RavenDB Studio 3.0, and why we moved from Silverlight to HTML5

Summary: A big step for RavenDB: a new HTML5 Studio. Plus, some thoughts on the move from Silverlight to HTML5 and our experience in the transition.

Yesterday, I pulled the covers off something I’ve been working on for a few months, something I’m very proud of.

RavenDB, the most popular NoSQL database in the .NET world, announced a brand new RavenDB Management Studio, Raven Studio 3.0, built from the ground up using HTML5 and modern web technologies. Yes, we’re moving away from Silverlight and onto HTML5.

Ayende and myself demoed the new Raven Studio just yesterday in a live webinar:

This has been my pet project for the last few months, and it’s something I’m quite proud of! I believe this is a huge step forward for RavenDB (more on that in a minute), and the reception from the Raven community has been awesome, ego-stoking, and totally energizing.

The old Raven Studio was built in Silverlight. Some Silverlight fans have asked, why did we move to HTML5? Are we making a big mistake moving away from Silverlight and to HTML5?

No, on the contrary, we believe HTML5 is a damn good option.

  • The RavenDB community wants an HTML5 Studio. This has probably been the most-requested item from the RavenDB community. It came up multiple times in the Raven 3.0 Wishlist, it’s come up whenever Oren talks about the Studio, it comes up when we speak to the external developer community, heck, when I was in Israel for RavenDB training last year, one of the students brought it up right there in the Hibernating Rhinos office. Silverlight has served us well over the years, but Silverlight is a dying technology that our community doesn’t want to be tied to any longer. 
     
  • Silverlight tooling is a perceived barrier to Raven adoption. I speak at Code Camps and user groups, and when I speak on RavenDB, the love flows and the excitement grows…until I show Silverlight tooling. I get the raised eyebrow. “Silverlight? Oh. I see.” Others in the Raven community have reported this as well. For some, Silverlight is a stumbling stone. 
     
  • HTML5 is a step towards cross-platform Raven. RavenDB is the best NoSQL database for .NET. But, in time, we want Raven to spread her wings and be not just the best NoSQL solution for .NET, but the best NoSQL database, period. Moving to an HTML5 toolset is a step towards this goal. 
     
  • The software industry is moving away from plugins. Plugins like Silverlight added abilities you couldn’t do on the native web, such as audio, video, gaming, 2d drawing, documents, voice, and more. Plugins filled these gaps, but with HTML5, these gaps are disappearing. We don’t need Adobe Acrobat plugins anymore to view that high fidelity document. We don’t need Java applets anymore to run that simulation. We don’t need websites built entirely with Flash. And we don’t need Silverlight for Raven Studio. There is little reason today to build something in JavaFX, Flash, or Silverlight: the native web has supplanted them. Just as it’s rare – and often undesired – to see a Java applet out in the wild, so too it will be with Silverlight in the coming years.
     
  • The native web platform is a solid foundation for the future. Microsoft products come and go. 3 years ago, Microsoft was pushing Silverlight as the platform for line of business apps and islands of richness on the web. Today, Silverlight is prevented from running in the default browser of their newest operating systems.

    HTML, on the other hand, has been a stable, ever-evolving technology for decades, and because it is the very fabric of the web, things built in HTML live indefinitely. There’s a reason you can still visit and use the 17-year old Space Jam Website. Smile But your MS Silverlight app from last year? It won’t run even on the latest MS operating system’s default browser.

As a Silverlight developer who has built professional apps (e.g. 3M Visual Attention Service) and spoken at Code Camps and user groups on Silverlight, truth be told, Silverlight is a great developer platform. C# is an excellent language, Visual Studio probably the best development environment.

But, in the words of Miguel de Icaza, creator of Moonlight (open source, cross-platform Silverlight),

“I felt that Silverlight had a bright future, and that it could turn to fill an important void, not only for web development, but for desktop development in general.  And this was largely one of my motivators. I am very sad that Microsoft strategy cut the air supply to Silverlight.”

This, coupled with the mobile computing explosion and the software industry’s shift away from plugins, results in a sickly future for Silverlight and Silverlight apps.

RavenDB rocks, and we want the tooling to rock as well. Having our tooling tied to this technology was not an attractive proposition, and it was time for us to move on.

A new technology stack for Raven Studio 3.0

After much deliberation and considering all the options available to us, we moved off of Silverlight.

Instead of Silverlight, HTML5.

Instead of C#, TypeScript.

TypeScript is awesome. TypeScript is new language, a superset of JavaScript designed for building apps on the web. Silverlight fans will be happy to know it’s built by none other than Anders Heijlsberg, the much-respected language designer and author of C#.

In TypeScript, all JavaScript is valid TypeScript code, so it’s familiar to any web developer, but it gives us nice things like an optional, flexible type system, classes, modules, and enums, and features proposed for future versions of JavaScript, but compiles to plain old JavaScript that runs in every browser.

TypeScript tooling is Visual Studio, with all the nice debugging and refactoring that brings, but it can also be written in any text editor and debugged in any browser.

For infrastructure, because we wanted the look & feel of a web application, rather than a set of web pages, we opted to build a single page application (SPA). Durandal.js gives us exactly that: a nice means to load pages on demand and compose them into a cohesive web application.

For UI, Durandal uses Bootstrap for a consistent, pleasing aesthetic, and KnockoutJS for data binding and MVVM.

Using data binding, MVVM, and Durandal makes a great developer experience, one not too foreign to the MVVM stuff in Silverlight. (Indeed, the author of Durandal.js is the same author of the popular Silverlight MVVM framework Caliburn Micro.) Look at the code and judge for yourself; you’ll see classes separated out into small, logical view models, and a clean separation between view and logic.

What has been our experience moving to HTML5?

One immediate, measurable gain was performance:

  • Memory usage dropped from 140MB to 20MB.
  • Cold starts dropped from ~7s to ~2s.
  • Warm starts dropped from ~3s to ~1s.
  • General snappiness: XAML is rather heavyweight, and you’ll notice just moving around the application, loading your documents, collections, or editing – it’s all faster in HTML5. Snappy and responsive.
  • This doesn’t happen:
    image
  • This doesn’t happen either:
    image

A lot of the above we get for free simply by Doing Less Stuff™. No .xap files to download, no dlls to load, no CLR runtime to start, no plugin host process for the browser, no browser-to-plugin communication, no managed code to start executing.

This translates into faster start times and less memory usage.

Another free item we get is JavaScript and the blazing-hot modern JS browser runtimes. The major browsers – IE, Firefox, Chrome, Safari – are in a cut-throat competition to get the fastest JavaScript runtime, to squeeze every possible ounce of performance out of JavaScript. You’ll regularly see these browser vendors advertising their JS benchmarks as proof of performance improvement. This is a free win for the new HTML5 Raven Studio: as browser vendors continue to improve their engines in this cut-throat, cross-company competition of speed, Raven Studio will reap the performance improvements.

Moving to the native web platform fixes some plugin-induced workflow hiccups. For example, keyboard shortcuts: Silverlight and other plugins eat the keyboard. So say you’re got Raven Studio opened, and you want to open a new browser tab, so you hit CTRL+T. Surprise, nothing happens. Why? Silverlight ate your keyboard shortcuts, your browser never received them, and your workflow was just interrupted.

If you’ve ever used one of those old all-Flash websites, or full-page Java applets, you’ve probably noticed some things just don’t feel right. So it was with the old Silverlight Studio. Moving to HTML5 fixes these issues.

Conclusion

Transitioning out of the plugin ghetto and moving to HTML5 has been a delight, but more importantly, it’s good for RavenDB users as we move to a faster, more lightweight tool. It’s good for the future of RavenDB to have our tooling built on the solid rock of the native web.

I understand the Silverlight fans who are sad to see the old Silverlight Studio go. I’m a Silverlight fan myself, I understand their concerns. The most I can ask of you guys is to give us the opportunity to earn your trust. It will take time, but with a faster, more lightweight, stable tool that does what you need and gets out of your way, I believe that trust will be earned.

The new HTML5 Raven Studio is on GitHub and we’d love for you to give it a spin or even contribute to the code. I’m pleased to say we already have had a few contributions since it was released just yesterday. I’m proud of this work, and I really hope you guys enjoy it!

My beautiful web development soup

Summary: Web development is chaotic, overwhelming, and beautiful. How many technologies does it take to build something useful on the web? You might be surprised. Advice for developers getting into web development.


3654636770_3b1a5d470bThis week I was working on my open source startup project, BitShuva Radio. I acquired a new client this week, but this client was unique because he’s a Java developer and wanted to know how to edit the open source code for his new radio station.

As I typed out my answer, it dawned on me that the sheer number of  technologies utilized in building a useful web application is staggering. That they actually work together is a testament of the beautiful soup that is web development today.

How many technologies does it take to build something useful on the web? Here’s my answer:

  • I use TypeScript, rather than plain JavaScript, for all client-side code. A superset of JavaScript with optional types, TypeScript is a well-structured, typed language with excellent tooling. I use it for practical reasons: as the app grew in complexity, the need for a more structured language became apparent. I ported the JS to TypeScript last month, and I’m quite pleased with the results.
  • KnockoutJS is used heavily absolutely relied on. BitShuva is a dynamic single page application, meaning we’re doing lots of dynamic updates to the UI in real time. Knockout lets us make dynamic UIs very easily without resorting to tons of DOM manipulation code. Instead of DOM manipulation, Knockout lets you change your objects, and your UI automatically updates. For example, songsOnScreen.push(new Song()) will automatically show the song in the UI, no jQuery DOM manipulation required. 
  • I use knockout.postbox for decoupled pub/sub communication between client-side code components. For example, when I hit the play button, the click handler doesn’t need to know about HTML5 audio or any infrastructure concerns. Instead, it calls ko.postbox.publish(“Play”). Whoever is in charge of playing audio simply calls ko.postbox.subscribe(“Play”, …). This way, the click handler and the HTML5 audio component don’t need to know about each other. Nice and clean. Bonus: it works with Knockout observables, so you can say currentSong.publishOn(“SongChanged”) to automatically publish messages on a particular topic.
  • UbaPlayer is used for playing HTML5 audio with a Flash fallback. 
  • For styling, we use LESS, the CSS superset. We use LESS because CSS is redundant; LESS lets us use variables and functions for code reuse, but still compiles down to plain CSS.
  • jQuery is still there, but not as much as you might think. Because KnockoutJS handles the DOM <—> code interaction, jQuery is only needed in rare cases where we need special DOM manipulation (e.g. to fade in/out an element). We also use jQuery to do AJAX calls.
  • The site generally uses Twitter Bootstrap UI toolkit for styling and consistency across the UI.
  • We’re using Google Web Fonts for typography. 
  • For images like thumb-up/down, play button, etc. these are actually font characters, from the special Font Awesome web font. Because it’s just a font, they scale infinitely without losing fidelity, and you can change their appearance (size, color, etc.) using plain old CSS stylings. Icon fonts are awesome.
  • We use Asp.NET MVC Razor view engine to actually render our UI. Because of heavy reliance on KnockoutJS, Razor isn’t needed much; it’s mostly just plain HTML that is dynamically swapped in through AJAX and KnockoutJS.
  • The server code is written in C# 5 using Asp.NET MVC 4 + MVC Web API
  • ASP.NET Web Optimization Framework is used for minifying and bundling the scripts and CSS.
  • The database is RavenDB, a modern, 2nd generation document database, easily the best NoSQL database in the .NET world. I use it because it’s blazing fast and simple to just start dumping objects into.
  • ReactiveExtensions (RX)  is used on the C# server end to treat events as observable streams, which can then write LINQ queries on top of. We also use its sister project, Interactive Extensions, for some useful extension methods to IEnumerable objects (such as .Do and .ForEach).
  • To load up the whole shebang and edit the project top-to-bottom, I use Visual Studio 2012 + TypeScript tools plugin (for editing/compiling TypeScript) + Web Essentials (LESS compilation, better tooling for HTML5 web standards, and more).

Holy cow!

Here’s the final tally of languages, frameworks, and tools used:

  • 5 languages – C#, TypeScript, LESS, .cshtml Razor syntax, plus bits of plain JavaScript here and there
  • 5 UI JavaScript frameworks – KnockoutJS, jQuery, UbaPlayer, Knockout.postbox, Twitter Bootstrap.
  • 6 server-side .NET frameworks – RavenDB, MVC 4, MVC Web API, Reactive Extensions, Interactive Extensions, Web Optimization Framework.
  • 3 tools – Visual Studio 2012, TypeScript tools plugin, Web Essentials plugin.

The sheer number of tools required to really build something useful on the web is staggering. You get the sense of why desktop developers — who often work in just a single language + UI framework (e.g. C# + WPF) — are hesitant to embrace the web. It requires massive retooling.

And to be truthful, this is only my beautiful web soup. I chose C# + KnockoutJS + TypeScript + LESS, but other web devs might prefer Ruby + BackboneJS + CoffeeScript + SASS. Or something entirely different.

The great thing is, there is so much evolution going on in web development space today, and this produces unique and specialized toolsets that help move the web forward. For example, languages like CoffeeScript and TypeScript are influencing the future direction of JavaScript.

It’s a fun time to be in web development.

My advice for developers? Learn a few web technologies that pique your interest and cook up your own beautiful web soup.

Stop writing for loops in Javascript

You have an array; you want to find an element in the array. What do you do?

Most developers go with the old procedural for loop:

var items = [
   { Name = "Matthew", Id = 1 },
   { Name = "Mark", Id = 2 },
   { Name = "Luke", Id = 3 },
   { Name = "John", Id = 4 }
];

// Find the item with Id === 3
var matchingItem = null;
for (i = 0; i < items.length; i++) {
     if (items.Id === 3) {
       matchingItem = item;
       break;
     }
}

  // Do something with matchingItem
}

What’s wrong with this?

It’s the classic procedural programming affliction: over-specification. It painfully specifies all the steps the program must take to reach the desired state.

All you want is the item with an Id of 3, but instead you create a new loop variable, increment that variable each loop, use special loop iteration language syntax, index into the array, check if it matches your condition, store the match in a mutable variable.

The how obscures the what.

Thank you sir, may I have another.

This obfuscates the intent of your code via over-specificying pageantry.

It’s also error prone. The above code omitted the var declaration for the loop variable i, (did you catch that?), thus adding it to the global scope (oops!). Developers often forget to break; out of the for loop when a match is found, resulting in superfluous loop iterations at best, unintended behavior at worst.

Can we do better?

How about .filter?

Today’s JavaScript (1.5+) includes the built-in .filter function:

var items = [
     { Name = "Matthew", Id = 1 },
     { Name = "Mark", Id = 2 },
     { Name = "Luke", Id = 3 },
     { Name = "John", Id = 4 }
  ];

  // Find the item with Id === 3
  var matchingItem = items.filter(function(item) {
     return item.Id === 3;
})[0];

  // Do something with matchingItem...

.filter is a higher-order function that returns all elements in the array that match your predicate. The above code calls .filter(…) and then gets the first matching item.

This is better, but still not right.

What’s wrong with this? Like a for loop that forgot to break; out of the loop after finding a match, .filter will iterate over your entire array, even if it already found a match. This will result in superfluous iterations. And if your code doesn’t account for multiple items matching your condition, you’ll get unexpected behavior.

Solution

There’s no way to do this efficiently or succinctly using any of the built-in JavaScript array functions.

Because of this, most developers just do the ugly one, or the inefficient one.

A proper solution would be a function added to the Array prototype chain. In lieu of the browser vendors doing that, we can do it like this:

if (!Array.prototype.first)
{
   Array.prototype.first = function(predicate)
   {
     "use strict";   
     if (this == null)
       throw new TypeError();      
     if (typeof predicate != "function")
       throw new TypeError(); 
     
     for (var i = 0; i < this.length; i++) {
       if (predicate(this[i])) {
         return match;
       }
     }      
     return null;
   }
}

Here, we’ve defined a .first method on the Array prototype. It uses the efficient-but-ugly for loop. But instead of coding up this for loop every time, we can now just call our higher-order function:

var items = [
   { Name = "Matthew", Id = 1 },
   { Name = "Mark", Id = 2 },
   { Name = "Luke", Id = 3 },
   { Name = "John", Id = 4 }
];

// Find the item with Id === 3
var matching = items.first(function(i) { return i.Id === 3 });

// Do something with match...

Much cleaner, syntactically similar to .filter, but without the superfluous iterations.

JavaScript has no lambda syntax (yet), so it’s not as clear as it could be. Ideally, we’d write it more succinct, something like:

var matchingItem = items.first(item => item.Id === 3);

We’ll have to wait for the proposed arrow function syntax for the next version of JavaScript before we can get that succinct.

In the meantime, Array.prototype.first is a nice polyfill with decent syntax, allowing your code to focus on what, rather than how.

TypeScript, CoffeeScript, and the state of web development in late 2012

Summary: The problems facing web development today. Microsoft’s new TypeScript language, what problems it addresses, and what it leaves wanting.


Earlier this year, I predicted Microsoft would be utilizing their still-baking Roslyn compiler service to transform C# into targets besides IL, such as JavaScript. That prediction didn’t come true, but I was close: Microsoft is indeed getting into the compile-to-JavaScript games, but they’re doing so with a brand new language announced today, TypeScript.

JavaScript is a messy, powerful language.

The whole compile-to-JavaScript trend addresses a real problem in web development today: JavaScript just isn’t a great language for big web apps. Some argue it’s not a great language for anything besides quick DOM + event handler glue. And honestly, they kind of have a point.

Half baked, escaped the lab too early, leaving us silly things like implicit conversions for equality comparison. Quick, what’s the result of this?

foo = 0 == (100 + 23 === “123”)

If you guessed the boolean true, you’re right (I think!) but it shows kind of the trouble you get into when languages try to do too much for the developer.

But silly syntax puzzles aren’t the real reason JavaScript is bad for big apps.

No, it’s for a number of more deeply-seated reasons the difficulties arise when coding a large JS app with a team of developers:

  • The automatic, often accidental, inclusion in global scope. (The above snippet adds foo to the window object, woops!) This results in code littering the global namespace at best, and overwriting other code’s functionality at worst.
  • No modules or namespaces for code organization. In addition to contributing to the above problem, a lack of built-in modules means we have to resort to 3rd party libraries for dependency detection and loading. Often, developers will just take the easy route and load everything, resulting in web apps that load slowly and use excessive memory.
  • A 10-ways-to-do-this inheritance pattern that kinda sorta looks like classical inheritance but is really prototypal. Some codebases standardize on a 3rd party library (such as MooTools or Prototype), while other codebases become the Wild West of inheritance problems.
  • Without any types, refactoring and symbol analysis becomes painful. For example, want to find all callers of a function? Have fun with CTRL+F, and weed through who’s calling your function and who’s calling a function of the same name. Want to rename a property? CTRL+H and pray that no other code file has a property of the same name. And so on.

Plus other uglies like optional semicolons to end lines, resulting in production-level frameworks being filled with, in the words of JavaScript creator Douglas Crockford, “insanely stupid code.”

But one thing JavaScript is, is capable. It’s a functional, prototypal, object-oriented language. General purpose.

So rather than wait for JavaScript to slowly evolve through standards bodies and bickering, we build our own fast-evolving languages that compile down to JavaScript.

CoffeeScript: A (slightly) better JavaScript

At the time of this writing, the compiles-to-JavaScript language de jour is CoffeeScript: a superset of JavaScript that adds things like classes, and removes things like semicolons, parenthesis, and other staples of the C family of languages.

But CoffeeScript still lacks a proper compiler. Oh, sure, there’s the CoffeeScript compiler, but it’s not really a compiler. It can’t tell you if you’re passing the wrong type into a function. It can’t tell you if foo.Blah will error at runtime, because there’s no symbol analysis.

CoffeeScript also suffers from its straddle-the-fence stance on compatibility with JavaScript. Want to include parens? OK, valid! Want to exclude parens? OK, valid! While it sounds nice in theory, in practice you end up with an inconsistent codebase, and an awkward rule set about when optional syntax isn’t really optional.

And though CoffeeScript tried its hand at list comprehensions, it falls far short of what Python and C#/LINQ devs have been enjoying for years.

To make matters worse, there’s still no real standard library for JavaScript. Just a mash of tiny libraries that do things differently ($.map? or array.map?)

And finally, we still have terrible debugging support. Got Coffee? OK! How do I set a breakpoint on this CoffeeScript line? Silly developer, you can’t! You have to look at your Coffee, figure out which .js files were emitted, find those in your favorite web browser developer debugger tool, and break into that. Grahgghhlhlhlhsl.

End result? As of late 2012, we don’t have any great options for building big web applications. We need a language that supports symbol analysis (refactoring and the like), type error detection, modules, classes. We need a web development platform with a standard library. We need a web development platform that supports debugging in the language you authored.

Enter Microsoft’s TypeScript

TypeScript is a Microsoft attempt at solving these problems, in particular through the adding of types. It’s JavaScript with types.

Does it solve all the problems?

Too early to say.

It was announced only today, and I just downloaded the damn thing. At a glance, they solved the classes and modules problem. And by adding types and tooling support in Visual Studio, you can actually perform refactorings in your big codebase; something Java and C# developers have been enjoying for over a decade, but web developers have been suffering without.

It doesn’t appear to have a standard library – sorry, LINQ fans.

Nor does it appear to support debugging in Visual Studio.

UPDATE: The TypeScript compiler supports an experimental –sourcemap command line argument that generates a source map file. These source map files can then be used to debug TypeScript in the browser. I predict that Microsoft will use these source map files to integrate into Visual Studio’s debugging experience, likely through a special debugging browser similar to the existing Page Inspector.

So that answer for now appears to be, TypeScript is an incremental improvement upon CoffeeScript: types and modules and symbol analysis are a nice addition. We’re still missing some important things, namely, a standard library and a powerful source-language debugger. Until we get these things, web development is still a second class experience.