Monday, May 02, 2005

Don Box and HailStorm

The other day I stumbled on an interesting post by Don Box. In it, he discusses noticing his old HailStorm (a.k.a., ".Net My Services") reference manual. Don, I am not sure why that old book is in your living room? Mine is on a shelf in my basement, along with all my old .NET books...

Don's claim is that "the technology never saw the light of day". While it is true that Microsoft never shipped the system (at least not yet anyway), I wonder if others are successfully shipping the essence of HailStorm?

For the duration of this post, and for the entire comment stream, lets for a minute, forget the over-sensationalized past. Forget that Microsoft was involved in the design, development, and marketing of this system. Forget that many believed that Microsoft was attempting to associate HailStorm with a global identity system, and that some within Microsoft believed that Microsoft should house all of the data exposed by HailStorm in it's own data centers. I am asking you to put this behind you, and instead, lets just look at the essence of HailStorm, and decide if some of these ideas are shipping, or should be shipping.

As I look back on HailStorm, and try to distill it into some of its core concepts, I come up with the following list:

  • Network Centric, Extensible Data Model, for Everyday Data

  • Data Decoupled from Applications

  • Anytime, Anyplace, and from Any Device Access

  • Identity Centric Data Access


HailStorm was based on an XML data model. The system defined several core data types (calendar events, address book entries, bookmarks, profiles, etc.). Each core type had a set of required elements and attributes, and allowed for arbitrary extensions in both the element and attribute dimension, as long as those extensions are defined within a unique XML namespace. HailStorm had the notion of an <address/> type which defined a set of base properties associated with an address. Anyone could easily extend an address by including arbitrary, well formed XML from an arbitrary namespace. The data model was simple to use, simple to extend, and simple to process. There was no need to buy proprietary tools to crack, parse, manipulate, re-transmit, or re-purpose any type.

Looking at HailStorm through this facet, there are clear similarities between it, and RSS 2.0 and Atom. These two core systems are very powerful. Their ease of use, their simple extensibility, their inherent network centricity have unleashed many clever and useful applications.

Both of these systems specify a very simple container model, in both systems we see a variety of "everyday types including events, people and their properties, rich categorization, etc.

When Don claims that "the technology never saw the light of day", I agree with him that as a whole, HailStorm was never released, but that the essence of HailStorm, in this dimension is alive and well.

HailStorm embraced the idea of decoupling the data from the application. The idea was to allow a variety of applications to process an manipulate your calendar data, your address book, your email, your favorite web sites, your travel preferences and itineraries, etc. This is not a new, novel idea, but was certainly something that was important and core to the system. Simple applications that we were trying to enable included the ability to overlay your personal calendar with the calendar of your favorite band, or your favorite sports team, or your spouse, etc. We wanted to enable a unified "address book" where your contacts could be used across applications written by any vendor.

In keeping with our RSS/Atom theme, I wonder if these two systems have similar goals? Isn't it true that we all love this idea of decoupling data from applications? That we are able to use applications like bloglines, newsgator, or a host of other aggregators/notifiers to keep track and reuse all that is going on out there? What would the blogsphere be like without this technology? What would it be like if you could only read blogger blogs on blogspot? Or read Don's stuff on pluralsight? Or read Jon Udell on infoworld? I think that we can all agree that this theme that was central to the vision of HailStorm is extremely valuable, and has certainly "seen the light of day". In fact, I think we can all also agree that we would rather see more information work like this...

HailStorm embraced the idea of accessing your data anytime, anywhere, and from any device. In fact, during our first public demonstration of the system, we showed read/write HailStorm access from:

  • Solaris written in Java

  • Linux written in perl

  • Windows written in perl, javascript, C

  • Palm VII (written in I forget, sorry, I think C)


I don't remember if we demonstrated Windows access from .NET, but for sure, that would have worked too.

In our system, if you had a crude XML parser, and if you had an ability to send/receive HTTP, you could interact with HailStorm. We did not force you to purchase any special tools, install complex DLL's or type libraries, generate proxy classes through a WSDL parser, etc. HailStorm was based on an approachable protocol that was easy to understand and use from any platform and from any device.

Looking at both Atom publishing protocol for updates, and A9's OpenSearch API for query is an interesting excercise. These systems all are designed to provide an approachable, read/write API that will work from any network capable device, without the need for a proprietary runtime or tool suite. These systems put you, the developer first. I would argue, and I think most of you would agree, that uniform access, any time, any place, and from any device, independent of any proprietary runtime stack is exactly what it takes for an information access protocol to succeed. Has this technology shipped? For sure, in pockets of the broader industry, this approach is thriving. In other areas, the illusion of openness is there, but the reality is that expensive and complex proprietary tools act as your gateway...

The final facet of HailStorm, is the idea of identity centric information access. In HailStorm, the model was based on an identity space where an identity could represent a person, an organization, a group, etc. Given an identity, you could determine where to send messages to interact what information related to that identity. Both the identity space and the services bound to that identity were fully federated. We did this so for instance, the calendar information for Callie could be served by servers at Microsoft, Yahoo, Google, etc. So that the travel data for Nikki could be served by Expedia, American Express, or Cisco... Our system was based on a discoverable, distributed directory known as the myServices service designed to answer these simple queries.

Architecturally, HailStorm provided a level playing field where service providers could compete for a customer's business, and where because of all the facets listed above, a customer had complete freedom to move from provider to provider, with no impact on applications running on her behalf.

Let us look back one more time on the central concepts, or the essence of HailStorm:

  • Network Centric, Extensible Data Model, for Everyday Data

  • Data Decoupled from Applications

  • Anytime, Anyplace, and from Any Device Access

  • Identity Centric Data Access


I believe that there are systems out there today that are based in large part on a similar set of core concepts. My feeling is that the various RSS/Atom based systems share these core concepts and are therefore very similar, and more importantly, that a vibrant, open and accessible, developer friendly eco-system is forming around these systems...

Thursday, March 03, 2005

Sorry About That

Sorry about my blog being down. Looks like someone actually read it... Oh yeah, and Google had absolutely nothing to do with my Blog being down. I took it down, on my own, so that I could shut down the inbound email comment stream. One of the many email messages I received was something from one of my financial advisors. He was making a joke about eWeek being so on top of the hot breaking news...

My inbox was full this morning, and your comments keep coming in... I'll figure out how to shut the inbound comment email stream so I can concentrate on my day job. You know writing, and delivering software.

Anyway, I think some are reading a bit too much into this message. I am in no way trying to trash Microsoft. All I was trying to point out is that, in my opinion, software delivery has changed. I know windows update makes things some things better, but you know, this article is not about delivering bug fixes...

Stay tuned though. I have some other things on my mind, what I would like to see in IE7 for instance.

Saturday, February 12, 2005

Shipping Software

A few weeks ago I had lunch with the now famous "Mark Jen". I never knew Mark while we were at Microsoft, even though we both worked in the same group. Funny how large groups at Microsoft can get...

We had a great Google style lunch at a sunny table in Mountain View. I was too dense to notice that Mark was doing research for his blog. One thing he said got me thinking... Something that many have said over the years, that Microsoft "knows how to ship software".

Being a 16 year Microsoft veteran, a Distinguished Engineer, key architect and code writer for windows, architect of the largest source code control and build system ever attempted, I deeply believed that Microsoft knows how to ship software. We know how to build it, test it, localize it, manufacture it, charge lots of $$$ for it, etc.

Mark and I talked about this briefly at lunch that day, and I have been thinking about it from time to time since...

I am not sure I believe anymore, that Microsoft "knows how to ship software". When a Microsoft engineer fixes a minor defect, makes something faster or better, makes an API more functional and complete, how do they "ship" that software to me? I know the answer and so do you... The software sits in a source code control system for a minimum of two years (significantly longer for some of the early Longhorn code). At some point, the product that the fix is a part of will "ship" meaning that CD's will be pressed and delivered to customers and OEM's. In best case scenarios, the software will reach end users a few months after the Release To Manufacturing (RTM) date. In many cases, particularly for users working in large corporations, they won't see the software for a year or more post RTM...

Consider the .NET framework for a second. Suppose you wrote something innocent like a screen saver, written in C# based on the .NET framework. How would you as an ISV "ship your software"? You can't. Not unless you sign up to ship Microsoft's software as well. You see, the .NET Framework isn't widely deployed. It is present on a small fraction of machines in the world. Microsoft built the software, tested it, released it to manufacturing. They "shipped it", but it will take years for it to be deployed widely enough for you, the ISV to be able to take advantage of it. If you want to use .NET, you need to ship Microsoft's software for them. Isn't this an odd state of affairs? Microsoft is supposed to be the one that "knows how to ship software", but you are the one doing all the heavy lifting. You are the one that has to ship their software the last mile, install it on end user machines, ensure their machines still work after you perform this platform level surgery.

When an Amazon engineer fixes a minor defect, makes something faster or better, makes an API more functional and complete, how do they "ship" that software to me? What is the lag time between the engineer completing the work, and the software reaching its intended customers? A good friend of mine investigated a performance problem one morning, he saw an obvious defect and fixed it. His code was trivial, it was tested during the day, and rolled out that evening. By the next morning millions of users had benefited from his work. Not a single customer had to download a bag of bits, answer any silly questions, prove that they are not software thieves, reboot their computers, etc. The software was shipped to them, and they didn't have to lift a finger. Now that's what I call shipping software.

I would argue that Microsoft used to know how to ship software, but the world has changed... The companies that "know how to ship software" are the ones to watch. They have embraced the network, deeply understand the concept of "software as a service", and know how to deliver incredible value to their customers efficiently and quickly.