A WS-I standards body for Hadoop? -1 to that

There's an article from IBM which argues that Hadoop needs to copy WS-I and Oasis to have a set of standards:

In the early 2000s, the service-oriented architecture world didn’t begin to mature until industry groups such as OASIS and WS-I stabilized a core group of specs such as WSDL, SOAP, and the like.

I despair. Anyone who credits OASIS and WS-I for this does not know their history -or is trying to rewrite it.

The initial interop came from the soapbuilders mailing list, of which Sam Ruby (IBM, ASF),  Davanum Srinivas , CA->WSO2->IBM and dims at apache, Glen Daniels (at the time, Macromedia + apache), all played key parts. Anyone in IBM curious about Soapbuilders should ask Sam about it.

Soapbuilders was the engineers: finding problems, fixing them. Agile, fast, focused on problems, fast turnaround in the evolving SOAP stacks.

It died. Killed by WS-I

WS-I wasn't the engineers, it was the standards people. Suddenly the battle lines were drawn over whose idea was going to be standardised. Take the problem of shipping binary data around. Base-64? works, but inefficiently. Microsoft's DIME?  Soap with Attachments?  MTOM?  WS-I was where the battles were fought, sometimes won, sometimes solved with "all of them"

I have been known to express opinions on the cause of interop problems in SOAP; I'm not going to revisit that, except to note that the focus of SOAP interop settled on Java<->.NET interop, which was addressed not by standards bodies but by plugfests, the standards themselves being too vague to cover the hard issues, especially defining which proper subsets of WS-* different stacks would support, and which proper subsets of WSDL could be generated and parsed. Ideally the parseable-subset was superset of the generateable subset, but, well, there can always be surprises. Those discussions would end up on soapbuilders, where maybe the developers could fix them without the standards team getting in the way.

I'm going to pick on one specification as the worst of all standards. WS-A.

1. Replaced a simple concept "URL" with an arbitrary lump of XML.
2. Went through an iterative development process that resulted in multiple versions
3. Used xml namespaces to identify the versions. 
4. Used three namespaces: 2003, 2004, 2005/08 to identify four different versions. The 2005/08 one has both an "interim' and "final" release.

That's what happens when you put a standards body like WS-I in charge of something as simple as a URL. They not only make a vast mess of it, you get a vast mess in different XML namespaces.

As for Oasis? WS-RF. That's all I need to say for anyone involved in; WS-*, or any of the grid proposals. A standard for managing "resources" -leased things at the end of WS-A addresses, across a network. A standard that managed to include two different WS-A versions in it.

Think about that for a minute. You have a "standard" that is produced by a standards body that is somehow affiliated with the UN and has some official status, yet they cannot push out one of the WS-* specification suites without incorporating different concepts of "how to address a SOAP endpoint" in the context of that single 'coherent' suite of documents -end up including non-normative drafts as well as the final things.

I am not going to make any statements on Hadoop standardisation in this context as it will probably be taken for an official stance rather than my personal opinions.  There is a section on Hadoop Compatibility that I wrote in Defining Hadoop; it sounds like some people and organisations ought to read that article.

 I do want to close with the following point

WS-I and Oasis were not bodies capable of producing "standard", where "standard" could be defined as a coherent, consistent and testable set of protocols. Instead they were places where vendors could push their own agendas, where the winners became the organisations capable of funding the most participants in the standards process, or those willing to do the most back-room deals with others.

The compromises needed to get anything out the door in even a hopelessly untimely manner produced an incoherent mess of XML namespaces, schemas and protocol issues that anyone working on SOAP stacks still has to deal with today.

REST did not win just because it was architecturally cleaner, because it was more powerful. It won because the alternative was the set of WS-* specifications that came out of WS-I and OASIS. Those organisations did not set WS-* on its route to global success; they condemned it to the niche of intra-enterprise Java/.NET communications, a decade after CORBA could have done the same thing better.

[photo: sunset on Nelson Street from St Michael's Hill]