Inspirational Technology

Wow, I never thought it would happen but XQuery 1.0, XSL-T 2.0 and XPath 2.0 have finally been released as W3C recoommendations. Eight specs in total make up this behemoth. This is an interesting claim.

The XML Query Working Group catalogued over forty implementations of XQuery and reported on how fourteen of them satisfy a test suite consisting of more than 14,000 test cases, demonstrating unprecedented levels of interoperability. XML Query is already available in products from all of the major relational database vendors as well as in XML-native database systems, middleware, XML editing systems and numerous open source products. W3C Member organizations have also announced implementations of XQuery or plans for implementations.

Just shows that with a enough blood, sweat and tears anything can be pushed through a standards body. So does anyone other than vendors actually care anymore? The wait for these things was excruciating and the complexity that comes along with them is extremely unfortunate. XML was supposed to be simple, this stuff plain and simply isn’t.

It’s been a while since I’ve seen a new release of Berkeley DB XML, I probably just wasn’t paying attention, but I did see Oracle has released Berkeley DB XML 2.3.8.

This release of Berkeley DB XML improves many aspects of query planning
and execution. Using indexed node storage, users will generally
experience significant speed increase. A new event-style layer allows
for tight coupling between Berkeley DB XML and other XML processing
code. This greatly enhances integration with programming languages and
XML parsing libraries by eliminating the need to create and then
re-parse XML content. The W3C XQuery 1.0 specification is nearing
completion and this release of Berkeley DB XML is compliant with the
current Proposed Recommendation.

Pete Lacey has an excellent post up about why all the teeth knashing over REST vs. WS-* and W3C XML Schema vs. Relax NG doesn’t really matter within the corporate space. I’ve certainly argued about this enough to know the sad reality that he’s right … as much as I hate to admit it. No matter how much simpler things could be if we chucked the whole WS-* stack, in the corporate world nobody is listening. All they hear is what the big vendors say, and all they talk about is SOAP and WS-* and W3C XML schema and a bunch of other really complex crap that just makes people’s jobs harder.

I think it’s naive to believe that this isn’t intentional, the complexity is there because it requires tools to make it usable and it just so happens that all those vendors that corporate developers are so fond of, sell tools. It’s good for their business to make technology as complex as possible. In fact it’s my distaste for what the big vendors are doing that keeps me from leaving the startup world. In that world we pick technologies that we can actually understand and that we know will work. Simple is always better than complex, and anything that requires an expensive suite of tools to make it simple is looked at with great skepticism. It also helps that we simply can’t afford the stuff the big vendors sell, but even if we could, there is very little I would actually want to use.

Tim Bray is pushing an effort to choose Relax NG for XML schemas. I simply say yes, if you have to have a schema, at least use a schema language that makes sense. W3C XML Schema is an abomination that has done more harm to XML based systems than anything else. Every time I’ve seen a team try to make use of W3C XML Schemas, it results in nothing but headaches. I just look on with frustration over my knowledge that it doesn’t have to be that way. It’s really quite sad to see how bad the W3C screwed up with that technology and how widely it penetrated other specifications without much thought being given to it’s real quality.

I’ve been out of the SOAP vs. REST debate for a while, but Pete Lacey’s “The S stands for Simple” post just has to be mentioned. For XML geeks who’ve been constantly been annoyed with the abomination that is SOAP/WSDL/UDDI it’s quite amusing. I’ll be so happy the day that SOAP is finally dead. Of course coming along right behind it will be another bloated, horribly complex and completely unnecessary vendor designed technology to do the exact same thing. After all, if it’s actually simple enough for average developers to understand then there’s no market for tools and that’s something we just can have now can we.

It’s been an unfortunately common misconception that Structured Blogging and Microformats are competitors. Now, even Marc Canter whose company hired me to work on Structured Blogging late last year, has fallen into the trap of thinking this way when he says “However with the demise (and inevitable dissolution of PubSub) it looks like Dave Sifry’s stampeding marketing efforts have won.”. How can they win when they’re not even competeting? I’ll lay it out in simple terms here.

Structured Blogging is a Open Source project that builds TOOLs to publish microcontent including Microformats

Microformats is an open effort to build FORMATs for microcontent data.

Tools and formats are not the same thing. These projects are fully in line with each other and both efforts are necessary.

In fact, from looking through the results of the Technorati Microfomats search, next to the various Yahoo services, users of the Structured Blogging plugins are probably the biggest generators of Microformated Review data.

Why is this? It’s because the Structured Blogging plugins are a TOOL that makes it really easy to create the reviews. People aren’t using the tool because they want to publish structured data, even I don’t really use it for that reason, they use it because it makes writing reviews easier. A side effect of that is that the review gets published in the hReview Microformat as well as the Structured Blogging XML format.

Now I hear you say, but why the other format? There are a number of reasons the XML format still needs to exist.

1. There aren’t yet Microformats for every type of data that people want to publish.
2. The Microformats that do exist often don’t cover the full range of data that the tool needs to track.
3. There is data that needs to be tracked, but should not have a visible presence on the page.
4. The tool uses the format internally for all editing operations because of the first item above and because it’s a lot easier to work with the XML using XML tools like XPath, than it is to deal with the Microformatted data.
5. Historical - The first version of the Structured Blogging plugins published only that XML block and the requirements when we built the new version dictated that we maintain the external publication of the data in the same manner. Those requirements were dictated by PubSub, more on this later.

Those formats are then published to the world for most of the same reasons. Now however, in the cases where there is an existing Microformat it will be used as well. So currently you can use the Structured Blogging tools to generate hReview, hCalendar, hCard, XOXO, rel-tag and rel-license Microformatted data and it is my explicit goal to continue to use Microformats where ever they fit. So again how can we be competing when a big focus of our efforts revolves around publishing Microformatted data?

Over time as more Microformats emerge we’ll continue to add support until the time where everything we want to do has an associated Microformat. At that point we’ll probably still have an associated XML format for each content type because of the internal needs of the tool, but whether those formats are used widely outside the tool is an open question and really not relevant in looking at things today.

Marc Canter also seems to see the death of PubSub as the death of Structured Blogging as well. Now here’s what Marc really doesn’t get, Structured Blogging is supposed to be an Open Source project. And one of the great things about Open Source projects is that they can survive the death of any supposed backing companies. And that’s especially true when the backing companies aren’t actually doing any of the development work which was the case with PubSub.

In this case I see the death of PubSub as a good thing. PubSub and Technorati were competitors and even though the Structured Blogging effort isn’t a competitor to Microformats, many people still extended the PubSub/Technorati competition to the Structured Blogging/Microformats efforts. At one point in time that may have even been true, but now, as I’ve tried to communicate here, it no longer is. Personally, I’m happy to see PubSub out of the picture. Now we can refocus the Structured Blogging effort on being truly open and focused around building really cool tools to create fun applications of Microcontent. We can also be free to fix some of the more controversial elements of the tools, deepen the support of Microformats and bring the concept of microcontent publishing into many other systems.

So contrary to what Marc says I don’t see the death of PubSub as any kind of loss for the Structured Blogging effort, in fact I see it as an opportunity for a new beginning.

Looks like one of my older posts about how to think when building REST XML protocols is making the rounds in the blogosphere.

That post was originally written to vent a little while I was trying to save a project that was consuming a very poorly designed XML web service. It was a continual process of me just shaking my head about the poor design decisions made on both sides of the system and how easy it would have been to avoid if they could have just thought a little bit differently. There are many other things to consider beyond what the post talks about, but my major point there is to simply think XML.

Hmm, “Designing REST protocols by thinking in XML”. Sounds like a good title for an article.

Note: comments are turned off on the original post so this post can be used instead.

This is from an email I wrote to clarify my concerns with the existence of RDFa. It was in response to an email prompted by my post on the premature creation of standards in the microcontent space.

I had looked at RDFa and I agree it’s not all that complicated, in fact RDF as a basic technology isn’t really that complicated. Where RDF fails is the semantic web vision and the complexity pulled in by trying to build a globally scalable data framework. To me that’s too aggressive of an approach for real world adoption. Anyone who actually gets out in the field and works with average skill level developers will know that a disturbingly large percentage are still struggling with simple XML, let alone the ideas behind RDF. The RDF vision is great and I don’t dispute the theoretical value of it, my problem with it is born 100% from the fact that I just don’t believe it can succeed outside the realm of the above average skill level developers who are already interested in it. That’s basically been my opinion since I read the early working drafts for the RDF specs and I’ve seen nothing since then that’s allowed me to think otherwise. I’m always looking.

Now, that being said, I will admit that RDFa is the cleanest use of RDF I’ve seen. It does have some potential, but it’s crippled in the short term by the dependence on XHTML 2.0 and in the long term by the fact you still need to understand the big picture of RDF to really leverage it. I know there is some work to try to fit it into XHTML 1.0, but that 2.0 issue is a really big risk. If people start evangelizing this as the “standard” way of doing microcontent and it depends on XHTML 2.0 and XHTML 2.0 isn’t widely used yet then that means people will wait to deploy microcontent based applications and that’s simply something that I can’t stand to see. That’s exactly what happened with XQuery and native XML databases, everybody got caught up waiting for XQuery to be finished and stopped innovating on alternative solutions. Is XQuery really the right language for managing XML data? I’m not convinced that it is, nor am I convinced that RDF is the right way of managing microcontent. Hence, why I consider it very damaging that the W3C is already playing in a space where quite frankly, they aren’t wanted. It’s fine if it’s being worked on for a specific creative commons use case, but I’m already seeing it being evangelized as a competitor to microformats and other forms of microcontent and that’s what I find troubling.

Evan Prodromou has posted what I guess can be considered a call to action in regard to microformats and RDFa. Those are clearly two ways of solving the problem of microcontent publication, with RDFa bringing all the weight of RDF along with it. Evan’s concern is that they’ll end up competing and slowing the adoption of the technology. Probably a legitimate concern given the history of other recent W3C killer specs like XQuery and XML Schema. It actually makes me cringe to think about the presence of a W3C spec in this area. I think XQuery was the worst thing that could have happened to XML databases, it basically killed innovation and the stupid thing still isn’t done.

Microcontent publication is a new area, ripe for innovation and it’s WAY TOO EARLY to worry about standards. Standards will just ramp up the complexity and kill innovation. I keep seeing the argument that RSS and HTML became such a mess because there were no standards or whatever. I just don’t buy that as a problem. The only thing that matters in both of those cases is that they were both successful, broadly successful, hugely successful, massively successful in a way that almost no standardized technology has ever been successful. Are they a mess now? Yeah sure I suppose you could argue that, but so what! At least we got to the point where it actually makes sense to worry about that. If RSS had been based on a committee designed standard for syndication it would have never succeeded. And that isn’t just theory, there were committee designed specifications for that exact thing before RSS existed and now I can’t even remember what they were called.

As technologists we tend to get caught up in trying to avoid mistakes, and we look at things like RSS and HTML in retrospect and think of all the ways things could have been done better technically. However, in doing that we also tend to forget or simply miss the reason that we’re even in a position to look back on the successful technologies. So here’s my message to the other technologists working in this space. Ignore your desire for standards, it’s too early. The path to success is paved with published microcontent, real world microcontent. The format of the microcontent is unimportant, what’s important is that it simply exists, that people are actually publishing stuff in some structured form. Any structured form is better than no structured form. Once that happens and people are used to the idea and there’s at least some value there, then we can worry about real standards to take it to the next level. For now even the Microformats effort is almost pushing things in my opinion.

And I’ll also say this, if microcontent publishing requires people to understand anything about RDF, it’s game over, kaput, fini, forget about it. RDF is what eight, nine years old now and there’s still only a very small handful of people who actually understand it. Hmm, maybe there’s a reason for that. So put your resources, reifications and ontologies on the ground and take a step back.

It’s going to be a rowdy and wild time on the microcontent frontier and that’s just how it should be. So grab your six shooters, free your mind and shootdown the standards. Yee Haw!