XML is a Dumb Format For Storing Data

originally wrote by Joel Spolsky

I'm not sure why XML got so sexy. It has its advantages; it's sure a good idea for data interchange or for all those little files you need to store settings. But for real work it just can't do what a solid, multiuser, relational database can do. The next time some uninformed analyst at Gartner or Giga or Forrester tells you "in the future, everything will be XML," ask them how to do "SELECT author FROM books" fast with XML. Hint: you can't. It has to be slow. XML is not the way to store a lot of data. Now tell me how to insert a new book at the beginning of the table without massive bitblts. Of course, I doubt if there is an analyst in one of those companies who would even understand that sentence, but that's life. Now lets look at the books table in XML.

<?xml blah blah>
<books>
    <book>
        <title>UI Design for Programmers</title>
        <author>Joel Spolsky</author>
    </book>
    <book>
        <title>The Chop Suey Club</title>
        <author>Bruce Weber</author>
    </book>
</books>

Quick question. What is the code to move to the next record?

Uh...

At this point a good programmer would say, well, let's parse the XML into a tree in memory so that we can operate on it reasonably quickly. The amount of work that has to be done here by the CPU to SELECT author FROM books will bore you absolutely to tears. As every compiler writer knows, lexing and parsing are the slowest part of compiling. Suffice it to say that it involves a lot of string stuff, which we discovered is slow, and a lot of memory allocation stuff, which we discovered is slow, as we lex, parse, and build an abstract syntax tree in memory. That assumes that you have enough memory to load the whole thing at once. With relational databases, the performance of moving from record to record is fixed and is, in fact, one CPU instruction. That's very much by design. And thanks to memory mapped files you only have to load the pages of disk that you are actually going to use. With XML, if you preparse, the performance of moving from record to record is fixed but there's a huge startup time, and if you don't preparse, the performance of moving from record to record varies based on the length of the record before it and is still hundreds of CPU instructions long.

What this means to me is that you can't use XML if you need performance and have lots of data. If you have a little bit of data, or if what you're doing doesn't have to be fast, XML is a fine format. And if you really want the best of both worlds, you have to come up with a way to store metadata next to your XML, something like Pascal strings' byte count, which give you hints about where things are in the file so that you don't have to parse and scan for them. But of course then you can't use text editors to edit the file because that messes up the metadata, so it's not really XML anymore.

------
By the way, when I published this digest from Joel, I have no idea how to put the HTML code in my blog contents. Here is the solution, and the full table of escape characters are here.