Deciding on when to use XmlDocument vs XmlReader
C#Xml SerializationXmldocumentXmlreaderC# Problem Overview
I'm optimizing a custom object -> XML serialization utility, and it's all done and working and that's not the issue.
It worked by loading a file into an XmlDocument
object, then recursively going through all the child nodes.
I figured that perhaps using XmlReader
instead of having XmlDocument
loading/parsing the entire thing would be faster, so I implemented that version as well.
The algorithms are exactly the same, I use a wrapper class to abstract the functionality of dealing with an XmlNode
vs. an XmlReader
. For instance, the GetChildren
methods yield returns either a child XmlNode
or a SubTree XmlReader
.
So I wrote a test driver to test both versions, and using a non-trivial data set (a 900kb XML file with around 1,350 elements).
However, using JetBrains dotTRACE, I see that the XmlReader
version is actually slower than the XmlDocument
version! It seems that there is some significant processing involved in XmlReader
read calls when I'm iterating over child nodes.
So I say all that to ask this:
What are the advantages/disadvantages of XmlDocument
and XmlReader
, and in what circumstances should you use either?
My guess is that there is a file size threshold at which XmlReader
becomes more economical in performance, as well as less memory-intensive. However, that threshold seems to be above 1MB.
I'm calling ReadSubTree
every time to process child nodes:
public override IEnumerable<IXmlSourceProvider> GetChildren ()
{
XmlReader xr = myXmlSource.ReadSubtree ();
// skip past the current element
xr.Read ();
while (xr.Read ())
{
if (xr.NodeType != XmlNodeType.Element) continue;
yield return new XmlReaderXmlSourceProvider (xr);
}
}
That test applies to a lot of objects at a single level (i.e. wide & shallow) - but I wonder how well XmlReader
fares when the XML is deep & wide? I.e. the XML I'm dealing with is much like a data object model, 1 parent object to many child objects, etc: 1..M..M..M
I also don't know beforehand the structure of the XML I'm parsing, so I can't optimize for it.
C# Solutions
Solution 1 - C#
I've generally looked at it not from a fastest perspective, but rather from a memory utilization perspective. All of the implementations have been fast enough for the usage scenarios I've used them in (typical enterprise integration).
However, where I've fallen down, and sometimes spectacularly, is not taking into account the general size of the XML I'm working with. If you think about it up front you can save yourself some grief.
XML tends to bloat when loaded into memory, at least with a DOM reader like XmlDocument
or XPathDocument
. Something like 10:1? The exact amount is hard to quantify, but if it's 1MB on disk it will be 10MB in memory, or more, for example.
A process using any reader that loads the whole document into memory in its entirety (XmlDocument
/XPathDocument
) can suffer from large object heap fragmentation, which can ultimately lead to OutOfMemoryException
s (even with available memory) resulting in an unavailable service/process.
> Since objects that are greater than 85K in size end up on the large object heap, and you've got a 10:1 size explosion with a DOM reader, you can see it doesn't take much before your XML documents are being allocated from the large object heap.
XmlDocument
is very easy to use. Its only real drawback is that it loads the whole XML document into memory to process. Its seductively simple to use.
XmlReader
is a stream based reader so will keep your process memory utilization generally flatter but is more difficult to use.
XPathDocument
tends to be a faster, read-only version of XmlDocument, but still suffers from memory 'bloat'.
Solution 2 - C#
XmlDocument is an in-memory representation of the entire XML document. Therefore if your document is large, then it will consume much more memory than if you had read it using XmlReader.
This is assuming that when you use XmlReader you read and process the elements one-by-one then discard it. If you use XmlReader and construct another intermediary structure in memory then you have the same problem, and you're defeating the purpose of it.
Google for "SAX versus DOM" to read more about the difference between the two models of processing XML.
Solution 3 - C#
Another consideration is that XMLReader might be more robust for handling less-than-perfectly-formed XML. I recently created a client which consumed an XML stream, but the stream didn't have the special characters escaped correctly in URIs contained in some of the elements. XMLDocument and XPathDocument refused to load the XML at all, whereas using XMLReader I was able to extract the information I needed from the stream.
Solution 4 - C#
There is a size threshold at which XmlDocument becomes slower, and eventually unusable. But the actual value of the threshold will depend on your application and XML content, so there are no hard and fast rules.
If your XML file can contain large lists (say tens of thousands of elements), you should definitely be using XmlReader.
Solution 5 - C#
The encoding difference is because two different measurements are being mixed. UTF-32 requires 4 bytes per character, and is inherently slower than single byte data.
If you look at the large (100K) element test, you see that the time increasesw by about 70mS for each case regardless of the loading method used.
This is a (nearly) constant difference caused specifically by the per character overhead,