Archive of published articles on July, 2010

Back home

Against Serialization with XmlSerialization

31/07/2010

There’s a terrible monster in .NET’s closest, and its name is XmlSerialization. I love serialization to XML (or whatever your node-based text format of choice is), and I love XML in .NET and C#, but XmlSerialization is a horror. Let’s look at the requirements for XML serialization with the classes in System.Xml.XmlSerialization:

  1. XML serialization serializes only the public fields and read/write property values of an object.
  2. XML serialization requires a default constructor to be declared in the class that is to be serialized.

What is serialization useful for?

  1. Persist the state of objects.
  2. Modify XML documents without using the DOM.
  3. Pass an object across applications and domains.
  4. Pass an object anywhere as a string.

That is stuff that is not just useful, it is essential. The problems with XmlSerialization are the constraints it puts on class design. They are deal-breakers for me- any technology that places such requirements on class design as XmlSerialization is terrible in my eyes. If I wrote some technology that had the requirements of XmlSerialization, and distributed it, I’d get harangued, and rightly so (there are complaints against nHibernate because every property must be virtual, this is infinitely worse in comparison). So what are the constraints/problems?

Note: I’m going to use the word ‘entity’ for a class that should only have one instance per-identity. There should only ever be one instance of Widget with the ID ’50935′ which other objects refer to.

Only public properties with public getters and settings are serialized.
This is the first of two huge slaps to good class design. One goal of designing good code is exposing the minimal public interface possible. Many classes only need a few public property setters, but this requires making them all public. Also, you cannot persist data that only exists in protected properties. Finally, you need to err on the side of making everything public, rather than choosing the most protected access possible from the start. These requirements are in direct contradiction to the ‘do not expose the internals of your class’- anyone consuming your class needs to have full access to all persistent components.

Serializable classes require a parameterless constructor.
Yikes. Few classes I write have a parameterless constructor, especially ones that need to persist. Entities need some sort of identification that is usually part of the constructor- serializable classes can have many instances of the same data.

The above two points are the most severe, since they fly in of both entities, which should have a single instance, and immutable classes, which have no public setters.

Little control over how objects serialize.
This is important. If you have a property that holds a reference to a class, it will only serialize if that class is serializable. And if it is serializable, but it is an entity, it will serialize a new instance. Interfaces will not serialize AT ALL.

Overcoming constraints is complex!
You are able to program around most of these problems. But if you’re going to do that, you need to really examine why you are using XmlSerialization. I know from experience with multiple teams here that most XML serialization with XmlSerialization does what is much better handled through custom serialization routines. I’d always first try to use custom serialization, and only use the built-in XmlSerialization is absolutely necessary, namely when doing much more advanced things with XML, such as, assuring certain typesafeties, namespaces, standard compliance, etc, or for the representation of relatively simple data objects, such as user preferences. But I see it most often used for simple data persistence or passing of data between .NET applications, and in those cases it is never my method of choice.

So that’s just a fraction of my case against XmlSerialization. You can find many more problems on the internet, but you’ll have to look into it, because they are rarely phrased as serious problems- it is one of those technologies that unassuming cargo-cultists gravitate towards. They have little understanding of the intended usage, power, costs, and other options, but they use it anyway. In a future post, I’ll go over strategies for custom serialization and how they address the problems associated with XmlSerialization.

No Comments

Extending LinqToXml for better XML interaction

25/07/2010

Many tools programmers or tech artists work with XML on a daily basis.  For .NET developers, there are three ways to work with XML.  You can use XML serialization (built in or custom), the System.Xml namespace, or LinqToXml (System.Xml.Linq).  I usually advocate XML serialization (with custom serialization routines!), but there are times when that isn’t practical.

In our case, we had a few hand-maintaned xml files that supported our animation system, which was undergoing a heavy rewrite by the gameplay team. They don’t maintain the animation tools code, though, so the serializing logic we wrote for the xml could easily break. Even worse, they could break our entire system logic if they make a logical change! So it was really impossible to maintain a fully logical system with serializing classes, as we always desire.

So we wrote ‘utility’ methods to do what we needed to do, and only what we needed to do, using System.Xml and XPath expressions. However, as anyone that has had to decipher tools written this way will tell you, this is a maintenance nightmare. You need to understand all the nuance of the underlying code to write, and each time you write something you can easily break the xml, since you’re working really, truly with raw xml. So we developed these ‘LinqToXml’ extensions to create a very light logical layer between underlying XML, representative classes, and procedural (utility) code.

Let’s take a look at an example XML file (vastly simplified):

<metadata>
  <actions>
    <input name="Stance">
      <value name="Standing" />
      <value name="Sitting" />
    </input>
    <input name="Weapon">
      <value name="Saber" />
      <value name="Gun" />
    </input>
  </inputs>
  <actions>
    <action name="Attack" blendInTime="0.1" blendOutTime="0.2">
      <path value="action|attack" />
      <input name="Stance" value="Standing" />
      <input name="Weapon" value="Saber" />
    </action>
    <action name="Heal" blendInTime="0.05" blendOutTime="0.03">
      <path value="action|heal" />
      <input name="Stance" value="Sitting" />
    </action>
  </actions>
</metadata>

Looking at the file, the reasons for a full logical system are obvious- validating the ‘input’ element on an action, for example.  But like I said, we needed to trade what’s right for what will work (we wrote the ‘right’ system for the xml 6 months ago, and the metadata system was changed right after that, invalidating some amount of work and resulting in a stagnation of the logical and serializing system for metadata).  Below is an example of the classes we would create to represent that XML.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml.Linq;

namespace FluentXml
{
    public class MetadataAction : XElement
    {
        public string ActionName
        {
            get { return this.Attribute("name").Value; }
            set { this.Attribute("name").Value = value; }
        }
        public decimal BlendInTime
        {
            get { return decimal.Parse(this.Attribute("blendInTime").Value); }
            set { this.Attribute("blendInTime").Value = value.ToString(); }
        }
        public string NetworkPath
        {
            get { return this.Element("path").Attribute("value").Value; }
            set { this.Element("path").Attribute("value").Value = value; }
        }
        public Dictionary Inputs
        {
            get { return this.Elements("input").ToDictionary(input => input.Attribute("name").Value, input => input.Attribute("value").Value); }
            set
            {
                this.Elements("input").Remove();
                foreach (KeyValuePair kvp in value)
                {
                    this.Add(new XElement("input", new XAttribute("name", kvp.Key), new XAttribute("value", kvp.Value)));
                }
            }
        }

        public MetadataAction(XElement other)
            : base(other)
        {
        }
    }
    public class MetadataInput : XElement
    {
        public string InputName
        {
            get { return this.Attribute("name").Value; }
            set { this.Attribute("name").SetValue(value); }
        }
        public IEnumerable InputValues
        {
            get
            {
                foreach (XElement inputVal in this.Elements("value"))
                {
                    yield return inputVal.Attribute("name").Value;
                }
            }
            set
            {
                this.Elements("value").Remove();
                foreach (string valname in value)
                {
                    this.Add(new XElement("value", new XAttribute("name", valname)));
                }
            }
        }

        public MetadataInput(XElement other)
            : base(other)
        {
        }
    }

    public class MetadataFile : XElement
    {
        public MetadataFile(XElement other)
            : base(other)
        {
        }

        public IEnumerable Inputs()
        {
            foreach (XElement child in this.Element("inputs").Elements().ToList())
            {
                MetadataInput result = child as MetadataInput;
                if (result == null)
                {
                    result = new MetadataInput(child);
                    child.ReplaceWith(result);
                }
                yield return result;
            }
        }
        public IEnumerable Actions()
        {
            foreach (XElement child in this.Element("actions").Elements().ToList())
            {
                MetadataAction result = child as MetadataAction;
                if (result == null)
                {
                    result = new MetadataAction(child);
                    child.ReplaceWith(result);
                }
                yield return result;
            }
        }
        public MetadataAction Action(string actionName)
        {
            foreach (MetadataAction action in this.Actions())
            {
                if (action.ActionName.Equals(actionName))
                {
                    return action;
                }
            }
            return null;
        }
        public MetadataInput Input(string inputName)
        {
            foreach (MetadataInput item in this.Inputs())
            {
                if (item.InputName.Equals(inputName))
                {
                    return item;
                }
            }
            return null;
        }
    }
}

This is really just ‘glue’ code that works with the underlying XML directly, but in a much more easily maintained format; instead of difficult-to-decipher XPath expressions, the classes make an easy-to-understand interface to the xml. In addition, we can write very fluent and concise utility expressions, for querying or processing, such as:

MetadataFile aam = new MetadataFile(XElement.Load("XMLFile1.xml"));
IEnumerable allActionNames = aam.Actions().Select(a => a.ActionName);
aam.Input("Weapon").InputValues = new string[] { "test1", "test2" };
aam.Actions().Where(a => a.BlendInTime < 0.5m).ToList().ForEach(a => a.BlendInTime = 0.5m);
var inputsAndActionsUsing = aam.Inputs().Select(input => new { Input = input, Actions = from a in aam.Actions() where a.Inputs.ContainsKey(input.InputName) select a });

This is a bit of an advanced topic but for people who are used to LINQ (and especially Linq2Xml), it should make sense, and is very, very powerful. We replaced all of our XPath utility methods for Metadata with this system in a couple hours, and boy was it worth it. In a future post I’ll go over a few improvements we can make, such as using a base ‘MetadataElement’ class for validation and better reuse, improving our IEnumerable yield methods, and improving MetadataFile by making it an XDocument to preserve top-level comments and provide more intuitive use.

2 Comments

Blogging again!

18/07/2010

I’m going to start blogging again (and more seriously for the first time).  I’ve been doing most of my posting over at www.tech-artists.org the past few years, but now with increased free time (no more crunch for me), and an expertise in a relevant area (I feel like I’m finally a competent enough .NET developer), I have some renewed reason to blog.  I’m going to aim to blog once week, no less than once a month, on topics mostly about game pipeline and .NET development.

1 Comment

Switch to our mobile site