Archive of published articles on June, 2012

Back home

Why bother with python and config files?


I’ve never understood why people design systems in python to use config files. IMO there are two types of data and we can handle them both in dead simple ways:

First: Read/write data, like UI/user preferences. I say, just use python dictionaries, and serialize them however the hell you want (json, yaml, pickle, cPickle, I don’t care). I don’t understand why anyone would build anything more complex (except maybe a wrapper over this functionality), and especially why anyone would bother using a non-serialized format like ini or cfg. Can there be a good reason?

Second: Read-only data, like application configuration (nothing that is edited by your app). This, too, is very commonly in config files with special like xml or ini. Well in this case I say, why bother even with serialized data at all? Just make it code! This data, by definition, does not have to be statically known and serialized, it is only needed at runtime. So why not make it a python module, have your code import/exec the python file, and access attributes from the python module? There’s a good deal of data we may only know at runtime, or we cannot statically represent, so we end up making the systems that consume this configuration data much more complex. (For an example, see Sphinx’s config, I think it does it right).

An example of where I think python-as-configuration has huge power is with complex configuration data like for a plugin system. Instead of relying on some static functionality, the configuration can even be callables that return a plugin instance- so now you don’t have to refer to globals inside of your plugins, you can have your own initializer with a much larger signature, and the plugin system can have its well-defined initialization interface.

I’m not sure I did a great job explaining that example- it deserves its own post. We’ve used it extensively for new tools and systems we’ve developed and it seems to work great (and results in zero globals or singletons used by the plugins, which is an artifact of most plugin systems I’ve seen).

These are my opinions and practices and I know there are plenty good modules which rely on ini/xml/config files. Other than legacy decisions or to be compatible with pre-existing systems, why would someone choose a config file/format over 1) serialized python objects and/or 2) python code?


Never build upon closed-source frameworks


A poster on who is using Autodesk’s CAT asks:

 The problem I´m having: I can control the ears now manually, after setting up the reactions, but apparently the CAT system isn´t respecting any keyframes I set on the facial controls, neither in setup nor animation mode.

He already hinted at the answer in the same post:

Even though I´ve heard a couple of times not to rely on CAT in production…

So there’s your answer.

Never depend upon closed-source frameworks that aren’t ubiquitous and proven.

It’s true of CAT, it’s true of everything. And in fact it is one reason I’ll never go back to developing with .NET on Windows (the 3rd party ecosystem, not the framework itself). If you don’t have the source for something, you 1) will never fully understand it, and 2) never be able to sustain your use of it. When I was at BioWare Austin, and the Edmonton guys decided to switch to CAT, I was up in arms for exactly this reason. We had an aging framework- PuppetShop- but it worked well enough, we had the source, and acceptable levels of support (Kees, the author, is/was readily available). Instead, they switched to a closed-source product (CAT) from a vendor that has repeatedly showcased its awful support (Autodesk), and headache ensued. Fortunately I never had to deal with it and left some months after that. Without the source, you’re dependent upon someone whose job it is to provide support just good enough so you don’t leave their product (which is difficult since they own everything), and bad enough that you have to hire them to fix problems (they pride themselves on this level of support, which I call extortion).

As for the ubiquitous and proven part: unless this closed source solution is incredibly widespread (think Max, Maya, etc.), and has a lot of involved power users (ie, Biped is widespread but difficult to get good community support for because it attracts so many novices), it means you have to figure out every single workaround because you can’t actually fix the problem because you don’t have source. Imagine working with no Google- that’s what you give up when you use a closed-source framework that isn’t an industry standard.

So don’t do it. Don’t let others do it. If you’re currently doing it, think about getting the source and if you can’t do that, start making a backup plan.