Website and Blog of Rob Galanakis: Tools Programmer, Technical Artist/Animator

There was a recent AltDevBlogADay post about Negative Developers and Team Stability that hit home. It’s not that I think the advice was particularly interesting (good, standard stuff), it’s that it reminded be that I’ve been a negative developer.

I don’t know what I could have done differently. I just wasn’t happy at work, and there was little I could do to change it. The quality of my work was apparently very good, I was just terrible for morale, because I was either 1) pissing people off or 2) encouraging people to be pissed off at the problems I/we saw. Eventually I got the best advice I’ve ever gotten (which deserves its own blog post), and left the company. I went to the right place and became a positive developer.

And that’s sort of what struck me about the article and about how we typically deal with negative developers. Some developers are just not a good fit, regardless of how amazing their work is. If someone is negative because she is “culturally incompatible”, because there’s nothing you or your manager can do to fix it. And it is worth it to have a frank discussion about whether that person can ever be happy without changes to the studio, and if that person says ‘no’, you should discuss plans to part with mutual respect at a mutually agreed date.

I had to put in my two weeks at my last job to have this advice given to me by the President (GM? Can’t remember) at the time. It convinced me to un-quit, and to stay on another year. It ended up being a miserable year in many ways, but it was the right thing to do and worked out for the best. As managers- and friends and team members of negative developers- we need to keep this advice in mind when dealing with negative developers (and ourselves).

Some people can find me a bit over-earnest in my quest for automation. I’ve finally figured out how to know whether something is worthwhile to automate.

Are you making any decisions when you do this?

And if someone is making decisions that may be unnecessary:

Can we get from A to B without making any decisions?

I value the people I work with and I want them to be more effective. Sometimes people are afraid I’ll automate them out of a job, or they like a manual part of their workflow. More often they take pride in being the manual bottleneck behind a tricky or brittle process.

Humans are valuable because we can make complex decisions very quickly. When I automate an area someone isn’t immediately comfortable with, it’s not because I don’t value what he or she does. On the contrary: because I value their skills and time, I want to see them doing something that requires their unique abilities.

Writing up some performance reviews reminded me of my opinionated fact that a minority of people programming python really understand python. I think in statically typed languages, this is somewhat more acceptable. I think of how interesting “CLR via C#” was. It helped me understand the CLR and IL, as well as some much more general design patterns (damn you, properties!), but had minimal impact for how I used C# (except perhaps regarding performance and GC).

But in a fully dynamic language that has so much power- not understand how the language itself works puts you at a severe disadvantage. I’m not just talking about the one thousand aspects of meta-programming, but even just taking advantage of duck-typing (ie, instead of creating/passing a type, can you just pass a callable that returns a new instance?). Or how decorators work, or how importing works.

I am pretty sure I asked this when I started python but, are there any books (or courses) about python, written for more-or-less experienced programmers (even python programmers), that aren’t about ‘teaching python’ in the normal introductory (or cookbook/reference) way? More along the lines of CLR via C#, or some of the better python guides out there (Idiomatic Python). I am sure there are a large number of people out there who would get huge benefit from such a book or comprehensive guide.

The wife is interested in learning programming so she has been doing the Code Academy JavaScript course. It’s incredible to observe a person learning programming from the ground up, and how their mind works. It’s almost impossible to remember those early weeks of learning how to code yourself.

It’s been fun to see the frustrations, problems, and questions she’s had:

• For loops. So hideous. I don’t know who invented such the syntax “for (i=0; i<100;i++)”, but really I blame the people that continued the tradition into later languages where it is a truly stupid syntax.
• Speaking of i++/i–. What are these things and why do they exist?
• Curly braces. What a waste of space!
• var. When do I use it and when do I not use it?
• Why do I need to end my lines with semicolons?

They also have a python course, I’ll be interested to see how that one goes. I suspect it’ll make less sense since she’ll now expect all languages to work as poorly as Javascript :)

Also to preempt the comments about how this doesn’t mean anything about a language and we shouldn’t design languages for complete programming newbies- yes, I agree. Mostly I found this interesting because the grievances she’s had all have their roots in C-style languages yet they are found in JavaScript which was designed to be novice-friendly. Though admittedly the novice in 1995 and 2012 start with very different dispositions.

I’ll certainly have much more to say once she moves into the rest of JS, and then into Python.

Also, Code Academy is really awesome.

In my line of work I deal with junior programmers constantly. One of the most common non-code mistakes I see them make is using capital letters in their python file names. I’m not sure why it happens so often, to such a diverse crowd, but I guess it comes down to bad examples, and stylistic preference.

Except it isn’t stylistic preference. And unlike my rant about tabs-vs-spaces, it isn’t easy to fix either.

Imagine a simple scenario (note, this only applies to Windows- I should mention in my line of work we are usually on Windows) :

1. You create a file, myModule.py, you put some awesome code in it, and check it into source control (probably Perforce but AFAIK any source control has the same problems).
2. A while later, you rename it to mymodule.py, fix up your code references, and check it in.
3. The code breaks on all your users machines, and you can’t clean it up without manual intervention, or changing your module to be called something else.

Did you see what happened? When you checked in the code, everyone synced down ‘myModule.py.’ When you changed the capitalization, everyone else’s VCS synced down the new file contents but, because the file was already there, didn’t delete myModule.py. So everyone else got the new code calling ‘import mymodule’ but keeps the old filename ‘myModule.py’. Fixing this problem either requires removing the file from people’s client and force syncing (manually or by mass-deploying a fixup script), or just renaming ‘mymodule’ to something else entirely.

The problem stems from python, which is case-sensitive, relying on the filenames of paths, which rely on Windows, which is case insensitive. Don’t bother blaming anyone, it’s just the way it is.

So the way to deal with it? Always use lowercase, which is exactly what the PEP8 style guide suggests.

We’ve finally gotten our test coverage reports published to an accessible place and I’ve been wondering whether including test files as part of the coverage report is generally preferable to excluding them.

Initially I thought they should be excluded- after all, it’s your ‘real’ code people are generally worried about- but now I’m not so sure. You should probably make sure your test files are actually run and covered, as once you have a large amount of tests that often bend in various ways, it isn’t always a given. And actually that’s just the case  we were in today, we have tests that are still not runnable on our CI server, but the report doesn’t show which ones.

Not to mention having your test source easily browseable in the form of a coverage report, which is easier for test-curious people to browse than a source depot.

I think I’m going to re-include tests in the report, despite it skewing our coverage percentage (favorably :)). But I’m really curious if there’s accepted practice or good reasons either way I haven’t thought of.

I’ve never understood why people design systems in python to use config files. IMO there are two types of data and we can handle them both in dead simple ways:

First: Read/write data, like UI/user preferences. I say, just use python dictionaries, and serialize them however the hell you want (json, yaml, pickle, cPickle, I don’t care). I don’t understand why anyone would build anything more complex (except maybe a wrapper over this functionality), and especially why anyone would bother using a non-serialized format like ini or cfg. Can there be a good reason?

Second: Read-only data, like application configuration (nothing that is edited by your app). This, too, is very commonly in config files with special like xml or ini. Well in this case I say, why bother even with serialized data at all? Just make it code! This data, by definition, does not have to be statically known and serialized, it is only needed at runtime. So why not make it a python module, have your code import/exec the python file, and access attributes from the python module? There’s a good deal of data we may only know at runtime, or we cannot statically represent, so we end up making the systems that consume this configuration data much more complex. (For an example, see Sphinx’s config, I think it does it right).

An example of where I think python-as-configuration has huge power is with complex configuration data like for a plugin system. Instead of relying on some static functionality, the configuration can even be callables that return a plugin instance- so now you don’t have to refer to globals inside of your plugins, you can have your own initializer with a much larger signature, and the plugin system can have its well-defined initialization interface.

I’m not sure I did a great job explaining that example- it deserves its own post. We’ve used it extensively for new tools and systems we’ve developed and it seems to work great (and results in zero globals or singletons used by the plugins, which is an artifact of most plugin systems I’ve seen).

These are my opinions and practices and I know there are plenty good modules which rely on ini/xml/config files. Other than legacy decisions or to be compatible with pre-existing systems, why would someone choose a config file/format over 1) serialized python objects and/or 2) python code?

A poster on tech-artists.org who is using Autodesk’s CAT asks:

 The problem I´m having: I can control the ears now manually, after setting up the reactions, but apparently the CAT system isn´t respecting any keyframes I set on the facial controls, neither in setup nor animation mode.

He already hinted at the answer in the same post:

Even though I´ve heard a couple of times not to rely on CAT in production…

So there’s your answer.

Never depend upon closed-source frameworks that aren’t ubiquitous and proven.

It’s true of CAT, it’s true of everything. And in fact it is one reason I’ll never go back to developing with .NET on Windows (the 3rd party ecosystem, not the framework itself). If you don’t have the source for something, you 1) will never fully understand it, and 2) never be able to sustain your use of it. When I was at BioWare Austin, and the Edmonton guys decided to switch to CAT, I was up in arms for exactly this reason. We had an aging framework- PuppetShop- but it worked well enough, we had the source, and acceptable levels of support (Kees, the author, is/was readily available). Instead, they switched to a closed-source product (CAT) from a vendor that has repeatedly showcased its awful support (Autodesk), and headache ensued. Fortunately I never had to deal with it and left some months after that. Without the source, you’re dependent upon someone whose job it is to provide support just good enough so you don’t leave their product (which is difficult since they own everything), and bad enough that you have to hire them to fix problems (they pride themselves on this level of support, which I call extortion).

As for the ubiquitous and proven part: unless this closed source solution is incredibly widespread (think Max, Maya, etc.), and has a lot of involved power users (ie, Biped is widespread but difficult to get good community support for because it attracts so many novices), it means you have to figure out every single workaround because you can’t actually fix the problem because you don’t have source. Imagine working with no Google- that’s what you give up when you use a closed-source framework that isn’t an industry standard.

So don’t do it. Don’t let others do it. If you’re currently doing it, think about getting the source and if you can’t do that, start making a backup plan.

In December, I made a blog post about being happy. A anonymous (of course) commenter said:

Hey Rob. 5 months? Are you really this shortsighted? It’s like watching a kid say he’ll NEVER get tired of THIS toy. Frankly, if you don’t get tired of it you stopped growing. Take it as a good learning experience for the time when you forge your own destiny, but if you aren’t back to being a grumpy asshole in 6 months I’ll eat my hat. Not that I wear hats.

So 6 months (and a released expansion) later, have things changed? No, not one bit. In fact, late-night-drunk-rants have growth even more positive all around. Few of the bitch-sessions I’ve been used to in my career. It is an unfamiliar feeling. But addictive.

So why am I not back to being a ‘grumpy asshole’? Because there’s nothing to be grumpy about. There’s plenty wrong, but we’re moving at a steady pace in the right direction. And everyone is on board that it’s the right direction. No “we’ve decided this is the right direction so STFU and do what you’re told”. This happens because the talented people that complain loudly have several constructive outlets (I put myself in that group, at the risk of sounding egotistical).

One of the most exciting are communities of practice. Read the Wikipedia article if you like, but really they are a company-sanctioned forum for people with axes to grind to argue. Then we figure out what we want to do, and how to do it. And then we do it on some 15% time for skunkworks projects we’re able to reserve. Being able to talk about our common problems, across teams and responsibilities- we have everyone from tech artists to server programmers to the CTO there- is a refreshing experience.

(If you’re at a large company with lots of programmers who all aren’t on the same team, I’d highly suggest forming your own communities of practice).

The people who get shit done and can convince you of the way forward are given the responsibility to get more shit done. It’s nice. I think it has to do with the open source and python mentality.

This is a post for the Tech Artists and new programmers out there. I am going to answer the common question “why do my GUI’s lock up” and, to a lesser extent, “what can I do about it.”

I’m going to tell a story of a mouse click event. It is born when a user clicks her mouse button. This mouse click goes from the hardware, to the OS. The OS determines that the user wants to interact with your GUI, so it sends the mouse click event to your GUI’s process. Here, the mouse click sits in a queue of other similar events, just hanging out. At some point, usually almost immediately, your process (actually the main thread in the process) goes through and, one by one, dispatches the items (messages) in the queue. It has determined our mouse click is over a button and then tells the button it’s been clicked. The button then usually repaints itself so it looks pressed, and then invokes any callbacks that are hooked up. The process (main thread) goes into one such callback you hooked up, that will look at 1000 files on disk. This takes a while. In the mean time, the user is clicking, but the messages are just piling up in the queue. And then someone drags a window over your GUI, because they’re tired of their clicks not doing anything and want to see what’s new on G+. The OS sends a message to your UI that it needs to repaint itself, but that message, too, just sits in the queue. At some point, your OS may even realize your window is not responding, and fade it out and change the title bar. Finally your on-button-click callback finishes, the process (thread) is done processing our initial mouse click, and then goes back to processing the messages that may have accumulated in the queue, and your UI will refresh and start responding again.

All this happens because the thread that processes messages to draw the UI was also responsible for looking at 1000 files on disk, so it wasn’t around to respond to the paint and click messages. A few pieces of info:

1. You can’t just ‘update the UI’ from the middle of your code. In addition to being terrible form code-wise, clearing the message queue would just cause other things to block the main thread, and it’d all get into one giant asynchronous mess. Some programs may have their own UI framework that supports this. Don’t trust it. You really just need the main/GUI thread clear as much as possible to respond to events.
2. Your GUI process has a single ‘main thread.’ A thread roughly corresponds to, and I’m being not nuanced here, the software concept of a hardware CPU core. Your GUI objects can only be created and manipulated by the main thread.

This means, you want to keep your main thread free so it can act on GUI stuff (paint events, mouse clicks) only. The processing, such as your callback that looks at 1000 files, should happen on another thread (a background thread). When the processing is complete, it can tell the GUI thread that it is finished, and the GUI thread can update the UI. Your background thread can also fire events or invoke a callback that will be picked up by the GUI thread, so the GUI can update a progress bar or whatever.

How you actually do this varies with each UI framework. .NET, including WinForms and WPF, is quite easy to use (look at the BackgroundWorker class, but the Tasks Parallel Library and Async CTP make that less necessary). Python GUI frameworks are a bit worse off- multithreading in python in general is worse off- so it’ll be different for each one, and probably not as simple as .NET. There’s no excuse for python GUI’s to lock up, it just takes a little more effort to get it completely right (like callbacks to update a UI are a bit tricky).

There is one other vital thing to keep in mind- DCC programs generally require you to interact with the API or run all their script on the main thread, which as discussed should also be kept clear. Bummer! So the best thing we can do is block while we get our data from the scene, put the processing on a background thread, and report back to the main thread when done, applying the new data back to the scene if necessary. Unfortunately, if your processing interacts with the API in any way, you probably need to put it in the main thread as well. So, right now, your GUI’s in DCC apps may need to lock up, by design. There are, in theory, ways to avoid this, but they’re well outside of the scope of what you can handle if you’re learning anything from this article.

Whatever your language and program, those are the essentials of why your GUI locks up.

Note: This info is not nuanced (and is less accurate the lower down things go), may not be terminologically perfect (though it should be vulgarly comprehensible), and is Windows-only, though it should be enough to know how any higher-level GUI framework (such as Qt) would work on a non-Windows system).