Website and Blog of Rob Galanakis: Tools Programmer, Technical Artist/Animator

We’re currently undergoing a push to do more ‘Agile testing’ at work. At CCP, we “do” Agile pretty well, but I don’t think we “code” Agile really well. A large part (the largest part?) of Agile coding and Agile testing is TDD/Unit Testing, of which I’m a huge fan if not an experienced practitioner.

I know how to do TDD for what I do- that is, side-by-side replacement of an internal legacy codebase in a high-level language. What I don’t have experience in is TDD for expansion and maintenance of a huge, high performance, very active, legacy codebase, and specifically the graphics components and the C++ bits.

So if you have experience in these sorts of things, I’d love to hear them.

At this point I’m sticking my neck out as a TDD and unit testing advocate studio-wide, but am reluctant to evangelize too strongly outside of my areas of expertise. I don’t think it’d be fair to the very talented people in those areas, and I also don’t want to be wrong, even if I know I’m right :) So I’d really like to hear about your experiences with TDD and unit testing in the games and graphics space, and on legacy codebases, because people are coming to me with questions and I don’t have good answers. I’d love to give them places to turn, articles to read, people to contact.

Thanks for any help.

I have no problem saying that I write good code. I place a focus on TDD and thorough unit and integration testing. I document everything I write (not just function documentation- I document classes, modules, and systems). The fact is, since I’ve been doing these two things somewhat religiously, the amount of time I have spent debugging code has gone down dramatically. There are just not many bugs to find in most of the code I write, they are easy to narrow down when I find them, and they rarely regress.

This is a good thing, isn’t it? So why do I hate TDD?

Because debugging is fun. There, I said it. I love debugging. I think lots of clever people like debugging. I love someone having a problem, coming to me, looking at it together, getting up to walk around, look at the ceiling, talk to myself, stand in front of a whiteboard, draw some lines that spark some idea, try it, manually test a fix out, slouch down in my chair staring at my computer lost in thought, and repeating this until I actually find and fix the problem. Not just think I fixed it, but really sure that I fixed it because suddenly it all makes sense. At which point I spring a terrific boner for my obviously superior brain power that was able to find this problem that plagued mere mortals.

So the sad fact is, since I’ve been doing TDD, I haven’t been able to go on this ego-trip a single time in our TDD-written codebase. Sure, sometimes we get a bug in the UI, but those usually manifest easily. Oh, and I’ve definitely used some frameworks and API’s improperly and caused bugs because of that. But those are the annoying types of debugging we all have to do, not the  magical mystery tour described above.

I didn’t realize how much I missed debugging until it was gone. Fortunately, there’s still lots of legacy code and API’s to get my fix from.

I was doing a code review today on a feature added to some poorly-architected system, and compared it to the code required for features added to well-architected systems. I suspect the quality of a system can be determined by how many ‘if’ statements are required to add a new feature. Poorly architected systems have the peculiar attribute that adding features is often a matter of littering or adjusting ‘if’ statements through the system, often with a class or function or two added somewhere. Well architected systems can normally see features added by scaling “horizontally”- creating new classes or functions, passing those ‘into’ the system, without any actual adjustments to the system code itself. Furthermore I usually find well architected systems easier to refactor, so places that would require an ‘if’ statement to be added or changed, can be refactored to avoid or remove the ‘if’ statement, often making the core of the system itself simpler overall.

Games Developers Conference and PyCon are both coming up, which means lots of familiar international travelers in the US. A post on G+ asked about how the tipping system works in the US. I’m not going to list the percentages and people (though I will say aim for 20% and always tip taxis and servers), but I will explain three very important things about ‘how the tipping system works.’

The first thing to understand is, tips are generally a part of wages. The federal minimum wage is $7.25, but may only be $3 or so for an employee that earns tips, because the rest of the money ‘must’ be made up in tips. Tipping is never “extra” money to a tip earner (takeaway, baristas, etc., are generally not ‘tip earners’).

Second is, servers do not generally declare how much they made- there is a standard percentage of sales they must be declare as tips (differs per state but generally something like 10-13%). So if they make $1000 of sales and make $250 in tips, they are allowed to say they only made $130 (%13) in tips and only pay taxes on that (I say allowed, this is ‘illegal’ but everyone does it and there is no expectation to report everything you earn). However, this has an inverse- if you make $1000 in sales and only make $100 in tips, you pay taxes on the $30 you didn’t even make! So if you tip someone 10%, they may be paying taxes on money they didn’t make- you are taking money out of their pocket.

Last and most important is, many servers need to ‘pay out’ to other staff, such as bussers and food runners. This can often be 5% of sales or so. So they may only be taking home 10% of your 15% tip. And the blowback from not tipping is far worse here- if you tip 5%, that server may be earning exactly zero dollars from your table. If you skip on the tip entirely, you are taking money out of your server’s pocket.

So, Europeans, and Americans too, please understand- tipping should never be considered optional, even for bad service. Unless you think it is OK to take money out of someone’s pocket for a job poorly done, or even just if they made some mistakes. Imagine if your pay was docked for each bug you wrote! The only time I would ever not tip is if you were to walk out of the restaurant (for lack of service or some other dealbreaker). Likewise, your server is almost never getting as much of your tip as you write (taxes, payouts to other staff). So if you get good service, tip generously, then add a dollar or two. And if you get bad service, tip anyway.

Enjoy the conferences!

PS- laws are different in each state and restaurants are different. These are just general guidelines. Please don’t nitpick exceptions.

I’ve been following John’s blog for about 9 months and it is one of my favorite blogs. Poignant, digestable posts, that usually get me thinking. The programming posts are the best around, though the math and statistics posts often go over my head (thankfully there aren’t a ton of those). He seems to read a new book every 3 days so a handful of his posts are just really interesting quotes from those books. He has an honesty and seriousness that is missing in some other bloggers I like (Scott Hanselman), but posts much more regularly and in more digestable chunks than some other bloggers I love (Eric Lippert, Jon Skeet). I’d highly suggest subscribing to John’s blog.

My favorite recent posts:

• http://www.johndcook.com/blog/2012/02/16/csharp-batch-file/
• http://www.johndcook.com/blog/2012/02/14/hard-soft/
• http://www.johndcook.com/blog/2012/02/09/programming-langauge-popularity/
• http://www.johndcook.com/blog/2012/02/06/ceiling-of-complexity/
• http://www.johndcook.com/blog/2012/02/03/limiting-your-options-creates-better-options/

http://www.johndcook.com/blog/

In the comments on post “Internal Tools Only Require the Critical Path“, Robert Kist points out a few problems to think about when developing internal tools that may have to be used by outsourcers.

I absolutely agree and writing outsource-compliant tools are something many TA’s (including myself) can struggle with. The biggest things are:

Security: Assume your users do not have administrator rights. This shouldn’t be a problem if you are doing things the “right way”, rather than the “hard-coded expedient at the moment” way.

SCM Clients: Do your outsourcers have SCM clients? If not, what does that mean if you scatter your SCM interaction throughout your tools? Are you observing best practices and trying to make your SCM interaction transactional, or are you just making calls at the earliest possible moment?

Network resources: Do your tools require access to network resources, and if so, do the outsourcers have access to them? This could be a database, or a file on the network. Consider how you can break or mock these dependencies (or get rid of them entirely).

Latency: If your tools do require network resources, are they hitting the database/network constantly? If so, expect your tools to be incredibly slow, since instead of access time measured in milliseconds over distances of hundreds of meters, expect access time requiring many seconds over several thousand kilometers. Figure out how to reduce/batch your DB calls/network IO, or mock them out (perhaps provide read-only copies on disk, and point your DB at them).

Machine setup/bootstrapping: What sort of configuration are you relying on, in terms of file setup/folder structure, and in terms of global state of the machine (environment variables, drive letters). Not building in any dependencies on global machine state ensure your tools are much more portable.

Localization/internationalization: Not every international outsourcer speaks English. Many places will translate documentation when settings things up, but that documentation can get stale. We should start thinking about writing localizable code and documentation.

Not so coincidentally- writing good code helps you make more portable tools. A lot of these problems aren’t really problems at all and they don’t really require extra work- they just require care and craftsmanship while developing your tools and code. Focusing on the critical path doesn’t give you a license to write bad code. In fact the contrary. The focus on the critical path means you should have excellent code at all times because the critical path is critical. So if you write good code, as you should, supporting outsourcers will be a lot easier when that time comes.

Do you have any experience or stories writing tools for outsourcers? I’d love to hear about them in the comments.

Refactoring is defined as a “disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior” by Martin Fowler.

But its everyday usage takes a very different meaning. We use the word refactor to everything from the original meaning, to a complete side-by-side rewrite of a barely-functional system. How can you use the same word to refer to something that does not change external behavior, with building an entirely new replacement system?

More than once, I’ve elicited sighs for my efforts to clarify the language we use. But when you have people that use wildly different vocabularies- artists, programmers, project managers- this is of paramount importance. So the fact that we say ‘refactor’ to meaning any type of rewriting of code or functionality irks me.

So recently I’ve begun a dictionary in my head for the type of tasks we do.

Refactor: The original and ‘precise’ meaning of restructuring a module without altering external behavior, up to a rewrite of parts of a larger system, that may change internal behavior of the system (including the external behavior of large internal components). While I’d love to keep only the more limited original meaning, when we’re dealing with large legacy codebases that often have zero tests, the original meaning doesn’t apply often enough.

Renovate: Rewriting a system so that its external behavior changes considerably but it still fulfills the same purpose (obviously). I name it as such because it is like renovating a building- the exterior and interior may greatly change, but what goes on inside may stay the same. Similar to ‘Rewrite’ but generally applies to a smaller (module/system) scale. An example would be, if you have a library for dealing with source control, and you no longer like the API. So you greatly change how the module works, and it still fulfills its fundamental purpose of dealing with source control.

Recycle: Writing a replacement system side-by-side with the old system, using components from the old system (by either referencing or copy/pasting), with the goal that once the new system is working, the old system will be shut down. The goal is a replacement that is easier to use but fulfills similar requirements. An example would be replacing legacy procedures for data transformations, that may have a lot of imperative code and poor reuse. You would replace the system with something better written, often taking chunks of logic from the old system, or writing tests that verify it produces the same results, then hook up calling code to use the new system, then delete the old one entirely.

Rewrite/Rebuild: When a system is to be written from the ground up, using the original implementation as an example or in a prototype role only, resulting in a system that fulfills the business requirements of the original system but does not necessarily preserve anything else about it. Similar to ‘Renovating’ but generally applies to a larger (application) scale. An example would be, if you have a website/game feature that you want to replace, you’d build the new one to fulfill the same business requirement (we need a website/some feature), but the features may be completely different.

So that’s my personal language right now. My hope is that it will expand to my team and then out from there. I don’t know if I’ll refine it much more- too granular and it would become unwieldy. Do you have suggestions or a language of your own?

I have seen this in both python and C#:

class Spam:
    def __init__(self):
        self.eggs = list()
    ...more code here...
    def clear():
        self.eggs = list()

What do I consider wrong here? The variable ‘eggs’ is what I call ‘doubly mutable’. It is a mutable attribute of a mutable type. I hate this because I don’t know a) what to expect and b) what the usage pattern for the variable is. This confusion applies to both people using your class (if they take references to the value of the attribute, either knowingly or unknowingly), as well as coders who are maintaining your class who will have to look every time to know which pattern you follow (clearing the list, or reassigning to a new list).

IMO you should have one of the following. This first example is preferred but as it involves an immutable collection, isn’t always practical as a replacement for this antipattern (which usually involves mutating a collection).

class Spam:
    def __init__(self):
        self.eggs = tuple() #Now immutable, so it must be reassigned to change
    ...more code here...
    def clear():
        self.eggs = tuple()

in which case, you know it is a mutable attribute of an immutable type. So you can’t change the value of the instance, you must reassign it.

Or you can do:

class Spam:
    def __init__(self):
        self.eggs = list()
    ...more code here...
    def clear():
        del self.eggs[:] #Empty the list without reassigning the attribute.

in which case, you understand it is to be considered an immutable attribute of a mutable type. So you would never re-set the attribute, you always mutate it. If you can’t use the tuple alternative, you should always use this alternative- there’s never a reason for the original example.

In C#, we could avoid this problem entirely by using the ‘readonly’ field keyword for any mutable collection fields. In python, since all attributes are mutable, we must do things by convention (which is fine by me).

I always fix or point out this pattern when I find it and I consider it an antipattern. Does it bother you as much, or at all?

I see this a lot:

class Foo:
    spam = None
    eggs = None

def frob(foo):
    return sprocket(str(foo.eggs))

f = Foo()
s = frob(f)

It tends to be more sinister, and difficult to see, in verbose examples. But generally it is easily identified by the called method using a single attribute or
method from the object passed in (or multiple in longer functions that should be split up ;) ). Sometimes I bring this up and say, “pass in the value directly,” and the ‘why’ clicks right away. Sometimes people (including my older self) say “but taking in a ‘foo’ encapsulates my method!”

I guess.  It certainly hides the detail that `frob` needs only `.eggs` and doesn’t also need `.spam`. But you’ve also coupled the implementation of `frob` to the interface of `Foo`. So you’ve achieved encapsulation by greatly increasing coupling.

Of the two, I’d vastly prefer a method that must take additional parameters if its implementation changes (ie, if it needs access to `.spam`), than increase coupling. High coupling leads to brittle, untestable, and non-reusable code. Changing the interface of a method leads to… what exactly?

Not only that but the contract of a method is much clearer (to both callers and maintainers) if it takes in meaningful parameters, rather than a single object which it accesses a bunch of properties of. It conveys more information for callers, and establishes what it is supposed to do to maintainers (who will not be able to just get or set the attribute of an object that happened to be passed into that method because it was a convenient place to do so).

So it is usually vastly preferable to take in the values the function uses, rather than pass around complex objects, and in fact this is a common design paradigm in functional programming. But obviously I’m not just using strings and ints everywhere. So what guidelines do I follow?

1. Immutable objects are fine to pass around (though prefer the advice about just listing what the function takes as per above).
2. Mutable objects should never be passed around, as I consider creating an object and passing it to a method that mutates it one of the greatest sins in OOP.

Exposing a UI to twiddle all the individual aspects of an object’s state is rarely a good UI design. Figure out how to abstract your system’s workings to simplify the UI. Graphics settings are a good example of this- there may be hundreds of bits to twiddle for your graphics settings, but generally you expose some slider or broad categories, and allow the user to override ‘advanced’ settings.

It is more difficult to apply it to many of the tools we write than it is for the trivial example of a settings menu, but we rarely try. We too often just expose every setting to a UI and call it a tool. It’d be easier to use a config file! Figure out how to simplify things for the user and their common use cases.

You may also think about moving this abstraction out of the UI layer, into an abstraction that sits on top of your lower ‘data’ layers. Then your UI can just bind to this simpler abstraction layer.

Tools can be difficult to use and simple, or easy to use and complex. Best of all is easy to use and simple. We know simplicity is a goal with systems design. It should also be a goal in tools design, and tools that end up just exposing widgets for the underlying data should be considered as poor as systems that are too complex.