Archive of articles classified as' "Tools and Pipeline"

Back home

Behavioral testing is the bee’s knees

10/11/2014

I have been doing Test Driven Development (TDD) with xUnit-based frameworks (like unittest and NUnit) for a number of years now, and started with RSpec in Ruby and Jasmine in JavaScript a few months ago. RSpec and Jasmine are behavioral testing frameworks that facilitate Behavioral Driven Development (BDD). BDD is really no different from “normal” TDD except for the frameworks used. BDD frameworks facilitate a different way of designing tests.

Behavioral testing is awesome and makes xUnit-style testing seem barbaric and uncivilized in comparison. I didn’t see the big deal for a long time. My xUnit style tests served me just fine. But now I’m hooked.

If you are a TDD person (or do any unit testing), I encourage you to try out BDD. You should get the hang of it quickly. It can take a little while to learn BDD idioms, but once you start doing things like custom matchers, it’s absolutely amazing.

In future posts I’ll try to discuss more of the differences between TDD and BDD, but for now I just provide a hearty endorsement of BDD and frameworks like RSpec and Jasmine.

3 Comments

GeoCities and the Qt Designer

25/08/2014

In a review of my book, Practical Maya Programming with Python, reviewer W Boudville suggests my advice of avoiding the Qt Designer is backwards-looking and obsolete, such as writing assembler instead of C for better performance, or using a text file to design a circuit instead of a WYSIWYG editor. I am quite sure he (assuming it is a he) isn’t the only person with such reservations.

Unfortunately, the comparison is not at all fair. Here’s a more relevant allegory:

Hey, did you hear about this awesome thing called geocities? You can build a website by just dragging and dropping stuff, no programming required!

We’ve had WYSIWYG editors for the web for about two decades (or longer?), yet I’ve never run into a professional who works that way. I think WYSIWYG editors are great for people new to GUI programming or a GUI framework, or for mock-ups, but it’s much more effective to do production GUI work through code. Likewise, we’ve had visual programming systems for even longer, but we’ve not seen one that produces a result anyone would consider maintainable. Sure, we’ve had some luck creating state machine tools, but we are nowhere close for the more general purpose logic required in a UI. And even these state machine tools are only really useful when they have custom nodes written in code.

Finally, WYSIWYG editors can be useful in extremely verbose frameworks or languages. I wouldn’t want to use WinForms in C# without the Visual Studio Designer. Fortunately for Pythonistas, PySide and PyQt are not WinForms!

I have no doubt that at some point WYSIWYG editors will become useful for GUI programming. Perhaps it will require 3D displays or massively better libraries. I don’t know. But for today and the foreseeable future, I absolutely discourage the use of the Qt Designer for creating production GUIs with Python.

12 Comments

Deploying a C# app through pip

28/05/2014

“If we bundle up this C# application inside of a Python package archive we can deploy it through our internal CheeseShop server with very little work.”

That sentence got me a lot of WTF’s and resulted in one of the worst abuses of any system I’ve ever committed.

We had a C# service that would run locally and provide an HTTP REST API for huge amounts of data in our database that was difficult to work with.* However, we had no good way to deploy a C# application, but we needed to get this out to users so people could start building tools against it.
I refused to deploy it by running it out of source control. That is how we used to do stuff and it caused endless problems. I also refused bottlenecks like IT deploying new versions through Software Center.

So in an afternoon, I wrote up a Python package for building the C# app, packaging it into a source distribution, and uploading to our Cheese Shop. The package also had functions for starting the packaged C# executable from the installed distribution. The package then became a requirement like anything else and was added to a requirements.txt file that was installed via pip.

What I initially thought was an unsightly hack ended up being perfect for our needs. In fact it caused us to eliminate a number of excessive future plans we had around distribution. I certainly wouldn’t recommend this “technique” for anything more than early internal distribution, but that’s what we needed and that’s what it did so effectively.

Getting something up and running early was extremely useful. It’s important to walk the line between the “we need to do several weeks of work before you see any result” and “I can hack something together we’re going to regret later,” especially for infrastructure and platform work. If code is well-written, tested, documented, and explained, you shouldn’t have qualms about it going into (internal) production. If the hackiness is hidden from clients, it can be easily removed when the time comes.


* Reworking the data was not an option. Creating this service would have allowed us to eventually rework the data, by providing an abstraction, though.

7 Comments

Global Glob

4/04/2014

I am cleaning out my drafts and found this two year old post titled “Globals Glob” with no content. The story is worth telling so here we go.

There was a class the EVE client used to control how missiles behaved. We needed to start using it in our tools for authoring missiles with their effects and audio. The class and module was definitely only designed (I used the term loosely) to run with a full client, and not inside of our tools, which are vanilla Python with a handful of modules available.

My solution was the GlobalsGlob base class, which was just a bag of every singleton or piece of data used by the client that was unavailable to our tools. So instead of:

serviceManager.GetService('someservice').DoThing(self.id)

it’d be:

self.globalsGlob.DoThing(self.id)

The ClientGlobalsGlob called the service, but FakeGlobalsGlob did nothing. The GlobalsGlob allowed us to use the missile code without having to rewrite it. A rewrite was out of the question, as it had just been rewritten, except using the same code. (sigh)

Unsurprisingly, GlobalsGlob was super-fragile. So we added a test to make sure the interface between the client and fake globs were the same, using the inspect module. This helped, but of course things kept breaking.

This all continued until the inevitable and total breakdown of the code. Trying to use the missile code in tools was abandoned (I think it was, I have no idea what state it’s in). This was okay though, as we weren’t using the missile tools much after those few months. GlobalsGlob served its purpose, but I will never be able to decide if it was a success or failure.

2 Comments

Introducing Host The Docs

22/03/2014

A month or so ago I created Host the Docs, and it’s been quietly running at work some success, so I figure it’s time to talk about it. The code is on GitHub at https://github.com/rgalanakis/hostthedocs, and here’s a description from the readme:

Host the Docs is a simple way to host static code documentation. Its main use is as a self-hosted server for your organization’s private documentation. Better alternatives are available for open source projects, such as Read the Docs or Github Pages.

Host the Docs was created after a long day of banging my head against the wall trying to get Read the Docs set up with private GitHub repositories, and having helped develop a plugin to get it to work with Perforce previously. What the world needed was a way to easily host documentation from several projects, from any source or language or SCM.

Seriously, let other people generate their own docs, I just want to Host the Docs!

It’s super easy to get set up, and requires no database. You can even just serve everything through Flask, if you don’t want to set up a webserver like Apache or nginx. It’s pretty well tested and configurable and thoroughly documented. We had it running on a new Linux VM in just a few minutes, and use it to host HTML documentation for Python and C# projects (we used Sphinx to generate the Python docs and Doxygen for C#).

I encourage you to try it out if you are interested in painless documentation hosting! I’m also very open to fixes/changes/pull requests, as long as it doesn’t overcomplicate things.

No Comments

Using Sonar for static analysis of Python code

15/02/2014

I’ve been doing static analysis for a while, first with C# and then with Python. I’ve even made an aborted attempt at a Python static code quality analyzer (pynocle, I won’t link to it because it’s dead). About a year ago we set up Sonar (http://www.sonarqube.org/) to analyze the Python code on EVE Online. I’m here to report it works really well and we’re quite happy with it. I’ll talk a bit about our setup in this post, and a future post will talk more about code metrics and how to use them.

Basic Info and Setup

Sonar consists of three parts:

  • The Sonar web interface, which is the primary way you interact with the metrics.
  • The database, which stores the metrics (Sonar includes a demonstration DB, production can run on any of the usual SQL DBs).
  • The Sonar Runner, which analyzes your code and sends data to the database. The runner also pulls configuration from the DB, so you can configure it locally and through the DB.

It was really simple to set up, even on Windows. The web interface has some annoyances which I’ll go over later, and sometimes the system has some unintuitive behavior, but everything works pretty well. There are also a bunch of plugins available, such as for new widgets for the interface or other code metrics checks. It has integrations with many other languages. We are using Sonar for both C++ and Python code right now. Not every Sonar metric is supported for Python or C++ (I think only Java has full support), but enough are supported to be very useful. There are also some worthless metrics in Python that are meaningful in Java, such as lines in a file.

The Sonar Runner

I’ll cover the Runner and then the Website. Every night, we have a job that runs the Runner over our codebase as a whole, and each sub-project. Sonar works in terms of “projects” so each code sub-project and the codebase as a whole have individual Sonar projects (there are some misc projects in there people manage themselves). This project setup gives higher-level people the higher-level trends, and gives teams information that is more actionable.

One important lesson we learned was, only configure a project on the runner side, or the web site. An example are exclusions: Sonar will only respect exclusions from the Runner, or the Web, so make sure you know where things are configured.

We also set up Sonar to collect our Cobertura XML coverage and xUnit XML test result files. Our Jenkins jobs spit these out, and the Runner needs to parse them. This caused a few problems. First, due to the way files and our projects were set up, we needed to do some annoying copying around so the Runner could find the XML files. Second, sometimes the files use relative or incomplete filenames, so parsing of the files could fail because the Python code they pointed to was not found. Third, the parsing errors were only visible if you ran the Runner with DEBUG and VERBOSE, so it took a while to track this problem down. It was a couple days of work to get coverage and test results hooked into Sonar, IIRC. Though it was among the most useful two metrics and essential to integrate, even if we already had them available elsewhere.

The Sonar Website

The Website is slick but sometimes limited. The limitations can make you want to abandon Sonar entirely :) Such as the ability to only few metrics for three time periods; you cannot choose a custom period (in fact you can see the enum value of the time period in the URL!). Or that the page templates cannot be configured differently for different projects (ie, the Homepage for the ‘Entire Codebase’ project must look the same as the Homepage for the ‘Tiny Utility Package’ project). Or that sometimes things just don’t make sense.

In the end, Sonar does have a good deal of configuration and features available (such as alerts for when a metric changes too much between runs). And it gets better each release.

The Sonar API

Sonar also has an API that exposes a good deal of metrics (though in traditional Sonar fashion, does not expose some things, like project names). We hook up our information radiators to display graphs for important trends, such as LoC and violations growth. This is a huge win; when we set a goal of deleting code or having no new violations, everyone can easily monitor progress.

Summary

If you are thinking about getting code metrics set up, I wholeheartedly recommend Sonar. It took a few weeks to get it to build up an expertise with it and configure everything how we wanted, and since then it’s been very little maintenance. The main struggle was learning how to use Sonar to have the impact I wanted. When I’ve written code analysis tools, they have been tailored for a purpose, such as methods/functions with the highest cyclomatic complexity. Sonar metrics end up giving you some cruft, and you need to separate the wheat from the chaff. Once you do, there’s no beating its power and expansive feature set.

My next post will go into more details about the positive effects Sonar and the use of code metrics had on our codebase.

3 Comments

Maya and PyCharm

15/01/2013

Recently at work I had the opportunity of spending a day to get Maya and PyCharm working nicely. It took some tweaking but we’re at a point now where we’re totally happy with our Maya/Python integration.

Remote Debugger: Just follow the instructions here: http://tech-artists.org/forum/showthread.php?3153-Setting-up-PyCharm-for-use-with-Maya-2013. We ended up making a Maya menu item that will add the PyCharm helpers to the syspath and start the pydev debugger client (basically, the only in-Maya instructions in that link).

Autocomplete: To get autocomplete working, we just copied the stubs from “C:\Program Files\Autodesk\Maya2013\devkit\other\pymel\extras\completion\py” and put them into all of the subfolders under: “C:\Users\rgalanakis\.PyCharm20\system\python_stubs” (obviously, change your paths to what you need). We don’t use the mayapy.exe because we do mostly pure-python development (more on that below), and we can’t add the stubs folder to the syspath (see below), so this works well enough, even if it’s a little hacky.

Remote Commands: We didn’t set up the ability to control Maya from PyCharm, we prefer not to develop that way. Setting that up (commandPort stuff, PMIP PyCharm plugin, etc.)  was not worth it for us.

Copy/Paste: To get copy/paste from PyCharm into Maya working, we followed the advice here:  http://forum.jetbrains.com/thread/PyCharm-671 Note, we made a minor change to the code in the last post, because we found screenshots weren’t working at all when Maya was running. We early-out the function if “oldMimeData.hasImage()”.

And that was it. We’re very, very happy with the result, and now we’re running PyCharm only (some of us were using WingIDE or Eclipse or other stuff to remote-debug Maya or work with autocomplete).

————————

A note about the way we work- we try to make as much code as possible work standalone (pure python), which means very little of our codebase is in Maya. Furthermore we do TDD, and we extend that to Maya, by having a base TestCase class that sends commands to Maya under the hood (so the tests run in Maya and report their results back to the pure-python host process). So we can do TDD in PyCharm while writing Maya code. It is confusing to describe but trivial to use (inherit from MayaTestCase, write code that executes in Maya, otherwise treat it exactly like regular python). One day I’ll write up how it works in more depth.

3 Comments

Why bother with python and config files?

26/06/2012

I’ve never understood why people design systems in python to use config files. IMO there are two types of data and we can handle them both in dead simple ways:

First: Read/write data, like UI/user preferences. I say, just use python dictionaries, and serialize them however the hell you want (json, yaml, pickle, cPickle, I don’t care). I don’t understand why anyone would build anything more complex (except maybe a wrapper over this functionality), and especially why anyone would bother using a non-serialized format like ini or cfg. Can there be a good reason?

Second: Read-only data, like application configuration (nothing that is edited by your app). This, too, is very commonly in config files with special like xml or ini. Well in this case I say, why bother even with serialized data at all? Just make it code! This data, by definition, does not have to be statically known and serialized, it is only needed at runtime. So why not make it a python module, have your code import/exec the python file, and access attributes from the python module? There’s a good deal of data we may only know at runtime, or we cannot statically represent, so we end up making the systems that consume this configuration data much more complex. (For an example, see Sphinx’s config, I think it does it right).

An example of where I think python-as-configuration has huge power is with complex configuration data like for a plugin system. Instead of relying on some static functionality, the configuration can even be callables that return a plugin instance- so now you don’t have to refer to globals inside of your plugins, you can have your own initializer with a much larger signature, and the plugin system can have its well-defined initialization interface.

I’m not sure I did a great job explaining that example- it deserves its own post. We’ve used it extensively for new tools and systems we’ve developed and it seems to work great (and results in zero globals or singletons used by the plugins, which is an artifact of most plugin systems I’ve seen).

These are my opinions and practices and I know there are plenty good modules which rely on ini/xml/config files. Other than legacy decisions or to be compatible with pre-existing systems, why would someone choose a config file/format over 1) serialized python objects and/or 2) python code?

23 Comments

Never build upon closed-source frameworks

20/06/2012

A poster on tech-artists.org who is using Autodesk’s CAT asks:

 The problem I´m having: I can control the ears now manually, after setting up the reactions, but apparently the CAT system isn´t respecting any keyframes I set on the facial controls, neither in setup nor animation mode.

He already hinted at the answer in the same post:

Even though I´ve heard a couple of times not to rely on CAT in production…

So there’s your answer.

Never depend upon closed-source frameworks that aren’t ubiquitous and proven.

It’s true of CAT, it’s true of everything. And in fact it is one reason I’ll never go back to developing with .NET on Windows (the 3rd party ecosystem, not the framework itself). If you don’t have the source for something, you 1) will never fully understand it, and 2) never be able to sustain your use of it. When I was at BioWare Austin, and the Edmonton guys decided to switch to CAT, I was up in arms for exactly this reason. We had an aging framework- PuppetShop- but it worked well enough, we had the source, and acceptable levels of support (Kees, the author, is/was readily available). Instead, they switched to a closed-source product (CAT) from a vendor that has repeatedly showcased its awful support (Autodesk), and headache ensued. Fortunately I never had to deal with it and left some months after that. Without the source, you’re dependent upon someone whose job it is to provide support just good enough so you don’t leave their product (which is difficult since they own everything), and bad enough that you have to hire them to fix problems (they pride themselves on this level of support, which I call extortion).

As for the ubiquitous and proven part: unless this closed source solution is incredibly widespread (think Max, Maya, etc.), and has a lot of involved power users (ie, Biped is widespread but difficult to get good community support for because it attracts so many novices), it means you have to figure out every single workaround because you can’t actually fix the problem because you don’t have source. Imagine working with no Google- that’s what you give up when you use a closed-source framework that isn’t an industry standard.

So don’t do it. Don’t let others do it. If you’re currently doing it, think about getting the source and if you can’t do that, start making a backup plan.

8 Comments

Why GUI’s Lock Up

8/04/2012

This is a post for the Tech Artists and new programmers out there. I am going to answer the common question “why do my GUI’s lock up” and, to a lesser extent, “what can I do about it.”

I’m going to tell a story of a mouse click event. It is born when a user clicks her mouse button. This mouse click goes from the hardware, to the OS. The OS determines that the user wants to interact with your GUI, so it sends the mouse click event to your GUI’s process. Here, the mouse click sits in a queue of other similar events, just hanging out. At some point, usually almost immediately, your process (actually the main thread in the process) goes through and, one by one, dispatches the items (messages) in the queue. It has determined our mouse click is over a button and then tells the button it’s been clicked. The button then usually repaints itself so it looks pressed, and then invokes any callbacks that are hooked up. The process (main thread) goes into one such callback you hooked up, that will look at 1000 files on disk. This takes a while. In the mean time, the user is clicking, but the messages are just piling up in the queue. And then someone drags a window over your GUI, because they’re tired of their clicks not doing anything and want to see what’s new on G+. The OS sends a message to your UI that it needs to repaint itself, but that message, too, just sits in the queue. At some point, your OS may even realize your window is not responding, and fade it out and change the title bar. Finally your on-button-click callback finishes, the process (thread) is done processing our initial mouse click, and then goes back to processing the messages that may have accumulated in the queue, and your UI will refresh and start responding again.

All this happens because the thread that processes messages to draw the UI was also responsible for looking at 1000 files on disk, so it wasn’t around to respond to the paint and click messages. A few pieces of info:

  1. You can’t just ‘update the UI’ from the middle of your code. In addition to being terrible form code-wise, clearing the message queue would just cause other things to block the main thread, and it’d all get into one giant asynchronous mess. Some programs may have their own UI framework that supports this. Don’t trust it. You really just need the main/GUI thread clear as much as possible to respond to events.
  2. Your GUI process has a single ‘main thread.’ A thread roughly corresponds to, and I’m being not nuanced here, the software concept of a hardware CPU core. Your GUI objects can only be created and manipulated by the main thread.

This means, you want to keep your main thread free so it can act on GUI stuff (paint events, mouse clicks) only. The processing, such as your callback that looks at 1000 files, should happen on another thread (a background thread). When the processing is complete, it can tell the GUI thread that it is finished, and the GUI thread can update the UI. Your background thread can also fire events or invoke a callback that will be picked up by the GUI thread, so the GUI can update a progress bar or whatever.

How you actually do this varies with each UI framework. .NET, including WinForms and WPF, is quite easy to use (look at the BackgroundWorker class, but the Tasks Parallel Library and Async CTP make that less necessary). Python GUI frameworks are a bit worse off- multithreading in python in general is worse off- so it’ll be different for each one, and probably not as simple as .NET. There’s no excuse for python GUI’s to lock up, it just takes a little more effort to get it completely right (like callbacks to update a UI are a bit tricky).

There is one other vital thing to keep in mind- DCC programs generally require you to interact with the API or run all their script on the main thread, which as discussed should also be kept clear. Bummer! So the best thing we can do is block while we get our data from the scene, put the processing on a background thread, and report back to the main thread when done, applying the new data back to the scene if necessary. Unfortunately, if your processing interacts with the API in any way, you probably need to put it in the main thread as well. So, right now, your GUI’s in DCC apps may need to lock up, by design. There are, in theory, ways to avoid this, but they’re well outside of the scope of what you can handle if you’re learning anything from this article.

Whatever your language and program, those are the essentials of why your GUI locks up.

Note: This info is not nuanced (and is less accurate the lower down things go), may not be terminologically perfect (though it should be vulgarly comprehensible), and is Windows-only, though it should be enough to know how any higher-level GUI framework (such as Qt) would work on a non-Windows system).
2 Comments