Archive of articles classified as' "Programming"

Back home

Old towns, and legacy software


On our road trip from Austin to Portland, we stopped in a handful of towns that were booming in the late 19th century. In particular, Pendleton, Oregon made an impression. They were exhibiting serious effort and success revitalizing the town. Pendleton has a rich and interesting history, but has shrunk (relatively) over the last hundred years. It apparently used to be Oregon’s 4th largest city. There has clearly been a big effort to keep buildings in good condition, create interesting businesses, establish modern dining and arts, and maintain a beautiful and safe river walk. The people who live there seemed to hold a special bond.

Pendleton was quite different from many other towns we saw, which were in decline, derelict, or abandoned.

Pendleton reminded me of my experiences working on legacy software projects. Projects that no one wants to own, that have a successor “coming soon,” or are seen as unimportant; these are depressing to work on. On the other hand, where people have stepped up to really take control of the situation and invest serious passion into a legacy project; special bonds form that are as strong or stronger than can be forged when working on something new.

A town like Pendleton, like a legacy software project, is not going anywhere soon. You can invest in it, make it special, meaningful, and historical. Or you can make it a place for people too old or passive to move from. Towns and legacy software can deteriorate and die. Many have met this fate. Alternatively, they can embrace their legacy and reinvent themselves, with great pride, in the glow of their former selves.

A country of Pendletons is probably not very healthy, but a country with none would lack character and history.

No Comments

Two weeks is the worst sprint length


Mike Cohn over at Mountain Goat Software says this in My Primary Criticism of Scrum:

In today’s version of Scrum, many teams have become overly obsessed with being able to say they finished everything they thought they would. This leads those teams to start with the safe approach. Many teams never try any wild ideas that could lead to truly innovative solutions.

I believe a great deal of this is the result of the move to shorter sprints, with two weeks being the norm these days. Shorter sprints leave less time to recover if a promising but risky initial approach fails to pan out.

Most teams I have joined have been working in two week sprints, and it has always seemed like the wrong length. In fact, at CCP, I championed “anything but two weeks.” I saw teams unable to get features done and tested in two weeks, which caused teams to not take the “shippable increment” idea very seriously. Nearly every sprint was a failed sprint in one way or another.

My preferred solution was to move to one week sprints. I’ve written about one week sprints before. If Agile is all about feedback, then halving your sprint length gives you double the feedback and chances to improve. You can try something out for a week, find it doesn’t work, and move on without putting a release at risk. A sprint twice as long carries twice the risk. I feel this, more than longer sprints, much better addresses Mike’s goal to get back to creativity. That said, it requires an intense commitment from the team to adopt Agile and eXtreme Programming development practices, such as through automated testing, shared code ownership, and more. It also requires a team that can function and perform well, and someone to coach. Without these things, one week sprints will fail.

My backup solution, when I feel a team isn’t yet ready to go through the transformation one-week sprints bring about, is to move to longer sprints. This does not encourage the behavior I ultimately want to see, but at least it allows more experimentation. More importantly, teams can take the “shippable increment” ideal seriously.

This reasoning can be taken to absurdity, but avoid it. I once had someone seriously suggest two releases a year was too often, and we should go down to one release a year, so we could make sure it was high quality and bug free. These people just need a reminder of the history of software (and often their own company). Even stuff like printer firmware can be iterated on less than 4 weeks.

In the end, I still prefer one week sprints, which isn’t surprising as I’m an XP advocate. But if a team isn’t able or willing to move to one week sprints, don’t default to two weeks.

1 Comment

Escaping the Windows prison


My friend Brad Clark over at Rigging Dojo is organizing a course on Maya’s C++ API (I am assuming it is Maya but could be another program). He had a question about student access to Visual Studio, to which I responded:

As a programmer, the experience on Windows is pretty horrific. No built-in package manager. A shell scripting language designed in hell. A command line experience that is beyond frustrating. A tradition of closed-source and GUI-heavy tools that are difficult or impossible to automate. A dependence on the registry that still confounds me.

My eyes weren’t opened to this reality until I switched to Linux. I was on a Windows netbook that was barely working anymore. I installed Linux Mint XFCE and suddenly my machine was twice as fast. But the much more important thing that happened was exposure to modes of developing software that I didn’t even know existed (Python, Ruby, and Go made a lot more sense!). Programming in Windows felt like programming in prison. Developing on Linux made me a better programmer. Furthermore, If I didn’t end up learning the Linux tools and mindset on my own, working in the startup world would be basically impossible.

Programming on Windows revolves around Visual Studio and GUI tools. If you need evidence at how backwards Windows development is, look no further than Nuget. This was a revolution when it was released in 2010, changing the way .NET and Windows development was done. In reality, the release of Nuget was like electricity arriving in a remote village. It took ten years for C# to get a package manager. And it is for the rich villagers only: older versions of Visual Studio don’t work with Nuget.

The ultimate problems Windows creates are psychological. The technical differences change the way you think. It echoes the “your Python looks like Java” problem. Is the Windows mentality what we want students to take on? My last two jobs have been on Linux/OSX. I honestly cannot imagine having kept my head above water if I didn’t have a few years of self-taught Linux experience.

Windows still dominates in desktop OS popularity. That is all the more reason to make sure we are exposing student programmers to alternative ways of developing. I want new developers to be exposed to things outside the Microsoft ecosystem. It is too easy to become trapped in the Windows prison.


Metaprogramming with the type function


In my book, Practical Maya Programming with Python, I use Python’s type function for dynamically creating custom Maya nodes based on specifications, such as input and output attributes. I really love the type function*, so I thought I’d post another cool use of it.

I recently wrote this gem as a way to dynamically create exception types from error codes (it was for a now-defunct REST client):

def get_errtype(code):
    errtype = _errtype_cache.get(code)
    if errtype is None:
        errtype = type('Error%s' % code, (FrameworkError,), {})
        _errtype_cache[code] = errtype
    return errtype

Next is an uncommon form of Python's except statement. Its argument can be any expression that evaluates to a single exception type or sequence of exception types. You can actually call a function in the argument to except!

    return framework.do_something()
except framework.catch(404):
    return None

The framework.catch function is below. It looks up (and potentially creates) error types based on the error codes being caught:

def catch(*codes):
    return [get_errtype(c) for c in codes]

This sort of utility is why I wrote the type of book I did. Learning how to program in Python instead of MEL is all well and good. But you need to really see what Python is capable of to make big strides. I hope that with a more advanced understanding of Python, 3D developers can start creating frameworks and libraries, just like PyMEL, that other developers will work with and on for many years to come.

* I love the type function for a vain reason. It's more obscure than decorators, but not as difficult to understand as metaclasses.

No Comments

GeoCities and the Qt Designer


In a review of my book, Practical Maya Programming with Python, reviewer W Boudville suggests my advice of avoiding the Qt Designer is backwards-looking and obsolete, such as writing assembler instead of C for better performance, or using a text file to design a circuit instead of a WYSIWYG editor. I am quite sure he (assuming it is a he) isn’t the only person with such reservations.

Unfortunately, the comparison is not at all fair. Here’s a more relevant allegory:

Hey, did you hear about this awesome thing called geocities? You can build a website by just dragging and dropping stuff, no programming required!

We’ve had WYSIWYG editors for the web for about two decades (or longer?), yet I’ve never run into a professional who works that way. I think WYSIWYG editors are great for people new to GUI programming or a GUI framework, or for mock-ups, but it’s much more effective to do production GUI work through code. Likewise, we’ve had visual programming systems for even longer, but we’ve not seen one that produces a result anyone would consider maintainable. Sure, we’ve had some luck creating state machine tools, but we are nowhere close for the more general purpose logic required in a UI. And even these state machine tools are only really useful when they have custom nodes written in code.

Finally, WYSIWYG editors can be useful in extremely verbose frameworks or languages. I wouldn’t want to use WinForms in C# without the Visual Studio Designer. Fortunately for Pythonistas, PySide and PyQt are not WinForms!

I have no doubt that at some point WYSIWYG editors will become useful for GUI programming. Perhaps it will require 3D displays or massively better libraries. I don’t know. But for today and the foreseeable future, I absolutely discourage the use of the Qt Designer for creating production GUIs with Python.


A short letter to a unit testing newcomer


One of my friends asked how to get started with unit testing and Test Driven Development and figured I could write a short post. I also mention TDD a few times in my book so I think it could use some more attention.

I got started with unit testing when I wrote a smallish Python project, and realized I couldn’t maintain large Python projects without tests*. So I wrote unit tests for my code. I then quickly got into TDD. I don’t remember what resources I used, but there are plenty.

I’d encourage you to start unit testing by writing your code first, then the tests. As soon as this becomes easy and is paying off, start writing the tests first (TDD). You will come to appreciate that this is a powerful way of thinking and working. Even if you don’t ultimately stick with this approach, it is worth becoming skilled with it. You will uncover many suboptimal areas in your coding process that are otherwise extremely difficult to find and fix.

Keep in mind that learning TDD isn’t like already knowing Mercurial, then reading a book about Git**, and then being skilled with Git because you are skilled with Mercurial. You are embarking on a long journey, and will need to refer to many tutorials, blogs, and books. You will do some TDD, and look back on that code and process in a year and be horrified, just like you were when you started programming. Do not think of unit testing and TDD like learning a new framework or library. Think of it like learning how to program all over again.

So I don’t know exactly where to get started, only that you must, and keep going once you do start.

* I’d soon realize that no project was really maintainable without tests.

** Pro Git is my favorite Git book, by the way.

1 Comment

Code separators and headers are more than a matter of style


IDontCareIfYouPreferPascalCase, ifCamelCaseIsForYou, or_if_you_prefer_lowercase_underscores. They all have their merits. However, there is one element of many style guides that I have come to vehemently disagree with: function/class headers* and separators. I bring it up because I’ve encountered them in pretty much every codebase I’ve worked with, but never found it difficult to convince people to stop using them.

Many people have the belief that there’s no inherent superiority between having and not having headers and separators. Separators vs. no separators is like PascalCase vs. camelCase. “It’s just the way it is, follow the guide, be consistent.”

It’s a reasonable opinion, but wrong.

Headers and separators are more like the comments that say #Open the file directly above the line that says with open(somepath) as f. Or more likely, a comment that says #Don't write the file yet above the line that says requests.get(someurl). Wait, what does that comment refer to? No idea, because someone edited the code but not the comment.

We’ve known these sorts of comments are harmful for a long time. They involve out-of-band, redundant information that quickly rots. Having to create purely formal, redundant information (“a new function becomes here!”) is extra work that cannot be justified. Headers and separators are the same. “Added missing separator above function” is a commit message no one should ever have to write or read. Separators and headers are not a debate about readability. They are harmful because they are an impediment to changing code.

Please, if you use separators and headers, stop it immediately, or at least listen to the next jerk that comes in and wants to stop using them.

* : I am also against file headers, but I know they can be useful in some cases, such as when your code is mingled with client code and it can be important for people to know what code came from where. I can tolerate overhead that has a demonstrable benefit.


Practical Maya Programming with Python is Published


My book, Practical Maya Programming with Python has been finally published! Please check it out and tell me what you think. I hope you will find it sufficiently-but-not-overly opinionated :) It is about as personal as a technical book can get, being distilled from years of mentoring many technical artists and programmers, which is a very intimate experience. It also grows from my belief and observation that becoming a better programmer will, due to all sorts of indirect benefits, help make you a better person.

If you are using Python as a scripting language in a larger application- a game engine, productivity software, 3D software, even a monolithic codebase that no longer feels like Python- there’s a lot of relevant material here about turning those environments into more standard and traditional Python development environments, which give you better tools and velocity. The Maya knowledge required is minimal for much of the book. Wrapping a procedural undo system with context managers or decorators is universal. A short quote from the Preface:

This book is not a reference. It is not a cookbook, and it is not a comprehensive guide to Maya’s Python API. It is a book that will teach you how to write better Python code for use inside of Maya. It will unearth interesting ways of using Maya and Python to create amazing things that wouldn’t be possible otherwise. While there is plenty of code in this book that I encourage you to copy and adapt, this book is not about providing recipes. It is a book to teach skills and enable.

Finally, to those who pre-ordered, I’m truly sorry for all the delays. They’re unacceptable. I hope you’ll buy and enjoy the book anyway. At least I now have a real-world education on the perils of working with the wrong publisher, and won’t be making that same mistake again.

Thanks and happy reading!
Rob Galanakis


goless 0.7 released, with Python3 support and bug fixes


goless version 0.7.0 is out on PyPI. goless facilitates writing Go language style concurrent programs in Python, including functionality for channels, select, and goroutines.

I forgot to blog about 0.6 at the start of July, which brought Python 3.3 and 3.4 support to goless (#17). I will support pypy3 as soon as Travis supports it.

Version 0.7 includes:
– A “fix” for a gevent problem on Windows (socket must be imported!). #28
– Errors in the case of a deadlock will be more informative. For example, if the last greenlet/tasklet tries to do a blocking send or recv, a DeadlockError will be raised, instead of the underlying error being raised. #25
– goless now has a small exception hierarchy instead of exposing the underlying errors.
– Better PyPy stackless support. #29 can be called with (case1, case2, case3), etc., in addition to a list of cases (ie, ([case1, case2, case3])). #22

Thanks to Michael Az for several contributions to this release.

Happy concurrent programming!

No Comments

goless now on PyPI


goless is now available on the Python Package Index: . You can do pip install goless and get Go-like primitives to use in Python, that runs atop gevent, PyPy, or Stackless Python. You can write code like:

channel = goless.chan()

def goroutine():
    while True:
        value = channel.recv()
        channel.send(value ** 2)

for i in xrange(2, 5):
    squared = channel.recv()
    print('%s squared is %s' % (i, squared))

# Output:
# 2 squared is 4
# 3 squared is 9
# 4 squared is 16

I’ve also ported the goless benchmarks to Go, for some basic comparisons to using goless on various Python runtimes (PyPy, CPython) and backends (gevent, stackless):

Thanks to Rui Carmo we have more extensive examples of how to use goless (but if you’ve done or read any Go, it should be relatively straightforward). Check them out in the examples folder:

And just a compatibility note (which is covered in the docs, and explained in the exception you get if you try to use goless without stackless or gevent available): goless works out of the box with PyPy (using in its stdlib) and Stackless Python. It works seamlessly with gevent and CPython 2.7. It works with PyPy and gevent if you use the tip of gevent and PyPy 2.2+. It will support Python 3 as soon as gevent does.

Thanks and if you use goless, I’m eager to hear your feedback!

No Comments