Archive of articles classified as' "All Posts"

Back home

Grabbing for good enough

15/12/2014

Uncle Bob, who I consider my favorite programming writer, had a post a few weeks ago titled “Thorns around the Gold“. In it he describes how writing tests for your core functionality first can be harmful. Instead, Uncle Bob prefers to probe for “thorns” around the “gold” first.

I shy away from any tests that are close to the core functionality until I have completely surrounded the problem with passing tests that describe everything but the core functionality. Then, and only then, do I go get The Gold.

I haven’t been doing TDD for nearly as long as Uncle Bob but I was shocked to read this. I’ve always learned and taught that you should create positive tests first, and only need as many negative tests as you feel are warranted. While you may not grab the gold immediately, you at least step towards the gold. How many thorns you expose is a judgement call. In Python, most people don’t even bother validating for None inputs, and instead just let things raise (or not). Of course, this depends on your users. For libraries limited to one internal application, I wouldn’t “probe many hedges.” For open source libraries, I validate pretty aggressively.

Of particular interest was this:

I often find that if all the ancillary behaviors are in place before I approach the core functionality, then the core functionality is much easier to implement.

I always thought you should only program what you need and no more. It seems very strange to assume the ancillary behaviors will be needed. It seems like a violation of YAGNI.

I have been trying to reconcile Uncle Bob’s advice here, and the TDD best practices I’ve learned and developed. But I cannot. Either I’ve been receiving and giving years of bad advice, or Uncle Bob has made a rare mistake.

No Comments

Qt Designer is harmful, exhibit A

11/12/2014

Last week, iker j. de los mozos posted a Qt tutorial on his blog. The post was retweeted a number of times, so I figure people liked it.

The post exemplifies what is wrong with the Qt Designer, and also how a little more investment in learning can pay dividends for your career.

I know it’s unfair to give people code reviews on things they just put out for free, but I consider it even worse to allow people to continue to use the Qt Designer with a clear conscience. I thank Ike for his post, and for syndicating his feed on Planet Tech Art, and hope that no one takes my writeup below personally. It’s not an attack on a person, it’s trying to point out there there is a much better way to do things.

There are 117 lines of Python code in Ike’s tool for locking and unlocking transformation attributes. This sounds like a small amount, but is for an experienced Pythonista it indicates huge waste. For comparison, the entire ZMQ-based request/reply client and server I built for Practical Maya Programming with Python is the same size or smaller (depending on the version). If we take a closer look at his code (see link above), we can see a ton of copy and pasted functionality. This is technically a separate concern from the use of the Designer, but in my experience the two go hand-in-hand. The duplication inherent in GUI tools carries over to the way you program.

Let’s look at some refactored code where we build the GUI in code (warning, I haven’t tried this code since I don’t have Maya on this machine):

from functools import partial
from PySide import QtGui
import pymel.core as pmc
import qthelpers

OPTIONS = [
    dict(label='Position', btn='POS', attr='translate'),
    dict(label='Rotation', btn='ROT', attr='rotate'),
    dict(label='Scale', btn='SCALE', attr='scale')
]

def setLock(obj, attr, value):
    obj.setAttr(attr, lock=value, keyable=not value)

def isAttrLocked(obj, attr):
    return obj.getAttr(attr, q=True, lock=True)

def toggleRowCallback(attr):
    obj = pmc.ls()[0]
    value = isAttrLocked(obj, attr + 'X')
    for axis in 'XYZ':
        setLock(obj, attr + axis, value)

def toggleCellCallback(attr, state):
    obj = pmc.ls()[0]
    setLock(obj, attr, state)

def makeRow(options):
    return qthelpers.row(
        [QtGui.QLabel(options['label'])] +
        map(lambda axis: qhelpers.checkbox(onCheck=partial(toggleCellCallback, options['attr'] + axis)), 'XYZ') +
        qhelpers.button(label='lock ' + options['btn'], onClick=partial(toggleRowCallback, options['attr']))
    )

def main():
    window = qthelpers.table(map(makeRow, OPTIONS), title='lockAndHide UI', base=QtGui.QMainWindow)
    window.show()

Why is this code better? Well, for starters, it’s less than a third of the size (37 lines) and there’s less duplication. These are very good things. When we want to change behavior- such as auto-updating the checkboxes when our selection changes- we can put it in one place, not nine or more.

So the code is better, but what other benefits are there to not using the Designer?
– We pull common primitives, like a “row” (QWidget with HBoxLayout) and “table” into a qthelpers module, so we can use this across all GUIs. This saves huge amounts of boilerplate over the long run, especially since we can customize what parameters we pass to it (like onClick being a callback).
– The GUI is clear from the code because the UI is built declaratively. I do not even need to load the UI into the Designer or run the code to understand it. I can just read the bottom few lines of the file and know what this looks like.
– You learn new things. We use functools.partial for currying, instead of explicit callbacks. This is more complicated to someone that only knows simple tools, but becomes an indispensable tool as you get more advanced. We are not programming in Turtle Graphics. We are using the wonderful language of Python. We should take advantage of that.

Again, I thank Ike for his blog post, and hope I haven’t upset anyone. Ike’s code is pretty consistent with the type of code I’ve seen from Technical Artists. It’s time to do better. Start by ditching the Designer and focusing on clean code.

5 Comments

Long live Slack, down with egotistical email

17/11/2014

We use Slack for team communication at Cozy. I struggled with the transition. When I reflected on my struggles, it made me better understand what a destructive format email is for workplace communication.

A quick disclaimer. This is only about work communication and not personal communication. I love email. I think email will be around for a long time and I will lament if and when it goes away. I just don’t think we should be using email for work.

Oration is the highest form of feeding an ego. You craft your message carefully. You research, write, and rehearse. Finally, you take the stage. You command everyone’s attention. And once you’re done, an important topic has been thoroughly addressed and everyone can go on with their lives, better off after hearing what you said.

Email is oratory without the speaking* (or skill). My problems with email stem from when it is used for one-way communication. I suspect that most emails I’ve ever received from anyone in management have been one-way. Generally these emails are meant to, first and foremost, communicate the sender/manager’s self-importance. Often the email contains a nugget of actual information which should be hosted elsewhere. Sometimes the email is an announcement no one understands. And as a rule, you can’t rely on people reading the email you send anyway.

When you craft a long email, like an orator crafts a speech, it is an ego boost. Each one is a masterpiece. You are proud of your fine writing. When you craft a long chat message, on the other hand, you look like a dramatic asshole. It puts in stark perspective how awful the written format is for important or high-bandwidth communication. I’ve never seen someone post a 300-word message to chat. How many 300-word emails do you have in your inbox?

Removing email also levels the playing field for communication. You don’t need to be a manager or orator. Everything you write has a visibility you can’t change. You choose your audience based on topic. Is there a question about a product’s design? Well, it goes into the product or design channel, whether you are Executive Emperor or QA Associate II. Also, no one really wants to read your dramatic flair so please keep it short and to the point.

I used to get frustrated when I’d write an excellent email, send it out, and within a few minutes someone would reply with a message like “Yeah, just to build on what Rob said, it’d be a good idea to do X.” You idiot! You are an Ice Cream Truck driving through the State of the Union. But of course, the problem was mine, playing a manipulative game, focusing too much on this amazing message I’d created. Sometimes these emails would be about the manipulative games people were playing and how we weren’t focused on the employees and customers and things that were actually important.

Email in the workplace is a systematic problem. We take it for granted. We use it constantly. We don’t question it. But email has a cost. It feeds into the already inflated ego of managers. It encourages one-way communication. It is wonderful for grandstanding. We spend a lot of time crafting museum-quality correspondence no one wants to read. And in the end, there are better ways to accomplish what we use it for.


* One of the greatest “speeches” of all time, Pro Milone by Cicero, was written, not spoken. We know great orators by their writing, not their speaking.

No Comments

If you hear “perception is reality” you’re probably being screwed

27/10/2014

I was once told in a performance review that “perception is reality.” I was infuriated, and the words stuck in my mind as the most toxic thing a manager could say to an employee. I have avoided writing about it, but the “This American Life” episode about Carmen Segarra’s recordings at the Fed has inspired me to change my mind. Here’s the relevant section, emphasis mine:

Jake Bernstein: Carmen says this wasn’t an isolated incident. In December– not even two months into her job– a business line specialist came to Carmen and told her that her minutes from a key meeting with Goldman executives were wrong, that people didn’t say some of the things Carmen noted in the minutes. The business line specialists wanted her to change them. Carmen didn’t.

That same day, Carmen was called into the office of a guy named Mike Silva. Silva had worked at the Fed for almost 20 years. He was now the senior Fed official stationed inside Goldman. What Mike Silva said to Carmen made her very uncomfortable. She scribbled notes as he talked to her.

Carmen Segarra: I mean, even looking at my own meeting minutes, I see that the handwriting is like nervous handwriting. It’s like you can tell. He started off by talking about he wanted to give me some mentoring feedback. And then he started talking about the importance of credibility. And he said, you know, credibility at the Fed is about subtleties and about perceptions as opposed to reality.

Well shit, if that doesn’t sound familiar. Here I was, doing work that was by all measures extremely successful, yet pulled into a feedback meeting to be told “perception is reality.”

Let me tell you what “perception is reality” means, and why you should plan on leaving your job the moment you hear it:

The arbitrary opinions of your manager’s manager defines your situation. And they don’t like what you’re doing.

Your manager may be well-meaning (mine was, as was Mike Silva), but at the point you get this “perception is reality” bullshit, you can be sure there’s nothing that they are going to do to help you. Someone above them has taken notice, your manager has probably heard hours of complaints, and you can either shut up or get out. Perception isn’t reality everywhere; it is only the mantra in sick organizations totally removed from reality.

1 Comment

Japanese vs. Western models of decision making

11/09/2014

I was reading a book about The Toyota Way last year (sorry, can’t remember which) and something that stuck with me was a section on Japanese versus Western decision making*. The diagram was something like this:

japanese-vs-western-decisionmaking

The crux of it is, Japanese companies like Toyota spend more time preparing for a decision, and less time executing. Western companies reach a decision earlier, and then execution has all sorts of problems. It’s also important to note that Japanese companies often reach the “end goal” sooner.

Preparation involves building consensus, but that doesn’t mean it’s all talk. Japanese companies pioneered set based design and prototyping, which allows them to try out a huge number of ideas cheaply. By the time a decision is made, they’ve “thrown out” about 90% of the work they’ve done!** However, once the decision is made the execution goes smoothly.

It is easy to see how Japanese decision making maps to businesses with a clear distinction between preparation/pre-production and execution/production, such as traditional manufacturing. It can be difficult to map to pure product development. Keep a few things in mind.

  • Consensus isn’t just discussion and design. Big-Design-Up-Front is not preparing for execution. BDUF is a woefully incomplete form of execution. Building consensus means actually implementing things in order to rapidly learn.

  • Consensus isn’t a democracy. Toyota has a very hierarchical structure and people have very clear responsibilities. The goal of consensus-based decisions is to make better decisions, not to achieve some Platonic ideal. Difficult decisions are still made, but the role of the executive is to manage the consensus-building process, rather than “just hurry up and making a decision.”

The Japanese model is built in to cross-functional teams. Consensus isn’t achieved by the heads of Engineering and QA signing some agreement in blood. If that agreement ends up not being a good idea- and it almost never is!- you end up with the Western execution experience. On the other hand, cross-functional teams have much more interactivity and iteration going on, and overall execution is much faster.

People will get upset. Take responsibility for your decisions, and acknowledge some people will get upset by them. However by making sure their concerns are thoroughly heard and discussed before execution, you will make sure they don’t keep pulling things off-track.

Anyway, there’s lots of good writing on this topic, and I highly suggest checking some of it out if it interests you. This is just my software development-centric take.


* I’m just calling these the Western and Japanese models of decision making. They are clearly not the way all decisions are reached in the respective countries. In fact these generalizations may not even be true anymore. Whether Japanese or American companies behave this way is irrelevant. The names are just proxies for different types of decision making.

** Obviously the work isn’t thrown out, since they’ve learned so much from it. But if the effort were raw materials, I suspect 90% of it would end up in the trash, depending on how a set is developed and culled.

5 Comments

Escaping the Windows prison

8/09/2014

My friend Brad Clark over at Rigging Dojo is organizing a course on Maya’s C++ API (I am assuming it is Maya but could be another program). He had a question about student access to Visual Studio, to which I responded:

As a programmer, the experience on Windows is pretty horrific. No built-in package manager. A shell scripting language designed in hell. A command line experience that is beyond frustrating. A tradition of closed-source and GUI-heavy tools that are difficult or impossible to automate. A dependence on the registry that still confounds me.

My eyes weren’t opened to this reality until I switched to Linux. I was on a Windows netbook that was barely working anymore. I installed Linux Mint XFCE and suddenly my machine was twice as fast. But the much more important thing that happened was exposure to modes of developing software that I didn’t even know existed (Python, Ruby, and Go made a lot more sense!). Programming in Windows felt like programming in prison. Developing on Linux made me a better programmer. Furthermore, If I didn’t end up learning the Linux tools and mindset on my own, working in the startup world would be basically impossible.

Programming on Windows revolves around Visual Studio and GUI tools. If you need evidence at how backwards Windows development is, look no further than Nuget. This was a revolution when it was released in 2010, changing the way .NET and Windows development was done. In reality, the release of Nuget was like electricity arriving in a remote village. It took ten years for C# to get a package manager. And it is for the rich villagers only: older versions of Visual Studio don’t work with Nuget.

The ultimate problems Windows creates are psychological. The technical differences change the way you think. It echoes the “your Python looks like Java” problem. Is the Windows mentality what we want students to take on? My last two jobs have been on Linux/OSX. I honestly cannot imagine having kept my head above water if I didn’t have a few years of self-taught Linux experience.

Windows still dominates in desktop OS popularity. That is all the more reason to make sure we are exposing student programmers to alternative ways of developing. I want new developers to be exposed to things outside the Microsoft ecosystem. It is too easy to become trapped in the Windows prison.

5 Comments

Metaprogramming with the type function

1/09/2014

In my book, Practical Maya Programming with Python, I use Python’s type function for dynamically creating custom Maya nodes based on specifications, such as input and output attributes. I really love the type function*, so I thought I’d post another cool use of it.

I recently wrote this gem as a way to dynamically create exception types from error codes (it was for a now-defunct REST client):

def get_errtype(code):
    errtype = _errtype_cache.get(code)
    if errtype is None:
        errtype = type('Error%s' % code, (FrameworkError,), {})
        _errtype_cache[code] = errtype
    return errtype

Next is an uncommon form of Python’s except statement. Its argument can be any expression that evaluates to a single exception type or sequence of exception types. You can actually call a function in the argument to except!

try:
    return framework.do_something()
except framework.catch(404):
    return None

The framework.catch function is below. It looks up (and potentially creates) error types based on the error codes being caught:

def catch(*codes):
    return [get_errtype(c) for c in codes]

This sort of utility is why I wrote the type of book I did. Learning how to program in Python instead of MEL is all well and good. But you need to really see what Python is capable of to make big strides. I hope that with a more advanced understanding of Python, 3D developers can start creating frameworks and libraries, just like PyMEL, that other developers will work with and on for many years to come.


* I love the type function for a vain reason. It’s more obscure than decorators, but not as difficult to understand as metaclasses.

No Comments

A short letter to a unit testing newcomer

18/08/2014

One of my friends asked how to get started with unit testing and Test Driven Development and figured I could write a short post. I also mention TDD a few times in my book so I think it could use some more attention.

I got started with unit testing when I wrote a smallish Python project, and realized I couldn’t maintain large Python projects without tests*. So I wrote unit tests for my code. I then quickly got into TDD. I don’t remember what resources I used, but there are plenty.

I’d encourage you to start unit testing by writing your code first, then the tests. As soon as this becomes easy and is paying off, start writing the tests first (TDD). You will come to appreciate that this is a powerful way of thinking and working. Even if you don’t ultimately stick with this approach, it is worth becoming skilled with it. You will uncover many suboptimal areas in your coding process that are otherwise extremely difficult to find and fix.

Keep in mind that learning TDD isn’t like already knowing Mercurial, then reading a book about Git**, and then being skilled with Git because you are skilled with Mercurial. You are embarking on a long journey, and will need to refer to many tutorials, blogs, and books. You will do some TDD, and look back on that code and process in a year and be horrified, just like you were when you started programming. Do not think of unit testing and TDD like learning a new framework or library. Think of it like learning how to program all over again.

So I don’t know exactly where to get started, only that you must, and keep going once you do start.


* I’d soon realize that no project was really maintainable without tests.

** Pro Git is my favorite Git book, by the way.

1 Comment

“Do you expect too much from people?”

11/08/2014

Last year, a coworker asked me if perhaps I expect too much from other people. I thought about it a moment and said:

No. I do not accept the argument that I’m somehow inherently superior to most others. In fact it is because I know I am not superior that I have high expectations of others.

In the intervening year, I’ve come to see that this belief drives a lot of my management philosophy. In general, I assume the best of people I work with. If someone is not performing, I do not blame them; I blame myself (or whoever their manager is) and systematic problems that they are not in control of (but hopefully ones I am).

Of course people have different innate abilities and experiences. Some people have a high aptitude for certain types of work, and some have chosen a path that may not be a good fit. But the realities of business are that these things can quickly change, and an asset one day can be a liability the next. When a company has grown past a dozen people, I believe its time to start favoring nurture over nature. If someone isn’t performing, it is management’s problem.

This is true of not just employees, but other managers, and it was specifically about two other managers that this question was posed. The times were a-changin’, but these individuals were in roles they were ill suited for. They simply did not have the experience or competence to drive through the changes that needed to happen. It was up to their (our) management to take responsibility, but instead I heard apologies that “maybe they aren’t the best suited” and other meaningless explanations. I didn’t expect them to magically change; I expected management to do their job: get involved and well, manage!

If I expect something, it’s that people can both teach and learn. If the ability of people to grow is not an organization’s chief expectation- if management is not set up to grow employees, or management is not prepared to mature itself- I can’t imagine what they think their long-term prospects are. Perhaps they aren’t expecting much.

No Comments

Portland, here we come!

6/08/2014

Tomorrow, my family leaves on a four day trip to begin a new life in Portland, OR, as the Engineering Manager at real-estate startup www.cozy.co. I am incredibly excited to join the company for all sorts of reasons (the role, product, team, founders, location, pretty much everything). It’ll also allow me to focus on what I’m most passionate about: engineering and development management. So there should be fertile topics for new blog posts, though I suspect the purely engineering or Python-centric posts will get fewer (except for goless, which I plan to maintain).

Now for the heavy stuff. In addition to a new chapter opening, several chapters are finally closing. Since deciding to leave Iceland earlier this year, life has been a depressing adventure. After a promising start at CCP Atlanta, the studio was shut down. This caused us to become homeless. We were 2 days from closing on a house (for which we lost a very large amount of earnest money). So a few days later we drove to Austin and moved in with my in-laws for about 2 months. I found a new job at The Foundry, but I ended up not enjoying working remotely. I was looking for a new job just a few weeks after starting (the job was just for a 3 month contract so they were aware I was looking). We left my in-laws mid-June and moved into their lake house, which I describe as 30 minutes outside of nowhere (over an hour into Austin). We haven’t had running water for weeks due to pump-house problems. My wife and son have gotten cabin fever, unable to make friends with a culture we don’t seem very compatible with. I haven’t been able to give a concise answer to “where I live” or “where I work” since March. Moving costs keep building. My son is behind on his vaccinations due to moving around. It’s been the most difficult time of our lives.

And oh my god is it hot outside.

So here’s to a new job, a new city, a new start. See you on the other side!

10 Comments