Everyone should take vacation at the same time

by Rob Galanakis on 31/07/2014

Throughout my career I’ve always seen people struggle with taking vacation. People are too wrapped up in what they’re doing. Managers can’t allow critical people to go missing. There are weeks of trepidation and handover and “I don’t know how to fix that” emails. To a large extent, this can be fixed with shared code ownership, comprehensive automated testing, and all those types of good development practices. I have a better idea, which I saw in great use at CCP (which for a long time did not have those good development practices):

Everyone vacations at the same time.

In Iceland, this was a cultural thing. From what I understand, employers aren’t allowed to deny you vacation between May and September. Everyone goes on vacation in July. The office is empty. Things just go relatively smoothly as no one expects anything to get done during July (its a great time for side projects). This “July slowdown” wasn’t limited to CCP, as people who need visa renewals in the summer no doubt learn.

In Atlanta, the studio just closed down for two weeks in July.

In both cases, there may be a skeleton crew to keep things running, people on call, etc. Its just that no one expects anything non-immediate to get done. This has many benefits: its easier to plan for, office costs are cheaper, there’s a single silent period rather than months of rolling disruption, everyone takes a refreshing vacation, and much more. It’s pretty much the only vacation policy I’ve seen that was largely resilient to the pressures that keep people from taking vacation. To be sure, some people were screwed over by bad managers, but (in contrast to most other management offenses) this was largely due to particular managers and not underlying cultural causes.

If you see the people around you failing to take the proper vacations everyone needs to keep going, I’d encourage you to try having everyone go on vacation at the same time.

6 Comments

Practical Maya Programming with Python is Published

by Rob Galanakis on 28/07/2014

My book, Practical Maya Programming with Python has been finally published! Please check it out and tell me what you think. I hope you will find it sufficiently-but-not-overly opinionated :) It is about as personal as a technical book can get, being distilled from years of mentoring many technical artists and programmers, which is a very intimate experience. It also grows from my belief and observation that becoming a better programmer will, due to all sorts of indirect benefits, help make you a better person.

If you are using Python as a scripting language in a larger application- a game engine, productivity software, 3D software, even a monolithic codebase that no longer feels like Python- there’s a lot of relevant material here about turning those environments into more standard and traditional Python development environments, which give you better tools and velocity. The Maya knowledge required is minimal for much of the book. Wrapping a procedural undo system with context managers or decorators is universal. A short quote from the Preface:

This book is not a reference. It is not a cookbook, and it is not a comprehensive guide to Maya’s Python API. It is a book that will teach you how to write better Python code for use inside of Maya. It will unearth interesting ways of using Maya and Python to create amazing things that wouldn’t be possible otherwise. While there is plenty of code in this book that I encourage you to copy and adapt, this book is not about providing recipes. It is a book to teach skills and enable.

Finally, to those who pre-ordered, I’m truly sorry for all the delays. They’re unacceptable. I hope you’ll buy and enjoy the book anyway. At least I now have a real-world education on the perils of working with the wrong publisher, and won’t be making that same mistake again.

Thanks and happy reading!
Rob Galanakis

5 Comments

goless 0.7 released, with Python3 support and bug fixes

by Rob Galanakis on 25/07/2014

goless version 0.7.0 is out on PyPI. goless facilitates writing Go language style concurrent programs in Python, including functionality for channels, select, and goroutines.

I forgot to blog about 0.6 at the start of July, which brought Python 3.3 and 3.4 support to goless (#17). I will support pypy3 as soon as Travis supports it.

Version 0.7 includes:
- A “fix” for a gevent problem on Windows (socket must be imported!). #28
- Errors in the case of a deadlock will be more informative. For example, if the last greenlet/tasklet tries to do a blocking send or recv, a DeadlockError will be raised, instead of the underlying error being raised. #25
- goless now has a small exception hierarchy instead of exposing the underlying errors.
- Better PyPy stackless support. #29
- goless.select can be called with (case1, case2, case3), etc., in addition to a list of cases (ie, ([case1, case2, case3])). #22

Thanks to Michael Az for several contributions to this release.

Happy concurrent programming!

No Comments

goless now on PyPI

by Rob Galanakis on 12/06/2014

goless is now available on the Python Package Index: https://pypi.python.org/pypi/goless . You can do pip install goless and get Go-like primitives to use in Python, that runs atop gevent, PyPy, or Stackless Python. You can write code like:

channel = goless.chan()

def goroutine():
    while True:
        value = channel.recv()
        channel.send(value ** 2)
goless.go(goroutine)

for i in xrange(2, 5):
    channel.send(i)
    squared = channel.recv()
    print('%s squared is %s' % (i, squared))

# Output:
# 2 squared is 4
# 3 squared is 9
# 4 squared is 16

I’ve also ported the goless benchmarks to Go, for some basic comparisons to using goless on various Python runtimes (PyPy, CPython) and backends (gevent, stackless): https://goless.readthedocs.org/en/latest/#a-benchmarks

Thanks to Rui Carmo we have more extensive examples of how to use goless (but if you’ve done or read any Go, it should be relatively straightforward). Check them out in the examples folder: https://github.com/rgalanakis/goless/tree/master/examples

And just a compatibility note (which is covered in the docs, and explained in the exception you get if you try to use goless without stackless or gevent available): goless works out of the box with PyPy (using stackless.py in its stdlib) and Stackless Python. It works seamlessly with gevent and CPython 2.7. It works with PyPy and gevent if you use the tip of gevent and PyPy 2.2+. It will support Python 3 as soon as gevent does.

Thanks and if you use goless, I’m eager to hear your feedback!

No Comments

goless Benchmarks

by Rob Galanakis on 9/06/2014

I benchmarked how goless performs under different backends (goless is a library that provides a Go-like concurrency model for Python, on top of stackless, PyPy, or gevent). Here are the results, also available on the goless readthedocs page:

Platform Backend   Benchmark      Time
======== ========= ============== =======
PyPy     stackless chan_async     0.08400
CPython  stackless chan_async     0.18000
PyPy     gevent    chan_async     0.46800
CPython  gevent    chan_async     1.32000
~~~~~~~~ ~~~~~~~~~ ~~~~~~~~~~~~~~ ~~~~~~~
PyPy     stackless chan_buff      0.08000
CPython  stackless chan_buff      0.18000
PyPy     gevent    chan_buff      1.02000
CPython  gevent    chan_buff      1.26000
~~~~~~~~ ~~~~~~~~~ ~~~~~~~~~~~~~~ ~~~~~~~
PyPy     stackless chan_sync      0.04400
CPython  stackless chan_sync      0.18000
PyPy     gevent    chan_sync      0.44800
CPython  gevent    chan_sync      1.26000
~~~~~~~~ ~~~~~~~~~ ~~~~~~~~~~~~~~ ~~~~~~~
PyPy     stackless select         0.06000
CPython  stackless select         0.38000
PyPy     gevent    select         0.60400
CPython  gevent    select         1.94000
~~~~~~~~ ~~~~~~~~~ ~~~~~~~~~~~~~~ ~~~~~~~
PyPy     stackless select_default 0.00800
PyPy     gevent    select_default 0.01200
CPython  stackless select_default 0.19000
CPython  gevent    select_default 0.25000

The trends are that PyPy with its built-in stackless support is fastest, then Stackless Python (2-5x), then PyPy with gevent (5-10x), and finally CPython with gevent (15-30x).

Remember that these are benchmarks of goless itself; goless performance characteristics may be different in real applications. For example, if you have lots of C extensions, CPython/stackless may pull ahead due to stack switching.

If you’re using goless for anything, I’d love to get some benchmarks, and find out how it’s working, so please comment below, or create a GitHub issue.

Disclaimers: It’s possible that goless is inefficiently using gevent, but the backend-specific code is so simple I doubt it (see goless/goless/backends.py). These benchmarks are all done against Python 2.7. Also I am no benchmark master (especially with PyPy) so there may be problems, look over goless/benchmark.py for benchmark code. These were done mostly for curiosity.

No Comments

Hiring your cake and eating it too

by Rob Galanakis on 6/06/2014

When evaluating candidates, I have always been a believer that cultural fit and potential to improve is more important than technical ability. Of course I like to review real code samples and give a programming test, but I rarely ask for whiteboard programming. I am not a master of algorithms or math and don’t have a CS background, so I don’t require that of most candidates*. As Five Thirty Eight explains and shows, people just want to date themselves.

This policy is amazingly ego-centric. But it’s also worked pretty well. Without exception, I’m proud of the hires I’ve had a primary stake in. But it is the best policy, at least for the type of work I’m doing?

I think I am a pretty darn good hire. Why would I want to exclude people like me? Well, the fact is that I’m not special. There are way more gifted people around than me. There are plenty of people who are a great fit, and are great leaders, managers, developers, and coaches, and are awesome at math and algorithms. Should I overlook myself to try and get someone better?

Clearly companies like Google can do this. There are anecdotes about people who they hired the second or third time around. People like Steve Yegge. On the other hand, there are plenty of managers that hire “to fill a seat.” Can every company interview like Google, and reject otherwise promising candidates that do not ace a challenging technical interview?

Who you choose to hire does not define what your company is, it defines what your company will become. If you give in and make unenthusiastic hires, your amazing company will be dragged into mediocrity. If you hold the line and hire only the best, you’ll continue to excel, if you can find people. In reality, most companies are somewhere in the middle.

The “best” policy is the one that fits your situation. In an isolated job market, you may choose to value fit and culture. A tight-knit culture may help retention, and make it easier for the inevitable low performers to improve. At a startup, you may have limited headcount, short timelines, and immediate technical needs. Cultural fit would be trumped by technical expertise and experience. On the other hand, cultural fit may be most important at an established company. The biggest risk is a dilution of the culture, and there are plenty of senior people around to do mentoring.

Recognizing what you should select for is much more important than simply always trying to be as selective as possible.


* Obviously there are exceptions. If you need expertise, you need expertise. My thoughts apply to the 80% case.

1 Comment

Optimize iteration length for feedback

by Rob Galanakis on 4/06/2014

If there is one theme that is weaved through all of Agile’s principles and practices, it is feedback. TDD. Pair programming. Continuous delivery. Stories. Estimation. Reviews. Retrospectives. On-site customers. Feedback comes up against and again. Feedback from code, the team, and users.

As I mentioned in my previous post, I am a big fan of short (one week) iterations. Feedback is why. If your team works in three-week sprints, and mine in one-week sprints, we have three times the feedback. I have three times the opportunities to learn and improve. While you debate whether to try something new in a future sprint, we try it out immediately and evaluate whether it’s working every week, until we accept or reject it. We become three times as good at estimation. We know in a third of the time you do that a solution to a problem works.

Perhaps in some situations one-week is too long. You could try two-day iterations early on with a new project and new team. Or perhaps, on a legacy project, even two weeks is too short and you don’t get enough work done to get good feedback. You can get faster feedback going to three week sprints, rather than having to wait four weeks (two two-week sprints). The point is to optimize for feedback. This means you should prefer shorter sprints, but there can be exceptions.

The benefit of feedback cannot be overemphasized. In a previous post, I said no single aspect of the Toyota Production System is more important than the others. That’s not entirely true. “Continuous Improvement” can bootstrap the other TPS components. You improve through getting feedback. It follows that you should do anything you can to get better and faster feedback.

2 Comments

Should a team be able to abort a sprint?

by Rob Galanakis on 2/06/2014

After my second retrospective on a new project, I unloaded some pretty harsh criticism about what we were building. I felt it was a “solution in search of a problem” and “not high value.” After proposing an alternative and convincing everyone to change direction, our sort-of Product Owner blasted me for not bringing my issues up sooner and basically wasting two weeks of work. I said I had voiced my concerns, but not strongly or formally, because I was focused on getting the sprint work done. My feeling was that it wasn’t constructive to second-guess things in the middle of a sprint, and that I trusted the people who made the decisions. Especially as the new guy on the team, I leaned towards agreement.

We both had a point. As a senior person, I had a responsibility to speak up. As a team members, I had a responsibility to get our work done. I don’t know if I made the right choice in this instance. Perhaps if I argued too loud too early, I wouldn’t have had enough credence for people to believe me, and would have been in a worse spot at the end. Or perhaps I would have saved two weeks or work.

This is one more reason I prefer one-week iterations. A week is too small to break up, so you just go heads down. But you don’t work on the wrong thing for too long. You get two or three times the chances to learn and improve.

1 Comment

There is no essence of Agile

by Rob Galanakis on 30/05/2014

real agile is: talk to the users directly, know their pain point, address it, repeat. -someone on Twitter who I disagree with

In many conversations about Agile, especially as of late, I read something like the above tweet. I don’t know where the idea that Agile can be distilled down into one or two practices or principles comes from. Thinking this way is extremely harmful. If you think like this, I’d love to hear your explanation.

Agile methodologies come out of Lean thinking which comes out of the Toyota Production System (TPS). The TPS is incredible. It mixes explicit practices such as 5S, ideals such as JIT, and principles such as “respect for people” into a unified, harmonious way. TPS is often represented using a house metaphor as in the following image:

house-of-lean1

The house is an apt metaphor because every single component is structurally vital. “Talking to users” and “addressing pain points” corresponds pretty closely to “customer satisfaction,” which is the roof of the TPS house. The roof is elevated by the walls, which are secured by the foundation. The roof is integral to the function of a house, but no more so than any other component. Customers can only help inform what you produce. If you are unable to produce those things at high quality, speed, and efficiency, and improve over time, then it doesn’t matter how much you talk to customers.

Adding an Andon cord to an assembly line does not make a manufacturer Lean. Being Lean requires a whole set of practices, ideals, and principles working in unison. It can be TPS, or your adaptation, but it requires incredible rigor, skill, and learning, and it’s not simple. Likewise, no single practice, from TDD to iterations to talking to users, creates a well-performing Agile organization.

There are plenty of pseudo-Lean companies, just like there are plenty of pseudo-Agile shops. On the plus side, the damage from Six Sigma Black Belts is far more severe than the Certified Scrum Master racket.

Companies that are Lean are rare, and have been at it for a while. It’s silly that every JIRA jockey thinks they have learned the essence of Agile. Being Agile is difficult, complicated, and takes a while. Let’s not try and distill it down so much that we totally dilute it.

5 Comments

Deploying a C# app through pip

by Rob Galanakis on 28/05/2014

“If we bundle up this C# application inside of a Python package archive we can deploy it through our internal CheeseShop server with very little work.”

That sentence got me a lot of WTF’s and resulted in one of the worst abuses of any system I’ve ever committed.

We had a C# service that would run locally and provide an HTTP REST API for huge amounts of data in our database that was difficult to work with.* However, we had no good way to deploy a C# application, but we needed to get this out to users so people could start building tools against it.
I refused to deploy it by running it out of source control. That is how we used to do stuff and it caused endless problems. I also refused bottlenecks like IT deploying new versions through Software Center.

So in an afternoon, I wrote up a Python package for building the C# app, packaging it into a source distribution, and uploading to our Cheese Shop. The package also had functions for starting the packaged C# executable from the installed distribution. The package then became a requirement like anything else and was added to a requirements.txt file that was installed via pip.

What I initially thought was an unsightly hack ended up being perfect for our needs. In fact it caused us to eliminate a number of excessive future plans we had around distribution. I certainly wouldn’t recommend this “technique” for anything more than early internal distribution, but that’s what we needed and that’s what it did so effectively.

Getting something up and running early was extremely useful. It’s important to walk the line between the “we need to do several weeks of work before you see any result” and “I can hack something together we’re going to regret later,” especially for infrastructure and platform work. If code is well-written, tested, documented, and explained, you shouldn’t have qualms about it going into (internal) production. If the hackiness is hidden from clients, it can be easily removed when the time comes.


* Reworking the data was not an option. Creating this service would have allowed us to eventually rework the data, by providing an abstraction, though.

7 Comments