Blog of Rob Galanakis (@robgalanakis)

Posts by: Rob Galanakis


When advice turns out to be a mistake

I remember reading Clinton Keith’s “Agile Game Development” and getting to an anecdote about CCP Games, where I worked at the time. The following is from a Gamasutra repost of the book’s “Teams” chapter: In the fall of 2008, CCP undertook the development of its tenth expansion pack called Apocrypha. Apocrypha was the most ambitious expansion of the EVE universe. It added major technical features and significantly extended the size of the EVE world. This ambitious goal required CCP’s worldwide development studios to work in parallel. Features and content developed simultaneously across three continents had to seamlessly come together to achieve their goal. Normally this would be the introduction to a disaster story, but CCP pulled it off. CCP is a longtime adopter of agile methods, specifically Scrum. In the case of Apocrypha, with more than 120 developers in 13 Scrum teams spread across three continents, 9 product owners were required. These product owners took their direction for the game from a project management group in Iceland. What!? I was reading this in early 2012 (I joined CCP in July 2011). Twenty percent of the company had just been laid off, a number of scandals had just unfolded, there were reorgs and more reorgs. This monumental failure could be directly traced back to the globalization of CCP’s development. Yet here I was reading about what a great model it was in the seminal work about Agile game development. Yet, Apocrypha was pretty great. EVE Online and CCP were growing and making money from 2007 through 2010. The company, by most measures, appeared successful when “Agile Game Development” was written. But it all came crashing down in 2011. In the soul searching that followed the layoffs, it was clear CCP was successful in spite of its globalized development and centralized...

Read more


Linting pull requests

A couple weeks ago on Twitter, I joked about adding a way to bypass Cozy’s Pull Request linter by including #YOLO in the pull request description. It spawned an interesting discussion and a few people asked for more details about how the linter works. Some devs at @CozyCo said our new PR linter was too strict, so I made it less strict. #YOLO pic.twitter.com/pjaqdc2ct2 — Rob Galanakis (@techartistsorg) March 18, 2016 We have some pretty thorough commit and pull request guidelines at Cozy, based on recognized best practices like these from Chris Beam. In other words, “update this” as a commit message is won’t do. After a few too many cut off commit messages and PRs missing links to the issues they were fixing, I finally got fed up enough to enforce these guidelines with a linting bot. The entire linting project was just a few hours work*. When the web works, the productivity is incredible. When it doesn’t, you spend 3 hours debugging a JavaScript binding error. This was definitely a positive and productive experience. Github has some excellent tutorials for building these sorts of bots. Check out Building a CI server. Your server gets some webhooks, and uses the Statuses API to update the PR. And, your bot can use the API to get whatever data you need. It doesn’t need a Git client or anything. The jury is still out whether this is a good idea or not. I much prefer to have agreed standards that are continuously discussed and improved while being rigorously enforced, rather than have things happen haphazardly. There’s no room for poor quality to sneak in. Standards are clear. This strategy warrants its own post, but this post is just about the experience of building the linter. I’ll dive into some example...

Read more


Hiring in a time of scarcity

I’ve never understood the hypergrowth mindset that’s guided the tech industry for the last few years, and part of me is happy that the investment bubble has popped. I’m not interested in throwing around money to solve problems; my experience is this creates more problems than it solves, and then you have twice the problems to solve when the money dries up! You can see how lean companies like Toyota deal with economic troubles compared to their historically gluttonous American counterparts. Toyota has been able to take problems in stride. More wasteful companies, who were making large profits due to a booming market rather than their own ingenuity, were in turmoil. This is obviously not unique to car manufacturing. Scarcity requires hard decisions. Let’s say you have 12 months of runway, and each new hire brings in runway 2 weeks. Do you keep your runway as long as possible, so you have the longest time to experiment and learn? Do you hire a few people immediately, so they can get ramped up and begin making an impact as soon as possible? Or do you need to hire for easier to fill positions, and defer the hire to as late as possible to keep yourself open to opportunity? There’s no single answer because every situation is different. But these are the tough conversations you are forced to have when money is scarce. You really analyze what you’re doing, why, and how to improve. I want scarcity to drive ruthless efficiency, effectiveness, and innovation. I was talking to someone from a public company recently who said they knew they wouldn’t be able to fill all their dev spots, so they are using one of those spots to fund open source work. Brilliant! But how many companies are just looking to put...

Read more


Questions to ask before you make that hire

When I was at CCP, I blogged a lot. I was able to try out many new ideas, and my disagreement with certain decisions sparked a lot of good writing. I’ll get to why that matters in a minute. At one point, one of the American executives hired by the Icelandic management brought in a number of cronies, and gave one the title of “Director of Communications” or something equally ridiculous. This was a new position. None of us knew we needed it. We had no idea what it was supposed to contribute. It would have been an inexplicable move were it not clearly to make room for an executive minion. When you hire for a new role with a leadership title, you better be damn sure of two things. First thing: hire the right person. This person needs to share similar values to the rest of your org. They are going to define a department and make a mark across the company. It’s incredibly difficult to predict success here, so I won’t try (much easier to predict failure!). The second consideration is less commonly discussed, but equally vital: the new role and its importance must be widely understood outside of management. If it’s not- if the first thing people say is “why do we need person X”- you have one of two problems, equally bad. The first problem may be that your senior management is disconnected from the rest of the company. Management has a clear and pressing (and, I will assume, legitimate) need for Senior VP of Synergy, but few outside of management understand why, or what the role is. This disconnect is a bad sign. You’re not communicating properly and an unhealthy gap between senior leadership and everyone else is opening. The gap will only get...

Read more


Anger is my muse

Last week, a coworker asked me to start blogging again. It’s hard to find material to blog about when things are going well and you’re so excited to do work, though! It is for me at least. Anger is my muse. I find it really difficult to stop having fun doing and start writing about doing. When I’m angry about something, writing helps collect my thoughts and discuss the ideas with others. Writing is a way of doing. So, let’s see if I can muster up enough anger to start blogging again? I’ll dig up some skeletons from my past, and see if I can write about all the great stuff we’re doing at Cozy.

Read more


On choosing software by “what is best for the business”

When people are discussing what language/framework/library to use for something, the general criteria people talk about is “what best solves the business problem.” This criteria is used to justify rewriting backend services in Go, rather than sticking with Python. Or not. It’s used explain why you wrote a new CRUD app in node, even though you’re already using Ruby. Or not. It’s used to choose between frameworks like Watir or Capybara, even though they’re basically the same thing. Or not. It’s used to introduce superior programming patterns into legacy codebases. Or not. It’s used to introduce new unit test runners or libraries. Or not. “Best solution for the business problem” is used to justify all manner of decisions that are risky and without worthwhile benefits. Likewise, it’s used to justify all manner of decisions that are restrictive and regressive. I’m sorry to rat on my fellow developers but choosing by “what best solves the business problem” is a load of bullshit. It’s much more honest to just admit that technology choices are made from a desire to work with a new technology, or because a technology is familiar. “We have approval to rewrite this service. I’m tired of working in dynamic languages, and I’d like to try Go.” “This is a small internal app, and I wanted to try node.” “I am familiar with Capybara, and will be writing most of the tests, so I’d like to use that.” “I am uncomfortable introducing a new programming pattern that I am unfamiliar with, even though it’s better.” “I don’t like using this library, and the one I do like can live side-by-side, so I’d like to add it.” I actually don’t have a problem with deciding this way. In fact, I think it’s a good thing! I want to keep...

Read more


What do you want next?

We are focusing on A and B, and in a month or so we’ll start focusing on C, while also keeping focus on A and B. Sound familiar? When we do prioritization at work, I insist we have a single column of priorities or coarse features. In other words, “what do you want delivered next?”* If a team or person isn’t working on one of the top two or three priorities, they’re doing unimportant, and possibly counter-productive, work. You’d be amazed how many people are working on things someone arbitrarily said was important, which aren’t inline with actual priorities. You’d be even more amazed how unimportant most “high priority” work is when it needs to be stacked along with everything else. A feature can easily sit at the number 4 spot for months. Just be careful work doesn’t move up the queue just because it’s been in the queue. I don’t think this is a problem, though, because when you tell a product person “we are only executing on the next 2 things to deliver” they are going to have to make hard decisions. I’ve worked on projects from 10 to 500 people, and generally the times we were humming along was when we had one or two priorities. We ended up producing crap when we had n priorities (where n is often the number of people or teams). Big teams don’t mean more priorities. It is just the granularity of the priorities that changes. This sort of rigid, columnar prioritization communicates to product people that work only gets done when it’s at the top of the column. I’ve run across countless people, both managers and developers, who just sort of, well, expect that stuff just sort of, well, gets done, somehow. And generally it appears as if things...

Read more


Anxiety causes selfish behavior

BPS Research Digest is a great site, highly recommended for anyone interested in why people behave the way that they do. A little while ago, they reported on a study where anxious participants were more likely to cheat and excuse their own unethical behavior than the control group. When we’re stressed out and feeling threatened, our priority becomes self-preservation. According to new research, this defensive mode even affects our morality, making us more likely to cheat and excuse our own unethical behaviour. What’s striking is the cause of the anxiety: they listened to Bernard Herrmann’s Psycho score. Compare this to the stress of micromanagement, yearly review season, project bonuses and deadlines, or even general water cooler politics, and it’s no surprise what goes on in most corporate offices. I’ve written before about how a manager’s primary job is to build trust and this is a good, concrete example why. It’s also a good example of why it’s a company’s job to remove anxiety-causing policies. The less anxiety you cause your employees, the healthier they are and the healthier your culture and company is (we want people working together, not behaving selfishly). These policies include: Annual performance reviews. Much has been written about this. Individual performance-based bonuses. They have been proven to be counter-productive, without a single shred of evidence supporting their utility. Limiting career and salary growth based on positions. People should not compete for a single “senior” spot. Limiting PTO and not having separate sick days. Being sick is not a vacation. Not forcing/encouraging people to take a vacation. This causes paranoia, burnout, and envy. Limiting the flow of information. People will worry if they know what they need to know. The list goes on and on. And the lesson is very simple: When you reduce anxiety, you...

Read more


Could employees choose their own manager?

Someone once brought up to me a plan about enabling employees to choose their own manager. The idea has stuck with me for a while, and being in my current position of authority I’ve pondered it more actively. I’ll use this post to collect my thoughts, and maybe present some ideas for discussion. I’m not going to evaluate the benefits or if this is a good idea, but only think about the practicalities. First, let’s define the role of “manager”. There are many ways the role of manager can be split up or changed or redefined, but I’m specifically going to talk about the extremely popular and stubborn setup of Dev->Manager->Director->VP->CO, or whatever similar hierarchy. I do believe a better structure exists (or that the lack of structure is better!), but I have not seen it, and this arrangement is certainly the most popular, so let’s work from that. Second, let’s define the manager’s responsibilities. There is the leadership aspect (setting direction for a team/group), and there is the procedural aspect (hiring, firing, raises). These can be found in the same person, or separate. If we operate in a strict hierarchy, where everyone in a team reports to a team’s lead, leadership and procedure must be handled by the same person. If people report to a “department manager” or someone else who is not a team, leadership and procedure are handled by different people. That established, how would “choosing your manager” work? “Choosing your manager” would mean individually choosing only the procedural person. The leadership person must be chosen collectively. The reasons for this are obvious. They could be the same person, though. Collectively choosing the leader but having an assigned procedural manager will not work. The person doing the hiring, firing, and appraising ends up with the power...

Read more


Automated testing shows a respect for employees

In the tech-artists.org G+ community page there was a comment on a thread about unit testing: A key factor in TA tools is the speed at which we need to deliver them, and our audience is considerably smaller than, say, engine tools code. Therefor it becomes somewhat hard to justify the time spent on writing the unit tests, and then maintaining them as the tools change or are ported or updated to match new APIs. In other words: Testing is great, but we don’t have time for it. Or the common alternative: Testing is great, but it’s not feasible to test what we’re doing. Codebases without tests manifest themselves in teams that are stressed and overworked due to an ever-increasing workload and firefighting. Velocity goes down over time. Meanwhile, I’ve never known a team with thorough test coverage that delivered slower than a team without automated tests. In fact I’ve observed teams that had no tests and crunched constantly, added tests and became predictable and successful, then removed the tests after idiotic leadership decisions to artificially increase velocity, and watched their velocity drop way down once the testing infrastructure, and especially culture, fell into disrepair. Companies that do not require automated tests do not respect their employees, and do not care about the long-term health of the company. It’s that simple (or they are incompetent, which is equally likely). We know that no testing results in stress, overwork, and reduced quality. We know that more testing results in more predictability, higher quality, and happier teams. I would love to blame management, but I see this nonchalant attitude about testing just as often among developers. The “do it fast without tests, or do it slow with tests” attitude is not just wrong, but poisonous. You are going to be the...

Read more