real agile is: talk to the users directly, know their pain point, address it, repeat. -someone on Twitter who I disagree with

In many conversations about Agile, especially as of late, I read something like the above tweet. I don’t know where the idea that Agile can be distilled down into one or two practices or principles comes from. Thinking this way is extremely harmful. If you think like this, I’d love to hear your explanation.

Agile methodologies come out of Lean thinking which comes out of the Toyota Production System (TPS). The TPS is incredible. It mixes explicit practices such as 5S, ideals such as JIT, and principles such as “respect for people” into a unified, harmonious way. TPS is often represented using a house metaphor as in the following image:

The house is an apt metaphor because every single component is structurally vital. “Talking to users” and “addressing pain points” corresponds pretty closely to “customer satisfaction,” which is the roof of the TPS house. The roof is elevated by the walls, which are secured by the foundation. The roof is integral to the function of a house, but no more so than any other component. Customers can only help inform what you produce. If you are unable to produce those things at high quality, speed, and efficiency, and improve over time, then it doesn’t matter how much you talk to customers.

Adding an Andon cord to an assembly line does not make a manufacturer Lean. Being Lean requires a whole set of practices, ideals, and principles working in unison. It can be TPS, or your adaptation, but it requires incredible rigor, skill, and learning, and it’s not simple. Likewise, no single practice, from TDD to iterations to talking to users, creates a well-performing Agile organization.

There are plenty of pseudo-Lean companies, just like there are plenty of pseudo-Agile shops. On the plus side, the damage from Six Sigma Black Belts is far more severe than the Certified Scrum Master racket.

Companies that are Lean are rare, and have been at it for a while. It’s silly that every JIRA jockey thinks they have learned the essence of Agile. Being Agile is difficult, complicated, and takes a while. Let’s not try and distill it down so much that we totally dilute it.

“If we bundle up this C# application inside of a Python package archive we can deploy it through our internal CheeseShop server with very little work.”

That sentence got me a lot of WTF’s and resulted in one of the worst abuses of any system I’ve ever committed.

We had a C# service that would run locally and provide an HTTP REST API for huge amounts of data in our database that was difficult to work with.* However, we had no good way to deploy a C# application, but we needed to get this out to users so people could start building tools against it.
I refused to deploy it by running it out of source control. That is how we used to do stuff and it caused endless problems. I also refused bottlenecks like IT deploying new versions through Software Center.

So in an afternoon, I wrote up a Python package for building the C# app, packaging it into a source distribution, and uploading to our Cheese Shop. The package also had functions for starting the packaged C# executable from the installed distribution. The package then became a requirement like anything else and was added to a requirements.txt file that was installed via pip.

What I initially thought was an unsightly hack ended up being perfect for our needs. In fact it caused us to eliminate a number of excessive future plans we had around distribution. I certainly wouldn’t recommend this “technique” for anything more than early internal distribution, but that’s what we needed and that’s what it did so effectively.

Getting something up and running early was extremely useful. It’s important to walk the line between the “we need to do several weeks of work before you see any result” and “I can hack something together we’re going to regret later,” especially for infrastructure and platform work. If code is well-written, tested, documented, and explained, you shouldn’t have qualms about it going into (internal) production. If the hackiness is hidden from clients, it can be easily removed when the time comes.

* Reworking the data was not an option. Creating this service would have allowed us to eventually rework the data, by providing an abstraction, though.

Do not tolerate brilliant jerks. The cost to teamwork is too high. – Reed Hastings, Netflix CEO

So I was all prepared to write about how much I hate this quote, but Freddy Nager already did. It is thorough and insightful and explains how out of context this quote is. Thanks to Freddy for doing a far better job ripping this apart than I did. (It also reminds me the difference between real writers/bloggers and people who just have a blog, like me…) Here’s his conclusion, but I suggest you read the whole thing:

In short, Netflix wants only stars who are passionate and courageous and innovative and always do A-level work while abhorring process and questioning assumptions yet working as a team — otherwise they get fired. Sounds brilliant. And jerky.

Why is the Hastings quote so popular? The Netflix presentation is a really excellent one and full of interesting advice and strong statements. I’d even say the brilliant jerk of corporate culture presentations! Why does this quip about “brilliant jerks” resonate with people so much? Probably because we’ve all run into the “brilliant jerk” and the idea of just firing him or her is so pleasing. It also remains cowardly.

This hits particularly close to home for me because I have seen the mistreatment of far too many brilliant jerks. Brilliant jerks are necessary to grow and innovate. The difficult part is to figure out how they can be brilliant but be less jerky.

Firing brilliant jerks is the absolute worst thing to do for teamwork, or indeed the health of the company as a whole. I could spend more time convincing you, or you could view the Netflix slideshow that spawned the quote!

In response to a previous post explaining the phrase “Results are not the point“, commenter RenRen Gabás says:

Both approaches have their own place. It’s easy to see why Toyota/Lean works well with manufacturing and operations. Continuous service and operations needs continual improvements. However, there are times when you need to forget all about process and workflow in order to break new ground. I would place breakthroughs in research and product development right in the Danny category.

Unfortunately, history (and logic) shows that Jess (continuous improvement) is still going to out-innovate Danny (gets drunk and makes stuff) and come up with far more breakthroughs.

• Exhibit A: The Prius (first successful hybrid car) and Lexus (Toyota’s first luxury line) demonstrate that continuous improvement is not limited to operations. These were successes of product development and marketing.
• Exhibit B: Google, a company filled with innovation, research, products, and big ideas, is also a world leader in analytics and iteration! Who do you think their process most closely resembles?

Do not confuse the stifling bureaucracy of a large company to embody Jess, and the creative chaos of a startup to embody Danny. This is a fallacy. Large companies are more stifling, and startups are more creative. But this is due to intrinsic properties, not continuous improvement.

Another way of saying “results are not the point” is “do not trust your fortune to randomness.” I don’t know anyone who would disagree with that! Yet when we take the Danny approach, that is exactly what we are doing, no matter the nature of what we are working on.

Consistency with this style guide is important. Consistency within a project is more important. Consistency within one module or function is most important.

The EVE source code, being initially developed before PEP8 existed, was based on Microsoft’s C++ style. Mostly this is manifested in UpperCamelCase function and method names, mixedCase parameter and variable names, and file headers and class/method delimiters. There was also a style guide that was probably larger than PEP8 (though included a bit more than basic style).

Today, most new code is PEP8 compliant. “What!?” I hear you say. “A foolish consistency is the hobgoblin of little minds!” I agree, and I’ll get back to that at the end. But first, what was involved in introducing PEP8?

The move started when I asked my team if anyone wanted to keep using the headers and delimiters. No one did. So we took out the headers and delimiters before we did any modifications to a file.

A while later, I asked all the programmers on the project if anyone felt strongly about keeping the existing style. No one did. So we started writing new packages with PEP8. The rationale was that our code was already calling code using lowercase_underscore in the stdlib, so it’d be no different for that same code to call a relatively independent, though still project-related, library.

Yet a while later, I started slipping some PEP8 code into modules or packages that weren’t PEP8. I was worried people would get upset. No one did.

And finally, in preparing some basic utility libraries for usage on other projects, I converted the function and method names to PEP8. This involved find-and-replace or CamelCase aliases. I bet you can guess who cared about these “superfluous” edits, or objected to the fact that parameter and variable names remained mixedCase? No one did.

Emerson’s quote “a foolish consistency is the hobgoblin of little minds” is often used as a reason to not change existing code to PEP8, or introduce it into a non-PEP8 codebase. But the quote means the opposite. “Foolish consistency” is being unable to change your mind because it would contradict something you’ve believed or said. So PEP8 has it so right but so wrong: the foolish consistency is to keep things the way they are, even though we know what the right thing to do is.*

* I would recommend introducing PEP8 incrementally. I also suggest keeping things backwards compatible when it is more prudent, such as if the changes are expansive, or you do not control all the code. I also don’t suggest being a stylistic nitpicker when it comes to non-functional aspects such as line length. There may also be times to break the rules (a class that behaves like a function, or function that behaves like a class def). As always, good sense should be applied when considering any viewpoints on this blog.

The phrase “results are not the point” often confuses people new to Lean thinking. It confused the shit out of me, not having really understood it even after my first few books. This is a shame, because it’s such a simple thing.

On Friday night, Danny got really drunk, coded a game, and the game was a hit. Danny did this again the following Friday, with the same results. And once more that third Friday.
Jess codes on sober Saturday nights instead (still drinks on Friday). Jess programs a game, and it runs poorly, crashes often, and isn’t fun. The following Saturday, Jess makes a new game, which runs fast but still isn’t fun and crashes often. That third Saturday, Jess creates a new well-performing, fun game, though it still crashes.
Would you bet on the long-term success of Danny or Jess?

Clearly, the better bet here is Jess. Jess has discovered a process which can be continuously improved. There is good reason to believe Jess will eventually create reliable success. The fact that Danny has been successful three times is basically irrelevant, since Danny’s process is totally haphazard.

This is the idea behind results are not the point. Focusing on the results, and not how those results were achieved, doesn’t improve anything in the long term. The point is to create a repeatable, empirical, continuously improving process. If we can create a reliable, successful process (which here includes culture and practices), we can get reliable, successful results.

I have a saying I like to use when discussing Scrum: Scrum is an Agile project management methodology, not an Agile development methodology. Scrum delivers tools for managing the project (planning, scheduling), but very little for how to develop (design, program, test) it. To do Agile properly, you really need both. This is why eXtreme Programming (XP) and Scrum “fit together” so nicely, with XP telling you how to build your product and Scrum usually taking a higher-level view (and the overlaps are usually the same practices).

The complimentary nature of project management and development methodologies is important to understand and embrace. It is also one reason I don’t believe you can implement Scrum effectively without a programming background. Ultimately this comes down to a very simple thing for me:

If you do not have comprehensive automated tests, you cannot be Agile.(1) I consider this a fundamental truth. Yet rarely do books about Scrum spend more than a few pages talking about how vital this is, rarely is it discussed enough in Scrum Master training courses, rarely do the project managers now running most Agile implementations seem to understand this.

Automated tests are the fundamental building blocks from which all other Agile practices flow. This also informs how I treat Agile as a whole: Agile project management should be free-flowing and not rigid, but Agile development should be rigorously adhered to. That is sure to be inflammatory so let me elaborate.

Rigorous Agile development

1. If automated tests are fundamental, and Test Driven Development (TDD) is the only way to get comprehensive test coverage, that means TDD is not optional. You don’t need to use it everywhere, but TDD should be the rule, not the exception.
2. In the same way, you cannot be Agile if one person ends up being a bottleneck, so collective code ownership is required. I think it’s less important whether this is done through pairing or code reviews (probably both), and more that collective code ownership is a rule and any exceptions are seen as a big problem. Automated tests are required to collectively own code, as it’s the only way people can reliably make changes.
3. Continuous integration is equally important, and depends on automated tests to be reliable and problems to be quickly fixed (it relies on the other two practices). You must always have a high quality build available and the code people are writing should be quickly (once a day or more) and easily getting into that build.

These three practices I consider absolute.(2). Maybe you can add some, but you are not allowed to decide you want to exclude any of those three from your Agile implementation.(3) To do so invalidates the principles on which Agile rests. So it follows that you should not be allowed to have zero experience in them if you’re an Agile leader. You cannot be Agile without them, no matter how briefly Scrum literature covers these topics; I would bet most of the writers of that literature would agree.

Flexible Agile project management

Opposite the rigorous adherence to specific development practices is experimentation with general project management practices. In this area, things are much more about principles (primarily feedback and continuous improvement) and less about practices or processes. Your sprint should be as long as you need to get good feedback, which varies depending on project/team/technology. Your retrospectives should be run so you can continuously improve. Your planning should be so you can get more accurate and insightful. Ten solutions can work well at ten different workplaces. Even more, ten solutions can work well at the same workplace(4). Just make sure you are continually improving, and keep trying new things!

Having your cake and eating it too

This distinction between development and project management is how I navigate the rift between Agile nihilists and Agile purists. The former say with disdain, “Whatever works for you!” The latter chant fervently, “You cannot pick and choose!” It turns out they can both be right, but it’s important to understand how and why. The nihilists end up floating, never realizing the transformative power of Agile because they refuse to adhere to the three vital, but initially taxing, processes. The purists can drive change, but not transform, because they do not create new practices that fully embrace each unique situation.(5) Rather than trying for a happy medium between the poles, I find Agile is best done by being at the extremes simultaneously.

1. I don’t know how to rebut arguments against this point, other than asking “have you worked on a project that was well-developed with TDD?” If not, I would try it out before you make excuses for a compromised form of Agile without comprehensive automated tests.
2. More accurately, what they achieve I consider absolute. If you wanted to get rid of any of them, you’d need to replace them with something suitable. For example, if you didn’t want to use TDD, you’d need to demonstrate some other way to reliably build comprehensive automated tests. And actually I have great hopes for some automated alternative to TDD one day…
3. Uncle Bob recently posted about how software projects need this sort of discipline, perhaps by having a “foreman”. I don’t agree on the solution, but I do agree that we do need to rigorously adhere to certain practices. http://blog.8thlight.com/uncle-bob/2014/02/21/WhereIsTheForeman.html
4. This is probably attributable to the Hawthorn Effect: http://en.wikipedia.org/wiki/Hawthorne_effect
5. To be fair, many purists (especially outside of programming) overlook the three vital development practices because they are so keen on implementing the easier Scrum project management tools that require less training and invasive changes.

Whenever I see the words “delighting customers” (which is, let’s face it, an awkward phrase) in a non-Lean context like a job description, I can feel the author winking at me. It tells me “we try to be Lean and if you get our drift you probably want to join us.” It instantly gives said company plenty of extra points.

I just wonder how long until “delighting customers” becomes a played out catchphrase (if it’s not already)?

Refraktor:  (verb) When you refactor some code and in the process change or mess it up so completely that you need to revert all your changes.

A while ago, one of the junior TA’s here was frustrated with all the code critiquing I was giving him, and exclaimed: “In 6 months, we haven’t had an argument about my code which I’ve won.”

I replied, “Well, get better at coding or get better at arguing.”

He got better at both.