Change should be the ally of quality
In The Beauty of Testing, Steven Sinofsky writes:
…great testers understand one the cardinal rules of software engineering—- change is the enemy of quality.
This is not a cardinal rule. This is a outdated and obsolete mode of thinking. Change is how you discover great UX. Change is how you refactor and reduce technical debt. Change is how you incrementally improve both your product and code quality.
Maybe that’s too obvious, and clearly Sinofsky isn’t arguing for static software. More nuanced (and the rest of the piece provides that nuance) would be “change inevitably introduces bugs, and bugs reduce quality.”
This too I take issue with. Your codebase should be verifiably better after you fix a bug: you’ve found a shortcoming in your automated tests, so you add a test, and maybe refactor some stuff as well. Or, you’ve identified a bad experience, and can change it to be better in a controlled manner. A bug is an opportunity for improvement. Without bugs, it can be very difficult to improve.*
It can be difficult for anyone who hasn’t worked in a codebase with extensive testing to understand this. In most cases, fixing bugs is playing whack-a-mole. Whack-a-mole is unacceptable to me. Every change we make at Cozy is making the code clearer, simpler, better tested. It’s making the product smoother, faster, and more intuitive.
Change is necessary; it is up to you to determine if it is a friend or foe.
If you’re practicing disciplined development and automated testing and not creating many bugs, good job! This post isn’t for you :)
It’s not just about bugs – change has more side effects than that. Since you mentioned UX, just think what happened with Windows 8’s new start menu. It’s not a bug as such, but it caused serious troubles in a larger system – that of how users interact with their OS. In the view of many users this change reduced the quality of the software. This is where automated tests fall short, and where you have to do acceptance testing.
Especially when it comes to UX, change – and experimentation can affect quality, even if it’s just perceived quality, that has nothing to do with the way a programmer defines bugs related to code.
Still, change is good, and it’s gonna break things. But that’s how things evolve, and that’s how we learn – from breaking stuff ;)
Right Jon, nor would any sort of “Acceptance Testing” stop the new start menu. It was *exactly* what MS wanted, I promise you, and it was a total failure of product design. That wasn’t a failure of programming or QA. Also, think hard about the implications of *not* changing anything about Windows. MSFT would watch its share dwindle more and more. In the end, these changes will have benefits, and didn’t need to have the problems they did, but this is neither here nor there. That is a management failure (one caused largely by Sinofsky himself, by the way!), having nothing to do with construction quality, which is the foundation of product quality.
But “testers” (whatever exactly that means, I’m not comfortable with its implications of a person just testing) wouldn’t prevent the new start menu. I would not be surprised to hear MS ran user feedback courses to gauge how people reacted, and found it acceptable in achieving whatever goals they were pursuing. Just like they did with the (frequently ridiculed for some reason) Office ribbon. That’s not software quality for testers, it’s.. product quality and alignment to long term goals.
Anyway,
With proper design and automated tests, code really can be changed without introducing a myriad of potential bugs and regressions and the resulting feeling of low confidence. That’s not to say it’s easy to achieve that (certainly I don’t find it easy and frequently fall off the path), but Rob is right, the potential is always there.
I think the idea that some sort of quality is a foundation for all other quality is not correct, nor is the view that there is something like “product quality” vs “software quality”. If your product is a piece of software, then it’s one and the same. The lack of having a holistic view on quality is exactly what causes the “not my problem” attitude we see so often in development.
And it’s definitely not a management problem. Gathering requirements, forming use cases, and defining test cases based on that clearly falls into the software engineering domain. Hence it’s the responsibility of the dev team.
Although you may be correct that acceptance testing could not have caught MS’ start menu problem. If you get your requirements wrong and test against wrong requirements, then your tests will still show up as “passed”. And that the requirements were wrong clearly showed when the majority of customers complained about the new feature.