It’s deployment day, and you feel that familiar dread that comes with knowing that your code is going into production. You whisper a silent prayer: “Please God, don’t let my code take down production.” As the servers are restarted you feel the pit in your stomach growing. Beads of sweat roll down your forehead as you watch the logs, check the alarms, and monitor the sales.
If this sounds familiar, you may be working in an environment that practices fear-driven development.
There are two critical factors that need to be present in an enterprise for fear-driven development to take hold.
The first factor is the creation of an environment where no one can actually be sure that their changes won’t break production.
This can happen a number of ways: Your test environment doesn’t accurately reflect your production environment, your code and applications interact in ways that no one fully understands or can predict, you have no ability to do load testing, or you simply don’t have the resources required to test all paths within your applications before pushing a change to production.
When no amount of due diligence can provide a reasonable guarantee that your code works as expected in production, it is inevitable that problems will arise regardless of how good your developers are and how much care they take while writing code.
And the inevitability of production issues is the first requirement in establishing a culture of fear-driven development.
The second factor is assigning personal blame when problems do inevitably arise.
It is entirely possible to have development processes that don’t provide a reasonable guarantee of a successful production deployment and still not create a culture of fear.
If production goes down due to a service being overwhelmed with traffic, an enterprise can choose to use this as a learning experience and incorporate load testing into their development cycle. This solves the root cause of the issue, developers are reassured that it was a fault in the process that lead to the failure, and it ensures that in future this particular kind of problem can be addressed before it becomes a production problem.
If instead, an enterprise chooses to find those responsible for the production outage and hold them personally responsible, the cycle of fear is complete. Developers soon realise that there is no reasonable way to prevent their code from eventually breaking production, and they learn that when that day comes, management will not bear any responsibility for creating an environment where breaking production is unavoidable.
Consequences of the fear
The only sane choice in an environment that practices fear-driven development is to never leave your fingerprints on those dominos that have fallen. This means never participating in any kind of development that, through action or inaction, could lead to an outage.
You may also like: Diverting Blame Effectively in Software Development
Does you code need refactoring to make it more maintainable? Forget it. Developers will just hope and pray that they are never asked to implement a feature that uses this old unmaintainable code.
Do you need tests written to ensure your applications don’t break with future changes? That’s not going to happen, because it violates the unwritten tenet of fear-driven development of plausible deniability:
If you didn’t write the code, and you didn’t test the code, then you get to look the other way when management begins blamestorming after an outage.
Do you need people to just stand up and take responsibility for the solution? Good luck with that, because in a fear-driven culture the only measure of success if how infrequently your name is uttered in anger after a deployment. Taking responsibility for the solution almost certainly means touching code that will fail at some point, and such actions are to be avoided at all costs.
Fear-driven development is a vicious cycle that discourages the kind of personal responsibility that developers need to take to address the root cause of issues, which leads to more issues in production, which leads to more blame being assigned, which goes right back to discouraging personal responsibility.
Next time you lose sleep over a deployment because you fear being called out for breaking production despite your best efforts, it might be time to admit that you are caught in the cycle of fear-driven development.
PS. There are always better jobs.
Originally published December 2015