Software Is Bananas
Posted September 2017
I posted about software development recapitulating the evolution of software development in general. Rather than harp on problems and specifics, I’m going to constructively talk about software development principles and values. I’ll also address one issue at a time and take time to listen before following up.
Ed: I had gotten strident & aggressive. Yes, me. I was toning it down.
The fundamental truth about software development is that we might be wrong. Every decision we make might be a mistake. One step off the path and we’re on the shoulder. Two and we’re in the weeds. Three and we’re over the cliff.
The lack of automated testing is a big surprise. Over time I came to realize one good reason for skipping automated testing: scale. If problems only appear at scale, then the first time you can possibly catch them is in production.
Some software is like this—the probability & cost of a simple logic error is low, the probability & cost of systemic errors is high.
Carefully testing for problems that won’t be problems is absurd.
So What?
How do we change software development if:
Mistakes are inevitable,
Mistakes compound,
Mistakes only manifest in production?
Well, we should:
Reduce the latency between decisions and feedback from production
Reduce the number of decisions we push at once
Invest in reversibility so we can continue to reduce latency and batch sizes
There are sensible ideas for software development that violate these principles. (Kinder, gentler me is going to let you work out what those are in case one of them is your pet technique.) If you violate these principles, you are invoking a tradeoff. Whatever you gain, you are giving up a precious resource—safe feedback. Whether the tradeoff is worth it is up to you.
Conclusions
Software is bananas. Buy a banana and you’d better eat it. You’re not going to be happy with the experience in a month.
Interesting to see me using the “rotting food” metaphor in another context. I think I’m trying to sensitize people to the cost of delay. Somehow some folks just don’t feel it.
This is particularly pernicious when your code is a service that executes other people's (as yet unknown) code. There are infinities of unknowns when your workload is arbitrary. We test as much as the budget will allow and monitor the shit out of everything we can in production to catch failure modes we couldn't've even imagined. People will do weird shit with your bananas that you never intended nor could anticipate.
Slightly different context, but the same basic theme: Why Milk is Thicker Than Water https://flowchainsensei.wordpress.com/2012/11/06/why-milk-is-thicker-than-water/