Software design is like salt in a recipe: too much and everyone hates it, too little and everybody asks for more. With just the right amount you get no credit and nobody thinks it added anything!
Oct 15, 2023·edited Oct 15, 2023Liked by Kent Beck
Kent Beck
I feel you. I've also been flying this banner for the last ~20 years in my own work and have proceeded with the same rationale as you, despite working with many many others who don't have the same resolve.
I have, however proven (to myself at least) as a technical founder of my own tech startup, (on the tools) that the big ball can absolutely be avoided - even after 8 years in market.
But I can only provide evidence of this at the small end of the scale of team sizes. Teams of less than 8-10.
The real challenge I see, based on lived experience across a very diverse career, working with literally 100s of codebases and teams, is that there simply is not enough evidence available to past or future developers/engineers of this working well enough - so that they just adopt it as a mater of course (lack of availability bias). and thus, we have not reached critical mass yet across the industry.
Every single one of them (after a certain stage of competence) gets it intellectually (and knows they should be doing it), but applying the necessary rigor and discipline is where most just fall back to what is easier for them. (We have a long history of geeks avoiding taking risks and responsibility, and delegating that to their bosses)
It has a lot to do with age and stage, and massively to do with the context they are in (most are just incentivized to care about problem-solving rather than the outcome they create, and too many are working in project contexts where they are not responsible for the future mess they create).
I write a lot about these contextual differences, that few programmers seem to recognize, let alone appreciate. https://jezz-santos.medium.com
Each and every developer/engineer that I have demonstrated this discipline to firsthand has been a long drawn-out campaign of its own, person by person. I have a high success rate of doing that at the individual level, but I can only affect so many.
There are just too many commoditized developers in the world today, and not enough teaching them hands-on what good looks like.
Haha I think any project with more than a few devs does turn into this due to time constraints.
It can’t be underestimated enough that a good design that is future proof saves insane amounts of time and bug frustrations in the future though...
some of the bad code becomes so messy nobody can visualize all the different states in their head, and it becomes impossible to confidently extend without introducing bugs.
Having observed lots of really smart (and plenty of not-so-smart) development people (programmers, developers, architects, testers, etc.*) in action over the years, I've concluded that at least part of the answer to "what's up with that?" is that we don't leverage the influence we have often enough and with enough gusto. Probably many times a Development Person doesn't even realize the influence or power he or she has. Other times I'm sure it's just a matter of personality. Whatever the reasons, I think we are partly to blame because we don't use leverage for good.
Me, I'm extremely outspoken and seldom hesitant to rock the boat. So I speak up - too much sometimes. I'm still learning (but practice a lot) to leverage my influence to advocate for, and sometimes insist on, the activities that yield good software design. I encourage my fellow Development People to do the same - speak up, don't let tech debt go unchecked, pad your estimates, speak up, - whatever it takes to prove that good design _does_ matter and can save time/money or expand their value in the future.
*I need a good label to encompass all of us that doesn't alienate any of us - maybe another article idea for you, KB :-)
I use the labels "geeks" or "changers". That's the part of the team who can first hand go make changes.
As for influence, that's the topic of book 3--relationships with the customers & how to make software design a positive contribution to those relationships.
I suppose it's just the the way the human brain works. We all tend to choose the short-term benefit, it's more obvious and seems more valuable than the long-term gain.
It always takes a lot of time, training and thinking to switch that default behavior off. It's a lot of hard work with no short-term gains though, so you can guess that very few people do that.
The good news is that it's a one-way street. Once you experience the benefits of good design first hand, there is no turning back, you literally cannot work the old way.
i know... i think it's because higher up people can't see the time saved, they just see whats in front, and if 2 hours a day is fighting fires but the new feature is out fast enough, they can't comprehend that even more could be done without all the wasted time due to the bad design
It just makes me glad that I work where I do, where the business leaders in the company understand that a well-designed code base makes future changes easier (and cheaper).
It wasn't always that way: when I joined there were no tests for the legacy system and no automation or repeatability, and it took time to get that all in place for the next gen system I came on to help build, and it's really only gotten smooth and fully-automated with the third gen system we have now (which was an incremental rewrite rather than the "big bang" of the 2nd gen rewrite).
The business leaders understand now that tech debt gets in the way of progress and when we want to clean code up and refactor it to get rid of tech debt, they give us the space to do so.
I'd love it if we never created tech debt but even with small steps and good design, I'm not sure if that's entirely achievable?
At this stage of my career, I think the first big problem is that most developers don't have a fully-formed, principled idea of what good design is, in part because they've only ever seen balls of mud.
Add in superstitious beliefs, compulsive adherence to the latest fashions, and failures of courage, and there's not much opportunity for improvement without exceptional leadership (which is rarer and harder to acquire)
What you have done so far in your career is already the best on offer. If only more developers availed themselves of it!
I'm trying to avoid "good" & "bad". Compared to what? I talk about improving instead. "When we improve the design, tests are less coupled to structure," instead of, "Good designs decouple tests from structure." See if that helps.
Very stoic! At some point we must believe what's been created is "good", otherwise we never stop. However, down the line, design fails and something else takes over - practicality, time-to-market, poor communication, pick your poison. I think it's an interesting topic to dive into.
I think doing some design upfront protects us from the path dependence. If we do design along the way, as we make changes, the ordering of the changes will have significant impact on the quality of the final design.
I've seen it in a big company I worked for -- every time I would ask why something is so wrong I would get a history lecture. As the company had billions of lines of code by then, I was deemed impossible to make any real structural changes.
> Would making those design decisions earlier make path dependence better or worse?
It allows the possibility of making them better.
I imagine design to be a process of imagining the system we're gonna build in our heads (with docs and visual aids), so we can examine its properties without investing in building it (presumably cheaper). Say, we can imagine 4 sprints of code changes and discover properties of the system we really don't like? We can then change our minds and make the system different (still in our hands) and use this to inform how we code starting from sprint 1.
I think even on this example we can see all that can go wrong and how hard it is to get design right. Can we imagine 4 sprints of system upfront? Are we ready to be critical about our design? Are we ready to redesign if we discover in sprint 2 something we missed completely?
Note: I'm bad at design, and didn't even read your book yet, so apologies if this is all rehashing stuff you already addressed, or using definitions that are known to be wrong, etc.
Design matters in as much as it helps describe intent within the team. In the same way that a good metaphor can focus design decisions, "good" design helps members get up to speed with regard to system behaviors. That level of communication and understanding is vital (I believe) to how the system continues to evolve. Without that ability to communicate the whys and the wherefores, it becomes easy to cut corners in the service of expediency, thus diminishing the power of the design and "muddying the waters".
One interesting data point in support of design being valuable is that Google has found a causal relationship between subjective satisfaction with code quality and productivity metrics: https://dl.acm.org/doi/pdf/10.1145/3540250.3558940.
IIRC, your mission isn't to help business become successful; it's to help software engineers stay sane. By that measuring stick, you're focused on the right things. Businesses can make money either way.
Is there a "sweet spot" on the curve of economic performance where optimal conditions for safety exist?
Adam Grant has a quadrant called the four deadly sins of organizational dysfunction. I have to wonder whether the pursuit of economic performance puts an organization at greater risk of falling into the toxic culture quadrant. In that quadrant, organizations put performance over people to a fault.
This also makes me think of the concept of "good value" where the cost-benefit ratio is equal or better. What's the equivalent of a "good value" for an organization WRT performance-safety? Is it economic stability? Even then, the organization might stifle creative pursuits. IME, it's not the financial success of the org that leads to safety, rather it's the people and especially the leader of your unit.
And there's a problem with orgs not having economic security - they fail and you lose your job.
Haven't looked for hard data on this question and this is just my own observation/experiences. Just wanted to raise the question of whether "superior economic performance" is the right condition or if something like "sustainable economic performance" would more likely lead to better safety outcomes.
OIC. What would make economic performance "superior" vs whatever else there is in the lexicon? IOW, how can you tell when an org has superior economic performance.
"superior" as observed by gross margin & growth rate--basically NPV. One challenge is software-as-a-business is a team sport. Great engineering * bad marketing = squadoosh. Boo hoo. Great engineering * great marketing * wrong product. Also squadoosh. That's why leaders get paid to see the whole.
Sadly, many non-technical leaders don't see the whole at all. They only see blah-blah-meaningless-blah * MARKETING * whatever-I'm-not-listening-anymore = $$$$$
What if the user was more important than our code? Crazy idea I know! As we build, a good enough design can be much more efficient and pragmatic.
It is when we fail to see the cost of our technical debts that we are failing. We are also failing when not shipping features for a perfect design. It's all about balance and context.
I think this points to where the problem is. In most organizations, people believe the culture won’t fully let them make it better later. To compensate, they choose an extreme on either side: too much too early, or too little too late. Both extremes offer something that’s easier to measure. Your middle position requires the culture to have faith in the value of “making it better” work that’s harder to measure.
Very insightful take, thanks! If I can make a request though: Would you consider scheduling the ones like this for Monday morning when we have a whole week of day-to-day stuff to distract us from reexamining our life choices? Thanks (also for ruining my weekend)!
"...they were determined to only write good code. By the time they were wildly successful, their code was the same ball of mud as everybody else. Maybe the 2 states of a system are ‘disaster’ & ‘almost disaster’."
I'd love to know how you (or this CEO) define "good code", "ball of mud", "disaster", and "almost disaster". To me, these seem almost orthogonal. I don't think balls of mud are always bad or disastrous.
I work on a multi-million-line system that's rife with tight coupling and accidental complexity. It _might_ be a big ball of mud. But it's neither a disaster nor almost one. Although some parts of it take more effort to change than I'd like, I'm not afraid to change it, and we deploy to production a dozen times per day.
I think of software design as holding space for complexity. Good code creates a complexity vacuum, which "wants" to be filled. You can fill it wisely (with value-generating functionality) or unwisely (with cruft or speculative features), but it's gonna get filled. The complexity of the system will expand to fill the space made by the design, however big or small that space is.
This, I think, is why everyone's code ends up as "the same ball of mud". Code tends toward a kind of high-entropy, hard-to-change equilibrium. The thing we control by doing design is how much value-generating complexity we'll be able to fit into the system before reaching that equilibrium.
(oversimplifying, of course—design can be improved at any time—but unless we make a continuous effort toward good design, entropy increases.)
Does software really need design? Do trees really need roots and woody stems? After all, mosses and algae seem to be doing just fine.
There is no such thing as "good design in a vacuum". When people say that design is "good" they really mean some optimization criteria which they like and how well a design conforms that criteria. A better way to define "good design" would be to enumerate the criteria we could optimize for and check if we (and business) indeed want to optimize for it. It can be a very insightful exercise.
Software design absolutely matters. The existence of the role of solutions architect is vital now. It contributes to system's testability (as you mentioned), reliability, scalability, maintenability and more. Business people do not understand it and they put pressure for fast delivery but this does not mean that software design does not matter.
A little off topic, but shouldn't all those quality attributes you listed for the "solution architect" role be the responsibilities of every person writing the code? If not, and we leave it to only the those experienced at it (with fancy titles, and status bumps), are we going to disseminate these vital skills across teams so they all take care of these things? Quality cannot be inspected in, it has to be designed in from the start, no?
That's why the sequence of books is the way it is. TF? is intended to make structure changes just a normal part of daily work, without need for approvals or permission. Then the team book will make structure changes part of daily, communal work, again without need for ceremony. Only when those habits are in place will we talk about architectural change that affects customers.
Software design is like salt in a recipe: too much and everyone hates it, too little and everybody asks for more. With just the right amount you get no credit and nobody thinks it added anything!
A beautiful summary. I’ll swipe it.
Kent Beck
I feel you. I've also been flying this banner for the last ~20 years in my own work and have proceeded with the same rationale as you, despite working with many many others who don't have the same resolve.
I have, however proven (to myself at least) as a technical founder of my own tech startup, (on the tools) that the big ball can absolutely be avoided - even after 8 years in market.
But I can only provide evidence of this at the small end of the scale of team sizes. Teams of less than 8-10.
The real challenge I see, based on lived experience across a very diverse career, working with literally 100s of codebases and teams, is that there simply is not enough evidence available to past or future developers/engineers of this working well enough - so that they just adopt it as a mater of course (lack of availability bias). and thus, we have not reached critical mass yet across the industry.
Every single one of them (after a certain stage of competence) gets it intellectually (and knows they should be doing it), but applying the necessary rigor and discipline is where most just fall back to what is easier for them. (We have a long history of geeks avoiding taking risks and responsibility, and delegating that to their bosses)
It has a lot to do with age and stage, and massively to do with the context they are in (most are just incentivized to care about problem-solving rather than the outcome they create, and too many are working in project contexts where they are not responsible for the future mess they create).
I write a lot about these contextual differences, that few programmers seem to recognize, let alone appreciate. https://jezz-santos.medium.com
Each and every developer/engineer that I have demonstrated this discipline to firsthand has been a long drawn-out campaign of its own, person by person. I have a high success rate of doing that at the individual level, but I can only affect so many.
There are just too many commoditized developers in the world today, and not enough teaching them hands-on what good looks like.
Haha I think any project with more than a few devs does turn into this due to time constraints.
It can’t be underestimated enough that a good design that is future proof saves insane amounts of time and bug frustrations in the future though...
some of the bad code becomes so messy nobody can visualize all the different states in their head, and it becomes impossible to confidently extend without introducing bugs.
That's the conundrum. We see design saving all kinds of time & headache & yet we don't do it. What's up with that?
Having observed lots of really smart (and plenty of not-so-smart) development people (programmers, developers, architects, testers, etc.*) in action over the years, I've concluded that at least part of the answer to "what's up with that?" is that we don't leverage the influence we have often enough and with enough gusto. Probably many times a Development Person doesn't even realize the influence or power he or she has. Other times I'm sure it's just a matter of personality. Whatever the reasons, I think we are partly to blame because we don't use leverage for good.
Me, I'm extremely outspoken and seldom hesitant to rock the boat. So I speak up - too much sometimes. I'm still learning (but practice a lot) to leverage my influence to advocate for, and sometimes insist on, the activities that yield good software design. I encourage my fellow Development People to do the same - speak up, don't let tech debt go unchecked, pad your estimates, speak up, - whatever it takes to prove that good design _does_ matter and can save time/money or expand their value in the future.
*I need a good label to encompass all of us that doesn't alienate any of us - maybe another article idea for you, KB :-)
I use the labels "geeks" or "changers". That's the part of the team who can first hand go make changes.
As for influence, that's the topic of book 3--relationships with the customers & how to make software design a positive contribution to those relationships.
I suppose it's just the the way the human brain works. We all tend to choose the short-term benefit, it's more obvious and seems more valuable than the long-term gain.
It always takes a lot of time, training and thinking to switch that default behavior off. It's a lot of hard work with no short-term gains though, so you can guess that very few people do that.
The good news is that it's a one-way street. Once you experience the benefits of good design first hand, there is no turning back, you literally cannot work the old way.
i know... i think it's because higher up people can't see the time saved, they just see whats in front, and if 2 hours a day is fighting fires but the new feature is out fast enough, they can't comprehend that even more could be done without all the wasted time due to the bad design
It just makes me glad that I work where I do, where the business leaders in the company understand that a well-designed code base makes future changes easier (and cheaper).
It wasn't always that way: when I joined there were no tests for the legacy system and no automation or repeatability, and it took time to get that all in place for the next gen system I came on to help build, and it's really only gotten smooth and fully-automated with the third gen system we have now (which was an incremental rewrite rather than the "big bang" of the 2nd gen rewrite).
The business leaders understand now that tech debt gets in the way of progress and when we want to clean code up and refactor it to get rid of tech debt, they give us the space to do so.
I'd love it if we never created tech debt but even with small steps and good design, I'm not sure if that's entirely achievable?
At this stage of my career, I think the first big problem is that most developers don't have a fully-formed, principled idea of what good design is, in part because they've only ever seen balls of mud.
Add in superstitious beliefs, compulsive adherence to the latest fashions, and failures of courage, and there's not much opportunity for improvement without exceptional leadership (which is rarer and harder to acquire)
What you have done so far in your career is already the best on offer. If only more developers availed themselves of it!
Perhaps the best follow-up topic would be "Why do good designs still result in a 'ball of mud'?"
I'm trying to avoid "good" & "bad". Compared to what? I talk about improving instead. "When we improve the design, tests are less coupled to structure," instead of, "Good designs decouple tests from structure." See if that helps.
I assume people have seen this analysis of the upside of the big ball of mud: https://www.researchgate.net/publication/2938621_Big_Ball_of_Mud/link/54d4eb900cf2970e4e63e44d/download
I think a perfectly clean design is impossible. That's why in the Empirical Software Design world, "perfect" is a verb, not an adjective.
Very stoic! At some point we must believe what's been created is "good", otherwise we never stop. However, down the line, design fails and something else takes over - practicality, time-to-market, poor communication, pick your poison. I think it's an interesting topic to dive into.
I think doing some design upfront protects us from the path dependence. If we do design along the way, as we make changes, the ordering of the changes will have significant impact on the quality of the final design.
I've seen it in a big company I worked for -- every time I would ask why something is so wrong I would get a history lecture. As the company had billions of lines of code by then, I was deemed impossible to make any real structural changes.
1. Would making those design decisions earlier make path dependence better or worse?
2. I disagree that in situ structural changes are impossible given a) skills and b) strong human relationships.
> Would making those design decisions earlier make path dependence better or worse?
It allows the possibility of making them better.
I imagine design to be a process of imagining the system we're gonna build in our heads (with docs and visual aids), so we can examine its properties without investing in building it (presumably cheaper). Say, we can imagine 4 sprints of code changes and discover properties of the system we really don't like? We can then change our minds and make the system different (still in our hands) and use this to inform how we code starting from sprint 1.
I think even on this example we can see all that can go wrong and how hard it is to get design right. Can we imagine 4 sprints of system upfront? Are we ready to be critical about our design? Are we ready to redesign if we discover in sprint 2 something we missed completely?
Note: I'm bad at design, and didn't even read your book yet, so apologies if this is all rehashing stuff you already addressed, or using definitions that are known to be wrong, etc.
Definitely read the book, then I'd be glad to discuss further.
Design matters in as much as it helps describe intent within the team. In the same way that a good metaphor can focus design decisions, "good" design helps members get up to speed with regard to system behaviors. That level of communication and understanding is vital (I believe) to how the system continues to evolve. Without that ability to communicate the whys and the wherefores, it becomes easy to cut corners in the service of expediency, thus diminishing the power of the design and "muddying the waters".
One interesting data point in support of design being valuable is that Google has found a causal relationship between subjective satisfaction with code quality and productivity metrics: https://dl.acm.org/doi/pdf/10.1145/3540250.3558940.
I'm not surprised, but that's my bias.
IIRC, your mission isn't to help business become successful; it's to help software engineers stay sane. By that measuring stick, you're focused on the right things. Businesses can make money either way.
Superior economic performance isn’t my goal, but it’s a necessary condition for safety.
Is there a "sweet spot" on the curve of economic performance where optimal conditions for safety exist?
Adam Grant has a quadrant called the four deadly sins of organizational dysfunction. I have to wonder whether the pursuit of economic performance puts an organization at greater risk of falling into the toxic culture quadrant. In that quadrant, organizations put performance over people to a fault.
This also makes me think of the concept of "good value" where the cost-benefit ratio is equal or better. What's the equivalent of a "good value" for an organization WRT performance-safety? Is it economic stability? Even then, the organization might stifle creative pursuits. IME, it's not the financial success of the org that leads to safety, rather it's the people and especially the leader of your unit.
And there's a problem with orgs not having economic security - they fail and you lose your job.
Haven't looked for hard data on this question and this is just my own observation/experiences. Just wanted to raise the question of whether "superior economic performance" is the right condition or if something like "sustainable economic performance" would more likely lead to better safety outcomes.
“Superior” is important because it creates incentive for change. “Maximum” is a goal that leads to all kinds of dysfunctions.
OIC. What would make economic performance "superior" vs whatever else there is in the lexicon? IOW, how can you tell when an org has superior economic performance.
"superior" as observed by gross margin & growth rate--basically NPV. One challenge is software-as-a-business is a team sport. Great engineering * bad marketing = squadoosh. Boo hoo. Great engineering * great marketing * wrong product. Also squadoosh. That's why leaders get paid to see the whole.
Sadly, many non-technical leaders don't see the whole at all. They only see blah-blah-meaningless-blah * MARKETING * whatever-I'm-not-listening-anymore = $$$$$
What if the user was more important than our code? Crazy idea I know! As we build, a good enough design can be much more efficient and pragmatic.
It is when we fail to see the cost of our technical debts that we are failing. We are also failing when not shipping features for a perfect design. It's all about balance and context.
We can always make it better later. That's a basic belief of mine that doesn't seem to be widely held.
I think this points to where the problem is. In most organizations, people believe the culture won’t fully let them make it better later. To compensate, they choose an extreme on either side: too much too early, or too little too late. Both extremes offer something that’s easier to measure. Your middle position requires the culture to have faith in the value of “making it better” work that’s harder to measure.
Very insightful take, thanks! If I can make a request though: Would you consider scheduling the ones like this for Monday morning when we have a whole week of day-to-day stuff to distract us from reexamining our life choices? Thanks (also for ruining my weekend)!
Just wow 😳
"...they were determined to only write good code. By the time they were wildly successful, their code was the same ball of mud as everybody else. Maybe the 2 states of a system are ‘disaster’ & ‘almost disaster’."
I'd love to know how you (or this CEO) define "good code", "ball of mud", "disaster", and "almost disaster". To me, these seem almost orthogonal. I don't think balls of mud are always bad or disastrous.
I work on a multi-million-line system that's rife with tight coupling and accidental complexity. It _might_ be a big ball of mud. But it's neither a disaster nor almost one. Although some parts of it take more effort to change than I'd like, I'm not afraid to change it, and we deploy to production a dozen times per day.
I think of software design as holding space for complexity. Good code creates a complexity vacuum, which "wants" to be filled. You can fill it wisely (with value-generating functionality) or unwisely (with cruft or speculative features), but it's gonna get filled. The complexity of the system will expand to fill the space made by the design, however big or small that space is.
This, I think, is why everyone's code ends up as "the same ball of mud". Code tends toward a kind of high-entropy, hard-to-change equilibrium. The thing we control by doing design is how much value-generating complexity we'll be able to fit into the system before reaching that equilibrium.
(oversimplifying, of course—design can be improved at any time—but unless we make a continuous effort toward good design, entropy increases.)
Does software really need design? Do trees really need roots and woody stems? After all, mosses and algae seem to be doing just fine.
There is no such thing as "good design in a vacuum". When people say that design is "good" they really mean some optimization criteria which they like and how well a design conforms that criteria. A better way to define "good design" would be to enumerate the criteria we could optimize for and check if we (and business) indeed want to optimize for it. It can be a very insightful exercise.
Software design absolutely matters. The existence of the role of solutions architect is vital now. It contributes to system's testability (as you mentioned), reliability, scalability, maintenability and more. Business people do not understand it and they put pressure for fast delivery but this does not mean that software design does not matter.
A little off topic, but shouldn't all those quality attributes you listed for the "solution architect" role be the responsibilities of every person writing the code? If not, and we leave it to only the those experienced at it (with fancy titles, and status bumps), are we going to disseminate these vital skills across teams so they all take care of these things? Quality cannot be inspected in, it has to be designed in from the start, no?
Absolutely they should do it. This is what I want developers to do.
However, In my experience many times this does not happen and the experienced one/s have to step in and do the design but also follow up.
Yeah, I know. Too many inexperienced devs, not enough to teach them
That's why the sequence of books is the way it is. TF? is intended to make structure changes just a normal part of daily work, without need for approvals or permission. Then the team book will make structure changes part of daily, communal work, again without need for ceremony. Only when those habits are in place will we talk about architectural change that affects customers.