I love seeing the discussion of how front-end development mixes into the wider software design mix. Last year I posted about what frontend and backend even mean. There is little shared understanding leading to frequent misunderstanding https://www.industrialempathy.com/posts/frontend-backend/
In my post I call your usage of front-end uppercase-Frontend to disambiguate from lowercase-frontend which is just the part of any distributed system talking to its clients.
Some general topics that are interesting about frontend development:
- Skew boundaries involve more long-lived clients not under deployment control of the system author
- More-commonly stateless services making techniques like immutable deployments more feasible
- The general direction of the last 10 years in state-of-the-art frontend development has been to blur the client-server boundary and to eliminate separation of concerns: A move from MVC architecture to unidirectional dataflow (which now completely dominates web development). The implications of unidirectional dataflow for backend technologies remain underexplored.
Working on Screenshotbot, I have unique insights into this, and it's usually part of my pitch to potential customers. Unit tests fail relatively rarely (used as a proxy for how often the system behavior changes), UI changes very frequently (using screenshot tests as a proxy for frontend behavior changes).
For instance, most screenshot test changes we see on Screenshotbot are intentional changes. Very rarely are they actually "rejected" for being a bug. I could run the exact numbers.
(To complete the sales pitch I use, many companies use screenshot tests like unit tests, or what some people call acceptance tests. They store screenshots in the repo and the tests fail when screenshots change. But that introduces a lot of overhead for UI devs because UI changes so damn frequently, and UI devs have to keep re-recording screenshots manually. It's the wrong way to think about screenshot tests.)
Okay, so keep in my startup is real small. And the data isn't as clean as I hoped.
I've restricted the data to 90 days, to keep it clean.
Roughly speaking, here's how this data looks like: For every PR, the customer's CI generates images and uploads it to us, this creates a "run". If we notice any difference we create a "report" and notify them on GitHub/BitBucket etc, and they get to accept or reject the changes. Runs are also generated on main branches, but I'm skipping those since users can't accept/reject them.
In theory, there might be multiple runs per commit, because some companies will break up their project into smaller sub-projects, but in the data I picked I think it's rare. Also each version of the PR will have its own run (i.e. you update the PR and it generates a new run). I skipped companies that use a mono-repo because that behaves very differently (number of runs grows quadratically, and that might skew the data).
In the last 90 calendar days, there were 8148 runs generated attached to a PR. Of these 4221 generated a report (pretty high IMO, but my customers are skewed towards heavy UI changes, perhaps having Screenshotbot makes them more confident to make UI changes). Of these 1420 were accepted, and only 14 were rejected. 2787 had no action (probably because it was an intermediate state of the PR and they just chose to create new version, very hard to assume if it's either a bad or good change)
Can you sprinkle magic AI fairy dust on it to reduce the number of false positives? I'd be especially interested in combining the contents of the PR with the changes to the appearance to get even better accuracy.
But you're not thinking like a UI dev! You're thinking like a backend dev!
Say you change a core button, say from rounded to square border or some such. As a UI dev, you want to see how that change affects the rest of the app. You know what the change is, but you still want to visualize it (and communicate that visual change to code reviewers).
Conversely, if you are in a product team, and a core UI developer changes a button, you want to be notified that your screen has changed. You've spent so much time making it pixel perfect, and you want to be able to keep it that way.
So a "false positive" is still a very valuable notification for Screenshotbot. Of course, it would be nice if the AI can tell you if this might be something important or not.
Btw, these numbers are consistent with what I saw at Facebook. (I used to tell people that only about 2% of UI changes were actual bugs, and 98% were just UI developers making real changes.)
I see that in the User Interface, every time you give the users something, it provides inspiration for even more things you could do for them.
(I'm not blaming the users! I'm the same way! When I'm writing software for myself, I have to control the scope vigorously, as it tends to explode with great new ideas of even more things I could do!)
I don’t understand why people complain about this. If I’m mining and I come to the end of one vein of ore and it splits into 2 veins, that’s good news. I suppose if you’re looking for an answer to “when will it be finished”, you’re not happy with “there is no ‘it’ to be finished.”
I was working on a project and every time we released a new version to the business, they asked us to add even more features to the product. After a few releases, the project manager asked me in frustration, "When will this project ever be FINISHED?!?!?"
I said, "Never. Not until they stop using it, and hence don't care any more. As long as they're using the system, they will want changes and enhancements to it, to make it more valuable. And that's a good thing."
We often get hung up on the mental model of "doing a project" or "buying a thing" -- that it's a predictable transaction, rather than an ongoing, essentially "living" process of changing and improving processes over time.
In most cases, if it "ends," then you have failed.
My observations were that before agile, I saw several consultancies that specialized in evaluating legacy systems to inform top management of the (objectively, mathematically) best/right time to rewrite them -- when the cost of maintenance exceeds the cost (and risk) of building a new system.
What I felt agile (and refactoring) brought to the table was that there was no longer any rational reason nor economic justification for old code/systems to "inevitably" become unmaintainable.
Not unless there was a *radical* change in your underlying business model that required "ripping up" and completely redesigning how your business *fundamentally* works. (A rare event, indeed!)
I'd like to reframe Vladimir's, very good, take like so:
API's are consumed by emotional monkeys you can tell to "RTFM", and some will. UI's, on the other hand, is consumed by emotional monkeys who will misinterpret every though you put into it, and the docs, and refuse to buy your service/product/stuff if told you tell them to.
So the requirements of UI's to "make sense" in order to create value/revenue somehow scales, by a factor, with every monkey interacting with it (thousandfold that of an API), increasing the pressure and stress to get "get it right" when working on it (at least for me).
Does anyone else think this phenomenon observed in the original tweet is at least somewhat due to the difference in maturity* between the BE and FE technologies? Comparing Java/Spring, .Net, Python - heck even Ruby/Rails or PHP - against the seemingly constantly re-born Javascript framework du jour, and it doesn't surprise me that there's more churn in FE work.
Similarly, the acceptance and near ubiquitous application of patterns (such as GoF patterns, Enterprise Architecture Pattens, etc) in BE work, seems to be longer-standing than in FE.
I'm not saying Kent's theory is wrong or even not the most weighted factor, just that churn in platform/language probably produces increased churn in output.
* Perhaps "stability" or "ripeness" would be better, less inflammatory word choices.
I rather think that the churn in front-end frameworks reflects, rather than causes, the kind of churn in requirements. That "spawning" effect doesn't seem to me to be related to framework at all. What am I missing?
I see backend developers as far more mature than frontend developers. In some of the larger organizations I've worked, the backend guys aren't developers at all, they're engineers...they understand the intricacies of the build.
I rarely find a front end developer that understands the mechanics of a single threaded language like JavaScript, and how to create something that spawns additional thread in the OS, something that is easily done in JS.
Modern FE developers are tool experts, not technology experts.
Check yourself. "Mature"? FE & BE engineers work under very different constraints, part of which I outlined in my post. Seems to me you're projecting, which we call out here when it happens.
And here I was worried about the inflammatory interpretation of the word "mature"... :-)
My thought was that there's an additional aspect to the idea that FE work spawns more FE work, in perpetuity. That aspect isn't about the forever-evolving, ever-expanding requirements (which is where's Kent's analysis went); rather it's about contributing to the fact that the UI isn't ever "done." It contributes by forcing more compromise between cost-effective implementation and at-the-time ideal product, since every FE-framework-du-jour adds distractions and often, IME, loses capabilities (or makes them more expensive).
By the way, I'm OK if the answer to my initial sentence, "Does anyone else think..." is "No."
Honestly, I often feel sorry for FE developers because, like you said, the jobs feels like it's never done, because there's always something new! There are so many new "things" being churned, and few of them are really new, just a "better", "more optimized", "[insert more useless words here]" way of doing the same thing. The FE has become way over-engineered (just look at state management).
As opposed to the backend: we have a fixed number of languages, sure some people use frameworks, but I feel they really understand what's going on with them.
Notice that the sources of variation come from outside the system. There's no amount of focused thinking that can make the unpredictable predictable. That's why I want to learn to surf instead of building seawalls.
Ah, I see, I meant the whole company requires focus in product thinking. I wasn't sure what you mean by "outside the system". Yes, if you're a frontend dev and hopelessly exposed to product and business not thinking on product focus, your job is very miserable. Not sure if surfing helps.
When I hear "focus", I hear, "We have a plan. We just have to execute on it & not get distracted." Which is the opposite mindset to what I aspire to. "Plan the work, work the plan" is the seawall. Yes there's all this variation out there. We're going to ignore it. Surfing puts you out there right on the wave bringing in variation. The only thing predictable in that situation is the fact that you can cope.
Ah, I see. Focus for me is something else. It's more along the lines of what are we doing here exactly, why are we doing it, etc. In a more general way. Totally agree that software product development is unpredictable.
If I understand you correctly, you're talking about a metaphor that I've used a lot in recent years in my thinking, designing, and team leading: APIs are LEGO bricks, UIs are LEGO sets. A good set of fundamental bricks can be assembled into a nearly endless variety of constructions; conversely, that specialty brick that was designed to be a specific part for a helicopter or something, isn't very useful in a different set. So build the APIs in such a way as to be assembleable without being so simplistic as to be annoying (the single-stud brick).
This is a great explanation. Like the existence of API debounces the amount of work that makes more work trickle-down to backend.
I'll give my (intentionally provocative) take.
Backend engineers at big tech have a cornucopia of tough problems to solve at scale. However, their extremely smart front-end engineers will blow their brains out if they have to work on *yet another* A/B test or design update. The result? Inventing interesting problems to solve, which is how we got the framework explosion of the early 2010s.
p.s. I love the sketch, reminds me of a Feynman diagram in particle physics!
Great insights on front-end versus back-end development! If you're interested in exploring how these concepts intersect with IT audit and compliance, check out ThatAuditor.me for expert insights and valuable resources. See you there!
I love seeing the discussion of how front-end development mixes into the wider software design mix. Last year I posted about what frontend and backend even mean. There is little shared understanding leading to frequent misunderstanding https://www.industrialempathy.com/posts/frontend-backend/
In my post I call your usage of front-end uppercase-Frontend to disambiguate from lowercase-frontend which is just the part of any distributed system talking to its clients.
Some general topics that are interesting about frontend development:
- Skew boundaries involve more long-lived clients not under deployment control of the system author
- More-commonly stateless services making techniques like immutable deployments more feasible
- The general direction of the last 10 years in state-of-the-art frontend development has been to blur the client-server boundary and to eliminate separation of concerns: A move from MVC architecture to unidirectional dataflow (which now completely dominates web development). The implications of unidirectional dataflow for backend technologies remain underexplored.
Working on Screenshotbot, I have unique insights into this, and it's usually part of my pitch to potential customers. Unit tests fail relatively rarely (used as a proxy for how often the system behavior changes), UI changes very frequently (using screenshot tests as a proxy for frontend behavior changes).
For instance, most screenshot test changes we see on Screenshotbot are intentional changes. Very rarely are they actually "rejected" for being a bug. I could run the exact numbers.
(To complete the sales pitch I use, many companies use screenshot tests like unit tests, or what some people call acceptance tests. They store screenshots in the repo and the tests fail when screenshots change. But that introduces a lot of overhead for UI devs because UI changes so damn frequently, and UI devs have to keep re-recording screenshots manually. It's the wrong way to think about screenshot tests.)
I'd love to see those numbers, Arnold (Hi!)
Okay, so keep in my startup is real small. And the data isn't as clean as I hoped.
I've restricted the data to 90 days, to keep it clean.
Roughly speaking, here's how this data looks like: For every PR, the customer's CI generates images and uploads it to us, this creates a "run". If we notice any difference we create a "report" and notify them on GitHub/BitBucket etc, and they get to accept or reject the changes. Runs are also generated on main branches, but I'm skipping those since users can't accept/reject them.
In theory, there might be multiple runs per commit, because some companies will break up their project into smaller sub-projects, but in the data I picked I think it's rare. Also each version of the PR will have its own run (i.e. you update the PR and it generates a new run). I skipped companies that use a mono-repo because that behaves very differently (number of runs grows quadratically, and that might skew the data).
In the last 90 calendar days, there were 8148 runs generated attached to a PR. Of these 4221 generated a report (pretty high IMO, but my customers are skewed towards heavy UI changes, perhaps having Screenshotbot makes them more confident to make UI changes). Of these 1420 were accepted, and only 14 were rejected. 2787 had no action (probably because it was an intermediate state of the PR and they just chose to create new version, very hard to assume if it's either a bad or good change)
Can you sprinkle magic AI fairy dust on it to reduce the number of false positives? I'd be especially interested in combining the contents of the PR with the changes to the appearance to get even better accuracy.
I've dabbled with using AI to compare two screenshots, but it hasn't shown much success yet https://www.linkedin.com/posts/arnoldnoronha_i-built-an-ai-to-summarize-screenshot-changes-activity-7141116226285604864-6Zpc?utm_source=share&utm_medium=member_desktop
I think current available vision models aren't great at image comparison.
As for PR contents, my tool doesn't have access to customers' source code or PR contents.
I'm just imagining that a data source that's interesting 2% of the time would get annoying after a while. (Just judging from marketing emails.)
Ah, I see what you're getting at now.
But you're not thinking like a UI dev! You're thinking like a backend dev!
Say you change a core button, say from rounded to square border or some such. As a UI dev, you want to see how that change affects the rest of the app. You know what the change is, but you still want to visualize it (and communicate that visual change to code reviewers).
Conversely, if you are in a product team, and a core UI developer changes a button, you want to be notified that your screen has changed. You've spent so much time making it pixel perfect, and you want to be able to keep it that way.
So a "false positive" is still a very valuable notification for Screenshotbot. Of course, it would be nice if the AI can tell you if this might be something important or not.
Btw, these numbers are consistent with what I saw at Facebook. (I used to tell people that only about 2% of UI changes were actual bugs, and 98% were just UI developers making real changes.)
I see that in the User Interface, every time you give the users something, it provides inspiration for even more things you could do for them.
(I'm not blaming the users! I'm the same way! When I'm writing software for myself, I have to control the scope vigorously, as it tends to explode with great new ideas of even more things I could do!)
I don’t understand why people complain about this. If I’m mining and I come to the end of one vein of ore and it splits into 2 veins, that’s good news. I suppose if you’re looking for an answer to “when will it be finished”, you’re not happy with “there is no ‘it’ to be finished.”
I was working on a project and every time we released a new version to the business, they asked us to add even more features to the product. After a few releases, the project manager asked me in frustration, "When will this project ever be FINISHED?!?!?"
I said, "Never. Not until they stop using it, and hence don't care any more. As long as they're using the system, they will want changes and enhancements to it, to make it more valuable. And that's a good thing."
We often get hung up on the mental model of "doing a project" or "buying a thing" -- that it's a predictable transaction, rather than an ongoing, essentially "living" process of changing and improving processes over time.
In most cases, if it "ends," then you have failed.
"When will we be done paying lawyers?"
"When we're out of business."
“First thing we do, let’s kill all the lawyers.” – William Shakespeare
https://www.stoufferlegal.com/blog/did-shakespeare-really-mean-it-when-he-said-lets-kill-all-the-lawyers
>;->
I would think that, it is finished when the cost of changing outweighs the revenue it will potentially generate.
"dead", technically speaking.
Yes.
My observations were that before agile, I saw several consultancies that specialized in evaluating legacy systems to inform top management of the (objectively, mathematically) best/right time to rewrite them -- when the cost of maintenance exceeds the cost (and risk) of building a new system.
What I felt agile (and refactoring) brought to the table was that there was no longer any rational reason nor economic justification for old code/systems to "inevitably" become unmaintainable.
Not unless there was a *radical* change in your underlying business model that required "ripping up" and completely redesigning how your business *fundamentally* works. (A rare event, indeed!)
Super interesting article!
I'd like to reframe Vladimir's, very good, take like so:
API's are consumed by emotional monkeys you can tell to "RTFM", and some will. UI's, on the other hand, is consumed by emotional monkeys who will misinterpret every though you put into it, and the docs, and refuse to buy your service/product/stuff if told you tell them to.
So the requirements of UI's to "make sense" in order to create value/revenue somehow scales, by a factor, with every monkey interacting with it (thousandfold that of an API), increasing the pressure and stress to get "get it right" when working on it (at least for me).
/emotional monkey and frontend dev
Does anyone else think this phenomenon observed in the original tweet is at least somewhat due to the difference in maturity* between the BE and FE technologies? Comparing Java/Spring, .Net, Python - heck even Ruby/Rails or PHP - against the seemingly constantly re-born Javascript framework du jour, and it doesn't surprise me that there's more churn in FE work.
Similarly, the acceptance and near ubiquitous application of patterns (such as GoF patterns, Enterprise Architecture Pattens, etc) in BE work, seems to be longer-standing than in FE.
I'm not saying Kent's theory is wrong or even not the most weighted factor, just that churn in platform/language probably produces increased churn in output.
* Perhaps "stability" or "ripeness" would be better, less inflammatory word choices.
I rather think that the churn in front-end frameworks reflects, rather than causes, the kind of churn in requirements. That "spawning" effect doesn't seem to me to be related to framework at all. What am I missing?
I see backend developers as far more mature than frontend developers. In some of the larger organizations I've worked, the backend guys aren't developers at all, they're engineers...they understand the intricacies of the build.
I rarely find a front end developer that understands the mechanics of a single threaded language like JavaScript, and how to create something that spawns additional thread in the OS, something that is easily done in JS.
Modern FE developers are tool experts, not technology experts.
Check yourself. "Mature"? FE & BE engineers work under very different constraints, part of which I outlined in my post. Seems to me you're projecting, which we call out here when it happens.
And here I was worried about the inflammatory interpretation of the word "mature"... :-)
My thought was that there's an additional aspect to the idea that FE work spawns more FE work, in perpetuity. That aspect isn't about the forever-evolving, ever-expanding requirements (which is where's Kent's analysis went); rather it's about contributing to the fact that the UI isn't ever "done." It contributes by forcing more compromise between cost-effective implementation and at-the-time ideal product, since every FE-framework-du-jour adds distractions and often, IME, loses capabilities (or makes them more expensive).
By the way, I'm OK if the answer to my initial sentence, "Does anyone else think..." is "No."
Honestly, I often feel sorry for FE developers because, like you said, the jobs feels like it's never done, because there's always something new! There are so many new "things" being churned, and few of them are really new, just a "better", "more optimized", "[insert more useless words here]" way of doing the same thing. The FE has become way over-engineered (just look at state management).
As opposed to the backend: we have a fixed number of languages, sure some people use frameworks, but I feel they really understand what's going on with them.
And yes, I agree on the initial question :0)
Good user interfaces have to be more specific while backends can afford to be more generic.
It’s possible to reach moments of calm but it requires a lot more (?) focus in product thinking.
Notice that the sources of variation come from outside the system. There's no amount of focused thinking that can make the unpredictable predictable. That's why I want to learn to surf instead of building seawalls.
Ah, I see, I meant the whole company requires focus in product thinking. I wasn't sure what you mean by "outside the system". Yes, if you're a frontend dev and hopelessly exposed to product and business not thinking on product focus, your job is very miserable. Not sure if surfing helps.
What do you mean by surfing exactly?
Focus is not about making things predictable. It's about how well you're able to deal with the unpredictable.
When I hear "focus", I hear, "We have a plan. We just have to execute on it & not get distracted." Which is the opposite mindset to what I aspire to. "Plan the work, work the plan" is the seawall. Yes there's all this variation out there. We're going to ignore it. Surfing puts you out there right on the wave bringing in variation. The only thing predictable in that situation is the fact that you can cope.
Ah, I see. Focus for me is something else. It's more along the lines of what are we doing here exactly, why are we doing it, etc. In a more general way. Totally agree that software product development is unpredictable.
If I understand you correctly, you're talking about a metaphor that I've used a lot in recent years in my thinking, designing, and team leading: APIs are LEGO bricks, UIs are LEGO sets. A good set of fundamental bricks can be assembled into a nearly endless variety of constructions; conversely, that specialty brick that was designed to be a specific part for a helicopter or something, isn't very useful in a different set. So build the APIs in such a way as to be assembleable without being so simplistic as to be annoying (the single-stud brick).
Is that kinda what you're saying?
That's another way to look at it. I don't have a good explanation for *why* an API dampens the requirements "spawning".
This is a great explanation. Like the existence of API debounces the amount of work that makes more work trickle-down to backend.
I'll give my (intentionally provocative) take.
Backend engineers at big tech have a cornucopia of tough problems to solve at scale. However, their extremely smart front-end engineers will blow their brains out if they have to work on *yet another* A/B test or design update. The result? Inventing interesting problems to solve, which is how we got the framework explosion of the early 2010s.
p.s. I love the sketch, reminds me of a Feynman diagram in particle physics!
Great insights on front-end versus back-end development! If you're interested in exploring how these concepts intersect with IT audit and compliance, check out ThatAuditor.me for expert insights and valuable resources. See you there!