Additional Testing After Refactoring

Should you write more tests after refactoring? Following up

Jeff Grigg

’s observation:

Yes and no. Yes, in the sense that I’m about to describe. No, in the sense that you should have the same level of confidence in your tests (the “predictive” property from https://www.testdesiderata.com) as you had before because of course you’ve been careful not to change any behavior.

But you probably have an opportunity to improve your tests in some cases.

---

Thanks to today’s sponsor, me!

My friend Flee dubbed me the Rick Rubin of software development, a moniker I wear proudly. Individuals & teams seem to do better around me, even if you can’t see how I’m having that effect. Consider engaging me for an internal talk or for high-stakes consulting. Difficult situations preferred.

Also, this space is for rent. If you want to get your message out to a hundred thousand of the best geeks on the planet, contact me about sponsorships.

---

Differentiation

One common form of design change is to take a big thing & break it into parts. From Christopher Alexander’s The Timeless Way of Building:

Within this process, every individual act of building is a process in which space gets differentiated. It is not a process of addition, in which preformed parts are combined to create a whole, but a process of unfolding, like the evolution of an embryo, in which the whole precedes the parts, and actually gives birth to them, by splitting.

Say we’ve done this. We had a big element. We’ve split it into parts. The tests remain. We are just as confident of the behavior as we’ve ever been. What happens next is what I think Jeff is getting at.

Unit-izing Tests

Now that we have these smaller sub-elements, presumably with narrower APIs, we can, if we so choose, write smaller tests for the sub-elements:

This is particularly important if:

• We want to use those sub-elements elsewhere. Extracted sub-elements tend to be reusable through some mysterious process of the universe I do not yet understand, only observe.

• We want to change those sub-elements. We need the caller’s view of the sub-element to be fixed.

This gives us a question about the timing of writing those smaller-scale tests. Do we do it, as Jeff seems to be suggesting, as part of the refactoring process? Or do we write the tests as a “tidy first”, when we’re about to reuse or modify them?

Redundancy

Some folks would say not to bother writing the sub-element tests. I disagree. I like the pressure that sub-element level tests put on me to clarify & improve the design.

Once we’ve written those tests, we are redundantly double checking the behavior of the system. Here’s where we have another genuine decision to make. Some people would say that we should never remove the element-level tests because OMG we don’t want to lose any coverage. Again, I disagree.

I want the fewest possible system-level tests to give me the desired confidence (again, that “predictive” property). System-level tests tend to be slow to run, which incentivizes me to work in larger steps. I don’t need any more incentive to lengthen my feedback loops.

The Dilemma

This topic produced more discussion than I expected, and more polarized:

• Of course you delete redundant tests

• Of course you never delete tests.

Reduced to it’s simplest form, imagine we have 2 system-level tests, S1 & S2, & one component-level test C1. We conclude through analysis that if C1 fails then S1 fails because that part of the system calls the component. Also if S2 fails then S1 fails because the system is wired together wrong. If we assume C1 & S2 pass, then S1 is completely redundant. Its pass or failure creates no new feedback for us.

In this perfect scenario, what do you do with S1? Leave it around consuming resources (maintenance, power consumption, delay seeing test results)? Or delete it? That’s the question.

A more concrete example one commenter brought up. Suppose we have a bunch of system-level tests checking for a bunch of error conditions. Now we extract components & add tests ensuring that the components throw exceptions under those same conditions. I would tend to leave one of the system-level tests around to ensure that if an exception is thrown then it is handled correctly, but I’d delete the other, now redundant, system-level tests.

Conclusion

So, yes & no, Jeff. Yes, you have an opportunity to write new tests after refactoring. No, the timing is up for grabs. Do you write the tests as a tidy after or a tidy first for your next change?

Upon re-reading Jeff’s comment I missed his point. The dilemma posed here is still valid. His use of “refactoring” suggests he may have been speaking ironically. Usually my reflexes are fast enough to catch irony as it goes whizzing over my head. Not always.