Desirable Unit Tests
A Test Desiderata Perspective
The Test Desiderata collect desirable properties for tests. You can think of them as sliders—more of this, sometimes less of that. Alternatively, they form a 12-dimensional space of all possible tests.
I wrote the Test Desiderata when I noticed that some people fixate on a single kind of test, a single point in that space, and claim that those are the only valuable tests. We aren’t done exploring all of what testing can do for us as programmers or for those whose lives we affect with our programs.
Tim Ottinger recently tweeted about properties of unit tests, in particular the FIRST principles he compiled with Jeff Langr. They looked at good unit tests and described their properties. I realized I could just the opposite—look at the properties and use “settings” on each property to describe the attractor we call “unit tests”.
Here they are:
Isolated — Unit tests are completely isolated from each other, creating their own test fixtures from scratch each time. (Note that I’m not saying these are the only useful tests, just that if tests aren’t isolated you’re going to have a hard time making the case that they are “unit tests”.)
Composable — Follows from isolation.
Deterministic — Should be. Code that uses a clock or random numbers should be passed those values by their unit tests.
Specific — If a unit test fails, the code of interest should be short.
Behavioral — If the behavior changes accidentally, a unit test should fail.
Structure-insensitive — This can be a challenge for unit tests. Too much mocking, especially strict mocking, is a structure sensitivity nightmare.
Fast — Yep.
Writable — Good interface design makes writing unit tests easier. Alternatively, difficult-to-write unit tests are the canary in the bad interface coal mine.
Readable — It can be challenge to write readable unit tests, because you are seeing so little context compared to the whole system.
Automated — Yep.
Predictive — Unit tests passing likely gives little confidence that the whole system is working. Unit tests failing should give great confidence that the whole system is not working.
Inspiring — A frequently-run unit test suite gives great confidence the programming is progressing. Sometimes I run my unit tests 2 or 3 times, just because it feels good (and they’re wicked fast).