Regarding "when I noticed that some people fixate on a single kind of test, a single point in that space, and claim that those are the only valuable tests"
hope it's clear that I wasn't suggesting that only one kind of test has value. The FIRST qualities of microtests (we didn't have the term back then) suggested that these are the ones best for TDD, because you can afford them in the tight, inner loop of coding.
Other tests have value, but slower tests are unlikely to be run after each minor refactoring step, like a variable rename.
Inspired by your tweet, I define a unit test as "a unit test is a small test." It is not about the size of the code under test. Small means fewer lines of test code, specifically the first A of the AAA pattern. If the tests can differentiate from each other by just a few lines of code, they are unit tests to me. I usually can achieve everything in your list, even if my test exercises thousands of lines of code, using a database, and there are hundreds of them. My e2e tests using multiple services can achieve almost everything on your list, except they are a bit slow. You might not want to run all of them 2 or 3 times for no reason, but they are bearable.
In summary, I don't do unit testing. My units (whatever that means) do not have their tests. I do unit tests, which are small tests decoupled from the code structure.
Would you consider tests that touch database and/or the file system isolated if they dont affect other tests? I dont have much experience programming professionally but there have been some instances where I found out I wasn't really testing anything by verifying the mocks were called with the right arguments. The "behvaior" I wanted to test was does this record gets inserted into the database given the username is not taken.
What about attributes like performance, security, etc.? Wouldn't they need to be kept in mind while writing the tests? In which dimension are they included in?
Great list. Glad it started from isolation. I've seen too many code bases treating tests as grand architecture of reusable crap that no one can reason about.
Regarding "when I noticed that some people fixate on a single kind of test, a single point in that space, and claim that those are the only valuable tests"
hope it's clear that I wasn't suggesting that only one kind of test has value. The FIRST qualities of microtests (we didn't have the term back then) suggested that these are the ones best for TDD, because you can afford them in the tight, inner loop of coding.
Other tests have value, but slower tests are unlikely to be run after each minor refactoring step, like a variable rename.
Inspired by your tweet, I define a unit test as "a unit test is a small test." It is not about the size of the code under test. Small means fewer lines of test code, specifically the first A of the AAA pattern. If the tests can differentiate from each other by just a few lines of code, they are unit tests to me. I usually can achieve everything in your list, even if my test exercises thousands of lines of code, using a database, and there are hundreds of them. My e2e tests using multiple services can achieve almost everything on your list, except they are a bit slow. You might not want to run all of them 2 or 3 times for no reason, but they are bearable.
In summary, I don't do unit testing. My units (whatever that means) do not have their tests. I do unit tests, which are small tests decoupled from the code structure.
Would you consider tests that touch database and/or the file system isolated if they dont affect other tests? I dont have much experience programming professionally but there have been some instances where I found out I wasn't really testing anything by verifying the mocks were called with the right arguments. The "behvaior" I wanted to test was does this record gets inserted into the database given the username is not taken.
What about attributes like performance, security, etc.? Wouldn't they need to be kept in mind while writing the tests? In which dimension are they included in?
Great list. Glad it started from isolation. I've seen too many code bases treating tests as grand architecture of reusable crap that no one can reason about.