“When you have a lot of tests; What do you do when you want to change something, and the tests get in your way?”
I hear this question - in one form or another - every now and then. I hear it during the trainings I teach, when coaching or when I am working on some code with a team and try to convince them to write more automated tests.
It is one of the objections I often hear against doing test driven development (TDD): “When we have more tests, changing things later will become harder”.
This Should not Happen
In an ideal world, you would:
- only add new tests when adding functionality - and not change any existing tests
- only change related tests when changing functionality
- not change any tests when refactoring - they will always stay green
At least this is the world we strive for when automating tests: A world where
- Tests will provide a safety net
- Tests will protect us against regressions -and-
- Tests will not get in our way otherwise
This Will Happen
But most of us, most of the time, do not live in that world.
Sometimes you have tests that always get in your way. Like a test that is unstable: It is green most of the time, but sometimes, only on the build server, it breaks. Such tests are unconditionally bad.
Sometimes, a test will become “Red” even though the code change did not change relevant functionality. Sometimes you will have to change tests when adding functionality. Sometimes changing functionality becomes hard because of the tests involved.
When that happens, first look at the type of test.
Business-Facing Tests
There is one kind of test - Business-facing tests that verify that the applications works correctly from a user’s point of view - that should never during refactorings. We only expect to change them when we change functionality. But there, they might get in our way, especially if you have a lot of them. They might get in your way because the changes to the tests are too hard.
If such tests get in your way when changing code, try to find the root cause. Do you have many overlapping tests? Do you have tests that are not focussed, that are testing too much at once? Is there another reason?
When you find the root cause, see if you can refactor your test or replace them with a suite of better ones.
When such a test breaks during a refactoring, you have found a bad test. Try to find the root cause why the test was bad and remember to avoid it in the future. For now, try to make the test better, or delete it if you cannot do that.
Technology-Facing Tests
The other kind of test - Technology-facing tests that verify that our code works correctly from an internal point of view - might require some changes during a refactoring.
If you have contract tests that verify that a component uses an interface correctly, they might break when you inline some of the functionality that was in the interface before.
If you split a large class into two smaller ones, it might also be necessary to move the tests around or to delete the bigger tests and write new ones for the smaller classes.
On the other hand, those tests might survive changes to the functionality - But only in rare cases.
About Deleting Tests
“Did he really just write delete tests?” – Are you thinking this right now?
Yes. Delete the test. Especially if you have a bad test. Even if you do not have a replacement right now (but do strive to find or write a replacement).
“But then this piece of functionality is not tested anymore” - OK, but a bad test is often worse than having no test at all.
“But we already invested so much time in writing and maintaining this test” - That’s the Sunk Cost Fallacy - Don’t fall for it!
Also, some tests were not always bad, not always worth deleting. Some tests have just out-lived their usefulness. When a test was useful in the past, it was already worth the investment. When it now is not useful anymore, delete it without looking back.
Test Driven Development vs. Test First vs. Test After
So, here’s the problem with usefulness: When you write the tests after the fact, and even sometimes when you write the “tests first”, you are writing tests to protect you against regressions. You want your tests to notify you when your code has changed.
The usefulness of your tests comes from protecting you against invalid code changes. Having to delete such a test because of a valid code change almost proves that the test was not useful and not worth the investment.
When you do TDD, things are different. You write the tests to drive the implementation and design of your code. You write the tests to help you think and to know when to stop.
In the very moment when the test becomes green, you already have proof that the test was useful. Protection against regressions comes as a side effect. When you have to delete such a test later, it’s no big deal: The test was useful once, but has now out-lived its purpose.
To Recap...
When a test gets in your way, try to find out why. Write the root cause down in your engineering notebook or wherever, and discuss it with your team. Consider the type of test to know how big of a “problem” you have.
Try to find or write a replacement for the test, and try to make it better. Try to learn something and write better tests in the future.
And when you find a bad test or a test that has out-lived its usefulness, delete it. But do try to write or find a replacement.
