Test-Driven Bugfixing
18 May 2016
Test-Driven Development (TDD) tells us to write the tests first and only then develop the code. It may seem like a good idea. Like a way how to force lazy developers to write tests. How to make sure that the code is good and does what it should do. But there's the problem. If you are doing something new, something innovative, how the hell are you supposed to know what the code should do?
If you are doing something new you probably do not know what will be the final result. You are experimenting, improving the code, changing the specification all the time. If you try to use TDD for that you are going to fail miserably. You will have no idea how to write the tests. And if you manage to write it somehow you will change them every time. This is a wasted effort. A lot of wasted effort. But we need the tests, don't we? And there is no known force in the world that will make the developer to write good and complete tests for the implementation once the implementation is finished. Or ... is there?
What are we using in midPoint project is Test-Driven Bugfixing (TDB). It works like this:
- You find a bug.
- You write an (automated) test that replicates the bug.
- You run the test and you check that the test is failing as expected.
- You fix the bug.
- You run the test and you check that the test is passing.
That's it. The test remains in the test suite to avoid future regressions. It is a very simple method, but a very efficient one. The crucial part is writing the test before you try to fix the bug. Even if the bugfix is one-liner and the test takes 100 lines to write. Always write the test first and see that it fails. If you do not see this test failure how can you be sure that the tests replicates the bug?
We are following this method for more than 5 years. It works like a charm. The number of tests is increasing and we currently have several times more tests that our nearest competition. Also the subjective quality of the product is steadily increasing. And the effort to create and maintain the tests is more than acceptable. That is one of the things that make midPoint great.
(Reposted from Evolveum blog)