When we work with automated tests in our project, a common and controversial metric is the code coverage. Code coverage is a metric that evaluates the percentage of the lines or the decisions graph that has been evaluated during the test execution.
I say controversial because its promiscuous use generally drops the tests quality, after all we are what we measure, right? But it is a useful measure, because although good coverage don't mean good tests, bad coverage mean bad tests.
I was thinking in a different approach to the subject, test coverage measure the number of lines affected by a test execution, what if we could invert this operation and measure the number of tests affected by a line of code? If we could do that, we could give a grater meaning to this measure, in a sense that it not only tests the quality of your production code but also the quality of your tests.
Why? because imagine if we have a line that when we remove it completely, not only the project builds but it passes all tests? this could lead to two conclusions. If the tests are really testing what matters this results will show that this line of code doesn't matter, it could be a leftover of a refactoring, or simply dirty code. But if the line is correct then the tests are not testing the operation of this line of code, showing that although the line is covered, the result of its existence is not tested, which is a lot more powerful that the simple coverage. I came across this idea because, sometimes when using TDD I tend to be a bit compulsive in doing exactly this, after making the test pass, I some times comment the line that caused the green light just to see that the test fails again.
Okay, but even if it is a good idea, it is a bit hard to implement this, right? if we have a project with let's say 10,000 logic lines of code, which may correspond to a medium sized project, and if our project builds a runs all tests in a good machine in 30 seconds (highly optimistic!) to remove each line, build and run would take more than 3 days! That's a lot! but lines selected to be changed could be weighted by its age in the project or some other criteria, or it could be a IDE integrated tool such as JUnit Max.
