This has basically been said already above, but I’m impressed once again that, when doing “test after” like I’m doing this week, and when tackling a codebase that is not easily testable, I find myself writing long and complicated test fixture setups just to enable a single test with a single assert. The side effect is that all that setup covers a lot of code, but does **not** test it. This is a little scary to me, that even if someone with good intentions tries to retroactively add tests to legacy code like I am doing, it’s almost unavoidable that the resulting coverage stats will be skewed due to the aforementioned complexity in test fixture setup. As Julio stated, and I heartily agree, with TDD in theory you can have much more confidence in raw coverage statistics, since every line of production code should be backed by at least one test. But if you find yourself in a situation like I’m in I’m not sure what can be done other than to be very skeptical of test coverage stats at all. And, as Kent Beck says, to write the tests you wish you had.

re: code generators, in response to Eric and Alex, I hadn’t really thought about it before, but I would be in favor of generating tests for simple getters/setters if for no other reason than they would be great, obvious placesetters for future tests if and when (as Alex said) the getters or setters had to change to have more complex behavior. I don’t feel any particular need to have n-percent code coverage, but I am in favor of doing anything that eases other developers’ future enhancement/maintenance of a codebase I work on.

I think the main point is that simply calling a method (coverage) does not verify anything about whether it did the right thing. And there are lots of ways for indirect coverage (from testing another method) to execute a method without actually testing it. This can give you a false sense of security that coverage == testing. In the case of test-first, you’re right that theoretically every method should have an associated test. However, the overwhelming majority of developers do not work test-first.

The comments were made in the context of testing “simple” getters and setters, and in that case I think I could make an argument that writing a bunch of tests like:

public void testGetName() {
  Person p = new Person();
  p.setName("SpongeBob");
  assertEquals("SpongeBob", p.getName());
}

for auto-generated get/set methods is testing for the sake of coverage. Now, if that setter is doing a bunch of name validation or the getter is actually fronting a cache or talking to a database or constructing the name from multiple fields or something, then by all means test it.

My experience has been that 75-80% coverage is where the rate of return falls off considerably. That is to say, coverage beyond that point is valuable, but is not valuable enough to justify the effort. Note, that that target is an average. For really important or critical or complex stuff, I would strive for something much higher. And for less important code I would shoot for something lower.

I must admit I’m guilty of being a 100%-coverage advocate, and trying hard to get there (98% on my current project). In part because I know I make mistakes and in part because 90% of my nastiest bugs are in the un-covered few percent.

More importantly to your point, and why it’s a strawman argument, is that I have never seen anyone writing tests that don’t test anything (or the wrong thing) just for he sake of coverage. No one is writing “assert_equal 3, calc.add(1,1)” just so calc gets coverage. So yes, in practice coverage == testing.