TotT: The Invisible Branch
Full statement coverage may be necessary for good testing coverage, but it isn't sufficient. Two places where statement coverage will be inadequate are branches and loops. In this episode, we'll look at branches, and specifically the differences between statement coverage and branch coverage.
Let's consider a case where branch coverage and statement coverage aren't the same. Suppose we test the following snippet. We can get complete statement coverage with a single test by using a berserk EvilOverLord:
bool DeathRay::ShouldFire(EvilOverLord& o, Target& t) {
double accumulated_rage = 0.0;
if (o.IsBerserk())
accumulated_rage += kEvilOverlordBerserkRage;
accumulated_rage += o.RageFeltTowards(t);
return (accumulated_rage > kDeathRayRageThreshold);
}
But what if DeathRay should fire at this Target even with a non-berserk Overlord? Well, we need another test for that. What should the test be? Let's rewrite the code a little bit. We would never see code like this in the real world, but it'll help us clarify an important point.
bool DeathRay::ShouldFire(EvilOverLord& o, Target& t) {
double accumulated_rage = 0.0;
if (o.IsBerserk()) {
accumulated_rage += kEvilOverlordBerserkRage;
} else {
}
accumulated_rage += o.RageFeltTowards(t);
return (accumulated_rage > kDeathRayRageThreshold);
}
Why do we add an else clause if it doesn't actually do anything? If you were to draw a flowchart of both snippets (left as an exercise – and we recommend against using the paper provided), the flowcharts would be identical. The fact that the else isn't there in the first snippet is simply a convenience for us as coders – we generally don't want to write code to do nothing special – but the branch still exists... put another way, every if has an else. Some of them just happen to be invisible.
When you're testing, then, it isn't enough to cover all the statements – you should cover all the the edges in the control flow graph – which can be even more complicated with loops and nested ifs. In fact, part of the art of large-scale white-box testing is finding the minimum number of tests to cover the maximum number of paths. So the lesson here is, just because you can't see a branch doesn't mean it isn't there – or that you shouldn't test it.
Hey Dave,
ReplyDeleteYour urinating friends might be interested in a handy tool for measuring coverage. In Java land Emma, Corbutura and Clover are by far the most common coverage tools.
Clover is the only tool that gives you per test coverage. That is, it tells you which test hit which class/method/statement/conditional. So you can see that the right test covered your if, rather than just any test. Check out the covererage report for Google-Guice.
Apologies for the blatant plug boys. now focus on your aim.
Cheers,
Pete.
Check out mutation testing, it's a technique for testing the quality of your test suite. The principle is simple:
ReplyDeletea) in the current state, all tests pass
b) make an arbitrary modification to the code under test that breaks functionality
c) if your tests still pass, they are lacking
This comment has been removed by the author.
ReplyDelete