Testing on the Toilet: Writing Descriptive Test Names
Thursday, October 16, 2014
by Andrew Trenk
This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.
How long does it take you to figure out what behavior is being tested in the following code?
You probably had to read through every line of code (maybe more than once) and understand what each line is doing. But how long would it take you to figure out what behavior is being tested if the test had this name?
isUserLockedOut_lockOutUserAfterThreeInvalidLoginAttempts
You should now be able to understand what behavior is being tested by reading just the test name, and you don’t even need to read through the test body. The test name in the above code sample hints at the scenario being tested (“invalidLogin”), but it doesn’t actually say what the expected outcome is supposed to be, so you had to read through the code to figure it out.
Putting both the scenario and the expected outcome in the test name has several other benefits:
- If you want to know all the possible behaviors a class has, all you need to do is read through the test names in its test class, compared to spending minutes or hours digging through the test code or even the class itself trying to figure out its behavior. This can also be useful during code reviews since you can quickly tell if the tests cover all expected cases.
- By giving tests more explicit names, it forces you to split up testing different behaviors into separate tests. Otherwise you may be tempted to dump assertions for different behaviors into one test, which over time can lead to tests that keep growing and become difficult to understand and maintain.
- The exact behavior being tested might not always be clear from the test code. If the test name isn’t explicit about this, sometimes you might have to guess what the test is actually testing.
- You can easily tell if some functionality isn’t being tested. If you don’t see a test name that describes the behavior you’re looking for, then you know the test doesn’t exist.
- When a test fails, you can immediately see what functionality is broken without looking at the test’s source code.
There are several common patterns for structuring the name of a test (one example is to name tests like an English sentence with “should” in the name, e.g., shouldLockOutUserAfterThreeInvalidLoginAttempts). Whichever pattern you use, the same advice still applies: Make sure test names contain both the scenario being tested and the expected outcome.
Sometimes just specifying the name of the method under test may be enough, especially if the method is simple and has only a single behavior that is obvious from its name.
This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.
How long does it take you to figure out what behavior is being tested in the following code?
@Test public void isUserLockedOut_invalidLogin() { authenticator.authenticate(username, invalidPassword); assertFalse(authenticator.isUserLockedOut(username)); authenticator.authenticate(username, invalidPassword); assertFalse(authenticator.isUserLockedOut(username)); authenticator.authenticate(username, invalidPassword); assertTrue(authenticator.isUserLockedOut(username)); }
You probably had to read through every line of code (maybe more than once) and understand what each line is doing. But how long would it take you to figure out what behavior is being tested if the test had this name?
isUserLockedOut_lockOutUserAfterThreeInvalidLoginAttempts
You should now be able to understand what behavior is being tested by reading just the test name, and you don’t even need to read through the test body. The test name in the above code sample hints at the scenario being tested (“invalidLogin”), but it doesn’t actually say what the expected outcome is supposed to be, so you had to read through the code to figure it out.
Putting both the scenario and the expected outcome in the test name has several other benefits:
- If you want to know all the possible behaviors a class has, all you need to do is read through the test names in its test class, compared to spending minutes or hours digging through the test code or even the class itself trying to figure out its behavior. This can also be useful during code reviews since you can quickly tell if the tests cover all expected cases.
- By giving tests more explicit names, it forces you to split up testing different behaviors into separate tests. Otherwise you may be tempted to dump assertions for different behaviors into one test, which over time can lead to tests that keep growing and become difficult to understand and maintain.
- The exact behavior being tested might not always be clear from the test code. If the test name isn’t explicit about this, sometimes you might have to guess what the test is actually testing.
- You can easily tell if some functionality isn’t being tested. If you don’t see a test name that describes the behavior you’re looking for, then you know the test doesn’t exist.
- When a test fails, you can immediately see what functionality is broken without looking at the test’s source code.
There are several common patterns for structuring the name of a test (one example is to name tests like an English sentence with “should” in the name, e.g., shouldLockOutUserAfterThreeInvalidLoginAttempts). Whichever pattern you use, the same advice still applies: Make sure test names contain both the scenario being tested and the expected outcome.
Sometimes just specifying the name of the method under test may be enough, especially if the method is simple and has only a single behavior that is obvious from its name.
Good one, certainly it helps to trace out the left over requirements, moreover if Priority of test cases is also added e.g., "descriptive_name_Priority01", it gives the added advantage in identifying the test cases to be run on basis of priority whenever required.
ReplyDeleteVery well-written article. I personally use a unit test naming pattern along the lines of:
ReplyDeletepublic void [FUT]_When[SomeContextOrScenario]_Returns[(or Updates..., or Affects..., etc.)SomeValueOrResult]()
{
/* ... */
}
Please note:
- This example is in C#, with bracketed "parameters" and alternatives in parentheses.
- In this example, FUT is the "function under test" (the overall test fixture, or "[TestClass]" will correspond/relate to the SUT, or "system under test"). I've found this part of the convention to be extremely useful, as it allows me to select a sub-section of tests, as applicable to a particular function, by simply using [Shift + left-click]. This is possible because many "Test Explorer" implementations alphabetize unit tests.
- People are often upset initially by the underscores, and sometimes resist their inclusion, but those characters help me identify the segregated responsibilities of the test name (what requirements the name must satisfy, as discussed in this article). Once, an open-minded lead developer asked me to remove the underscores (on a "for now..." basis), and then come back and actually SUGGESTED underscores, but under strict guidelines, for readability. :-) (That's how I feel more lead developers should approach conflicting ideas from their teammates. He even credited me when formalizing his suggestion. Thanks, Jeff!)
Thanks for this well-presented information and I look forward to future installments of TotT. I apologize if anything in this post is "superfluous" or "noise". I get excited about large companies embracing topics dear to me.
Sincerely,
Mike Manard
Name of test only, since formatting seems a bit off above...
ReplyDelete[FUT]_When[SomeContextOrScenario]_Returns[(or Updates..., or Affects..., etc.)SomeValueOrResult]()
Thanks for sharing.
ReplyDelete