Testing on the Toilet: Prefer Testing Public APIs Over Implementation-Detail Classes
Wednesday, January 14, 2015
by Andrew Trenk
This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.
Does this class need to have tests?
A public API can be called by any number of users, who can pass in any possible combination of inputs to its methods. You want to make sure these are well-tested, which ensures users won’t see issues when they use the API. Examples of public APIs include classes that are used in a different part of a codebase (e.g., a server-side class that’s used by the client-side) and common utility classes that are used throughout a codebase.
An implementation-detail class exists only to support public APIs and is called by a very limited number of users (often only one). These classes can sometimes be tested indirectly by testing the public APIs that use them.
Testing implementation-detail classes is still useful in many cases, such as if the class is complex or if the tests would be difficult to write for the public API class. When you do test them, they often don’t need to be tested in as much depth as a public API, since some inputs may never be passed into their methods (in the above code sample, if UserInfoService ensured that UserInfo were never null, then it wouldn’t be useful to test what happens when null is passed as an argument to UserInfoValidator.validate, since it would never happen).
Implementation-detail classes can sometimes be thought of as private methods that happen to be in a separate class, since you typically don’t want to test private methods directly either. You should also try to restrict the visibility of implementation-detail classes, such as by making them package-private in Java.
Testing implementation-detail classes too often leads to a couple problems:
- Code is harder to maintain since you need to update tests more often, such as when changing a method signature of an implementation-detail class or even when doing a refactoring. If testing is done only through public APIs, these changes wouldn’t affect the tests at all.
- If you test a behavior only through an implementation-detail class, you may get false confidence in your code, since the same code path may not work properly when exercised through the public API. You also have to be more careful when refactoring, since it can be harder to ensure that all the behavior of the public API will be preserved if not all paths are tested through the public API.
This article was adapted from a Google Testing on the Toilet (TotT) episode. You can download a printer-friendly version of this TotT episode and post it in your office.
Does this class need to have tests?
class UserInfoValidator { public void validate(UserInfo info) { if (info.getDateOfBirth().isInFuture()) { throw new ValidationException()); } } }Its method has some logic, so it may be good idea to test it. But what if its only user looks like this?
public class UserInfoService { private UserInfoValidator validator; public void save(UserInfo info) { validator.validate(info); // Throw an exception if the value is invalid. writeToDatabase(info); } }The answer is: it probably doesn’t need tests, since all paths can be tested through UserInfoService. The key distinction is that the class is an implementation detail, not a public API.
A public API can be called by any number of users, who can pass in any possible combination of inputs to its methods. You want to make sure these are well-tested, which ensures users won’t see issues when they use the API. Examples of public APIs include classes that are used in a different part of a codebase (e.g., a server-side class that’s used by the client-side) and common utility classes that are used throughout a codebase.
An implementation-detail class exists only to support public APIs and is called by a very limited number of users (often only one). These classes can sometimes be tested indirectly by testing the public APIs that use them.
Testing implementation-detail classes is still useful in many cases, such as if the class is complex or if the tests would be difficult to write for the public API class. When you do test them, they often don’t need to be tested in as much depth as a public API, since some inputs may never be passed into their methods (in the above code sample, if UserInfoService ensured that UserInfo were never null, then it wouldn’t be useful to test what happens when null is passed as an argument to UserInfoValidator.validate, since it would never happen).
Implementation-detail classes can sometimes be thought of as private methods that happen to be in a separate class, since you typically don’t want to test private methods directly either. You should also try to restrict the visibility of implementation-detail classes, such as by making them package-private in Java.
Testing implementation-detail classes too often leads to a couple problems:
- Code is harder to maintain since you need to update tests more often, such as when changing a method signature of an implementation-detail class or even when doing a refactoring. If testing is done only through public APIs, these changes wouldn’t affect the tests at all.
- If you test a behavior only through an implementation-detail class, you may get false confidence in your code, since the same code path may not work properly when exercised through the public API. You also have to be more careful when refactoring, since it can be harder to ensure that all the behavior of the public API will be preserved if not all paths are tested through the public API.
Isn't this counterintuitive to the testing pyramid as it encourages larger tests? I'm sort of playing devils advocate here since I agree that it's often better to test to the API.
ReplyDeleteThe important thing is that you're getting the proper coverage from your tests. Often this might mean you need to pull in additional classes that your code uses in order to properly test your class.
DeleteI don't think this advice necessarily conflicts with the testing pyramid idea of focusing on smaller tests since you can still test small pieces of code this way. But having them exercise several classes instead of one class shouldn't be a problem.
I disagree with the conclusion of the article.
ReplyDeleteUserInfoValidator requires unit test (userInfo should be mocked), simple code requires only simple tests.
UserInfoService#save also requires unit testing, mock userInfo and the validator, also write an integration test for UserInfoService#save.
Testing indirectly causes bad defect localization and stronger coupling between tests and seemingly unrelated classes, thus making the tests less useful by not being able to pinpoint the defect line of code as a result of a broken test and also the test is more fragile.. changes in 3 classes could cause an non-isolated test to fail that only tests UserInfoService directly and the UserInfoValidator and UserInfo classes indirectly.
Mocks are good for some specific cases but they shouldn't be used as a general tool to remove dependencies on every single class you happen to be using. These two posts have some more details about this:
Deletehttp://googletesting.blogspot.com/2013/05/testing-on-toilet-dont-overuse-mocks.html
http://googletesting.blogspot.com/2015/01/testing-on-toilet-change-detector-tests.html
Defect localization isn't an issue here as long as you're still testing each public API. If an implementation detail class is complex enough that you wouldn't be able to figure out the problem from testing the public API then the implementation detail class should have tests too.
A public method is a public API irrespective of whether it is used by one other method or not. The whole discussion over implementation detail made no sense.
ReplyDeleteA public method should be tested irrespective of how it is implemented. The mistake the author is makes is that he/she has looked at the source code of the method and gone:
• “[oh it’s using some implementation method, I can’t be bothered testing it]”
• “[oh it’s only being used by one other method, I will adjust my testing accordingly]”
A class to be tested should be treated as a black box; make no assumptions over it; and just test it as potentially toxic. You should never look at the source code for something you are intending to test.
The only time you would do the latter if you are attempting to break the code to uncover a weakness or security flaw.
You're right that a class should be tested as a black box. But not all types of classes are equal. Just because a developer decides to put some logic in its own class doesn't automatically mean the logic needs to be tested in depth.
ReplyDeleteThe decision on how much testing a class needs isn't based on its usage, not its implementation: if a class is only used in one place and no one else can access it then it's not a public API. So this has nothing to do with its implementation.