Wednesday, July 15, 2009

The Advantages of Unit Testing Early

by Shyam Seshadri

Nowadays, when I talk with (read: rant at) anyone about why they should do test driven development or write unit tests, my spiel has gotten extremely similar and redundant to the point that I don't have to think about it anymore. But even when I do pairing with skeptics, even as I cajole and coax testable code or some specific refactorings out of them, I wonder, why is it that I have to convince you of the worth of testing ? Shouldn't it be obvious ?

And sadly, it isn't. Not to many people. To many people, I come advocating the rise of the devil itself. To others, it is this redundant, totally useless thing that is covered by the manual testers anyway. The general opinion seems to be, "I'm a software engineer. It is my job to write software. Nowhere in the job description does it say that I have to write these unit tests." Well, to be fair, I haven't heard that too many times, but they might as well be thinking it, given their investment in writing unit tests. And last time I checked, an engineer's role is to deliver a working software. How do you even prove that your software works without having some unit tests to back you up ? Do you pull it up and go through it step by step, and start cursing when it breaks ? Because without unit tests, the odds are that it will.

But writing unit tests as you develop isn't just to prove that your code works (though that is a great portion of it). There are so many more benefits to writing unit tests. Lets talk in depth about a few of these below.

Instantaneous Gratification

The biggest and most obvious reason for writing unit tests (either as you go along, or before you even write code) is instantaneous gratification. When I write code (write, not spike. That is a whole different ball game that I won't get into now), I love to know that it works and does what it should do. If you are writing a smaller component of a bigger app (especially one that isn't complete yet), how are you even supposed to know if what you just painstakingly wrote even works or not ? Even the best engineers make mistakes.

Whereas with unit tests, I can write my code. Then just hit my shortcut keys to run my tests, and voila, within a second or two, I have the results, telling me that everything passed (in the ideal case) or what failed and at which line, so I know exactly what I need to work on. It just gives you a safety net to fall back on, so you don't have to remember all the ways it is supposed to work in. Something tells you if it is or not.

Also, doing Test Driven Development when developing is one of the best ways to keep track of what you are working on. I have times when I am churning out code and tests, one after the other, before I need to take a break. The concept of TDD is that I write a failing test, and then I write just enough code to pass that test. So when I take a break, I make it a point to leave at a failing test, so that when I come back, I can jump right back into writing the code to get it to pass. I don't have to spend 15 - 20 minutes reading through the code to figure out where I left off. My asserts usually tell me exactly what I need to do.

Imposing Modularity / Reusability

The very first rule of reusable code is that you have to be able to instantiate an instance of the class before you can use it. And guess what ? With unit tests, you almost always have to instantiate an instance of the class under test. Therefore, writing a unit test is always a first great step in making code reusable. And the minute you start writing unit tests, most likely, you will start running into the common pain points of not having injectable dependencies (Unless of course, you are one of the converts, in which case, good for you!).

Which brings me to the next point. Once you start having to jump through fiery hoops to set up your class just right to test it, you will start to realize when a class is getting bloated, or when a certain component belongs in its own class. For instance, why test the House when what you really want to test is the Kitchen it contains. So if the Kitchen class was initially part of the House, when you start writing unit tests, it becomes obvious enough that it belongs separately. Before long, you have modular classes which are small and self contained and can be tested independently without effort. And it definitely helps keep the code base cleaner and more comprehensible.

Refactoring Safety Net

Any project, no matter what you do, usually ends up at a juncture where the requirements change on you. And you are left with the option of refactoring your codebase to add / change it, or rewrite from scratch. One, never rewrite from scratch, always refactor. Its always faster when you refactor, no matter what you may think. Two, what do you do when you have to refactor and you don't have unit tests ? How do you know you haven't horribly broken something in that refactor ? Granted, IDE's such as Eclipse and IntelliJ have made refactoring much more convenient, but adding new functionality or editing existing features is never simple.

More often than not, we end up changing some undocumented way the existing code behaved, and blow up 10 different things (it takes skill to blow up more, believe me, I have tried). And its often something as simple as changing the way a variable is set or unset. In those cases, having unittests (remember those things you were supposed to have written?) to confirm that your refactoring broke nothing is godsend. I can't tell you the amount of times I have had to refactor a legacy code base without this safety net. The only way to ensure I did it correct was to write these large integration tests (because again, no unit tests usually tends to increase the coupling and reduce modularity, even in the most well designed code bases) which verified things at a higher level and pray fervently that I broke nothing. Then I would spend a few minutes bringing up the app everytime, and clicking on random things to make sure nothing blew up. A complete waste of my time when I could have known the same thing by just running my unit tests.

Documentation

Finally, one of my favorite advantages to doing TDD or writing unit tests as I code. I have a short memory for code I have written. I could look back at the code I wrote two days ago, and have no clue what I was thinking. In those cases, all I have to do is go look at the test for a particular method, and that almost always will tell me what that method takes in as parameters, and what all it should be doing. A well constructed set of tests tell you about valid and invalid inputs, state that it should modify and output that it may return.

Now this is useful for people like me with short memory spans. But it is also useful, say, when you have a new person joining the team. We had this cushion the last time someone joined our team for a short period of time, and when we asked him to add a particular check to a method, we just pointed him to the tests for that method, which basically told him what the method does. He was able to understand the requirements, and go ahead and add the check with minimal handholding. And the tests give a safety net so he doesn't break anything else while he was at it.

Also useful is the fact that later, when someone comes marching through your door, demanding you fix this bug, you can always make sure whether it was a a bug (in which case, you are obviously missing a test case) or if it was a feature that they have now changed the requirements on (in which case you already have a test which proves it was your intent to do it, and thus not a bug).

13 comments:

  1. While there is no question that key functional code should have unit tests, the more pragmatic development groups I work with say, "Why should I test an obscure UI method? It will take longer to make the test than to write the six or eight lines of code for the whole method and besides, if I use Eclipse right, five of those lines are written for me before I write anything." The other line I hear is, "We're a manufacturing company, not a software vendor! Of course I expect our vendors to use cutting edge practices, but that's not us." How would you respond to these?

    ReplyDelete
  2. Right on!

    I’d like to supplement your reasoning with a couple of guidelines for writing and organizing tests:

    http://www.bugfree.dk/blog/2009/06/19/my-unit-testing-guidelines

    For some reason it’s a challenge to convince developers to test. The Return on Investment isn’t obvious to many developers.

    I think one reason is that people mix the concepts of unit testing and integration testing, partly because a unit testing framework is used for both.

    People new to unit testing frameworks often start writing integration tests because they’re the easiest to write. Unfortunately they’re also the most brittle, and tend to fail for no apparent reason — when some dependency isn’t properly configured. After a few such false positives developers grow tired of testing because they find the maintenance cost too high.

    Another reason is that unit testing is hard. Getting used to think in terms of testability doesn’t come natural to many developers. The same goes for dependency injection and mocking frameworks.

    In addition many frameworks used by the application aren’t written with testability in mind. And so extra effort must go into mocking their APIs.

    Finally, in my experience, organizing tests so that a team of developers are comfortable with the structure is paramount to success. Basically, a common set of guidelines should be in place from early on and at least one team member must be passionate about testing for it to take off.

    ReplyDelete
  3. I agree strongly with your main points, about the value of both unit testing and test-driven development. But I do have some worries about subsidiary points, and also think you need to distinguish the two more emphatically. Unit testing when you don’t already have a framework to use can be hard, perhaps even impractical—but even in such cases test-driven development is still useful.

    And I think we all need to be wary of Beizer’s Pesticide Paradox and Whittaker’s False Sense of Thoroughness. Google, for all its devotion to unit testing, and the overall quality level of its products, is not immune to this. (Specifics on request.) You call some simple high-level testing “A complete waste of my time when I could have known the same thing by just running my unit tests.” But even though unit tests may well be better than high-level tests, they do tell you different things. I’ve seen a lot of products which passed their unit tests but had glaring high-level bugs.

    ReplyDelete
  4. @Jay Mumper, for the first one, while I don't argue that it takes more time, the six or eight lines more often than not grows into a much bigger / longer method on an active project. And before long, you have this long, delicate method where any mucking around leads to bugs. Wouldn't spending a few extra minutes writing some tests around that save you hours later down ? As for the second one, if you are the one developing a software, I don't are what company you are, you are a software engineer, and it becomes your responsibility to make sure you test your application. Now whether you want to drain thousands of dollars in manual testing or write some unit tests and reduce the strain on qa efforts down the line, thats a call you will have to make.

    @Flash, I completely agree that you do need higher levels of test. There is no way unit tests are going to catch integration issues and issues in interaction between components. What I do strongly hate is the lack of either and then just running your app and manually testing it. That is what I was complaining about in that line you quoted. Focussing just on higher level testing and ignoring unit testing is a huge glaring mistake in my opinion.

    ReplyDelete
  5. Can someone give an best example for these guidelines in a real opensource project? That would be a lot of help if exists. Great Thanks!

    ReplyDelete
  6. This is really good stuff, thanks for posting!

    Does your team have a "Top 10" list of books that discuss the art of TDD that you could recommend?

    ReplyDelete
  7. Good question Liu. I'm convinced that thorough unit tests are something that are written about, but don't exist in real life.

    ReplyDelete
  8. @Liu, check out the Testability Explorer project (http://testability-explorer.googlecode.com/) which has a pretty good hygiene of writing tests along with all their code, as well as integration tests for some of the more complicated interactions. In particular, check out the core plugin under src.

    @Dave, if you are planning to work on an entirely new codebase and want to know how to tdd in such an environment, then Kent Beck's TDD book ( http://www.amazon.com/Test-Driven-Development-Addison-Wesley-Signature/dp/0321146530) comes to mind. But it doesn't talk about working with legacy code bases much and assumes you are familiar with the XUnit pattern. Most of the stuff I learnt has been from pairing with other developers and trying out different techniques and seeing what works and what doesn't. In the end, it does become as simple as, write a test, write the code to get it to pass, and iterate. And as long as you follow the base principles listed at (http://misko.hevery.com/code-reviewers-guide/), then you should be good to go.

    ReplyDelete
  9. Request for an article.
    How would you unit test a form class.
    Example code:
    $form->text('username', array('class' => 'users'));
    This creates an input text field called username
    Thankyou

    ReplyDelete
  10. This is all fine and great, but what do you do in the case where you have code that is inherently not machine testable, because the output requires human judgement? This is often the case in my projects as I work on Natural Language Processing - unit testing this is entirely nontrivial.

    Second, my biggest gripe with unit testing is that it can lull you into a false sense of security. You can be inclined to think your code works and does what it should, when in reality it only behaves well for the test cases covered in your unit tests. For more complex systems it is very easy to miss out on many corner cases in your unit tests, and as a developer it is very frustrating to be handed code that "works because the unit tests say so" only to find out that the unit tests didn't cover my use cases.

    ReplyDelete
  11. @Shyam Seshadri, Great Thanks!
    Misko Hevery is a great Coach! Thanks all your guys hard work!

    ReplyDelete
  12. @Toby
    This is an interesting issue (the false sense of security).
    What I have found is that by adopting more traditional QA processes, such as test plans, it's much easier to see unit tests in context. This tends to help in avoiding any particular type of test becoming "god".
    Now, it can sometimes be challenging to adapt existing organizational practices to incorporate automated testing organically, because they are often tuned for manual testers. But the rewards are usually worth it, especially if there are a wide variety of tests in use.

    ReplyDelete
  13. Hi Shyam,

    Good post. I've posted a "made me think" response in the Typemock Blog . We're doing a webinar next week about this very topic - how to motivate people to start unit testing in an organization.

    ReplyDelete

The comments you read and contribute here belong only to the person who posted them. We reserve the right to remove off-topic comments.