Wednesday, December 17, 2008

Static Methods are Death to Testability

by Miško Hevery

Recently many of you, after reading Guide to Testability, wrote to telling me there is nothing wrong with static methods. After all what can be easier to test than Math.abs()! And Math.abs() is static method! If abs() was on instance method, one would have to instantiate the object first, and that may prove to be a problem. (See how to think about the new operator, and class does real work)

The basic issue with static methods is they are procedural code. I have no idea how to unit-test procedural code. Unit-testing assumes that I can instantiate a piece of my application in isolation. During the instantiation I wire the dependencies with mocks/friendlies which replace the real dependencies. With procedural programing there is nothing to "wire" since there are no objects, the code and data are separate.

Here is another way of thinking about it. Unit-testing needs seams, seams is where we prevent the execution of normal code path and is how we achieve isolation of the class under test. seams work through polymorphism, we override/implement class/interface and than wire the class under test differently in order to take control of the execution flow. With static methods there is nothing to override. Yes, static methods are easy to call, but if the static method calls another static method there is no way to overrider the called method dependency.

Lets do a mental exercise. Suppose your application has nothing but static methods. (Yes, code like that is possible to write, it is called procedural programming.) Now imagine the call graph of that application. If you try to execute a leaf method, you will have no issue setting up its state, and asserting all of the corner cases. The reason is that a leaf method makes no further calls. As you move further away from the leaves and closer to the root main() method it will be harder and harder to set up the state in your test and harder to assert things. Many things will become impossible to assert. Your tests will get progressively larger. Once you reach the main() method you no longer have a unit-test (as your unit is the whole application) you now have a scenario test. Imagine that the application you are trying to test is a word processor. There is not much you can assert from the main method.

We have already covered that global state is bad and how it makes your application hard to understand. If your application has no global state than all of the input for your static method must come from its arguments. Chances are very good that you can move the method as an instance method to one of the method's arguments. (As in method(a,b) becomes a.method(b).) Once you move it you realized that that is where the method should have been to begin with. The use of static methods becomes even worse problem when the static methods start accessing the global state of the application. What about methods which take no arguments? Well, either methodX() returns a constant in which case there is nothing to test; it accesses global state, which is bad; or it is a factory.

Sometimes a static methods is a factory for other objects. This further exuberates the testing problem. In tests we rely on the fact that we can wire objects differently replacing important dependencies with mocks. Once a new operator is called we can not override the method with a sub-class. A caller of such a static factory is permanently bound to the concrete classes which the static factory method produced. In other words the damage of the static method is far beyond the static method itself. Butting object graph wiring and construction code into static method is extra bad, since object graph wiring is how we isolate things for testing.

"So leaf methods are ok to be static but other methods should not be?" I like to go a step further and simply say, static methods are not OK. The issue is that a methods starts off being a leaf and over time more and more code is added to them and they lose their positions as a leafs. It is way to easy to turn a leaf method into none-leaf method, the other way around is not so easy. Therefore a static leaf method is a slippery slope which is waiting to grow and become a problem. Static methods are procedural! In OO language stick to OO. And as far as Math.abs(-5) goes, I think Java got it wrong. I really want to write -5.abs(). Ruby got that one right.

22 comments:

  1. Misko,
    This is a great writeup. I've been looking for a good explanation of why statics almost always lead to "badness".

    The slow evolution of "simple statics" to "bad statics" is hard for people to spot. It's sort of like a glacier moving. But eventually you blink and take a fresh look and realize a LOT has changed.

    Thanks!

    Patrick

    ReplyDelete
  2. You can unit test procedural code. Take a file of functions... add it to a new project. Mock up a file with all the dependant methods and fill them in.

    Instead of wiring up objects in the usual way you do for OO, for procedural, you have to recompile the file with its mocked dependencies. You recompile instead of building mocks on the fly and injecting them.

    So.. if procedural code is unit-testable... lots of people do it! Then so are static methods, it's just the process is a little different.

    However, the advice in this post does sound very useful if you want to avoid recompiling for each unit test... which is a bit more time consuming, and takes a few more steps to automate.

    ReplyDelete
  3. As Scott says you can unit test procedural code if you try hard enough. Michael Feathers has some things to say about this in his book "Working Effectively with Legacy Code" (which incidentally should really be entitled something like "Refactoring for Testing"). You can instantiate part of your application for testing with clever linking or with a preprocessor.

    But of course life is much nicer in an OO environment such as Java or Ruby.

    Btw Misko I'm really impressed with your blog. Keep up the good work - it is always worth reading!

    ReplyDelete
  4. After reading your explanation, it seems to me that the testing of a static method is not the actual problem. The problem is to test code that calls static methods, because the call graph is hardwired ("there is no way to override"). This is also a reason to avoid static methods even as leaves, since the code that calls it becomes more difficult to test.

    ReplyDelete
  5. By the way, in Python it is possible to override static methods:

    class A(object):
    __@staticmethod
    __def m():
    ____print 'A.m'

    __def f(self):
    ____print 'A.f'
    ____self.m()

    class B(A):
    __@staticmethod
    __def m():
    ____print 'B.m'

    b = B()
    b.f()

    This will print:
    A.f
    B.m

    In Java you always call a static method on a class: although you can write "variable.staticMethod()", the compiler takes the static type of the variable to determine which method to call. In Python the runtime type of the variable is used, so if the static method is overridden, the overridden version will be called.

    Therefore I think that when unit testing Python code, static methods are not a problem, as long as you call them on instances or classes passed to you instead of hardcoded class names.

    ReplyDelete
  6. It seems the OO world is in conflict over this very subject, and you have done an extremely good job of addressing it. I really enjoy your views on testability.

    However, I do have some thought's I'd like to share. First, what are we as developers supposed to do when faced with API's that don't "comply" to a "no-static-methods-allowed" philosophy? I can't really rewrite Java to change Integers to have a sqrt() method. As developers we are largely stuck with what we're given in that aspect. I don't think you addressed this forgivingly enough in your post :). I think something to the point of "though your language may make heavy use of static methods, under no circumstances create your own static methods" would have come across well.

    Also, I understand the significant difference between a singleton and a single shared reference of an object, for testing purposes, but isn't a container-managed (eg. Spring-configured) object nearly the same as a singleton object? I mean, if I inject a single shared instance of an object to all dependent classes via Spring, I could easily achieve the same thing with a singleton. I'm not advocating the use of a singleton, because I know that they are truly death to OOP, I agree with you on that one. But I want to understand why you argue that even factory methods are bad! The factory method pattern hides a lot of ugly details from developers and exemplifies OOP encapsulation. I really want to understand where you're coming from.

    Anyway, these are just my thoughts on this matter. Honestly, thank you for a very conclusive post on this matter :)

    ReplyDelete
  7. Misko, so your Java guru Joshua Bloch got it all wrong when he recommends replacing constructors with static factories?

    You are using Math.abs() in your example, which is just mathematical function. So, if abs() was an instance, non final method, that you could override with some goofy implementation to create a mock that returns negative values ( yes you can do it on ruby ), would you wire this mock object as a dependancy in your client objects?

    I think static methods are fine as long as they do not modify any state, basically utility methods, that will never change. I think Math.abs() and all other utility methods in Math class are the perfect example of a valid usage of static methods. No one should ever change them. And it is not that hard to test them.
    assertTrue(Math.abs(Integer.MIN_VALUE) > 0)

    I certainly agree, that static methods that modify a state should never be used.

    ReplyDelete
  8. Static methods and constructors can be mocked in Java, using PowerMock. http://code.google.com/p/powermock/

    I've tried it; it works, much to my surprise.

    ReplyDelete
  9. Great article.

    Did you deliberately not account for functional programming style? In that case, a lot of methods could be thought of as "static", but could actually be very testable:
    * Much less reliance on state (so very little work for tests to set up state)
    * Functions accepting other functions as parameters gives you a perfect hook for testability.

    I would _guess_ that functional programmes are perhaps some of the easiest to test?


    Having said all that, true functional programming seems to be used quite rarely outside academia, so I agree wholeheartedly with what you've said, when put in the context of every codebase I've ever worked on! :)

    ReplyDelete
  10. One problem I have with this generalization:

    Chances are very good that you can move the method as an instance method to one of the method's arguments. (As in method(a,b) becomes a.method(b).) Once you move it you realized that that is where the method should have been to begin with.

    What about when a and b are not of object types that you created? In a web app I have a utility class with a method:

    public String htmlHighlightSearchTerm(String resultTxt, String searchTxt)

    This method returns a copy of resultTxt with <b> and </b> inserted around occurrences of searchTxt. This seems to me the perfect case for a static helper method, and it should be easily testable.

    I do agree that using static methods when object methods make more sense is a good coding practice that will lead to testable code, but when the need arises for static helper methods, it's important to make sure they have a clear mapping of inputs to expected outputs.

    ReplyDelete
  11. If you change the title to "Static State is Death to Testability", then I agree with you. However, as others have pointed out, static methods that are pure functions are examples of functional, not procedural code, and are thus (typically) easy to test. Math.abs() falls into this category, and using it as your primary example really weakens your argument.

    I'm quite sure that you know how to test an implementation of Math.abs(), and this have nothing to do with it's algorithmic simplicity. It is easy to test because it does not modify any static state. Having nothing to mock is exactly the point!

    ReplyDelete
  12. Turning everything into an instantiable object is also a bad idea. It leads to confusion, is error-prone, and is cognitively wasteful.

    The real solution is for languages to allow for interfaces containing just static methods, and for the language to allow you to parametrize over different implementations of those interfaces.

    The ML-style module system is EXACTLY this. A signature is an interface which is just a bag of functions and values. The implementation of this signature is called a structure. A functor is a structure that takes other structures of a given signature as arguments. More generally, the module system also includes something called abstract data types... which allows you to perform polymorphism to do EXACTLY what you talk about in your getting-rid-of-ifs talks.

    Haskell's typeclasses is another approach to this.

    Please please please... read about this. Please please please.

    ReplyDelete
  13. On the algorithm side we are seeing intensive semantic methods emerge, which may in time challenge statistical methods like pagerank.

    ReplyDelete
  14. so you can't test anything that has no internal state.
    for tests you can probably wrap around any object and delegate to the static methods.

    btw - the line below's the best in the entire article :)

    assertTrue(Math.abs(Integer.MIN_VALUE) > 0)

    [should be asserFalse]. Kudos!

    ReplyDelete
  15. Stanimir,
    you are right, it should be assertFalse to pass the test, because of the numeric overflow. I should had thought about it before posting, or run the test :)

    ReplyDelete
  16. too bad x.x
    I thought 'twas a satiric moment. (indeed it was the best one)

    ReplyDelete
  17. Will google give us any audio hosting. sunil kumar lalwani sksoftmind

    ReplyDelete
  18. "If your application has no global state than all of the input for your static method must come from its arguments. Chances are very good that you can move the method as an instance method to one of the method's arguments. (As in method(a,b) becomes a.method(b).) Once you move it you realized that that is where the method should have been to begin with."

    That's the money quote right there. Excellent point!

    ReplyDelete
  19. I don't know that I fully understand what you mean by not "wireing" something up. What if you have a simple static append function (fair warning I use groovy)...

    class TestClass{
    String foo
    void someMethod(){
    String extractedInfo = extract(foo)
    println testMethod(extractedInfo)
    }

    static String testMethod(String test){
    return "${test}end"
    }
    }

    The point I am trying to make here is that the value you are passing can be any value, and needs no relation to the class itself, the class is just the logical place to store it.


    this seems like a simple enough method to test(and maybe I even want to limit override access). I do get what your saying when it comes to maybe limiting extending of the class, however, I am just not convinced that you shouldn't ever use statics except purely procedural classes.

    ReplyDelete
  20. Nicely written. My sentiments exactly!

    ReplyDelete
  21. It seems the main argument against static methods is that they are too easy to violate good programming practice with. I think the same can be said with a purely object oriented approach. Just replace "if the static method calls another static method" with "if the instance method creates another object using 'new' and calls its instance method" and you've got the same problem.

    More importantly, banning static methods goes against the trend towards adding functional programming features to the language. An example would be filtering values in an Iterable. Mandating instance methods means either adding the filter method to all instances of Iterable or creating a wrapper object with a filter method. Neither are optimal solutions, hence the move towards higher order programming. Higher order programming requires functions, which in the Java world translate to static methods. Extension methods are essentially syntactic sugar for static functions. Furthermore, the closer you get to funcational programming, the more sure you can be of your tests. So how does this trend fit with the no statics mandate?

    ReplyDelete

The comments you read and contribute here belong only to the person who posted them. We reserve the right to remove off-topic comments.