TotT: Contain Your Environment

Many modules must access elements of their environment that are too heavyweight for use in tests, for example, the file system or network. To keep tests lightweight, we mock out these elements. But what if no mockable interface is available, or the existing interfaces pull in extraneous dependencies? In such cases we can introduce a mediator interface that's directly associated with your module (usually as a public inner class). We call this mediator an "Env" (for environment); this name helps readers of your class recognize the purpose of this interface.

For example, consider a class that cleans the file system underlying a storage system:

// Deletes files that are no longer reachable via our storage system's
// metadata.
class FileCleaner {
public:
  class Env {
  public:
    virtual bool MatchFiles(const char* pattern, vector* filenames) = 0;
    virtual bool BulkDelete(const vector& filenames) = 0;
    virtual MetadataReader* NewMetadataReader() = 0;
    virtual ~Env();
  };
  // Constructs a FileCleaner. Uses “env” to access files and metadata.
  FileCleaner(Env* env, QuotaManager* qm);
  // Deletes files that are not reachable via metadata.
  // Returns true on success.
  bool CleanOnce();
};

FileCleaner::Env lets us test FileCleaner without accessing the real file system or metadata. It also makes it easy to simulate various kinds of failures, for example, of the file system:

class NoFileSystemEnv : public FileCleaner::Env {
  virtual bool MatchFiles(const char* pattern, vector* filenames) {
  match_files_called_ = true;
  return false;
  }
  ...
};

TEST(FileCleanerTest, FileCleaningFailsWhenFileSystemFails) {
  NoFileSystemEnv* env = new NoFileSystemEnv();
  FileCleaner cleaner(env, new MockQuotaManager());
  ASSERT_FALSE(cleaner.CleanOnce());
  ASSERT_TRUE(env->match_files_called_);
}

An Env object is particularly useful for restricting access to other modules or systems, for example, when those modules have overly-wide interfaces. This has the additional benefit of reducing your class's dependencies. However, be careful to keep the “real” Env implementation simple, lest you introduce hard-to-find bugs in the Env. The methods of your “real” Env implementation should just delegate to other, well-tested methods.

The most important benefits of an Env are that it documents how your class accesses its environment and it encourages future modifications to your module to keep tests small by extending and mocking out the Env.


Remember to download this episode of Testing on the Toilet and post it in your office.

Permalink | Links to this post | 0 comments  

GUI Testing: Don't Sleep Without Synchronization

Posted by Philip Zembrod, Software Engineer in Test, Sweden

So you're working on TheFinalApp - the ultimate end-user application, with lots of good features and a really neat GUI. You have a team that's keen on testing and a level of unit test coverage that others only dream of. The star of the show is your suite of automatic GUI end-to-end tests — your team doesn't have to manually test every release candidate.

Life would be good if only the GUI tests weren't so flaky. Every once and again, your test case clicks a menu item too early, while the menu is still opening. Or it double-clicks to open a tree node, tries to verify the open too early, then retries, which closes the node (oops). You have tried adding sleep statements, which has helped somewhat, but has also slowed down your tests.

Why all this pain? Because GUIs are not designed to synchronize with other computer programs. They are designed to synchronize with human beings, which are not like computers:

• Humans act much more slowly. Well-honed GUI test robots drive GUIs at near theoretical maximum speed.


• Humans are much better at observing the GUI, and they react intelligently to what they see.


• Humans extract more meaningful information from a GUI.

In contrast to testing a server, where you usually find enough methods or messages in the server API to synchronize the testing with the server, a GUI application usually lacks these means of synchronization. As a result, a running automated GUI test often consists of one long sequence of race conditions between the automated test and the application under test.

GUI test synchronization boils down to the question: Is the app under test finished with what it's doing? "What it's doing" may be small, like displaying a combo box, or big, like a business transaction. Whatever "it" is, the test must be able to tell whether "it" is finished. Maybe you want to test something while "it" is underway, like verify that the browser icon is rotating while a page is loading. Maybe you want to deliberately click the "Submit" button again in the middle of a transaction to verify that nothing bad happens. But usually, you want to wait until "it" is done.

How to find out whether "it" is done? Ask! Let your test case ask your GUI app. In other words: provide one or several test hooks suitable for your synchronization needs.

The questions to ask depend on the type, platform, and architecture of your application. Here are three questions that worked for me when dealing with a single-threaded Win32 MFC database app:

The first is a question for the OS. The Win32 API provides a function to wait while a process has pending input events:
DWORD WaitForInputIdle(HANDLE hProcess, DWORD dwMilliseconds). Choosing the shortest possible timeout (dwMilliseconds = 1) effectively turns this from a wait-for to a check-if function, so you can explicitly control the waiting loop; for example, to combine several different check functions. Reasoning: If the GUI app has pending input, it's surely not ready for new input.

The second question is: Is the GUI app's message queue empty? I did this with a test hook, in this case a WM_USER message; it could perhaps also be done by calling PeekMessage() in the GUI app's process context via CreateRemoteThread(). Reasoning: If the GUI app still has messages in its queue, it's not yet ready for new input.

The third is more like sending a probe than a question, but again using a test hook. The test framework resets a certain flag in the GUI app (synchronously) and then (asynchronously) posts a WM_USER message into the app's message queue that, upon being processed, sets this flag. Now the test framework checks periodically (and synchronously again) to see whether the flag has been set. Once it has, you know the posted message has been processed. Reasoning: When the posted message (the probe) has been processed, then surely messages and events sent earlier to the GUI app have been processed. Of course, for multi-threaded applications this might be more complex.

These three synchronization techniques resulted in fast and stable test execution, without any test flakiness due to timing issues. All without sleeps, except in the synchronization loop.

Applying this idea to different platforms requires finding the right questions to ask and the right way to ask them. I'd be interested to hear if someone has done something similar, e.g. for an Ajax application. A query into the server to check if any XML responses are pending, perhaps?

Permalink | Links to this post | 10 comments  

Testability Explorer: Measuring Testability

Testability Explorer: Using Byte-Code Analysis to Engineer Lasting Social Changes in an Organization’s Software Development Process. (Or How to Get Developers to Write Testable Code)

Presented at 2008 OOPSLA by Miško Hevery a Best Practices Coach @ Google

Abstract

Testability Explorer is an open-source tool that identifies hard-to-test Java code. Testability Explorer provides a repeatable objective metric of “testability.” This metric becomes a key component of engineering a social change within an organization of developers. The Testability Explorer report provides actionable information to developers which can be used as (1) measure of progress towards a goal and (2) a guide to refactoring towards a more testable code-base.
Keywords: unit-testing; testability; refactoring; byte-code analysis; social engineering.

1. Testability Explorer Overview

In order to unit-test a class, it is important that the class can be instantiated in isolation as part of a unit-test. The most common pitfalls of testing are (1) mixing object-graph instantiation with application-logic and (2) relying on global state. The Testability Explorer can point out both of these pitfalls.

1.1 Non-Mockable Cyclomatic Complexity

Cyclomatic complexity is a count of all possible paths through code-base. For example: a main method will have a large cyclomatic complexity since it is a sum of all of the conditionals in the application. To limit the size of the cyclomatic complexity in a test, a common practice is to replace the collaborators of class-under-test with mocks, stubs, or other test doubles.

Let’s define “non-mockable cyclomatic complexity” as what is left when the class-under-test has all of its accessible collaborators replaced with mocks. A code-base where the responsibility of object-creation and application-logic is separated (using Dependency Injection) will have high degree of accessible collaborators; as a result most of its collaborators will easily be replaceable with mocks, leaving only the cyclomatic complexity of the class-under-test behind.

In applications, where the class-under-test is responsible for instantiating its own collaborators, these collaborators will not be accessible to the test and as a result will not be replaceable for mocks. (There is no place to inject test doubles.) In such classes the cyclomatic complexity will be the sum of the class-under-test and its non-mockable collaborators.

The higher the non-mockable cyclomatic complexity the harder it will be to write a unit-test. Each non-mockable conditional translates to a single unit of cost on the Testability Explorer report. The cost of static class initialization and class construction is automatically included for each method, since a class needs to be instantiated before it can be exercised in a test.

1.2 Transitive Global-State

Good unit-tests can be run in parallel and in any order. To achieve this, the tests need to be well isolated. This implies that only the stimulus from the test has an effect on the code execution, in other words, there is no global-state.

Global-state has a transitive property. If a global variable refers to a class than all of the references of that class (and all of its references) are globally accessible as well. Each globally accessible variable, that is not final, results in a cost of ten units on the Testability Explorer.

2. Testability Explorer Report

A chain is only as strong as its weakest link. Therefore the cost of testing a class is equal to the cost of the class’ costliest method. In the same spirit the application’s overall testability is de-fined in terms of a few un-testable classes rather than a large number of testable ones. For this reason when computing the overall score of a project the un-testable classes are weighted heavier than the testable ones.

3. How to Interpret the Report

By default the classes are categorized into three categories: “Excellent” (green) for classes whose cost is below 50; “Good” (yellow) for classes whose cost is below 100; and “Needs work” (red) for all other classes. For convenience the data is presented as both a pie chart and histogram distribution and overall (weighted average) cost shown on a dial.

[-]ClassRepository [ 323 ]
 [-]ClassInfo getClass(String) [ 323 ]
   line 51:
     ClassInfo parseClass(InputStream) [318]
     InputStream inputStreamForClass(String) [2]
 [-]ClassInfo parseClass(InputStream) [318]
   line 77: void accept(ClassVisitor, int) [302]
   line 75: ClassReader(InputStream) [15]

Clicking on the class ClassRepository allows one to drill down into the classes to get more information. For example the above report shows that ClassRepository has a high cost of 318 due to the parseClass(InputStream) method. Looking in closer we see that the cost comes from line 77 and an invocation of the accept() method.

73:ClassInfo parseClass(InputStream is) {
74:  try {
75:    ClassReader reader = new ClassReader(is);
76:    ClassBuilder v = new ClassBuilder (this);
77:    reader.accept(v, 0);
78:    return visitor.getClassInfo();
79:  } catch (IOException e) {
80:    throw new RuntimeException(e);
81:  }
82:}

As you can see from the code the ClassReader can never be replaced for a mock and as a result the cyclomatic complexity of the accept method can not be avoided in a test — resulting in a high testability cost. (Injecting the ClassReader would solve this problem and make the class more test-able.)

4. Social Engineering

In order to produce a lasting change in the behavior of developers it helps to have a measurable number to answer where the project is and where it should be. Such information can provide in-sight into whether or not the project is getting closer or farther from its testability goal.

People respond to what is measured. Integrating the Testability Explorer with the project’s continuous build and publishing the report together with build artifacts communicate the importance of testability to the team. Publishing a graph of overall score over time allows the team to see changes on per check-in basis.

If Testability Explorer is used to identify the areas of code that need to be refactored, than compute the rate of improvement and project expected date of refactoring completion and create a sense of competition among the team members.

It is even possible to set up a unit test for Testability Explorer that will only allow the class whose testability cost is better than some predetermined cost.

Permalink | Links to this post | 5 comments  

Dependency Injection Myth: Reference Passing

by Miško Hevery

After reading the article on Singletons (the design anti-pattern) and how they are really global variables and dependency injection suggestion to simply pass in the reference to the singleton in a constructor (instead of looking them up in global state), many people incorrectly concluded that now they will have to pass the singleton all over the place. Let me demonstrate the myth with the following example.

Let's say that you have a LoginPage which uses the UserRepository for authentication. The UserRepository in turn uses Database Singleton to get a hold of the global reference to the database connection, like this:

class UserRepository {
 private static final BY_USERNAME_SQL = "Select ...";

 User loadUser(String user) {
   Database db = Database.getInstance();
   return db.query(BY_USERNAME_SQL, user);
 }
}

class LoginPage {
 UserRepository repo = new UserRepository();

 login(String user, String pwd) {
   User user = repo.loadUser(user);
   if (user == null || user.checkPassword(pwd)) {
     throw new IllegalLoginException();
   }
 }
}

The first thought is that if you follow the advice of dependency injection you will have to pass in the Database into the LoginPage just so you can pass it to the UserRepository. The argument goes that this kind of coding will make the code hard to maintain, and understand. Let's see what it would look like after we get rid of the global variable Singleton Database look up.

First, lets have a look at the UserRepository.

class UserRepository {
 private static final BY_USERNAME_SQL = "Select ...";
 private final Database db;

  UserRepository(Database db) {
   this.db = db;
 }

 User loadUser(String user) {
   return db.query(BY_USERNAME_SQL, user);
 }
}

Notice how the removal of Singleton global look up has cleaned up the code. This code is now easy to test since in a test we can instantiate a new UserRepository and pass in a fake database connection into the constructor. This improves testability. Before, we had no way to intercept the calls to the Database and hence could never test against a Database fake. Not only did we have no way of intercepting the calls to Database, we did not even know by looking at the API that Database is involved. (see Singletons are Pathological Lairs) I hope everyone would agree that this change of explicitly passing in a Database reference greatly improves the code.

Now lets look what happens to the LoginPage...

class LoginPage {
 UserRepository repo;

  LoginPage(Database db) {
   repo = new UserRepository(db);
 }

 login(String user, String pwd) {
   User user = repo.loadUser(user);
   if (user == null || user.checkPassword(pwd)) {
     throw new IllegalLoginException();
   }
 }
}

Since UserRepository can no longer do a global look-up to get a hold of the Database it musk ask for it in the constructor. Since LoginPage is doing the construction it now needs to ask for the Databse so that it can pass it to the constructor of the UserRepository. The myth we are describing here says that this makes code hard to understand and maintain. Guess what?! The myth is correct! The code as it stands now is hard to maintain and understand. Does that mean that dependency injection is wrong? NO! it means that you only did half of the work! In how to think about the new operator we go into the details why it is important to separate your business logic from the new operators. Notice how the LoginPage violates this. It calls a new on UserRepository. The issue here is that LoginPage is breaking a the Law of Demeter. LoginPage is asking for the Database even though it itself has no need for the Database (This greatly hinders testability as explained here). You can tell since LoginPage does not invoke any method on the Database. This code, like the myth suggest, is bad! So how do we fix that?

We fix it by doing more Dependency Injection.

class LoginPage {
 UserRepository repo;

  LoginPage(UserRepository repo) {
   this.repo = repo;
 }

 login(String user, String pwd) {
   User user = repo.loadUser(user);
   if (user == null || user.checkPassword(pwd)) {
     throw new IllegalLoginException();
   }
 }
}

LoginPage needs UserRepository. So instead of trying to construct the UserRepository itself, it should simply ask for the UserRepository in the constructor. The fact that UserRepository needs a reference to Database is not a concern of the LoginPage. Neither is it a concern of LoginPage how to construct a UserRepository. Notice how this LoginPage is now cleaner and easier to test. To test we can simply instantiate a LoginPage and pass in a fake UserRepository with which we can emulate what happens on successful login as well as on unsuccessful login and or exceptions. It also nicely takes care of the concern of this myth. Notice that every object simply knows about the objects it directly interacts with. There is no passing of objects reference just to get them into the right location where they are needed. If you get yourself into this myth then all it means is that you have not fully applied dependency injection.

So here are the two rules of Dependency Injection:

• Always ask for a reference! (don't create, or look-up a reference in global space aka Singleton design anti-pattern)
• If you are asking for something which you are not directly using, than you are violating a Law of Demeter. Which really means that you are either trying to create the object yourself or the parameter should be passed in through the constructor instead of through a method call. (We can go more into this in another blog post)

So where have all the new operators gone you ask? Well we have already answered that question here. And with that I hope we have put the myth to rest!

BTW, for those of you which are wondering why this is a common misconception the reason is that people incorrectly assume that the constructor dependency graph and the call graph are inherently identical (see this post). If you construct your objects in-line (as most developers do, see thinking about the new operator) then yes the two graphs are very similar. However, if you separate the object graph instantiation from the its execution, than the two graphs are independent. This independence is what allows us to inject the dependencies directly where they are needed without passing the reference through the intermediary collaborators.

Permalink | Links to this post | 2 comments  

Test Engineering at Google

By Roshan Sembacuttiaratchy, Software Engineer in Test, Google Zurich, Switzerland

When I tell people that I'm a Test Engineer at Google, I get a confused look from them and questions as to what I actually do. Do I sit in front of a keyboard clicking every button and link on screen? Do I watch over the infinite number of monkeys, checking to make sure they produce Shakespeare's work while clicking said links? Or do we create better and smarter pigeons? Well, it's actually a bit of everything, but not quite in the way you might think at first.

The people working in Test Engineering are software developers with a passion for testing and test automation. Whereas most software engineers at Google are working on developing products, Test Engineering's objective is to help improve the quality of the projects we're involved in. We work integrated with the project team, developing test frameworks and setting up test systems. One of our key mantras is automation, so our first task whenever we're assigned to a new project is to evaluate the current state of test automation, identify what we could do to improve the results obtained, reduce the total run time, and make the best use of available resources. Load and performance testing is another important task we perform, along with analysis of the results. We're also not afraid to get our hands dirty by diving deep into the project's code to identify what re-factoring might be useful to help test the system better, creating and introducing stubs, fakes and mocks as necessary. As you might guess, we're big fans of Test Driven Design, Continuous Integration and other agile techniques and play the role of evangelists, spreading the word on new tools, techniques and best practices.

Individual project assignments tend to be limited in duration, as we're a relatively small group of people in Test Engineering, helping support a much larger group of software developers. So once we've completed our tasks for one project, we move on to the next, working with a different team of people and a different set of problems, changing projects every few months.

That pretty much describes what I do, and how our team in Zurich works. So to answer the questions posed initially, I write software which performs the task of clicking every button and every link, and which then verifies the result of these actions. I run this on a few hundred machines to mimic virtual monkeys, and hand it over to the development team so that they can use it to check that their product is working, even after I've left their project. And if it produces the works of Shakespeare, that's just an added bonus! :-)

Permalink | Links to this post | 7 comments  

TotT: Floating-Point Comparison

If your code manipulates floating-point values, your tests will probably involve floating-point values as well.

When comparing floating-point values, checking for equality might lead to unexpected results. Rounding errors can lead to a result that is close to the expected one, but not equal. As a consequence, an assertion might fail when checking for equality of two floating-point quantities even if the program is implemented correctly.

The Google C++ Testing Framework provides functions for comparing two floating-point quantities up to a given precision.

In C++, you can use the following macros:

ASSERT_FLOAT_EQ(expected, actual);
ASSERT_DOUBLE_EQ(expected, actual);
EXPECT_FLOAT_EQ(expected, actual);
EXPECT_DOUBLE_EQ(expected, actual);

In Java, JUnit overloads Assert.assertEquals for floating-point types:

assertEquals(float expected, float actual, float delta);
assertEquals(double expected, double actual, double delta);

An example (in C++):

TEST(SquareRootTest, CorrectResultForPositiveNumbers) {
  EXPECT_FLOAT_EQ(2.0f, FloatSquareRoot(4.0f));
  EXPECT_FLOAT_EQ(23.3333f, FloatSquareRoot(544.44444f));
  EXPECT_DOUBLE_EQ(2.0, DoubleSquareRoot(4.0));
  EXPECT_DOUBLE_EQ(23.33333333333333, DoubleSquareRoot(544.44444444444444));
    // the above succeeds
  EXPECT_EQ(2.0, DoubleSquareRoot(4.0));
    // the above fails
  EXPECT_EQ(23.33333333333333, DoubleSquareRoot(544.44444444444444));
}

Remember to download this episode of Testing on the Toilet and post it in your office.

Permalink | Links to this post | 0 comments  

To "new" or not to "new"...

by Miško Hevery

Dependency injection asks us to separate the new operators from the application logic. This separation forces your code to have factories which are responsible for wiring your application together. However, better than writing factories, we want to use automatic dependency injection such as GUICE to do the wiring for us. But can DI really save us from all of the new operators?

Lets look at two extremes. Say you have a class MusicPlayer which needs to get a hold of AudioDevice. Here we want to use DI and ask for the AudioDevice in the constructor of the MusicPlayer. This will allow us to inject a test friendly AudioDevice which we can use to assert that correct sound is coming out of our MusicPlayer. If we were to use the new operator to instantiate the BuiltInSpeakerAudioDevice we would have hard time testing. So lets call objects such as AudioDevice or MusicPlayer "Injectables." Injectables are objects which you will ask for in the constructors and expect the DI framework to supply.

Now, lets look at the other extreme. Suppose you have primitive "int" but you want to auto-box it into an "Integer" the simplest thing is to call new Integer(5) and we are done. But if DI is the new "new" why are we calling the new in-line? Will this hurt our testing? Turns out that DI frameworks can't really give you the Integer you are looking for since it does not know which Integer you are referring to. This is a bit of a toy example so lets look at something more complex.

Lets say the user entered the email address into the log-in box and you need to call new Email("a@b.com"). Is that OK, or should we ask for the Email in our constructor. Again, the DI framework has no way of supplying you with the Email since it first needs to get a hold of a String where the email is. And there are a lot of Strings to chose from. As you can see there are a lot of objects out there which DI framework will never be able to supply. Lets call these "Newables" since you will be forced to call new on them manually.

First, lets lay down some ground rules. An Injectable class can ask for other Injectables in its constructor. (Sometimes I refer to Injectables as Service Objects, but that term is overloaded.) Injectables tend to have interfaces since chances are we may have to replace them with an implementation friendly to testing. However, Injectable can never ask for a non-Injectable (Newable) in its constructor. This is because DI framework does not know how to produce a Newable. Here are some examples of classes I would expect to get from my DI framework: CreditCardProcessor, MusicPlayer, MailSender, OfflineQueue. Similarly Newables can ask for other Newables in their constructor, but not for Injectables (Sometimes I refer to Newables as Value Object, but again, the term is overloaded). Some examples of Newables are: Email, MailMessage, User, CreditCard, Song. If you keep this distinctions your code will be easy to test and work with. If you break this rule your code will be hard to test.

Lets look at an example of a MusicPlayer and a Song

class Song {
 Song(String name, byte[] content);
}
class MusicPlayer {
 @Injectable
 MusicPlayer(AudioDevice device);
 play(Song song);
}

Notice that Song only asks for objects which are Newables. This makes it very easy to construct a Song in a test. Music player is fully Injectable, and so is its argument the AudioDevice, therefore, it can be gotten from DI framework.

Now lets see what happens if the MusicPlayer breaks the rule and asks for Newable in its constructor.

class Song {
 String name;
 byte[] content;
 Song(String name, byte[] content);
}
class MusicPlayer {
 AudioDevice device;
 Song song;
 @Injectable
 MusicPlayer(AudioDevice device, Song song);
 play();
}

Here the Song is still Newable and it is easy to construct in your test or in your code. The MusicPlayer is the problem. If you ask DI framework for MusicPlayer it will fail, since the DI framework will not know which Song you are referring to. Most people new to DI frameworks rarely make this mistake since it is so easy to see: your code will not run.

Now lets see what happens if the Song breaks the rule and ask for Injectable in its constructor.

class MusicPlayer {
 AudioDevice device;
 @Injectable
 MusicPlayer(AudioDevice device);
}
class Song {
 String name;
 byte[] content;
 MusicPlayer palyer;
 Song(String name, byte[] content, MusicPlayer player);
 play();
}
class SongReader {
 MusicPlayer player
 @Injectable
 SongReader(MusicPlayer player) {
   this.player = player;
 }
 Song read(File file) {
   return new Song(file.getName(),
                   readBytes(file),
                   player);
 }
}

At first the world looks OK. But think about how the Songs will get created. Presumably the songs are stored on a disk and so we will need a SongReader. The SongReader will have to ask for MusicPlayer so that when it calls the new on a Song it can satisfy the dependencies of Song on MusicPlayer. See anything wrong here? Why in the world does SongReader need to know about the MusicPlayer. This is a violation of Law of Demeter. The SongReader does not need to know about MusicPlayer. You can tell since SongReader does not call any method on the MusicPlayer. It only knows about the MusicPlayer because the Song has violated the Newable/Injectable separation. The SongReader pays the price for a mistake in Song. Since the place where the mistake is made and where the pain is felt are not the same this mistake is very subtle and hard to diagnose. It also means that a lot of people make this mistake.

Now from the testing point of view this is a real pain. Suppose you have a SongWriter and you want to verify that it correctly serializes the Song to disk. Why do you have to create a MockMusicPlayer so that you can pass it into a Song so that you can pass it into the SongWritter. Why is MusicPlayer in the picture? Lets look at it from a different angle. Song is something you may want to serialize, and simplest way to do that is to use Java serialization. This will serialize not only the Song but also the MusicPlayer and the AudioDevice. Neither MusicPlayer nor the AudioDevice need to be serialized. As you can see a subtle change makes a whole lot of difference in the easy of testability.

As you can see the code is easiest to work with if we keep these two kinds objects distinct. If you mix them your code will be hard to test. Newables are objects which are at the end of your application object graph. Newables may depend on other Newables as in CreditCard may depend on Address which may depend on a City but these things are leafs of the application graph. Since they are leafs, and they don't talk to any external services (external services are Injectables) there is no need to mock them. Nothing behaves more like a String like than a String. Why would I mock User if I can just new User, Why mock any of these: Email, MailMessage, User, CreditCard, Song? Just call new and be done with it.

Now here is something very subtle. It is OK for Newable to know about Injectable. What is not OK is for the Newable to have a field reference to Injectable. In other words it is OK for Song to know about MusicPlayer. For example it is OK for an Injectable MusicPlayer to be passed in through the stack to a Newable Song. This is because the stack passing is independent of DI framework. As in this example:

class Song {
 Song(String name, byte[] content);
 boolean isPlayable(MusicPlayer player);
}

The problem becomes when the Song has a field reference to MusicPlayer. Field references are set through the constructor which will force a Law of Demeter violation for the caller and we will have hard time to test.

Permalink | Links to this post | 2 comments  

TotT: Simulating Time in jsUnit Tests

Sometimes you need to test client-side JavaScript code that uses setTimeout() to do some work in the future. jsUnit contains the Clock.tick() method, which simulates time passing without causing the test to sleep.
For example, this function will set up some callbacks to update a status message over the course of four seconds:

function showProgress(status) {
  status.message = "Loading";
  for (var time = 1000; time <= 3000; time+= 1000) {
    // Append a '.' to the message every second for 3 secs.
    setTimeout(function() {
      status.message += ".";
    }, time);
  }
  setTimeout(function() {
    // Special case for the 4th second.
    status.message = "Done";
  }, 4000);
}

The jsUnit test for this function would look like this:

function testUpdatesStatusMessageOverFourSeconds() {
  Clock.reset(); // Clear any existing timeout functions on the event queue.
  var status = {};
  showProgress(status); // Call our function.

  assertEquals("Loading", status.message);

  Clock.tick(2000); // Call any functions on the event queue that have been
                    // scheduled for the first two seconds.
  assertEquals("Loading..", status.message);

  Clock.tick(2000); // Same thing again, for the next two seconds.
  assertEquals("Done", status.message);
}

This test will run very quickly - it does not require four seconds to run.

Clock supports the functions setTimeout(), setInterval(), clearTimeout(), and clearInterval(). The Clock object is defined in jsUnitMockTimeout.js, which is in the same directory as jsUnitCore.js.

Remember to download this episode of Testing on the Toilet and post it in your office.

Permalink | Links to this post | 2 comments  

Six Hats of Software Testing

Posted by Jessica Tomechak, Test Engineering Team

Julian Harty, one of our senior test engineers, is presenting a keynote at the STARWEST conference today (Wednesday, October 1) on Six Thinking Hats for Software Testers. Expanding on the Thinking Hats concept, originated many years ago by Edward De Bono and used by large numbers of people and various types of businesses, Julian's talk will add his experience and views about how Thinking Hats can be used in software testing. At Google, we have found it delivers breakthroughs in the short term and great results in the longer term -- one Googler called it the "universal unblocker."

For those of you unable to attend the conference, a video recording will be available (UPDATE: Video now available). Julian has also written an article on this topic to appear in Better Software magazine within the next few months.

Permalink | Links to this post | 2 comments