Taming the Beast (a.k.a. how to test AJAX applications) : Part 1

Posted by Markus Clermont, John Thomas

This is the first in a two part blog series titled 'Taming the Beast: How to test AJAX applications". In this part we discuss some philosophies around web application testing and how it should be done the 'right' way. In part 2, we walk through a real example of designing a test strategy for an AJAX application by going 'beyond the GUI.'

Background
Typically we address the problem of testing an AJAX application through a plethora of big end-to-end tests and (hopefully) high unit-test coverage. Here we outline the main problems with this approach and demonstrate an effective testing strategy for a sample GWT-based application which goes beyond "testing just through the GUI."

Problems with GUI-only testing

In general testing through the GUI:

• is expensive (takes long to write the tests and execution is resource-intensive)
  
• gives limited insight into the system
• often take only the 'happy paths' into account
• combines multiple aspects in a single test
  
• is slow and brittle
• needs a lot of maintenance
  
• is hard to debug

And while unit tests don't suffer from many of these problems, they alone are not sufficient mainly because they:

• give little insight how the components interact with each other
• don't give confidence that the business logic and functionality of the system meets the requirements

Solution
Although there is no 'one size fits all' solution, there are some basic principles we can use to solve the testing problem for web applications:

• Invest in integration tests (identify the smallest sub-system)
• Separation of Concerns (don't do the set-up through the interface you are testing)
• Test each interface separately (mock out everything that you are not testing)
• Consider dependencies in production (figure out how dependencies can fail, and test that)
  
• Use a mix of strategies and tools. There is no silver bullet.
• And no... you cannot scrap all of your end-to-end tests

A recipe for testing goodness
Using the principles above we can build up a recipe for testing web applications. In the second part of our blog we will walk through each of these steps for a real web application.

1. Explore the system's functionality
2. Identify the system's architecture
3. Identify the interfaces between components
4. Identify dependencies and fault conditions
5. For each function:

• Identify the participating components
• Identify potential problems
• Test in isolation for problems
• Create a 'happy path' test

End note: value of a test
A question commonly asked by developers when writing tests is, "is this really worth my time?" The short answer is "always!". Since fixing a bug is way more expensive than preventing it in the first place, writing good tests is always worth the time.

While there are many different classifications of tests, the most common way of classifying them is based on their size and the areas of a product they test. Each test answers specific questions about the product:

• Unit test: is the method fulfilling its contract?
• Small integration test: Can two classes interact with each other?
• Medium integration test: Is a class interacting properly with all its dependencies? Does it anticipate and handle errors correctly? Are the needed functions exposed on an API/GUI?
• Sub-system test: Can two sub-systems interact with each other? Does one of them anticipate all errors of the other and does it deal with them appropriately?
  
• System test: Does the entire system behave as expected?
  

Keeping this questions in mind, testing at various levels allows us to write more focused and meaningful tests. Remember that effective tests are those that provide quick and useful feedback, i.e. quickly identify issues and pin point the exact location of the issue.

In the next episode we will walk through the recipe proposed above using a real web application.

Permalink | Links to this post |