TotT: Refactoring Tests in the Red

With a good set of tests in place, refactoring code is much easier, as you can quickly gain a lot of confidence by running the tests again and making sure the code still passes.

As suites of tests grow, it's common to see duplication emerge. Like any code, tests should ideally be kept in a state that's easy to understand and maintain. So, you'll want to refactor your tests, too.

However, refactoring tests can be hard because you don't have tests for the tests.

How do you know that your refactoring of the tests was safe and you didn't accidentally remove one of the assertions?

If you intentionally break the code under test, the failing test can show you that your assertions are still working. For example, if you were refactoring methods in CombineHarvesterTest, you would alter CombineHarvester, making it return the wrong results.

Check that the reason the tests are failing is because the assertions are failing as you'd expect them to. You can then (carefully) refactor the failing tests. If at any step they start passing, it immediately lets you know that the test is broken – undo! When you're done, remember to fix the code under test and make sure the tests pass again.
(revert is your friend, but don't revert the tests!)

Let's repeat that important point:

When you're done...remember to fix the code under test!

Summary

• Refactor production code with the tests passing. This helps you determine that the production code still does what it is meant to.
• Refactor test code with the tests failing. This helps you determine that the test code still does what it is meant to.

Remember to download this episode of Testing on the Toilet and post it in your office.

Permalink | Links to this post | 5 comments  

Test Engineering in Zurich

Posted by Phil Rollet, Senior Software QA Engineer

Zurich, Switzerland is the location of one of Google's largest engineering offices in Europe. We like to say it is no longer necessary to live in Silicon Valley to develop great software for Google, and the Zurich office has over 100 software engineers working on Google products and infrastructure. As a result the Zurich Test Engineering team is kept very busy.

Producing great software is the driving motivation for each member of the Test Engineering team, which is relatively small by industry standards. But we're growing quickly and are passionate about software development, software quality, and testing.

We work on several projects: some are customer-facing and some are infrastructure. We currently work on Google Maps, which has testers and developers in several of our offices around the world. Our team builds tools to check the consistency of the data feeds that support the local search features in Maps. Another project mainly developed in Zurich is Google Transit, which provides public transportation itineraries and schedule information for commuters who take trains, buses, and trams. On this project, we build tools to verify the proper alignment of the transportation layer of the map with the actual location coordinates of transit stops. We also focus on many projects related to Google’s infrastructure. For example, with Google Base API, we work with the Software Engineering team to measure response time and to track bottlenecks during large-scale bulk updates.

Our aim is to assign local test engineers to most projects developed in this office, so hiring for this team is always a priority. Candidates are from all over the world, and many different nationalities are represented in our office. Adapting to Zurich is quite easy, because it is already an international place: many companies have their headquarters here, and Zurich has been named the best city in the world for its quality of life in 2005, 2006, and 2007.

Permalink | Links to this post | 3 comments  

Test Engineering around the world

Posted by Allen Hutchison, Engineering Manager

The Test Engineering group at Google is truly a global organization. Our team is made up of test engineers in every corner of the world, working on all kinds of interesting projects. Today we are starting a series of blog posts to introduce you to our teams in different offices. Each post will give you some information on the testing projects we have in that particular office, and a little bit of the flavor of that region of the world. The global effort that makes up our test engineering team, and the diversity of ideas that are brought to bear on the challenges that we face, is one of our greatest strengths as a team.

In the coming weeks and months, join me in learning more about Google Testing from a global perspective. Perhaps you'll find that we have a team in your part of the world.

Permalink | Links to this post | 1 comments  

GTAC Speaker Submission Window Closed

Posted by Allen Hutchison, Engineering Manager

Thank you to everyone who submitted a proposal to speak at the Google Test Automation Conference. We'll be reading each proposal closely and selecting those that we think are best for the conference. If you submitted a proposal, then we'll be getting in touch with you directly to let you know if it was accepted or not.

If you didn't submit a proposal, but are interested in attending, then check back on the Google Testing Blog on May 7 to see the conference schedule and instructions for how to apply to attend.

Permalink | Links to this post | 1 comments  

Test Engineering Internships

Posted by Zuri Kemp, Lead Software Quality Assurance Engineer

Thinking about becoming a Google test engineering intern? Want to help us organize the world's information and make it universally accessible and useful -- but you're not sure what it's really like here? Over the next several months we'll be featuring articles by former Test Engineering interns. Until then, here's a top-10 list of adjectives to describe software engineers at Google:

• Global - Google has engineering offices (and, of course, customers) all over the world. So not only do we have engineers from everywhere, every engineer gets the chance to make software that will be as great in Singapore as it is in Finland.
• Comfortable - Engineers need to be comfortable to be effective...and Google gives us the equipment to be comfortable. And if our muscles get tight from sitting, we can get an on-site professional massage.
• Flexible - Getting work done is more important than what time we work.
• Unhampered - We're a company designed by engineers for engineers, so functional merit tends to outweigh other considerations at decision-making time.
• Entertained - We have lots of fun activities. Every winter, there's a company-wide ski trip. Every summer, there's an all-engineer picnic on the Santa Cruz beach boardwalk. Google has bought out theaters for opening days of films like Star Wars and Lord of the Rings.
• Listened to - Google has a very open environment and truly values each employee's opinions. Every Friday, there's a company-wide meeting where Larry and Sergey (our founders) report on significant events and have an open-to-all Q&A session.
• Stuffed - In most offices, Google provides free gourmet-quality breakfast, lunch, and dinner. If you come in too late to get breakfast (or you get hungry between meals), you can always head to the nearby mini-kitchen and grab some fruit, cereal, and tea (or crisps, chocolate, and espresso for the less healthily inclined).
• Cutting-edge - We have the audacious goal of organizing the world's information -- and to meet it, we've built the largest distributed computer system in the world and written some of the world's most widely-used software.
• High-impact - People in every country and every language use our products.
• Dependable - We try to keep a high bar for hiring, so you can depend on the ability of your teammates to handle most any engineering problem.

If all of that (and this) wasn't enough to convince you that Google is a fantastic environment for a software engineer in test, consider being able to "make and break" cool Google products. That is exactly what test engineers at Google do. We not only manually find flaws in software engineers' code, we also build smart, home-grown tools that effectively break and test the limits of web software.

While the deadline for Test Engineering internships for this summer is past, we want to help you make a decision for your internship next year. Stay tuned to our blog, we'll be posting a regular series on Test Engineering internships at Google.

Permalink | Links to this post | 2 comments  

TotT: Stubs Speed up Your Unit Tests

Michael Feathers defines the qualities of a good unit test as: “they run fast, they help us localize problems.” This can be hard to accomplish when your code accesses a database, hits another server, is time-dependent, etc.

By substituting custom objects for some of your module's dependencies, you can thoroughly test your code, increase your coverage, and still run in less than a second. You can even simulate rare scenarios like database failures and test your error handling code.

A variety of different terms are used to refer to these “custom objects”. In an effort to clarify the vocabulary, Gerard Meszaros provides the following definitions:

• Test Double is a generic term for any test object that replaces a production object.
• Dummy objects are passed around but not actually used. They are usually fillers for parameter lists.
• Fakes have working implementations, but take some shortcut (e.g., InMemoryDatabase).
• Stubs provide canned answers to calls made during a test.
• Mocks have expectations which form a specification of the calls they do and do not receive.

For example, to test a simple method like getIdPrefix() in the IdGetter class:

public class IdGetter {  // Constructor omitted.
  public String getIdPrefix() {
    try {
      String s = db.selectString("select id from foo");
      return s.substring(0, 5);
    } catch (SQLException e) { return ""; }
  }
}

You could write:

  db.execute("create table foo (id varchar(40))");  // db created in setUp().
  db.execute("insert into foo (id) values ('hello world!')");
  IdGetter getter = new IdGetter(db);
  assertEquals("hello", getter.getIdPrefix());

The test above works but takes a relatively long time to run (network access), can be unreliable (db machine might be down), and makes it hard to test for errors. You can avoid these pitfalls by using stubs:

  public class StubDbThatReturnsId extends Database {
    public String selectString(String query) { return "hello world"; }
  }
  public class StubDbThatFails extends Database {
    public String selectString(String query) throws SQLException {
      throw new SQLException("Fake DB failure");
    }
  }
  public void testReturnsFirstFiveCharsOfId() throws Exception {
    IdGetter getter = new IdGetter(new StubDbThatReturnsId());
    assertEquals("hello", getter.getIdPrefix());
  }
  public void testReturnsEmptyStringIfIdNotFound() throws Exception {
    IdGetter getter = new IdGetter(new StubDbThatFails());
    assertEquals("", getter.getIdPrefix());
  }

Remember to download this episode of Testing on the Toilet and post it in your office.
 

Permalink | Links to this post | 5 comments  

Google Test Automation Conference submission deadline coming soon

Posted by Allen Hutchison, Engineering Manager

If you recall, we announced the Google Test Automation Conference here about a month ago. To everyone who has submitted a proposal to speak at the conference, thank you. To those of you who haven't yet submitted your proposal, I want to remind you that the deadline for submissions is this Friday, April 6.

We're looking for speakers with exciting ideas and new approaches to test automation. If you have a subject you'd like to talk about, please send an email to gtac-submission@google.com and include a description of your 45 minute session in 500 words or less (no attachments, please). Deadline for submissions is April 6.

Permalink | Links to this post | 3 comments