More bang for your testing buck

By James Whittaker

I am giving a webinar for uTest that may be of interest to some of you. Date: Friday Dec 10 at 8am PST. I tend to be pretty grouchy about that time of the morning and I have some pretty exhausting plans for the evening before so it might be extra fun.

Here's the link to register.

And here's the abstract:

"If you were going to invest more money in testing, where would you place those bets? Testing early in the cycle? Automation? Manual testing? Better requirements and planning? Better documentation? Torture devices for your devs? James Whittaker takes a critical look at such an investment and draws some very counterintuitive conclusions about maximizing such an investment. He then outlines a set of tools and practices that will help maximize the overall investment and make testing a happier place."

Permalink | Links to this post | 2 comments  

An Ingredients List for Testing - Part Seven (of Seven)

By James Whittaker

When to stop testing? It’s the age old testing question that many researchers have tried to quantify. In fact, the best answer requires no science whatsoever: never. Since testing is infinite, we can never really stop. A more practical answer also surfaces in the real world: when the software ships, you’re done. Of course this is only true for the duration of the ship party, after that testing continues on the next version.

At Google we are experimenting with test completeness measures that describe how well actual testing covers the risk landscape. In other words, we are measuring the extent to which our testing covers the things that require the most testing. Tests that cover the high risk areas well count for more than tests that cover lower risk features. Testing is, after all, a business of risk mitigation.

The set of tools necessary to accomplish this were described in my GTAC 2010 talk which should appear on YouTube soon and are collectively being called Google Test Analytics. More about these tools in future posts.

Permalink | Links to this post | 6 comments  

An Ingredients List for Testing - Part Six

By James Whittaker

The sixth ingredient is variation. Tests often get stale (i.e., they stop finding bugs) as they are run over and over on build after build as a product is being constructed. On the one hand, it is important to continue running tests to ensure the product still operates as specified. Indeed, I hesitate to throw any test away. However, becoming reliant on stale tests is too risky. Adding variation to existing tests can range from straightforward reordering of the sequence in which tests are run to more involved solutions of either modifying tests or adding new ones. Hopefully new tests will increase overall coverage and add to our confidence that we’ve tested the software in all the ways it needs to be tested.

Permalink | Links to this post | 4 comments  

Test your app from right to left

Can you spot the error in the following webpage?


Unless you are one of the 56 million Internet users who read Arabic, the answer is probably no. But BidiChecker, a tool for checking webpages for errors in handling of bidirectional text, can find it:


Oops! The Arabic movie title causes the line to be laid out in the wrong order, with half of the phrase "57 reviews" on one side of it and half on the other.

As this example demonstrates, text transposition errors can occur even if your web application is entirely in a left-to-right language. If the application accepts user input or displays multilingual content, this data may be in one of the right-to-left languages, such as Arabic, Hebrew, Farsi or Urdu. Displaying right-to-left text in a left-to-right environment, or vice versa, is likely to cause text garbling if not done correctly. So most user interfaces, whether left-to-right or right-to-left, need to be able to deal with bidirectional (BiDi) text.

Handling BiDi text can be tricky and requires special processing at every appearance of potentially BiDi data in the UI. As a result, BiDi text support often regresses when a developer adds a new feature–and fails to include BiDi support in the updated code.

Called from your automated test suite, BidiChecker can catch regressions before they go live. It features a pure JavaScript API which can easily be integrated into a test suite based on common JavaScript test frameworks such as JSUnit. Here's a sample test for the above scenario:


// Check for BiDi errors with Arabic data in an English UI.
function testArabicDataEnglishUi() {



 // User reviews data to display; includes Arabic data.



 var reviewsData = [



 

 {'title': 'The Princess Bride', 'reviews': '23'},


 

 {'title': '20,000 Leagues Under the Sea', 'reviews': '17'},


 

 {'title': 'ستار تريك', 'reviews': '57'} // “Star Trek”



 ];





 // Render the reviews in an English UI.


 var app = new ReviewsApp(reviewsData, testDiv);


 app.setLanguage('English');



 app.render();







 // Run BidiChecker.



 var errors = bidichecker.checkPage(/* shouldBeRtl= */ false, testDiv);



 // This assertion will fail due to BiDi errors!



 assertArrayEquals([], errors);

}

We’ve just released BidiChecker as an open source project on Google Code, so web developers everywhere can take advantage of it. We hope it makes the web a friendlier place for users of right-to-left languages and the developers who support them.

By Jason Elbaum, Internationalization Team

Permalink | Links to this post | 1 comments