How Google Tests Software - A Break for Q&A
Wednesday, May 04, 2011
By James Whittaker
New material for the this series is coming more slowly. I am beginning to get into areas where I want to start posting screen shots of internal Google tools and describe how our infrastructure works. This is material that takes longer to develop and also requires some scrutiny before being published externally. So in the meantime, I am pausing to answer some of the questions you've posted in the comments.
I am going to start with Lilia (because she likes Neil Young mainly, but also because she can run further than me and those two things combine to impress me to no small end) who asks about SET-SWE conversion and vice-versa and which I have seen the most. There is also the broader question of whether there is a ceiling on the SET career path.
SETs and SWEs are on the same pay scale and virtually the same job ladder. Both roles are essentially 100% coding roles with the former writing test code and the latter doing feature development. From a coding perspective the skill set is a dead match. From a testing perspective we expect a lot more from SETs. But the overlap on coding makes SETs a great fit for SWE positions and vice versa. Personally I think it is a very healthy situation to have conversions. Since I have both roles reporting to me I can speak from first hand experience that many of my best coders are former SETs and some of my best testers are former SWEs. Each is excellent training ground for the other. On my specific team I am even on the conversions from one role to the other. But I suspect that Google-wide there are more SETs who become SWEs.
Why convert in the first place? Well at Google it isn't for the money. It also isn't for the prestige as we have a lot more SWEs than SETs and it is a lot harder to standout. The scarcity of our SETs creates somewhat of a mystique about these folk. Who are these rare creatures who keep our code bases healthy and make our development process run so smoothly? Actually, most SWEs care more about making the SETs happy so they continue doing what they do. Why would any dev team force a conversion of a great developer from SET to SWE when finding a suitable SET replacement is so much harder than adding another feature developer? SWEs ain't that stupid.
Now pausing before I take another hit of the corp kool-aid, let me be honest and say that there are far more senior SWEs than SETs. Percentage wise we test folk are more outnumbered at the top of the org than at the middle and bottom. But keep in mind that developers have had a large head start on us. We have developers who have been at Google since our founding and testers ... well ... less time than that.
Where do TEs fit into this mix? TE is an even newer role than SET but already we have a number climbing to the Staff ranks and pushing on the senior most positions in the company. There is no ceiling, but the journey to the top takes some time.
Raghev among others has asked about the career path and whether remaining an IC (individual contributor) is an option over becoming a manager. I have mixed feelings about answering this. As a manager myself, I see the role as one with much honor and yet I hear in your collective voices a hint of why do I have to become a manager? Ok, I admit, Dilbert is funny.
For me, being a manager is a chance to impart some of my experience and old-guy judgement on less experienced but more technically gifted ICs. The combination of an experienced manager's vision and an ICs technical skill can be a fighting force of incredible power. And yet, why should someone who does not want to manage be forced to do so in order to continue their career advancement?
Well, fortunately, Google does not make us choose. Our managers are expected to have IC tasks they perform. They are expected to be engaged technically and lead as opposed to just manage. And our ICs are expected to have influence beyond their personal work area. When you get to the senior/staff positions here you are a leader, period. Some leaders lead more than they manage and some leaders manage more than they lead.
But either way, the view from the top means that a lot of people are looking to you for direction ... whether you manage them or not.
New material for the this series is coming more slowly. I am beginning to get into areas where I want to start posting screen shots of internal Google tools and describe how our infrastructure works. This is material that takes longer to develop and also requires some scrutiny before being published externally. So in the meantime, I am pausing to answer some of the questions you've posted in the comments.
I am going to start with Lilia (because she likes Neil Young mainly, but also because she can run further than me and those two things combine to impress me to no small end) who asks about SET-SWE conversion and vice-versa and which I have seen the most. There is also the broader question of whether there is a ceiling on the SET career path.
SETs and SWEs are on the same pay scale and virtually the same job ladder. Both roles are essentially 100% coding roles with the former writing test code and the latter doing feature development. From a coding perspective the skill set is a dead match. From a testing perspective we expect a lot more from SETs. But the overlap on coding makes SETs a great fit for SWE positions and vice versa. Personally I think it is a very healthy situation to have conversions. Since I have both roles reporting to me I can speak from first hand experience that many of my best coders are former SETs and some of my best testers are former SWEs. Each is excellent training ground for the other. On my specific team I am even on the conversions from one role to the other. But I suspect that Google-wide there are more SETs who become SWEs.
Why convert in the first place? Well at Google it isn't for the money. It also isn't for the prestige as we have a lot more SWEs than SETs and it is a lot harder to standout. The scarcity of our SETs creates somewhat of a mystique about these folk. Who are these rare creatures who keep our code bases healthy and make our development process run so smoothly? Actually, most SWEs care more about making the SETs happy so they continue doing what they do. Why would any dev team force a conversion of a great developer from SET to SWE when finding a suitable SET replacement is so much harder than adding another feature developer? SWEs ain't that stupid.
Now pausing before I take another hit of the corp kool-aid, let me be honest and say that there are far more senior SWEs than SETs. Percentage wise we test folk are more outnumbered at the top of the org than at the middle and bottom. But keep in mind that developers have had a large head start on us. We have developers who have been at Google since our founding and testers ... well ... less time than that.
Where do TEs fit into this mix? TE is an even newer role than SET but already we have a number climbing to the Staff ranks and pushing on the senior most positions in the company. There is no ceiling, but the journey to the top takes some time.
Raghev among others has asked about the career path and whether remaining an IC (individual contributor) is an option over becoming a manager. I have mixed feelings about answering this. As a manager myself, I see the role as one with much honor and yet I hear in your collective voices a hint of why do I have to become a manager? Ok, I admit, Dilbert is funny.
For me, being a manager is a chance to impart some of my experience and old-guy judgement on less experienced but more technically gifted ICs. The combination of an experienced manager's vision and an ICs technical skill can be a fighting force of incredible power. And yet, why should someone who does not want to manage be forced to do so in order to continue their career advancement?
Well, fortunately, Google does not make us choose. Our managers are expected to have IC tasks they perform. They are expected to be engaged technically and lead as opposed to just manage. And our ICs are expected to have influence beyond their personal work area. When you get to the senior/staff positions here you are a leader, period. Some leaders lead more than they manage and some leaders manage more than they lead.
But either way, the view from the top means that a lot of people are looking to you for direction ... whether you manage them or not.
James,
ReplyDeleteLove the blog posts! Getting to know how Google Testing operates is extremely interesting for those of us outside the company. I'm looking forward to seeing what types of tools you've developed in-house and even which tools, if any, you've purchased.
I've noticed on the jobs boards at Google, Microsoft, etc. you don't see the more traditional "Quality Assurance" role for a manual tester, intead they have become Test Engineers. Can you explain more about how you find and use TEs? Maybe, if you know the differences, compare them to traditional QA role? Are TE's more than just manual testers or is it a terminology debate? Thanks
Chris
Thanks for giving insight into the SET role. It really did answer my question. Keep posting!
ReplyDeleteRaghav.
Thank you for all the interesting articles you publish on the Google Testing Blog. They are very helpful and insightful.
ReplyDeleteOne question:
"Where do TEs fit into this mix? TE is an even newer role than SET but already we have a number climbing to the Staff ranks and pushing on the senior most positions in the company. There is no ceiling, but the journey to the top takes some time."
Does this mean that TEs are also on the same pay scale as SETs and SWEs?
Best regards,
Johan
Full disclosure.... James is my manager however we have not been together long enough for me to be biased - well no much :)
ReplyDeleteMuch like James I have seen SET to SWE and SWE to SET conversions. Personally I dont like to focus on either role and lets not leave out PM to SET or SET to PM conversions. What you should ask yourself is what do you like to do? If you like to analyze software, take it apart and figure out how to make it better SET is perfect for you. If you like to architect, construct, and engineer software you'll likely find SWE better. Now keep in ming both roles do what the other does and toward the more senior levels the lines almost totally blur however the perspective approaches to the problem remain intact.
One thing I've noticed is more scientists (Physics, Chemistry, Math - like analysis, etc) tend to wind up in test. I think it comes from that fact testers like to understand why something works.
Tony Voellm
Hi James,
ReplyDeleteSince you have explained about SWE and SET, how about TE roles and its career path which you said as new role in Google? Keen to know more about the day-to-day life of TE.
:)
Wanna be a tester for google, that would be awesome... I've been tester of games and applications for 4 years.
ReplyDeleteCurious to please know the answer to Chris's question above re: TE's vs traditional QA.
ReplyDeleteThanks,
Jim Kahl
James's grad student FIT '98
Hi James,
ReplyDeleteI know you worked at Microsoft for quite some time as well. Can you talk about the difference between the SET and SDET Roles?
Keep posting good stuff like this.
Philip
good post , thanks ..
ReplyDelete