An Inside Look At What Google’s Search Quality Raters Do

One of the long-standing myths about Google’s search results is that they’re completely computer-generated with no human input or involvement whatsover. But, since at least 2005, Google has had a program that involves regular searchers reviewing the quality of their search results.

There are several different job titles involved, but the people who get paid to review Google’s search results are commonly called Quality Raters.

Their work has largely been a mystery over the years, but over on Search Engine Land, we’ve just published what might be the first of its kind — an interview with a Google Search Quality Rater.

It’s a lengthy Q&A, but you’ll likely find it worth the time if you’re interested in search marketing. Some of the key takeaways:

• Raters are only hired after passing two tests, the second of which includes rating more than 140 different Google search results and demands a 90 percent score in order to pass.
• Each rating task is completed by multiple raters and, if they disagree in their ratings of Google’s search results, there’s a formal discussion and debate process that may involve higher-level moderators making the final decision on who’s correct.
• Raters don’t only look at Google’s organic results, they may rate Google’s paid search results as well as search results from News, Maps and other Google “universal search” products.
• Raters may also have tasks that involve grading the design and layout of specific web pages. One of the key considerations in those tasks involves the balance between content and ads on a page. (That’s reminiscent of Google’s new page layout algorithm that was just announced on Thursday.)
• Tasks are not tailored to the Rater’s skills or interests.

That’s a taste of what the interview covers. You can read the entire piece on Search Engine Land: An Interview With A Google Search Quality Rater.