Study: Google Home has more answers than Amazon Echo

Stone Temple Consulting asked both devices 5,000 questions to see which one is smarter. Google Home won, but how much does it matter?

Chat with MarTechBot

Echo Vs Home 4 1920

Fresh off its recent voice search study, Stone Temple Consulting compared the relative accuracy of Google Home and Amazon Echo. The SEO firm posed 5,000 questions to each unit “to see which device was smarter.”

In its discussion of results, the firm makes clear that the test was not a comparison of the full range of virtual assistant capabilities. Rather it sought to determine which device more often gave a correct answer to a wide range of factual questions. The specific list of questions wasn’t provided.

Percentage of 5,000 questions device attempted to answer

Screen Shot 2017 03 07 At 7.08.25 PM

Source: Stone Temple Consulting

There were some scoring nuances that Stone Temple describes in its own blog post, which I won’t reproduce here. As one might anticipate, however, Google was able to answer many more questions than Echo/Alexa. That’s consistent with my personal experience and makes sense given Google’s search index.

Google Home answered more than three times the number of questions that Amazon Echo could. The surprise is that Amazon Echo had nearly as good a percentage score for those questions it was able to answer.

Of the roughly 3,400 questions Google Home answered, 89.5 percent were correct, according to the study. By comparison, Amazon Echo got right roughly 87 percent of the just over 1,000 questions it was able to answer.

Percentage of questions answered correctly 

STCecho

Source: Stone Temple Consulting

For Stone Temple Consulting, Google Home was the clear winner. However, it’s not clear to me that consumers are drawn to these devices simply to replicate the search experience or to answer factual questions.

Ultimately, they’re entertainment devices and transactional tools to help people accomplish tasks such as ordering takeout, renting a car, buying an iPhone charger, making dinner reservations, creating shopping lists and so on.

I don’t think the battle of the intelligent assistants will be won or lost on the basis of index size. Rather a range of considerations, including general utility, cost and aesthetics (design and UX) will be significant factors. In these other categories, Amazon has more than a fighting chance, especially if it can build out a rich developer ecosystem.


Contributing authors are invited to create content for MarTech and are chosen for their expertise and contribution to the martech community. Our contributors work under the oversight of the editorial staff and contributions are checked for quality and relevance to our readers. The opinions they express are their own.


About the author

Greg Sterling
Contributor
Greg Sterling is a Contributing Editor to Search Engine Land, a member of the programming team for SMX events and the VP, Market Insights at Uberall.

Fuel up with free marketing insights.