User Experience Smackdown: Usability Testing Vs. User Testing
What's the difference between user testing and usability testing? Contributor Shari Thurow explains the distinction.
Throughout my user experience (UX) career, I have encountered many people who honestly, truly believe that their company conducts usability tests on a regular basis. But when I probe for details, I quickly learn the truth.
For example, I was hired to produce wireframes and advanced prototypes for the health section of a major brand’s website. After conducting the appropriate card sort tests to determine the optimal global, local, and contextual navigation, I presented my usability-tested wireframes to the new health manager.
The health manager essentially trashed all of my wireframes and test results because she claimed to have regularly conducted A/B and multivariate tests on another health website with great success. I have seen the end result over the years. Here’s a summary:
- The site’s information architecture and corresponding navigation system is based on date and pagination. In reality, patients and patient caregivers do not organize and health information by date and page number.
- Supplemental navigation is provided “By Topic” — but topics are not organized in a way that makes sense to users. The topic labels are paginated, and topic labels are not presented in any particular order. Heck, even alphabetizing the topics would make them easier to locate and discover.
- Not surprisingly, the health site does not rank well at all (on Google, Yahoo, and Bing) for most of their targeted keyword phrases.
In another scenario, I was at an event where I provided site reviews for usability and the user experience. One attendee claimed that his company worked with multiple firms that offered user testing. However, when I looked at the website, it only had one page. There was no site navigation…only a form to fill out. The page did not provide any benefit to users for filling out the form.
I realized the problem: the term user testing. Somehow, throughout the years, the term user testing has become synonymous with usability testing. Well, Marketing Land readers, I am here to tell you that they are not the same thing. Just because a person or company conducts A/B tests, multivariate tests, and focus groups does not mean that the company is a usability firm.
That previous sentence is a rather bold statement, isn’t it?
List Of Usability Tests
Usability is about contextual task completion. Website usability isn’t about one’s personal opinion. Website usability is not about a “coolness” or “wow” factor. It’s not about forcing your personal beliefs or design preferences onto users. It’s about task completion.
The ISO defines usability as:
The effectiveness, efficiency and satisfaction with which specified users achieve specified goals in particular environments.
User satisfaction is directly related to task completion. If users can complete their assigned tasks quickly and easily, they often report higher satisfaction, especially if there is an element of delight in the interaction. If users have a difficult time completing their tasks or cannot complete them at all, they often report low satisfaction.
To determine user mental models and the effectiveness of a website or application, usability professionals conduct ongoing usability tests. I don’t use the term testers when I refer to usability test participants.
Reason? What is being evaluated is the website or application, not the person participating in the usability test. If a test participant has a difficult time completing a task, the problem lies in the interface. Perhaps the labels are confusing. Perhaps the functionality is inconsistent from page to page, screen to screen.
Below is a short list of usability tests that professional usability firms typically conduct:
|USABILITY TEST||MEASURES||FORMATIVE or|
|Brand Perception Test||Brand message||Formative|
|Closed Card Sort Test||Organization of content; whether or not navigation labels make sense to users; user mental models||Formative|
|Expectancy Test||Initial mental model||Formative|
Research shows that where test participants look depends on what you have asked them. Please see Eyetracking Studies – 7 Traps to Avoid.
|5-Second Usability Test||Initial mental model, aboutness, keyword focus||Formative|
|Free Exploration Test||Actual usage||Summative|
|Functional Salience Test||Importance of functions||Summative|
|Open Card Sort Test||Organization of content||Formative|
|Performance Test||Navigation design||Formative|
|Performance Test||Organization of content, layout, task flow, controls and content||Summative|
|Sample-of-One Test||Roadblocks or errors in interface layout, labels, content organization, affordance, and functionality||Summative|
|Visual Affordance Test||Clickability/Tappability||Formative|
Why am I on a rant about usability testing vs. user testing? I listen to numerous sales pitches. I hear people claim that their site does ongoing testing without specifying the types of tests that are conducted. I observe people using the wrong methods to collect the wrong data, such as using keyword research tools to determine a website’s information architecture.
Use this list to help you determine whether or not you are working with a true usability professional or organization. Usability is a critical facet of the user and searcher experience. You can delight or entertain your website users all you want, but if they can’t complete their desired tasks? If they can’t locate or discover desired content? That is not much of an experience.
Okay, end of rant.