Usability testing

Usability testing is a crucial component of user experience design, and helps close the gap between the designer’s intentions and the user’s expectations. Whether positive or negative, the user’s feedback can expose what needs to be fixed. With the right data, you can better understand how to improve the user interface, design, navigation, and more.

What is usability testing?

Usability testing is a set of methods for evaluating a product or service by testing with representative users, then iterating based on direct feedback. Since no one can design the perfect product from the get-go, it’s a good way to expose flaws and gain new insights. It’s used for testing prototypes, websites, apps, devices, and other products and services.

Conducting a usability test

In-person tests allow you to capture nuances in behavior and expressions, and encourage lively post-test discussions. There’s also a greater opportunity to establish trust and rapport, which can lead to better results. When geographical distance prohibits in-person testing, remote sessions allow you to reach a higher number of users in more locations.

Depending on the location and level of expertise required, representative users can be difficult to recruit for in-person testing. Although putting the product in front of more users is a good idea, it’s proven more efficient to test each iteration with just five users. After six users, the cost-benefit ratio is greatly diminished. That is: Testing five users at a time and repeating the test is recommended over testing a single version with 10 users.

Usability testing methods

The most common usability testing methods include the concurrent think aloud, retrospective think aloud, and various remote options. They follow a general prepared script that begins when the moderator introduces the test to the user, and reassures him/her that it’s not a test of their ability, but of the system. The audio or video is recorded for future reference.

If a moderator is present for the test, he/she should not bias the user by favoring one choice over another with comments or facial expressions, or do anything to help the user complete the task. For example, saying “Scroll down, it’s just right there!” is helpful to the user but counterproductive to the test.

The system being tested does not need to be a fully-functioning website or application. When used earlier in the process, usability tests can help clear major roadblocks, and systems can be as simple as a paper sketch; later down the line, they’re helpful to expose more detailed usability problems with more complex prototypes.

After each round of testing, the findings are analyzed, and recommendations are made to inform the next version of the product.

Concurrent think aloud

This is the most traditional type of usability testing. The user is asked to verbalize their reactions, thoughts, feelings, and opinions as they perform a set of tasks. It’s often done in a lab, with one or two other people: A moderator observes and listens, with an optional notetaker. Although many more team members can observe the session, it’s best to keep the testing room limited to three since a bigger group can be intimidating to the user.

Retrospective think aloud

After the user completes a set of tasks, the system is reviewed with the user (or a recording can be played back). During the review, the user is asked to recall his/her thoughts, opinions, and reactions.

Remote unmoderated test

There is no moderator. Tasks are pre-selected, and instructions given through a website or app. The user is asked to verbalize their thoughts as they’re going through the system. Since there’s no moderator, there is less control and a greater risk of user confusion. The tasks should be fairly simple and quick to perform.

Remote moderated test

The user is asked to verbalize his/her reactions, thoughts, feelings, and opinions as they occur while they’re using the system. Thanks to screen-sharing tools, a moderator can interact in real time with the user.

How can we measure usability?

Usability can be measured behaviorally and attitudinally. ISO defines usability and user experience design as “designing products to be effective, efficient, and satisfying.” Let’s break down each of these factors:

1. Effectiveness: Did the users achieve their goals?

Ways to measure: errors rates, quality of completion, and accuracy

1. Efficiency: How much energy did it take for users to get there?

Ways to measure: time to task completion

1. Satisfaction: What are the users’ attitudes toward the system?

Ways to measure: Likert scale rating of satisfaction, verbal opinions

As an example, let’s say Jane filled out a form on a mobile application accurately, but struggled with reading it. Although she may have completed her task successfully (good effectiveness), she didn’t like it because the font was barely legible without her glasses (low satisfaction). In this case, it’s helpful to capture both objective and subjective data in order to fully understand the situation.

User testing vs. usability testing

You might have heard the terms “user testing” and “usability” get thrown around in product discussions frequently, with a lot of confusion about what kind of testing should actually occur. While they’re often used interchangeably, they play distinct roles in the UX design process.

User testing describes the research done to assess user needs and validate an idea before moving forward. You would usually do this during the problem discovery phase. You might have a new product idea, or simply want to understand how your users are currently solving their problems. It’s helpful when the question you have is, “Will the user use my product/service?”

User testing is done in order to:

• Understand user needs
• Validate product ideas
• Generate concepts

As a part of its methods, user testing can involve:

• Surveys
• Interviews
• Card sorting
• Heuristic evaluation
• Contextual inquiries
• Focus groups
• Quantitative methods such as A/B testing

Usability testing is used during the evaluative phase when an existing product is in any stage of development. This type of testing uncovers product issues and helps improve functionality. It’s important to note that usability testing is done once you’ve grasped the problem, and the concept has taken some shape or form. It helps answer, “Can the user use my product/service?”

Usability testing is done in order to:

• Evaluate a product/service
• Improve functionality
• Test a concept or product against user expectations

User testing can be useful in generating new ideas and is better for understanding user needs in context. Usability testing is used during the evaluative phase and is better for working towards an improved implementation of a solution.

This is a general overview of a complex subject, however, and it’s not to say that certain methods can only be used during either phase. There can be overlap, and usability and user testing can have shared methods.

Both types of testing are used to understand the user’s perspective, including expectations, behaviors, and opinions. They’re complementary to one another in the design process, and help create useful, usable, and delightful user experiences.

Don’t leave success up to chance

Usability testing is vital, but it might sound daunting to add more voices to your team’s discussion and process. Fortunately, InVision empowers your team to include users in a friction-free way—and early.

With InVision, it’s easy to share prototypes with people—both in and outside of your organization—in order to quickly gather meaningful usability feedback for effective iterations. From early low-fidelity freehands to true-to-life high-fidelity Studio screens, the feedback is there every step of the way.