Website Usability Guidelines

About Usability Testing (2of 6)

You might also have more specific questions. For example, if one of your concerns for this round of testing is how well the search function works for users, you might focus on these questions:

• Do participants click to pages or do they use search?
• What words do they use most when searching?
• Is the search box in a good location and is it large enough for most of the words used?
• Do the search results provide answers to users' questions?
• When search results do provide answers, are the answers usually on the first page of results?
• Does the search do a good job of detecting and helping to resolve typing errors?

What should you keep in mind when usability testing?

(1) You are testing the site not the users.
For some users the term testing has a negative connotation. We try hard to ensure that participants do not think that we are testing them. We help them understand that they are helping us test the prototype or website. In fact, don't use the term testing with participants at all. Instead, invite participants to help by trying out the prototype.

When users have difficulty completing a task, we fix the website, not the users. Both out loud and in your thoughts, ask "How well does the site allow you to meet their goals?"

(2) Rely more on what you learn about performance than preference. We can measure both performance and preference. Performance measures include success, time, errors, etc. Preference measures include users' self-report of their satisfaction and comfort.

Some designers believe that if they design to meet users' preferences, the site will enable users to perform well. The evidence does not support that.

In fact, peoples' performance and preference do not always match. One study found that about 70% of users had performance and preference measures that agreed with each other. That is, they either performed well and liked the website or performed poorly and disliked the site.

However, this leaves a relatively large percentage of people (30%) for whom performance and preference measures did not agree. They either performed well and disliked the site or performed poorly and liked the site.

Various reasons have been proposed for why people often rate a site more highly than their performance would lead us to expect. They may blame themselves for the problems that they have, rather than the site. They may think they would be hurting our feelings by giving the site a low score. They may not be aware of the problems they had and think they were successful when they were not. All of these reasons support the recommendation to rely more on what you learn through performance measures than through preference measures.

(3) Make use of what you learn. Usability testing is not just a milestone to be checked off on the project schedule. A usability test is not finished when the last participant leaves. The team must consider the findings, set priorities, and change the prototype or site based on what happened in the usability test.

(4) Try to find the best solution given the reality of your many users. Creating any product, including most websites and Web applications, requires considering many different users with different ways of working, different experiences, and different questions and needs. Most projects, including designing or revising websites, have to deal with constraints of time, budget, and resources. Balancing all those is one of the major challenges of most projects.

As you weigh constraints and trade-offs, push for creating the website or Web application that will enable the greatest percentage of your users to successfully answer their questions and complete their tasks. Research shows that the cost of supporting unsuccessful users after a product is launched is far greater than the cost of fixing the product to meet users' needs before it is launched.

As you consider your personas, their scenarios, and what you learn in usability testing, try to find the ideal solutions to your design problems. Settling for less than the best means having users be less successful than they otherwise would be. Evidence shows that even after extended use and experience, when they have a less than optimal product to work with, users never achieve their goals they would have if they had had the better interface.