If you want to know when new articles go
online,
subscribe to the WebWord.com
Usability Newsletter!
Usability
Metrics
by John S.
Rhodes
Summary:
First, when you are
conducting a usability test, it is important to understand exactly what data
you should be collecting. You should not run a test without first deciding
on what data is required to address your business challenges. Plan ahead!
Second, in a usability test, you don't just watch users. You must collect
data that reflects how customers actually use your products and services.
This is easier said than done.
Background
One of the core ideas of usability is that you need to
measure what people are actually doing. Sure, you need to watch them, but that is not enough. You have to understand what to record as they are using your web
site, product, or service.
Please remember that
you aren't really testing people. A usability test is not meant to
test intelligence! Instead, you are testing your products and services. You
are testing how people interact with your web site. Beware, if you start thinking that you are testing people, you will go down the path of blame. Before you know it, you
will blame users for problems with your technology. You will start to think that
customers are stupid, and that is something that goes completely against usability.
The failures you see and record are the failures of your web site.
Below is a list of metrics
that are based on software usability research (Mayhew, 1992; Nielsen, 1993; Whiteside, Bennet, &
Holtzblatt, 1988). Don't worry that the references are several years old.
The ideas are still very fresh, and useful. You can use them when testing almost any product or service, including cell phones, web sites,
or almost any other technology.
Usability Metrics
Effectiveness
-
Percent of tasks completed
-
Ratio of successes to failures
-
Workload
-
Number of features or commands used
Efficiency
-
Time to complete a task
-
Time to learn
-
Time spent on errors
-
Percent or number of errors
-
Frequency of help or documentation use
-
Number of repetition or failed commands
User Satisfaction
-
Rating scale for usefulness of the product or service
-
Rating scale for satisfaction with functions and features
-
Number of times user expresses frustration or anger
-
Rating scale for user versus technological control of task
-
Perception that the technology supports tasks as needed by the user
Brief Discussion
There are certainly other things you could measure, such as shopping cart abandonment ratios. However, for the purpose of this article, realize that the metrics listed above are general and are meant to help you construct your own. I strongly recommend that you develop metrics that will help you answer the questions you developed when you gathered your business requirements.
If you collect data, just for the sake of collecting data, you will be doing yourself a great disservice.
Focus on solving business problems! Remember also that if you capture the problems that your customers truly face, then you will be able to help them. If you help them, then your company will prosper.
Before you go off and start your own usability testing, there are a few things to remember. I have only given a
tiny piece of the pie. You still need to gather requirements, get a representative sample of users, use the appropriate testing tools, do the data analysis, write the report, and develop the strategy. Also, how you do your
testing depends entirely on the overall business goals of the company.
One of the main points of this article is that usability testing is much more than simply watching users. I estimate that "just watching users" will net you about
20-25% of your core usability problems. That is pretty damn good, but it
also means that you do not know
the 75-80% remaining usability problems! To really understand your customers, and to truly take on your business challenges, then you need to think through your whole research approach. Core usability problems will slip by without allocating resources and paying attention to the entire usability testing cycle.
References
If you would like to rate
this page or add a comment for others to read, click
here.
What next?
|