If you want to know when new articles go
subscribe to the WebWord.com
by John S.
First, when you are
conducting a usability test, it is important to understand exactly what data
you should be collecting. You should not run a test without first deciding
on what data is required to address your business challenges. Plan ahead!
Second, in a usability test, you don't just watch users. You must collect
data that reflects how customers actually use your products and services.
This is easier said than done.
One of the core ideas of usability is that you need to
measure what people are actually doing. Sure, you need to watch them, but that is not enough. You have to understand what to record as they are using your web
site, product, or service.
Please remember that
you aren't really testing people. A usability test is not meant to
test intelligence! Instead, you are testing your products and services. You
are testing how people interact with your web site. Beware, if you start thinking that you are testing people, you will go down the path of blame. Before you know it, you
will blame users for problems with your technology. You will start to think that
customers are stupid, and that is something that goes completely against usability.
The failures you see and record are the failures of your web site.
Below is a list of metrics
that are based on software usability research (Mayhew, 1992; Nielsen, 1993; Whiteside, Bennet, &
Holtzblatt, 1988). Don't worry that the references are several years old.
The ideas are still very fresh, and useful. You can use them when testing almost any product or service, including cell phones, web sites,
or almost any other technology.
Percent of tasks completed
Ratio of successes to failures
Number of features or commands used
Time to complete a task
Time to learn
Time spent on errors
Percent or number of errors
Frequency of help or documentation use
Number of repetition or failed commands
Rating scale for usefulness of the product or service
Rating scale for satisfaction with functions and features
Number of times user expresses frustration or anger
Rating scale for user versus technological control of task
Perception that the technology supports tasks as needed by the user
There are certainly other things you could measure, such as shopping cart abandonment ratios. However, for the purpose of this article, realize that the metrics listed above are general and are meant to help you construct your own. I strongly recommend that you develop metrics that will help you answer the questions you developed when you gathered your business requirements.
If you collect data, just for the sake of collecting data, you will be doing yourself a great disservice.
Focus on solving business problems! Remember also that if you capture the problems that your customers truly face, then you will be able to help them. If you help them, then your company will prosper.
Before you go off and start your own usability testing, there are a few things to remember. I have only given a
tiny piece of the pie. You still need to gather requirements, get a representative sample of users, use the appropriate testing tools, do the data analysis, write the report, and develop the strategy. Also, how you do your
testing depends entirely on the overall business goals of the company.
One of the main points of this article is that usability testing is much more than simply watching users. I estimate that "just watching users" will net you about
20-25% of your core usability problems. That is pretty damn good, but it
also means that you do not know
the 75-80% remaining usability problems! To really understand your customers, and to truly take on your business challenges, then you need to think through your whole research approach. Core usability problems will slip by without allocating resources and paying attention to the entire usability testing cycle.
If you would like to rate
this page or add a comment for others to read, click