WebWord.com Newsletter Issue #64

Subscribe to the WebWord.com Usability Newsletter!

ISSUE #64

==================================================
                WebWord.com Newsletter
           "Industrial Strength Usability"
==================================================
                 Newsletter Editor
                   John S. Rhodes
                     John@webword.com
..................................................
December 27, 2001                   Newsletter #64
..................................................

Table of Contents


1.  Triangulation in Usability Experiments
2.  Understanding Design Misfits
3.  The Perceived Information Architecture Test (PIA)
4.  Requirements Engineering Portal
5.  Wayback Machine: WebWord
6.  Deep Linking, WebWord, and The Vanguard Group
7.  Metafilter Text Ad Experience
8.  WebWord Hot Sites and Newsletter Archive
9.  What Can You Find at WebWord.com?

      
5260 subscribers


Tell a friend. Pass this along to
        a colleague. Recommend WebWord now!
--> http://webword.com


Need to subscribe?
--> http://webword.com/subscriptions.html



- - - - - - - - - - - - - - - - - - - - - - - - - -
ITEM #1

Editorial: Triangulation in Usability Experiments


In recent months I have started to really worry about
things people are writing about usability. So much of
what is written is not based on research. People in
the media are also very loose with their explanations
and depictions of usability. They throw around big ideas
while they provide very few details. In usability, the
details matter. It is a very bad idea to make gross
generalizations and it is a bad idea to throw around
ideas that are not well documented or supported.

But it happens all of the time.

We recently finished a rather large research project
for a major catalog retailer. We performed several
empirical tests that seemed redundant to our clients.
We gently explained that no single usability test can
cover all of the various angles, and no single test can
give you all the data you need. These are true statements
but more importantly, by running several tests, you get
what we call "converging evidence".

In plain terms, by running several tests that overlap
we find the same things happening again and again. We
see the same kind of issues and problems. We see trends
from test to test. We triangulate!

We see things that are different too, but that is
what too many people focus on...

The key is to realize that you need to run more than
one test if you want to be sure you are capturing
major problems. One simple test won't cut it. If you
really want to be sure you are addressing core issues,
I strongly suggest that you test using more than one
method.

There are similar ideas that you should consider. First,
you could two tests. That is, you can run the same test,
but run it twice, perhaps using entirely different
user experience specialists. Or, maybe you could run the
same test again, but change the scenarios or testing
materials in some minor way.

If you run your own experiments, think about these things.
If you employ usability consulting firms, such as
WebWord, your usability investment can be protected by
asking for multiple tests, particularly multiple types
of tests.

Think about it. Some people are good at True/False, others
are good at Multiple Choice, and still others are great
with Essays. The way you test matters. The number of
times you test also matters. Like it or not, you need
some level of redundancy in your testing. You need
triangulation to find different problems, but more
importantly, you need it to find the same problems
over and over.

Overkill is overkill, but overkill is also certainty.

Keep in mind that discount usability testing methods save
money, but they can't do the full duty. Instead, think
about how cheap methods can augment full testing. Unless
you are truly on a small budget, use discount methods to
double check the results of the more extensive methods.

If you have the budget and the time, you really should
demand multiple tests.

Cheers,

- John

p.s. My quick sales pitch: If you need help with your
usability research, contact us. We know what we are
doing, and we know what others are doing wrong. Protect
your investment with solid empirical research. 

John S. Rhodes
WebWord.com -- Industrial Strength Usability
http://webword.com
john@webword.com



- - - - - - - - - - - - - - - - - - - - - - - - - -
ITEM #2

A new WebWord.com article is now online:
--> http://webword.com/moving/rhodes0002.html


"Understanding Design Misfits"

The purpose of this paper is to briefly discuss design
misfits. A cursory overview of misfits is provided,
along with a working definition. Eight types of design
misfits are revealed. Three examples of each type of
misfit are provided for clarification.


You can read the new article here:
--> http://webword.com/moving/rhodes0002.html



- - - - - - - - - - - - - - - - - - - - - - - - - -
ITEM #3

A new WebWord.com article is now online:
--> http://webword.com/moving/pia.html


"The Perceived Information Architecture Test (PIA)"

The purpose of this article is to describe WebWord's
Perceived Information Architecture test. We explain
how the test is conducted, when it should be used, and
what value it provides.

 
You can read the new article here:
--> http://webword.com/moving/pia.html



- - - - - - - - - - - - - - - - - - - - - - - - - -
ITEM #4

A new WebWord.com resource is now online:
--> http://webword.com/services/rp.html


"Requirements Engineering Portal"

WebWord's Requirements Engineering Portal is a
collection of links about requirements engineering,
requirements management, systems science, use-case
scenarios, requirements engineering software, and
methods and tools related to requirements engineering. 


You can view the new resource here:
--> http://webword.com/services/rp.html



- - - - - - - - - - - - - - - - - - - - - - - - - -
ITEM #5

Wayback Machine: WebWord

If you are curious to see how WebWord has evolved over
the last several years, check out the Wayback Machine:

--> http://web.archive.org/web/*/http://webword.com


This is a service of the Internet Archive. In their
own words: "The Internet Archive is building a digital
library of Internet sites and other cultural artifacts
in digital form." I think it is rather slick. When I
have some spare time, I am definitely going to take
some screenshots of WebWord in case the archive is
taken down or lost.



- - - - - - - - - - - - - - - - - - - - - - - - - -
ITEM #6

Deep Linking, WebWord, and The Vanguard Group

I never thought that I would ever be involved in any
sort of deep linking debacle. I never think much about
linking -- I just find news and link to it. I thought
that since everyone benefits, no person or company
would complain or be upset by my linking.

I was wrong.

On 9-Nov-2001 Lorraine Bayler of the Vanguard Group
sent me an email that really shocked me:

--> http://webword.com/readers/bayler.txt

In effect, in her email she told me that I could still
deep link to the Vanguard Group's web site, but with some
very silly constraints. When I posted this on WebWord,
several people commented (they were incredulous), just as
I expected:

--> http://webword.com/weblog/00000601.html

It has been well over a month since this issue came
about but it still makes me sad. It is so frustrating.
I merely want to point to news that I find. I'm not
trying to do anything else and I'm not looking to do
any harm. I certainly don't want to fight.

What a shame.



- - - - - - - - - - - - - - - - - - - - - - - - - -
ITEM #7

Metafilter Text Ad Experience

On 23-Oct-2001 I decided to set up two text banner ads
on Metafilter. A text ad is a simple idea. You pay a
web site some money to post your advertisement on their
site. Much like a banner ad, it gets rotated along with
other advertisements. The catch is the ad is only text;
no graphics allowed!

I think that Google started this trend. If nothing else,
they are best known for this kind of advertising. If you
are curious about it, go here:

--> https://adwords.google.com/AdWords/Welcome.html

You'll see that I left a few things out since Google does
more than simple post your text. They do some other
fancy stuff that I am not going to discuss here.

In any event, Metafilter started posting text ads. I
decided to jump on the bandwagon. I wanted to see how
their system worked and I wanted to support Metafilter.

So, here is what I did. I posted TWO advertisements,
both pointing directly to the WebWord home page. Here
are the ads (as best as I can reconstruct):


AD ONE
Click Here, Punk (header)
Usability, usability, usability,
usability, usability, usabil...

AD TWO
WebWord.com Usability (header)
Usability news, articles, interviews
newsletter, and more.


Both ads ran for 5,000 impressions. That is, they were
displayed 5,000 times each over the course of about
2 weeks.

Which ad did better? Not surprisingly, AD ONE did much
better (5.06% click through) than AD TWO (1.98% click
through). AD ONE was fun, innovative, and catchy. A couple
of people even sent me comments about the ad -- they
thought it had an edge, but they liked it.

I paid $10 for 5,000 impressions (a real deal). However,
even at this low price, and even for the better ad, I
paid about $0.05 per click through to WebWord. For some
people, that would be great and the payoff would be
excellent. But for WebWord, this is probably not
something I will do very often. We shall see.

If you are curious about Metafilter's text advertising,
go here to learn more:

--> http://www.metafilter.com/textads.mefi?webword

(Tell Matt that John said hello!)



- - - - - - - - - - - - - - - - - - - - - - - - - -
ITEM #8

WebWord Hot Sites and Newsletter Archive

I made two recent updates to WebWord. First, I finally
made some changes to the Hot Sites page. I made it more
friendly (alphabetical order) and I also added a short
description of the site. Second, I updated the WebWord
Newsletter Archive. If you missed any of the previous
63 issues (of this newsletter!), visit the archive now.

WebWord Hot Sites
--> http://webword.com/hotsites.html


WebWord Newsletter Archive
--> http://webword.com/archive/index.html


Enjoy!!



- - - - - - - - - - - - - - - - - - - - - - - - - -
ITEM #9
   
What Can You Find at WebWord.com?


WebWord.com Services
>> http://webword.com/services/index.html

Articles (Moving WebWord)
>> http://webword.com/moving/

Expert Interviews
>> http://webword.com/interviews/

Career Center
>> http://careers.webword.com/

Recommended Books
>> http://webword.com/books/booksindex.html

Newsletter Archive
>> http://webword.com/archive

Weblog Archive
>> http://www.webword.com/weblog

Usability Reports
>> http://webword.com/reports

Recommended Web Sites
>> http://webword.com/hotsites.html



----- End of WebWord.com Usability Newsletter #64 -----

      (c)2001 by WebWord.com and John S. Rhodes.
                 All rights reserved.

Go Back to the WebWord.com Newsletter Archive