If you want to know when new articles go
online,
subscribe to the WebWord.com
Usability Newsletter!
Perceived
Information Architecture: User Feedback
Article by John
S. Rhodes
Abstract
The purpose of this article is to
explain how the Perceived Information Architecture test was used by BBC New
Media. The testing procedure is outlined, along with a list of pros and cons
of the method. Several actual user diagrams are provided.
The Email
In late January, Damian Rees, a
Usability Engineer at BBC New Media, told me that he was going to try using
the Perceived Information Architecture test
(PIA). I asked him to provide me with feedback when he was done. He was
kind enough to send me a detailed email message. Below
is the full text of that email (unedited), reprinted with his
permission.
To: "'John Rhodes'" <john@webword.com>
Subject: RE: PIA method
Date: Fri, 15 Feb 2002 13:48:02 -0000
Hey John
Well, we had a go at using the PIA - as
promised here's an outline of what we found.
Our aim was to understand users' mental
models for online community services (Chatrooms, celeb/expert Q &A
sessions, and messageboards). We conducted an unstructured usability
evaluation - what I mean by this is that we didn't want a traditional
usability study as we only wanted a snapshot of how they use online
community services. So we let them walk us through how they use email,
instant messengers, chatrooms and any other application/site they use
regularly. We then walked them through our site before they moved on to the
PIA method.
We wanted to gain an understanding of how
they conceptualise our services in relation to their current experience,
knowledge, preconceptions and so on. Tricky task using verbal/auditory
channel. So, they were asked to use one of those electronic whiteboard
thingies to draw on. We gave them the following scenario which is an
extension of one my usual post-test questions:
"imagine you are in a bar/coffee
shop/at a friends house and you are explaining what you saw today. They have
no experience of the web and just don't understand what you mean. So you
decide to draw it on a beer mat/napkin/scrap piece of paper. What would you
draw to show them what you mean?"
See the attached files for the output
ended up with (you might not be able to make much sense of them, but
hopefully you'll get the idea). [Editors Note: Diagram
1, Diagram 2, Diagram
3, Diagram 4.]
Pros :
- it really helped us get an
understanding of how they perceived the different services
- they did see a clear difference between
them.
- I didn't run any of these sessions, I
got the designers to run them. I found this really interesting as they
all saw the value of the method immediately.
Cons :
- some people just recreated our site -
rather than their interpretation of the services. I think this was
partly due to the fact that they didn't get a lot of exposure to our
site and therefore didn't fully use all the services. Also, due to
time, in a couple of sessions, we had to speed through the final
walkthrough in order to cover everything, so our descriptions probably
influenced their perceptions.
- some people just didn't know how to
start, and we didn't know how to get them started. Maybe we should
have given them some other task which got them used to drawing first.
I think also that the logical leap they had to make, from sitting at a
computer walking through websites with us, to standing in front of a
blank sheet being asked to perform a very abstract task threw some
people. Interestingly, one of our subjects was an artist and she had
the worst time out of all of them in getting started.
So, in summary. It was a hit, however I
think we need to think about how we introduce it to test participants. We
also need to think carefully about what we get them to do before, as what
they experience in the usability session is key. Because we weren't doing a
traditional usability study, they didn't have enough exposure to the
concepts they were asked to represent.
I hope this helps you with understanding the
different ways the PIA can, and is, being used by others.
I'll keep you updated with our future
findings
Regards
Damian
Damian Rees
Usability Engineer
BBC New Media
rm. 217 NW wing
Bush House, Strand
London WC2B 4PH
020 755 71615
Reflections
I first want to thank Damian
for providing us with this feedback and graciously allowing us to post it.
This benefits everyone. It is good press for BBC New Media because it shows
they care about usability. It is good for WebWord because we get to see how
our innovative usability testing method works for other people. It also
helps us refine our tools. Finally, it benefits readers because you get a
chance to see how the method works outside of WebWord, and our bias is
minimized.
I next want to talk about the
diagrams that Damian provided to us. You'll notice that the diagrams are not
pretty and seem very rough. However, they have some commonalities, and that
is a core purpose of the PIA method. The method helps experimenters see (and
hear) how the web site is the same from user to user. Another core point
about the diagrams is that they are useless outside of the testing context.
In plain terms this means that the descriptions users give as they are
drawing are crucial. The PIA method is a combination of user diagrams and
what they say.
Finally, I think Damian is
exactly right about the directions and getting people to start drawing. It
is a challenge to give users enough flexibility to do what they want while simultaneously
giving them some general rules to follow so that commonalities can be found
in the diagrams and explanations. This is something we are working on now to
make easier for experimenters.
Comments?
Please send them to me: john@webword.com
I want to know what you think about this article.
What next?
|
