# Presentation: 268015

## Usability Testing to Improve 			Your Website

**	****	****	****Colleen Blessing**
**	****	****Energy Information Administration**
**	****	****	****UNECE Work Session**
**	****	****Statistical Output for Dissemination to **
**	****	****Information Media, Voorburg, Netherlands**
**	****	****	****September 25-27, 2000**
- colleen.blessing@eia.doe.gov

## What does “usability” mean?

- Usability = the quality of user experience when interacting with something.
- Easy to learn
- Efficient to use
- Easy to remember
- Designed to minimize errors
- Satisfying to use.

## Why do we have usability problems?

- Technical = make it work
- Artistic = make it pretty
- Content = put everything up there
- Usability = make it usable to customers

## What is usability testing?

- Collecting information about users
- thinking process
- problem solving approaches
- navigation issues-where do they go?
- understanding the site
- feelings about the site
- measuring their success

## The actual usability test

- Observing and recording user actions and reactions (watch them work)
- What they like
- What they find easy
- Where they stumble
- “Scavenger hunt” test - timed exercises
- Think-aloud process
- Empathetic listening
- Audio/Videotaping

## First Steps to Usability Testing

- Training or reading
- Setting research goals
- Can users locate specific info on our site?
- Where do they have difficulty?
- Wrong “right answers”
- Use of footnotes, definitions
- Are they frustrated?
- Choosing an interview process and site

## Developing User Tasks

- Representative tasks based on the objectives of the site
- Not too obvious but not too difficult
- Broad representation across the site
- Questions in clear language
- Feedback from help desk specialists
- Test questions on yourself

## Moderator’s Guide

- Why they are here
- What they will be doing
- Encouragement to think aloud
- Speak in non-bureaucratic English
- Testing the site, not you
- Confidentiality
- Permission to audio/videotape

## Recruiting Interview Subjects

- Decide whom and how many you want to test
- Get the right mix
- Screening
- Overbooking
- Small thank-you fee ($25)
- Logistics

## Demographic Questions

- Level of expertise on the Internet
- Novice, intermediate, expert
- Familiarity with energy industry and terms
- Used our site before?
- Age

## Energy Information Administration Experience

- Collects and disseminates energy information
- Web site since 1995
- 400,000 user sessions per month
- 50,000 files
- Award winning but challenged
- No central web control

## Identification of Web Problems

- Calls to the help desk for web help
- Emails to webmaster
- Internal recognition of problems
- Publication/office centric vs. information centric
- Discussions at Web meetings

## How our usability tests work

- Subjects sit at a work station
- One interviewer in the room
- Observers behind one-way glass
- Four 10-minute exercises
- One hour total
- No help- “work as if you were in your office or at home”
- Probes to keep them talking

## Usability Test Results

- Pretest
- Group testing situation or 1:1?
- Tested senior management
- Actual Test #1
- Tested 17 subjects
- Most from Government

## More Results

- Too many buttons/choices
- Search engine failures
- Too much jargon
- Needed knowledge of publications
- Frustrated by long text files
- Low correlations between correct answers and familiarity with:
- energy terms
- our web site
- Internet

## Moving Results to Actions

- Ok, so now what do we do?
- JAD session with facilitator
- Developed a new design
- Based on input from key staff people
- “What goes where” exercise
- Parallel design
- Paper prototype exercise

## Usability Test of Prototype

- Tested 15 subjects in formal lab
- Broader representation of customers
- more serious recruiting/no screening
- Most users preferred our new home page
- Higher correlation between correct answers found and confidence in the answers
- Easier first button selection

## Prototype Test Results

- Clickable items didn’t look clickable
- Confusing navigation
- Search is good (Google)
- Generic categories not selected
- Contacts are critical
- Readers skim
- Exact text for repeat buttons
- Some text is too small

## Presentation of Results

- Support conclusions with facts, quotes, videoclips
- Invite staff, management
- Make recommendations
- Non-confrontational environment
- Classic resistance
- do it because you can do it
- the “stupid customer” syndrome

## Thoughts on Usability Testing

- Easier than you think
- Tiring but fascinating
- Pretest questions
- Maximum 6 per day
- Start with as few as 5 subjects
- Maximum 15 subjects
- Repeat tests after making changes
- Richer results than customer surveys

## Good Usability Sources

- Jakob Nielson.  *Designing Web* *Usability.  2000.*
- Jared Spool*.  Web Site Usability:  A Designer’s Guide.  *1997 and updates*.*
- www.**useit.com**  Jakob Nielson’s site**
- www.uie.com   Jared Spool’s site
- **  A must visit for usability information