# Presentation: 760843

## Find out what they like, and how they like it, And let ‘em have it just that way.  			Fats Waller

**Usability Testing 101 **
**Gina Pearson, Web Manager**
- January 29, 2003

## Today’s Goals

- Primary purpose is to learn how ERS is integrating usability testing into its Web design & development process
- Review why user-centered design and testing (usability engineering) are important
- Explain the usability testing process at ERS

## Today’s Goals

- You will learn how to:
- Assemble your test team
- Recruit & select test participants
- Formulate a test plan
- Create testing tasks & scenarios
- Facilitate a test session
- Test without a lab
- Record, compile & analyze results
- Communicate results effectively

## Why Is Usability Important?

## Ballot Usability

- What should have been addressed?
- Font size
- Lighting
- Text vs. background color
- Instructions
- Layout
- Act of voting (punching ballot card)

## Ballot Usability

- Costs
- $20,000 to run usability tests addressing
- Layout
- Multiple votes
- Dimpled ballots
- About $1 billion spent by both candidates to try to get elected

## Failure of Web Sites

- Users could find information only 42% of time (15 large commercial sites - Spool study)
- Sixty-two percent of web shoppers gave up looking for an item (Zona study)
- Only 51% of sites complied with simple Web usability principles (Forrester study of 20 major sites)

## User-Centered Design

**Philosophical & Practical Approach**
- User-centric -  ***not*** developer, designer nor organization-centric
- Who are the users?
- What information do **they** want?
- Based on research and data ***not*** opinions
- Testable & verifiable

## The Traditional Development Process

## User-Centered Design

- A methodical approach to producing a web site or any user interface. It is a practical and systematic way to deliver a product that works for users.
- Test early & often
- Build a little, test a little, fix a little
- At ERS, we are incorporating usability testing & other user assessment techniques as a regular part of our web design & development process

## User-Centered Design

- Types of user assessment techniques
- Interviews
- Ethnographic observation / field research
- Focus groups
- User personas
- Expert (heuristic) reviews
- Usability testing
- Customer satisfaction surveys
- Customer email trend analysis
- Search engine query analysis
- What else?

## Myths About Usability Testing

- Usability testing:
- Is time consuming, expensive & otherwise resource intensive
- Typically involves hiring a specialized consulting company
- Requires a formal testing lab
- Must be videotaped
- Always involves writing a detailed analysis and report of results

## Diagnostic User Testing 

- Find out what is and is not working well on the site. In a diagnostic usability test, you usually want to answer questions like these:
- Do users complete a task successfully?
- If so, how fast do they do each task? Is that fast enough to satisfy them?
- What paths do they take in trying? Do those paths seem efficient enough to them?
- Where do they stumble?— What problems do they have?— Where do they get confused?
- What words or paths are they looking for that are not now on the site?

## Methods

- Assemble test team
- Administrator
- Facilitator
- Observers / recorders
- Video recordings operator (optional)

## Methods

- Define goals & scope of what you are testing
- Recruit & compensate participants
- Develop testing tasks & script
- Set up testing environment
- Draft participation agreement
- Rehearse & refine test protocol
- Conduct tests
- Analyze & report out results

## Methods

- Test Environment

## Methods

- How many participants are needed for a usability test?
- It depends!
- Typical range is from 5 -12 users in each test
- Our approach:  3 users per test but conduct several rounds of testing throughout an application or feature’s development
- If you have different potential user groups (for example, small business owners, unemployed, blue-collar workers), try to include representatives of all these groups

## Methods

- Participant recruitment & compensation
- Typically time and labor intensive - consider outsourcing (average $200 per participant)
- Go through professional networks & contacts for volunteers
- Conduct testing at conferences & other events with volunteer representatives from your target audiences
- Always give volunteers a small token of your appreciation (t-shirt, mug, etc)

## Methods

- Task construction:  where do ideas for tasks come from?
- Concerns about information architecture, design, terminology, functionality, etc that have surfaced among the product development team
- Trend analysis of emails from site customers
- Search engine query analysis
- Trend analysis of call center inquiries
- What else?

## Methods

- Constructing usability testing tasks
- Find or scavenger hunt tasks
- Verb-based tasks
- Scenario-based tasks

## Methods

- Conducting the Test (about 2 hours)
- Pre-test briefing for observers
- Greet participant & administer participation agreement
- Introduce recorder & observers
- Explain testing process to participant
- Administer pre-test interview & activities (if any)
- Facilitate testing tasks
- Administer post-test interview questions
- Involve observers in general Q&A session at the end
- After participant leaves, debrief recorder & observers

## Methods

- Developing the testing script & facilitating the test
- 3 parts of a usability test:
- Pre-test interview
- Testing tasks (“think out loud” methodology)
- Post-test interview
- See another sample script at:
- _[http://www.sensible.com/Downloads/script.doc](http://www.sensible.com/Downloads/script.doc)_

## Methods

- Characteristics of a Good Facilitator
- Quick learner
- Ability to establish instant rapport
- Empathic “people person”
- Good listening skills
- Patient
- Flexible & adaptable
- Long & focused attention span
- Impartial:  not wedded to a particular design

## Methods

- Analyzing & reporting the test results using the KJ method
- Assemble your testing team immediately after the last test for a brainstorming session.
- Ask each participant to write down each major observed problem on an index card or sticky note.
- Put all index cards or sticky notes on a large wall or board, no discussions!
- Read each others cards silently, add additional problems.

## Methods

- Analyzing & reporting the test results using the KJ method
- Sort problems by area.  Eliminate duplicates if total agreement.
- Name each group of problem areas.
- Vote for most important problems, each participant has ten votes.
- Assign one team member the task of writing up the results of brainstorming session.
- Disseminate results of brainstorming session to development team via email

## For Further Information

- Contact:
- Gina Pearson, Web Manager
- 694-5119
- Brenda Powell, Web Analytics Manager
- 694-5134

## For Further Information

**Don't Make Me Think! A Common Sense Approach to Web Usability**
- Steve Krug
- The book every manager should read!
**Handbook of Usability Testing:  How to Plan, Design & Conduct Effective Tests**
- By Jeffrey Rubin
- Practical, step-by-step guidelines, requires no engineering or human factors training.
**A Practical Guide to Usability Testing**
- By Joseph Dumas & Janice Redish
- Premier text in the field.

## For Further Information

- Jakob Nielsen (www.useit.com)
- Why You Only Need to Test With 5 Users
- http://www.useit.com/alertbox/20000319.html
- Success Rate: The Simplest Usability Metric
- http://www.useit.com/alertbox/20010218.html
- Usability Metrics
- http://www.useit.com/alertbox/20010121.html

## For Further Information

- Jakob Nielsen (www.useit.com)
- Cost of User Testing a Web Site
- http://www.useit.com/alertbox/980503.html
- When to Outsource the Recruiting of Test Users
- http://www.useit.com/alertbox/outsource_recruiting.html

## For Further Information

- Jared Spool (www.uie.com)
- Using Paper Prototyping to Manage Risks
- http://world.std.com/~uieweb/paper.htm
- Usability Labs:  Our Take
- http://world.std.com/~uieweb/labs.htm
- Eight Is More Than Enough
- http://world.std.com/%7Euieweb/eight.htm

## For Further Information

- Interaction Architect
- 13 Common Objections Against User Requirements Analysis & Why You Should Not Believe Them
- http://www.interactionarchitect.com/articles/article20000609b.htm
- Observation Methods & Tips for Usability Testing
- http://www.interactionarchitect.com/knowledge/article19991212shd.htm

## For Further Information

- DialogDesign (www.dialogdesign.dk)
- Ethics in Human Computer Interaction
- http://www.dialogdesign.dk/chi2001.html
- Usability Test of www.TowerRecords.com
- http://www.dialogdesign.dk/utestreports.html#sample

## For Further Information

- Usability.gov
- Why Should I Do Iterative Usability Testing?
- How Can I Encourage People in My Organization to Conduct Usability Engineering and Testing?
- Do I Need a Lab to Do Usability Testing?
- How Many Participants Are Needed for a Usability Test?
- How Much Does It Cost to Do Usability Testing?

## For Further Information

- Training Opportunities
- National Cancer Institute
- 	Usability Seminar Series & Education Training Program, http://usability.gov/events/seminars.html
- Society for Technical Communication WDC Chapter, www.stcwdc.org
- User Interface Engineering
- Human Factors International
- User Experience Conferences (Nielsen Norman Group)