Relationship of 

Usability and Patient Safety Usability and Patient Safety

with Health Information Technolog

Decemb

Emily S Patterson PhD Emily S. Patterson, PhD
Acknowledgments

• Robert Schumacher• Chris Gibbons Chris Gibbons• Patti Abbott• Jiajie Zhang Jiajie Zhang• Robert North• Svetlana Lowry Svetlana Lowry• Matthew Quinn• Mala Ramaiah Mala Ramaiah• Jason Saleem• Michelle Rogers Michelle Rogers• Emilie Roth• David Woods

2

David Woods
Poll

Which best describes you? Which best describes you?A. ResearcherB. ProgrammerC. Usability expertD. Patient safety expert Patient safety expertE. Administration/policymaker

3
Objectives

1.HIT: Tra s o g Co p e o nsforming Complex Work2.Definition of usability3.Relationship y

 between usability and 

4.Usabilityg g testing methodologies5.Evaluation scenarios that ensure co

4
Th E f HIT d C l 

The 

Era o

f HIT an

d

Comp

lex

5
HIT Transforms Work HIT Transforms Work

“Adopt a papproachnew tech...for thresafety and them befoaccidents 

IOM report IOM report

“T

T

p. 150

6
Laments with Transformative H

• Some HIT workflows that do not match clinicaprocesses create inefficiencies processes create inefficiencies• Poorly designed HIT screens that slow down and sometimes endanger patients• Large numbers of files containing historical p

information that are difficult to search, navigate, 

efficiently and identify trends over time efficiently, and identify trends over time• Confusing, and often conflicting, error messa• Alert fatigue Alert fatigue leading to users ignoring potenti

leading to users ignoring potenti

messages• Excessive mouse clicks, cursor movements

keystrokes, etc. during frequent tasks

7
Poll

What is your knowledge of usability tesWhat is your knowledge of usability tesA. Little knowledgeB. Know what it is Know what it isC. Have done a usability testD. Expert in the area Expert in the area

8
Objectives

1.HIT: Tra s o g Co p e o nsforming Complex Work2.Definition of usability3.Relationship y

 between usability and 

4.Usabilityg g testing methodologies5.Evaluation scenarios that ensure co

9
Definitions

Usability: Extent to which a product can be used by specified users to achieve specified goaleffectiveness, efficiency, and satisfaction specified context of usePatient safety:System attribute that influences the risk oharm due to errors

Q: What is the relationship between these for HIT

10

Q: What is the relationship between these for HIT
Objectives

1.HIT: Tra s o g Co p e o nsforming Complex Work2.Definition of usability3.Relationship y

 between usability an

4.Usabilityg g testing methodologies5.Evaluation scenarios that ensure co

11
Framework: Use Errors and Patient 

12
Use Error: Wrong Patient Record O

Imaging: PatImaging: Pat

EHR P ti t A EHR: 

Patien

t

A EHR P ti t B EHR: 

Patien

t

B

13
Use Error: Wrong Mode for Actio

Direct Dose Mode (mcg/min)W i ht D M d ( /k / i ) Weight

Dose 

Mo

de 

(mcg/k

g

/min

)

Test Mode Test ModeProduction Mode

14
Use Error: Inaccurate Data Displa

Lidocaine Hydrochlor

15
Use Error: Incomplete Data Displa

80 mg

16
Use Error: Non-standard measurem

system, convention, nv

e

n

tio

n, or terms

Kilograms or pounds?

17
Use Error: User Required to Recall Inf

One Time Dose

18
Use Error: Inadequate Feedbac

about Automation uto

matio

n

1 tablet

19
Use Error: Corrupted Data Stora

Next Finish

20
Poll

Have you or a family member experienHave you or a family member experien

least partially due to a design flaw w

A. Wrong patient Wrong patientB. Wrong treatmentC. Wrong medication Wrong medicationD. Delay of treatmentE. More than one More than one

21
Framework: Use Errors and Patient 

22
Adverse Events

Wrong patient: Actions with potentially harmful consequences are performed for one patient thaconsequences are performed for one patient tha

intended for another patient or a patient is not inthe need for treatment 

Wrong treatment: Treatments that were not inta patient are provided or missedW di ti Wrong medication: A ti t i th A patient receives the wron

medication type, dose, or route

Delay of treatment: Delay of treatment: A patient receives a signifi

A patient receives a signifi

in the provision of care activities Unintended or improper treatment: A patient 

unintended care due to confusion or due to actioto test software, train users, or demonstrate soft

t ti l t

23

potential customers
Framework: Use Errors and Patient 

24
Solution 1: No Action

25
Solution 2: Sue the Builder

 Code of Hamm

Code of Hamm

229

If a builder buiIf

a builder bui

house for som

does not consdoes not cons

properly, and twhich he built 

and kills its owthat builder shto death to death. 

26
Solution 3: Name and Shame

27
Solution 4: Summative Usabilit

y 

 T

NISTIR 7804 (DRAFT)

28
Poll

What is your preferred primary approacWhat is your preferred primary approac

making HIT safer for patients?

A. No action No action

B. Patients sue the vendor

C. Anonymous reporting Anonymous reportingD. Usability testingE. Something else Something else

29
Objectives

1.HIT: Tra s o g Co p e o nsforming Complex Work2.Definition of usability3.Relationship y

 between usability and 

4.Usabilityg g testing methodologies5.Evaluation scenarios that ensure co

30
Usability Evaluation Methods: Ov

Do not involve users Involve users

User Experience Participatory design

Card sorting

User Experience

(UX) Methods

Eye tracking

Usability

Summative 

usability testing

Ethnographic 

research

Eye tracking

Cognitive 

walkthrough

Usability

Evaluation 

Methods

usability testing

Formative 

usability testing

Expert evaluation

Focus groups

Surveys Expert evaluation

KLMGOMS

31
Summative is Not Formative Usabili

Behavior

Formative 

S

Eye tracking

ve

usability 

testing

Qualitativ

Ethnographic research

Q

Focus

groups

Participatory 

design Card sorting

ocus g oups

S

Attitude
Usability Testing Process

Usability Testing Process
Use Error Checklist Items: Exam

1.A Patient Identification Error

Wh d ti t’ d i i th fWhen a secon

d patient’s recor

d is open, is the f

record automatically closed?

When a second user opens a patient chart is thWhen

a second user opens

a patient chart, is th

automatically Iogged out?

When another application (e g imaging) is opeWhen another application (e.g., imaging) is ope

within the EHR, does the display have a title or 

with an accurate unique patient identifier? with an accurate unique patient identifier?If an action will cause data to be destructively ovwith another patient with another patient s’ data is the user alerted? data, is the user alerted?

34
Use Error Reporting Form: Exam

No. Potential use error Mitigation Plan 

1.

C.1 D t a

ta accuracy error: 

Medication doses truncated in pick list 

Do not truncate names at 40 characters, but instead display 7characters and the remainder v

menu makes it easy to pick the wrong dose 

upon mouse roll-over 

1F6 . . Recall error: Recall error: Provide pop up

“Are you sure?

” 

Physicians might forget that patients have 

ll i t di ti

Provide pop

-up Are you sure? 

when a physician orders and a pharmacist verifies a medication 

d t hi h ti t h

allergies 

to medications 

while ordering, even though it is displayed 

or

der 

to whi

c

h a patien

t

has an 

allergy 

35
Use Error Tracking Form: Exam

No. 

Date 

Found 

Date 

Fixed 

Date Fix 

Released Reported? Contact Resolution 

2011-

1.C.1 

5/31/11 6/2/11 6/6/11 Yes Smith, 

John 

Medication 

doses 

truncated in 

pick list 

menu makes 

it easy to 

pi k th c

k the 

wrong dose 

Clear Cl d ose

d

Green Awaiting fixY ll e ow Ali i na ysis ongoing

Red Newly reported, awaiting analy

36
Summative Usability Report Eleme

 Introduction

 Results

 Method

 Participants

D i

 Discussion

 Overall Res



Design P t ti l U 

 Tasks

 Procedure



P

o

tenti

al Us

 Effectivenes

 Procedure

 Efficiency

 Test location and 

environment

 Efficiency Satisfaction

 Appendices

 Usability metrics

 Test plan Screener Moderation 

Tas

k

s

37

as s
Objectives

1.HIT: Tra s o g Co p e o nsforming Complex Work2.Definition of usability3.Relationship y

 between usability and sa

4.Usabilityg g testing methodologies5.Evaluation scenarios that ensure c

38
Scenarios: Target Characteris

Level Integrity Targ

Surface validity Professionals judge face 

are engaged

Mdl f t Model of support I t iti i l 

Impact on cognition; inclu

capability gaps

J stification for Justification for Assess claims sers’ an

implementation

Assess claims; users’ an

organization’s perspectiv

Representative Nominal and challenging 

complexity

Nominal and challenging 

Performance Externally observable ac

observability

Externally observable ac

utterances

39
Comp y lexit Factors: Domain-Inde

 Data overload (Needle in a haystack

Data overload (Needle in a haystack

 Attention demands (Attention bottlen Mi i i f ti (I f ti 

Missing information (Information gap

 Uncertain Information (Unreliable da No predefined procedure (Novel situ Overconstrained task (Can task (Can t’ do it all

do it all

 Workload (Time pressured)

40
Embedded HIT Complexity Facto

Approach Real Approach Real

-World Comple

World Comple

• Increase dose of existing medication Increase dose of existing medication• Drug interaction warnings: false alarm• Taper dose for steroids Taper dose for steroids• First dose now and subsequent doses • Verbal order Verbal order• Change form of medication (PO to IV)• Handoff• Interruptions• Follow

-up documentation of prior work up documentation of prior work

• Batch transfer of medications

41
Scenario #1 Complexity Factor

• Documentation of activities dependent oprovid ll (R l f ider recall (Removal o

f td l rans

dermal pa

• Dealing with PRN Meds • D l ti /”T ” D i Dose escalation/”Taper

”

Dosing

• Sensitive Dx Documentation (Substance 

Ambulatory Care Mi

d

-ll d eve Provi

der

Diabetic Patient

42
Scenario #2 Complexity Factor

• Verbal order documentation• W kfl i t ti Workflow in

terruptions

• Documentation of patient handoffs

• I ti t t t ti t di ti 

Inpatien

t

to ou

tpatien

t medication proces

Inpatient CarePhysician ProviderCardiac Patient

43
Scenario #3 Complexity Factor

• Document change in DNR status (Remo• D t I/O’ Documen

t I/O’

s

• Documentation of medication administra

Critical CareNurse ProviderCardiac Patient

44
Concluding Thoughts: Infrastru

I t t A E i E li Inves

tmen

ts 

Are 

Eas

ier 

Earli

With present equipment flying is so With present equipment, flying is so

difficult that many individuals cannot learn to 

p y ilot an aircraft safel

y, 

and...human errors account for a major proportion of aircraft accid t A i ft b iden

ts...

As aircraft

become more 

complex and attain higher speeds, the 

necessity for designing the machine necessity for designing the machine

to suit the inherent characteristics of the human o

perators becomes 

Fitts, 1947, reprinted in Karsh et al., 2010, p. 621

increasingly apparent.

45
References: HIT/Usability

Schumacher RM, Patterson ES, North R, Zhang J, Lowry SZ, Quinn MT2011. Technical Evaluation, Testing and Validation of the Usability of EleRecords. Baltimore, MD: National Institute of Standards and TechnologyNISTIR 7804) NISTIR 7804).

Patterson ES, Roth EM, Woods DD. 2010. Facets of Complexity in SituaMacrocognition Metrics and Scenarios: Design and Evaluation for Real-

Edited by Patterson ES Miller J Hampshire UK: Edited by Patterson ES, Miller

J. Hampshire, UK: Ashgate Ashgate Publishing 22

Publishing. 22

Saleem JJ, Patterson ES, Militello L, Anders S, Falciglia M, Wissman JASM. 2007. Impact of Clinical Reminder Redesign on Learnability, Efficien

Workload for Ambulatory Clinic Nurses. Workload for Ambulatory Clinic Nurses. Journal of the American Medica

Journal of the American Medica

Association. Vol. 14, no. 5. : 632-640.Patterson ES, Nguyen AD, Halloran JM, Asch SM. 2004. Human Factors Effective Use of Ten HIV Clinical Reminders. Journal of the American M

Association. Vol. 11, no. 1. : 50-59.Patterson ES, Rogers ML, Render ML. 2004. Fifteen Best Practice RecoImprove the Effectiveness of Bar Code Medication Administration. Joint Journal on Q Sf uality and 

S

a

fety: 355-365.

Saleem JJ, Patterson ES, Militello L, Render ML, Orshansky G, Asch SABarriers and Facilitators to the Use of Computerized Clinical Reminders

American Medical Information Association Vol 12 : 438 447

46

American Medical Information Association. Vol. 12. : 438-447.
References: Patient Safety

Chipps E, Wills CE, Tanda R, Patterson ES, Elfrink V, Brodnik M, SchweWenger N. 2011. Registered Nurses' Judgments of the Error Classificatiof Patient Care Errors. Journal of Nursing Care Quality. Vol. 26, no. 4. (310.Patterson ES, Cook RI, Woods DD, Render ML. 2004. Examining the Coa Medication Error: Generic Patterns in Communication. IEEE TransactM d C b ti Man an

d

Cybernetics 

- P tA S t dH Part

A: Sys

tems an

d

Humans. V l 34 6 74

Vol. 34, no. 

6. : 74

Ebright P, Urden L, Patterson ES, Chalko BA. 2004. Themes SurroundinNear-Miss and Adverse Event Situations. Journal of Nursing Administra

11 : 531 11. : 531-538.

Patterson ES, Woods DD, Roth EM, Cook RI, Wears RI. 2006. Three Ke

Achieving Resilience with Medication Delivery with Information Technolo

Patient Safety Patient Safety. Vol 2 no 1 : 33 Vol. 2, no. 1. : 33-38.

Patterson ES, Rogers ML, Chapman RJ, Render ML. 2006. Compliance Use of Bar Code Medication Administration in Acute and Long-Term CarObservational Stud

y. Human Factors,p y S ecial Issue on Patient Safety. V

15-22.Powell-Cope G, Nelson AL, Patterson ES. 2008. Patient Care TechnologEvidence-Based Patient Safety: A Handbook for Nurses. Edited by Hugh

47

MD: Agency for Healthcare Research and Quality. 1-14.
Thank you for Thank you for your attenti

your attenti

Contact Information:

patterson.150@osu.edu

48