VIReC CyberSeminar Series 2007

Using Human Factors Principles in the (Re)Design of Bar Code Medication Administration

Emily S. Patterson, PhD

Research Scientist

VA Getting at Patient Safety (GAPS) Center

Cincinnati VAMC

The Ohio State University

Presentation Overview

  Define: Human Factors

  Literature Review: Effectiveness & Challenges

  BCMA System

  Research Studies:

  Study 1 (pre-post obs): “Side effects”

  Study 2 (small, medium, large obs): Workarounds by setting

  Translating Findings into Practice

  Concluding Remarks

Human Factors:interdisciplinary human-centered approach to addressing design challenges

Cognitive Systems Engineering (CSE):engineering a system of human and machine agents performing cognitive tasks in a domain   

(e.g. planning in anesthesiology)

CSE studies cognition...
        -   situations
        -   scenarios
        -   tasks
        -   domains
        -   expertise
        -   knowledge
        -   strategies
        -   artifacts
        -   other agents

In challenging:

With experts:

Supported by:

Cognitive Triad

        - Mixed results on medication error rates
        - With “wrong time” errors:
        - Ward 1: 	17% (pre)  vs. 11% (post)
        - Med/surg: 	  5% (pre)  vs.   9% (post)
        - Tele: 		  6% (pre)  vs.   7% (post)
        - Without “wrong time” errors:
        - Ward 1: 	11% (pre)  vs.   5% (post)
        - Med/surg: 	  4% (pre)  vs.   4% (post)
        - Tele: 		  6% (pre)  vs.   2% (post)

Literature Review: Evidence for Effectiveness Hospital Gamma: Barcode System

(Barker, 2004, personal communication)

~500 doses each

        - Lower medication error rate:
        - 0.17% (pre) vs. 0.07% (post) vs. 0.05% (one-year post) 
        - Improved medication records
        - Improved scheduling of medications
        - Better communication between nursing & pharmacy 
        - More efficient drug monitoring
        - More accurate and timely billing

Challenges:
        - Resistance to the change from a manual system 
        - Steep learning curve for some nurses and physicians
        - Some meds not barcoded (unit dose oral, injectables)

Literature Review: Evidence for Effectiveness 326 Bed Hospital: CliniCare

(Puckett, 1995)

Pre: 	450 medication administrations directly observed 	compared with physician order in record 

Post: 	7,013 doses checked by software

9.09% (pre) vs. 1.21% (post) error rate 
        - Improper dose; wrong dosage form eliminated 
        - Omitted dose decreased 92% 
        - Wrong time decreased 77%
        - 3.2% of doses scanned were intercepted
        - Self-reported errors decreased 79%
        - 42% increase in overall nursing satisfaction

Literature Review: Evidence for Effectiveness University of Wisconsin: BCSS (handheld) Pilot Unit

(Rough, 2005)

Challenges:
        - Some nursing resistance from additional time 
        - New sources of error: 
        - interface usability
        - miss trends in past doses
        - integration with IV “smart pumps”
        - Self-reported errors higher in hospital:
        - IV pump programming
        - pharmacy order entry
        - prescribing near-misses

Literature Review: Evidence for Effectiveness University of Wisconsin: BCSS (handheld) Hospital-wide

Literature Review: Evidence for EffectivenessBCMA: Topeka VAMC

% med errors/ total meds administered

(Johnson, 2002)

(Coyle & Heinen, 2005)

Literature Review: Evidence for Effectiveness BCMA: Martinsburg VAMC

Literature Review: Evidence for Effectiveness BCMA: Martinsburg VAMC

(Coyle & Heinen, 2005)

“Has BCMA improved patient safety?”
        -    19% yes
        -    52% partially
        -    28% no

Survey: 
        - N = 54
        - 45% response rate
        - Convenience sample of nursing staff

Literature Review: Evidence for Effectiveness BCMA: Asheville VAMC 1/2001

Bar Code Medication Administration (BCMA)

Radio antenna

Color LCD touch screen

Bar code scanner 

Keyboard

Touch  pen

Medication cart 

Patient medication drawer

© 1999 Coelho

BCMA Objectives
        - Enhance accuracy

drug

dose

patient

route

time
        - Reduce time to administer missing medications
        - Improve documentation efficiency

Physician

Pharmacist

Patient

orders

dispenses

administers

Nurse

validates

(From training materials)

***FICTITIOUS PATIENT RECORD***

Study 1 (Observational): Pre-Post Implementation

Objective: 
        - Identify “side effects” from BCMA implementation 

Methods:
        - Direct observation of medication administration
        - Prior to implementation (21 hrs, 7 nurses, 1 site) 
        - Following implementation (60 hrs, 26 nurses, 3 sites)
        - “Process tracing” protocol analysis
        - Mini-case selection (67), classification (12), theme identification (5)

Investigating Side Effects of Change

© 1999 Woods

“Adopt a proactive approach: examine new technologies ...for threats to safety and redesign them before accidents occur.”

IOM report “To err is human” p. 150

Aviation accidents, lab research

Degraded team coordination when transition from paper to electronic

Monitoring raises priority

Automation increases rigidity for unanticipated situations

Electronic flight strips

in air traffic control 

“Explanation” systems in anesthesiology 

On-time departures in aviation 

Automated SOPs in nuclear power 

1992 Strasbourg, France; 87/96 died 

Surprises when automation is “strong and silent”

Advisory systems increase workload during escalating situations

1) “Automation Surprise” with BCMA

11:00

[Chemotherapy 11 a.m.]

IV site unavailable

IV site 

available

Record in use; 

administered 2:45 p.m.

Dr.1: Given yet?

Dr.2: Given yet?

16:00

Medication automatically removed

Nurse scans; automatically documented 3:50 p.m.

© 2000 Patterson

Nurse surprised not in BCMA; cannot document

Pharmacy re-enters order

Aviation accidents, lab research

Degraded team coordination when transition from paper to electronic

Monitoring raises priority

Automation increases rigidity for unanticipated situations

Electronic flight strips

in air traffic control 

“Explanation” systems in anesthesiology 

On-time departures in aviation 

Automated SOPs in nuclear power 

1992 Strasbourg, France; 87/96 died 

Surprises when automation is “strong and silent”

Advisory systems increase workload during escalating situations

Physician

Pharmacist

Nurse

Patient

orders

dispenses

administers

Do expired orders need to be continued?

Can you co-sign?

Are they ready?

No pending, discontinued information

2) Degraded team coordination

© 2000 Patterson

Can you order these?

Are you sure?

***FICTITIOUS PATIENT RECORD***

Aviation accidents, lab research

Degraded team coordination when transition from paper to electronic

Monitoring raises priority

Automation increases rigidity for unanticipated situations

Electronic flight strips

in air traffic control 

“Explanation” systems in anesthesiology 

On-time departures in aviation 

Automated SOPs in nuclear power 

1992 Strasbourg, France; 87/96 died 

Surprises when automation is “strong and silent”

Advisory systems increase workload during escalating situations

5) Increased rigidity: Refused medication

***FICTITIOUS PATIENT RECORD***

Study 1: Limitations
        - Improvements made since observations (2000-2001)
        - Missed side effects since data collection and analysis shaped by frameworks
        - Unclear if findings generalize to other hospitals, software, devices with barcode input
        - Not examined: 
        - Positive “side effects”
        - Effectiveness in reducing med errors
        - Time spent administering medications
        - Impact on others (pharmacists, physicians, respiratory therapists, nursing aides, patients)

Study 2 (Observational): Background
        - Medication errors most common cause of hospital AEs
        - Rate: 3.7% (chart review) to 17.7% (observations)
        - Cost: $1.33/$1 for medication
        - When: 38% during nursing administration
        - Long-term care:
        - Larger patient:nurse ratio
        - More medications (per patient, per “medication pass”)
        - Longer patient stays

Study 2 Methods

Objective: 
        - ID BCMA workarounds in acute and long-term care 

Prospective ethnographic study:
        - Direct observation of medication administration; opportunistic interviews
        - Setting: 1 small, 1 medium, 1 large hospital
        - Data collection: Field notes, BCMA data
        - Data analysis: 
        - Workarounds from “process tracing” protocols
        - Barriers to desired use from interviews

No. Observations by Facility

Patient Identification Strategies

p=0.016, Fisher’s exact test

Barriers to Scanning Wristbands
        - Medium hospital: tethered scanners
        - Carts stationary in long-term care
        - Larger carts (more medications)
        - More battery replacements (longer med passes)
        - Longer patient stays in long-term care
        - Dirty, twisted, torn, missing, wet wristbands
        - Nurses more familiar with patients

“Best Practice” Recommendations

Study 2: Limitations
        - Improvements made since observations (2000-2002)
        - Hawthorne effect: participants less likely to use workarounds
        - Facilities, wards, nurses convenience sample
        - Small sample size relative to target population
        - Unclear if findings generalize to other hospitals, software, devices w/barcodes

Study 2: Summary
        - In long-term care vs acute care:
        - Less scanning of wristbands to identify patients
        - More “pre-pouring” as administration strategy
        - Less detailed reports to detect errors
        - “Workarounds” (in both settings)
        - Reduce effectiveness 
        - Reduce accuracy of documentation
        - Redesign and organizational changes recommended (not sanctioning, training) 

Paper Reports Used by Nurses

p<0.0001, Fisher’s exact test

Report Information
        - Long-Term Care: Ward Administration
        - Number of medications/patient/hr
        - Acute Care: End of Shift, Medication List
        - Medication name
        - Dose
        - Route
        - Special instructions (physician, pharmacist) 

Cover Sheet Overview Display (Prototype)

***FICTITIOUS PATIENT RECORD***

Translating Findings into Practice

15 “best practices” 

Dissemination via National BCMA Joint Program Office, BCMA workgroup

Joint Commission Journal 

Prioritized design modifications

V1.0: 5/10 high priority, 2/9 medium priority, and 1/8 low priority implemented

V2.0: 4/5 participants missed all IV medications for at least 1/6 fake patients

V2.0 redesign: 1/4 participants missed one IV medication for 1/6 fake patients

“Cover sheet” enhancement  

Concept storyboard designed collaboratively with developers

Beyond VHA

Press releases for key publications (~70 requests for reprints)

Postings on NPSF patient safety listserv (~1500 recipients)

FDA cited research as evidence for need for flexibility

Consultations (hospitals, device companies, “best practice” groups, researchers)

Concluding Remarks
        - Very high failure rates for software internationally
        - Human factors experts (partnering with medical experts) can radically improve usefulness, usability, adoption rates
        - Need for “bridge” funding for HF experts:
        - Steep learning curve if no healthcare experience
        - Significant ramp-up time before competitive as PI
        - Structural challenges pervasive in obtaining funding
        - 1 grant submission per round, expectation that PI is 10-20% FTE, no equivalent for “clinical funding,” methods foreign to reviewers, recent MREP and PSCI funding issues, traditional tenure initiating units do not have 5/8 appointments, research in other domains easier to fund and conduct
        - Translating into practice requires “infrastructure” funds
        - “Good citizen” expectations high
        - Long tails post-award to translate research into practice

References 

Patterson, E.S., Cook, R.I., Woods, D.D., Render, M.L. (in press).  Gaps and resilience.  In MS Bogner (Ed.) Human Error in Medicine 2nd Edition.  Lawrence Erlbaum Associates, Hillsdale, NJ.

Patterson ES, Woods DD, Cook RI, Render ML. (August 2007).  Collaborative Cross-Checking to Enhance Resilience.  Cognition, Technology and Work 9(2), 155-162.

Patterson, E.S., Rogers, M.L., Chapman, R.J., Render, M.L. (2006). Compliance with Intended Use of Bar Code Medication Administration in Acute and Long-Term Care: An Observational Study.  Human Factors, Special Issue on Patient Safety 48(1), 15-22.

Patterson, E.S., Woods, D.D., Roth, E.M., Cook, R.I., Wears, R.L. (2006).  Three key levers for achieving resilience in medication delivery with information technology.  Journal of Patient Safety 2(1):33-38.

Patterson, E.S., Rogers, M.L., Render, M.L. (2004). Fifteen best practice recommendations to improve the effectiveness of bar code medication administration.  Joint Commission Journal on Quality and Safety, 355-365.

Patterson, E.S., Rogers, M.L., Render, M.L. (2004).  Simulation-based embedded probe technique for human-computer interaction evaluation.   Cognition, Technology, and Work, 6(3), 197-205.

Patterson E.S., Cook, R.I., Woods, D.D., Render, M.L. (2004).  Examining the complexity behind a medication error: Generic patterns in communication.  IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans 34(6), 749-756.

Patterson, E.S., Cook, R.I., Render, M.L. (2002).  Improving patient safety by identifying side effects from introducing bar coding in medication administration.  Journal of the American Medical Informatics Association, 9(5),540-553.