# Presentation: 623961

##   VIReC CyberSeminar Series 2007

**Using Human Factors Principles in the (Re)Design of Bar Code Medication Administration**
- Emily S. Patterson, PhD
- Research Scientist
- VA Getting at Patient Safety (GAPS) Center
- Cincinnati VAMC
- The Ohio State University

##   Define: Human Factors
  Literature Review: Effectiveness & Challenges
  BCMA System
  Research Studies:
  Study 1 (pre-post obs): “Side effects”
  Study 2 (small, medium, large obs): Workarounds by setting
  Translating Findings into Practice
  Concluding Remarks

-   Literature Review: Effectiveness & Challenges
-   BCMA System
-   Research Studies:
-   Study 1 (pre-post obs): “Side effects”
-   Study 2 (small, medium, large obs): Workarounds by setting
-   Translating Findings into Practice
-   Concluding Remarks

**Presentation Overview**

## Human Factors:interdisciplinary human-centered approach to addressing design challenges

**Cognitive Systems Engineering (CSE)**:engineering a system of human and machine agents performing cognitive tasks in a domain
- (e.g. planning in* *anesthesiology)

## CSE studies cognition...

-   situations
-   scenarios
-   tasks
-   domains

-   expertise
-   knowledge
-   strategies

-   artifacts
-   other agents

- In challenging:

- With experts:

- Supported by:

- Cognitive Triad

## Mixed results on medication error rates

With “wrong time” errors:
Ward 1: 	17% (pre)  vs. 11% (post)
Med/surg: 	  5% (pre)  vs.   9% (post)
Tele: 		  6% (pre)  vs.   7% (post)

Without “wrong time” errors:
Ward 1: 	11% (pre)  vs.   5% (post)
Med/surg: 	  4% (pre)  vs.   4% (post)
Tele: 		  6% (pre)  vs.   2% (post)

- With “wrong time” errors:
- Ward 1: 	**17%** (pre)  vs. **11%** (post)
- Med/surg: 	  **5%** (pre)  vs.   **9%** (post)
- Tele: 		  **6%** (pre)  vs.   **7%** (post)
- Without “wrong time” errors:
- Ward 1: 	**11%** (pre)  vs.   **5%** (post)
- Med/surg: 	  **4%** (pre)  vs.   **4%** (post)
- Tele: 		  **6%** (pre)  vs.   **2%** (post)

**Literature Review: Evidence for Effectiveness**** Hospital Gamma: Barcode System**

**(Barker, 2004, personal communication)**

- ~500 doses each

## Lower medication error rate:
0.17% (pre) vs. 0.07% (post) vs. 0.05% (one-year post) 
Improved medication records
Improved scheduling of medications
Better communication between nursing & pharmacy 
More efficient drug monitoring
More accurate and timely billing

Challenges:
Resistance to the change from a manual system 
Steep learning curve for some nurses and physicians
Some meds not barcoded (unit dose oral, injectables)

**0.17% **(pre) vs. **0.07% **(post) vs. **0.05% **(one-year post)** **
- Improved medication records
- Improved scheduling of medications
- Better communication between nursing & pharmacy
- More efficient drug monitoring
- More accurate and timely billing
- Challenges:
- Resistance to the change from a manual system
- Steep learning curve for some nurses and physicians
- Some meds not barcoded (unit dose oral, injectables)

**Literature Review: Evidence for Effectiveness**** 326 Bed Hospital: CliniCare**

**(Puckett, 1995)**

## Pre: 	450 medication administrations directly observed 	compared with physician order in record 
Post: 	7,013 doses checked by software

9.09% (pre) vs. 1.21% (post) error rate 
Improper dose; wrong dosage form eliminated 
Omitted dose decreased 92% 
Wrong time decreased 77%
3.2% of doses scanned were intercepted
Self-reported errors decreased 79%
42% increase in overall nursing satisfaction

- Post: 	7,013 doses checked by software
**9.09%** (pre) vs. **1.21%** (post) error rate
- Improper dose; wrong dosage form eliminated
- Omitted dose decreased 92%
- Wrong time decreased 77%
- 3.2% of doses scanned were intercepted
- Self-reported errors decreased 79%
- 42% increase in overall nursing satisfaction

**Literature Review: Evidence for Effectiveness**** University of Wisconsin: BCSS (handheld) Pilot Unit**

**(Rough, 2005)**

## Challenges:
Some nursing resistance from additional time 
New sources of error: 
interface usability
miss trends in past doses
integration with IV “smart pumps”
Self-reported errors higher in hospital:
IV pump programming
pharmacy order entry
prescribing near-misses

- Some nursing resistance from additional time
- New sources of error:
- interface usability
- miss trends in past doses
- integration with IV “smart pumps”
- Self-reported errors higher in hospital:
- IV pump programming
- pharmacy order entry
- prescribing near-misses

**Literature Review: Evidence for Effectiveness**** University of Wisconsin: BCSS (handheld) Hospital-wide**

## Literature Review: Evidence for EffectivenessBCMA: Topeka VAMC

- % med errors/ total meds administered

**(Johnson, 2002)**

## Literature Review: Evidence for Effectiveness BCMA: Martinsburg VAMC

**(Coyle & Heinen, 2005)**

**Literature Review: Evidence for Effectiveness**** BCMA: Martinsburg VAMC**

## Literature Review: Evidence for Effectiveness BCMA: Martinsburg VAMC

**Literature Review: Evidence for Effectiveness**** BCMA: Martinsburg VAMC**

**(Coyle & Heinen, 2005)**

## “Has BCMA improved patient safety?”
   19% yes
   52% partially
   28% no

Survey: 
N = 54
45% response rate
Convenience sample of nursing staff

-    19% yes
-    52% partially
-    28% no
- Survey:
- N = 54
- 45% response rate
- Convenience sample of nursing staff

**Literature Review: Evidence for Effectiveness**** BCMA: Asheville VAMC 1/2001**

## Bar Code Medication Administration (BCMA)

**Bar Code Medication Administration (BCMA)**

- Radio antenna

- Color LCD touch screen

- Bar code scanner

- Keyboard

- Touch  pen

- Medication cart

- Patient medication drawer

- © 1999 Coelho

## BCMA Objectives

**BCMA Objectives**

- Enhance accuracy
***drug***
***dose***
***patient***
***route***
***time***
- Reduce time to administer missing medications
- Improve documentation efficiency

- Physician

- Pharmacist

- Patient

- orders

- dispenses

- administers

- Nurse

**validates**

**(From training materials)**

## ***FICTITIOUS PATIENT RECORD***

## Objective: 
Identify “side effects” from BCMA implementation 

Methods:
Direct observation of medication administration
Prior to implementation (21 hrs, 7 nurses, 1 site) 
Following implementation (60 hrs, 26 nurses, 3 sites)
“Process tracing” protocol analysis
Mini-case selection (67), classification (12), theme identification (5)

- Identify “side effects” from BCMA implementation
- Methods:
- Direct observation of medication administration
- Prior to implementation (21 hrs, 7 nurses, 1 site)
- Following implementation (60 hrs, 26 nurses, 3 sites)
- “Process tracing” protocol analysis
- Mini-case selection (67), classification (12), theme identification (5)

**Study 1 (Observational): ****Pre-Post Implementation**

## Investigating Side Effects of Change

**Investigating***** *****Side Effects***** *****of Change**

- © 1999 Woods

## Aviation accidents, lab research

- Degraded **team coordination **when transition from paper to electronic

- Monitoring raises **priority**

- Automation increases **rigidity **for unanticipated situations

- Electronic flight strips
- in air traffic control

- “Explanation” systems in anesthesiology

- On-time departures in aviation

- Automated SOPs in nuclear power

- 1992 Strasbourg, France; 87/96 died

**Surprises **when automation is “strong and silent”

- Advisory systems increase **workload **during escalating situations

## 1) “Automation Surprise” with BCMA

- 11:00

- [Chemotherapy ***11 a.m.***]

- IV site unavailable

- IV site
- available

- Record in use;
- administered ***2:45 p.m.***

- Dr.1: Given yet?

- Dr.2: Given yet?

- 16:00

- Medication _**automatically removed**_

- Nurse scans; _**automatically documented **_***3:50 p.m.***

- © 2000 Patterson

- Nurse _**surprised**_ not in BCMA; cannot document

- Pharmacy re-enters order

## Aviation accidents, lab research

- Degraded **team coordination **when transition from paper to electronic

- Monitoring raises **priority**

- Automation increases **rigidity **for unanticipated situations

- Electronic flight strips
- in air traffic control

- “Explanation” systems in anesthesiology

- On-time departures in aviation

- Automated SOPs in nuclear power

- 1992 Strasbourg, France; 87/96 died

**Surprises **when automation is “strong and silent”

- Advisory systems increase **workload **during escalating situations

## 2) Degraded team coordination

- Physician

- Pharmacist

- Nurse

- Patient

- orders

- dispenses

- administers

*Do expired orders need to be continued?*

*Can you co-sign?*

*Are they ready?*

**No pending, discontinued information**

**2) Degraded ****team coordination**

- © 2000 Patterson

*Can you order these?*

*Are you sure?*

- *****FICTITIOUS** PATIENT RECORD***

## Aviation accidents, lab research

- Degraded **team coordination **when transition from paper to electronic

- Monitoring raises **priority**

- Automation increases **rigidity **for unanticipated situations

- Electronic flight strips
- in air traffic control

- “Explanation” systems in anesthesiology

- On-time departures in aviation

- Automated SOPs in nuclear power

- 1992 Strasbourg, France; 87/96 died

**Surprises **when automation is “strong and silent”

- Advisory systems increase **workload **during escalating situations

## 5) Increased rigidity: Refused medication

## ***FICTITIOUS PATIENT RECORD***

- *****FICTITIOUS** PATIENT RECORD***

## Improvements made since observations (2000-2001)
Missed side effects since data collection and analysis shaped by frameworks
Unclear if findings generalize to other hospitals, software, devices with barcode input
Not examined: 
Positive “side effects”
Effectiveness in reducing med errors
Time spent administering medications
Impact on others (pharmacists, physicians, respiratory therapists, nursing aides, patients)

- Missed side effects since data collection and analysis shaped by frameworks
- Unclear if findings generalize to other hospitals, software, devices with barcode input
- Not examined:
- Positive “side effects”
- Effectiveness in reducing med errors
- Time spent administering medications
- Impact on others (pharmacists, physicians, respiratory therapists, nursing aides, patients)

**Study 1: Limitations**

## Medication errors most common cause of hospital AEs
Rate: 3.7% (chart review) to 17.7% (observations)
Cost: $1.33/$1 for medication
When: 38% during nursing administration
Long-term care:
Larger patient:nurse ratio
More medications (per patient, per “medication pass”)
Longer patient stays

- Rate: 3.7% (chart review) to 17.7% (observations)
- Cost: $1.33/$1 for medication
- When: 38% during nursing administration
- Long-term care:
- Larger patient:nurse ratio
- More medications (per patient, per “medication pass”)
- Longer patient stays

**Study 2 (Observational): Background**

## Objective: 
ID BCMA workarounds in acute and long-term care 

Prospective ethnographic study:
Direct observation of medication administration; opportunistic interviews
Setting: 1 small, 1 medium, 1 large hospital
Data collection: Field notes, BCMA data
Data analysis: 
Workarounds from “process tracing” protocols
Barriers to desired use from interviews

- ID BCMA workarounds in acute and long-term care
- Prospective ethnographic study:
- Direct observation of medication administration; opportunistic interviews
- Setting: 1 small, 1 medium, 1 large hospital
- Data collection: Field notes, BCMA data
- Data analysis:
- Workarounds from “process tracing” protocols
- Barriers to desired use from interviews

**Study 2 Methods**

## No. Observations by Facility

## Patient Identification Strategies

- p=0.016, Fisher’s exact test

## Barriers to Scanning Wristbands

- Medium hospital: tethered scanners
- Carts stationary in long-term care
- Larger carts (more medications)
- More battery replacements (longer med passes)
- Longer patient stays in long-term care
- Dirty, twisted, torn, missing, wet wristbands
- Nurses more familiar with patients

## “Best Practice” Recommendations

## Improvements made since observations (2000-2002)
Hawthorne effect: participants less likely to use workarounds
Facilities, wards, nurses convenience sample
Small sample size relative to target population
Unclear if findings generalize to other hospitals, software, devices w/barcodes

- Hawthorne effect: participants less likely to use workarounds
- Facilities, wards, nurses convenience sample
- Small sample size relative to target population
- Unclear if findings generalize to other hospitals, software, devices w/barcodes

**Study 2: Limitations**

## Study 2: Summary

- In long-term care vs acute care:
- Less scanning of wristbands to identify patients
- More “pre-pouring” as administration strategy
- Less detailed reports to detect errors
- “Workarounds” (in both settings)
- Reduce effectiveness
- Reduce accuracy of documentation
- Redesign and organizational changes recommended (***not*** sanctioning, training)

## Paper Reports Used by Nurses

- p<0.0001, Fisher’s exact test

## Report Information

*Long-Term Care:* Ward Administration
- Number of medications/patient/hr
*Acute Care:* End of Shift, Medication List
- Medication name
- Dose
- Route
- Special instructions (physician, pharmacist)

## Cover Sheet Overview Display (Prototype)

- *****FICTITIOUS** PATIENT RECORD***

## Translating Findings into Practice

- 15 “best practices”
- Dissemination via National BCMA Joint Program Office, BCMA workgroup
- Joint Commission Journal
- Prioritized design modifications
- V1.0: 5/10 high priority, 2/9 medium priority, and 1/8 low priority implemented
- V2.0: 4/5 participants missed all IV medications for at least 1/6 fake patients
- V2.0 redesign: 1/4 participants missed one IV medication for 1/6 fake patients
- “Cover sheet” enhancement
- Concept storyboard designed collaboratively with developers
- Beyond VHA
- Press releases for key publications (~70 requests for reprints)
- Postings on NPSF patient safety listserv (~1500 recipients)
- FDA cited research as evidence for need for flexibility
- Consultations (hospitals, device companies, “best practice” groups, researchers)

## Very high failure rates for software internationally
Human factors experts (partnering with medical experts) can radically improve usefulness, usability, adoption rates
Need for “bridge” funding for HF experts:
Steep learning curve if no healthcare experience
Significant ramp-up time before competitive as PI
Structural challenges pervasive in obtaining funding
1 grant submission per round, expectation that PI is 10-20% FTE, no equivalent for “clinical funding,” methods foreign to reviewers, recent MREP and PSCI funding issues, traditional tenure initiating units do not have 5/8 appointments, research in other domains easier to fund and conduct
Translating into practice requires “infrastructure” funds
“Good citizen” expectations high
Long tails post-award to translate research into practice

- Human factors experts (partnering with medical experts) can radically improve usefulness, usability, adoption rates
- Need for “bridge” funding for HF experts:
- Steep learning curve if no healthcare experience
- Significant ramp-up time before competitive as PI
- Structural challenges pervasive in obtaining funding
- 1 grant submission per round, expectation that PI is 10-20% FTE, no equivalent for “clinical funding,” methods foreign to reviewers, recent MREP and PSCI funding issues, traditional tenure initiating units do not have 5/8 appointments, research in other domains easier to fund and conduct
- Translating into practice requires “infrastructure” funds
- “Good citizen” expectations high
- Long tails post-award to translate research into practice

**Concluding Remarks**

## References 

- Patterson, E.S., Cook, R.I., Woods, D.D., Render, M.L. (in press).  Gaps and resilience.  In MS Bogner (Ed.) *Human Error in Medicine* 2nd Edition.  Lawrence Erlbaum Associates, Hillsdale, NJ.
- Patterson ES, Woods DD, Cook RI, Render ML. (August 2007).  Collaborative Cross-Checking to Enhance Resilience.  *Cognition, Technology and Work 9(2),* 155-162.
- Patterson, E.S., Rogers, M.L., Chapman, R.J., Render, M.L. (2006). Compliance with Intended Use of Bar Code Medication Administration in Acute and Long-Term Care: An Observational Study.  *Human Factors, Special Issue on Patient Safety 48(1),* 15-22.
- Patterson, E.S., Woods, D.D., Roth, E.M., Cook, R.I., Wears, R.L. (2006).  Three key levers for achieving resilience in medication delivery with information technology.  *Journal of Patient Safety 2(1)*:33-38.
- Patterson, E.S., Rogers, M.L., Render, M.L. (2004). Fifteen best practice recommendations to improve the effectiveness of bar code medication administration.  *Joint Commission Journal on Quality and Safety, *355-365.
- Patterson, E.S., Rogers, M.L., Render, M.L. (2004).  Simulation-based embedded probe technique for human-computer interaction evaluation.   *Cognition, Technology, and Work, 6(3),* 197-205.
- Patterson E.S., Cook, R.I., Woods, D.D., Render, M.L. (2004).  Examining the complexity behind a medication error: Generic patterns in communication.  *IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans 34(6),* 749-756.
- Patterson, E.S., Cook, R.I., Render, M.L. (2002).  Improving patient safety by identifying side effects from introducing bar coding in medication administration.  *Journal of the American Medical Informatics Association*,* 9(5)*,540-553.