Information Overload and the Design and Implementation of Computerized Decision Support

Charlene R. Weir, PhD

Associate Director Education and Evaluation, SLC GRECC

Associate Professor

Department of Biomedical Informatics

University of Utah

OBJECTIVES
        - Identify concepts and causes of Information Overload in terms of computerized clinical environments.
        - Discuss the relationship between information overload and different memory systems.
        - Describe how to match clinical task characteristics and implementation strategies in order to minimize information overload in the clinical setting. 

OVERVIEW
        - Definition of Computerized Decision Support
        - The Goal and the Mandate
        - Are we meeting the goal?
        - Central Thesis
        - The Adaptive User
        - Memory, attention and control
        - Recommendations

The Socio-Technical Perspective

“Providing clinicians, patients or individuals with knowledge and person specific or population  information, intelligently filtered or presented at appropriate times, to foster better health processes, better individual patient care, and better population health.”

                                                    Consensus Definition of DSS by A Roadmap for National Action on Clinical Decision Support, AMIA 2006

Types of Decision Support
        - Alerts
        - Reminders
        - Guidelines
        - Information Displays
        - Provider Order Entry
        - Electronic Text/Templates

“to ensure that optimal, usable and effective clinical decision support is widely available to providers, patients, and individuals where and when they need it to make health care decisions.” 

A Roadmap for National Action on Clinical Decision Support

AMIA 2006

THE GOAL

ARE WE MEETING THE GOAL?Systematic Reviews - Outcomes
        - Garg, et al.(2005) Effects of CDSS on Practitioner Performance and Patient Outcomes:  . . . . .
        - Sig improvements in practitioner performance in 64% of studies and improvements in patient outcomes for only 13%.
        - Unable to aggregate due to significant unexplained variation
        - Recommendations: “Important issues include CDSS user acceptance, workflow integration . . .  (p. 1236)

ARE WE MEETING THE GOAL?Systematic Reviews - Outcomes
        - Kawamoto, et al (2005). Improving clinical practice using clinical decision support systems: a systematic review.
        - Sig improvements in practitioner performance in 68%. Patient outcomes were not examined.
        - Unable to aggregate effects due to significant unexplained variation
        - Recommendations: “The promise of evidence based medicine will be fulfilled only when strategies for implementing best practice are rigorously evidence-based themselves.”  (p. 7)

ARE WE MEETING THE GOAL?Systematic Reviews - Outcomes
        - Shekelle P. Costs and Benefits of Health Information Technology. AHRQ Publication. Santa Monica, CA.
        - Unable to aggregate due to significant unexplained variation

Also published in: Chaudhry B, Want J, Wu S, et al.  Systematic review:  Impact of HIT on quality, efficiency, and costs of medical care.  Annals of Internal Medicine 2006;144:E12-E22

CONLUSION
        - “In summary, we identified no study or collection of studies, outside of those from a handful of HIT leaders, that would allow a reader to make a determination about the generalizable knowledge of the system’s reported benefit.
        - Even if further randomized, controlled trials are performed, the generalizability of the evidence would remain low unless additional systematic, comprehensive, and relevant descriptions and measurements are made regarding how the technology is utilized, the individuals using it, and the environment it is used in.” (Shekelle, et al, p. 4)

“What do users say?”

The Results of Qualitative Studies

Types of Unintended Consequences Related to CPOE

“Great care must be taken to balance the risks of over-alerting with not alerting. Developers should re-work clinical system interfaces to: 1) reduce collection of redundant information; b) display relevant information in logical locations . . . “ (p. 553)

 Campbell, Sittig, Ash, Guappone and Dykstra, JAMIA. 13:547-556.

Information-System Related Errors
        - Interface not suitable for highly interruptive context
        - Causes cognitive overload due to overly structured information entry and retrieval
        - Misrepresenting collective, interactive work as a linear, clear-cut and predictable workflow
        - Misrepresenting communication as information transfer.

Ash, J, Berg,M. Coiera, E. JAMIA  2004; 11:104-112

Barriers to Effective Use of VA Clinical Reminders
        - Patterson, et al (2004)
        - Workload was the primary barrier
        - Inapplicability to the situation
        - Lack of utility and ease of use
        - Workflow - not related to core work - duplication
        - “Assembly line medicine”
        - “Having physicians do clerical entry tasks”

Issues in Electronic Documentation

“overwhelmed”

“takes too much effort to sort through everything”

“I avoid reading nursing notes, they just have pages and pages of blank fields”

“There is so much stuff put into a note, I can’t find what I need.”
        - Access versus Availability
        - Tools to identify relevance not available
        - Accuracy goals are competing with efficiency goals

Weir, C and Nebeker, J (2007). Critical Issues in an Electronic Documentation System. AMIA Proceedings.

**CENTRAL THESIS**
        - Inattention to work processes in the implementation process is experienced as information overload to clinicians. 
        - Because:
        - Deviations in work-flow are perceived as interruptions
        - Changes in information location and timing increases cognitive effort
        - The process of adaptation results in the creation of  innovative strategies to decrease cognitive burden
        - Information Overload is really a “mismatch” between available cognitive resources and the task

Changing Work Processes

Information Overload

Adaptive Strategies

 TASK ANALYSISInformation Management Strategies
        - Systematic selection of 13 / 133 VA sites
        - Random selection of a primary care clinic
        - Procedures
        - Site visit, observations and interviews
        - Goal-based interviews (“in order to . .”; “by . .”)
        - 88 participants (14 nurses, 53 ordering providers, 8 pharmacists, 2 dieticians)
        - About 60 hours of observation
        - Qualitative Analysis ( tasks, common components, and goals)

Weir, CR, et al (2007). A cognitive task analysis of information management strategies in a computerized provider order entry environment. JAMIA 14(1):65-75

Information Management Goals
        - Relevance Screening
        - Ensuring Accuracy
        - Minimizing Memory Load
        - Negotiating Responsibility

 COPING STRATEGIES FOR INFORMATION INPUT OVERLOAD
        - Omission
        - Reduced Precision
        - Queuing
        - Filtering
        - Cutting Categories
        - De-centralization
        - Escape

Hollnagel, E and Woods, D (2005) Joint Cognitive Systems. CRC Press (p. 80)

CONCEPTS of “INFORMATION OVERLOAD” 
        - Mismatch between us and context
        - Disorientation/ lost
        - Inability to determine relevance
        - Distracting/forget goal
        - High Effort
        - Lack of situational awareness
        - Inability to “think” or analyze

MEMORYA Tale of Two Processes 
        - Associative Processing
        - Symbolic/Rule-Based Processing

ASSOCIATIVE MEMORY PROCESSING
        - Associative Learning: Gradual accretion of knowledge through progressive associations; expert performance is an example.
        - Thinking: fast, pattern-completion, effortless
        - Awareness: Not required for performance
        - Errors: common heuristics or “rules of thumb”
        - Change: change is slow,hard; like “breaking bad habits.”

VERY RESISTANT TO IMPACT OF COGNITIVE LOAD

SYMBOLIC MEMORY PROCESSING
        - Symbolic Learning: Fast increase in knowledge through rules/symbols/language.
        - Thinking: slow, effortful, requires attention
        - Awareness: Required for performance
        - Errors: miss-identifed task, not understanding
        - CHANGE: change may be fast

HIGHLY SENSITIVE TO COGNITIVE LOAD

IMPLICATIONS
        - Both types of cognition are “working” simultaneously.
        - Humans prefer to minimize cognitive load, hence their behavior will likely be under the control of what they know as much as possible.
        - Adaptive strategies or “work-arounds” are geared to “think less.”
        - Experts do much more with less attention and effort.

MOTIVATION
        - As cognitive load increases (work, distractions), then the work will be taken care of by the associative system (less thinking, more automatic, pattern-recognition processing).
        - As motivation to be accurate increases, more of the work will be done by the symbolic system (new material, high patient acuity, social pressure)

RECOMMENDATIONSTask-Person-Technology Fit
        - Decision support for easy tasks should not require attention (they will be seen as interruptions). Increase Control
        - Order sets and protocols
        - Standing Orders
        - Administrative Control (e.g. formulary)
        - Documentation / Order Combinations
        - Embedded tracking of behavior
        - “just in time” heuristic

RECOMMENDATIONSTask-Person-Technology Fit
        - Decision support for hard/complex tasks should assist the human in active problem-solving - not replace him/her.
        - Provide information early in the planning phase
        - Display information by tasks (e.g.problems)
        - Slow down the process in order to facilitate “deeper processing”
        - Info buttons, access to other experts/consults; and/or scientific authoritative sources
        - Enhance team communication

RECOMMENDATIONSTask-Person-Technology Fit
        - Humans vary in expertise, current conditions of cognitive load and need for choice and flexibility.
        - User expertise and role identification - based views.
        - Search, query and custom views
        - Tailor views to situations - busy settings may have different views

EXAMPLES
        - Antibiotic prescribing support
        - Assessing decision-making capacity
        - Patient Education
        - Diagnoses
        - Identification of Adverse Drug Events
        - Management of HPT
        - Flu vaccine
        - Depression Screening
        - DVT prevention protocols
        - Prevention of constipation resulting from narcotics
        - Fall Screening

High Task Complexity/Difficulty

Low Task Complexity/Difficulty

Embed Interventions and Increase Control

Increase Information/

High Flexibility

Current VA Research Examples

Minimizing Harm from ADEs by Improving Nurse-Patient Communication - Weir (PI)

VA funded research project

Medication Management is a complex, multi-disciplinary activity

Analyze Medication Management Communication Patterns

Design an Intervention that enhances information exchange between providers on medication management issues

“VA Integrated Medication Manager” Nebeker (PI)
        - VA-based development project
        - Improves information display for management of hypertension
        - Includes symptoms, medication history changes and relevant lab data.
        - Uses human factor methods to identify task characteristics.

THE END