The Orderly and Effective Visit: 

Impact of the Electronic Health

Record on Modes of Cognitive 

Control

Charlene Weir, PhD & Jonathan R. Nebeker

Frank A Drews, PhD

Molly K Leecaster, PhD

Robyn J Barrus, MS

James L Hellewell, MD
OVERVIEW

• Joint Cognitive Systems

• Contextual Control Model

• Concepts of effectiveness

• Aims of the Study

• Methods

• Results

• Conclusion
Overarching Goal

• Medical care is information intensive

• Current user interfaces are hard and 

unpleasant to use

• Must develop user information environments 

that improve efficiency and quality of care.

– Efficiency measured in time (easy)

– Quality measured in patient outcomes (hard)

• Need model to predict and measure effects of 

user interfaces on quality of reasoning
Joint Cognitive Systems

• An interdependent emergent system

• “Cognition” is distributed across system

– computers, people, data, displays

• Dimensions of a joint cognitive system:

– Goal-orientation – increase order/alignment

– Orderly/ Under Control – a cycle that detects 

deviations from goal and institutes adaptive actions

– Co-agency – Collective energy of the system
As-is EHR

• Building blocks of EHR are data domains

– Meds, Labs, Rad, Notes, Diagnoses, etc.

• Results in bad thought-flow and workflow

5
Contrasts with Usual Approaches

• Cognitive support not decision support

– quality of reasoning not right answer

• Joint not separate systems

– Joint best not human best and computer best

• Cognitive-social psychology not ergonomics

– Thought cycles over seconds to minutes not ms
Contextual Control Cycle
Contextual Control Components

• Goal Interactions - the degree to which goals are 

defined and integrated 

• Time Horizon - breadth of information, change over 

time and future forecasting

• Assessment of Uncertainty –Limitations of available 

information is identified, explained and adapted to

• Decision heuristics - Action plan is customized to the 

current situation 
Dependent Variables – Control Mode
 Description of Study
Aims / Hypotheses

• Aim 1: The intensity of pre-visit planning 

would differ performance level

• Aim 2: Higher level of performance associated 

with lower frequency of searching during visit. 
METHODS

• Design: Correlational

• Settings: 5 VA hospitals

• Participants: 45 primary care providers (35 

MDs, 10 mid-levels) 
METHODS

Procedures

• 3 phases (audio-taped)

– Phase 1: Interview prior to visit / prep

– Phase 2: Interviews in person (audio-tapes); 

screen shots captured manually (notes, 

orders, medications, consults, and 

labs/procedure results). 

– Phase 3: Post-visit interview
Dependent Variables 

• Control Mode (coded orderliness) (IRR > 0.80)

– Strategic – highest level of control/orderliness

– Tactical – structured, but rigid, guideline based

– Opportunistic – scattered, bouncing from highly salient 

– Scrambled – very unstructured and chaotic

• Pre-planning intensity: coded on a 1 (low) to 7 (high) 

scale. Independent of Mode

• Searching Activity: the proportion of overall screen 

changes dedicated to searching alone

RESULTS - Descriptives
% Cognitive Mode

Strategic

15%

Tactical

39%

Opportunistic

46%

Scrambled

0%
Screen Changes

• Mean # of screen changes = 18.5 per visit 

– 60% searching

– 40% entering notes or orders

Searching, 

60%

Entering, 

40%

, 0 , 0
• Control mode NOT related to years since 

graduation, familiarity with the patient, 

ratings of levels of mental stress, and overall 

time of the visit. 
Pre-planning Measurement: CTA

“Please think aloud what is in your mind while 

you do this task. Please indicate what you are 

trying to do. There are no right or wrong things 

to say.” 

“Please tell me what you’re doing explicitly 

during the preparation templating and 

updating process. You’re doing this in order to do 

what? And you’re doing that by doing what?”
Intensity of Preparation Coding

• 1 (not well) to 7 (very-well) scale

• 2 independent coders, ( R>0.80)

– Medication review

– Pre-review of prior visits and intervening events 

(reading progress notes)

– Review by problems/associated labs

– Establishment of goals of visit
Aim 1: The intensity of pre-visit 

planning would differ across mode

• Mean = 4.9

• ANOVA (F2,23=5.64; p=0.01). 

• Statistical significance remained after 

controlling for years of experience (F 2,23 = 

6.62; p=0.01). 
RESULTS – Pre-Planning (1 to 7) 
Aim 2: Higher level of performance associated 

with lower frequency of searching during visit

• Higher levels of performance associated with 

lower incidence of search activities 

 (F 2,30 = 6.54; p=0.004)

• After controlling for pre-planning:

 (F 2,22 = 4.2; p=0.03) 
Measurement of Search Activity

• Research assistants in room with patient and 

provider

• Used tablet with time-stamped software

• Tape-recorded visit

• Noted every change of screen 
RESULTS – Search Activity
Entering Screens

• Visits rated at a higher level of control had 

higher proportion of computer interactions 

involving ordering.

 ANOVA, (F2,22 = 9.26; p=0.001) 

• After controlling for pre-visit planning:

 (F 2,22 = 4.9; p=0.02). 
RESULTS – Entering Notes/Orders
Conclusions

• Pre-visit preparation of electronic note was significantly associated 

with higher levels of visit control after controlling for years of 

clinical experience.

• Searching activity significantly inversely associated with visit 

performance after controlling for degree of pre-planning efforts.

• Implication is that the EHR is not providing adequate cognitive 

support for visit workflow.

– Information has to be very organized to support a complex primary care 

visit.

– Searching during a visit is very ditsracting and decreases performance
“IT applications appear designed largely to 

automate tasks or business processes. They 

are often designed in ways that simply 

mimic existing paper-based forms and 

provide little support for the cognitive tasks 

of clinicians or the workflow of the people 

who must actually use the system.” (p. 3) 

Stead WW, Lin HS. Computational technology for effective health care: 

immediate steps and strategic directions. Washington, D.C.: National 

Academies Press; 2009. 
Changing Work Processes

Information Overload

Adaptive Strategies
Limitations

• Not a randomly controlled trial

• Personality/cognitive factors may be 

associated with pre-planning and ALSO 

associated with visit performance (thus 

producing a confound)

• Study done in the VA which may not be 

generalizable
Acknowledgements

Research supported by:

AHRQ 1R18HS017168-01 (Nebeker, J)