Measuring EHR Usage and Patient-Provider 

Communication During Outpatient Visits

Dr. Zia Agha

Director VA San Diego HSR&D

Professor of Medicine UCSD

Conflict of Interest Disclosure

Have no real or apparent 

conflicts of interest to report.

Research funded by Veterans Administration

HSR&D Project IIR-07-196-2 
Health IT Usability

Science & Engineering

• Define usability as “effectiveness, efficiency

and satisfaction with which users can perform 

intended tasks”

– Measuring usability (a non-functional, lifecycle 

property) is a science. 

– Improving usability is achieved through 

engineering (functional requirements)
Learning Objectives 

• Describe EHR usage and task efficiency leveraging 

time-domain process measures

• Recognize challenges posed by current 

technology to providers who multitask between 

EHR workflow and addressing patient needs

• Identify and rank-order the most commonly used 

EHR components

• Discuss how to Correlate EHR usage with patient- provider communication patterns

• Recognize how to Leverage analysis results for 

EHR design improvements
PACE Study Motivation

• Little quantitative data from real clinical 

settings (vs. lab experiments with 

predetermined tasks)

• Clinical settings enable sampling realistic: 

variability across patients, multitasking (patientprovider-EHR interaction)
Profile EHR usage and task efficiency

Leveraging time-domain measures

• PACE study:

– Outpatient visits; established patients 

• Time-stamped visit process data:

– EHR activity; Non-Verbal workflow; Verbal

Communication

• Additional data:

– patient & provider satisfaction; agreement; 

interviews
Multi-Level EHR Activity Data

Usability software logs time-stamped mouse 

activity + manual review of screen video:

– Screens: Notes, Labs, Medications, Reports, 

Clinical Reminders

– Tasks: View v. Order Entry; Current v. Previous 

Note

– Modes: Text Box, v. Radio Button v. Scroll Bar

– Order Entry: Meds, Labs, Consults, Imaging

– Alerts & Reminders 
Rank-order EHR component usage

• Hierarchical Task Analysis

– Rank-Order: screens, tasks by visit, by provider... 

• Sequential Task Analysis

– Most frequent transitions between screens, 

tasks...
Temporal analysis of EHR + Clinical 

Workflow 

• Time-at-task profiling excluding physical 

exams, interruptions

• Conditional profiling of EHR activity while 

sharing screen 

• Verbal analysis during 3-minute windows of 

highest & lowest EHR activity excluding 

interruptions
Types of Visit Process Data by Source

Room 

Video

NonVerbal & 

Clinical 

Workflow

Vocalization, 

Verbal

Usability 

Software

Mouse 

Activity

EHR Screen 

Recording
Synchronizing EHR Activity + 

NonVerbal

2 sample visits Top Chromogram=EHR 

Mouse Clicks

Bottom Chromogram=NonVerbal

& Clinical Workflow
Categories of User-EHR Interactions 

User-EHR interactions:

– Information Retrieval: browsing, searching, 

decision-support

– Documentation: progress notes, reports

– Order Entry: meds, labs, consults, imaging

– Coordination: reminders, care team 

communication
Estimate average number of mouse clicks per 

office visit ?

A) 20

B) 100

C) 250

D) 500
Variability in EHR Activity by Physician
Visit Length vs. EHR Activity
Distribution EHR Screens (Mouse 

Clicks)

Notes 40%

Orders 27%

Meds 10%

Labs 10%

Reports 4%

Consults 3%

Labs 10%
Distribution of View Tasks while in 

Notes

View Current 

Notes 83%

View 

Previous 

Notes by PCP 

7% 

View Notes by 

Other Providers 

7% 

Create New Note 

3% 

By mouse clicks
Distribution of Clinical Work

Gaze to Patient 35%

Gaze to EHR 40%

Physical Exam 16%

Gaze to 

Paper/Meds 6%
Variability of EHR Activity Across 

Visits

Low 

EHR 

Activity

High EHR 

Activity
Longer Visits, More Gaze to EHR

In longer visits, 

physicians look 

at the computer 

more
How do we explain the variability in EHR usage ?

Discuss factors that may explain this.
Variability of EHR Sharing by 

Physician

Physician ID

Width= # Visits 

Sharing EHR

Height= Shared EHR 

Time (Sec)
What Screens Do Physicians Share?

Meds 25%

Labs 45%

Notes 14%

Orders 8%

Cover 

Sheet 5%
Transitions Between EHR Screens

© 2013 HIMSS

Notes

Labs

Orders Review/Sign

Meds

Patient 

Selection

Consults

Other

Cover Sheet

Reports
Transitions Between EHR Tasks

© 2013 HIMSS

Notes

Meds

Reports

Consults

Reminders

Orders

Views

Reminders 

(Notes)

Other

Review/

Sign Consults

Recent 

Labs

Meds

Previous 

Note

Cover/Vita

ls

Cover

Other Provider Notes
Physicians Multitask Throughout Visit

Orders

EHR Screens

NonVerbal

Vocalizations

Example ~60 min visit 

EHR Labs

(Info Retrieval)

EHR Notes

(Documentation + 

Info Retrieval)

EHR Orders

(Order Entry)

Physical 

Exam

VOC: Black = 

Silence
EHR Activity: Mouse vs Time-at-Task

Left bars: 

distribution 

by mouse 

clicks 

Right bars: 

distributio

n by timeat-task
Order Entry Analysis (Mouse Activity)

Bottom: EHR 

Screens

Top: Orders

(vertical axis = 

# items)

2 example visits

Many items 

ordered with 

few clicks per 

item

Few items 

ordered with 

Many clicks
Order Entry Analysis (Meds)

165 mouse 

clicks to order 

6 meds

Each row = 1 visit

This one has no Meds Orders
Order Entry Analysis (Consults)

82 mouse 

clicks to order 

2 Consults

We also profile Labs, Imaging, Reminders... 
Order Entry Analysis Summary
Leverage Results  Drive EHR 

redesign

• Documentation & Order Entry

– Notes: unstructured ASCII file  Structure data

searchable tags, links, autocomplete

– Pull down lists  Automate direct order entry 

• Information Retrieval & Visualization:

– Better Formatting, help Share

– Labs, Meds, clinical events on multiple screens 

leverage visualization like ZUI timeline browsers 
Thank You! 

Zia: zia.agha@ucsd.edu 

Alan: acalvitti@ucsd.edu

...and the PACE Team:

Kristin Bell – UCSD & VA San Diego

Yunan Chen – UCI

Neil Farber – UCSD

Mark Gabuzda – VA San Diego

Barbara Gray – VA HSRD

Lin Liu – UCSD & VA HSRD

Richard Street – Texas A&M U

Danielle Zuest – VA HSRD