Using VistAWeb for Stroke 

Quality of Care Research

Linda S. Williams, MD

Research Coordinator, VA HSR&D Stroke QUERI

Roudebush VAMC

Professor of Neurology, Indiana University

Indianapolis, IN
Overview 

• The INSPIRE QUERI SDP

• Why central chart review?

• Examples of using VistAWeb and CAPRI for 

research 

• Lessons learned

• Questions
Audience poll question 1

What is the coldest temperature (oF) you’ve experienced 

at your VA during 2014?

a. < -20 

b. -19 to -10 

c. -9 to 0

d. +1 to 10

e. +11 to 20

f. +21 to 30

g. +31 to 40

h. >+40 
INSPIRE SDP Background

Chose improving 

stroke care as one 

element of their 

VISN-wide focus 

for FY08

VISN 11 

Pilot Project

Office of Quality 

and 

Performance

Stroke 

QUERI

Conducted 1st 

national 

measurement of 

inpatient stroke 

care in FY07-09 

Worked with OQP to develop and measure VA stroke care, 

used these data to pilot a Systems Redesign-based 

collaborative in VISN 11 FY08-09 
National Results by Phase of Hospitalization

Consolidated measure = number of passes

 number of opportunities

IN-HOSPITAL 

(87%) 

Antithrombotic (95.6%)

Pressure ulcer (91.8%)

Early ambulation (86.1%) 

Rehab/FIM (86.0%) 

Fall risk (79.3%)

VTE Prophylaxis (78.2%) 

DISCHARGE 

(73%) 

Antithrombotic (96.4%)

Smoking cessation (94.9%)

Lipids (82.1%)

Anticooag for AF (75.3%) 

Stroke education (18.1%)

EARLY 

(25%) 

NIHSS (27.7%)

Dysphagia (23.4%) 

tPA (8.4%)
INSPIRE SDP: 

Intervention for Stroke Performance 

Improvement using Redesign Engineering

• Study aims: 

1. To evaluate the effect of public reporting on 

organizational change to improve stroke care 

quality in VA

2. To conduct a formative developmental evaluation 

in 12 of the largest volume VAMCs to understand 

organizational barriers and facilitators to the 

delivery of high-quality stroke care

3. To test a Systems Redesign/OSE-based 

intervention vs. performance feedback alone 

in improving 2 stroke indicators
INSPIRE Intervention (Aim 3)

• 11 sites randomized to intervention vs. control

– Sites with at least 50 stroke admissions annually

– Intervention sites: In-person collaborative with OSE/SR 

training, followed by 6 months of coaching through rapid 

tests of change; monthly quality indicator feedback

– Control sites: quarterly quality indicator feedback

• Quality indicators collected for 1 year prior and 12 

months post-intervention

• Analyses: 

– Primary: improvement in individual and composite 

performance indicators from baseline to post-intervention

– Secondary: temporal pace of change in performance, 

sustainability of change, qualitative assessments of 

local context and relationship to changes observed
Quality indicator assessment

• Primary outcome is improvement in two QIs:

– Venout thromboembolism (VTE) prophylaxis and 

dysphagia screening before oral intake

• Eight other Joint Commission-based QIs and 

one VA QI also collected

– E.g. thrombolysis (tPA) for eligible patients, 

antithrombotic medication by hospital day two, 

discharge on cholesterol lowering medication
Question 1: Why chart review? 

• None of the inpatient stroke quality indicators 

were being collected as part of routine VA 

care in either paper or electronic form

– IPEC Stroke Module available for self-reporting 3 

quality indicators as of July 2012

– tPA for eligible patients, dysphagia screening, NIH 

Stroke Scale completion

– FY 2013 had about 1/3 of facilities self-reporting
Question 2: Why VistAWeb?

• Data required to construct the QIs are not part 

of VA EHR data

– Many key elements not present in the EHR, e.g. 

dysphagia screening status, mechanical VTE 

prophylaxis devices, documentation of comfort 

measures do not have standard health factors 

across facilities
VTE prophylaxis flowchart

INSPIRE Collaborative, 4/27/11

4. Were appropriate 

medications or 

mechanical 

prophylaxis given by 

HD2?

2. Was the patient ambulatory 

by hospital day (HD) 2? 3. Were “comfort measures 

only” documented by HD2?

1. Was the patient hospitalized 

for at least 2 days?

5. Were any contraindications to 

meds and mechanical prophylaxis 

recorded by the provider?
Question 2: Why VistAWeb?

• Data required to construct the stroke QIs are 

not part of VA EMR data

• CDW/VINCI not operational at the time the 

study began

• Local chart review

– Expense, training, quality control
INSPIRE chart review planning

• 3.0 FTE chart reviewers working for 12 

months: 

– 118+ variable chart review form, 11 quality 

indicators

• Some multiple response variables (medications on 

admission and discharge)

– Weekly abstractor meeting

• Review questions, clarify and update manual, bring 

examples

– Opened 2,305 charts

• 1,600 full review

• ~160 random 10% inter-rater reliability
Chart review expense example 

• 11-site study, reviewing 2.5 years of stroke 

admissions (estimated 2,400 charts fully reviewed)

– Site level load is approximately 75 stroke cases per year

– Effort for that number of fully reviewed charts would be 

low (~2 charts per week)

– Difficult to find sites willing and able to hire some small 

% of a research assistant 

• Even 25% of a GS7 RA at 12 sites (total 3.0 FTE) would not save 

any $$ compared to hiring only at the coordinating site

– Training, maintaining, retaining all these personnel over 

a 3-year study is not feasible 
Central Chart Review Quality

– Data quality extremely high

• 113/118 variables > 0.8 ICC/kappa, none of these in the 

quality indicator algorithms (e.g. past history variable)

• QI result agreement (ineligible, passed, failed) excellent 

with kappas 0.84-0.96

0.9

0.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1

ICC_1k

ICC_1k

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

0.6

Gender

nonstrdx2

assessrhb

dcdocwarn

dcthrom4

ipafib

whynowar

norxpro

tpacon5c

wicthrom7

pasthx8

pasthx22pasthx32

assesev1

ltarm

kapp

kapp
Advantages to using 

central chart review

• Access to multiple sites from a central location

– Facilitates training, review, and quality control for chart 

abstractors

• Collecting text-based data elements with high inter-site 

variability (difficult to use text mining/NLP)

– E.g. different staff and methods for dysphagia screening, 

assessment of ambulatory status, different locations/notes 

for recording stroke severity data or stroke education data

• Data elements that reflect clinician judgment

– E.g. documentation of reasons not to anticoagulate

• Facilitates data feedback and discussion with site teams

– Research team “sees” what the site team sees
Audience poll question 2

What experience do you have using central 

chart review for VA research?

a. Never tried to do this before

b. Used VistAWeb

c. Used CAPRI

d. Used both VistAWeb and CAPRI

e. Used some other platform for central chart 

review
Options for central EHR review

• VistAWeb: 

– Provides read-only EHR access for a given patient 

across all sites of care for that patient

• CAPRI: Compensation & Pension Record 

Interchange

– Provides read-only EHR access for a given patient 

at a single site at a time (can switch to other sites)

– Has VistAWeb tab that can shift the viewer into 

VistAWeb view of the EHR 

Access request forms through the VA Data Portal: 

intranet only, contact VIReC@va.gov for link
CAPRI vs. VistAWeb

Characteristic CAPRI VistAWeb

Requires installation of special software Yes No

Requires unique access/verify code pair (i.e., 

does not use local VistA access/verify code pair)

Yes No

Views EHR one site at a time Yes No

Views EHR of all sites visited in one consolidated 

view, organized chronologically

No Yes

Provides progress notes (i.e., text) search 

function 

Yes No

Provides seamless transition between EHR 

applications (CAPRI/VistAWeb)

Yes No

http://www.virec.research.va.gov/CAPRI-VistAWeb/CAPRI-VistAWeb.htm

My personal recommendation: get access to both 

for maximum flexibility
Examples: Possible considerations in 

using VistAWeb

• Variable time to load pages

– Can take seconds to as much as 15 minutes to load notes 

from a selected date range

– Varies by time of day, location, and ???

– This seems to be improved with the current access as a 

tab in CAPRI

– Text searching of some data, e.g., orders, note titles (not 

note text)

• Easy to make abstraction errors when separate 

locations share a single facility identifier

• If tracking transfers between facilities is important, 

then VistAWeb may make that more obvious, but you 

still have to be careful viewing notes from stations with 

multiple facilities!
Example of admission data with 

site identifiers: VistAWeb

Patient cared for at 

Nashville VAMC

Patient cared for at 

Murfreesboro VAMC
Example of notes with site identifiers: 

VistAWeb
CAPRI Log In
CAPRI Patient Selection Screen
CAPRI view of a patient’s record

VistAWeb tab
Example of text Search Function 

within notes in CAPRI

In the “Clinical Documents/Notes” 

tab, all notes from one facility are 

shown
Using the search term “carotid” 

(bottom right corner) only notes 

containing this word are shown, 

and the word is highlighted within 

the note.
The find command can also be 

used with some types of data in 

VistAWeb, here for example to 

search within the orders for 

“aspirin” 
Lessons learned about using 

CAPRI/VistAWeb for research

• Optimal way to collect complex clinical chartrecorded data

– Consider complexity and data quality requirements

– Develop standard chart review manual and update 

with local examples as they are noted 

– Standardize search features and terms

• Increase estimated time per chart review from 

local CPRS by some small factor (10-15%) to take 

into account view switching and page loading 

issues

– However, additional search features may end up 

saving time depending on what you are looking for 
Lessons learned about using 

VistAWeb for research

• Organize process for access requests

– Designate one person from your study to submit 

and stay in communication via the DART process

• Keep your chart reviewers happy!

– Breaks for other types of work

– Shared positions if possible

– Regular team meetings to discuss 

questions/resolve differences

– Prizes for “Best Story of the Week”
Resources/Help

• VHA Data Portal: Access for CAPRI and 

VistAWeb 

– intranet only, contact VIReC@va.gov for link

• VistA Documentation Library: 

– http://www.va.gov/vdl/default.asp 

• VIReC tutorial “Using VistAWeb for the First 

Time for Research”: 

– intranet only, contact VIReC@va.gov for link
Thanks to the most amazing group of 

chart reviewers and data managers:

Nick

Catherine

Dani

Annie

Flossy 

Jeff

Jenn 
INSPIRE Study Team

• Investigators: 

– Linda Williams, MD

– Teresa Damush, PhD

– Ginger Daggett, PhD

– Marianne Matthias, PhD

– Edward Miech, EdD

– Laurie Plue, MA 

– Joe Ross, MD, MS

– Heather Woodward-Hagg, MS

• Biostats and Data 

Management: 

• Zhangsheng Yu, PhD

– James Slaven, MS

– Jeff Barnd, MS

– Jeff Fahner, MS, BSN

• Engineers:

– Balmatee Bidasee, PhD

– Marianne Pilat, PhD

– Angie Howard, BSN

• Project management: 

– Dani Sager

– Ginger Daggett

• Research assistants/chart 

reviewers: 

– Catherine Bauer-Martinez 

– Nick Burris

– Flossy Lincoln

– Jenn Myers

– Annie Plahitko

– Josh Robinson

Collaboration between Stroke QUERI and the VA Center for Applied Systems Engineering (VA-CASE)
Questions?