Outpatient Outpatient Waiting Waiting Time Measu

Measu

Patient Satisfaction

Julia Prentice and Steven Pizer

HERC Cyber Seminar

May 16 2012 May 16, 2012 

Funding for this work was provided by the VA Systems Redesign Office . We are indebted to Matthew Neuman of the VA Boston Healthcare System and Stacey Campbell of VSSC for pr

Th i e v ews expressedin thi s presentati f on are those of the authors and d onot necessaril y repre

policy of the Department of Veterans Affairs or Boston University. 

www.hcfe.research.va.gov 
decade

Wait Times are a Key Policy F

• VA has monitored wait times for over 

decade

‐ Before 1999, anecdotal evidence on waits

• Congress Congress requested requested wait time data• VA began systematically collecting wa

data
Interventions Interventions to Decrease Decrease W

• Performance measures• Advanced Advanced Clinic Access in six target clinics clinics

‐ 2000

• Primary care panel sizes

• Limited enrollment to priority 7/8

‐ 2003 to 2009
Time 

to first next available 

appoin

tments for new patient 

Wait Times Have Decrease

100 

Mean 10% 90% 

80 90 60 70 40 50 20 30 

0 

10 

Jan-02

Apr-02

Jul-02

Oct-02

Jan-03

Apr-03

Jul-03

Oct-03

Jan-04

Apr-04

Jul-04

Oct-04

Jan-05

Apr-05

Jul-05

Oct-05

Jan-06

Apr-06

Jul-06

Oct-06

Jan-07

Apr-07

Jul-07

Oct-07

Jan-08

Apr-08

Jul-08
Concerns Concerns About Waits Still Re

• VA OIG audits wait time policies policies

‐ Access to mental health care‐ April 2012

• Congressional Congressional hearings hearings on access

‐ SVAC April 2012

‐ HVAC May 2012
Reliability liability of Wait Time Measures Time Measures is U

U

• VA has used a variety of wait time measu• Initiatives Initiatives to decrease decrease wait times require require 

measures

‐ Little research research has used wait time measures measures t

predict outcomes

• This study aims to fill this knowledge gap

‐ Today focusing focusing on patient patient satisfactio satisfaction results

‐ Future analyses focus on health outcomes
Wait Time Measures

• Capacity measures

‐ First next available (FNA)

• Time stamp measures

‐ Create date (CD)

‐ Desired date (DD)

• Access list measures measures

‐ Create date (CD)‐ Desired date (DD)
FNA Calculation

• New patient A requests to be seen as 

possible on January 5, 2010

• First next available appointment is Jan

2010

• Wait time = 5 days (O‐X)

X

1/5/2010 1/10/2010 O
FNA Measure

• Overall supply in system

‐ Patient availability/preferences not considere

• Schedulers distinguish between follow‐u

and urgent care appointments

‐ More problematic problematic for established established vs. new pa

pa

• Appointment Appointment type/Multiple type/Multiple physician physician pr

pr

‐ FNA appointment type is not what patient ne

‐ Cannot consult all sch de uling profiles for sam
Previous Research

• Veterans visiting VA facilities with longer 

have poorer health outcomes

• Mortality, preventable hospitalization for 

veterans

• Mortality Mortality, preventable preventable hospitalization hospitalization, AM

stroke, HbA1c

‐ Veterans with diabetes

‐ Veterans over age 70 and with greater comorb
FNA Measure Limitations

• Patient availability/preferences not cons• VA managers explored other options
Create Date Time Stamp Calcu

• New patient A requests to be seen as 

possible on January 5, 2010

• Cannot take January 10, 2010 appoint

• Appointment is scheduled for January 

• Wait time= 16 days (Y‐X)

Time Stamp CD FNA 

O Y 

1/5/2010 1/10/2010 1/21/

X 
Use o eca ste s e sus sc edu

Create Date (CD) Time Stamp M

M

• Little information required of schedu

clerks

• Based on completed appointments

‐ Excludes Excludes no‐shows, cancellations cancellations

• Use of recall systems versus schedulin

follow‐up appointments right away

‐ Influences wait time‐ More problematic for established versus 

patients
Desired Date Time Stamp Calc

• Established Patient B requests an Apri

follow‐up appointment on January 1, 

• Appointment Appointment is scheduled scheduled for April 20• Wait time= 15 days (Z‐W)

Desired Date T

W 

1/5/2010 4/5/2010 4/2X
Desired Desired Date (DD) Time Stamp Me

Me

• In 2010, VA shifted to desired date me• Not influenced by use of recall systems

• Takes into account patient preferences
Desired Desired Date (DD) Time Stamp Me

Me

• Schedulers must correctly enter desire

‐ Oi i r g nal DD k t ep when negotiating appoit 

n 

‐ E.g. May 1st versus May 5th

• Extensive Extensive training training of schedulers schedulers

‐ Implemented in 2010

‐ Audits find date is entered correctly 90% of 
Prospective Prospective Access Measur

Measur

• Time stamp measures are retrospectiv

‐ Only includes includes completed completed appointments appointments

‐ Patient no‐shows not included‐ Cancellations Cancellations that are not rescheduled rescheduled not 

• Access list measures are prospective

‐ Calculate waits off of pending appointmen

‐ Includes no‐shows and cancellations
Access List Create Date Calcu

• New patient A requests an appt ASAP on Janua

• Appointment is scheduled for February 10, 20

• Bi‐monthly report dates (1 and 15th of each mo

• Appointment is not eligible for calculation unti

equal to or before report date

• 1/1/2010 report‐ appt. not included• 1/ / 15 2010 report‐Wait time = 10 days (R2‐O)• 2/1/2010 report‐ Wait time = 26 days (R3‐O)

Access CD 

O 

Access CD 

R1 R2 R3 

1/1/2010 

1/5/2010 

1/15/2010 2/1/2010 
Access List CD Measure

• Performance Performance measure measure is percent percent of ap

that have less than a 14 day wait

‐ We average waits to match other measur

• Influenced by how follow‐up appts. a

sch de ul de (e.g. recall systems)
Access List Desired Date Calcu

• Established Established Patient Patient B requests requests an April 

follow‐up appointment on January 5, 2

• Appointment is schdld e u e for May 5, 20• Report dates are 1st and 15th of each m

• 4/15/2010 report‐ Wait time= 10 days (• 5/1/2010 report‐ Wait time= 25 days (R

Access DD Access DD Access DD 

X O R1 R2 

1/5/2010 

4/5/2010 

4/15/2010 5/1/2010 
Access List DD Measure

• Performance measure is percent of ap

that have less than a 14 day wait

‐ We average average waits to match other measure

measure

• Schedulers must correctly enter DD
Summary of Wait Time Meas

• New versus established patients

• Time Stamp Retrospective

• FNA‐Access Prospective

A CD 

Time Stamp CD 

Access CD 

O FNA X R W

1/5/2010 1/5/2010 1/10/2010 1/15/2010 1/21/2010

Access DD 

W 

DD Time 

X W Y 

1/5/2010 4/5/2010 4/15

X 
Research Question

• How well do alternative measures of 

times predict patient satisfaction?
Surveys Surveys on Patient Patient Satisfac

Satisfac

• Access is a key component of satisfaction

• Difficult to judge technical quality• Patients focus on practical aspects of their healthcare experience
Satisfaction Data

• 2010 Survey of Healthcare Experiences of Patients (SHEP)

‐ Managed by Office of Quality and Performance‐ Modeled after Consumer Assessment of Healthcare

Providers Providers and Systems Systems

• Simple random sample of patients with completed appointments each month

• Visit date of appointment is recorded• n=221,924 people
Access Satisfaction Measu

1) Appt. as soon as wanted (Timely appt.)

2) Ease of getting test or treatment in last 12 months? (Treatment 

3) Ease of accessing specialist visit (Specialist access)

• Asked for the last 12 months

‐ Most recent visit is likely in mind

• Coded as Always/Usually vs. Sometimes/Never
General General Satisfaction Satisfaction Meas

Meas

4) Rate VA health care in last 12 months (V

‐ On a scal f 1 e o (worst) 10 to (best)‐ 9 or 10 versus <=8

5) Satisfaction Satisfaction with VA at recent visit (sati

(sati

‐ Likert scale 1 (least) to 7 (most) of satisfacti

‐ 6 or 7 versus <=5
atc ed to s t date e de t as se ected o sa 

Analyses

• Logistic Logistic regression regression predicting predicting satisfaction satisfaction• Wait time measures

‐ High volume clinic stops

‐ Patient/provider interactions

‐ All major medical sub‐specialties‐ Facility level monthly averages (including Access measures)

‐ Matched to visit date when respondent was selected for sam

‐ Quartiles
‐

Analyses Continued

• Risk adjustors adjustors from SHEP

‐ Sex‐ Race‐ Age‐ Education‐ Health care utilization‐ Health status
Descriptive Descriptive Statistics Statistics of Sam

Demographics Mean Age 67Male 95%Had some college 53%White 79%Black 10%Hispanic 5%Other 6%> 5= visits to a doctor’s office in last 12 31%

months

Excellent/very Excellent/very good self‐reported reported health 25%

status in last 12 months

25%
Descriptive Statistics of Satisfa

Measures

Satisfaction Measure %

Timely visit

Always/Usually vs. Sometimes/Never 83Treatment accessAlways/Usually vs. Sometimes/Never 85

Specialist access

Always/Usually vs. Sometimes/Never 82VA rating in last 12 months9 or 10 versus <9 78VA satisfaction at most recent visit6 or 7 versus <6 82
Mean Wait Times

353025

Wait in 

Days 20

1510

50 

Tim

Tim

30.91

30.18 

FN

20.07 

17.97 

Ac

15.58 

Ac

7.89 7.91 

4.72 

2.52 2.72

New Patients Established Patients
New Patient Patient Results Results

• How well do alternative measures of 

times predict patient satisfaction?

‐ Longer waits predict lower satisfaction
New Patient FNA/CD Timely 

1

0.90.80.7

Odds 0.6

Ratio 

0.50.40 3.0.20 1.

0

FNA Time Stamp CD

Access CD

*Everything is significant at P<0.05 

Reference group is VA facilities in quartile 1 
New Patient DD Timely Vi

1.15 

* 

1.1 * * 

Odds 

Ratio 

1.05 

* * 

1 

0.95 

0 9. 

Time Stamp DD Access DD 

*Significant at P<0.05 in wrong direction 

Reference group is VA facilities in quartile 1 
0.8 0.8 0.8 0.9 

New Patient Waits Predicting Satisf

Measure Timely Treatment Specialist VA rating 

Appt. Access Access 

FNA (ref=Q1)Q2Q3

0.89*0.82*

0.93*

0.84*

0.93*

0.84*

0.95*

0.92*

Q3 Q4 0.73* 0.73* 0.74* 0.86* 

Time Stamp CD

( f re =Q1)

Q2 0.86* 0.87* 0.86* 0.96* Q3 0.78* 0.80* 0.81* 0.91* Q4 0.67* 0.65* 0.67* 0.83* 

Q4 0.73* 0.72* 0.72* 0.89* Access CD(ref=Q1)

Q2Q3

Q4

0.83*0.85*0 73*

0.84*0.84*0 72*

0.86*0.85*0 72*

0.93*0.93*0 89*

Numbers are odds ratio from logistic regression. * Signi
New Patient Waits Predicting Satisf

Measure Timely Treatment Specialist VA rating 

Appt. Access Access 

Time Stamp DD(ref=Q1)Q2 1.06* 1.01 1.05* 1.00Q3Q4

1.061.10*1.06*

1.011.06*1.02

1.051.10*1.07*

1.000.981.01

Access DD

(ref=Q1)Q2 0.98 0.97 0.98 0.94* Q3 1.12* 1.09* 1.11* 1.00 Q4 1.10* 1.06* 1.11* 1.00 

Numbers are odds ratio from logistic regression Numbers are odds ratio from logistic regression. 

* Significant at P<=0.05 * Significant in wrong direction 
New Patient Results

• FNA and create date measures are m

reliable

‐ Predict all 5 satisfaction measures

• New patients patients want to be seen as soon a

‐ Date an appointment request was origin

made is reliable
Established Patient Resu

• How well do alternative measures of 

times predict patient satisfaction?

‐ Longer waits predict lower satisfaction
*

Established Patient FNA/C

1.041 02 . 1.00 0.98 

Odds 0.96 

Ratio 

0.94 0.92 0.900 88 . 0.86 0.84

FNA

* 

* 

* 

* 

* 

Timely Vi it s

Time Stamp CD

Access CD

*Significant at P<0.05 Reference group is VA fac
Established Patient DD Timely 

1.20 

* 

1.00 * 

* 

* 

* 

Odds 

Ratio 0 60 

0.80 

* 

0 40 0.60 0.20 0.40 0.00 

Time Stamp DD Access DD 

*Significant at P<0.05 *Significant in wrong direction at P<0

Reference group is VA facilities in quartile 1 
Established Patient Waits Predicting Sa

Measure Timely Treatment Specialist VA rating 

Appt. Access Access 

Q4 0.90 0.90 0.90

( Q )

Q2Q3Q4

1.021.000 90*

FNA (ref=Q1)

0.991.02

0 90*

1.011.03

0 90*

0.970.95*

0 90 . *

Time Stamp CD

(ref=Q1)Q2 0.92* 0.90* 0.94* 0.96* 

Q3 0.91* 0.87* 0.89* 0.94* Q4 1.02 0.96* 0.99 1.07* 

Access CD(ref=Q1)Q2Q3Q4

0.94* 0 93* 0 94* 0 94* 0.96*1.02

0.93*0.95*0.98

0.94*0.93*0.97

0.94*0.980.95*

Numbers are odds ratio from logistic regression. *Significant at P<=0.05 *Significant in the wrong direction 
Established Patient Waits Predicting Sa

Measure Timely Treatment Specialist VA rating 

Appt. Access Access 

Time Stamp DD(ref=Q1)Q2 0.99 0.97 0.98 0.94*Q3Q4

0.991.05*0.96*

0.971.030.94*

0.981.020.97

0.940.980.95*

Access DD

(ref=Q1)Q2 0.91* 0.90* 0.88* 0.94* Q3 0.86* 0.86* 0.83* 0.86* Q4 0.80* 0.77* 0.76* 0.86* 

Numbers are odds ratio from logistic regression Numbers are odds ratio from logistic regression. 

*Significant at P<=0.05 

*Significant in wrong direction 
Established Established Patient Patient Resu

Resu

• Access list desired date is most reliable

‐ Includes no‐shows/cancellations

‐ More accurate accurate measure measure of supply in system

• DD reflects reflects established established patient patient prefere

prefere

versus FNA/CD measures
Policy Implications

• M lti l Multiple wait time measures need de

‐ New versus established patients

• New patients want to be seen right awa

‐ Change in health status

‐ Appointment request dates are reliable‐ Capacity and create date measures
Policy Implications Implications Contin

• E t bli Establish de patients may not pri iti or ze wait times

‐ Continuity of care

‐ Convenient time

• VA is a leader in recognizing new versus established 

compl it ex y

• Future work will predict predict health outcomes outcomes
Questions or Comments

Julia Prentice Julia Prentice

Julia.Prentice@va.gov

(857)-364-6057

www.hcfe.research.gov

Resources

1. Prentice J; Pizer SD. (2007) Delayed Access to Health Care and Mortality. Health Services Research. 

42(2): 644-662. 

2. Prentice JC, Pizer SD. (2008) Waiting Times and Ambulatory Care Sensitive Condition Hospitalizations. He

Services and Outcomes Research Methodology. 8(1):1-18. 

3. Pizer SD Prentice JC (2011) What Are the Consequences of Waiting for Health Care in the Veteran Popula

Pizer, SD, Prentice JC. (2011) What Are the Consequences of Waiting for Health Care in the Veteran Popula

Internal Medicine 26(supplement 2):676-682. 

4. Pizer, SD, Prentice JC. (2011) Time is money: Outpatient waiting times and health insurance choices of elde

States. Journal of Health Economics 30(4):626-636. 

5. Prentice JC, Fincke BG, Miller DR, Pizer SD. (2011) Outpatient Waiting Times and Diabetes Care Quality 

Improvement. American J ournal of Managed Care 17(2):e43-e54 .

6. Prentice JC, Fincke BG, Miller DR, Pizer SD. (2012) Waiting for Primary Care and Health Outcomes among 

Diabetes. Health Services Research 47(1):46-67