Department of Veterans Affairs 

Office of Inspector General 

Audit of 

 Veterans Health Administration's 

Efforts to Reduce 

 Unused Outpatient Appointments 

Report No. 08-00879-36 December 4, 2008

VA Office of Inspector General 

Washington, DC 20420 
To Report Suspected Wrongdoing in VA Programs and Operations 

Call the OIG Hotline – (800) 488-8244 

 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Contents 

Page 

Executive Summary ..............................................................................................i

Introduction...................................................................................................................... 1

Purpose........................................................................................................................... 1

Background .................................................................................................................... 1

Scope and Methodology ................................................................................................ 2

Results and Conclusions ............................................................................................. 4

Reducing Unused Outpatient Appointments Could Improve Timeliness of Veterans 

Patient Care and Enable Better Use of Resources ......................................................... 4

Recommendations ....................................................................................................... 10

Appendixes

A. Review of Unused Outpatient Appointments ........................................................ 12

B. Review of Consults and Patients Waiting for Care................................................ 15

C. Review of Missed Opportunity Strategies ............................................................. 17 

D. Cost of Unused Appointments ............................................................................... 19 

E. Monetary Benefits in Accordance with IG Act Amendments ............................... 20

F. Under Secretary for Health Comments................................................................... 21

G. OIG Contact and Staff Acknowledgements........................................................... 28

H. Report Distribution................................................................................................. 29

VA Office of Inspector General 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Executive Summary 

Results in Brief 

The Office of Inspector General (OIG) conducted an audit to evaluate the effectiveness of 

Veterans Health Administration’s (VHA) efforts to minimize the number of unused 

outpatient appointments. The objectives for the audit were to determine if VHA has an 

effective method to accurately track and report unused outpatient appointments, whether 

VA medical facilities implemented effective processes for reducing the number of patient 

no-shows, and whether unused appointments could be used for patients who are waiting 

for care. 

VHA did not have an effective method to accurately measure and report unused 

outpatient appointments, has not implemented effective processes for reducing the 

number of missed opportunities, and needs to ensure that, where possible, unused 

appointments are filled with patients currently on lists waiting for care. 

We projected about 4.9 million (18 percent) of the scheduled 26.5 million annual 

outpatient appointments in VHA’s 10 performance measure clinics were unused in Fiscal 

Year (FY) 2008. Minimizing the number of unused outpatient appointments could 

enable veterans to receive more timely patient care and VHA to make better use of 

resources valued at about $76 million annually and $380 million over a 5-year period. 

Background 

A steadily increasing demand for VHA’s outpatient services is making it more 

challenging for VHA to provide veterans and their dependents with timely health care. 

Ensuring every possible effort is made to use available appointments helps to address this 

challenge and enhances the use of VHA resources. The three primary groups of unused 

appointments are:

• A patient does not show up for a scheduled appointment. VHA refers to these 

3.1 million “no-shows” as missed opportunities. 

• A patient cancels before their scheduled appointment, but facility personnel do not 

schedule a different patient into the available appointment. In FY 2008, we projected 

this resulted in an additional 1.8 million unused appointments. 

• A medical facility never schedules a patient into an available appointment. 

In some cases, this is done intentionally to ensure providers have the ability to see 

patients needing care that same day by leaving some appointments unscheduled. 

VHA does not have sufficient records to identify these appointments. 

VA Office of Inspector General i 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

In FY 2006, VHA established a performance measure to monitor and track the number of 

missed opportunities for scheduled outpatient appointments. The measure was included 

in the Veterans Integrated Service Network (VISN) Directors’ Performance Plans. This 

measure identified the percentage of scheduled appointments that facility personnel did 

not use because the patients did not show up for their appointments. VHA also included 

those appointments in this measure that were canceled by the patient and clinic, but VHA 

staff did not enter the cancellations into the Veterans Health Information Systems and 

Technology Architecture (VistA) scheduling package until after the appointment times. 

To identify successful practices for reducing missed opportunities, a task force chartered 

by VHA’s National Leadership Board in January 2006 reviewed strategies used by clinics 

with low missed opportunity rates. In their August 2007 report, they identified 

18 strategies in use but concluded that no single strategy worked as well as using multiple 

strategies to reduce missed appointments. The team recommended that facilities should 

assess their own processes and then design and implement solutions targeted to their 

unique needs. They also noted that a future study on clinics that were struggling with 

high missed opportunity (no-show) rates would provide more insight. As of 

October 2008, no subsequent study had been done. 

VHA established missed opportunity targets for 10 performance measure clinics—

audiology, cardiology, dermatology, eye care, gastroenterology, mental health, 

orthopedics, podiatry, primary care, and urology. VHA Directive 2006-028, “Process For 

Ensuring Timely Access to Outpatient Clinical Care” (May 8, 2006), charges VA medical 

facility directors with responsibility for ensuring that a process is in place for monitoring 

and decreasing missed opportunities. 

We projected the number of unused appointments that were canceled by patients before 

the appointment time from a random statistical sample of appointments. Similarly, we 

estimated the number of patients waiting for care as the total number of patients on the 

electronic wait list—patients waiting more than 30 days past their desired dates of care 

for appointments and patients waiting for specialty care. We projected the number of 

patients waiting for specialty care using a random statistical sample of consult referrals 

for specialty care. 

We calculated the average cost of an unused appointment as $182 using cost data from 

VHA’s Decision Support System (Appendix D, Table 1.) Although our methodology 

provided us with a reasonable basis for identifying the overall financial impact of unused 

appointments, it did not allow us to match up available appointments to individual 

patients waiting for an appointment. 

VA Office of Inspector General ii 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Findings 

1. VHA Lacked an Effective Method to Accurately Track and Report Unused 

Appointments. 

VHA did not have an adequate process to monitor and track the number of unused 

appointments. Instead, VHA monitored and tracked only missed opportunities 

(no-shows). In FY 2008, approximately 4.9 million (18 percent) of VHA’s available 

outpatient appointments were unused. Approximately 3.1 million of those unused 

appointments occurred because the scheduled patients did not show up for them. 

Additionally, about 1.8 million of the appointments not used occurred because patients 

canceled them prior to the appointment times, and facility personnel did not refill the 

available appointment slots. 

VHA personnel stated that facilities sometimes use overbooking to minimize the effect of 

vacant appointment slots but could not provide us with reliable information to determine 

the number of overbooked appointments. During our site visits, we asked facility 

personnel about their use of overbooking. We found that although overbooking occurred, 

it was not used very often because the practice of overbooking patients has the potential 

to increase the risk of patients having to wait longer for their appointments. 

We also reviewed the facility Clinic Utilization Statistical Summary (CUSS) report that 

shows the total number of available appointments. We did this to determine how well 

each facility used their available appointments but found it to be inaccurate. For 

example, for the 1st Quarter FY 2008, the CUSS report showed that the six medical 

facilities we reviewed had about one million available appointments, of which about 

745,000 were shown as unused (74 percent). Our interviews with VHA personnel and 

reviews of the clinic schedules revealed that facility staff did not keep schedules current 

to show the true number of appointments available for use. For example, the provider 

covering Tuesdays in one clinic left, and the facility had not replaced the provider. As a 

result, all of the Tuesday appointment slots were reported as unused because the clinic 

schedule had not been modified to show the actual number of providers. This 

erroneously overstated the facility’s available appointments. 

2. VHA Had Not Implemented Effective Processes for Reducing Missed 

Opportunities. 

At the six facilities we reviewed, VHA had processes in place to reduce the number of 

missed opportunities but VHA had not adequately assessed the effectiveness of these 

processes. We interviewed key personnel and reviewed performance data at the six 

medical facilities to determine how many of the strategies were being used and whether 

positive outcomes resulted. Staff at these facilities said that the strategies used by the 

clinics to address this issue had minimal effect in reducing missed opportunities. They 

VA Office of Inspector General iii 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

cited various factors, including weather, lack of transportation, fuel costs, and patient 

preferences as contributing factors to the low effectiveness of the reduction strategies. 

We focused on strategies at the 6 medical facilities and found little correlation in the use 

of specific strategies with the reduction of missed opportunities. For example, about 

61 percent of clinics used human reminder calls while about 39 percent did not. Both 

groups’ results showed no statistically significant difference in terms of missed 

opportunity targets—45 percent of the clinics that used human reminder calls met targets 

while 47 percent of the clinics that did not use human reminder calls met targets. 

Appendix C presents more information on the results on the six strategies reviewed. 

In FY 2008, about 3.1 million patients who did not contact the medical facility to cancel 

their appointment did not show up for their scheduled appointment. Although the 

3.1 million patients represented a downward trend from past years (a reduction of 

3 percent from FY 2007), VHA could not determine which strategies, if any, were 

responsible for the reduction. The number of unused appointments is significant and 

costly to the medical facility (about $564 million annually). Reducing the number of 

missed opportunities by even one third (1.03 million patients) could allow facilities to 

more efficiently use $188 million annually. Our calculation is based on an average cost 

of an outpatient appointment of $182. 

3. VHA Needs to Ensure Unused Appointments are Used for Patients Waiting for 

Care. 

VHA has not established procedures for facility personnel to follow to ensure that when 

patients cancel appointments, other patients currently waiting for appointments are 

contacted first and offered the available appointment slot. VHA Central Office personnel 

told us that when appointments become available from a patient cancellation, schedulers 

should try to fill the available appointment with a patient on the electronic wait list, 

waiting for a consult with a specialist, or that currently has a scheduled appointment more 

than 30 days past the desired date of care. 

Facility personnel said that if an appointment became available, they would generally 

offer that appointment to the next patient requesting an appointment which essentially 

puts these patients ahead of those already waiting for an appointment. Personnel told us 

that occasionally they would contact a patient, if they knew the patient was waiting for an 

appointment and could come in on short notice. However, it was usually easier just to fill 

the appointment with the next patient who called or walked in. Under this ad hoc system, 

facilities were able to refill and use about 2.7 million (60 percent) of the 4.5 million 

appointments that were canceled prior to the appointment. However, the remaining 

1.8 million appointments (40 percent) went unused. This number along with the 

3.1 million no-shows represents the 4.9 million unused appointments in FY 2008. 

VA Office of Inspector General iv 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Of the 1.8 million canceled appointments that facility personnel did not refill, 

830,000 were canceled with at least 3 days notice. These unused appointment slots 

provided facilities with an excellent opportunity to schedule patients who were waiting 

for appointments or had appointments that were more than 30 days past their desired 

appointment dates. During that same timeframe, we projected that about 1.2 million 

patients were waiting to be scheduled for appointments or had appointments that were 

more than 30 days past their desired appointment dates. 

Although providers may be accomplishing other patient care duties during an unused 

appointment, ensuring all appointments are used provides patients with timely health care 

in the most cost-effective manner. The financial impact to VHA by not using all of the 

830,000 canceled appointments, where the facility had at least 3 days notice to try and 

refill the appointment, could be as much as $151 million annually (830,000 patients X 

$182, our calculated average cost of an outpatient appointment). 

VHA Central Office personnel agreed that procedures should be established but told us it 

was not reasonable to expect facility personnel to refill all 830,000 unused appointments. 

We do not have adequate information to determine how successful facilities will be in 

scheduling patients for the unused appointments. However, even a conservative estimate 

that projects one half of the 830,000 appointments could be refilled equates to a better use 

of $76 million annually (415,000 patients X $182) and $380 million over a 5-year period. 

Conclusion 

Missed opportunities and unused appointments impact VHA’s ability to provide veteran 

patients with timely health care. A conservative estimate projects that these issues 

represent substantial annual resources that could be more efficiently utilized. Although 

providers may be accomplishing other patient care duties during an unused appointment, 

ensuring all appointments are used allows VHA to work towards their goal of providing 

patients with timely health care in the most cost-effective manner. Providing 

management with more accurate information on clinic utilization will help ensure that 

resources are devoted to areas where they are most needed. We estimate that VHA could 

better use $76 million annually and $380 million over a 5-year period by refilling one 

half of the unused appointments that occur from known patient cancellations. 

Recommendations 

1. We recommended the Under Secretary for Health establish procedures to ensure 

facilities measure and track all unused outpatient appointments, including those from 

no-shows, patient cancellations, and unscheduled appointment slots. 

2. We recommended the Under Secretary for Health establish a system to measure the 

effectiveness of processes employed to reduce the number of missed opportunities 

and then implement any best practices nationwide.

VA Office of Inspector General v 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

3. We recommended the Under Secretary for Health establish procedures requiring 

facility directors ensure scheduling personnel offer appointments to patients who are 

either on the electronic wait list, waiting for appointments with specialists, or 

currently have appointments more than 30 days past the desired dates of care, when 

appointments become available and the facility has at least 3 days notice. 

Under Secretary for Health’s Comments 

The Under Secretary agreed with our findings, recommendations and monetary benefits. 

The Under Secretary’s implementation plans for the recommendations were acceptable. 

See Appendix F for the full text of the Under Secretary’s comments. 

OIG Response 

In paragraph’s 3, 4, and 5 of the Under Secretary’s response, he acknowledges that our 

sampling methodology was statistically valid but has concerns with our use of national 

projections. Our use of national projections is consistent with generally accepted 

auditing standards and is appropriately disclosed in the scope and methodology section of 

the report as well as in Appendixes A and B. In this case, the Under Secretary’s concern 

about the sample size is not applicable as our sample is large enough to give statistically 

valid, unbiased projections of the population and we identified the margins of error for 

each of the projections used in our report. 

In paragraph 6, the Under Secretary stated that the 1.2 million patients we reported who 

were waiting for care was incorrect. As we stated in the report, the 1.2 million patients is 

an estimate and our methodology was not designed to match up available appointments to 

individual patients waiting for an appointment. We took steps that provided us 

reasonable assurance that patients were not counted twice. 

In paragraph 7, the Under Secretary states that our report implies that VHA should 

attempt to fill every unused appointment slot. Although we recommended that VHA 

should try to fill unused appointments, we did not expect that it would be successful in 

using all appointment slots. In fact, our estimate of monetary benefits is based on only 

filling 50 percent of patient cancellations and does not include those cancellations that 

occurred within 3 days of the scheduled appointment or those appointment slots where 

the veteran did not show up for their scheduled appointment. 

The Under Secretary adds that VHA’s desire is to leave about 15 percent of the 

appointment slots open to account for variations in demand, which is comparable to our 

findings that 18 percent of all appointment slots were unused. That comparison is not 

valid. The 15 percent referred to by the Under Secretary would be considered unused 

appointment slots and are typically left open by facilities to accommodate patient 

VA Office of Inspector General vi 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

requests for same-day care. Our finding was that 18 percent of FY 2008 appointment 

slots went unused because of patient cancellations that were never refilled and because of 

patient no-shows. Our number does not include the 15 percent that may have been 

intentionally left open to accommodate unexpected patients. 

In paragraph’s 8, 9, and 10, the Under Secretary concurs with our estimate of potential 

monetary benefits but notes that achieving the savings is dependent upon many 

uncontrollable variables. The Under Secretary also states that open clinic time is almost 

always leveraged by staff to perform other clinic-related activities, and that we did not 

fully consider the added cost of filling the open slots. The Under Secretary did not 

provide us with a better estimate of benefits. 

We acknowledge that achieving the savings is dependent upon many variables. This is 

why we used a conservative estimate based on filling only 50 percent of canceled 

appointments that occurred at least 3 days prior to the scheduled appointment. We did 

not determine what facility staff did during unexpected open clinic time. Finally, we do 

not see significant additional costs to fill the unused appointments since scheduling and 

medical staffs are already on hand at the VHA medical facilities. 

In paragraphs 11 and 12, the Under Secretary stated that the method we used to review 

no-show strategies was not reliable to establish a cause and effect relationship between 

the actual strategy used and the results. The Under Secretary added that an evidencebased standard for strategies for reducing no-shows does not exist. We agree. To correct 

this shortcoming, we recommended that the Under Secretary establish a system to 

measure the effectiveness of processes employed to reduce the number of missed 

opportunities and then implement any best practices nationwide.

 (original signed by:)

BELINDA J. FINN 

Assistant Inspector General 

for Auditing 

VA Office of Inspector General vii 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Introduction 

Purpose 

The purpose of the audit was to evaluate the effectiveness of VHA’s efforts to minimize 

the number of unused outpatient appointments. The objectives for the audit were to 

determine if VHA had an effective method to accurately track and report unused 

outpatient appointments, whether VA medical facilities implemented effective processes 

for reducing the number of patient no-shows, and whether unused appointments could be 

used for patients who are waiting for care. 

Background 

The demand for VHA’s outpatient services have steadily increased making it more 

challenging for VHA to provide veterans and their dependents with timely health care. 

One way to effectively address this challenge is to ensure every possible effort is made to 

use available appointments. 

In FY 2006, VHA established a performance measure to monitor and track the number of 

missed opportunities for scheduled outpatient appointments. The measure was included 

in the VISN Directors Performance Plans. This measure identified the percentage of 

scheduled appointments that facility personnel did not use because the patients did not 

show up for their appointments. VHA also included those appointments in this measure 

that were canceled by the patient and clinic, but VHA staff did not enter the cancellations 

into the VistA scheduling package until after the appointment times. 

VHA established missed opportunity targets for ten performance measure clinics—

audiology, cardiology, dermatology, eye care, gastroenterology, mental health, 

orthopedics, podiatry, primary care, and urology. VHA Directive 2006-028, “Process For 

Ensuring Timely Access to Outpatient Clinical Care” (May 8, 2006), charges VA medical 

facility directors with responsibility for ensuring that a process is in place for monitoring 

and decreasing missed opportunities. 

The three primary groups of unused appointments are:

• A patient does not show up for a scheduled appointment. (VHA refers to these 

3.1 million “no-shows” as missed opportunities.) 

• A patient cancels before their scheduled appointment, but facility personnel do not 

schedule a different patient into the available appointment. (In FY 2008, we 

projected this resulted in an additional 1.8 million unused appointments.) 

• A medical facility never schedules a patient into an available appointment. 

(In some cases, this is done intentionally to ensure providers have the ability to see 

VA Office of Inspector General 1 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

patients needing care that same day by leaving some appointments unscheduled. 

VHA does not have sufficient records to identify these appointments.) 

Scope and Methodology 

We conducted our work from March to October 2008 and reviewed outpatient scheduling 

activities for the 1st Quarter FY 2008. We reviewed applicable laws, regulations, 

policies, procedures, and guidelines. We also interviewed employees at VA Central 

Office and at six medical facilities in six VISNs. We randomly selected the following six 

medical facilities for our review: 

• VA Boston Healthcare System (HCS), Jamaica Plain Campus (Jamaica Plain, MA)—

VISN 1 

• Northport VA Medical Center (Northport, NY)—VISN 3 

• Lexington VA Medical Center, Cooper Division (Lexington, KY)—VISN 9 

• Jack C. Montgomery VA Medical Center (Muskogee, OK)—VISN 15 

• Sheridan VA Medical Center (Sheridan, WY)—VISN 19 

• VA San Diego Healthcare System (San Diego, CA)—VISN 22 

To determine if VHA had an effective method to accurately measure and report unused 

outpatient appointments, we reviewed VHA’s: 

• Missed opportunities performance measure and interviewed VHA officials to 

understand the methodology used for calculating these missed opportunities. 

• CUSS report from VistA and compared the report to clinic schedules. 

To determine whether facility staff refilled canceled appointments, we reviewed a 

random statistical sample of 600 appointments canceled by patients prior to their 

appointment times from the 10 performance measure clinics for 1st Quarter FY 2008—

100 appointments at each of the six medical facilities. Appendix A describes our sample 

design and results. In addition, we reviewed data in the VistA scheduling package to 

determine if facility staff filled the appointments when they were canceled. 

To determine the average cost of an unused appointment, we obtained the total cost for 

specialty care and the cost for primary care from VHA’s Decision Support System and 

divided that amount by the number of scheduled appointments. (See Appendix D, 

Table 1.) 

VA Office of Inspector General 2 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

To determine if facility staff implemented effective strategies to reduce the number of 

missed opportunities, we interviewed facility staff and analyzed the information to assess 

whether any particular strategy or combination of strategies were successful. 

To determine if patients were waiting to be scheduled for appointments, we reviewed the 

electronic wait list and a random statistical sample of 600 consults for specialty care for 

the 1st Quarter FY 2008—100 consults at each of the six medical facilities—to evaluate 

whether the patients were scheduled in a timely manner. Appendix B details the results 

of this sample. We used data in the VistA scheduling package and the Computerized 

Patients Record System to determine how long the patients waited before facility staff 

took action on the patients’ request for appointments. We also reviewed reports on 

patients waiting more than 30 days past their desired dates of care. Although our 

methodology provided us with a reasonable basis for identifying the overall financial 

impact of unused appointments, it did not allow us to match up available appointments to 

individual patients waiting for an appointment. 

We assessed the reliability of automated data by comparing missed opportunity data to 

no-show and clinic canceled reports in VistA. We also compared the CUSS report to 

clinic schedules for the number of scheduled appointments and available appointments. 

We concluded that the missed opportunity data used to accomplish the audit objectives 

was sufficiently reliable. However, the CUSS report data was not reliable, and its data 

was not used to reach audit conclusions. 

We conducted this performance audit in accordance with generally accepted government 

auditing standards. Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our findings and 

conclusions based on our audit objectives. We believe that the evidence obtained 

provides a reasonable basis for our findings and conclusions based on our audit 

objectives. 

VA Office of Inspector General 3 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Results and Conclusions 

Issue: Reducing Unused Outpatient Appointments Could 

Improve Timeliness of Veterans Patient Care and Enable 

Better Use of Resources 

Findings 

VHA did not have an effective method to accurately measure and report unused 

outpatient appointments. VHA had not implemented effective processes for reducing the 

number of missed opportunities or ensuring that unused appointments were used for 

patients who were waiting for care. 

1. VHA Lacked an Effective Method to Accurately Track and Report Unused 

Appointments. 

VHA’s process to monitor and track the number of unused appointments focused entirely 

on missed opportunities. In addition to patients not showing up for scheduled 

appointments, a significant number of appointments went unused because patients 

canceled prior to their appointment times, and facility personnel did not refill the 

available appointment slots. 

In FY 2008, we estimated that VHA reported about 3.1 million missed opportunities 

(12 percent of the scheduled appointments). However, this is not the total number of 

unused appointments. When clinic personnel canceled appointments before the 

appointment times, the appointments became unscheduled appointments and were not 

counted as missed opportunities. Based on our analysis of appointments, including those 

canceled by patients prior to the appointment times, we determined that in FY 2008, 

approximately 4.9 million (18 percent) of VHA’s available outpatient appointments were 

unused as shown in Table 1. 

Table 1. OIG Calculation of Unused Appointments—FY 2008 

Description 

Number 

(millions)

Missed Opportunities Per VHA’s Measure 3.1 

Canceled Prior To Appointment But Not Refilled 1.8 

Total Unused Appointments 4.9 

Total Completed Appointments per VHA 21.6 

Total Scheduled Appointments 26.5 

Total Unused Appointments/Total Scheduled Appointments 18% 

VA Office of Inspector General 4 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Canceled Prior to Appointment but Not Refilled

We reviewed canceled appointments for VHA’s 10 performance measure clinics and 

projected that in FY 2008 about 4.5 million patients canceled appointments prior to their 

appointment times. VHA has not established procedures for facility personnel to follow 

to ensure that patients currently waiting for appointments were contacted first and offered 

the available slot. Facility personnel said that if an appointment became available, they 

would generally offer that appointment to the next patient requesting an appointment. 

Under this ad hoc system, facilities were able to refill about 2.7 million (60 percent) of 

the appointment slots canceled prior to the scheduled appointment. We agreed with 

facility personnel that appointments canceled by the patient with less than 3 days notice 

of the appointment would be hard to reschedule with a different patient. Our review of 

the remaining 1.8 million appointments (40 percent) found that patients canceled about 

830,000 appointments (46 percent) at least 3 days in advance. During that same 

timeframe, we also projected that about 1.2 million patients were waiting to be scheduled 

for appointments or had appointments that were more than 30 days past their desired 

appointment dates. 

The financial impact to VHA from not using the 830,000 canceled appointments, where 

the facility had at least 3 days notice to try and refill the appointment, could be as much 

as $151 million annually. Our calculation is based on an average cost of an outpatient 

appointment of $182. Although providers may be accomplishing other patient care duties 

during an unused appointment, ensuring all appointments are used allows VHA to work 

towards their goal of providing all patients with timely health care in the most cost 

effective manner. 

VHA Central Office personnel agreed that procedures should be established but told us it 

was not reasonable to expect facility personnel to refill all 830,000 unused appointments. 

Because we do not have adequate information to determine how successful facilities will 

be in scheduling patients for the unused appointments, we believe that, with appropriate 

procedures and concerted efforts, VHA could realistically refill one half of the 

830,000 appointments. This more conservative estimate equates to a better use of 

$76 million annually and $380 million over a 5-year period. Appendix D details the 

average cost of an outpatient appointment and Appendix E details our estimate of 

potential monetary benefits. 

VHA personnel told us that we did not consider that facility personnel may have 

scheduled more than one person in an appointment (overbooking). However, VHA also 

confirmed to us that they had no reliable information to determine the number of 

overbooked appointments. During our site visits we asked facility personnel about their 

use of overbooking and found that although used, overbooking was not used frequently 

because of the risk that patients would have to wait for care. 

VA Office of Inspector General 5 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Available Appointments

We also reviewed the facility CUSS report to determine how well each facility used their 

available appointments. Although this report showed the total number of available 

appointments, we found the CUSS report was often inaccurate. For the 1st Quarter 

FY 2008, the CUSS report showed that the six medical facilities we reviewed had about 

1 million available appointments, of which they only used about 255,000 (26 percent). 

During our interviews with VHA personnel and reviews of the clinic schedules, we found 

that facility staff did not keep schedules current to show the true number of appointments 

available for use. For example, the provider covering Tuesdays in one clinic left, and the 

facility had not replaced that provider. As a result, all of the Tuesday appointment slots 

were reported as unused because the clinic schedule had not been modified to show the 

actual number of providers. 

 

2. VHA Had Not Implemented Effective Processes for Reducing Missed 

Opportunities. 

VHA’s efforts to reduce missed opportunities resulted in some improvements as shown in 

the table below, but VHA continued to have problems with the number of patients who 

did not show up for their scheduled appointments. In FY 2008, we estimated that about 

3.1 million patients did not show up for their scheduled appointments. Although the 

3.1 million patients represented a downward trend from past years (a reduction of 

3 percent from FY 2007), VHA could not determine which strategies, if any, were 

responsible for the reduction. The number of unused appointments is significant and 

costly to the medical facility (about $564 million annually). Reducing the number of 

missed opportunities by even one third could allow facilities to more efficiently use about 

$188 million annually. 

Table 2. Nationwide Trend of VHA Reported Missed Opportunities 

Fiscal 

Year 

Total Missed 

Opportunities per VHA 

Percent 

Change

2005 4,164,456 

2006 3,704,514 -11% 

2007 3,196,707 -14% 

2008 3,100,2231 -3% 

VHA established missed opportunity targets for the ten performance measure clinics. For 

the 1st Quarter FY 2008, 894 (58 percent) of the 1,542 performance clinics nationwide 

 

1

OIG estimation for FY 2008 based on first 11 months. 

VA Office of Inspector General 6 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

met their targets. Of the six medical facilities reviewed, only 24 (44 percent) of the 

55 performance clinics met their targets as shown in Table 3. 

Table 3. Missed Opportunity Percentages—FY 2008 1st Quarter 

Performance 

Measure Clinic 

FY 08 

Target 

Muskogee Lexington San 

Diego Sheridan Boston 

Northport 

Audiology 7 8 5 7 10 6 6 

Cardiology 12 6 14 13 6 8 12 

Dermatology 12 * 9 18 * 16 14 

Eye Care 13 16 8 19 12 18 13 

Gastroenterology 17 26 8 17 * 26 19 

Mental Health 17 15 32 19 21 19 18 

Orthopedics 12 9 9 15 * 14 11 

Podiatry 12 16 24 17 15 17 11 

Primary Care 11 11 11 15 7 13 10 

Urology 13 17 11 15 * 17 16 

 *Care was not provided at this facility by VHA-employed providers. 

Task Force Study 

To identify successful practices for reducing missed appointments, a task force chartered 

by VHA’s National Leadership Board in January 2006 reviewed strategies used by clinics 

with low missed opportunity rates. In its August 30, 2007, report—“Systems Approach 

to Reduce Missed Appointment Opportunities”—this team concluded that no single 

strategy worked as well as using multiple strategies to reduce missed appointments. They 

identified 18 strategies in use. The team recommended that facilities should assess their 

own processes and then design and implement solutions targeted to their unique needs. 

They also noted that a future study on clinics struggling with high missed opportunity 

rates would provide more insight. As of October 2008, VHA had not developed a 

national plan, issued guidance on using the various strategies discussed in the report, or 

conducted the subsequent study. 

Implementation of Strategies to Reduce Missed Opportunities

VHA had not assessed the effectiveness of strategies employed to reduce missed 

opportunities. With the assistance of National Leadership Board personnel, we identified 

six primary strategies used by facilities to reduce missed opportunities. During our site 

visits to six medical facilities, we interviewed key personnel and reviewed performance 

data to determine how many of the strategies were being used and resulted in positive 

outcomes. (Detailed results are in Appendix C.) Staff at the six medical facilities said 

that the missed opportunity strategies used by the clinics had minimal effect in reducing 

VA Office of Inspector General 7 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

missed opportunities. They cited various factors, including weather, lack of 

transportation, fuel costs, and patient preferences as contributing factors to the low 

effectiveness of the reduction strategies. 

We did not find a correlation with the use of specific strategies and a reduction of missed 

opportunities as illustrated by the following examples: 

• The clinics that used human reminder calls showed no statistically significant 

difference in terms of meeting missed opportunity targets compared to those clinics 

that did not use human reminder calls—45 percent of the clinics that used human 

reminder calls met targets while 47 percent of the clinics that did not use human 

reminder calls met targets. 

• About 45 percent of clinics that used automated calls to remind patients of their 

appointments achieved their 1st Quarter FY 2008 targets. However, about 60 percent 

of clinics that did not provide automated calls to patients achieved their 1st Quarter 

FY 2008 targets. 

• About 58 percent of clinics that used a recall system met their 1st Quarter FY 2008 

targets compared to 40 percent of clinics that did not use a recall system. (In a recall 

system, facilities notify patients before their desired appointment date telling them to 

call in to make their appointments.) 

3. VHA Needs to Ensure Unused Appointments are Used for Patients Waiting for 

Care. 

Facility personnel did not have procedures to ensure that patients currently waiting for 

appointments were contacted and offered available appointments. Facility personnel said 

that if an appointment became available, they would generally offer that appointment to 

the next patient requesting an appointment. Personnel told us that occasionally they 

would contact a patient, if they knew the patient was waiting for an appointment and 

could come in on short notice. However, it was usually easier just to fill the appointment 

with the next patient that called or walked in. 

VHA Central Office personnel told us that when an appointment becomes available from 

a patient cancellation, schedulers should try to fill the available appointment with a 

patient on the electronic wait list, waiting for a consult with a specialist, or that currently 

has a scheduled appointment more than 30 days past the desired date of care. Based on 

VHA’s data, we projected that in FY 2008, about 1.2 million patients nationwide were 

either waiting to be scheduled for appointments or had appointments more than 30 days 

past their desired dates of care. 

• On January 1, 2008, about 7,000 patients were on the electronic wait list. 

VA Office of Inspector General 8 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

• In FY 2008, about 651,000 patients waited more than 7 days for facility personnel to 

take action on their requests for specialty care. 

• In FY 2008, about 496,000 patients had outpatient appointments with wait times of 

more than 30 days beyond their desired dates of care. 

Although our methodology did not allow us to match up available appointments for 

individual patients waiting for an appointment, we did identify some examples where 

patients were waiting for care even though there were available appointments. The 

following situations highlight the need to address this issue. 

• A patient waited 50 days for an audiology assessment clinic visit after his provider 

had requested an audiology consult—it took 27 days for VHA to schedule the patient 

for a clinic appointment and 23 additional days until his appointment. Our review of 

the clinic schedule found unused appointments on nearly all of the days he waited for 

the visit. 

• A patient at another facility waited 219 days for an ophthalmology exam after his 

provider requested the consult—it took 186 days for the exam to be scheduled and 

33 more days from scheduling until his appointment date. We reviewed the clinic 

schedule and found four unused appointments within 30 days of the request date. 

Although some facilities had strategies in place to reduce the number of unused 

appointments, facility personnel did not have corresponding procedures to ensure patients 

currently waiting for appointments were contacted and offered the available appointment. 

Conclusion 

VHA had a high number of unused appointments—about 4.9 million (18 percent) of the 

scheduled 26.5 million annual outpatient appointments were unused in FY 2008. This 

number did not include the appointments that were never scheduled with patients because 

VHA did not have sufficient records to identify these appointments. Therefore, VHA 

was unable to track and measure the total number of available outpatient appointments. 

Using $182 as the average cost of an outpatient appointment, we estimated that VHA 

could better use $76 million annually and $380 million over a 5-year period by refilling 

half of the 830,000 unused appointments that resulted from patient cancellations. 

Missed opportunities for patient care impact VHA’s ability to provide patients with 

timely health care. More accurate information on clinic utilization will help management 

ensure that resources are devoted to areas where they are most needed. This involves 

revising the missed opportunity performance measure to include all available 

appointments and/or ensuring the clinic schedules and the CUSS report are kept current 

VA Office of Inspector General 9 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

and accurate. In addition, emphasizing the need and establishing procedures to reduce 

the number of unused appointments will help ensure facilities see patients in a timely 

manner and use resources in the most cost-effective manner. 

Recommendations 

1. We recommended the Under Secretary for Health establish procedures to ensure 

facilities measure and track all unused outpatient appointments, including those from 

no-shows, patient cancellations, and unscheduled appointment slots. 

Under Secretary Response 

VHA will establish policy guidance to ensure facilities strengthen procedures to 

identify and schedule unused outpatient appointments and will conduct a study of the 

CUSS report for potential ways to improve its ability to report and track unused 

outpatient appointments within the current scheduling system. Ultimately, VHA will 

be able to measure and track unused appointments system-wide when the 

Replacement Scheduling Application is fully implemented. VHA has requested that 

deployment of the new system begin in FY 2009. 

OIG Response

The improvement plans are acceptable, and we will follow up on the planned actions 

until they are implemented. 

2. We recommended the Under Secretary for Health establish a system to measure the 

effectiveness of processes employed to reduce the number of missed opportunities 

and then implement any best practices nationwide.

Under Secretary Response

The Under Secretary for Health agreed and stated VHA would ascertain the feasibility 

and value of a study on the effectiveness of processes employed to reduce the number 

of patient no-shows. This would occur by December 31, 2008. 

Additionally, VHA will survey medical facilities to gauge the range and effectiveness 

of the various no-show strategies currently in use. The Systems Redesign Steering 

Committee will review the results of this survey by July 31, 2009, and make 

recommendations concerning whether VHA may be able to implement any best 

practices nationwide. 

VA Office of Inspector General 10 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

OIG Response

The improvement plans are acceptable, and we will follow up on the planned actions 

until they are implemented. 

3. We recommended the Under Secretary for Health establish procedures requiring 

facility directors ensure scheduling personnel offer appointments to patients who are 

either on the electronic wait list, waiting for appointments with specialists, or 

currently have appointments more than 30 days past the desired dates of care, when 

appointments become available and the facility has at least 3 days notice. 

Under Secretary Response

The Under Secretary for Health agreed and stated that VHA will establish policy 

guidance, most likely in the currently pending revised scheduling policy, to make 

certain facilities establish procedures requiring facility directors ensure scheduling 

personnel offer appointments to waiting patients. The VHA System Redesign Office 

will have responsibility for writing the policy. Specifically, this policy guidance will 

establish and strengthen procedures so that appointments cancelled at least 3 days 

before their scheduled times are offered to other patients. In addition, the Scheduling 

Process Group will include training to implement these procedures in the new 

scheduler training package. 

OIG Response

The improvement plans are acceptable, and we will follow up on the planned actions 

until they are implemented. 

VA Office of Inspector General 11 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix A 

Review of Unused Outpatient Appointments 

Universe

The universe for VHA reported missed opportunities came from VHA’s Quality and 

Performance Measures reports. The universe during the period of October 1 to 

December 31, 2007, consisted of 172 VA medical facilities and 44,342 clinics 

nationwide. These clinics had missed opportunities totaling 788,000 in the 1st Quarter 

FY 2008. 

Sample Design

To identify our audit scope, we used a three-stage random statistical sample. In the first 

stage, we randomly identified six sample VA medical facilities. For the second stage of 

the sample, we identified 615 random sample clinics out of a possible 1,811 performance 

measure clinics within those six sample VA medical facilities. Using VistA’s Canceled 

Clinic Report for the six sample facilities, we identified the universe of patient-canceled 

appointments prior to the appointment times within our sample clinics—a universe of 

13,323 cancellations. For the third stage of our sample, random number generation 

software was used to select 100 canceled appointments for each of the six sample VA 

medical facilities. In total, we reviewed 600 canceled appointments. 

Table 1. Universe of Patient-Canceled Appointments—FY 2008 1st Quarter 

Facility Universe Sample 

Boston HCS 3,670 100 

Lexington 1,111 100 

Muskogee 2,134 100 

Northport 1,941 100 

San Diego HCS 3,987 100 

Sheridan 480 100 

Total Universe 13,323 600 

Sample Results

We analyzed 600 random patient cancellations prior to the scheduled appointment times 

to determine if the appointments were refilled after the cancellations. In total, 253 of 

600 canceled appointments were not refilled. Of the 244 patient cancellations that 

occurred less than 3 days prior to the appointment, 152 were not refilled. Of the 

356 cancellations that occurred at least 3 days prior to the appointment, 101 were not 

refilled. 

VA Office of Inspector General 12 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix A 

Based on the factors of our three-stage random sample, we projected nationwide that 

about 460,000 patient-canceled appointments were not refilled as shown in Table 2. 

Table 2. Canceled Appointments Not Refilled—FY 2008 1st Quarter 

Days 

Number of 

Canceled 

Appointments 

Number of 

Canceled 

Appointments 

Not Refilled 

Percent of 

Appointments 

Not Refilled 

Less Than 3 Days 413,737 252,994 61% 

3 or More Days 710,956 207,450 29% 

Total 1,124,693 460,444 41% 

These projections were computed from a sample of appointment records. This was a 

complex, multi-stage sample with unequal weights. We used a replication-based method 

to compute the sampling errors for our estimates, which takes the complexity of the 

sample design into account. The margins of error in this report give the upper and lower 

bounds of a 90 percent confidence interval for each projection as shown below. This 

means that 90 percent of the possible samples we could have selected of the same size 

and design would have resulted in an estimate within these bounds. 

Table 3. Projection of Patient Cancellations Prior to the Appointment Times— 

FY 2008 1st Quarter 

Value Projection 

Margin of 

Error 

Lower 

90% 

Upper 

90% Sample 

Total Canceled Prior To 

The Appointment 1,124,693 577,733 546,960 1,702,426 600 

Canceled and Refilled 664,249 378,459 285,790 1,042,708 347 

Canceled and Not Refilled 460,444 215,276 245,168 675,720 253 

Canceled Less Than 3 

Days And Not Refilled 252,994 113,291 139,703 366,284 152 

Canceled With At Least 3 

Days Notice And Not 

Refilled 207,450 138,837 68,613 346,287 101 

VA Office of Inspector General 13 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix A 

In order to project through FY 2008, we annualized the number of unused outpatient 

appointments for each category as shown below. 

Table 4. Projection of Patient Cancellations Prior to Appointment Times—FY 2008 

Value Projection 

Margin of 

Error 

Lower 

90% 

Upper 

90% Sample 

Total Canceled Prior To 

The Appointment 4,498,772 2,310,932 2,187,840 6,809,704 600 

Canceled and Refilled 2,656,996 1,513,836 1,143,160 4,170,832 347 

Canceled and Not Refilled 1,841,776 861,104 980,672 2,702,880 253 

Canceled Less Than 3 

Days And Not Refilled 1,011,976 453,164 558,812 1,465,140 152 

Canceled With At Least 3 

Days Notice And Not 

Refilled 829,800 555,348 274,452 1,385,148 101 

VA Office of Inspector General 14 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix B 

Review of Consults and Patients Waiting for Care 

Universe 

The universe of consults was obtained from the VistA Consult Tracking Report for the 

period of October 1 to December 31, 2007. The universe consisted of 28,611 consults 

from the six sample facilities’ performance measure clinics. 

Sample Design

The sample design used to project consults not acted on within 7 days consisted of two 

stages. In the first stage, we randomly identified six sample VA medical facilities. For 

the second stage of the sample, we identified a universe of 28,611 consults for the 

1st Quarter FY 2008 within the six sample facilities using VistA’s Consult Tracking 

Report. Random number generation software was used to select 100 consults from each 

of the six sample medical facilities. In total, we reviewed 600 consults to determine if 

facility personnel took action on the consults within 7 days. 

 

Table 1. Universe of Consults—FY 2008 1st Quarter 

Facility Universe Sample 

Boston HCS 3,232 100 

Lexington 6,747 100 

Muskogee 7,164 100 

Northport 2,237 100 

San Diego HCS 7,292 100 

Sheridan 1,939 100 

Total Universe 28,611 600 

Sample Results

In total, 111 of 600 consults were not acted on in a timely manner. Based on the factors 

of our two-stage random sample, we projected nationwide that approximately 

162,000 consults were not acted on within 7 days. 

Table 2. Consult Sample Projected Nationwide—FY 2008 1st Quarter 

Number 

Margin of 

Error 

Lower 

90% 

Upper 

90% 

Not Acted on Within 7 Days 162,671 122,268 40,403 284,939 

VA Office of Inspector General 15 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix B 

These projections were computed from a sample of the appointment records. This was a 

complex, multi-stage sample with unequal weights. We used a replication-based method 

to compute the sampling errors for our estimates, which takes the complexity of the 

sample design into account. The margins of error in this report give the upper and lower 

bounds of a 90 percent confidence interval for each projection. This means that 

90 percent of the possible samples we could have selected of the same size and design 

would have resulted in an estimate within these bounds.

Patients Waiting for Care

In addition to the patients waiting for consults, there were about 7,000 patients on the 

electronic wait list as of January 1, 2008, and 123,939 patients seen in the 1st Quarter 

FY 2008 with wait times of more than 30 days. We projected that in FY 2008 about 

1.2 million patients nationwide were either waiting to be scheduled for an appointment or 

had appointments more than 30 days past their desired dates of care as shown in Table 3. 

Table 3. Patients Waiting for Care Nationwide-- FY 2008 

1st Quarter FY 2008 

Not Acted on Within 7 Days 162,671 650,684 

On January 1, 2008 EWL 6,823 6,823 

Wait Times > 30 days 123,939 495,756 

Total Patients Waiting for Care 293,433 1,153,263 

VA Office of Inspector General 16 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix C 

Review of Missed Opportunity Strategies 

During our audit we focused on six strategies shown in the table below. We determined 

how many strategies were used and whether they were successful based on our 

615 sample clinics. 

Table 1. Review of Missed Opportunity Strategies 

Strategy Employed 

Percent of 

Clinics Met 

FY 2008 

Targets 

Margin 

of 

Error 

Number 

of Clinics 

in Sample

Scheduling Procedure

Recall System 58% 6% 196 

Traditional 40% 4% 419 

Automated Reminder Letter

Letter Sent 44% 4% 475 

Letter Not Sent 53% 7% 140 

Automated Reminder Call

Call Made 45% 4% 585 

Call Not Made 60% 15% 30 

Human Reminder Call

Call Made *45% 4% 378 

Call Not Made *47% 6% 237 

Designated Phone Line For 

Cancellations 

Designated Phone Line 41% 5% 314 

No Designated Phone Line 50% 5% 301 

Allocated Appointments for 

"Drop Ins"

Allocated Appointments 56% 6% 181 

No Allocated Appointments 41% 4% 434 

* Not a statistically significant difference 

VA Office of Inspector General 17 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix C 

• Recall System – patient due for follow-up appointment is notified to call and schedule 

appointment. 

• Traditional Scheduling – patient is scheduled for future appointment at time of current 

visit. 

• Automated Reminder Letter – patient receives reminder letter through mail. 

• Automated Reminder Call – patient receives recorded message prior to appointment. 

• Human Reminder Call – patient receives human reminder call prior to appointment. 

• Allocated Appointments for "Drop Ins" – open appointments available for patients to 

drop by or walk in as needed. 

The following correlations existed between strategies in use and clinics meeting missed 

opportunity targets. 

• Clinics using Recall System and/or allocating appointments for “drop ins” met targets 

more often. 

• Clinics not using automated reminder calls, automated reminder letters, or 

cancellation lines met targets more often than those that did use the strategies. 

• Clinics using human reminder calls showed no statistically significant difference in 

regards to meeting targets. 

These projections were computed based on analysis and interviews at six sample medical 

facilities. This was a complex sample with unequal weights. We used a replicationbased method to compute the sampling errors for our estimates, which takes the 

complexity of the sample design into account. The margins of error in this report give 

the upper and lower bounds of a 90 percent confidence interval for each projection. This 

means that 90 percent of the possible samples we could have selected of the same size 

and design would have resulted in an estimate within these bounds. 

VA Office of Inspector General 18 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix D 

Cost of Unused Appointments 

To determine the average cost of an unused appointment, we obtained the total cost for 

specialty and primary care from VHA’s Decision Support System and divided that 

amount by the number of scheduled appointments as shown below. 

Table 1. Factors for Determining Cost of Unused Appointment—FY 2008 

Primary Care 

Specialty 

Clinics 

Total Clinic 

Costs Per DSS 

Number of 

Scheduled Appts 

Average Cost of 

Scheduled Appts

$664,767,343 $534,505,255 $1,199,272,598 6,586,835 $182 

Appendix A, Table 4 shows that for FY 2008, the projected number of appointments 

canceled by the patient 3 or more days prior to the appointment time and not refilled with 

another patient was 830,000. Using $182 as the average cost of an outpatient 

appointment, the financial impact to VHA from not using these 830,000 appointments 

could be as much as $151 million annually. VHA Central Office personnel agreed that 

procedures should be established but told us it was not reasonable to expect facility 

personnel to refill all 830,000 unused appointments. Because we do not have adequate 

information to determine how successful facilities will be in refilling unused 

appointments, we believe that refilling one half is more realistic. This will allow VHA to 

better use $76 million annually and $380 million over a 5-year period. 

Table 2. Potential Monetary Benefits* 

FY 2008 

(1st Qtr Projection X 4) 

5-year 

Canceled (FY 2008 X 5) 

With At 

Least 3 Days 

Notice and 

not Refilled 

1st Qtr FY08 

(Projected 

Based on 

Sample) 

Unused 

Appts** 

Unused 

 Apt** 

Cost (Monetary 

Benefits)*** 

Unused 

Appts** 

Cost (Monetary 

Benefits)*** 

All 207,450 829,800 $151,023,600 4,149,000 $755,118,000

One Half 103,725 414,900 $75,511,800 2,074,500 $377,559,000

 * Table reflects the actual amounts. Amounts shown in the report may be different to 

account for rounding and consistency. 

 ** Unused appointments from appointments canceled by patient at least 3 days prior to 

the appointment time. 

*** Based on average outpatient appointment cost of $182. 

VA Office of Inspector General 19 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix E 

Monetary Benefits in Accordance with 

IG Act Amendments 

Recommendation

Explanation of 

Benefit(s)

Better Use of 

Funds

3. Estimated $76 million 

annually could have 

been used more 

efficiently to refill 

available appointments 

that were canceled prior 

to the appointment 

time. Over a 5-year 

period this amounts to 

$380 million. 

$380,000,000 

 Total $380,000,000 

VA Office of Inspector General 20 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix F 

Department of 

Veterans Affairs Memorandum

Date: November 12, 2008 

From: Under Secretary for Health (10) 

Subject: Audit of Veterans Health Administration's Efforts to 

Reduce Unused Outpatient Appointments 

To: Assistant Inspector General for Auditing (52) 

1. I have reviewed the draft report, and I concur with the recommendations and 

estimate of monetary benefit. While I believe there are important limitations to your 

methodology, which raise questions about the reasonableness of some of your 

conclusions, you have identified potential areas that need improvement. As you 

recommend, I agree that the ability to systematically track unused outpatient 

appointments, detect and fill open appointment slots, and measure the effectiveness 

of local no-show interventions will enhance VHA's capability to improve access for 

the health care needs of our Nation’s veterans. 

2. I am encouraged that VHA has made extensive efforts in improving waiting 

times. A recent report by an outside consulting firm on VA's patient scheduling and 

waiting times measurement concluded that "despite the difficulty of measuring 

actual wait times, VA has made a significant effort to confront these challenges, and 

has used patient wait time data to manage the timeliness of outpatient care and 

report on system-wide performance."2

 VHA has also intensively targeted and 

monitored no-shows and clinic cancellations through its missed opportunity and 

new patient wait time performance measures. As a result, VHA is currently refilling 

60 percent of all cancelled appointments. 

3. As previously noted, while I concur with the report's recommendations, I am 

concerned that some of the findings and conclusions in your report could be 

misconstrued. For instance, in the initial paragraphs of the report's executive 

summary, you immediately highlight the national projections of the number of 

unused appointments and number of patients waiting for care in VHA. However, 

you do not introduce the important qualifier that these projections are derived from 

 

2

Booz Allen Hamilton, Final Report on the Patient Scheduling and Waiting Times Measurement 

Improvement Study, July 11, 2008. 

VA Office of Inspector General 21 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix F 

a small sample taken from only six medical facilities until your discussion of scope 

and methodology well into the body of the report. 

4. Furthermore, while the complex multi-stage sample with unequal weights 

methodology that you utilized to make these national projections may be 

statistically valid, the report does not inform the reader just how small the sample 

sizes that you based your conclusions on actually are. 

For instance, to determine whether VHA refilled cancelled appointments, you 

reviewed a random statistical sample of 600 appointments canceled by patients prior 

to their appointment times from the 10 performance measure clinics at six medical 

facilities during first quarter FY 2008. You also state that in FY 2008, VHA had 

approximately 4.5 million cancelled appointments prior to appointment times. As 

such, you only reviewed 0.0133 percent (600 out of 4.5 million) of canceled 

appointments in FY 2008 to make national projections about whether VHA refilled 

cancelled appointments. 

5. In another example, to determine whether patients were scheduled in a timely 

manner, you reviewed a random statistical sample of 600 consults for specialty care 

for the first quarter FY08. However, in FY 2008, VHA had a total of 

3,640,807 Computerized Patient Record System consult requests. As such, you 

only reviewed 0.0165 percent (600 out of 3,640,807) of all VHA consults in 

FY 2008 to make national projections about whether VHA scheduled patients in a 

timely manner. As VHA staff has previously discussed, while these national 

projections may be statistically representative, one must be cautious in using such 

projections in estimating that an agency could have used millions of dollars more 

efficiently. 

6. I am also concerned about your methodology in deriving the total number of 

patients waiting for care in VA. In order to determine if patients were waiting to be 

scheduled for appointments, you reviewed the VHA data and found that there were 

about 7,000 patients on the electronic wait list as of January 1, 2008, 

496,000 patients seen in FY 2008 with wait times of more than 30 days, and 

651,000 patients with consults not acted on within seven days. From these three 

figures, you projected that approximately 1.2 million patients were waiting for care 

in FY 2008. However, as VHA staff has previously discussed, labeling this 

grouping as "total patients waiting for care nationwide in FY 2008" is incorrect 

since: 1) patients with consults not acted on within seven days may have already 

received their care in FY 2008; 2) your count of those consults not acted on within 

seven days could also be on the electronic wait list and thus be double counted; and 

3) your count of those consults not acted on within seven days could also have been 

double counted as patients seen in FY 2008 with a wait time of more than 30 days. 

VA Office of Inspector General 22 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix F 

7. As written, your report implies that VHA should optimally attempt to fill every 

unused appointment slot. However, non-healthcare industry studies and systems 

redesign principles state that filling any system to 100 percent capacity will cause 

waiting times to increase exponentially. I realize there is a trade off between patient 

waiting times and appointment slot use. General guidance is that systems filled 

beyond about 85 percent of capacity cannot respond to variation in demand and 

result in causing patient delay. 

Your report projects that about 4.9 million (18 percent) of the scheduled 

26.5 million annual outpatient appointments in VHA’s 10 performance measure 

clinics were unused in FY08. While I acknowledge there are improvements that can 

be made, the actual system performance that you calculated (18 percent open slots) 

and system redesign principles (15 percent open slots) is not very different. Lastly, 

your report should acknowledge that it is not reasonably possible for every unused 

appointment slot length to match the time needed for patients waiting. For example, 

a 15 minute slot may be open, but waiting patients may need 30 or more minutes for 

their care. 

8. While I concur with your estimate of monetary benefit, I am concerned that the 

accuracy of your estimated better use of funds amounts is highly dependent upon 

many potentially uncontrollable variables. Specifically, actual realization of the 

better use of funds amount is dependent upon timely and effective communication 

from the clinic to the veteran about the availability of a cancelled appointment, the 

availability and willingness of the veteran to change her/his schedule to accept the 

cancelled appointment, and the success of the veteran actually completing the 

rescheduled appointment. If all of the above happened, then there would be a more 

effective use of the funds, but if those things did not occur, it is not correct to 

assume that there was an ineffective use of the funds. 

9. Additionally, your estimate of monetary benefit calculation assumes that clinic 

staff is idle during the visit time of a “no-show”. However, this has not been VHA’s 

experience. Any open clinic time is almost always leveraged by staff to perform 

other important clinic-related activities. For the provider, this could entail: 

reviewing view alerts; calling patients to discuss next steps in care; preparing for the 

next patient by reviewing clinical notes, etc.; completing documentation on the 

previous patient; completing encounters; spending extra time with patients who are 

being seen; and, teaching activities. For non-providers, similar use of time occurs. 

Lastly, in calculating your estimate of monetary benefit, you do not consider the 

added cost of filling additional numbers of open slots and thus have not properly 

reduced projected costs savings accordingly. 

10. To project the true costs of increasing the percentage of available slots to be 

filled, one would need to review tools, time, and contact information available to 

VA Office of Inspector General 23 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix F 

schedulers to enable their increased success in filling open slots. In the end, while 

VHA concurs with the intent to offer open clinic slots to waiting patients, this is not 

so easy to do, given the limitations of the current scheduling system. With the 

current scheduling software, if a patient is being scheduled and desires an 

appointment and the desired date is today (as for new patients), schedulers would 

see the next open appointment (cancelled or not). For a scheduler to offer open 

clinic slots to waiting patients, schedulers would have to be able to sequester 

cancelled appointments from the system for rescheduling of patients who were 

waiting for appointments. This is not possible in the current system. In the current 

system, in order to offer appointments to waiting patients when appointments 

become available and the facility has at least a one day notice, schedulers are 

required to check the electronic wait list, Recall/Reminder System, New Patient Call 

List, AEG Mumps Routine, pending consults, and Veterans Integrated Service 

Network Support Service Center Pending Reports. In addition, the scheduler must 

check patient registration to obtain current contact information for such patients, 

successfully make the contact, offer the appointment, assist as needed with travel, 

and coordinate lab and diagnostic work-ups. This is a very difficult expectation to 

add to the already overburdened schedulers, and effective implementation of this 

process with the current tools is unrealistic. I believe that the new scheduling 

system, Replacement Scheduling Application (RSA), which is currently in the 

development and testing phase, will allow more flexibility in scheduling when it is 

implemented throughout the VA system. This is because RSA will have the 

capability to provide a VA-wide view of resource availability, which should 

facilitate real-time centralized scheduling and improve the timeliness of care. 

11. As to your objective to determine if facility staff implemented effective 

strategies to reduce the number of missed opportunities, your review of six facilities 

identified six primary strategies used to reduce missed opportunities. In the end, 

you concluded that little correlation existed in the use of specific strategies to reduce 

missed opportunities, and VHA had not assessed the effectiveness of strategies 

employed to reduce missed opportunities. However, your report fails to provide the 

reader with a well-informed understanding because it does not: 1) fully define the 

six strategies you identified; 2) describe the extent that each strategy was 

implemented at each clinic to gauge whether they were effectively implemented; 

and 3) explain how and what standards you used to determine whether or not the 

strategy was effectively implemented. As such, your methodology of comparing 

your perceptions of no-show strategy implementation that staff conveyed to you 

during interviews and whether clinics met previously implemented no-show targets 

is not a reliable way to establish a cause and effect relationship between actual 

strategy use and results. I believe that in order to make informed conclusions in this 

area, we need to engage in a more focused short-term performance improvement 

initiative and potentially a formal long-term research study. 

VA Office of Inspector General 24 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix F 

12. While I ultimately agree that VHA needs to become more knowledgeable about 

the connection between no-show strategy implementation and effectiveness, it is 

important to point out that there are no evidence-based standards for strategies that 

reduce no-shows. Nonetheless, VHA has championed and distributed to the extent 

possible a number of successful strategies (see attached). 

13. Thank you for the opportunity to review the report and your willingness to work 

with us to refine its contents. Attached is VHA’s complete plan of corrective action. 

I would be glad to discuss any of concerns or comments you may have about this 

response or the action plan. If you have any questions, please contact Margaret 

Seleski, Director, Management Review Service (10B5) at (202) 461-8470. 

VHA No-show Strategies 

1) “Closing the visit” – The provider and patient discuss the next step in care 

including reasons and timing of the next visit and agree upon the plan. 

2) Reminder calls or letters – This strategy scripts the scheduling function to use 

the best words to be clear about the visit appears. Reports suggest this strategy 

to be extremely effective in some clinics, but not at all effective in others. 

3) Support staff – The VHA Office of Provider Productivity found a significant 

association of no-show rates with support staff sufficient to make reminder calls. 

4) Minimize transportation issues – Transportation failures or challenges can lead 

to cancellations and no-shows. 

 

5) Make cancellations easy – Patients are less likely to no-show if the facility 

makes cancellation easy. VHA has established an expectation that facilities 

make cancellation options appear high on the phone tree. 

6) Reduce the wait time – This strategy has been broadly and successfully 

implemented by VHA. 

7) Improve continuity – This strategy emphasizes that the patient sees their own 

provider. VA has taken the lead in this area for many years by establishing 

panels in primary care. 

8) Involve the patient in making the appointment – Involve the patient rather than 

sending the patient an appointment time and date. VA has prohibited autorebooking because it does not involve the patient, resulting in increased noshow, cancel, and reschedule rates. 

VA Office of Inspector General 25 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix F 

Under Secretary for Health Comments 

to Office of Inspector General’s Report 

The following comments are submitted in response to the 

recommendations in the OIG’s Report: 

OIG Recommendations

Recommendation 1. We recommend the Under Secretary for Health 

establish procedures to ensure facilities measure and track all unused 

outpatient appointments, including those from no-shows, patient 

cancellations, and unscheduled appointment slots. 

Concur Target Completion Date: 07/31/2009

VHA will establish policy guidance, most likely in the currently 

pending revised scheduling policy, to ensure facilities strengthen 

procedures to identify and schedule unused outpatient appointments 

as noted in the action plan to recommendation number 3 below. 

VHA System Redesign Office will have responsibility for writing 

the policy. In the near-term, VHA System Redesign Office will 

request that the VSSC conduct a study of the CUSS report for 

potential ways to improve its ability to report and track unused 

outpatient appointments within the current scheduling system. VHA 

System Redesign Office will submit this request to VSSC and 

request a response by July 31, 2009. Ultimately, VHA will be able 

to measure and track unused appointments system-wide when the 

new scheduling system, Replacement Scheduling Application, is 

fully implemented. The timeline for implementation of Replacement 

Scheduling Application depends on VA Office of Information and 

Technology's software development process but VHA has requested 

that deployment begin in FY 2009. 

Recommendation 2. We recommend the Under Secretary for Health 

establish a system to measure the effectiveness of processes 

employed to reduce the number of missed opportunities and then 

implement any best practices nationwide. 

Concur Target Completion Date: 07/31/2009

VA Office of Inspector General 26 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix F 

Deputy Under Secretary for Health for Operations and Management 

and VHA System Redesign Office will coordinate with VHA Office 

of Research and Development to ascertain the feasibility and value 

of an Office of Research and Development study on the 

effectiveness of processes employed to reduce the number of patient 

no shows. VHA System Redesign Office will coordinate this 

request by December 31, 2008. 

Additionally, the Deputy Under Secretary for Health for Operations 

and Management will survey VHA medical facilities to gauge the 

range and effectiveness of the various no-show strategies currently 

in use. The Systems Redesign Steering Committee will review the 

results of this survey by July 31, 2009 and make recommendations 

concerning whether VHA may be able to implement any best 

practices nationwide. 

Recommendation 3. We recommend the Under Secretary for Health 

establish procedures requiring facility directors ensure scheduling 

personnel offer appointments to patients who are either on the 

electronic wait list, waiting for appointments with specialists, or 

currently have appointments more than 30 days past the desired 

dates of care, when appointments become available and the facility 

has at least 3 days notice. 

Concur Target Completion Date: 07/31/2009

VHA will establish policy guidance, most likely in the currently 

pending revised scheduling policy, to ensure facilities establish 

procedures requiring facility directors ensure scheduling personnel 

offer appointments to waiting patients. The VHA System Redesign 

Office will have responsibility for writing the policy. Specifically, 

this policy guidance will establish and strengthen procedures to 

ensure that appointments cancelled within at least three days of its 

scheduled time are offered to other patients. In addition, the VHA 

System Redesign Office – sponsored Scheduling Process Group will 

include training to implement these procedures in the new scheduler 

training package. 

VA Office of Inspector General 27 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix G 

OIG Contact and Staff Acknowledgments 

OIG Contact Larry Reinkemeyer 

816-997-6940 

Acknowledgments Madeline Cantu 

Robin Frazier 

Lee Giesbrecht 

Joe Janasz 

Lance Kramer 

Russ Lewis 

Mary Lopez 

Gilbert Melendez 

Henry Mendala 

Daniel Morris 

Carla Reid 

Jason Schuenemann 

Lynn Scheffner 

Oscar Williams 

VA Office of Inspector General 28 
Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments 

Appendix H 

Report Distribution 

VA Distribution

Office of the Secretary 

Veterans Health Administration 

Veterans Benefits Administration 

National Cemetery Administration 

Assistant Secretaries 

Office of General Counsel 

Non-VA Distribution

House Committee on Veterans’ Affairs 

House Committee on Appropriations, Subcommittee on Military Construction, Veterans 

Affairs, and Related Agencies 

House Committee on Oversight and Government Reform 

Senate Committee on Veterans’ Affairs 

Senate Committee on Appropriations, Subcommittee on Military Construction, Veterans 

Affairs, and Related Agencies 

Senate Committee on Homeland Security and Governmental Affairs 

National Veterans Service Organizations 

Government Accountability Office 

Office of Management and Budget 

This report will be available in the near future on the OIG’s Web site at 

http://www.va.gov/oig/publications/reports-list.asp. This report will remain on the OIG 

Web site for at least 2 fiscal years after it is issued. 

VA Office of Inspector General 29