VHA’s health care system is geographically divided into 21 VISNs, each of which is headed by a VISN director. Each VISN is comprised of a network of VAMCs, and the VISN office serves as the basic budgetary and decision-making unit for providing health care services to veterans within that geographical area. Each VAMC and its affiliated CBOCs and ambulatory care centers are headed by a VAMC director who manages administrative functions, and a chief of staff who manages clinical functions for these facilities. VHA’s Central Office establishes system- wide scheduling policy. VHA’s scheduling policy establishes processes and procedures for scheduling medical appointments, and for ensuring the competency of staff directly or indirectly involved in the scheduling process. This policy is designed to help VAMCs meet VHA’s commitment to scheduling medical appointments with no undue waits or delays. Specifically, VHA’s scheduling policy includes, but is not limited to, the following requirements: Requires VAMCs to use VHA’s Veterans Health Information Systems and Technology Architecture (VistA) medical appointment scheduling system to schedule medical appointments. Requires VAMCs to keep appointment schedules open and available for patients to make medical appointments at least 3 to 4 months into the future. Requires schedulers to record in the VistA scheduling system the date on which the patient or provider wants the patient to be seen as the desired date. To determine the desired date, schedulers should be in communication with the patient when scheduling the medical appointment. Requires schedulers to record the desired date correctly and describes how to determine and record the desired date for new patients—patients who haven’t been seen by a health care provider in a clinic within the past 2 years, including those scheduled in response to a consult request—as well as specifying how to determine the desired date for established patients’ follow-up medical appointments—patients who have been seen within the past two years. Requires VAMCs to track new patients waiting for medical appointments using the electronic wait list within VistA and to remind established patients of follow-up medical appointments using the recall/reminder software within VistA, which enables clinics to create a list of established patients who need follow-up medical appointments more than 3 or 4 months in the future. Additionally, VHA has a separate directive that establishes policy on the provision of telephone service related to clinical care, including facilitating telephone access for medical appointment management. Officials at the VHA central office, VISN, and VAMC all have oversight responsibilities for the implementation of VHA’s scheduling policy. In the VHA central office, the Director of Systems Redesign, through the Office of the Deputy Undersecretary for Health for Operations and Management, is responsible for the oversight and implementation of medical appointment scheduling requirements. This oversight includes measurement and monitoring of ongoing performance. Each VISN director, or designee, is responsible for oversight of enrollment and medical appointment scheduling for eligible veterans. director, or designee, is responsible for ensuring that clinics’ scheduling of medical appointments complies with VHA’s scheduling policy, including clinics in affiliated CBOCs and ambulatory care centers. In addition, the VAMC director is responsible for ensuring that any staff who can schedule medical appointments in the VistA scheduling system has completed VHA scheduler training. To obtain VHA healthcare services, veterans generally must enroll with VHA and register at a specific VAMC. aspects of VHA’s scheduling policy as well as overall compliance, partial compliance, or noncompliance with VHA’s scheduling policy as a whole. According to officials, VHA’s central office does not penalize noncompliance with the certification and expects oversight to be managed locally. VHA’s central office uses this certification of compliance as a tool for VAMCs to identify and improve performance on important aspects of the policy. VistA is the single integrated health information system used throughout VHA in all of its health care settings. There are many different VistA applications for clinical, administrative, and financial functions, including VHA’s electronic medical record, known as the Computerized Patient Record System, and the scheduling system. As we reported in May 2010, the VistA scheduling system is more than 25 years old and inefficient in facilitating care coordination between different sites. In 2000, VHA began an initiative to modernize the scheduling system, but VA terminated the project in 2009. We also reported that VA’s efforts to successfully replace the scheduling system were hindered by weaknesses in its project management processes and lack of effective oversight. In 1995, VHA established a goal of scheduling primary and specialty care medical appointments within 30 days to ensure veterans’ timely access to care.for both primary and specialty care medical appointments based on improved performance reported in previous years. Specifically, VA’s reported wait times for fiscal year 2010 showed that nearly all primary care and specialty care medical appointments were scheduled within 30 days of desired date. In fiscal year 2012, VHA added a goal of completing primary care medical appointments within 7 days of the desired date. In fiscal year 2011, VHA shortened the wait time goal to 14 days To facilitate accountability for achieving its wait time goals, VHA includes wait time measures—referred to as performance measures—in its VISN and VAMC directors’ performance contracts known as Network Director Performance Plans (NDPP) and Facility Director Performance Plans (FDPP), respectively. Wait time performance measures also are included in VA’s budget submissions and performance reports to Congress and stakeholders; the performance reports are published annually in VA’s Performance and Accountability Report (PAR). However, the medical appointment wait time performance measures included in the NDPPs and FDPPs differ from the measures that are reported in the PAR. (See table 1.) For example, in fiscal year 2012, VHA’s wait time goal of 7 days for primary care medical appointments was reflected in the NDPP and FDPP performance measures, but the fiscal year 2012 PAR reported primary care wait time performance using a 14-day standard. The performance measures have also changed over time. At the time of our review, all of VHA’s medical appointment wait time performance measures reflected the number of days elapsed from the patient’s or provider’s desired date, which is recorded in the VistA scheduling system by VAMCs’ schedulers. According to VHA central office officials, VHA measures wait times based on desired date in order to capture the patient’s experience waiting and to reflect the patient’s or provider’s wishes; which is not reflected by other available wait time measures. Medical appointment wait times used for measuring and assessing performance toward VHA’s wait time goals are unreliable due to problems with recording the appointment desired date in the VistA scheduling system. Acknowledging limitations of the wait time measures, VHA uses additional information to monitor patients’ access to medical appointments. VHA measures its medical appointment wait times as the number of days that have elapsed from the patient’s or provider’s desired date. Consequently, the reliability of reported wait time performance is dependent on the consistency with which schedulers record the desired date in the VistA scheduling system. However, aspects of VHA’s scheduling policy and training documents regarding how to determine and record the desired date are unclear and do not ensure replicable and reliable use of the desired date. In addition, we found that some schedulers at select VAMCs did not correctly implement other aspects of VHA’s scheduling policy for recording the desired date. Aspects of VHA’s scheduling policy and related training documents on how to determine and record the desired date are unclear and do not ensure replicable and reliable recording of the desired date by the large number of staff across VHA who can schedule medical appointments in the VistA scheduling system. Specifically, VHA’s scheduling policy and related scheduler training documents do not provide consistent guidance about when or whether the desired date should be based on the patient’s or provider’s preference. While the policy defines desired date as “the date on which the patient or provider wants the patient to be seen,” it also instructs that the “the desired date needs to be defined by the patient” for new patient medical appointments, medical appointments scheduled in response to consult requests, and established patient follow-up medical appointments. When there is a conflict between the provider and patient desired date, the scheduler is instructed to contact the provider for a decision on the return time frame, but the policy and training documents do not clearly describe under what circumstances the provider’s date should be used as the desired date. Further, providers may designate a desired appointment time frame for a follow-up medical appointment rather than a specific date; in such cases, the policy is unclear as to which date within the provider’s designated time frame the scheduler should enter as the desired date. The scheduling policy and training do not provide sufficient guidance to ensure consistent use of desired date in these various scheduling scenarios. VHA central office officials responsible for developing VHA’s scheduling policy and related training documents told us that the desired date is intentionally broad to account for all of the scheduling scenarios that may exist. However, leadership officials from the four VAMCs we visited and their corresponding VISNs reported problems with the unclear guidance on the desired date definition, and difficulties achieving consistent and correct use of the desired date by their schedulers. In addition, given the ambiguity in the scheduling policy and related training documents, there are different interpretations of the desired date between officials at different levels. For example, a VISN director stated that if a provider gives a desired time frame, the scheduler is to use the earliest date in that range as the desired date; whereas a provider in a specialty care clinic at the VAMC we visited within that VISN stated that the clinic uses the latest date in the range to meet the 14-day specialty care medical appointment scheduling goal. Additionally, when presented with various scheduling scenarios, schedulers at the VAMCs we visited determined and recorded the desired date differently. For example, when posed with the question “What date do you enter into the scheduling system as the desired date for an established patient follow-up medical appointment?”, 12 schedulers said they would enter the patient’s desired date, 4 said the provider’s date, and the remaining 3 said they used the next available medical appointment date. When posed with the question “If the patient’s stated desired date conflicts with the provider’s designated desired date or time frame, what date do you enter as the desired date?”, 1 scheduler said that the patient’s desired date would be entered, while another said the desired date has to come from the provider. The variation in schedulers’ interpretation of the desired date suggests confusion about its correct use in different scheduling scenarios. Although unclear about when to use the patient’s or provider’s desired date, VHA’s scheduling policy clearly instructs that, in all circumstances, the desired date should be defined without regard to schedule capacity, and should not be altered once established to reflect a medical appointment date the patient accepts because of lack of medical appointment availability on the desired date. However, we found that at least one scheduler from each of the VAMCs we visited did not correctly implement these aspects of the policy when recording the desired date in the VistA scheduling system for specific hypothetical scheduling situations. As summarized in table 2, we identified the following three types of errors, each of which would have resulted in desired dates that did not accurately reflect the patients’ or providers’ desired date, as well as potentially result in the reporting of more favorable wait times for those medical appointments. Determined appointment availability prior to establishing desired date: Although VHA’s scheduling policy requires schedulers to establish the desired date for a medical appointment without regard to the schedule capacity, four schedulers from three VAMCs determined the clinic’s next available medical appointment dates before establishing a desired date. Therefore, reported wait times for these appointments may not have accurately reflected how long patients actually waited. Altered original desired date based on appointment availability: Three schedulers from two VAMCs established a desired date that was recorded in the VistA scheduling system independent of schedule capacity, but later altered the desired date because of appointment availability. Specifically, two of the three schedulers altered the originally established desired date to match the agreed-upon appointment date, which would have incorrectly resulted in no wait time reported for the appointment. The third scheduler altered the established desired date when there was no appointment availability within 2 weeks of that date; which would have resulted in an incorrectly reported wait time that was shorter than the patient actually waited from his or her original desired date. Recorded a new desired date when rescheduling appointment: Additionally, eight schedulers from three VAMCs incorrectly recorded a new desired date when rescheduling an appointment cancelled by the clinic rather than keeping the original desired date as required by VHA’s scheduling policy. Changing the desired date in this way would incorrectly decrease the reported wait times for the rescheduled appointments; veterans actually would wait longer than the reported wait times indicated. During our site visits, staff at some clinics told us they change medical appointment desired dates to show clinic wait times within VHA’s performance goals. A scheduler at one primary care clinic specifically stated that she changes the recorded desired date to the patient’s agreed-upon appointment date in order to show shorter wait times for the clinic. A provider at a specialty care clinic at another VAMC said providers in that clinic change the desired dates of their follow-up appointments if a patient cannot be scheduled within the 14-day performance goal. In addition, the reported wait times, derived from desired date, for one of the specialty care clinics we visited were inconsistent with the VAMC’s account of appointment scheduling backlogs and scheduling challenges, indicating reported wait time inaccuracies. At the time of our site visit, officials from this clinic indicated that long waits for new patient appointments had existed prior to our visit and told us that the next available appointment for a new patient was in 6 to 8 weeks. However, reported wait time data for the month we visited showed that the clinic completed all new patient appointments on the desired date, resulting in an unlikely high percentage of appointments with zero-day wait times that was inconsistent with information gathered during our site visit, raising questions about whether the desired date was recorded in accordance with VHA’s scheduling policy. Furthermore, according to reported wait times for the VAMC, this clinic completed nearly all new patient appointments within 14 days of the desired date for the 2 months prior to our visit; and, similarly, in the 2 months after our visit, reported wait times for this clinic show completion of all new patient appointments within the 14-day time frame. VHA central office officials told us that they recognized the potential reliability issues of using the desired date for measuring wait times, but stated that use of the desired date is the best approach for capturing patient experience and preference. Officials told us that there is no single industry standard for measuring how long patients wait for appointments and commonly used measures—such as capacity measures—do not account for patient preference or reflect how long the patient actually In addition, officials told us that the VistA waited for an appointment.scheduling system was not designed to capture data for management purposes, which has limited VHA’s options for developing wait time measures. Over the years, VHA has tried using many different approaches to measuring wait times, such as capacity measures and using the date the appointment was created rather than the desired date to determine wait times.used for performance accountability or reported on the PAR or NDPP and FDPP in fiscal year 2012, data on these measures are available to VISNs and VAMCs for performance monitoring. Officials told us that improving how wait times are measured is an ongoing effort, and they have conducted research to identify wait time measures that most closely correlate with patient satisfaction and positive outcomes. At the time of our review, VHA had not implemented changes to wait time performance measures based on the results of this research. Although these measures were not officially In addition to measuring medical appointment wait times, VHA central office officials reported that VHA also uses other information to monitor patients’ access to medical appointments and to assist VISNs and VAMCs in managing clinics. Patient Satisfaction Measures: VHA central office and VISN officials with whom we spoke identified patient satisfaction as another important indicator of patient access to medical appointments and VA has incorporated measures of self-reported patient satisfaction in its performance assessments. Specifically, the annual PAR includes a measure of overall patient satisfaction with VHA inpatient and outpatient healthcare in addition to the wait time measures derived from desired date. Separate measures related to patient satisfaction with obtaining outpatient care were also among the measures available for VISN and VAMC directors to include in their fiscal years 2011 and 2012 performance plans (NDPP and FDPP). VHA also makes the satisfaction measures available to VISNs and VAMCs for continuous performance monitoring as well as available to the public. One of the four VAMCs we visited included the satisfaction measures in their performance plan for fiscal year 2012, and officials cited monitoring these measures on a regular basis. Officials from one VISN also specifically cited comparing its VAMCs’ patient satisfaction scores to reported wait times to identify inconsistencies. However, the director of another VAMC said he does not rely on the satisfaction measures to monitor access because the data are dated by the time the VAMC sees the results, and instead, he relies on the scheduling data derived from wait time measures. Clinic Management Information: In addition to wait time measures, VHA has other information available for VISNs and VAMCs to manage clinics and monitor and improve clinic access, such as no-show rates and consult lists. Several clinic officials reported monitoring no-show rates— the rate at which patients do not appear for their scheduled appointment—in order to reduce unused appointments, for example, by identifying and providing additional appointment reminders to patients with frequent no-shows. Officials from multiple specialty clinics said they monitor lists of consults—requests for specialty care appointments—to ensure they are acted upon in a timely manner. Although the time between when the provider requests a consult and when the specialty clinic reviews the consult can affect the total time a patient waits for a specialty appointment, this time is not reflected in current wait time performance measures. The four VAMCs we reviewed did not consistently implement certain elements of VHA’s scheduling policy, including oversight requirements, which may result in increased wait time or delays in scheduling medical appointments. VAMCs also described other problems with scheduling timely medical appointments, including outdated technology, gaps in staffing of schedulers and providers, and telephone access problems. The four VAMCs we visited did not consistently implement VHA’s scheduling policy, which is intended to facilitate the creation of medical appointments that meet patients’ needs with no undue waits or delays. This policy includes the use of the VistA scheduling system to schedule medical appointments, and the use of the electronic wait list to track new patients waiting for medical appointments. (See table 3 for information on the number of clinics we visited that did not implement selected elements of the VHA’s scheduling policy.) Inconsistent implementation of VHA’s scheduling policy can result in increased wait time or delays in obtaining medical appointments. One of the clinics we visited did not use the VistA scheduling system to determine available medical appointment dates and times, and to schedule medical appointments, as required by VHA’s scheduling policy. Officials noted that this clinic lacked a full-time staff person dedicated to scheduling, and therefore, the providers called their patients to schedule their own medical appointments. Clinic staff reported that providers recorded medical appointments on sheets of paper and gave those sheets to a scheduler, who maintained a paper calendar of all medical appointments; this scheduler later recorded the appointment into the VistA scheduling system. Failing to use VistA to schedule medical appointments could create additional backlogs or scheduling errors because the schedule in VistA may not accurately reflect providers’ availability. According to one provider in this clinic, for example, “staff from other departments look in VistA and it looks like the clinic is not booked, so they’ll send their patients as walk-in appointments. However, the clinic is really fully booked and patients are waiting.” Officials from six clinics across two different VAMCs reported that staff scheduled new patient or established patient follow-up medical appointments without speaking to patients, and then notified patients of the scheduled medical appointment by letter, if the appointment was at least a few weeks away. This method of scheduling—referred to as “blind” scheduling by one official —is not in accordance with VHA’s scheduling policy and could result in missed medical appointments for patients who do not receive the letters, or are not available at the scheduled time because patients are not involved in the scheduling process. One scheduler noted that he sent medical appointment letters because he didn’t have time to call all patients to schedule appointments as he performs scheduling duties for 27 different clinics. Furthermore, outdated or incorrect patient contact information is an impediment to scheduling appointments via letters; an official in one of the six clinics told us that the databases containing patient contact information used to send such letters often do not have veterans’ correct or up-to-date contact information. Officials in four clinics across three VAMCs that had backlogs of patients waiting for medical appointments stated that they do not use the electronic wait list, the official VHA wait list used to track patients with whom a clinic does not have an established relationship. Clinics that do not use the electronic wait list may be at risk of losing track of new patients waiting for medical appointments. For example, at one specialty clinic with a backlog of consult requests, medical appointments for new patients were backed up almost 3 months; VAMC officials reported tracking patients waiting for medical appointments by printing paper copies of the consult requests from the electronic medical record. A provider at this clinic expressed concern that the clinic manager “has a tall stack of unscreened consult referrals just sitting on her desk, and no one is addressing them.” Officials from one VAMC stated that it did not have the required recall/reminder software to facilitate reminders for patients who need to return to the clinic for follow-up medical appointments more than 3 to 4 months into the future; therefore, none of its clinics, including the five clinics that we visited, were able to use it as intended. Instead clinics at this VAMC use a work-around in the scheduling system to remind clerks to print and send letters reminding patients to call and schedule their follow-up medical appointments. However, this work-around is not automated and relies on schedulers to remember to generate a list of patients who need follow-up medical appointments, and print and send those letters. The VAMC is in the process of implementing recall/reminder software, according to officials. One clinic in each of the four VAMCs visited did not keep their medical appointment schedules open 3 to 4 months into the future as required by VHA’s scheduling policy. appointments to be booked only 1 to 2 months into the future. Limiting the future medical appointment schedule may limit patients’ ability to schedule a follow-up medical appointment before leaving the clinic, as recommended by the policy, and also may result in additional work for clinic staff to send recall/reminder letters to patients for medical appointments less than 3 to 4 months away. VHA’s scheduling policy states that for clinics to most efficiently operate, “schedules must be open and available for the patient to make appointments at least three to four months into the future. Permissions may be given to schedulers to make appointments beyond these limits when doing so is appropriate and consistent with patient or provider requests. Blocking the scheduling of future appointments by limiting the maximum days into the future an appointment can be scheduled is inappropriate and is disallowed.” completion of the training by all staff who were required to complete it.Although all VAMCs we visited provided a list of staff who can schedule appointments, three VAMCs did not provide documentation that all staff on the list had successfully completed the required training. For example, officials from one VAMC stated that it maintained a list of staff who can schedule appointments, and a separate list of staff who had completed the training, but only in response to GAO’s request for documentation did the VAMC identify staff with scheduling access who needed to complete the training. Further, three of the 19 schedulers we interviewed said they completed training other than the required VHA scheduler training. Completion of required VHA scheduler training and maintaining up-to- date documentation of schedulers’ completion of the training is particularly important for ensuring consistent implementation of VHA’s scheduling policy, given the high rates of scheduler turnover described by officials. All four of the VAMCs we visited completed the required self-certification of compliance with the VHA’s scheduling policy for fiscal year 2011, three of which certified overall compliance, and one certified overall noncompliance.including the only one of the four that certified overall noncompliance, were initially uncertain who completed the certification or the steps taken to complete it, indicating that VAMCs are not always using the self- certification process to identify and improve problems with compliance with VHA’s scheduling policy. VAMCs identified several problems that can impede the timely scheduling of medical appointments, which also may impact their compliance with VHA’s scheduling policy. VHA central office officials and officials from all of the VAMCs we visited said the VistA scheduling system is outdated and inefficient, which hinders the timely scheduling of medical appointments. In particular, officials said the scheduling system requires schedulers to use commands requiring many keystrokes and does not allow them to view multiple screens at once. Schedulers must open and close multiple screens to check a provider’s or clinic’s full availability when scheduling a medical appointment, which is time-consuming and can lead to errors. For example, providers have separate schedules within VistA to accommodate the various types of services they provide. Because the scheduling system cannot display multiple schedules on the same screen, schedulers have to enter and exit multiple screens to check a provider’s full daily schedule when scheduling a medical appointment. If schedulers do not open all of the necessary screens, they may unknowingly create scheduling errors such as booking two medical appointments at the same time in different sections of a provider’s schedule. Further, staff at one VAMC told us the problem of not being able to easily view a provider’s full schedule can result in the failure to ensure that appointments are cancelled when a provider requests it. This error could cause patients to come to the VAMC unnecessarily or a failure to reschedule cancelled appointments in a timely way, both of which might lead to increased wait times for those patients. Officials from all the VAMCs we visited also noted that the VistA scheduling system is not easily adapted to meet clinic needs. For example, staff cannot create a provider schedule in the scheduling system that is longer than 8 hours. If a provider wants to extend his or her schedule on certain days, staff must create additional clinic schedules in the scheduling system for that provider, which can result in more delays and possible errors because schedulers have to check additional screens for medical appointment availability. Furthermore, officials told us that the scheduling system does not automatically interface with VHA’s electronic medical record, which makes the scheduling process more time- consuming as schedulers alternate between the two software applications to ensure medical appointments are made in accordance with providers’ guidance. VAMC officials described steps they take to ensure schedulers use VistA in accordance with the scheduling directive, including ongoing scheduler training and supervisory reviews of scheduler performance. However, as noted above, a lack of clarity in the desired date training documents and a lack of documentation of scheduler training at certain facilities may limit the effectiveness of these interventions. One VAMC provides schedulers with dual monitors to enable them to open multiple screens at once. Another VAMC told us they considered this solution in their primary care clinic, but found that limited physical space in the clinic did not accommodate additional monitors. In response to ongoing problems with the VistA scheduling system, VHA undertook an initiative to replace it in 2000, but VA abandoned the replacement due to weaknesses in project management and a lack of effective oversight. VA released a new request for information in December 2011 to gather information about vendors and possible software packages that could replace the current scheduling system. In September 2012, VHA told us that vendors’ responses to the request for information indicated that VHA will be able to choose among several viable software packages. According to officials, VA’s next step is to compare different vendors’ software packages through the summer of 2013, and subsequently issue a request for vendor proposals. VHA central office officials and officials from all of the VAMCs we visited stated that shortages or turnover of schedulers also creates problems for the timely scheduling of medical appointments. Officials said that schedulers perform many important roles, including greeting patients, checking patients in and out of clinics, answering telephone calls, scheduling medical appointments for primary care, as well as specialty care consults, and performing other administrative support functions on behalf of the clinical staff. Officials explained, however, that high stress and a demanding workload as well as the entry-level pay grade of the scheduler position leads to high turnover. Further, officials told us that high-performing schedulers often are quickly promoted to other positions within VA. According to VHA officials, most scheduler positions are classified as a low grade within the government general schedule pay scale with little room for upward movement within the grade. Officials at two of the VAMCs we visited told us they are working to raise the pay level for schedulers; for example, one VAMC has begun to assess scheduler position descriptions to determine whether they can be reclassified to allow for more flexibility in determining scheduler salaries based on the variation in their assigned duties. Given the important role of schedulers in the scheduling process, officials said that even temporary staffing gaps or shortages can cause medical appointment delays or wait times. Staff with whom we spoke in several clinics said that when scheduler staffing is lacking, including when a scheduler is on short-term leave, it is difficult to cover all the scheduler’s duties, and that such gaps can cause delays for patients. Further, we were told that scheduler staffing gaps resulted in inefficient use of clinical staff time. For example, at one specialty clinic that lacked its own scheduler, providers routinely scheduled their own medical appointments, which took away from time seeing patients, and also resulted in incorrect scheduling practices. Given the training needs associated with using the VistA scheduling system, following VHA’s scheduling policy, and ensuring the correct use of desired date, high rates of scheduler turnover could contribute to inconsistent use of desired date in the scheduling process or other appointment scheduling problems. Officials at two VAMCs noted that scheduler staffing gaps are compounded by recent changes in their roles and responsibilities as VHA implements a new team-based model of primary care, which calls for one scheduler to be assigned to each primary care team. Officials told us that these changes generally increase the administrative demands placed on schedulers, as they are asked to respond to team duties while continuing to answer phones, greet patients, and register new patients, among other responsibilities. Officials from two VAMCs told us they had requested approval to hire additional staff to meet these added administrative needs. Scheduler staffing gaps may also create problems managing patient flow through clinics, which can impede scheduling of follow-up appointments, according to officials at two of the VAMCs we visited. Staff at these VAMCs told us that they sometimes do not have sufficient schedulers available to staff check-out desks, and staff at one VAMC added that as a result patients might “fall through the cracks,” leaving follow-up medical appointments unscheduled unless the patient remembers to call in to schedule the appointment. In addition, when patients do not check out, schedulers are responsible for tracking patients needing follow-up medical appointments. This situation may be exacerbated in clinics that do not use the required recall/reminder software to facilitate the scheduling of follow-up medical appointments more than 3 to 4 months in the future, adding further to the backlog of patients in need of follow-up medical appointments. Officials from all of the VAMCs we visited told us that provider shortages also contribute to scheduling backlogs in certain locations and specialties. Recruitment and retention of providers was a particular challenge for VAMCs in rural areas, areas with high costs of living, and for certain provider specialties. All of the VAMCs we visited described gaps in provider staffing in certain specialty care clinics. Officials at all VAMCs also stated that a lack of salary competitiveness or the length of time to hire new providers into the VA system also contributed to gaps in provider staffing and scheduling backlogs. Gaps in provider staffing also can result from providers being on extended or unexpected leave, including vacation time, sick leave, or military deployments. These absences may result in longer wait times for patients. For example, officials at one VAMC told us that even a brief absence of one provider on leave can cause significant wait times, and that it is difficult to catch up and eliminate the backlog. Staff from some clinics described steps they take to reduce backlogs caused by gaps in provider staffing, including overbooking provider schedules and scheduling temporary Saturday hours. Officials at one VAMC told us that they employ a “floater” primary care physician to provide coverage for providers on leave, but an official at another clinic told us that they were unable to hire additional providers to meet the demand for medical appointments. Officials at all of the VAMCs we visited told us that high call volumes and a lack of staff dedicated to answering the telephones impede the timely scheduling of medical appointments. Despite VHA’s telephone policy requiring the provision of continuous telephone service for clinical care and medical appointment management, VAMC officials noted that schedulers are frequently overwhelmed by high call volumes and are unable to respond to calls in a timely way. In addition, officials at one VAMC told us that outdated telephone technology, and the lack of a dedicated VAMC-wide call center, limited their ability to improve their telephone responsiveness. VHA has reported that telephone access to VHA health services has historically been a frustrating experience for veterans, including dropped calls, multiple transfers, and long waits to reach a staff person able to resolve their inquiries. Further, patients at all of the VAMCs we visited registered complaints about the difficulty of reaching outpatient clinic staff by telephone and unreturned telephone calls. According to information on patient complaints provided by the four VAMCs we visited, patient complaints about unreturned telephone calls ranked among the top two categories of complaints in fiscal year 2012 at all four VAMCs. Further, staff at two of the VAMCs reported that their telephone calls to outpatient clinics within their own VAMC went unanswered, and one added that their inability to reach staff in their own clinics also was an obstacle to timely medical appointment scheduling. In January 2012, VHA distributed suggested best practices for improving telephone design, service, and access in its Telephone Systems Improvement Guide. This guide outlines steps VHA found to be effective means of improving telephone service and maintaining health care access, including regularly monitoring the purpose and volume of telephone calls; establishing dedicated staff to answering calls, especially at times of peak call volume; and training staff responsible for answering telephones in call centers. To address telephone issues, officials at one VAMC we visited told us they were developing a proposal to establish a call center with a new telephone system, to be staffed by schedulers dedicated to answering the telephones. Officials at a different VAMC stated that a scheduling supervisor periodically checks schedulers’ telephones to ensure that voice mail messages are listened to and that calls are returned. VHA is implementing several initiatives to improve veterans’ access to medical appointments. Specifically, these initiatives focus on more patient-centered care; using technology to provide care, through means such as telehealth; and using care outside of VHA to reduce travel and wait times for veterans who are unable to receive certain types of outpatient care in a timely way through local VHA facilities. VHA officials told us they are monitoring the implementation of these initiatives; however, in some cases, more information is needed to determine their impact on timely access to care over time. VHA’s patient-centered medical home model for primary care, Patient Aligned Care Teams (PACT), is intended, in part, to improve access to medical appointments and care coordination through the use of interdisciplinary care teams and technology to communicate with patients. Implementation of PACT began in 2010, and is an ongoing effort, according to VHA officials. PACT differs from how primary care was previously delivered by assigning each patient to an interdisciplinary team. The PACT team is intended to be comprised of a primary care provider, registered nurse care manager, a clinical support staff member These teams offer such as a licensed practical nurse, and a scheduler.patients a centralized way to get questions answered by nurses or other clinical support staff and aim to reduce the need for face-to-face medical appointments, thereby enabling more efficient use of providers’ time. For example, at one of the VAMCs we visited, patients are given a direct telephone number to contact their PACT team and leave a voice mail message to be returned by the team’s registered nurse. Encouraging PACT teams’ use of telephone communication and telephone appointments is intended to enable patients to more quickly obtain answers to some of their administrative and medical questions, such as requests for prescription refills, without having to schedule a face-to-face medical appointment. VHA officials told us that they expect PACT teams’ use of telephone communication and telephone appointments will open up face-to-face medical appointment slots for patients who need them and might enable clinics to reduce backlogs and improve access to same- day primary care medical appointments. Officials at two VAMCs we visited told us that the transition to the PACT model has created some initial scheduling and staffing difficulties. For example, officials at these VAMCs noted that it is difficult for scheduling staff to respond to their PACT team duties in addition to meeting other responsibilities such as answering phones, checking in patients, registering new patients, and scheduling for more than one clinic. This is compounded by the fact that not all PACT teams have been assigned their own scheduler, as prescribed by the PACT model, so an individual scheduler is sometimes serving multiple PACT teams. Officials at these two VAMCs explained that they would need to hire more schedulers to meet the goal of assigning one to each PACT team. To measure the progress of PACT implementation and its impact on access to quality care, VHA is collecting data and tracking a series of measures in a monthly internal data report. Five of the PACT measures are (1) primary care medical appointments completed within 7 days of the desired date; (2) same day access with primary care provider, or the percentage of appointments completed within 1 day; (3) telephone utilization, or the percentage of total encounters that occur by telephone; (4) continuity of care, or the percentage of primary care appointments with the patient’s assigned primary care provider; and (5) post-hospital discharge contact, or percentage of patients discharged from the hospital who were contacted by their primary care provider within 2 days. As described earlier, accurate measurement of medical appointment wait times—including the first two PACT measures—is dependent upon the correct recording of the desired date in the VistA scheduling system. In fiscal year 2012, PACT measures were also included in the NDPP and FDPP. Part of VHA’s goal of achieving improved access to medical appointments is the increased use of technology such as telehealth and secure messaging. Use of these tools is intended to improve communication between patients and providers and open up providers’ schedules for needed face-to-face medical appointments, thereby improving access to face-to-face appointments. home telehealth for chronic disease management such as diabetes; real-time clinic-based video telehealth, in which patients at a local CBOC may connect with a VHA provider at a different location to receive services that are unavailable at the CBOC, such as mental health or speech pathology; and store-and-forward telehealth, in which digital images such as x-rays or images of skin problems, are taken, stored, and sent to an expert for review and consultation. VHA officials told us that the use of telehealth can reduce both travel and wait times for medical appointments and help meet the needs of patients with chronic conditions. All VAMCs we visited told us they were using telehealth to improve access to care. Another initiative that uses technology to reduce unnecessary face-to- face medical appointments is VHA’s My HealtheVet, a web-based program that enables veterans to create and maintain a web-based personal health record with secure access to health information; services such as prescription refill requests; and secure messaging. Secure messaging allows veterans to communicate electronically with their health care team. According to VHA, of the more than 8 million veterans enrolled in VHA, 1.4 million are registered in My HealtheVet as of August 2012, and more than 437,000 have created secure messaging accounts. A recent VA study reports that secure messaging may improve access, patient perceptions about access, and provides for better communication. VHA uses non-VA care to reduce wait times and backlogs and to provide veterans’ access to specialists not available through VHA.statutory requirement to help veterans receive care closer to home, VHA is piloting a new model of non-VA care known as Project ARCH (Access Under a Received Closer to Home). Project ARCH is a five-site, 3-year pilot program administered by the VHA Office of Rural Health to provide health care services through contracts with local community providers.According to VHA officials, Project ARCH might help alleviate wait times for specialty care services with high demand, or for which there is a shortage of local providers. At the Montana Project ARCH pilot site, which we visited as part of our site visit to the Montana VAMC, staff from the VAMC and the Billings Clinic, a non-VA provider delivering services to veterans through Project ARCH, identified both benefits and obstacles for patients enrolled in Project ARCH. For example, though VAMC and Billings Clinic staff noted that Project ARCH reduced both travel and wait times for Montana veterans in need of orthopedic care, Billings Clinic staff also noted that difficulties in coordinating care for veterans moving between VHA and non-VA providers at times resulted in delays in providing care to those and other veterans. Additionally, problems with processing authorizations for certain services were among the concerns raised in an April 2012 evaluation of the Montana Project ARCH program. Project ARCH contractors must submit monthly reports, including information on medical appointment scheduling timeliness, wait times, and other topics. For example, the contractor for the Project ARCH Program in Montana is required to report on the extent to which it is meeting VHA’s 14-day wait time goal for medical appointments— according to VHA officials, the contractor must meet a 90 percent target. These wait times may not accurately reflect how long patients are waiting for a medical appointment, however, because the wait time is counted from the time the contractor receives the authorization from VA, rather than from the time the patient or provider requests a medical appointment. VHA officials have expressed an ongoing commitment to providing veterans with timely access to medical appointments and have reported continued improvements in achieving this goal. However, unreliable wait time measurement has resulted in a discrepancy between the positive wait time performance VA has reported and veterans’ actual experiences. Ambiguity in what constitutes the medical appointment desired date—the date VHA uses as the basis for measuring wait time—as well as manipulation of the desired date to meet goals have contributed to these inaccuracies. With more than 50,000 schedulers making approximately 80 million medical appointments in fiscal year 2011, establishing a clear definition of the desired date or finding and reporting another acceptable measure of wait time is key to understanding how long veterans are actually waiting for medical appointments. Without reliable measurement of how long patients are waiting for medical appointments, VHA is less equipped to identify and address factors that contribute to wait times, or gauge the success of its initiatives to improve access to timely medical appointments, including efforts to improve primary care medical appointments. More consistent adherence to VHA’s scheduling policy and oversight of the scheduling process, as well as the allocation of staffing resources in accordance with clinics’ demands for scheduling of medical appointments, would potentially reduce medical appointment wait times. Furthermore, persistent problems with telephone access must be resolved to assure veterans’ ability to schedule timely medical appointments. Ultimately, VHA’s ability to ensure and accurately monitor access to timely medical appointments is critical to ensuring quality health care to veterans, who may have medical conditions that worsen if access is delayed. To ensure reliable measurement of veterans’ wait times for medical appointments, we recommend that the Secretary of VA direct the Under Secretary for Health to take actions to improve the reliability of wait time measures either by clarifying the scheduling policy to better define the desired date, or by identifying clearer wait time measures that are not subject to interpretation and prone to scheduler error. To better facilitate timely medical appointment scheduling and improve the efficiency and oversight of the scheduling process, we recommend that the Secretary of VA direct the Under Secretary for Health to take actions to ensure that VAMCs consistently and accurately implement VHA’s scheduling policy, including use of the electronic wait list, as well as ensuring that all staff with access to the VistA scheduling system complete the required training. To improve timely medical appointment scheduling, we recommend that the Secretary of VA direct the Under Secretary for Health to develop a policy that requires VAMCs to routinely assess clinics’ scheduling needs and resources to ensure that the allocation of staffing resources is responsive to the demand for scheduling medical appointments. To improve timely medical appointments and to address patient and staff complaints about telephone access, we recommend that the Secretary of VA direct the Under Secretary for Health to ensure that all VAMCs provide oversight of telephone access and implement best practices outlined in its telephone systems improvement guide. In reviewing a draft of this report, VA generally agreed with our conclusions and concurred with our recommendations. (VA’s comments are reprinted in app. I.) In summary, VA stated that VHA officials have closely followed our review and proactively taken steps in response to our findings. Specifically, VHA is revising and improving directives, policies, training, clinic management tools, and oversight related to scheduling practices. VA further stated that VHA is committed to routinely assessing clinics’ scheduling needs and resources and developing practices and guidelines to ensure adequate staffing resources for scheduling medical appointments. VA described its plans to address each recommendation as follows: In response to our recommendation that VA take actions to improve the reliability of wait time measures, VA concurred and stated that VHA will revise its scheduling policy to implement more reliable wait time measures and new processes to better define desired date with a targeted completion date of November 1, 2013. In response to our recommendation that VA take actions to ensure that VAMCs consistently and accurately implement VHA’s scheduling policy and ensure that all staff complete required training, VA concurred and stated that the revised scheduling policy will include improvements and standardization of the use of the electronic wait list. Additionally, VHA will require VISNs to update each VAMC’s scheduler master list and verify that all schedulers on the list have completed required training, and will require schedulers to complete a standardized training update on the revised scheduling policy. The targeted completion date for these activities is November 1, 2013. In response to our recommendation that VA develop a policy that requires VAMCs to routinely assess clinics’ scheduling needs and resources, VA concurred and stated that VHA will ask VAMCs to routinely assess clinics’ availability and ensure staff is distributed to meet access standards in clinics. However, VA has not specified requirements for VAMCs to complete these assessments nor has the agency provided a timeline for this process. Because schedulers are key to ensuring timely appointment scheduling, we believe that VA should establish a targeted completion date for requiring these assessments in policy or guidance. In response to our recommendation that VA ensure that all VAMCs provide oversight of telephone access and implement best practices outlined in its telephone improvement guide, VA concurred and stated that VHA will require each VISN director to assess current phone service and develop strategic improvement telephone service plans to improve service. Additionally, VHA will identify a process to monitor performance on a quarterly basis for at least 1 year after the assessment. The targeted completion date for the telephone service assessments and plans is March 30, 2013. As arranged with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 28 days after its issuance date. At that time, we will send copies of this report to appropriate congressional committees, the Secretary of Veterans Affairs and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or draperd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II. In addition to the contact named above, Bonnie Anderson, Assistant Director; Rebecca Abela; Jennie Apter; Rich Lipinski; Sara Rudow; and Ann Tynan made key contributions to this report. VA Mental Health: Number of Veterans Receiving Care, Barriers Faces, and Efforts to Increase Access. GAO-12-12. Washington, D.C.: October 14, 2011. Information Technology: Department of Veterans Affairs Faces Ongoing Management Challenges. GAO-11-663T. Washington, D.C.: May 11, 2011. Information Technology: Management Improvements Are Essential to VA’s Second Effort to Replace Its Outpatient Scheduling System. GAO-10-579. Washington, D.C.: May 27, 2010. VA Health Care: Access for Chattanooga-Area Veterans Needs Improvement. GAO-04-162. Washington, D.C.: January 30, 2004. VA Health Care: More National Action Needed to Reduce Waiting Times, but Some Clinics Have Made Progress. GAO-01-953. Washington, D.C.: August 31, 2001. Veterans’ Health Care: VA Needs Better Data on Extent and Causes of Waiting Times. GAO/HEHS-00-90. Washington, D.C.: May 31, 2000.

Summary: VHA provided nearly 80 million outpatient medical appointments to veterans in fiscal year 2011. While VHA has reported continued improvements in achieving access to timely medical appointments, patient complaints and media reports about long wait times persist. GAO was asked to evaluate VHAs scheduling of timely medical appointments. GAO examined (1) the extent to which VHAs approach for measuring and monitoring medical appointment wait times reflects how long veterans are waiting for appointments; (2) the extent to which VAMCs are implementing VHAs policies and processes for appointment scheduling, and any problems encountered in ensuring veterans access to timely medical appointments; and (3) VHAs initiatives to improve veterans access to medical appointments. To conduct this work, GAO made site visits to 23 clinics at four VAMCs, the latter selected for variation in size, complexity, and location. GAO also reviewed VHAs policies and data, and interviewed VHA officials. Outpatient medical appointment wait times reported by the Veterans Health Administration (VHA), within the Department of Veterans Affairs (VA), are unreliable. Wait times for outpatient medical appointments--referred to as medical appointments--are calculated as the number of days elapsed from the desired date, which is defined as the date on which the patient or health care provider wants the patient to be seen. The reliability of reported wait time performance measures is dependent on the consistency with which schedulers record the desired date in the scheduling system in accordance with VHA's scheduling policy. However, VHA's scheduling policy and training documents for recording desired date are unclear and do not ensure consistent use of the desired date. Some schedulers at Veterans Affairs medical centers (VAMC) that GAO visited did not record the desired date correctly. For example, three schedulers changed the desired date based on appointment availability; this would have resulted in a reported wait time that was shorter than the patient actually experienced. VHA officials acknowledged limitations of measuring wait times based on desired date, and described additional information used to monitor veterans' access to medical appointments, including patient satisfaction survey results. Without reliable measurement of how long patients are waiting for medical appointments, however, VHA is less equipped to identify areas that need improvement and mitigate problems that contribute to wait times. While visiting VAMCs, GAO also found inconsistent implementation of VHA's scheduling policy that impedes VAMCs from scheduling timely medical appointments. For example, four clinics across three VAMCs did not use the electronic wait list to track new patients that needed medical appointments as required by VHA scheduling policy, putting these clinics at risk for losing track of these patients. Furthermore, VAMCs' oversight of compliance with VHA's scheduling policy, such as ensuring the completion of required scheduler training, was inconsistent across facilities. VAMCs also described other problems with scheduling timely medical appointments, including VHA's outdated and inefficient scheduling system, gaps in scheduler and provider staffing, and issues with telephone access. For example, officials at all VAMCs GAO visited reported that high call volumes and a lack of staff dedicated to answering the telephones impede scheduling of timely medical appointments. In January 2012, VHA distributed telephone access best practices that, if implemented, could help improve telephone access to clinical care. VHA is implementing a number of initiatives to improve veterans' access to medical appointments such as expanded use of technology to interact with patients and provide care, which includes the use of secure messaging between patients and their health care providers. VHA also is piloting a new initiative to provide health care services through contracts with community providers that aims to reduce travel and wait times for veterans who are unable to receive certain types of care within VHA in a timely way. GAO recommends that VHA take actions to (1) improve the reliability of its medical appointment wait time measures, (2) ensure VAMCs consistently implement VHA's scheduling policy, (3) require VAMCs to allocate staffing resources based on scheduling needs, and (4) ensure that VAMCs provide oversight of telephone access and implement best practices to improve telephone access for clinical care. VA concurred with GAO's recommendations.