In October 2014, VA modified its existing contracts with two TPAs that were administering another VA community care program to add certain administrative responsibilities associated with the Choice Program. For the Choice Program, each of the two TPAs—Health Net and TriWest—is responsible for delivering care in a specific multi-state region (See figure 1.). Specifically, the TPAs are responsible for establishing networks of community providers, scheduling appointments with community providers for eligible veterans, and paying community providers for their services. As stated in VA’s December 2015 guidance, the Choice Program allows eligible veterans to opt to obtain health care services from the TPAs’ network providers rather than from VHA medical facilities when the veterans are enrolled in the VA health care system and meet any of the following criteria: the next available medical appointment with a VHA clinician is more than 30 days from the veteran’s preferred date or the date the veteran’s physician determines he or she should be seen; the veteran lives more than 40 miles driving distance from the nearest VHA facility with a full-time primary care physician; the veteran needs to travel by air, boat, or ferry to the VHA facility that is closest to his or her home; the veteran faces an unusual or excessive burden in traveling to a VHA facility based on geographic challenges, environmental factors, or a medical condition; the veteran’s specific health care needs, including the nature and frequency of care needed, warrants participation in the program; or the veteran lives in a state or territory without a full-service VHA medical facility. Over the life of the Choice Program, VA has taken various approaches to care for veterans for whom services are not available at a particular VHA medical facility. In May and October of 2015, VHA issued policy memoranda to its VAMCs that required them to offer veterans referrals to the Choice Program before they authorized care through one of VA’s other community care programs, which existed prior to the creation of the Choice Program. Before May 2015, VA provided VAMCs the flexibility to decide on a case-by-case basis whether to refer veterans to the Choice Program or one of VA’s other community care programs when services were not available. In June 2017, VHA issued another policy memorandum that rescinded the referral hierarchy that required VAMCs to refer to the Choice Program first. It directed VAMCs to refer veterans to the Choice Program only if they met the Choice Act’s wait-time, distance, and geographic eligibility criteria, and to instead use other VHA medical facilities, facilities with which VA has sharing agreements, and other VA community care programs to deliver care to veterans when services were not available at a VHA medical facility and veterans did not qualify under the Choice Act’s eligibility criteria. In August 2017, after Congress provided an additional $2.1 billion for the Choice Program, VHA again changed its guidance on referral patterns for the Choice Program and VA’s other community care programs. Specifically, VA issued a fact sheet saying that the new funding will allow VAMCs to refer veterans to the Choice Program to the maximum extent possible. This allowed VAMCs to again offer veterans Choice Program referrals when services are unavailable at VHA medical facilities (until available funds have been exhausted), and also permitted VAMCs to refer veterans to other VA community care programs when services are unavailable. Through policies and standard operating procedures for VAMCs and contracts with the TPAs, VA and VHA have established two separate processes for Choice Program routine and urgent appointment scheduling: one process for time-eligible veterans and another for distance-eligible veterans. Table 1 provides an overview of the appointment scheduling process that applies when a veteran is referred to the Choice Program because the veteran is time-eligible. (Appendix II contains additional detail about the Choice Program appointment scheduling process for time-eligible veterans—including differences between the routine and urgent care appointment scheduling process.) When veterans reside more than 40 miles from a VHA medical facility or meet other travel-related criteria, VHA uses the appointment scheduling process it developed for distance-eligible veterans. The process for distance-eligible veterans differs from that for time-eligible veterans in that VAMCs do not prepare a referral and send it to the TPA. Instead, distance-eligible veterans contact the TPA directly to request Choice Program care. See table 2 for an overview of the Choice Program appointment scheduling process that applies for distance-eligible veterans. (See appendix III for additional detail about the Choice Program appointment scheduling process for distance-eligible veterans—including differences between the routine and urgent care appointment scheduling process.) Data we obtained from the TPAs indicate that VHA and the TPAs used the time-eligible appointment scheduling process about 90 percent of the time from fiscal year 2015 through fiscal year 2016 (the first 2 years of the Choice Program’s implementation). More than half of the veterans who were referred to the Choice Program and for whom the TPAs scheduled appointments were referred because the services they needed were not available at a VHA medical facility. The second-most-common reason for referral was that the wait time for an appointment at a VHA medical facility exceeded 30 days. (See figure 2.) The distance-eligible appointment scheduling process was used for about 10 percent of the veterans who used the Choice Program between fiscal year 2015 through fiscal year 2016. In coordinating the furnishing of care to eligible veterans under the Choice Program, VA is required to ensure that veterans receive appointments for Choice Program care within the wait-time goals of VHA for the furnishing of hospital care and medical services. Although the Choice Act defined VHA’s wait-time goals as not more than 30 days from the date a veteran requests an appointment from the Department, the Choice Act gave VA the authority to change this definition if it did not reflect VHA’s actual wait- time goals. If VA wanted to exercise this authority, it was required to notify Congress of VHA’s actual wait-time goals within 60 days of the law’s enactment (i.e., by October 6, 2014). VA did so in an October 3, 2014, report to Congress. To “ensure that care provided through the Veterans Choice Program is delivered within clinically appropriate timeframes,” VA notified Congress that VHA’s wait-time goals were “not more than 30 days from either the date that an appointment is deemed clinically appropriate by a VA health care provider, or if no such clinical determination has been made, the date a Veteran prefers to be seen for hospital care or medical services.” By incorporating VHA’s reported wait- time goal, the Choice Act required VA to ensure the furnishing of care to eligible veterans within 30 days of the clinically indicated date or, if none existed, within 30 days of the veteran’s preferred date. VA has purchased health care services from community providers through various programs since as early as 1945. Currently, there are six community care programs other than the Choice Program through which VA purchases hospital care and medical services for veterans. These six community care programs offer different types of services and have varying eligibility criteria for veterans and community providers. VA’s six non-Choice community care programs include: Individually authorized VA community care. The primary means by which VHA has traditionally purchased community care is through individual authorizations, where local VAMC staff determine veteran eligibility, create authorizations, and assist veterans in arranging care with community providers that are willing to accept VA payment. Traditionally, VAMCs have approved the use of individually authorized community care when a veteran cannot access a particular specialty care service from a VHA medical facility because the service is not offered or the veteran would have to travel a long distance to obtain it from a VHA medical facility. (See appendix IV for an illustration of how appointment scheduling and care coordination processes for the Choice Program compare to those for individually authorized VA community care.) Two emergency care programs. When VA community care is not preauthorized, VA may reimburse community providers for emergency care under two different community care programs: 1) emergency care for a condition related to a veteran’s service-connected disability and 2) emergency care for a condition not related to a veteran’s service-connected disability, commonly referred to as Millennium Act emergency care. For emergency care to be covered through these two programs, a number of criteria must be met, including (1) community providers must file claims in a timely manner (within 2 years of the date services were rendered for service-connected emergency care and within 90 days for Millennium Act emergency care); (2) the veteran’s condition must meet the prudent layperson standard of an emergency; and (3) a VA or other federal medical facility must not have been feasibly available to provide the needed care, and an attempt to use either would not have been considered reasonable by a prudent layperson. Patient-Centered Community Care (PC3). In September 2013, VA awarded contracts to Health Net and TriWest to develop regional networks of community providers to deliver specialty care, mental health care, limited emergency care, and maternity and limited newborn care when such care is not feasibly available from a VHA medical facility. VA and the TPAs began implementing the PC3 program in October 2013, and it was fully implemented nationwide as of April 2014—prior to the creation of the Choice Program. In August 2014, VA expanded the PC3 program to allow community providers of primary care to join the PC3 networks. PC3 is a program VA created under existing statutory authorities, not a program specifically designed by law. To be eligible to obtain care from PC3 providers, veterans must meet the same criteria that are required for individually authorized VA care in the community services. Agreements with federal partners and academic affiliates. When services are not available at VHA medical facilities, VA may also obtain specialty, inpatient, and outpatient health care services for veterans through two types of sharing agreements—those with other federal facilities (such as those operated by the Department of Defense and the Indian Health Service), and those with university- affiliated hospitals, medical schools, and practice groups (known as academic affiliates). Dialysis contracts. In June 2013, VA awarded contracts to numerous community providers nationwide to deliver dialysis—a life-saving medical procedure for patients with end-stage renal disease (permanent kidney failure). When dialysis services are not feasibly available at VHA medical facilities, veterans may be referred to one of VA’s contracted dialysis providers, and veterans may receive dialysis at local clinics on an outpatient basis, or at home (if the contractors offer home-based dialysis services). The VA Budget and Choice Improvement Act, which was enacted on July 31, 2015, required VA to develop a plan for consolidating all of its community care programs into a new, single program to be known as the “Veterans Choice Program.” VHA submitted this plan, including proposed legislative changes, to Congress on October 30, 2015. VA has moved forward with some aspects of the planned community care program consolidation that it believes can be accomplished without statutory changes. In December 2016, VA issued a request for proposals (RFP) for contractors to help administer the consolidated community care program, through “community care network” contracts. The consolidated community care program VA described in the October 2015 plan it submitted to Congress and the December 2016 RFP, as amended, would be similar to the current Choice Program in certain respects. For example, VA is planning to award community care network contracts to TPAs, which would establish regional networks of community providers and process payments to those providers. In contrast, other aspects of the consolidated community care program VA has planned may differ from the existing Choice Program. For example, VA’s RFP for the community care network contracts, as amended, requires VAMCs—rather than TPAs—to carry out appointment scheduling, unless they exercise a contract option for the TPAs to provide such services. In fiscal year 2015, the first year of the Choice Program’s implementation, total obligations for Choice Program health care services accounted for about 4.7 percent of the $8.7 billion VA obligated for all community care services that year. However, as more care was provided through the Choice Program in fiscal years 2016 and 2017, obligations for Choice Program care grew steadily, while obligations for care delivered through other VA community care programs decreased. In fiscal year 2017, total obligations for Choice Program health care services accounted for about 39 percent of the $11.16 billion VA obligated for all community care services that year. See table 3. As shown in Table 4, below, of the $10.37 billion in Choice Program funds that were obligated between fiscal year 2015 and fiscal year 2017, $6.28 billion (or about 61 percent) of the funds were obligated for Choice Program health care services. About $1.76 billion (or 17 percent) of total Choice Program funds obligated between fiscal year 2015 and fiscal year 2017 were obligated for administrative costs. The remaining $2.33 billion (about 22 percent) were obligated for medical services other than those authorized under the Choice Program. As we previously reported, VHA experienced a projected funding gap in its medical services appropriation account in fiscal year 2015, largely due to lower-than-expected obligations for the Choice Program, higher-than-expected obligations for other VA community care programs, and unanticipated obligations for hepatitis C drugs. To address the projected funding gap, on July 31, 2015, VA obtained temporary authority to use Choice Program funds between July 31, 2015 and September 30, 2015 for amounts obligated on or after May 1, 2015 to furnish medical services other than those that it authorized under the Choice Program. Later, in fiscal year 2016 and fiscal year 2017, VA de-obligated about $420 million of the Choice Program funds it had obligated for other VA community care programs and hepatitis C drugs in fiscal year 2015 because they were never expended. Our analysis of VA’s scheduling process indicates that veterans who are referred to the Choice Program for routine care because they are time- eligible could potentially wait up to 70 calendar days to obtain care, if VAMCs and TPAs take the maximum amount of time allowed by VA’s process. About 90 percent of Choice Program referrals in fiscal years 2015 and 2016 were scheduled under the time-eligibility process, which means that the majority of veterans referred to the program would have been subject to this potential wait time for an appointment for routine care. This 70-day potential wait time is in contrast to the Choice Act’s required time frame, which is that eligible veterans receive Choice Program care no more than 30 days from the date an appointment is deemed clinically appropriate by a VHA clinician (referred to as the clinically indicated date), or if no such determination has been made, 30 days from the date the veteran prefers to receive care. According to VHA policy, a VHA clinician’s clinically indicated date determination must be based upon the needs of the patient, and it should be the earliest date that it would be clinically appropriate for the veteran to receive care. Therefore, if there is no clinical reason that care should be delayed, a veteran’s clinically indicated date could be the same date that the VHA clinician determined the veteran needed care. The potential wait time of about 70 calendar days for time-eligible veterans to receive routine care through the Choice Program encompasses 18 or more calendar days for VAMCs to prepare veterans’ Choice Program referrals and potentially another 52 calendar days for appointments to occur through the TPAs’ scheduling process, as follows: VAMCs’ process for preparing routine Choice Program referrals. According to VHA policies and guidance, VAMC staff have at least 18 calendar days to confirm that veterans want to be referred to the Choice Program and to send veterans’ referrals to the TPAs: They have 2 business days (or up to 4 calendar days) after a VHA clinician has determined the veteran needs care to begin contacting an eligible veteran by telephone to offer them a referral to the Choice Program. They have up to 14 calendar days after initiating contact to reach the veteran by telephone or letter and confirm that the veteran wants to be referred to the Choice Program. After confirming that a veteran wants to be referred to the Choice Program, however, VA has not set a limit on the number of days VAMCs should take to compile relevant clinical information and send referrals to the TPAs. TPAs’ process for scheduling routine Choice Program appointments. Through its contracts with the TPAs, VA has established a process under which a veteran could potentially wait another 52 calendar days from the date the TPA receives the VAMC’s Choice Program referral for a routine care appointment to take place. This includes up to 16 business days (or 22 calendar days) after receiving a referral to confirm the veteran’s decision to opt in to the Choice Program and create an authorization. The contracts further state that, for time-eligible veterans, an appointment shall take place within 30 calendar days of the clinically indicated date, the authorization creation date, or the veteran’s preferred date, whichever occurs later: The TPA has 2 business days to review the VAMC’s referral and accept it if it contains sufficient information to proceed with appointment scheduling. The TPA has 4 business days to contact the veteran by telephone and confirm they want to opt in to the Choice Program (which means that the veteran wants to receive care through the Choice Program and have the TPA proceed with appointment scheduling). If the veteran is not reached by telephone, the TPA has 10 business days for the veteran to respond to a letter confirming that they want to opt in, at which point the TPA creates the Choice Program authorization. If the authorization is created after the veteran’s preferred date or after the clinically indicated date on the VAMC’s referral has already passed, the TPA has 30 calendar days from the authorization creation date for an appointment for routine care to take place. The TPA can use up to 15 business days of this 30- calendar-day time frame to contact providers and successfully schedule the veteran’s Choice Program appointment. See figure 3 for an illustration of the potential wait time of approximately 70 calendar days for time-eligible veterans to receive routine care through the Choice Program. The process VA established for time-eligible veterans to receive routine care through the Choice Program—which could potentially take 70 days to complete—is not consistent with the requirement that veterans receive care within 30 days of their clinically indicated dates (where available) as applicable under the Choice Act. Furthermore, according to the federal internal control standard for control activities, agencies should design control activities—such as through policies and procedures—that will help ensure federal programs meet their objectives and respond to any risks to meeting those objectives. A key reason that veterans’ overall wait times for Choice Program care could potentially exceed the Choice Act’s 30-day wait-time requirement is that the process VA and VHA designed did not include a limit on the number of days VAMCs have to complete a key step of the process— compiling relevant clinical information and sending referrals to the TPAs after veterans have agreed to be referred to the Choice Program. While the process sets forth time frames for the other steps VAMCs and TPAs must complete to process referrals and schedule appointments, VA and VHA have not specified how many days VAMCs have to send veterans’ Choice Program referrals to the TPAs. VHA has no comprehensive policy directive for the Choice Program, and neither its consult management directive nor its outpatient appointment scheduling directive specifies an amount of time within which VAMCs should prepare Choice Program referrals. Another reason that veterans’ overall wait times for Choice Program care could potentially exceed the Choice Act’s 30-day wait-time requirement is that after VA and VHA implemented their policies, they did not review and address risks that were identified through their actual experience in operating the program. In response to a letter we sent in March 2017, VA’s Deputy General Counsel for Legal Policy said that, based on VA’s and VHA’s experiences with actual operation of the Choice Program since November 2014, “the practical reality” has been that the 30-day wait-time goal VA established just prior to the program’s implementation cannot always be met. VA has not disclosed what timeliness goals it would apply under a future consolidated community care program. We note, however, that VA currently has no timeliness goals for its existing individually authorized community care program and cannot determine the amount of time veterans wait, on average, to receive care through that program, which has accounted for a significant portion of veterans’ community care utilization. We recommended in May 2013 that VA analyze the amount of time veterans wait to see providers in its individually authorized community care program and apply the same wait-time goals to that care that it uses to monitor wait times at VHA medical facilities. VA concurred with the recommendation to conduct an analysis and reported that it was in the process of building wait-time indicators to measure wait-time performance for individually authorized VA community care. VHA has since updated its wait-time goal for care delivered within VHA medical facilities—which is that care must be delivered within 30 days of veterans’ clinically indicated dates (where available). However, VA has not applied that same goal to its individually authorized VA community care program nor begun measuring wait-time performance for that program. Timeliness of appointments is an essential component of quality health care; delays in care have been shown to negatively affect patients’ morbidity, mortality, and quality of life. Without specifying wait-time goals that are achievable, and without designing appointment scheduling processes that are consistent with those goals, VA lacks assurance that veterans are receiving care from community providers in a timely manner. It also lacks a means for comparing the timeliness of veterans’ community care with that of care delivered within VHA medical facilities. To examine selected veterans’ actual wait times to receive routine care and urgent care through the Choice Program, we conducted a manual review of a random, non-generalizable sample of 196 Choice Program authorizations. The TPAs created these authorizations between January 2016 and April 2016 in response to referrals sent by six selected VAMCs. Our manual review of veterans’ VA electronic health records and the TPAs’ records for our non-generalizable sample of 55 routine care authorizations and 53 urgent care authorizations for which the TPAs succeeded in scheduling appointments identified the following review times: For the 55 routine care authorizations in our sample, it took VAMC staff an average of 24 calendar days after the veterans’ need for routine care was identified to contact the veterans and confirm that they wanted to be referred to the Choice Program, compile relevant clinical information, and send veterans’ referrals to the TPAs. It took an average of 27 calendar days for the VAMCs to complete these actions for the 53 urgent care authorizations in our sample. For these routine care authorizations, it took the TPAs an average of 14 calendar days to accept referrals and reach veterans by telephone or letter for the veterans to opt in to the Choice Program. It took the TPAs an average of 18 calendar days to complete these actions for the urgent care authorizations in our sample. After the TPAs succeeded in scheduling veterans’ appointments for routine care, an average of 26 calendar days elapsed before veterans in our sample completed their initial appointments with Choice Program providers. For urgent care authorizations in our sample, it took an average of 18 days for the veterans to complete their initial appointments after the TPAs scheduled them. See the following text box for specific examples of the overall wait times experienced by some veterans in the samples of routine and urgent Choice Program authorizations we reviewed. Examples of Delays Experienced by Veterans for Whom the Choice Program Third Party Administrators (TPA) Scheduled Appointments One veteran was referred to the Choice Program for magnetic resonance imaging (MRI) of the neck and lower back because these services were unavailable at a Veterans Health Administration (VHA) medical facility. It took almost 3 weeks for Department of Veterans Affairs (VA) medical center (VAMC) staff to prepare his Choice Program referral for routine care and send it to the TPA, and then it took an additional 2 months after the VAMC sent the referral for the veteran to receive care. Notes in the veteran’s VA electronic health record indicated that his follow- up appointment with a VHA neurosurgeon was at risk of being rescheduled because the VAMC had not received the results of the MRI after the appointment with the Choice Program provider occurred. Ultimately, the veteran’s appointment with the VHA neurosurgeon—where the imaging results and treatment options were discussed—did not occur until almost 6 months after the VHA clinician originally identified the need for the MRI. One veteran was referred to the Choice Program because she needed maternity care, which is generally not available at VHA medical facilities. Almost a month and a half elapsed from the time VAMC staff confirmed her pregnancy (when she was 6 weeks pregnant) to when the VAMC sent the Choice Program referral for urgent care to the TPA. It then took 2 additional weeks for the TPA to make an unsuccessful attempt to contact the veteran to schedule a prenatal appointment; by that point, she was almost 15 weeks pregnant. The veteran called the TPA back, but when the TPA had yet to schedule an appointment by the time she was 18 weeks pregnant, the veteran finally scheduled her initial prenatal appointment herself, almost 3 months after her pregnancy was confirmed by VAMC staff. One veteran was referred to the Choice Program for thoracic surgery to address a growth on his lung because there was a wait for care at a VHA medical facility. TPA documentation we reviewed indicated that VAMC staff contacted the TPA four times to inquire about the status of the veteran’s appointment, and the TPA contacted five Choice Program providers in its unsuccessful attempts to schedule the urgent appointment for the veteran. Ultimately, the veteran scheduled his own initial appointment with a thoracic surgeon in the community and informed the TPA that he had done so. The veteran’s initial appointment occurred 3 weeks after the VAMC sent his referral to the TPA. We also found that veterans in our sample experienced lengthy overall wait times to receive care when the TPAs returned their authorizations to the VAMC without scheduling appointments. When veterans’ Choice Program authorizations are returned, VAMCs must attempt to arrange care through other means—such as through another VA community care program, a new Choice Program referral, or at another VHA medical facility. Among the 88 returned authorizations in our sample, we determined that 53 veterans eventually received care through other means after their authorizations were returned. These 53 veterans ended up waiting an average of 111 days after the VHA clinician originally determined they needed care until their first appointment with a VHA clinician or with a community provider occurred. See the text box below for some examples of delays experienced by veterans in the sample of 88 returned Choice Program authorizations we reviewed. Examples of Delays Experienced by Veterans Whose Authorizations Were Returned to Department of Veterans Affairs Medical Centers (VAMC) by the Choice Program Third Party Administrators (TPA) The VAMC took almost 3 ½ months to refer one veteran to a physical therapist to address her pelvic floor prolapse. When the preferred provider listed in the VAMC’s referral was outside the TPA’s network, the TPA sent a message to the VAMC via its web-based portal to ask if it should try scheduling the appointment with a different provider. By the time VAMC staff responded to the message in the TPA’s portal, the TPA had already returned the authorization—almost 2 weeks after accepting it. Two months later, the VAMC realized that the veteran still needed this care and sent a new Choice Program referral to the TPA. It then took the veteran another 2 ½ months to attend her first appointment. Overall, this veteran waited more than 8 months to receive physical therapy. One veteran who was eligible for the Choice Program because he resided more than 40 miles from a VHA medical facility contacted the TPA to request an appointment with a urologist. More than a month later, the TPA contacted the VAMC via its web-based portal to request a referral for the veteran. VAMC staff responded to the TPA two days later and stated (correctly) that because the veteran was distance-eligible, no referral was required. Four days after receiving the VAMC’s response, the TPA succeeded in scheduling an appointment. However, the veteran declined it because the TPA had scheduled the appointment with a neurologist (a specialist who treats conditions affecting the brain, spinal cord, and nerves) rather than a urologist (a specialist who treats conditions affecting the urinary tract and male reproductive organs). Ultimately, the veteran ended up seeing a urologist at a VAMC nearly 5 months after he originally contacted the TPA to request care. It took about 2 ½ weeks for the VAMC to send one veteran’s referral for pain management to the TPA after a VHA clinician originally determined he needed these services. However, information the TPA needed for scheduling the Choice Program appointment was missing from the VAMC’s referral. The TPA requested the information from the VAMC twice using its web-based portal, but VAMC staff did not reply, and the TPA returned the authorization 2 weeks after receiving it. It then took another month before the veteran ended up receiving pain management services at a VAMC. Overall, this veteran waited almost 2 ½ months for pain management services. After we shared the results of our preliminary analysis with VHA officials in December 2016, VHA required its medical facilities to manually review a sample of about 5,000 Choice Program authorizations that were created in July, August, and September of 2016 for four types of Choice Program care—mammography, gastroenterology, cardiology, and neurology. The purpose of this review was to analyze (1) the timeliness with which VAMCs sent referrals to the TPAs, and (2) veterans’ overall wait times for Choice Program care. VHA calculated the average wait times across these four types of care for each of its 18 Veterans Integrated Service Networks (VISN). VHA’s analysis of data collected by VAMCs identified the following average review times when veterans were referred to the Choice Program because there was a greater-than-30-day wait time for an appointment at a VHA medical facility. Referral wait times. VISN-level averages ranged from 6 to 53 days for VAMC staff to contact veterans and confirm that they wanted to be referred to the Choice Program, compile relevant clinical information, and send veterans’ referrals to the TPAs. The national average was 19 days. Overall wait times. From the time veterans’ need for care was identified until they attended initial Choice Program appointments, average overall wait times ranged from 34 to 91 days across VHA’s 18 VISNs. The national average was 51 days. When veterans were referred to the Choice Program because services were unavailable at a VHA medical facility, VHA’s analysis of VAMCs’ self-reported data identified the following average review times: Referral wait times. VISN-level averages ranged from 6 to 21 days for VAMC staff to contact veterans and confirm that they wanted to be referred to the Choice Program, compile relevant clinical information, and send veterans’ referrals to the TPAs. The national average was 15 days. Overall wait times. From the time veterans’ need for care was identified until they attended initial Choice Program appointments, average overall wait times ranged from 39 to 56 days across VHA’s 18 VISNs. The national average was 47 days. Our analysis indicates that VHA’s ability to monitor Choice Program access is limited because the data VHA uses are not always accurate and reliable, and VHA lacks certain data that are needed to effectively monitor the program. As discussed below, multiple factors contribute to these data limitations. According to federal internal control standards for information and communication and for monitoring, agencies should use quality information to achieve the entity’s objectives, internally and externally communicate quality information, and establish activities to monitor the quality of performance over time and evaluate the results. Without complete, reliable Choice Program data, VHA cannot determine whether the Choice Program has achieved the goals of (1) alleviating the wait times veterans have experienced when seeking care at VHA medical facilities, and (2) easing geographic burdens veterans may face to access care at VHA medical facilities. The data VHA currently uses to monitor the timeliness of Choice Program appointment scheduling and completion do not capture the days it takes for VAMCs to prepare veterans’ referrals and send them to the TPAs. This is because VHA has not standardized the manner in which VHA clinicians and VAMC staff categorize consults that lead to Choice Program referrals. We observed inconsistency in the titles of consults that were associated with the non-generalizable sample of Choice Program authorizations we reviewed. For example, consult titles sometimes included the word “Choice,” but in other cases they included the words “non-VA care.” Some of the consult titles indicated the criterion under which the veteran was eligible for the Choice Program and the type of care the veteran needed (for example, “Choice-First Physical Therapy”), while other consult titles only indicated the type of care the veteran needed (for example, “pain management”). We observed this variability among consult titles both within single VAMCs and across all six of the VAMCs we selected for review. According to documentation VHA officials provided to us in December 2016, they planned on implementing a process for standardizing the consult titles associated with Choice Program referrals over the course of calendar year 2017. Originally, they planned on piloting the process at four VAMCs beginning in February 2017 and expected to gradually roll out standardized consult titles across all other VAMCs over the remainder of calendar year 2017. However, in late June and early July 2017, we followed up with the six VAMCs in our sample, and at that time, managers from only one of the VAMCs said that they had implemented the new process for standardizing consult titles associated with Choice Program referrals. When we interviewed VHA officials again in September 2017, they acknowledged that they had been delayed in implementing standardized consult titles, and they provided documentation indicating that they were just beginning to roll out the new process nationwide. In the absence of standardized consult titles for the Choice Program, VHA has no automated way to electronically extract data from VA’s electronic health record and calculate the average number of days it takes for VAMC staff to prepare veterans’ Choice Program referrals after veterans have agreed to be referred to the program. Further, without standardized consult titles, VHA cannot monitor veterans’ overall wait times—from the time VHA clinicians determine veterans need care until the veterans attend their first appointments with Choice Program providers. The lack of standardized consult titles also prevents VHA from tracking average overall wait times and monitoring the timeliness of care for veterans whose Choice Program authorizations are returned by the TPAs without scheduled appointments. The data VHA currently uses to monitor the timeliness of Choice Program appointments capture only a portion of the process that the TPAs carry out to schedule veterans’ appointments after they receive referrals from VAMCs. Specifically, VHA’s data reflect the timeliness of appointment scheduling and completion after the TPAs create authorizations in their appointment scheduling systems, which (according to VA’s contracts, as of June 1, 2016) the TPAs must do only after they have received all necessary information from VA and the veteran has opted in to the Choice Program. Therefore, VHA’s timeliness data do not capture the time TPAs spend (1) reviewing and accepting VAMCs’ referrals, and (2) contacting veterans to confirm that they want to opt in to the Choice Program. Data related to the timeliness of Choice Program appointment scheduling. When we asked how they monitor the timeliness of Choice Program appointment scheduling, VHA officials provided us the following types of data, all of which reflect the time that elapses only after veterans have opted in to the Choice Program and the TPAs have created authorizations: the average number of business days the TPAs take after creating authorizations to schedule appointments for routine and urgent care, the percentage of appointments for routine care that the TPAs schedule within 5 business days after they create authorizations, and the percentage of appointments for urgent care that the TPAs schedule within 2 business days after they create authorizations. Data related to the timeliness with which initial Choice Program appointments occur. VHA officials provided us data on the timeliness with which initial Choice Program appointments have occurred; however, as shown below, almost all of these data reflect the timeliness with which appointments occur only after veterans have opted in to the Choice Program and the TPAs have either created authorizations or successfully scheduled veterans’ appointments: the average number of business days after the TPAs create authorizations in which appointments for routine and urgent care occur; the percentage of appointments for routine care that are completed within 30, 60, 90, and 120 business days or more after the TPAs create an authorization; the percentage of appointments for routine care that are completed within 30 calendar days of either (1) the TPA’s scheduling of the appointment, (2) the clinically indicated date on the VAMC’s referral, or (3) the veteran’s preferred date; and the percentage of appointments for urgent care that are completed within 2 calendar days of the TPAs creating the authorizations. See figure 4 for an illustration of how VHA’s data capture only a portion of the Choice Program process to obtain care. In September 2017, VHA officials told us that they recently began implementing an interim solution that would allow them to track veterans’ overall wait times for Choice Program and other VA community care— from the time VHA clinicians determine veterans need the care until the veterans attend their first appointments with community providers. Specifically, this interim solution requires VAMC staff to enter unique identification numbers on VHA clinicians’ requests for care and on the Choice Program referrals they send to the TPAs. This unique identification number is then carried over to the Choice Program authorizations that are created in the TPAs’ systems. According to VHA officials, the unique identification number creates a link between VHA’s data and the TPAs’ data, so that VHA can monitor the timeliness of each step of the Choice Program referral and appointment scheduling process. However, the success of VHA’s interim solution relies on VAMC staff consistently and accurately entering the unique identification numbers on both the VHA clinicians’ requests for care and on Choice Program referrals, a process that is prone to error. VHA officials said it is their long- term goal to automate the process by which VHA’s data are linked with TPAs’ data in the consolidated community care program they are planning to implement. Because, as previously explained, VHA lacks data on the average timeliness with which VAMCs prepare Choice Program referrals, and VHA also lacks data on the average amount of time that elapses between when the TPAs receive VAMCs’ referrals and when veterans opt in with the TPAs, VHA cannot track veterans’ overall wait times for Choice Program care—from the time VHA clinicians determine that veterans need care until the veterans attend their first appointments with Choice Program providers. In addition, the lack of data on the timeliness with which the TPAs have (1) accepted VAMCs’ referrals and (2) determined that veterans wish to opt in to the program also prevents VHA from assessing whether the TPAs’ average timeliness in completing these actions has improved over time. Our analysis of a sample of 196 Choice Program authorizations shows that another way in which VHA’s monitoring of veterans’ access to care is limited by available data is that the clinically indicated dates included on referrals that VAMCs send to the TPAs may not be accurate. We found that the clinically indicated dates on VAMCs’ referrals were not always identical to the clinically indicated dates that were originally entered into VA’s electronic health record by the VHA clinicians who treated the veterans. VHA’s policy directive on consult management and its Choice Program standard operating procedure for VAMCs state that the clinically indicated date is to be determined by the VHA clinician who is treating the veteran. However, in reviewing VA’s electronic health records for our sample of 196 Choice Program authorizations, we identified 60 cases where the clinically indicated dates VAMC staff entered on Choice Program referrals they sent to the TPAs differed from the clinically indicated dates that were originally entered by VHA clinicians. In 46 of these 60 cases, VAMC staff entered clinically indicated dates on the Choice Program referrals that were later than the dates originally determined by the VHA clinicians, which would make the veterans’ wait times appear to be shorter than they actually were. VHA could not explain why the dates differed. Clinically indicated dates are manually entered on VAMCs’ electronic referrals to the TPAs, a practice that is subject to error or manipulation. It is unclear if VAMC staff mistakenly entered incorrect dates, or if they inappropriately entered later dates when the VAMC was delayed in contacting the veteran, compiling relevant clinical information, and sending the referral to the TPA. If VAMCs’ Choice Program referrals have clinically indicated dates that are different from the ones VHA clinicians originally entered without additional supporting documentation, there is a risk that VHA’s data will not accurately reflect veterans’ actual wait times. Specifically, VHA will not be able to determine how often veterans receive Choice Program care within the Choice Act’s required 30-day time frame. Another limitation of VHA’s monitoring of veterans’ access to Choice Program care is that VAMCs and TPAs do not always categorize referrals in accordance with the contractual definition for urgent care when they are processing referrals and scheduling appointments for veterans. According to VA’s contracts with the TPAs, Choice Program referrals are to be marked as “urgent” when a VHA clinician has determined that the veteran needs care that (1) is considered essential to evaluate and stabilize conditions and (2) if not provided would likely result in unacceptable morbidity or pain when there is a significant delay in evaluation or treatment. It is VA’s goal that the TPAs schedule appointments for urgent care and ensure that they take place within 2 business days of accepting the referrals from VA. Among the sample of 53 Choice Program authorizations for urgent care we reviewed, VHA and TPA documentation showed that in 35 cases (about 66 percent), VHA clinicians originally determined that veterans needed routine care, but VAMC or TPA staff later re-categorized the referrals or authorizations as urgent. In 4 of these 35 cases, we found documentation showing that VHA clinicians had reviewed the pending referrals and determined that the veterans’ clinical conditions or diagnoses warranted re-categorizing the veterans’ routine care referrals or authorizations as urgent. In 31 other cases we reviewed, however, we found no documentation indicating that a VHA clinician had identified a clinical reason for the veteran to receive care faster. In at least 15 of these 31 cases, it appeared that the VAMC or TPA staff changed the status of the referral or authorization in an effort to administratively expedite appointment scheduling when they were delayed in sending referrals and scheduling veterans’ Choice Program appointments. According to the VA contracting officer who is responsible for the Choice Program contracts, VA’s contracts with the TPAs do not include provisions for separating clinically urgent Choice Program referrals and authorizations from those that the VAMC or the TPA has decided to expedite for administrative reasons (such as when the veteran or VAMC staff has expressed frustration with a delay in the referral or appointment scheduling process). If Choice Program referrals for routine care are inappropriately categorized as urgent care referrals under the Choice Program, VHA’s data on the timeliness of urgent appointment scheduling and completion will not accurately reflect the extent to which veterans who have a clinical need for urgent care are receiving it within the time frames required by the TPAs’ contracts. The authorization creation date is the primary starting point from which VHA monitors the TPAs’ timeliness in appointment scheduling and the extent to which veterans’ initial Choice Program appointments occur in a timely manner. However, when initially implementing the Choice Program—beginning in November 2014—the two TPAs had differing interpretations of contractual requirements relating to when they should create authorizations in their appointment scheduling systems. According to VA contracting officials and VHA community care officials we interviewed, at the start of the program, one of the TPAs was creating authorizations as soon as it accepted referrals from VAMCs, but the other was waiting until after veterans opted in to the Choice Program to create authorizations. It was not until May 2016 (about 18 months into the Choice Program’s implementation) that VA modified its contracts to clarify that the TPAs are to create Choice Program authorizations only after they have contacted the veterans and confirmed that they want to opt in to the program. Due to these differing interpretations, VA lacked comparable performance data for the two TPAs for the first 18 months of the Choice Program’s expected three-year implementation. Therefore, it could not compare the timeliness of access nationwide. In addition, since VA modified the TPAs’ contracts midway through the Choice Program’s implementation, officials can only comparatively examine whether the timeliness of both TPAs’ appointment scheduling and completion has improved since June 2016, which is when the relevant contract modification took effect. VHA collects data and monitors various reasons the TPAs return Choice Program authorizations to VAMCs without making appointments. Each month, VA monitors how each TPA performs on Choice Program performance measures related to the timeliness of appointment scheduling. Authorizations that are returned for reasons that are attributable to the TPA—such as a lack of network providers in close proximity to the veteran’s residence—negatively impact the TPAs’ monthly performance measures. In our sample, we found that VHA’s data on the TPAs’ reasons for returning Choice Program authorizations are not reliable. Specifically, we questioned the validity of the TPAs’ return of 20 out of the 88 authorizations in our sample, for the following reasons: In 11 of the 20 cases, we found VHA or TPA documentation that substantiated the return, but the TPAs selected the incorrect return reasons when they sent the authorizations back to VA. For example, in one case, the TPA was unable to schedule an appointment with a primary care provider—even after contacting 11 different network providers—but the TPA staff returned the authorization to the VAMC indicating that the veteran had declined care. TPA officials who reviewed this authorization with us agreed that it was inappropriate to mark this authorization as having been returned because the veteran declined care and that their staff instead should have indicated that they had been unable to schedule an appointment with a network provider. In the remaining 9 of the 20 cases, we could find no VHA or TPA documentation to substantiate the reasons the TPAs selected when they returned the authorizations to VA, nor any other reasons for return. For example, the TPAs incorrectly selected “missing VA data” as the reason they returned 5 of these 9 authorizations. Based on VHA and TPA documentation we reviewed, the VAMCs’ referrals were complete and not missing any of the information the TPAs needed to proceed with appointment scheduling. TPA officials could not explain why their staff selected incorrect return reasons or inappropriately returned authorizations for which they should have kept attempting to schedule appointments. However, TPA staff must manually select return reasons when they send authorizations back to VAMCs, a process that is subject to error or manipulation. There is a process by which VA’s contracting officer’s representatives validate the monthly data submitted by the TPAs, but it cannot identify the data reliability issues we found when manually reviewing VHA and TPA documentation associated with a sample of returned Choice Program authorizations. VHA officials told us that VA’s contracting officer’s representatives do not have access to veterans’ electronic health records, which means that they cannot check whether VHA documentation substantiates the return reasons selected by the TPAs. Without reliable data on reasons that veterans have been unable to obtain appointments through the Choice Program, VHA cannot properly target its efforts to address challenges—such as network inadequacy— that may be causing the TPAs to return authorizations without making appointments. In addition, the lack of reliable data makes it difficult for VA to monitor whether the TPAs are meeting their contractual obligations, such as establishing adequate networks of community providers. Another way in which VHA’s monitoring of veterans’ access is limited is that VA lacks contract performance measures that would provide VA and VHA with data related to veterans’ driving times to access care from the TPAs’ Choice Program network providers. Such performance measures would help VA monitor the TPAs’ network adequacy. In contrast, for PC3, VA does collect data from the TPAs to monitor urban, rural, and highly rural veterans’ maximum commute times to specialty care providers, providers of higher level care, primary care providers, and mammography and maternity care providers. When we asked why VA had not established driving time performance measures for the Choice Program, a VHA official responsible for monitoring the Choice Program contracts told us he thought that these performance measures had simply been overlooked in the haste to implement the Choice Program. VA concurred with a recommendation we made in our December 2016 report about VA health care for women veterans, in which (among other things) we stated that the department should monitor women veterans’ driving times to access sex-specific care through the Choice Program and VA’s future community care contracts. However, VA stated in its June and October 2017 written updates on actions it has taken to address this recommendation that it does not intend to modify the current Choice Program contracts to address our recommendation because the contracts will be ending soon and it would be too costly to do so. Without driving time performance measures for the Choice Program, VHA lacks assurance that the TPAs’ networks include a sufficient number of community providers in close proximity to where veterans live, and it cannot monitor the extent to which veterans’ geographic access to care has improved or diminished. Officials we interviewed from VA’s contracting office, VHA’s Office of Community Care, and both of the TPAs, along with leadership officials, managers, and staff from the six selected VAMCs told us about various factors that have directly or indirectly affected veterans’ access to care throughout the Choice Program’s implementation. Chief among these are (1) administrative burden associated with the Choice Program’s complex referral and appointment scheduling processes; (2) inadequate VAMC staffing and poor communication between VHA and the VAMCs; and (3) the TPAs’ slow development of a robust provider network. We also identified actions VA and VHA have taken to address these factors. (See appendix VI for additional information about actions that VA and VHA took to address these three access-related issues for the Choice Program.) To the extent that these factors persist under the consolidated community care program that VA plans to establish, they will continue to adversely affect veterans’ access to care. VHA and TPA officials, as well as managers and staff from the six selected VAMCs, told us they encountered administrative burden associated with the complexities of the Choice Program’s referral and appointment scheduling processes. Further, they lacked care coordination tools throughout the time they were operating the Choice Program, which affected their ability to provide timely care to veterans. Among the main issues cited were the following: Manual referral processes and lack of TPA access to veterans’ records. To prepare veterans’ Choice Program referrals, VAMC staff had to follow a manual, time-consuming process to retrieve and collate key contact and clinical information from veterans’ VA electronic health records. This was because—throughout most of the Choice Program’s implementation—VA had no system for automatically generating referral packages that contained all of this information; nor did TPA staff have access to veterans’ VA electronic health records. If VAMC staff made mistakes (such as mistyping or inadvertently omitting veterans’ telephone numbers or addresses) or if the referrals were missing clinical information that the TPAs needed for appointment scheduling purposes, the TPAs had to either contact the VAMC to correct or obtain the missing information or return the referrals to VA without attempting to schedule appointments. These manual processes impeded the VAMCs’ progress in preparing referrals and the TPAs’ progress in scheduling veterans’ Choice Program appointments. Limited availability of care coordination tools and dependence on telephone-based customer service for appointment scheduling. A lack of care coordination tools and near-constant telephone calls also delayed VAMC and TPA staff from efficiently processing veterans’ referrals for appointments. For example, the Choice Program had no web-based portal through which VAMC staff and veterans could view the TPAs’ step-by-step progress in scheduling appointments. While both of the TPAs had portals that allowed VAMC staff (but not veterans) to obtain certain information— such as whether the TPA had already scheduled an appointment—the portals did not show if, or when, veterans’ referrals had been accepted, the dates and times of the TPAs’ attempts to contact veterans, or the number of community providers the TPA had contacted in its attempts to schedule an appointment. VAMC staff we interviewed said that while they could submit written messages to the TPAs through the portals, TPA staff did not always answer these messages in a timely manner. This, in turn, made telephone calls between veterans, the VAMCs, and the TPAs the most effective form of follow-up regarding veterans’ Choice Program referrals, according to VAMC managers and staff. Officials from one selected VAMC estimated that their community care staff (which included about 30 employees) was answering approximately 10,000 calls per month, and another VAMC had hired a full-time staff person just to answer telephone calls. Workload associated with re-authorizing veterans’ care. VAMC and TPA staff also told us they faced a lengthy administrative process to re-authorize care if veterans’ Choice Program authorizations expired before veterans received care or if veterans needed services that were outside the scope of their original authorizations. The TPAs referred to these as “secondary authorization requests” or “requests for additional services.” Without these re-authorizations, veterans’ care from community providers could be delayed or interrupted. VAMC and TPA staff had to process a high volume of these requests for two main reasons. First, the Choice Program originally had a 60- day limit on episodes of care, which meant that all appointments within the episode of care had to be completed within 60 days of the initial date of service. Even if the veteran needed care that could routinely be expected to outlast this 60-day time frame (such as maternity care or cancer treatment), community providers and the TPAs would still have to request additional referrals from the VAMCs to authorize the remaining care. Second, TPAs would have to request additional referrals if an episode of Choice Program care was already in progress and the veteran needed services that were not specifically authorized in the VAMC’s original referral. According to some VAMC managers and staff, this generated significant workload for the VAMCs. Officials from one of the selected VAMCs said it had to hire a full-time nurse just to process secondary authorization requests. Manual post-appointment follow-up processes. According to VAMC managers and staff we interviewed, the manual processes used for post-appointment follow-up also added to delays for veterans seeking care through the Choice Program. After an episode of care is complete—whether services are delivered at a VHA medical facility or in the community—VHA’s policy requires VAMC staff to document that care was provided and make the results of encounters available to VHA clinicians by entering medical records or other clinical information into the veteran’s VA electronic health record. When medical records from the community provider became available, VAMC staff had to retrieve copies from the TPAs’ portals and scan them into veterans’ VA electronic health records. (See appendix V for an illustration of this process.) VAMC staff described this as a very time-consuming process because it could take months for claims or medical records from Choice Program appointments to appear in the TPAs’ portals. At the time of our interviews in the summer of 2016, managers from two of the VAMCs in our sample said they each had backlogs of more than 6,000 Choice Program and other community care consults to complete. These backlogs adversely affected veterans’ access to Choice Program care because the time VAMC staff spent attempting to complete some veterans’ consults could not be spent on preparing other veterans’ Choice Program referrals. Over the course of the Choice Program’s implementation, VA and VHA took multiple actions to address administrative burden, including the following. Opportunities exist to improve or build on these actions as VA moves forward with the consolidated community care program it plans to implement. Implementation of a web-based tool to automate Choice Program referral preparation. In early 2016, to improve the process of gathering information from veterans’ VA electronic health records to prepare Choice Program referrals, staff from two VAMCs developed a web-based tool—called the “referral documentation” (REFDOC) tool. According to VHA documentation, the REFDOC tool automates the process of gathering necessary information and assembling it in a standardized format for veterans’ Choice Program referrals. VHA’s initial analyses of the REFDOC tool’s effectiveness found that it sped up the process of preparing Choice Program referrals by about 20 minutes per referral, which helped reduce the administrative burden associated with preparing referrals. However, VHA’s nationwide dissemination of the tool to all of the VAMCs was slowed by limitations of VA’s information technology systems. As of November 2016 (about 9 months after the tool was created), it had only been implemented at 18 of VHA’s 170 VAMCs. VHA gradually made the tool available at the remaining VAMCs between November 2016 and May 2017. Standardized episodes of care. In April 2017, VHA approved standardized episodes of care—or “bundles” of clinically necessary medical services and procedures—that are to be authorized whenever veterans are referred to community providers for specified types of care. This was intended to help address administrative burden associated with clinical review processes and improve veterans’ access to care. To start, VHA approved standardized episodes of care for 15 different types of care, including physical therapy, maternity care, and optometry. VA and VHA documentation indicate that they intend to roll out additional standardized episodes of care over time and continue using them once VA transitions to the consolidated community care program it is planning to implement. Acquisition of a secure e-mail system and a mechanism for TPAs and community providers to remotely access veterans’ VA electronic health records. VA recently established two different care coordination tools that were intended to make the process of providing veterans’ medical records to Choice Program and other VA community care providers more efficient. Secure e-mail system. In the spring of 2017, VA acquired software that allows VAMC managers and staff to e-mail encrypted files containing veterans’ medical records to the TPAs and community providers. Only the intended recipient can decrypt and respond to messages containing the files. According to VHA documentation, this secure e-mail system was intended to improve the efficiency of coordinating veterans’ Choice Program care and address potential security risks associated with printing paper copies of veterans’ medical records and sending them to the TPAs or community providers via fax or U.S. mail. Remote access to veterans’ VA electronic health records. In May 2017, VHA began offering a secure, web-based application called the Community Viewer as a tool for community providers nationwide to have access to assigned veterans’ VA electronic health records. Like the secure e-mail system, this tool is intended to improve the efficiency of coordinating veterans’ Choice Program care. However, VHA’s ability to seamlessly coordinate care with community providers remains limited—even with the secure e-mail system and the Community Viewer—because these tools only facilitate a one-way transfer of the information needed to coordinate the care veterans receive at VHA medical facilities and in the community. For the purposes of care coordination, it is important that information sharing among all participants concerned with a veteran’s Choice Program or other VA community care—including VHA clinicians, the TPAs, community providers, and the veteran—is as seamless as possible. According to the federal internal control standard for information and communication, agencies should internally and externally communicate the necessary information to achieve their objectives. While the secure e-mail system and Community Viewer tool provide an interim solution for VAMCs to transfer information from veterans’ VA electronic health records to the TPAs and community providers, they do not provide a means by which VAMCs or veterans can (1) view step-by-step progress in scheduling appointments, or (2) electronically receive the clinical results of Choice Program or other VA community care encounters. Building such a capability into the future consolidated community care program that VA plans to implement would allow VHA to improve the care coordination processes that exist in the Choice Program. Pilot programs for VAMC staff to schedule Choice Program appointments. In July 2016 and October 2016, VHA began implementing pilot projects, whereby staff at two VAMCs took over from the TPAs the responsibility of scheduling veterans’ Choice Program appointments. Specifically, VA modified its contracts with TriWest and Health Net to implement the two VAMC scheduling pilots at the Alaska VA Health Care System and the Fargo VA Health Care System, respectively. In these two locations, VAMC staff schedule veterans’ appointments and send relevant clinical documentation to the Choice Program providers. According to VHA officials, this had the potential to improve veterans’ access to care by improving the efficiency of the Choice Program appointment scheduling process. The results of these two VAMC scheduling pilots are particularly relevant, given that VA’s RFP, as amended, for its planned consolidated community care program indicates that VAMCs—rather than TPAs—will carry out community care appointment scheduling, unless VA exercises a contract option for the TPAs to provide such services for VAMCs that request them. However, while VHA officials told us that while they have taken some steps to begin evaluating the effectiveness of the pilots in improving appointment scheduling, these efforts have not been completed. The lack of an evaluation of the two VAMC scheduling pilots is inconsistent with the federal internal control standard for risk assessment, which stipulates that an agency should identify, analyze, and respond to risks related to achieving defined objectives. In addition, the federal internal control standard for monitoring calls for ongoing monitoring to assess the effectiveness of management strategies, make needed corrections if shortcomings are identified, and determine if corrective actions are achieving desired outcomes. Without evaluating the results of the scheduling pilots at the Alaska and Fargo VA Health Care Systems, VA lacks assurance that VAMC staff have the potential to schedule veterans’ community care appointments in a more timely manner than TPA staff otherwise would schedule them. Furthermore, VA is missing an opportunity to inform its planning and decisions for scheduling under its planned consolidated community care program. TPA officials and managers and staff from the six selected VAMCs frequently discussed staffing- and communication-related factors that adversely affected the timeliness of veterans’ Choice Program care. During the course of our review, they cited the following factors that delayed VAMCs’ processing of veterans’ referrals and TPAs’ scheduling of appointments: Staff vacancies and turnover. TPA officials and managers and staff at selected VAMCs said that VAMCs and TPAs were initially understaffed as Choice Program implementation began. VAMCs. Managers at the six selected VAMCs told us that after implementing the Choice Program, they hired additional community care staff, with one of them increasing its community care staffing level almost five-fold by July 2016. Some VAMC managers told us in 2016 and again in 2017 that they still struggled with staff retention and vacancies—among both managers and staff. Five of the VAMCs said they relied on overtime for their existing staff to keep up with the Choice Program workload. According to community care managers from four of the selected VAMCs, it takes about 6 months to recruit, hire, and train new community care staff, and this process could take more time if the VAMC’s human resources office is also understaffed, which was the case for at least one of the six VAMCs. That VAMC had not had a permanent community care manager for more than 2 years as of July 2017—which covered the majority of the Choice Program’s original 3-year implementation. TPAs. Officials from both TPAs also told us that they initially underestimated the workload associated with scheduling Choice Program appointments, and they brought on additional staff, including sub-contractors, to better manage their workloads as utilization of the program increased. One TPA opened eight operations centers in addition to the two it already had when the Choice Program was initially implemented. Ineffective mechanisms for VAMCs to resolve problems. VAMC managers and staff we interviewed also said they lacked useful mechanisms and points-of-contact when they needed to resolve issues and problems they were having with referral and appointment scheduling processes. VHA established a web-based Choice Program “issue tracker” system for VAMCs to report problems to VHA’s Office of Community Care. However, staff at four of the selected VAMCs told us they rarely used the tracker and some had stopped using the tracker altogether because it took too long for VHA’s Office of Community Care or the TPAs to respond and resolve the issues (if they responded at all), and they did not see the value in taking the time to report them via this mechanism. Managers at one of the VAMCs also told us about a phone line that their TPA had established to escalate and resolve urgent issues, but the TPA told the VAMC only to use it for emergencies. VHA’s untimely communication of Choice Program policy and process changes. According to managers and staff at the six selected VAMCs, VA and VHA have issued numerous contract modifications and policy changes with little advanced notice throughout the Choice Program’s implementation. According to these VAMC managers and staff, the untimely communication of changes created confusion at the VAMC level that affected veterans’ access to Choice Program care. We reviewed documentation showing that from October 2014 (when it modified the TPAs’ contracts to add responsibilities related to Choice Program administration) until July 2017, VA modified each TPA’s contract about 40 times. Many of these contract modifications—along with other legislative and regulatory changes that VA implemented during this period—changed VAMC or TPA processes related to Choice Program referrals and appointment scheduling. Many of the VAMC managers and staff we interviewed said they struggled to keep up with the contract modifications and policy changes, that VHA’s Office of Community Care did not always leave adequate time to prepare for them, and they felt they were never really able to become proficient at new processes before additional changes occurred. This meant that training sometimes happened after the contract modifications or VHA policy changes had already gone into effect. For example, managers and staff at three of the selected VAMCs told us that they were not informed in advance about a June 2016 contract modification that required the TPAs to return Choice Program authorizations to VAMCs if they failed to schedule appointments within required time frames. This contract modification had the potential to significantly increase VAMCs’ workloads, because they would have to arrange veterans’ care through other means once the authorizations were returned. According to individuals at two of these three VAMCs, they first heard about this change from TPA staff, rather than from VHA. VHA took the following two actions intended to help address staffing- related factors that adversely affected the timeliness of veterans’ Choice Program care. Staffing tool for VAMCs to estimate needs. In the spring of 2017, VHA developed a tool that is intended to help VAMCs project their staffing needs for the consolidated community care program VA plans to implement. VHA used workload data and site visit observations to develop the tool. Among the six selected VAMC managers we interviewed, impressions about the reasonableness of the staffing estimates generated by the community care staffing tool were mixed. For example, managers at two of the VAMCs said that the tool likely underestimated the number of staff they would need to handle referrals and appointment scheduling once VA transitions to the consolidated community care program. In contrast, managers from two other VAMCs thought that the tool’s staffing estimates seemed about right. Co-locating TPA staff at selected VAMCs to assist with resolution of problems. To help facilitate problem resolution between VAMCs and the TPAs as they work to schedule veterans’ Choice Program appointments, VA modified the TPAs’ contracts in November 2015 to allow for TPA staff to be co-located at selected VAMCs. VHA officials expected that one potential benefit of co- locating TPA staff would be that fewer veterans’ Choice Program referrals would be returned to VAMCs because of missing clinical information because TPA staff could help resolve such problems locally before the TPA returned referrals. As of May 2017, TPA staff were working at 70 of VHA’s 170 VAMCs—or about 40 percent of all VAMCs. Similar care coordination arrangements may exist under the consolidated community care program VA is planning to implement, if VA exercises a contract option for the TPAs to provide such services at VAMCs that request them. However, the communication-related factors that VHA and TPA officials identified as affecting the timeliness of veterans’ Choice Program care remain. VHA relied on ad hoc communications such as memoranda, fact sheets, e-mails, national conference calls, and occasional web-based trainings to communicate policy and process changes to VAMCs throughout the Choice Program’s implementation. Our interviews with VAMC managers and staff suggest that these were not the most effective methods of communication because messages about key changes sometimes lacked sufficient detail or failed to reach the VAMC staff responsible for implementing them in a timely manner. According to the federal internal control standard for control activities, agencies should implement control activities through their policies and procedures, which document the responsibilities of managers and staff who are responsible for implementing a program. Among other things, this may include management reviewing and updating policies and day-to-day procedures in a timely manner after a significant change in the program has occurred. VHA has no comprehensive policy directive or operations manual for the Choice Program, and its broader policy directive for VA community care programs has not been updated since January 2013. As a result, VAMC staff have operated in an environment that is frequently changing with no definitive reference source or sources of up-to-date policy and processes to consult, such as a comprehensive policy directive or operations manual. Instead, VAMC staff have had to keep track of the Choice Program’s policy and process changes through VHA’s various ad hoc communications. This poses a risk to VHA, as it increases the likelihood that VAMCs will implement new policies and processes inconsistently. In addition, there is risk that VAMC managers and staff will not always be aware of the most current policies and processes. Unless a comprehensive policy directive or operations manual is created, those risks could remain for the consolidated community care program VA is planning to establish. According to VAMC managers and TPA officials we interviewed, the TPAs’ inadequate networks of community providers affected both the timeliness with which veterans received Choice Program care and the extent to which veterans were able to access community providers located close to their homes. In September 2015, about 11 months after the Choice Program was implemented, VA contracting officials sent corrective action letters to both TPAs, citing network adequacy (i.e., the number, mix and geographic distribution of network providers) as a concern. TPA officials we interviewed acknowledged that their networks initially were not adequate to meet demand for Choice Program care. From the TPAs’ perspective, the brief transition period before the Choice Program began operations in November 2014 was not enough time to strengthen the community provider networks they had previously established under PC3, another VHA community care program. Furthermore, the TPAs told us that VA had not provided them with sufficient data on the expected demand for Choice Program care—by clinical specialty and zip code—prior to or after the Choice Program’s implementation. The overall number of community providers participating in the TPAs’ Choice Program networks nationwide grew dramatically over the following year—from almost 39,000 providers in September 2015 to more than 161,000 providers as of September 2016. However, at the time of our review, managers at five of the six selected VAMCs told us that they still observed TPA network inadequacies that impeded veterans’ access to Choice Program care. Similarly, managers at three VAMCs in our sample said that key community providers—including large academic medical centers—have refused to join the TPAs’ networks or dropped out of the networks after joining them, often because the TPAs had not paid them in a timely manner for the services they provided. Establishing adequate networks of Choice Program providers in rural areas has been particularly difficult. Officials at two of the three of the rural VAMCs in our sample pointed to general health care workforce shortages in rural areas as one cause for the TPAs’ network inadequacy—a challenge that is not limited to the Choice Program or VA’s health care system. According to a December 2015 analysis by VHA researchers, the majority of network providers in two of the three VISNs examined were located within 40 miles of VAMCs, leaving large geographic areas of these VISNs (particularly rural areas) outside the 40- mile radius with few network providers. For example, only 3.8 percent of primary care providers and 3.2 percent of behavioral health providers in VISN 20 (which covers Alaska, Idaho, Oregon, and Washington) were located more than 40 miles from VAMCs within that VISN. While the areas lacking network providers generally have fewer veterans relative to other areas within these VISNs, the analysis by VHA researchers suggests that veterans living in these areas are likely to have difficulty accessing Choice Program network providers that are located closer to their homes than the nearest VAMC, which is over 40 miles away. VA and VHA have tried to address network inadequacy that existed under the Choice Program and either have taken or plan to take additional actions to address this issue for the community care program VA plans to implement, including the following. Establishment of Choice Program provider agreement process. To help address inadequacies in the TPAs’ provider networks and improve veterans’ access to care under the Choice Program, VHA established the Choice Program provider agreement process in February 2016. This process allowed VAMCs to establish agreements with community providers, schedule veterans’ appointments, and reimburse the providers directly (using Choice Program funds) when the TPAs failed to schedule veterans’ appointments for reasons relating to network inadequacy, among others. Originally, the VAMCs were required to send veterans’ referrals to the TPAs and wait for them to be returned before they could proceed with arranging care through a Choice Program provider agreement. While this process had the potential to increase the availability of providers for the Choice Program, it did not immediately improve the timeliness of veterans’ Choice Program care because veterans still had to wait for as long as it took the VAMCs to send their referrals to the TPAs and for TPAs to return them before the VAMCs could proceed with arranging care through Choice Program provider agreements. According to the policies and contractual requirements that were in effect at the time, it could have taken up to 40 calendar days after a VHA clinician first identified the veteran’s need for care until the TPA returned the referral and the VAMC could proceed with arranging care through a Choice Program provider agreement. However, in March 2017, VHA updated the Choice Program provider agreement process so that—if the TPAs were returning a high volume of a VAMC’s referrals for one or more types of care—the VAMC could seek approval from its VISN and VHA’s Office of Community Care to bypass the TPA and proceed directly to arranging that type of care through Choice Program provider agreements. This had the potential to improve the timeliness of veterans’ access to Choice Program care because it eliminated the steps of sending referrals to the TPAs and waiting for them to be returned. Improving quality of information given to future TPAs. To help inform the recruitment of network providers for the consolidated community care program VA plans to establish, VA plans to provide future TPAs more robust data than they provided the current TPAs at the start of the Choice Program. In particular, VA’s RFP for the consolidated community care program, as amended, indicates that VA will provide (1) zip-code-level data on the number of authorizations that were issued in fiscal year 2015 for specific types of care (e.g., chemotherapy and obstetrics) and (2) VAMC-level data on the clinical specialties with the greatest wait times for appointments at VAMCs. These local-level data could help TPAs estimate the number of network providers of various specialties they will need to recruit in specific localities if awarded a contract for the consolidated community care program that VA is planning to implement. Performing market assessments. In preparation for the consolidated community care program VA plans to establish, VA and VHA officials are planning to conduct market assessments in 96 markets nationwide. Through these market assessments, officials told us, VA will (1) examine the clinical capacity that currently exists within VHA medical facilities and among community providers, (2) assess veterans’ current and future demand for health care services, and (3) develop long-term plans for ensuring that veterans will have access to high-quality health care services—whether they receive care from VHA clinicians or from community providers. According to VHA officials, the market assessments will help inform network provider recruitment efforts for the consolidated community care program VA is planning to implement. In addition, VHA officials told us that the market assessments will help VISN- and VAMC-level leaders make more informed, strategic decisions about whether it is more efficient to maintain or build capacity for delivering particular types of care within VHA medical facilities, or if they should routinely purchase certain types of care in the community. In November 2017, VHA officials told us that they expect to begin conducting the market assessments early in calendar year 2018, and the officials estimate that it will take about 18 months to complete assessments for all 96 markets. The Choice Program is approaching the end of its life, and with plans to consolidate it with VA’s other community care programs, opportunities to improve the program are diminishing. Congress created the Choice Program in 2014 in response to longstanding challenges in veterans’ access to care delivered within VHA medical facilities. However, we found numerous operational and oversight weaknesses with VHA’s management of scheduling veterans’ medical appointments through the Choice Program. While it may not be feasible for VA and VHA to implement corrective actions to address all of our findings before the Choice Program ends, it is imperative that VA incorporate lessons learned from the Choice Program when it implements the consolidated community care program it has planned. First, we found VHA’s process for scheduling appointments for veterans through the Choice Program was not consistent with statutory requirements. The Choice Act requires veterans to receive care no more than 30 days from the date an appointment is deemed clinically appropriate or from the date the veteran prefers to receive care; however, we found that veterans could potentially wait up to 70 calendar days to receive routine care through the Choice Program. In effect, we found that in 2016, some veterans’ actual wait times far exceeded 30 days. Although VA has made some relevant contract modifications and issued guidance to address Choice Program wait times, VHA has not adjusted the Choice Program’s appointment scheduling process or established timeliness standards for all steps of the process. In addition, VHA’s monitoring of access to Choice Program care has been limited by incomplete and unreliable data. In particular, the data VHA uses preclude it from accurately identifying the number of days that occur within each phase of the process, from initial referral to the actual appointment. Furthermore, a lack of controls has allowed for inappropriate changes to be made in veterans’ clinically indicated dates and routine versus urgent care categorizations, affecting VA’s ability to monitor whether veterans are receiving Choice Program care in a timely manner. The lack of reliable data and performance measures also hinders VHA’s ability to oversee the program and identify problems and corrective actions. Further, we found that VHA is missing out on opportunities to enhance its design of the planned consolidated community care program. For example, VHA has not fully evaluated its pilot programs for scheduling appointments nor developed tools such as a mechanism that would allow the seamless sharing of information between VHA and the TPAs. Lastly, we found that VHA often relied on inefficient, ad hoc methods of sharing information (such as memoranda, fact sheets and emails), which often failed to reach the VAMC managers and staff responsible for implementing the program. After the Choice Program ends, VA anticipates that veterans will continue to receive care from non-VHA providers under the consolidated community care program that it is planning to implement. VA’s and VHA’s design of the future program can benefit from the lessons learned under the Choice Program. Ignoring these lessons learned and the challenges that have arisen under the Choice Program as VA and VHA design the future consolidated program would only increase VA’s risk for not being able to ensure that all veterans will receive timely access to care in the community. To ensure that VA and VHA incorporate lessons learned from the Choice Program as they develop and implement a consolidated VA community care program, we are making the following 10 recommendations: The Under Secretary for Health should establish an achievable wait- time goal for the consolidated community care program that VA plans to implement that will permit VHA to monitor whether veterans are receiving VA community care within time frames that are comparable to the amount of time they would otherwise wait to receive care at VHA medical facilities. (Recommendation 1) The Under Secretary for Health should design an appointment scheduling process for the consolidated community care program that VA plans to implement that sets forth time frames within which (1) veterans’ referrals must be processed, (2) veterans’ appointments must be scheduled, and (3) veterans’ appointments must occur, which are consistent with the wait-time goal VHA has established for the program. (Recommendation 2) The Under Secretary for Health should establish a mechanism that will allow VHA to systematically monitor the average number of days it takes for VAMCs to prepare referrals, for VAMCs or TPAs to schedule veterans’ appointments, and for veterans’ appointments to occur, under the consolidated community care program that VA plans to implement. (Recommendation 3) The Under Secretary for Health should implement a mechanism to prevent veterans’ clinically indicated dates from being modified by individuals other than VHA clinicians when veterans are referred to the consolidated community care program that VA plans to implement. (Recommendation 4) The Under Secretary for Health should implement a mechanism to separate clinically urgent referrals and authorizations from those for which the VAMC or the TPA has decided to expedite appointment scheduling for administrative reasons. (Recommendation 5) The Under Secretary for Health should (1) establish oversight mechanisms to ensure that VHA is collecting reliable data on the reasons that VAMC or TPA staff are unsuccessful in scheduling veterans’ appointments through the consolidated community care program VA plans to implement, and (2) demonstrate that it has corrected any identified deficiencies. (Recommendation 6) The Secretary of Veterans Affairs should ensure that the contracts for the consolidated community care program VA plans to implement include performance metrics that will allow VHA to monitor average driving times between veterans’ homes and the practice locations of community providers that participate in the TPAs’ networks. (Recommendation 7) The Secretary of Veterans Affairs should establish a system for the consolidated community care program VA plans to implement to help facilitate seamless, efficient information sharing among VAMCs, VHA clinicians, TPAs, community providers, and veterans. Specifically, this system should allow all of these entities to electronically exchange information for the purposes of care coordination. (Recommendation 8) The Under Secretary for Health should conduct a comprehensive evaluation of the outcomes of the two appointment scheduling pilots it established at the Alaska and Fargo VA Health Care Systems (where VAMC staff, rather than TPA staff, are responsible for scheduling veterans’ Choice Program appointments), which should include a comparison of the timeliness with which VAMC staff and TPA staff completed each step of the Choice Program appointment scheduling process, as well as the overall timeliness with which veterans received appointments. (Recommendation 9) The Under Secretary for Health should issue a comprehensive policy directive and operations manual for the consolidated community care program VA plans to implement and ensure that these documents are reviewed and updated in a timely manner after any significant changes to the program occur. (Recommendation 10) VA provided written comments on a draft of this report, which are reprinted in Appendix VII. In its comments, VA concurred with 8 of our 10 recommendations and described its plans for implementing them. VA stated that VHA’s Office of Community Care will work collaboratively with other VA and VHA offices to evaluate modifications to the current wait- time goals and measurement processes so that wait times for VA community care can be compared to wait times for care delivered at VHA medical facilities. VA did not concur with our recommendation to implement a mechanism to separate clinically urgent referrals and authorizations from those that are designated as urgent for administrative reasons. VA stated that because VAMC staff (rather than TPA staff) will be responsible for scheduling veterans’ appointments under the consolidated community care program it plans to implement, there would no longer be a need to separate clinically urgent referrals from those that need to be administratively expedited. However, we maintain that our recommendation is warranted. In particular, we found that VA’s data did not always accurately reflect the timeliness of urgent care because both VAMC and TPA staff inappropriately re-categorized some routine care referrals and authorizations as urgent ones for reasons unrelated to the veterans’ health conditions. Regardless of whether VAMC staff or TPA staff are responsible for appointment scheduling, VA will need to ensure that it uses reliable data to monitor the extent to which veterans receive urgent care within required time frames. Without a means of separating clinically urgent referrals and authorizations from ones for which the scheduling process must be administratively expedited, VA’s data on the timeliness of urgent care will continue to be unreliable. VA agreed in principle with our recommendation to issue a comprehensive policy directive and operations manual, but stated in its comments that it would wait to determine whether a comprehensive policy directive is needed until after the consolidated community care program has been fully implemented and any interim implementation challenges have been resolved. However, when implementing a new program, it is important that agencies establish the program’s structure, responsibilities, and authorities at the beginning to help ensure that the new program’s objectives are met. Relying on outdated policies and unreliable communication methods increases VA’s risk of encountering foreseeable challenges. Without issuing a comprehensive policy directive and operations manual before the start of the new program, VA risks experiencing untimely communication issues similar to those that affected veterans’ access to care throughout the Choice Program’s implementation. A comprehensive policy directive and operations manual that could be updated as changes occur would give VAMCs a definitive source of real-time, up-to-date information and reduce the likelihood that VAMCs will implement new policies and processes inconsistently under the future program. We are sending copies of this report to the Secretary of Veterans Affairs, the Under Secretary for Health, appropriate congressional committees, and other interested parties. This report is also available at no charge on the GAO Web site at http://www.gao.gov. If you or your staffs have any questions about this report, please contact Sharon M. Silas at (202) 512-7114 or silass@gao.gov or A. Nicole Clowers at (202) 512-7114 or clowersa@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs are on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VIII. To examine selected veterans’ actual wait times to receive routine care and urgent care through the Choice Program and the information VHA uses to monitor access to care under the program, we took five key steps. We (1) analyzed Choice Program appointment wait times for selected veterans using a sample of 196 Choice Program authorizations for routine and urgent care; (2) reviewed VHA’s analysis of Choice Program appointment wait times for a sample of about 5,000 Choice Program authorizations; (3) reviewed data VHA uses to monitor the timeliness of Choice Program care and reasons that the TPAs have returned Choice Program referrals without making appointments; (4) interviewed VA, VHA, and TPA officials; and (5) reviewed federal internal control standards, as follows. 1. Our analysis of Choice Program wait times for a sample of 196 authorizations. To analyze the timeliness of Choice Program appointment scheduling and completion for a sample of veterans, we selected six VAMCs and a random, non-generalizable sample of 196 authorizations for veterans who were referred to the Choice Program by those six VAMCs between January 2016 and April 2016. We judgmentally selected the six VAMCs to include variation in geographic location, with three VAMCs that serve rural veteran populations and three VAMCs that serve urban veteran populations. In addition, three of the VAMCs were served by Health Net, and three were served by TriWest. (See table 5.) To select our random, non-generalizable sample of 196 Choice Program authorizations, we obtained VA data on all authorizations created by the TPAs between January and April 2016 for veterans who were referred to the program by the six VAMCs we selected—a universe of about 55,000 authorizations. From these 55,000 authorizations, we randomly selected: 55 routine care authorizations (about 10 authorizations per VAMC) for which the TPAs scheduled appointments for veterans, 53 urgent care authorizations (about 10 authorizations per VAMC) for which the TPAs scheduled appointments for veterans, and 88 routine and urgent care authorizations (about 15 authorizations per VAMC) that the TPAs returned to VA without scheduling appointments for any one of the following three reasons—(1) VA requested the authorization be returned, (2) VA data were missing from the referral, and (3) the veteran declined or did not want Choice Program care. For all 196 Choice Program authorizations in our sample, we manually reviewed VHA documentation (specifically, the veterans’ VA electronic health records) and TPA documentation to track the number of calendar days that elapsed at each step of the Choice Program appointment scheduling process. For the authorizations that the TPAs returned to the VAMCs without making appointments, we examined VHA and TPA documentation to determine whether the veterans eventually obtained care through other means—such as through another VA community care program, a different Choice Program referral, or at a VHA medical facility—and how long it took to receive that care. Determining whether veterans in our sample experienced clinical harm or adverse clinical outcomes because of delays in the VAMCs’ or TPAs’ processing of their referrals and authorizations was outside the scope of our review. We selected our sample of 55 routine care and 53 urgent care authorizations for which the TPAs succeeded in scheduling appointments to include only authorizations for which the TPAs did not meet VA’s appointment scheduling goals at one phase of the appointment scheduling process: when the TPAs attempt to schedule appointments after the veterans have opted in to the program. This was to ensure that our sample included only authorizations for which scheduling was delayed, so that we could examine the potential causes of appointment scheduling delays and whether delays also occurred at other phases of the process (such as when VAMCs were preparing the veterans’ referrals or when the TPAs were attempting to reach the veterans for them to opt in to the program). We omitted this phase of the appointment scheduling process when calculating the timeliness of appointment completion for the 55 routine care authorizations and 53 urgent care authorizations in our sample. Rather than reporting veterans’ overall wait times for these authorizations, we report the average number of calendar days that elapsed (1) while VAMCs were preparing veterans’ Choice Program referrals, (2) while the TPAs were attempting to reach veterans for them to opt in to the program, and (3) while veterans waited to attend their appointments after the TPAs succeeded in scheduling them. To assess the reliability of the authorization data we used, we interviewed knowledgeable agency officials, manually reviewed the content of the data, and electronically tested it for missing values. We concluded that these data were sufficiently reliable for the purposes of our reporting objectives. The findings from our review of Choice Program authorizations cannot be generalized beyond the VAMCs and the veterans’ Choice Program authorizations we reviewed. 2. VHA’s analysis of Choice Program wait times for a sample of about 5,000 authorizations. We obtained from VHA’s Office of Community Care the results of a nationwide analysis of Choice Program appointment timeliness it conducted in February 2017. Specifically, VHA directed its VAMCs to manually review veterans’ health records and TPA documentation and report observations for a non-generalizable sample of about 5,000 randomly selected Choice Program authorizations that were created between July and September of 2016. The sample was limited to authorizations for Choice Program appointments that had been scheduled for time- eligible veterans who needed four types of specialty care— mammography, gastroenterology, cardiology, and neurology. According to VHA officials, they limited their analysis to these four types of care because delayed treatment for any of these specialties could cause adverse health outcomes for patients. To assess the reliability of VHA’s data, we manually reviewed the results of its analysis and interviewed knowledgeable agency officials about potential outliers. We concluded that VHA’s data were sufficiently reliable for the purposes of our reporting objective. The results of VHA’s analysis cannot be generalized beyond the sample of Choice Program authorizations that it reviewed. 3. VHA data on timeliness of Choice Program appointments and the reasons TPAs return referrals without making appointments. To evaluate the information VHA uses to monitor access to care under the Choice Program, we reviewed data that VHA collects to monitor the timeliness with which the TPAs schedule appointments and the timeliness with which appointments occur after the TPAs have scheduled them. We also reviewed and tested the reliability of VHA data on the reasons the TPAs have returned Choice Program referrals to VAMCs without scheduling appointments, which may offer insights about access to care (e.g., the percentage of referrals that are returned due to a lack of providers in the TPAs’ networks). 4. Interviews with officials. We interviewed VA, VHA, and TPA officials responsible for administering the Choice Program contracts and overseeing implementation of the program. We interviewed these officials to gain an understanding of the processes they followed and the information they used to monitor veterans’ access to Choice Program care. 5. Federal internal control standards. We examined the results of our and VHA’s analyses and the information VHA uses to monitor veterans’ access to care under the program in the context of federal standards for internal control for (1) information and communication and (2) monitoring. The internal control standard for information and communication relates to management’s ability to use quality information to achieve the entity’s objectives. The internal control standard for monitoring relates to establishing activities to monitor the quality of performance over time and evaluating the results. Appendix II: Process for Veterans to Obtain Department of Veterans Affairs (VA) Choice Program Care if They Are Time-Eligiblea If the veteran does not respond to the letter within 14 calendar days, a notification is sent to the veteran’s VA clinician so that they can determine if additional action should be taken. Appendix IV: Comparison of Processes for Arranging Choice Program and Individually Authorized Community Care The Veterans Health Administration (VHA) uses the time-eligible appointment scheduling process when the services needed are not available at a VHA medical facility or are not available within allowable wait times. We found 21 actions that the Department of Veterans Affairs (VA) and the Veterans Health Administration (VHA) took after the Choice Program’s November 2014 implementation that were intended to help address issues related to veterans’ access to care. Table 6, below, provides a chronological summary of the actions VA and VHA had taken as of August 2017 and the issues they were intended to address. In addition to the contact named above, Marcia A. Mann (Assistant Director), Alexis C. MacDonald (Analyst-in-Charge), Daniel Powers, and Michael Zose made major contributions to this report. Also contributing were Muriel Brown, Christine Davis, Helen Desaulniers, Krister Friday, Sandra George, Jacquelyn Hamilton, and Vikki Porter. Veterans’ Health Care: Preliminary Observations on Veterans’ Access to Choice Program Care, GAO-17-397T (Washington, D.C.: March 7, 2017). VA Health Care: Improved Monitoring Needed for Effective Oversight of Care for Women Veterans, GAO-17-52 (Washington, D.C.: December 2, 2016). VA’S Health Care Budget: In Response to a Projected Funding Gap in Fiscal Year 2015, VA Has Made Efforts to Better Manage Future Budgets, GAO-16-584 (Washington, D.C.: June 3, 2016). Veterans’ Health Care: Proper Plan Needed to Modernize System for Paying Community Providers, GAO-16-353 (Washington, D.C.: May 11, 2016). VA Health Care: Actions Needed to Improve Monitoring and Oversight of Non-VA and Contract Care. GAO-15-654T (Washington, D.C.: June 1, 2015). VA Health Care: Further Action Needed to Address Weaknesses in Management and Oversight of Non-VA Medical Care, GAO-14-696T (Washington, D.C.: June 18, 2014). VA Health Care: Actions Needed to Improve Administration and Oversight of VA’s Millennium Act Emergency Care Benefit, GAO-14-175 (Washington, D.C.: March 6, 2014). VA Health Care: Management and Oversight of Fee Basis Care Need Improvement, GAO-13-441 (Washington, D.C.: May 31, 2013).

Summary: Congress created the Choice Program in 2014 to address longstanding challenges with veterans' access to care at VHA medical facilities. The Joint Explanatory Statement for the Consolidated Appropriations Act, 2016 included provisions for GAO to review veterans' access to care through the Choice Program. This report examines for Choice Program care (1) VA's appointment scheduling process, (2) the timeliness of appointments and the information VHA uses to monitor veterans' access; and (3) the factors that have adversely affected veterans' access and the steps VA and VHA have taken to address them for VA's future community care program. GAO reviewed applicable laws and regulations, VA's TPA contracts, and relevant VHA policies and guidance. Absent reliable national data, GAO also selected 6 of 170 VAMCs (selected for variation in geographic location and the TPAs that served them) and manually reviewed a random, non-generalizable sample of 196 Choice Program authorizations. The authorizations were created for veterans who were referred to the program between January and April of 2016, the most recent period for which data were available when GAO began its review. The sample of authorizations included 55 for routine care, 53 for urgent care, and 88 that the TPAs returned without scheduling appointments. GAO also obtained the results of VHA's non-generalizable analysis of wait times for a nationwide sample of about 5,000 Choice Program authorizations that were created for selected services between July and September of 2016. Through the Veterans Choice Program (Choice Program), eligible veterans may receive care from community providers when it is not readily accessible at Veterans' Health Administration (VHA) medical facilities. The Department of Veterans Affairs (VA) uses two contractors—or third party administrators (TPA)—to schedule most veterans' Choice Program appointments after receiving referrals from VA medical centers (VAMC). GAO found that veterans who are referred to the Choice Program for routine care because services are not available at VA in a timely manner could potentially wait up to 70 calendar days for care if VAMCs and the TPAs take the maximum amount of time VA allows to complete its appointment scheduling process. This is not consistent with the statutory requirement that veterans receive Choice Program care within 30 days of their clinically indicated date (when available), which is the soonest date that it would be appropriate for the veteran to receive care, according to a VHA clinician. Without designing appointment scheduling processes that are consistent with this requirement, VA lacks assurance that veterans will receive Choice Program care in a timely manner. GAO and VHA found that selected veterans experienced lengthy actual wait times for appointments in 2016, after manually reviewing separate samples of Choice Program authorizations. For example, when GAO analyzed 55 routine care authorizations that were created between January and April of 2016, it found that the process took at least 64 calendar days, on average. When VHA analyzed about 5,000 authorizations created between July and September of 2016, it took an average of 51 calendar days for veterans to receive care. a GAO excluded from its analysis the amount of time the TPA took to schedule the appointment and the overall wait time because its sample selection methodology differed from VHA's in a way that would have skewed these two averages but not the averages for the other segments of the process. GAO also found that VHA cannot systematically monitor the timeliness of veterans’ access to Choice Program care because it lacks complete, reliable data to do so. The data limitations GAO identified include: A lack of data on the timeliness of referring and opting veterans in to the program. GAO found that the data VHA uses to monitor the timeliness of Choice Program appointments do not capture the time it takes VAMCs to prepare veterans’ referrals and send them to the TPAs, nor do they capture the time spent by the TPAs in accepting VAMCs’ referrals and opting veterans in to the Choice Program. VHA has implemented an interim solution to monitor overall wait times that relies on VAMC staff consistently and accurately entering unique identification numbers on VHA clinicians’ requests for care and on Choice Program referrals, a process that is prone to error. Inaccuracy of clinically indicated dates. GAO found that clinically indicated dates (which are used to measure the timeliness of care) are sometimes changed by VAMC staff before they send Choice Program referrals to the TPAs, which could mask veterans’ true wait times. GAO found that VAMC staff entered later clinically indicated dates on referrals for about 23 percent of the 196 authorizations it reviewed. It is unclear if VAMC staff mistakenly entered incorrect dates manually, or if they inappropriately entered later dates when the VAMC was delayed in contacting the veteran, compiling relevant clinical information, and sending the referral to the TPA. Unreliable data on the timeliness of urgent care. GAO found that VAMCs and TPAs do not always categorize Choice Program referrals and authorizations in accordance with the contractual definition for urgent care. According to the contracts, a referral is to be marked as “urgent,” and an appointment is to take place within 2 days of the TPA accepting it, when a VHA clinician has determined that the needed care is (1) essential to evaluate and stabilize the veteran’s condition, and (2) if delayed would likely result in unacceptable morbidity or pain. GAO reviewed a sample of 53 urgent care authorizations and determined that about 28 percent of the authorizations were originally marked as routine care authorizations but were changed to urgent by VAMC or TPA staff, in an effort to administratively expedite appointment scheduling. Without complete, reliable data, VHA cannot determine whether the Choice Program has helped to achieve the goal of alleviating veterans’ wait times for care. GAO found that numerous factors adversely affected veterans’ access to care through the Choice Program. These factors include: (1) administrative burden caused by complexities of referral and appointment scheduling processes, (2) poor communication between VHA and its VAMCs, and (3) inadequacies in the networks of community providers established by the TPAs, including an insufficient number, mix, or geographic distribution of community providers. VA and VHA have taken numerous actions throughout the Choice Program’s operation that were intended to help address these factors, though not all access factors have been fully resolved. For example, to help address administrative burden and improve the process of coordinating veterans’ Choice Program care, VA established a secure e-mail system and a mechanism for TPAs and community providers to remotely access veterans’ VA electronic health records. However, these mechanisms only facilitate a one-way transfer of necessary information. They do not provide a means by which VAMCs or veterans can view the TPAs’ step-by-step progress in scheduling appointments or electronically receive medical documentation associated with Choice Program appointments. While the Choice Program will soon end, VA anticipates that veterans will continue to receive community care under a similar program that VA plans to implement, which will consolidate the Choice Program and other VA community care programs. Incorporating lessons learned from the Choice Program into the implementation and administration of the new program could help VHA avoid similar challenges. For VA's future consolidated community care program, GAO is making 10 recommendations, which include: establishing an achieveable wait-time goal for the community care program that will permit VHA to monitor whether veterans are receiving care within time frames that are comparable to the amout of time they would otherwise wait for care at VHA medical facilities; designing an appointment scheduling process that (1) is consistent with the wait-time goal and (2) sets forth time frames within which veterans' referrals must be processed, appointments must be scheduled, and appointments must occur; allow VHA to systematically monitor the amount of time taken to prepare referrals, schedule appointments, and complete appointments; prevent veterans' clinically indicated dates from being modified by individuals other than VHA clinicians; and separate clinically urgent referrals and authorizations from those for which the VAMC or the TPA has decided to expedite appointment scheduling for administrative reasons; and establishing a system that will help facilitate seamless, efficient care coordination and exchanges of information among VAMCs, VHA clinicians, TPAs, community providers, and veterans. VA generally agreed with all but one of GAO's recommendations, which was to separate clinically urgent referrals from those that are administratively expedited. GAO maintains that implementing this recommendation will help improve future monitoring of urgent care timeliness for reasons explained in the report.