Over the last decade, VHA has increasingly provided care on an outpatient basis, including primary care and mental health care services. VHA Handbook 1006.02, VHA Site Classifications and Definitions, defines classifications for outpatient sites of care including CBOCs. VHA’s Directive 1229, Planning and Operating Outpatient Sites of Care, outlines the process for establishing new CBOCs. VHA provides outpatient care through CBOCs, health care centers, and other outpatient services sites, which are defined in VHA’s site classification policy: CBOCs are clinics that provide primary care and mental health care services, and also may provide specialty care services such as cardiology or neurology, in an outpatient setting. CBOCs can provide a wide array of services, ranging from a small, mainly telehealth clinic with one technician and a nurse, to a large clinic with several specialty care services and providers. Each clinic is overseen by, and separate from, its VAMC; each VAMC in turn is overseen by one of 18 VISNs. Health care centers are large multi-specialty outpatient clinics that provide primary care, mental health care, and on-site surgical services, in addition to other health care services. Other outpatient services sites provide nonclinical services, such as social services, homelessness services, and support services. They may also provide services that are clinical in nature through telehealth or other arrangements. (See fig. 1.) To establish a new CBOC, VHA’s policy states that the VAMC and VISN must ensure that one is needed by first exhausting existing VHA resources (such as changing clinic hours or staffing) and determining that VHA community care programs cannot meet the identified demand. The VAMC and VISN follow several steps to assess the need for a new clinic: Step 1—The VAMC and VISN identify an underserved area using VHA models that project changes in the veteran population and trends in veterans’ health care needs. Step 2—The VAMC develops a detailed proposal for the new clinic— an Access Expansion Plan—that includes information such as whether the proposed clinic will be VHA-operated or contracted, projected workload, scope of the services to be provided, and cost. It also describes, as required by VHA policy, how the VAMC has exhausted existing VHA resources before proposing a new clinic. Step 3—The VISN reviews the expansion plan, and if approved forwards it to an interdisciplinary panel at VHA’s central office, which reviews it. A list of approved clinics is then sent to the Under Secretary for Health for endorsement. Step 4—Endorsed clinics are included in the VISN’s Strategic Capital Investment Planning process submission for the fiscal year. Final approval and funding for a new CBOC is dependent on Office of Management and Budget approval of VA’s budget submission and VHA’s final appropriations. In fiscal year 2015, VHA suspended the establishment of new CBOCs beginning in fiscal year 2018 due to several factors, including budget constraints and an emphasis on the use of VHA community care programs. However, VISNs can submit requests for exceptions to the Deputy Under Secretary for Health for Operations and Management for review. VHA officials told us 11 exceptions had been granted as of February 2018. We found that VHA-operated CBOCs provided more specialty care and less primary care and mental health care as a proportion of their total provided services than contracted CBOCs in fiscal years 2014 through 2016. For example, in fiscal year 2016, specialty care (e.g., cardiology, gastroenterology, physical therapy) comprised 13 percent of services provided at VHA-operated clinics and 5 percent of services provided at contracted clinics. In contrast, VHA-operated clinics provided proportionally less primary care and mental health services (services offered at all CBOCs) in fiscal year 2016—these services comprised 66 percent of the services provided at VHA-operated clinics, but 84 percent of the services provided at contracted clinics. (See fig. 2.) We found that VHA-operated CBOCs provided several specialty care services that were not offered in contracted CBOCs. For example, dental care services and gastrointestinal endoscopy were provided by multiple VHA-operated clinics, but were not provided by any of the contracted clinics in fiscal year 2016. In addition, we found that VHA-operated clinics were generally larger and provided more complex services than contracted clinics. For example, multi-specialty CBOCs (clinics that provide two or more on-site specialty care services, and which may offer procedures requiring local anesthesia or sedation) were more often VHA- operated than contracted. Of the 733 CBOCs in fiscal year 2016, 210 were classified by VHA as multi-specialty, and nearly all of these (206) were VHA-operated. Officials from the four VAMCs and VISNs in our review told us decisions about what types of services CBOCs provide are made on a case-by- case basis according to local needs. For example, officials from one VAMC told us they decided to add physical therapy specialty care to one of their VHA-operated clinics based on analysis indicating that veterans’ need for this care in their community would increase. Also, officials said they wanted to alleviate the travel burden for veterans who needed the care, as the next closest VHA facility that offered this care was a 2.5-hour drive away. Officials from another VAMC told us that they approached the service needs at their clinics from a regional perspective, allowing for veteran demand for services to be met across multiple clinics in the same geographic area instead of relying on one clinic to meet the need. As a result of this approach, VAMC officials were in the process of expanding services at two of its clinics. From fiscal years 2014 through 2016, we found that VHA-operated CBOCs had higher per-encounter expenditures than contracted CBOCs—a difference ranging from 3 to 5 percent per encounter. (See table 2.) We also found that per-encounter expenditures for almost all service types were higher on average for VHA-operated CBOCs than contracted CBOCs in fiscal year 2016; the exception was mental health care services, where VHA-operated clinics’ per-encounter expenditures were 2 percent lower than for contracted clinics. The difference in per-encounter expenditures was greatest for specialty care services. For example, VHA- operated clinics’ per-encounter expenditures for specialty care services were 46 percent higher than for contracted clinics. This is in contrast to primary care, where VHA-operated clinics had 11 percent higher per- encounter expenditures, on average, compared to contracted clinics. (See fig. 3.) Officials told us that several factors can influence per-encounter expenditures, including (1) differences in provider compensation and types of providers (physicians vs. physician assistants); (2) the number of patients with complex health conditions that generally require longer visits and more costly services (as opposed to patients with well-managed conditions); and (3) geographic differences in the cost of providing care. One of our selected contracted CBOCs had one of the highest per- encounter expenditures for fiscal year 2016 among all clinics. Officials from this clinic’s VAMC told us this was due to the contractor being able to command a very high payment rate at the time of the contract award, due to temporarily strong local economic conditions, as well as being the only contractor in the area capable of providing the required services. Officials said the VAMC is in the process of awarding a new contract for this clinic. Although per-encounter expenditures were generally lower for contracted CBOCs, officials from the VISNs and VAMCs in our review told us they consider several factors in determining whether a new clinic will be VHA- operated or contracted. Such factors include the ability to directly monitor performance and implement new standards of care, as well as the ability to recruit and staff the clinic. For example, officials from two VAMCs in our review told us that VHA-operated clinics can be easier to manage because the VAMC has direct control of the clinic. Officials said this makes it easier to implement changes to VHA standards of care without the need to enter into contract modification negotiations. On the other hand, officials from three of the four VISNs and three of the four VAMCs in our review told us that contractors can be more flexible than VHA in recruiting staff (such as the ability to offer higher salaries), making a contracted clinic desirable for geographic areas where VHA has challenges recruiting or retaining providers. We found that VHA has implemented certain oversight requirements, but not others described in Directive 1229—its policy that outlines VHA’s oversight responsibilities for outpatient sites of care, including CBOCs. In terms of the oversight requirements that VHA implemented, we found it has provided reports on patient satisfaction to VISNs and VAMCs on a monthly basis. Specifically, VHA distributes the results of the VHA Survey of Healthcare Experiences, a monthly survey of veterans’ satisfaction with the care they received through VHA health care facilities. In addition, VHA implemented the requirement to make measures related to evaluating the progress of outpatient sites of care, such as data on wait times, workload, and costs, available on an internal VHA website. However, VHA has not implemented other oversight requirements, which is inconsistent with federal standards for internal control related to monitoring, which state that management should establish and operate monitoring activities to monitor the internal control system and evaluate the results. We found that VHA has not implemented the following requirements in Directive 1229: VHA has not developed guidelines for monitoring the quality and comprehensiveness of care in CBOCs. Officials from the three VHA offices with responsibility for collaborating to develop guidelines for monitoring the quality and comprehensiveness of care in CBOCs, as required in the policy, told us that they are not currently developing these guidelines and they have no plans to do so. First, officials from the office of the Assistant Deputy Under Secretary for Health for Policy and Planning told us they had not developed these guidelines because they no longer believed it was their office’s responsibility, despite the fact that officials from the office had helped to develop the recently issued policy. Second, officials from the office of the Deputy Under Secretary for Health for Organizational Excellence told us that their office was not responsible for addressing the broader issue of monitoring clinics. Third, officials from the office of the Deputy Under Secretary for Health for Operations and Management told us that although they do not have formal guidelines in place, they believe their office meets the Directive 1229 requirement as part of their regular VISN oversight. Officials said they collect and review VISN- level performance data, such as patient satisfaction data, which can be broken down to the level of the CBOC if there is a performance problem. However, VHA may miss clinic performance problems that are not identifiable in the VISN-level data. In addition, without developing such guidelines, VHA has not established standardized processes for how it monitors CBOCs, which can lead to inconsistent oversight. This poses the risk that veterans may be subject to different standards of care depending on the clinic visited. VISNs do not conduct continuous quality monitoring of CBOCs to ensure that consistent, quality care is being delivered. We found that three of the four VISNs in our review largely delegated oversight of the CBOCs to the VAMCs, rather than conducting continuous quality monitoring as required in the policy. Specifically, officials from these VISNs said that they largely focus their oversight on the VAMCs and do not separately review the performance of every CBOC unless the VAMC informs them of a quality problem at a particular clinic. Officials from the remaining VISN in our review said they do conduct CBOC-specific oversight activities. Specifically, this VISN had created a performance review survey tool that it sends to each clinic on an annual basis, and the results are reviewed by a workgroup made up of VISN staff. The workgroup examines trends across the CBOCs, including a comparison of VHA-operated and contracted performance. For example, one question in the tool asks how an individual CBOC’s performance compares with others overseen by the VAMC. The delegation of oversight responsibility for the CBOCs to the VAMCs without consistent VISN-level oversight creates the potential for inconsistencies in oversight, which does not align with VHA policy to provide one standard of care for all clinics. Consequently, veterans may be subject to different standards of care across clinics. The Deputy Under Secretary for Health for Operations and Management has not reviewed CBOC performance with VISNs as part of the quarterly VISN performance reviews. The Deputy Under Secretary for Health for Operations and Management is responsible for conducting reviews of VISN performance with each VISN director. Specifically, the office of the Deputy Under Secretary for Health for Operations and Management is required by VHA policy to review CBOC-level performance data during quarterly VISN performance reviews. However, officials from this office and two of the VISNs we contacted told us they do not specifically do this unless the VISN identifies a performance problem. Of the remaining two VISNs, officials at one VISN reported only having mid-year and year-end meetings with VHA central office at which they did not specifically discuss the CBOCs, and officials from the other VISN said they did not have any regular quarterly performance reviews with VHA central office. This lack of consistent oversight poses the risk that VHA is not providing one, high quality standard of care to veterans across CBOCs. Directive 1229 requires VHA to provide reports to the VISNs and VAMCs on CBOC quality of care on a quarterly and year-end basis. We found that the CBOC Report, which is VHA’s only report that allows for comparing clinical quality of care data across VHA-operated and contracted CBOCs, lacks accurate and complete information. These gaps limit the CBOC Report’s usefulness as a monitoring tool to determine whether VHA-operated and contracted CBOCs are providing the same standard of care. This is inconsistent with federal standards for internal control for information and communication, which state that management should use quality information to achieve the entity’s objectives. Specifically, VHA distributes the CBOC Report to VISNs and VAMCs on a quarterly and year-end basis, which compiles CBOC quality of care performance results based on the Healthcare Effectiveness Data and Information Set (HEDIS)—an industry standard set of quality measures. VISNs and VAMCs have access to other types of CBOC performance data, such as patient satisfaction data and wait time data, but these data are not used to assess clinical quality of care and they cannot be used to examine performance across all CBOCs or stratified by VHA-operated versus contracted CBOCs. In contrast, the CBOC Report allows for the comparison of clinical quality of care data across all CBOCs, which can be stratified according to whether the clinic is VHA-operated or contracted. However, we found the following issues with the CBOC Report: Incorrect classification of CBOCs. We compared CBOCs from the most recent CBOC Report at the time of our review (the first quarter of fiscal year 2017) against sites in the VAST system as of January 3, 2017, which is VHA’s listing of all VHA sites of care and their characteristics. We found that 22 percent of sites were incorrectly classified as CBOCs, based on the site classifications in VAST. Several of these sites were much more complex, such as health care centers and VAMCs. For example, a VAMC was included in the report as a CBOC, but this VAMC has three specialized intensive care units and serves as a regional referral center for intensive inpatient surgery, including open heart surgery. In addition, we also identified sites included in the report that provided less complex services than those that are provided in CBOCs, such as other outpatient services sites. VHA officials who produce the CBOC Report told us that, prior to the establishment of the VAST site classifications in 2014, they used their judgment to classify existing sites of care as CBOCs and they have not updated their classifications since then. For sites established since 2014, officials told us they use the VAST site classifications, but may also use their judgment in certain situations. For example, if a site’s classification changed in VAST from a non-CBOC to a CBOC, they would make a decision about whether to classify it as a CBOC in the report by examining various aspects of the facility, such as the services provided and encounters. This procedure differs from what is documented in the methodology section of the CBOC Report, which states that site classifications are based on VAST. Further, VHA officials said they did not have a document available that outlined how they make these decisions. Because the site classifications in the CBOC Report are based, in part, on officials’ judgment in addition to the classifications in VAST, the report does not present accurate information on CBOCs across VHA and is of limited usefulness to VHA as a tool to ensure that VHA-operated and contracted CBOCs are providing the same standard of care that is of high quality. Missing CBOCs. We found that 53 CBOCs (7 percent of all CBOCs) were missing from the CBOC Report from the first quarter of fiscal year 2017, rendering the data incomplete. VHA officials provided examples of why a CBOC might not be included in the report. For example, a newer CBOC might not be included because it did not have quality of care data available at the time the report was developed. However, we identified several other sites that were listed in the report, despite unavailable data. Inaccurate summary calculations. Due to the incorrect site classifications and missing CBOCs, the national- and VISN-level summary calculations of performance in the CBOC Report were also inaccurate. Specifically, the report includes national- and VISN-level averages for each HEDIS measure, which VHA officials can use as benchmarks for clinic performance. These averages were over- inclusive—incorporating performance results from additional sites that were not CBOCs, and under-inclusive—omitting performance results from CBOCs that were missing from the report. These inaccuracies may lead VHA officials to draw incorrect conclusions about the quality of care provided in CBOCs. For example, officials from one VAMC told us that they use the national averages as benchmarks against which they compare the performance of their CBOCs. Because this VAMC requires CBOCs with lower-than-average HEDIS performance results to develop a formal action plan to improve performance, officials may not be identifying clinics that are in need of an action plan due to the inaccuracy of the averages. In addition, VHA central office officials who develop the CBOC Report said that the results from recent reports have shown that VHA-operated and contracted clinics in general provided the same standard of care, but this conclusion may not be correct as it is based on unreliable data. No guidance or training for use of the CBOC Report. VHA central office officials do not provide guidance or training specific to the CBOC Report to assist VISNs and VAMCs in using it to oversee CBOCs. This is inconsistent with federal standards for internal control related to the control environment, which state that management should, among other things, develop personnel to achieve the entity’s objectives. Such development may include training to enable individuals to develop competencies appropriate for key roles. In our review of the CBOC Report from the first quarter of fiscal year 2017, we found that in several places in the report, shorthand text and acronyms were used, but not defined. In addition, although there is a methodology section, it is not clear that the measures described in the report are HEDIS measures, for which VHA makes training available. Several VAMC and VISN officials stated that guidance or training that is specific to understanding the CBOC Report would be helpful. If VISNs and VAMCs are not trained on how to use the report, they may not know how to use it to oversee CBOCs and ensure they are providing one standard of care that is of high quality. No requirement for VISNs or VAMCs to use the CBOC Report. VHA does not require that the CBOC Report be used as a tool to oversee CBOCs. As a result, we found that the report was not widely used. Specifically, an official from the office of the Deputy Under Secretary for Health for Organizational Excellence—which produces the CBOC Report—told us that the office’s role is to compile the reports and distribute them, but not to monitor performance. Officials from the office of the Deputy Under Secretary for Health for Operations and Management said that VISNs and VAMCs are expected to use the report as part of their CBOC oversight; however, we found there is no requirement that they do so. We found that officials from three of the four VISNs and three of the four VAMCs in our review were not regularly using the CBOC Report. Officials from one of the four VAMCs and one of the four VISNs in our review were using it as part of CBOC oversight activities at the time of our review. Officials from another VISN said that they planned to start using the CBOC Report after we made them aware of it during our interview. If VISN and VAMC officials do not use the report as a part of their oversight, they may be missing opportunities to compare VHA- operated and contracted CBOCs and ensure they are providing one standard of care that is of high quality. CBOCs are an integral part of VHA’s health care delivery system, and VHA requires that such clinics, whether VHA-operated or contracted, provide the same standard of care to veterans that is of high quality. Although VHA has implemented certain policy requirements for CBOC oversight, we found several weaknesses in its oversight that make it difficult to determine whether it is ensuring this consistent standard of care across the clinics. Specifically, VHA has not fully implemented oversight requirements that align with its established policies, including a requirement to establish guidelines for overseeing CBOC quality of care. The CBOC Report, as VHA’s only report comparing clinical quality of care across both VHA- operated and contracted clinics, could be an important part of those guidelines. However, as it currently stands, the report is inaccurate and incomplete and VISNs and VAMCs are not trained on or required to use it; thus, it is of limited use to VHA, including the VISNs and VAMCs that have responsibility for CBOC oversight. As a result, VHA lacks assurance that both VHA-operated and contracted CBOCs are providing one standard of care that is of high quality. We are making the following four recommendations to the VHA Undersecretary for Health: Implement oversight requirements that align with VHA’s existing policy, including developing guidelines for monitoring quality of care in CBOCs. (Recommendation 1) Establish a process for regularly updating the CBOC Report to ensure it contains an accurate and complete list of CBOCs that is consistent with VHA’s established site classifications. (Recommendation 2) Ensure that VISNs and VAMCs receive guidance or training on how to use the CBOC Report. (Recommendation 3) Require the use of the CBOC Report as an oversight tool for ensuring one standard of care that is of high quality across VHA-operated and contracted CBOCs. (Recommendation 4) We provided VA with a draft of this report for its review and comment. VA provided written comments, which are reprinted in appendix I. In its written comments, VA concurred with all four of the report’s recommendations, and identified actions it is taking to implement them. We are sending copies of this report to the appropriate congressional committees, the Acting Secretary of Veterans Affairs, the Under Secretary for Health, and other interested parties. In addition, the report is available at no charge on the GAO Web site at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or at draperd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II. Debra A. Draper, (202) 512-7114 or draperd@gao.gov. In addition to the contact named above, Janina Austin, Assistant Director; Malissa G. Winograd, Analyst-in-Charge; Jennie F. Apter; Zhi Boon; Keith Haddock; and Sarah-Lynn McGrath made key contributions to this report. Also contributing were Jacquelyn Hamilton and Vikki Porter.

Summary: In fiscal year 2016, VHA's 733 CBOCs provided care to more than 3 million veterans at a cost of $5.3 billion. Although most of these clinics are VHA-owned and -operated, 101 are operated through contracts with non-VHA organizations. VHA policy states that CBOCs, whether VHA-operated or contracted, must provide one standard of care that is of high quality. GAO was asked to review VHA's use of contracts to carry out core functions. This report examines, among other issues, the extent to which VHA oversees CBOC operations. To conduct this work, GAO reviewed VHA's policies and CBOC Report. GAO also interviewed officials from VHA's central office and from a nongeneralizable sample of eight CBOCs and their four respective VAMCs and VISNs. The CBOCs were selected for variation in factors such as contract status and geographic area. Community-based outpatient clinics (CBOC) are an important part of the Department of Veterans Affairs' (VA) Veterans Health Administration (VHA) health care delivery system. These clinics are geographically separate from VA medical centers (VAMC) and provide outpatient services, including primary care and mental health care. GAO found weaknesses in VHA's oversight of CBOCs: Incomplete policy implementation. VHA has not implemented certain CBOC oversight requirements as outlined in its policy. Specifically, VHA has not developed guidelines for monitoring the quality and comprehensiveness of care in CBOCs and officials said they have no plans to do so. Officials told GAO they believe the requirement was met as part of their regular oversight of Veterans Integrated Service Networks (VISN)—regional networks responsible for oversight of VAMCs and CBOCs. However, VHA may miss CBOC performance problems that are not identifiable in VISN-level data. Further, although policy requires VHA central office officials to review CBOC performance as part of quarterly VISN performance reviews, officials said they do not specifically do so unless the VISN identifies a problem. Officials from three of the four VISNs in GAO's review said they largely delegate CBOC oversight to VAMCs, and do not separately review clinic performance unless a VAMC identifies a problem. An inaccurate and incomplete CBOC Report. VHA's CBOC Report is prepared by VHA central office and distributed to VISNs and VAMCs quarterly and at year-end. The CBOC Report could be useful to compare clinical quality of care between VHA-operated and contracted CBOCs, but it is inaccurate and incomplete. Specifically, VHA officials have used their judgment to classify certain sites as CBOCs in the report, rather than use the official classifications in policy. GAO found that 22 percent of sites were incorrectly classified as CBOCs when they were other types of sites, including VAMCs. As a result, the report is of limited usefulness to VHA as an oversight tool. Lack of guidance or training on the CBOC Report. VHA central office officials do not provide guidance or training specific to understanding the CBOC Report to assist VISNs and VAMCs in their oversight of CBOCs. GAO found that in several places in the report, shorthand text and acronyms were used, but not defined. In addition, several VISN and VAMC officials stated that guidance or training would be helpful. No requirement to use the CBOC Report. VHA officials told GAO that VAMCs and VISNs are expected to use the CBOC Report as an oversight tool, but GAO found that VHA lacks a requirement that they do so. Officials from three of the four VISNs and three of the four VAMCs in GAO's review were not using the report. These weaknesses potentially lead to inconsistent oversight and create a risk that VHA is not providing one standard of care that is of high quality to veterans across VHA-operated and contracted CBOCs. GAO recommends that VHA (1) implement oversight requirements that align with existing policy; (2) establish a process to ensure the CBOC Report is accurate and complete; (3) provide guidance or training to VISNs and VAMCs on how to use the CBOC Report; and (4) require use of the CBOC Report as an oversight tool. VA concurred with all of GAO's recommendations and identified actions it is taking to implement them.