VA Primary Care: Improved Oversight Needed to Better Ensure Timely Access and Efficient Delivery of Care
Released: 2015-10-22T00:00:00

Letter

Background

Within VA Central Office, VHA’s Primary Care Services Office develops policies related to the management of primary care—including the recording and reporting of primary care panel size data—and VHA’s Primary Care Operations Office is responsible for executing policies related to primary care delivery and monitoring primary care. VHA’s Office of Finance develops policies related to the recording and reporting of primary care encounter and expenditure data. Each of VA’s 21 networks is responsible for overseeing the facilities within their network, and this responsibility includes overseeing facilities’ management of primary care. (See fig. 1.)

Panel Size

Based on a review of studies, VA established a baseline panel size of 1,200 patients at any given time for a full-time primary care physician provider. The Primary Care Services Office adjusts the baseline panel size for each facility based on a model VA officials said they developed in 2003 that uses data reported by facilities—including data on the number of FTE providers, support staff, and exam rooms—and projections on the average number of primary care visits. These projections are based on patient characteristics, such as the proportion of patients with chronic conditions. VA refers to the adjusted baseline for each facility as the “modeled panel size,” which in fiscal year 2014 ranged from 1,140 to 1,338 across VA’s facilities. VA generally updates the modeled panel size annually for each facility.

VA’s handbook on primary care management requires that facilities record and report primary care data using the Primary Care Management Module (PCMM) software. These data include the number of patients, FTE providers, support staff, and exam rooms, and the reported and modeled panel size. Each facility maintains its own PCMM software and is required to update its panel size data on an ongoing basis in PCMM, which electronically reports facilities’ data to a separate national database maintained by the Veterans Support Service Center. This national database allows the Primary Care Operations Office and VA’s networks to review the data.

Encounters and Expenditures

An encounter is a professional contact between a patient and a provider who has the primary responsibility for diagnosing, evaluating, and treating the patient’s condition. In addition to individual office visits, there are other types of encounters, such as telephone visits and group visits. Each facility identifies and tracks all of its expenditures associated with primary care encounters. Facilities transmit their encounter and expenditure data using the Decision Support System, which is maintained by the Office of Finance. This office is responsible for collecting and maintaining financial information for VA’s cost accounting—which identifies and assesses the costs of programs at the national, network, and facility levels—and for budgetary purposes.

VA Lacks Reliable Data on Medical Facilities’ Primary Care Panel Sizes and Does Not Have Effective Oversight Processes to Verify and Use Facilities’ Data to Monitor Primary Care Data Reported by VA Facilities on Primary Care Panel Sizes Are Unreliable

We found that VA lacks reliable data on primary care panel sizes across its facilities because the data that facilities record and report to VA Central Office and networks are sometimes inaccurate. Because reliable reported panel sizes were not available for all facilities, we calculated actual panel sizes at six of seven selected facilities and compared them to each facility’s modeled panel size for fiscal year 2014. We found that actual panel sizes across the six facilities varied from 23 percent below to 11 percent above their respective modeled panel size. Moreover, we found that VA Central Office and networks do not have effective oversight processes for verifying and using facilities’ panel size data to monitor facilities’ management of primary care.

We found that VA lacks reliable data on primary care panel sizes across its 150 facilities because the data that facilities record in the PCMM software and report to the Primary Care Operations Office and to networks are sometimes inaccurate. Federal internal control standards state that reliable information is needed to determine whether an agency is meeting its goals for accountability for effective and efficient use of resources.

However, our review of the reported panel size data for all of VA’s facilities for fiscal year 2014 revealed missing values as well as values that appeared to be unreasonably high or low, which raised concerns about these data. Officials from the Primary Care Operations Office, whom we interviewed about the reliability of these data, agreed that inaccuracies exist in the way facilities report data elements in PCMM, such as the number of patients assigned to primary care panels and the number of FTE providers, support staff, and exam rooms. Primary Care Operations Office officials pointed out that because the data are self- reported, facilities can and sometimes do record the data inaccurately or in a manner that does not follow VA’s policy on panel management. For example, the officials stated that some facilities may not count support staff and exam rooms as outlined in VA’s policy. These officials also stated that PCMM has limitations that may affect the reliability of facilities’ reported panel size data. For example, officials explained that the software makes it difficult for facilities to ensure that inactive patients (i.e., those who have not seen their primary care provider within the preceding two years or have died) are removed from providers’ panels.

We identified similar inaccuracies in our more in depth review of panel size data reported by the seven selected facilities. Specifically, at three facilities we found inaccuracies in the reported number of FTE primary care providers and the reported number of patients, which impacted the facilities’ reported or modeled panel sizes. For example, the number of FTE primary care providers reported by one of these facilities was too low because the facility incorrectly recorded each FTE provider as only 90 percent of a FTE. We did not identify inaccuracies in the data reported by the remaining four facilities. (See table 1.)

Because some medical facilities’ reported panel size data are unreliable, VA Central Office and network officials cannot readily determine each facility’s average primary care panel size nor compare these panel sizes to each facility’s modeled panel size to help ensure that care is being delivered in a timely manner to a reasonable number of patients. Moreover, having unreliable data can misinform VA in other aspects as well. For example, because VA’s model is based on historical data reported by facilities, unreliable data may result in VA’s modeled panel size being too high or too low for certain facilities. Also, if facilities are using unreliable data to manage their primary care panels—for example, using the data to assign patients to primary care providers—the facilities may be misinformed about the available capacity on primary care providers’ panels—information that is key to determining facilities’ staffing and other resource needs.

Primary Care Operations Office officials told us that they intend to address data reliability issues over time. Specifically, the Primary Care Operations Office is in the process of implementing new software, called web-PCMM, which officials believe will address some concerns about the reliability of the data because the software features controls to help ensure that facilities record and report the data accurately and consistently. For example, web-PCMM will automatically remove inactive patients from providers’ panels. In preparation for the implementation of web-PCMM, Primary Care Operations Office officials said they have been training network and facility staff on the features and capabilities of the new software and instructing facility staff to review and correct their panel size data to help improve data accuracy. It is not yet known the extent to which the new software will actually address the data reliability issues because facilities will continue to self-report data. The Primary Care Operations Office started piloting the new software at selected facilities in 2014 and had planned to implement it agency-wide after resolving software interoperability issues identified during the pilot. However, officials said that implementation is currently on hold because of a lack of funding, and the officials could not provide an updated timeframe for its system-wide implementation. According to these officials, VA has spent about $8.8 million through July 2015 on the development and implementation of web-PCMM and requires an additional $1.5 million to implement it agency-wide.

Wide Variations Exist between Actual Panel Sizes and VA’s Modeled Panel Size at Medical Facilities We Reviewed

Because reliable data on reported panel sizes were not available for all of VA’s facilities at the time of our review, we calculated actual panel sizes at six of the seven selected facilities using updated data from these facilities and correcting for the inaccuracies we found at two facilities. We compared the actual panel size to each facility’s modeled panel size for fiscal year 2014. Although Primary Care Operations Office officials recommend that facilities keep panel sizes 10 to 15 percent below modeled panel sizes to accommodate growth and provider attrition, we found that actual panel sizes ranged from 23 percent below to 11 percent above their respective modeled panel size. This wide variation may indicate that actual panel sizes at some facilities are too low—potentially leading to inefficiency and wasted resources—or too high—potentially leading to veterans experiencing delays in obtaining care, among other negative effects. It may also indicate that VA’s modeled panel sizes are determined incorrectly based on unreliable facility data or do not sufficiently account for patient acuity levels and demand for primary care services. Actual average panel sizes across the six facilities ranged from a low of 1,000 patients per provider to a high of 1,338 patients per provider. (See fig. 2.)

At the three facilities where actual panel sizes were the highest of the six for which we calculated the actual panel sizes, officials cited three key factors that contributed to the higher panel sizes.

Growing patient demand: Officials at all three facilities stated that the growing number of patients seeking primary care services at their facilities has required them to assign a larger number of patients to each provider. Officials at one of these facilities stated that not assigning new patients to a panel would result in a greater number of walk-in patients seeking emergency care and a loss of continuity of care.

Staffing shortages: Officials at all three facilities described difficulty recruiting primary care providers, which resulted in a shortage of providers. At one of these facilities, about 40 percent of primary care provider positions were vacant at the time of our review. Officials at all three facilities attributed recruiting difficulties to the rural location of these facilities, lack of academic affiliation of the facilities, and the lower pay that VA offers primary care providers compared to nearby private sector medical facilities. In addition, at one of these facilities, officials stated that non-compete clauses limited the facility’s ability to hire providers currently working in the private sector who might otherwise seek employment with VA.

Exam room shortages: Officials at two of the three facilities stated that a lack of available exam room space has limited their ability to hire additional primary care providers—and thereby reduce panel sizes. They stated that the process for acquiring additional space—whether through building additional space or leasing it—is cumbersome and requires extensive preplanning. For example, at one of these facilities, officials stated that expanding the facility’s existing exam room space or opening another CBOC to accommodate growing demand for primary care typically takes 5 to 6 years. The officials told us that while the Veterans Access, Choice, and Accountability Act of 2014 provided facilities with funds to acquire additional space, it did not simplify the process for acquiring space.

Officials at two of the three facilities stated that the higher actual panel sizes have contributed to provider burnout and attrition. At one facility— where actual panel sizes were 11 percent above the modeled panel size—officials stated that the facility has been unable to hire enough providers to make up for attrition. The officials added that providers have expressed concerns to facility leadership that high panel sizes were impeding their ability to provide safe and effective patient care. All three facilities have taken measures to address higher actual panel sizes. For example, in order to ease staffing shortages the facilities have contracted with non-VA providers to provide care at VA facilities and have offered evening and weekend clinic hours to fully utilize available exam room space. However, while these measures have helped address capacity shortages at these facilities, they do not fully address the longstanding concerns resulting from higher panel sizes.

In contrast, at the facility where actual panel size was the lowest of the six we reviewed—23 percent below its modeled panel size—officials said they have made a concerted effort to establish lower panel sizes while increasing the number of primary care providers. Officials stated that they had recently lowered providers’ panel sizes because they believed that the modeled panel size did not sufficiently account for factors affecting patients’ demand for primary care services, such as high acuity levels. These officials noted that they previously followed the modeled panel size but found that it was too high and resulted in primary care provider burnout and poor patient access to primary care providers. Since VA Central Office and network staff generally do not examine differences across medical facilities VA-wide, it is unclear whether the facility with lower panel sizes for providers was providing primary care services in an inefficient manner or whether VA’s modeled panel size for this facility was too high.

VA Central Office and Networks Do Not Have Effective Oversight Processes for Verifying and Using Facilities’ Panel Size Data to Monitor Primary Care

VA Central Office and networks do not have effective oversight processes for verifying and using facilities’ panel size data to monitor facilities’ management of primary care. VA’s panel management policy requires facilities to ensure the reliability of their reported panel size data, but the policy does not assign oversight responsibility to VA Central Office or the networks for verifying the reliability of these data or for using the data for monitoring purposes. Federal internal control standards state that agencies should clearly define key areas of authority and responsibility, ensure that reliable information is available, and assess the quality of performance over time.

However, officials from the Primary Care Operations Office told us that— except for a few isolated situations—they do not verify the panel size data recorded in PCMM to systematically identify unreliable data or to monitor panel sizes across all VA medical facilities. For example, these officials told us that in 2014, they conducted reviews of three facilities that were struggling with recording and reporting reliable data in PCMM to identify ways to improve the reliability of the facilities reported data. The officials said they have not validated facilities’ reported panel size data or used the data to monitor primary care because the office has a limited number of staff and mainly relies on the networks and facilities to ensure that the data are recorded and reported correctly and that monitoring is conducted.

Across the seven networks that oversee the seven selected facilities for which we conducted a more in-depth analysis, we also identified variations in the extent to which the networks verified facilities’ panel size data and used the data to monitor and address panel sizes that were too high or too low. Specifically,

Data verification: Officials from four of the seven networks told us that they took some steps to verify that facilities’ panel size data were reliable, such as reviewing the data for errors and large variations. For example, officials from one of these networks stated that if they identified large variability in the number of exam rooms—a relatively stable data element over time—it could indicate problems with data reliability, which the network officials would discuss with officials from the facility reporting the data. Officials from another network stated that they compared data reported by facilities to data previously reported by the facilities to identify large variations. Officials from the remaining three networks told us that they did not any take steps to verify that facilities’ reported panel size data were reliable. According to Primary Care Operations Office officials, VA networks can request access to facilities’ PCMM software, which would enable them to verify the data; however, the officials acknowledged that many of VA’s 21 networks are unaware of this capability.

Use of data for monitoring primary care: Officials from six of the seven networks said they discussed reported panel size data during monthly calls with facility officials, at primary care committee meetings, or during facility site visits. However, officials from only four of these six networks stated that they took steps to address panel sizes that are too high or too low compared to a facility’s respective modeled panel size. For example, officials at one network told us that they helped a facility recruit additional primary care providers to address high panel sizes. In another network, officials said that they were helping a facility secure additional exam room space to address high panel sizes. Officials at a third network told us that they recently had to curtail monitoring activities to address facilities’ panel sizes due to staffing shortages. In contrast, officials from the one network that does not use panel size data to monitor facilities’ management of primary care told us that they rely on the facilities to manage their own primary care panels and do not believe that the network should take an active role in this process. As a result, officials from this network were unaware that a facility within their network had made a concerted effort to establish panel sizes that were well below its modeled panel size.

Absent a robust oversight process that assigns responsibility, as appropriate, to VA Central Office and networks for verifying facilities’ panel size data and using the data to monitor facilities’ management of primary care—such as, examining wide variations from modeled panel sizes—VA lacks assurance that facilities’ data are reliable and that they are managing primary care panels in a manner that meets VA’s goals of providing efficient, timely, and quality care to veterans. Primary Care Operations Office officials stated that VA Central Office is in the process of revising its policy on primary care panel management and is developing additional guidance to require VA Central Office and VA networks to verify reported panel size data in addition to other monitoring responsibilities. However, as the revised policy and guidance are still under development, it is unknown when they will be implemented and whether they will fully address the issues we identified.

VA’s Encounter and Expenditure Data Show Wide Variation across Medical Facilities, and VA Central Office and Networks Do Not Use These Data to Monitor Primary Care

Expenditures per Primary Care Encounter Varied Widely across VA Facilities

Based on our review of fiscal year 2014 VA-wide primary care expenditure and encounter data, we found that expenditures per primary care encounter varied widely across VA facilities, from a low of $150 to a high of $396, after adjusting to account for geographic differences in labor costs. Expenditures per encounter at 97 of the 140 facilities we reviewed were within $51 or one standard deviation—a statistical measure of variance—of VA’s overall average of $242. According to officials from VHA’s Office of Finance, one standard deviation is typically used to identify potential outliers when examining encounter and expenditure data. For the remaining 43 facilities, our analysis found that expenditures per encounter at 20 facilities were at least one standard deviation above the average and at 23 facilities were at least one standard deviation below, which may indicate potential outliers that VA Central Office and the networks may need to examine further. (See fig. 3.) Among other things, this variation may indicate that primary care is being delivered efficiently at facilities with relatively low expenditures per encounter or inefficiently at facilities with relatively high expenditures per encounter. We also analyzed expenditures per unique primary care patient—that is, a patient with at least one primary care encounter in fiscal year 2014— and found similar variation across VA’s facilities. (See app. I.) We found that this variation remained when examining expenditures by encounter and per unique patient for facilities within the same complexity group.

Of the seven selected facilities, one was among the least expensive facilities across all VA facilities and another was among the most expensive, in terms of expenditures per primary care encounter. An official from the facility that was among the least expensive of the seven we reviewed, with expenditures per encounter of $158, identified an increased use of secure messaging and telephone primary care as primary factors that contributed to a lower expenditure per encounter. Officials from the network that oversees the facility that was among the most expensive of the seven we reviewed, with expenditures per encounter of $330, identified the high cost of living in the area—which resulted in higher leasing and labor costs—as the primary factor that contributed to a higher than average cost per encounter. However, our analysis largely accounted for the higher cost of living in that expenditure data provided by VA were adjusted to account for geographic differences in labor costs, which made up 71 percent of this facility’s costs in fiscal year 2014. The officials also explained that part of the reason for the high expenditures per encounter was that the facility was not appropriately accounting for telephone-based primary care services it provided for the entire network. As a result, primary care encounters and expenditures for the selected facility included encounters and expenditures for telephone primary care services for other facilities within the network. According to network officials, steps are being taken to ensure that the facility is allocating these expenditures appropriately going forward.

VA Central Office and Networks Verify and Use Facilities’ Encounter and Expenditure Data for Financial Purposes, But Not to Monitor Primary Care

While VA Central Office and networks verify and use facilities’ encounter and expenditure data for financial purposes, VA’s policies governing primary care do not require VA Central Office and networks to use these data to monitor facilities’ management of primary care. Federal internal control standards state that agencies need both operational and financial data to determine whether they are meeting strategic goals and should use such data to assess the quality of performance over time.

We found that the Office of Finance in VA Central Office independently verifies facilities’ encounter and expenditure data to help ensure their reliability and uses the data for cost accounting and budgetary purposes. Similarly, chief financial officers or their designees at six of the seven networks that oversee the facilities we reviewed routinely examine encounter and expenditure data to identify outliers for the purposes of ensuring data reliability and for cost accounting. However, the Primary Care Operations Office in VA Central Office does not use encounter and expenditure data, even though officials stated that examining such data would likely help them monitor facilities’ management of primary care. Furthermore, primary care officials at the seven networks we examined generally do not use these data to monitor facilities’ management of primary care. Some officials told us that they do not use encounter and expenditure data for monitoring primary care delivery because panel sizes are the most effective means of measuring efficiency within primary care.

By not using encounter and expenditure data to monitor facilities’ management of primary care, VA may be missing opportunities to identify facilities—such as those that experience higher than average expenditures per encounter or significant changes in expenditures over time—that may warrant further examination and to strengthen the efficiency and effectiveness of the primary care program. Using panel size data in conjunction with encounter and expenditure data, would allow VA Central Office and networks to assess facilities’ capacity to provide primary care services and the efficiency of care delivery.

Conclusions

The absence of reliable panel size data and oversight processes could significantly inhibit VA’s ability to ensure that facilities are providing veterans with timely, quality care and delivering that care efficiently. While VA planned to address some of the data reliability issues through new software to help VA facilities record data more accurately, development of this software is currently on hold, and VA could not provide any estimates of when the software would be implemented at its facilities. Even if this software is implemented, VA Central Office and networks will still be relying on self-reported data on primary care panel sizes from its facilities. By not having in place a process to verify the reliability of facilities’ panel size data or to monitor wide variations between facilities’ reported and modeled panel sizes, VA will likely continue to receive unreliable data and miss opportunities to assess the impact of panel sizes on veterans’ access to care. VA Central Office and the networks are also missing opportunities to use readily available encounter and expenditure data to potentially improve the efficiency of primary care service delivery. Consistent with federal internal control standards, using such data in conjunction with reliable panel size data could be a potent tool in “right- sizing” panel sizes to best serve veterans’ needs and deliver primary care efficiently.

Recommendations for Executive Action

We recommend that the Secretary of the Department of Veterans Affairs, direct the Undersecretary for Health to take the following two actions to improve the reliability of VA’s primary care panel size data and improve VA Central Office and the networks’ oversight of facilities’ management of primary care: Incorporate in policy an oversight process for primary care panel management that assigns responsibility, as appropriate, to VA Central Office and networks for (1) verifying each facility’s reported panel size data currently in PCMM and in web-PCMM, if the software is rolled- out nationally, including such data as the number of primary care patients, providers, support staff, and exam rooms; and (2) monitoring facilities’ reported panel sizes in relation to the modeled panel size and assisting facilities in taking steps to address situations where reported panel sizes vary widely from modeled panel sizes.

Review and document how to use encounter and expenditure data in conjunction with panel size data to strengthen monitoring of facilities’ management of primary care.

Agency Comments and Our Evaluation

VA provided written comments on a draft of this report, which we have reprinted in appendix II. In its comments, VA agreed with our conclusions, concurred with our two recommendations, and described the agency’s plans to implement our recommendations. VA also provided technical clarifications and comments on the draft report, including the recommendations contained in the draft report. We incorporated these comments, as appropriate. In particular, we modified our first recommendation in the draft report and now recommend that VA verify each facility’s panel size data in PCMM and, if the latter is available, in web-PCMM. We made this change to reflect the continued uncertainty over the implementation of the web-PCMM software. In addition, we modified our second recommendation in the draft report and no longer recommend VA incorporate into existing VA policy a requirement that the agency and its networks use encounter and expenditure data to strengthen the monitoring of facilities’ management of primary care. We made this change to reflect that VA officials were not prepared to incorporate such a requirement without first examining how to use these data for monitoring purposes.

To address our first recommendation, VA stated that it plans to issue guidance by September 2016 clarifying VA Central Office’s and the networks’ oversight responsibilities with regard to primary care panel size data. This guidance will include a process—developed by the Offices of Primary Care Services and Primary Care Operations—for addressing medical facilities whose panel sizes differ significantly from similar facilities’ panels. In its response, however, VA did not provide information on how it plans to address unreliable panel size data facilities record and report in PCMM. We would encourage VA, in the guidance it plans to issue in 2016, to assign responsibility for verifying each facility's reported panel size data as we recommended. To address our second recommendation, VA stated that it will take steps to understand encounter and expenditure data and determine how best to utilize these data to improve patient care with a target completion date for presenting its findings and decisions by September 2018.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 14 days from the report date. At that time, we will send copies to the appropriate congressional committees and the Secretary of Veterans Affairs. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-7114 or williamsonr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs are on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Expenditures per Unique Primary Care Patient in Fiscal Year 2014

We analyzed Department of Veterans Affairs (VA) fiscal year 2014 data on primary care expenditures and calculated expenditures per unique primary care patient. We found that expenditures per unique primary care patient varied widely across facilities in fiscal year 2014, ranging from $558 to $1,544 after adjusting to account for geographic differences in labor costs across facilities. We found that the expenditures per unique patient at 102 of the 140 facilities we reviewed were within $167 or one standard deviation—a statistical measure of variance—of VA’s overall average of $871. For the remaining facilities, expenditures per unique patient were at least one standard deviation above the average (19 facilities) or were at least one standard deviation below the average (19 facilities), which may indicate potential outliers that VA Central Office and the networks may need to examine further. (See fig. 4.)

Appendix II: Comments from the Department of Veterans Affairs

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Rashmi Agarwal, Assistant Director; James Musselwhite, Assistant Director; Kathryn Black; Krister Friday; Cathleen Hamann; Aaron Holling; Emily Wilson; and Michael Zose made key contributions to this report.

Related GAO Products

Department of Veterans Affairs: Expanded Access to Non-VA Care Through the Veterans Choice Program. GAO-15-229R. Washington, D.C.: Nov 19, 2014.

VA Health Care: Actions Needed to Ensure Adequate and Qualified Nurse Staffing. GAO-15-61. Washington, D.C.: Oct 16, 2014.

VA Health Care: Ongoing and Past Work Identified Access, Oversight, and Data Problems That Hinder Veterans’ Ability to Obtain Timely Outpatient Medical Care. GAO-14-679T. Washington, DC: Jun 9, 2014.

VA Health Care: VA Lacks Accurate Information about Outpatient Medical Appointment Wait Times, Including Specialty Care Consults. GAO-14-620T. Washington, D.C.: May 15, 2014.

VA Health Care: Ongoing and Past Work Identified Access Problems That May Delay Needed Medical Care for Veterans. GAO-14-509T. Washington, D.C.: Apr 9, 2014.