VA PRIMARY CARE

Improved Oversight 

Needed to Better Ensure 

Timely Access and 

Efficient Delivery of Care

Statement of Randall B. Williamson, 

Director, Health Care

Testimony 

Before the Subcommittee on Health, 

Committee on Veterans Affairs, House 

of Representatives

For Release on Delivery 

Expected at 10:00 a.m. ET

Thursday, October 22, 2015

GAO-16-114T

United States Government Accountability Office
Letter

Chairman Benishek, Ranking Member Brownley, and Members of the 

Subcommittee:

I am pleased to be here today to discuss our report on the provision and 

oversight of primary care services at Department of Veterans Affairs (VA) 

medical facilities, which is being publicly released today.1 As you know, 

primary care services are often the entry point to the VA health care 

system for most veterans, including an aging veteran population and a 

growing number of younger veterans returning from military operations in 

Afghanistan and Iraq. Veterans may obtain primary care services at VA’s 

150 medical facilities, which include medical centers and more than 800 

community based outpatient clinics (CBOC).2 Primary care services are 

delivered through patient aligned care teams consisting of a primary care 

provider and support staff—a nurse care manager, clinical associate, and 

administrative clerk.3 When other services are needed to meet patient 

goals and needs, these teams oversee and coordinate that care. VA’s 

medical facilities are overseen by 21 Veterans Integrated Service 

Networks—called networks–and by VA’s Central Office, which oversees 

the entire VA health care system.4

In recent years, VA has emphasized providing primary care as the way to 

enhance health care delivery to meet veterans’ needs. In support of this 

emphasis, VA has detailed guidelines regarding how primary care is to be 

provided and staffed, including a requirement that medical facilities record 

and report data on primary care panel size—that is, the number of 

patients for whom a patient aligned care team delivers primary care 

services.5 According to VA, panel size is an important factor in helping VA 

determine the total number of patients that can be cared for in the VA 

1

GAO, VA Primary Care: Improved Oversight Needed to Better Ensure Timely Access and 

Efficient Delivery of Care, GAO-16-83 (Washington, D.C.: Oct. 8, 2015).

2

VA medical facilities manage primary care clinics located within their respective medical 

centers and associated CBOCs. CBOCs provide outpatient primary care and general 

mental health services on site.

3

Primary care providers are physicians, nurse practitioners, or physician assistants. 

4

VA’s 21 Veterans Integrated Service Networks oversee the day-to-day functions of 

facilities that are within their network. 

5

See VHA Handbook 1101.02, Primary Care Management Module (PCMM) (April 29, 

2009) and VHA Handbook 1101.10, Patient Aligned Care Team (PACT) Handbook

(February 5, 2014). 

Letter

Page 1 GAO-16-114T 

 
health care system; moreover, maintaining appropriate panel sizes helps 

ensure that providers will be able to offer quality care in a timely manner 

to a reasonable number of patients. Panel sizes that are too high may 

lead to veterans experiencing delays in obtaining care among other 

negative effects, while panel sizes that are too low may be associated 

with inefficiency and wasted resources. Reliable data on panel size can 

also help VA facilities manage and VA Central Office and networks 

oversee VA’s primary care program. In addition to data on panel sizes, 

VA policy also requires facilities to record and report data on primary care 

encounters—that is, the number and types of visits to a primary care 

provider—and expenditures for delivering primary care. According to VA 

officials, data on encounters and expenditures may help VA oversee its 

primary care program by identifying potential inefficiencies.

In this context, my testimony today discusses the findings from our report 

analyzing VA’s panel size, encounter, and expenditure data. Accordingly, 

this testimony addresses (1) what VA’s panel size data show across 

facilities and how VA Central Office and networks use these data to 

oversee primary care, and (2) what VA’s encounter and expenditure data 

show across facilities and how VA Central Office and networks use these 

data to oversee primary care. In addition, I will highlight two key actions 

that we recommended in our report that VA can take to improve the 

reliability of its primary care panel size data and improve VA Central 

Office and the networks’ oversight of facilities’ management of primary 

care.

To conduct our work, among other things, we analyzed fiscal year 2014 

data for all VA facilities on 1) modeled panel size, which represents the 

number of patients on a primary care panel for whom a patient aligned 

care team is expected to deliver primary care, as projected by VA for 

each facility, 2) facility-reported panel size, which represents the average 

number of patients on a primary care panel; 3) the number of primary 

care patients, providers, support staff, and exam rooms reported by each 

facility; and 4) expenditures per primary care encounter. In addition, we 

conducted a more detailed, nongeneralizable examination of these data 

at seven VA facilities, which we selected based on geographic diversity 

Page 2 GAO-16-114T 
and differences in facility complexity.6 Our review revealed concerns 

about facilities’ reported panel sizes and other data elements that were 

attributable to inaccuracies in how facilities recorded the data, which 

precluded us from using these data to report panel sizes across all VA 

facilities. As a result of these data inaccuracies, we calculated actual 

panel sizes for six of the seven selected facilities where we were able to 

use updated data provided by each facility and correct the data for 

inaccuracies.7 In addition, we reviewed VA policy documents and 

interviewed officials from the Veterans Health Administration’s (VHA) 

Primary Care Operations Office and Office of Finance, within VA Central 

Office, as well as officials from selected facilities and their associated 

networks.8 We also assessed oversight activities for VA Central Office 

and across the networks we reviewed in the context of federal internal 

control standards for control environment, information and 

communication, and monitoring.9 The work this statement is based on 

was performed in accordance with generally accepted government 

auditing standards. Further details on our scope and methodology are 

included in our report.

6

VA assigns each facility to one of five complexity groups (1a, 1b, 1c, 2, and 3) using a 

facility complexity model where level 1a facilities are the most complex and level 3 

facilities are the least complex. VA’s complexity model uses multiple variables to measure 

facility complexity in four categories: patient population served, clinical services offered, 

education and research complexity, and administrative complexity.

7

We calculated actual panel sizes based on updated data each facility provided to us at 

the time of our review from November 2014 through March 2015. We verified the reliability 

of the data each facility provided us by checking for missing values and outliers as well as 

interviewing facility officials knowledgeable about the data. For one of the seven facilities, 

we were unable to correct for the inaccurate number of full-time equivalent providers.

Therefore, we did not use or report on the panel sizes for this facility and instead report on 

six facilities.

8

VA policy documents we reviewed included VA Handbook 1101.02, Primary Care 

Management Module (PCMM) (April 29, 2009) and VHA Handbook 1101.10, Patient 

Aligned Care Team (PACT) Handbook (February 5, 2014). VHA’s Primary Care 

Operations Office is responsible for executing policies related to primary care delivery and 

for monitoring primary care. VHA’s Office of Finance develops policies related to the 

recording and reporting of primary care encounter and expenditure data.

9

See GAO, Standards for Internal Control in the Federal Government, 

GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999) and GAO, Internal Control 

Management and Evaluation Tool, GAO-01-1008G (Washington, D.C.: August 2001).

Page 3 GAO-16-114T 

 
We found that VA data on primary care panel sizes are unreliable across 

VA’s 150 medical facilities. Federal internal control standards state that 

reliable information is needed to determine whether an agency is meeting 

its goals for accountability for effective and efficient use of resources. 

However, in contrast with these standards, we found missing values and 

other inaccuracies in VA’s 2014 data, such as values that appeared too 

high or too low. Officials from VHA’s Primary Care Operations Office 

confirmed that facilities sometimes record and self-report these data 

inaccurately or in a manner that does not follow VA’s policy and noted 

that this could result in the data reliability concerns we identified. Because 

some medical facilities’ reported panel size data are unreliable, VA 

Central Office and network officials cannot readily determine each 

facility’s average primary care panel size nor compare these panel sizes 

to each facility’s modeled panel size to help ensure that care is being 

delivered in a timely manner to a reasonable number of patients.

Primary Care Operations Office officials told us that they intend to 

address data reliability issues over time. Specifically, the Primary Care 

Operations Office is in the process of implementing new software, called 

web-Primary Care Management Module (PCMM), which officials believe 

will address some concerns about the reliability of the data because the 

software features controls to help ensure that facilities record and report 

the data accurately and consistently. In preparation for the 

implementation of web-PCMM, Primary Care Operations Office officials 

said they have been training network and facility staff on the features and 

capabilities of the new software and instructing facility staff to review and 

correct their panel size data to help improve data accuracy. It is not yet 

known the extent to which the new software will address the data 

reliability issues we identified because facilities will continue to self-report 

data. The Primary Care Operations Office started piloting the new 

software at selected facilities in 2014 and had planned to implement it 

agency-wide after resolving software interoperability issues identified 

during the pilot. However, officials said that implementation is currently on 

hold because of a lack of funding, and the officials could not provide an 

updated timeframe for its system-wide implementation. According to 

these officials, VA has spent about $8.8 million through July 2015 on the 

development and implementation of web-PCMM and requires an 

additional $1.5 million to implement it agency-wide.

VA Lacks Reliable 

Data on Medical 

Facilities’ Primary 

Care Panel Sizes and 

Does Not Have 

Effective Oversight 

Processes to Verify 

and Use Facilities’

Data to Monitor 

Primary Care

Page 4 GAO-16-114T 
Although reliable data on reported panel sizes were not available for all of 

VA’s facilities at the time of our review, we were able to obtain updated 

data from six of seven selected facilities and calculate the actual panel 

sizes for those six facilities.10 We found that for these six facilities, the 

actual panel size varied from 23 percent below to 11 percent above the 

modeled panel size. (See fig. 1.) Such wide variation raises questions 

about whether veterans are receiving access to timely care and the 

appropriateness of the size of provider workload at these facilities. 

Officials we interviewed at the three facilities where actual panel sizes 

were the highest cited growing patient demand, staffing shortages, and 

exam room shortages as factors contributing to higher panel sizes.

Figure 1: Comparison of Six Selected Department of Veterans Affairs (VA) Medical 

Facilities’ Actual Primary Care Panel Sizes to Their Fiscal Year 2014 Modeled Panel 

Sizes

10We did not calculate the actual panel size for the remaining selected facility because we 

were unable to correct for inaccurate data on full-time equivalent providers reported by 

this facility.

Page 5 GAO-16-114T 

 
Note: We were unable to compare reported panel sizes to fiscal year 2014 modeled panel sizes at all 

VA facilities because of inaccuracies we identified in facilities’ reported panel size data. For six 

selected facilities, we calculated actual panel sizes based on updated data obtained from each facility 

from November 2014 through March 2015 that we verified as reliable.

Moreover, we found that while VA’s primary care panel management 

policy requires facilities to ensure the reliability of their panel size data, it 

does not assign responsibility to VA Central Office or networks for 

verifying the reliability of facilities’ data or require the facilities to use the 

data for monitoring purposes. Primary Care Operations officials said that 

they have not validated facilities’ reported panel size data or used the 

data to monitor primary care because the office has a limited number of 

staff and mainly relies on the networks and facilities to ensure that the 

data are recorded and reported correctly and that monitoring is 

conducted. This practice is inconsistent with federal internal control 

standards that call for agencies to clearly define key areas of authority 

and responsibility, ensure that reliable information is available, and use 

this information to assess the quality of performance over time. Primary 

Care Operations Office officials stated that VA Central Office is in the 

process of revising its policy for primary care panel management and is 

developing additional guidance to require VA Central Office and VA 

networks to verify reported panel size data in addition to other monitoring 

responsibilities. However, as the revised policy and guidance are still 

under development, it is unknown when they will be implemented and 

whether they will fully address the issues we identified.

In our report, we concluded that the absence of reliable panel size data 

and oversight processes could significantly inhibit VA’s ability to ensure 

that facilities are providing veterans with timely, quality care that is 

delivered efficiently. To address these shortcomings, we recommended 

that VA incorporate in policy an oversight process for primary care panel 

management that assigns responsibility, as appropriate, to VA Central 

Office and networks for (1) verifying each facility’s reported panel size 

data currently in PCMM—and in web-PCMM, if the software is rolled-out 

nationally—including such data as the number of primary care patients, 

providers, support staff, and exam rooms; and (2) monitoring facilities’

reported panel sizes in relation to the modeled panel size and assisting 

facilities in taking steps to address situations where reported panel sizes 

vary widely from modeled panel sizes. VA agreed with this 

recommendation and said it plans to take steps to implement it, including 

issuing guidance by September 2016 clarifying VA Central Office and 

networks’ oversight responsibilities with regard to primary care panel size 

data. Given the importance of these panel size data, we will continue to 

monitor VA’s progress in implementing the recommendation.

Page 6 GAO-16-114T 
In contrast to VA’s panel data, we found that primary care encounter and 

expenditure data reported by all VA medical facilities are reliable, 

although the data show wide variations across facilities.11 For example, in 

fiscal year 2014, expenditures per primary care encounter ranged from a 

low of $150 to a high of $396 after adjusting to account for geographic 

differences in labor costs across facilities. (See fig. 2.) We also found that 

expenditures per encounter at about 40 facilities were potential outliers 

that VA Central Office and the networks may need to examine further.12

We also analyzed expenditures per unique primary care patient—that is, 

a patient with at least one primary care encounter in fiscal year 2014—

and found similar variation across VA’s facilities. Such wide variations 

may indicate that services are being delivered inefficiently at some 

facilities with relatively higher per encounter costs compared to other 

facilities.

11Data for some of VA’s 150 facilities that are part of a larger hospital system were 

combined with another facility’s data. As a result, the expenditure and encounter data that 

VA provided included 140 facilities. The analysis includes expenditure and encounter data 

for traditional and telephone primary care. 

12According to officials from VHA’s Office of Finance, one standard deviation is typically 

used to identify potential outliers when examining encounter and expenditure data. Our 

analysis found that expenditures per encounter at 20 facilities were at least one standard 

deviation above the average and at 23 facilities were at least one standard deviation 

below the average.

VA’s Encounter and 

Expenditure Data 

Show Wide Variation 

across Medical 

Facilities, and VA 

Central Office and 

Networks Do Not Use 

These Data to 

Monitor Primary Care

Page 7 GAO-16-114T 

 
Figure 2: Department of Veterans Affairs (VA) Expenditures per Encounter by Facility Adjusted for Geographic Variance in 

Labor Costs, Fiscal Year 2014

Note: Data for some of VA’s 150 facilities that are part of a larger hospital system were combined with 

another facility’s data. As a result, the expenditure and encounter data that VA provided included 140 

facilities. The analysis includes expenditure and encounter data for traditional and telephone primary 

care.

While VA verifies and uses these data for cost accounting and budgetary 

purposes, VA’s policies governing primary care do not require the use of 

the data to monitor facilities’ management of primary care. In contrast, 

federal internal control standards state that agencies need both 

operational and financial data to determine whether they are meeting 

strategic goals and should use such data to assess the quality of 

performance over time. By not using available encounter and expenditure 

data in this manner, we concluded that VA is missing an opportunity to 

potentially identify inefficient practices or processes at facilities and 

improve the overall efficiency of primary care service delivery. Using 

panel size data in conjunction with encounter and expenditure data would 

allow VA to assess facilities’ capacity to provide primary care services 

and the efficiency of their care delivery.

Page 8 GAO-16-114T 
To address this shortcoming, our report recommended that VA review 

and document how to use encounter and expenditure data in conjunction 

with panel size data to strengthen monitoring of facilities’ management of 

primary care. VA concurred with this recommendation and said it plans to 

take steps to fully implement it with a target completion date for 

presenting its findings and decisions by September 2018. Given the 

significant lead time that VA has indicated it needs to implement this 

recommendation, we will continue to actively monitor its progress.

Chairman Benishek, Ranking Member Brownley, and Members of the 

Subcommittee, this concludes my statement. I would be pleased to 

respond to any questions that you may have at this time.

If you or your staff members have any questions concerning this 

testimony, please contact me at (202) 512-7114 (williamsonr@gao.gov). 

Contact points for our Offices of Congressional Relations and Public 

Affairs may be found on the last page of this statement. Other individuals 

who made key contributions to this testimony include Rashmi Agarwal, 

Assistant Director; Aaron Holling; and Michael Zose.

GAO Contacts & Staff

Acknowledgments

(100318) Page 9 GAO-16-114T 
This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. 

Listen to our Podcasts and read The Watchblog.

Visit GAO on the web at www.gao.gov.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.