Page i GAO-18-356 GAO-18-356 VHA Patient Advocacy Program

VA HEALTH CARE

Improved Guidance 

and Oversight 

Needed for the 

Patient Advocacy 

Program

Report to Congressional Committees

April 2018

GAO-18-356

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-18-356 a report to 

congressional committees

April 2018

VA HEALTH CARE

Improved Guidance and Oversight Needed for the 

Patient Advocacy Program

What GAO Found

The Veterans Health Administration (VHA) provided limited guidance to 

Department of Veterans Affairs (VA) medical centers (VAMC) on the governance 

of its patient advocacy program and its guidance, a program handbook, has 

been outdated since 2010. VAMCs are still expected to follow the outdated 

handbook, which does not provide needed details on governance, such as 

specifying the VAMC department to which patient advocates should report.

Officials from most of the VAMCs that GAO reviewed noted that the VAMC 

department to which patient advocates report can have a direct effect on the 

ability of staff to resolve veterans’ complaints. The lack of updated and complete 

guidance may impede the patient advocacy program from meeting its

expectations, to receive and address complaints from veterans in a convenient 

and timely manner.

VHA also has provided limited guidance to VAMCs on staffing the patient 

advocacy program. VHA’s handbook states that every VAMC should have at 

least one patient advocate and appropriate support staff; however, it did not 

provide guidance on how to determine the number and type of staff needed.

Officials at all but one of the eight VAMCs in GAO’s review stated that their 

patient advocacy program staff had more work to do than they could accomplish. 

This limited guidance on staffing could impede VAMCs’ efforts to ensure that 

they have the appropriate number and type of staff to address veterans’ 

complaints in a timely manner.

Further, VHA has recommended training for patient advocates, but it has not 

developed an approach to routinely assess their training needs or monitored 

training completion. VHA officials stated that they relied on VAMC and Veterans 

Integrated Service Network (VISN) staff to conduct these activities. However, 

GAO found that for the eight VAMCs in its review, the training needs of patient 

advocates were not routinely assessed, and training completion was not always 

monitored. Without conducting these activities, VHA increases its risk that staff 

may not be adequately trained to advocate on behalf of veterans.

Finally, VHA has not monitored patient advocacy program data-entry practices or 

reviewed the data to assess program performance. VHA officials stated that they 

relied on VISN and VAMC officials to ensure that all complaints were consistently 

entered into VHA’s Patient Advocate Tracking System (PATS). However, GAO 

identified inconsistencies in the extent to which VAMC officials did so. VHA’s lack 

of monitoring may pose a risk that not all complaints are entered into this 

tracking system—a goal of the program. Additionally, VHA officials stated they 

did not systemically review data in the system to assess program performance 

and identify potential system-wide improvements because VHA considered this 

the responsibility of VAMCs. As a result, VHA officials may miss opportunities to 

improve veterans’ experiences.

VHA is beginning to address many of these governance, staffing, training, and 

data issues, including directing a workgroup to provide recommendations by 

spring of 2018. However, because the recommendations will be advisory, and 

because program deadlines have slipped in the past, the nature and timing of the 

actions needed to resolve these issues remain unclear.

View GAO-18-356. For more information, 

contact Debra A. Draper at (202) 512-7114 or 

draperd@gao.gov.

Why GAO Did This Study

VHA has designated patient advocates 

at each VAMC to receive and 

document feedback from veterans or 

their representatives, including 

requests for information, compliments, 

and complaints. In recent years, the 

importance of a strong patient 

advocacy program has taken on new 

significance given concerns with VHA’s 

ability to provide veterans timely 

access to health care, among other 

issues.

The Comprehensive Addiction and 

Recovery Act of 2016 included a 

provision for GAO to review VHA’s 

patient advocacy program. This report 

examines the extent to which VHA has 

(1) provided guidance on the 

governance of the program; (2) 

provided guidance on staffing the 

program; (3) assessed the training 

needs of patient advocates and 

monitored training completion; and (4) 

monitored patient advocacy program 

data-entry practices and reviewed 

program data. GAO reviewed VHA and 

VAMC documents, including 

summaries of program data. GAO 

interviewed VHA officials about the 

program, as well as officials from a 

non-generalizable selection of eight 

VAMCs and five VISNs selected based 

on the volume of veteran complaints 

and other factors. GAO also compared 

VHA policies and practices to federal 

internal control standards.

What GAO Recommends

GAO is making 6 recommendations to 

improve guidance for and oversight of 

the patient advocacy program, 

focusing on governance, staffing, 

training, and PATS data entry and 

assessment. VA concurred with GAO’s

recommendations.
Page i GAO-18-356 VHA Patient Advocacy Program 

Letter 1

Background 6

VHA Has Provided Limited, Outdated Guidance to VAMCs on the 

Governance of the Patient Advocacy Program 10

VHA Has Provided Limited Guidance to VAMCs on Staffing the 

Patient Advocacy Program 12

VHA Has Recommended Training for Patient Advocates, but Has 

Not Developed an Approach to Routinely Assess Their Training 

Needs or Monitored Training Completion 15

VHA Has Not Monitored Patient Advocacy Data-Entry Practices or 

Reviewed Patient Advocacy Data to Assess Program 

Performance and Identify System-Wide Improvements 18

Conclusions 22

Recommendations for Executive Action 23

Agency Comments and Our Evaluation 23

Appendix I Awareness of the Patient Advocacy Program 25

Appendix II Patient Advocate Tracking System (PATS) Data Entry 

and Management 27

Appendix III Approaches to the Governance and Staffing of the 

Patient Advocacy Program 36

Appendix IV Comments from the Department of Veterans Affairs 38

Appendix V GAO Contact and Staff Acknowledgments 41

Tables

Table 1: Selected Department of Veterans Affairs Medical Centers 

(VAMC) and Associated Veterans Integrated Service 

Networks (VISN) Included in Review 4

Contents
Page ii GAO-18-356 VHA Patient Advocacy Program 

Table 2: Issue Codes Used to Describe Veterans’ Feedback in the 

Veterans Health Administration (VHA) Patient Advocate 

Tracking System (PATS), Fiscal Year (FY) 2017 30

Table 3: The Most Common Issues Entered in the Patient 

Advocate Tracking System (PATS), according to the 

Department of Veterans Affairs (VA), Fiscal Year 2017 33

Table 4: Patient Advocate Tracking System (PATS) Data-Entry 

Practices at Eight Selected Department of Veterans 

Affairs Medical Centers (VAMC), as of September 2017 34

Table 5: The Number of Positions and Titles Used for Patient 

Advocates and Service-Level Advocates at Eight Selected 

Department of Veterans Affairs Medical Centers (VAMC), 

as of September 2017 36

Figures

Figure 1: Process at Department of Veterans Affairs Medical 

Centers (VAMC) for Entering and Closing Veterans’ 

Feedback in the Patient Advocate Tracking System 8

Figure 2: Examples of Patient Advocacy Program Signage at 

Selected Department of Veterans Affairs Medical Centers 26

Figure 3: Example of Information Patient Advocacy Program Staff 

at VAMCs Enter in the Patient Advocate Tracking System 

(PATS) to Add a Report of Contact (ROC) 28

Figure 4: Example of Information Patient Advocacy Program Staff 

at Department of Veterans Affairs Medical Centers 

(VAMC) Enter in the Patient Advocate Tracking System 

to Close a Report of Contact (ROC) 29
Page iii GAO-18-356 VHA Patient Advocacy Program 

Abbreviations

CARA Comprehensive Addiction and Recovery Act 

of 2016

FY fiscal year

OIG Office of Inspector General

OPA Office of Patient Advocacy

OPCC&CT Office of Patient Centered Care and 

Cultural Transformation

PATS Patient Advocate Tracking System

ROC report of contact

VA Department of Veterans Affairs

VAMC VA medical center

VHA Veterans Health Administration

VISN Veterans Integrated Service Network

VSO veterans service organization

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-18-356 VHA Patient Advocacy Program 

441 G St. N.W.

Washington, DC 20548

April 12, 2018

The Honorable Johnny Isakson

Chairman

The Honorable Jon Tester

Ranking Member

Committee on Veterans’ Affairs

United States Senate

The Honorable Phil Roe, M.D.

Chairman 

The Honorable Tim Walz 

Ranking Member

Committee on Veterans’ Affairs

House of Representatives

The Veterans Health Administration (VHA), within the Department of 

Veterans Affairs (VA), operates one of the nation’s largest health care 

systems. VA has faced a growing demand by veterans for its health care 

services due, in part, to (1) servicemembers returning from the United 

States’ military operations in Afghanistan and Iraq and (2) the growing 

needs of an aging veteran population—trends that are expected to 

continue. The total number of veteran enrollees in VA’s health care 

system rose from 7.9 million to almost 9 million from fiscal year (FY) 2006 

through FY 2016. As the number of veterans using VA health care 

services increases, it is important to ensure that feedback about their care 

is addressed in a convenient and timely manner. To that end, VHA has 

designated patient advocates at each VA medical center (VAMC) to 

receive and document feedback from veterans or their representatives, 

including requests for information, compliments, and complaints.1

Although VHA’s patient advocacy program has been in place since 1990, 

it has taken on new significance in recent years given concerns about 

veterans’ ability to receive timely and quality care, among other issues. 

Our work, along with that of the VA Office of Inspector General (OIG) and 

others, has cited longstanding concerns about VA’s oversight of its health 

1

In this report, the term “VAMC” refers to an individual VA medical center and any of its 

associated facilities such as community-based outpatient clinics.

Letter
Page 2 GAO-18-356 VHA Patient Advocacy Program

care system, including ambiguous policies and inconsistent processes.2

These concerns contributed to the addition of veterans’ health care to 

GAO’s High-Risk List in 2015, and its continued inclusion in the 2017 

update.3 In 2017, the VA OIG raised concerns about the effectiveness of 

VHA’s patient advocacy program, including a lack of monitoring of 

program data, such as feedback documented by advocates, to identify 

trends across VAMCs.4

Until recently, VHA’s patient advocacy program was overseen by the 

Office of Patient Centered Care and Cultural Transformation 

(OPCC&CT). However, the Comprehensive Addiction and Recovery Act 

of 2016 (CARA), included a provision for VHA to establish an Office of 

Patient Advocacy (OPA) by July 2017 to take on oversight responsibilities 

for the program, such as ensuring that patient advocates at VAMCs 

receive training.5

CARA also included a provision for us to review VHA’s patient advocacy 

program.6 This report examines the extent to which VHA has

 2

For example, see GAO, Veterans Health Care: Additional Actions Could Further Improve 

Policy Management, GAO-17-748 (Washington, D.C.: Sept. 22, 2017), and VA Health 

Care: Improved Policies and Oversight Needed for Reviewing and Reporting Providers for 

Quality and Safety Concerns, GAO-18-63 (Washington, D.C., Nov. 15, 2017). See also, 

for example, Department of Veterans Affairs, Office of Inspector General, Veterans Health 

Administration, Review of the Implementation of the Veterans Choice Program, Report 

No. 15-04673-333 (Washington, D.C.: Jan. 30, 2017), and Veterans Health 

Administration, Audit of Veteran Wait Time Data, Choice Access, and Consult 

Management in VISN 6, Report No. 16-02618-424 (Washington, D.C.: Mar. 2, 2017).

3

GAO maintains a high-risk list to focus attention on government agencies and programs 

that it identifies as high risk due to their greater vulnerabilities to fraud, waste, abuse, and 

mismanagement or the need for transformation to address economy, efficiency, or 

effectiveness challenges. See GAO, High-Risk Series: An Update, GAO-15-290 

(Washington, D.C.: Feb. 11, 2015). In our 2017 high-risk update, we reported that 

previously identified issues for policy management continue, in part, because VA lacks the 

capacity to effectively address the issues we raised. See GAO, High-Risk Series: 

Progress on Many High-Risk Areas, While Substantial Efforts Needed on Others, 

GAO-17-317 (Washington D.C.: Feb. 15, 2017).

4

Department of Veterans Affairs, Office of Inspector General, Veterans Health 

Administration, Audit of the Patient Advocacy Program, Report No. 15-05379-146. 

(Washington, D.C.: Mar. 31, 2017).

5

Comprehensive Addiction and Recovery Act of 2016, Pub. L. No. 114-198, § 924, 130 

Stat. 695, 767-769 (2016).

6

Pub. L. No. 114-198, § 923, 130 Stat. 695, 767 (2016).
Page 3 GAO-18-356 VHA Patient Advocacy Program

1. provided guidance to VAMCs on the governance of the program;

2. provided guidance to VAMCs on staffing the program;

3. assessed the training needs of patient advocates and monitored 

training completion; and

4. monitored patient advocacy program data-entry practices and 

reviewed program data.

For all four objectives, we interviewed officials involved in the patient 

advocacy program from eight VAMCs and their five associated Veterans 

Integrated Service Networks (VISN), regional networks of care. We 

selected six of our eight VAMCs for variation in (1) the number of 

complaints received and entered in VHA’s Patient Advocate Tracking 

System (PATS)—an electronic system used to describe and track the 

resolution of veterans’ feedback across VAMCs—in FY 2016, (2) facility 

complexity level, (3) geographic location, and (4) the type of staff VAMCs 

used to administer the program.7 We selected the remaining two VAMCs 

based on one’s involvement in piloting a new approach to recording 

patient advocacy program data, and one’s recent changes to the structure 

of its program. We also ensured that at least three of the VAMCs we 

selected received assessments from OPCC&CT on how they had 

implemented the patient advocacy program to gain perspectives on the 

office’s involvement with VAMCs.8 See table 1 for a list of the eight 

VAMCs we selected and their associated VISNs. Perspectives obtained 

from the eight VAMCs and five VISNs in our review cannot be 

generalized.

 7

VHA categorizes VAMCs according to complexity level, which is determined on the basis

of the characteristics of the patient population, clinical services offered, educational and 

research missions, and administrative complexity. We obtained the number of complaints 

received by VAMCs and recorded in PATS in FY 2016 from VISN officials in March 2017. 

We then ranked VAMCs by the number of complaints and split them into three equal 

groups. We then selected at least two VAMCs from each of these groups.

8

In 2015 and 2016, OPCC&CT conducted assessments of the implementation of the 

patient advocacy program at some VAMCs and provided recommendations on how to 

strengthen program implementation to better align with a proactive approach to patient 

advocacy.
Page 4 GAO-18-356 VHA Patient Advocacy Program

Table 1: Selected Department of Veterans Affairs Medical Centers (VAMC) and 

Associated Veterans Integrated Service Networks (VISN) Included in Review

VAMC VISN 

Edith Nourse Rogers Memorial Veterans Hospital (Bedford, Massachusetts) 1

VA Maryland Health Care System (Baltimore, Maryland) 5

Fayetteville VA Medical Center (Fayetteville, North Carolina) 6

Hampton VA Medical Center (Hampton, Virginia) 6

John J. Pershing VA Medical Center (Poplar Bluff, Missouri) 15

Kansas City VA Medical Center (Kansas City, Missouri) 15

Grand Junction Veterans Health Care System (Grand Junction, Colorado) 19

VA Eastern Colorado Health Care System (Denver, Colorado) 19

Source: GAO analysis of Veterans Health Administration (VHA) information. | GAO-18-356

We also interviewed VHA officials, including those who had overseen the 

program when it was under OPCC&CT and who transitioned to OPA once 

the office was established in 2017. In addition, we interviewed officials 

from five veterans service organizations (VSO) to obtain their 

perspectives on the patient advocacy program: American Legion, 

Disabled American Veterans, Iraq and Afghanistan Veterans of America, 

Veterans of Foreign Wars, and Vietnam Veterans of America.9

To examine the extent to which VHA has provided guidance to VAMCs on 

the governance of the patient advocacy program, we reviewed VHA’s 

handbook for the program to determine, among other things, whether it 

specified the VAMC department to which patient advocates should report 

and whether it identifies responsibilities for VHA staff overseeing the 

program.10 We also reviewed documentation of VHA’s planned efforts 

related to improving the governance of the program, such as a draft 

directive for the program. We evaluated the information we reviewed in 

VHA’s handbook against federal internal control standards.11

 9

We selected five VSOs based on several criteria including whether the organization 

published articles or reports about VHA’s patient advocacy program.

10For example, see Department of Veterans Affairs, Veterans Health Administration, VHA 

Patient Advocacy Program, VHA Handbook 1003.4 (Sept. 2, 2005).

11GAO, Standards for Internal Control in the Federal Government, GAO-14-704G 

(Washington, D.C.: September 2014). Internal control is a process effected by an entity’s 

oversight body, management, and other personnel that provides reasonable assurance

that the objectives of an entity will be achieved.
Page 5 GAO-18-356 VHA Patient Advocacy Program

To examine the extent to which VHA has provided guidance to VAMCs on 

staffing the patient advocacy program, we reviewed VHA’s handbook for 

the program. Specifically, we reviewed the handbook to determine the 

extent to which it provided guidance on how VAMCs should determine the 

appropriate number and type of staff needed to administer the program. 

We also reviewed documentation of VHA’s planned efforts related to 

staffing the program identified in a workgroup charter. We evaluated 

VHA’s efforts to provide guidance to VAMCs on staffing against key 

principles for effective strategic workforce planning and federal internal 

control standards.12

To examine the extent to which VHA has assessed the training needs of 

patient advocates and monitored training completion, we reviewed 

training materials VHA provided to VAMCs, such as a list of 

recommended training for patient advocates. We also reviewed 

documentation of VHA’s planned efforts related to assessing the training 

needs of patient advocates identified in a workgroup charter. We 

evaluated the extent to which VHA has monitored training of patient 

advocates against a guide for assessing strategic training and 

development efforts and federal internal control standards.13

To examine the extent to which VHA has monitored data-entry practices 

and reviewed data from the patient advocacy program, we reviewed 

VHA’s handbook for the program and summaries of data from PATS.14

We also reviewed documentation of VHA’s planned efforts related to 

PATS data-entry practices and reviewing program data identified in a 

workgroup charter. We evaluated the extent to which VHA has monitored 

data-entry practices and reviewed PATS data against a guide for 

assessing the reliability of computer-processed data and federal internal 

control standards.15

 12See GAO-14-704G and GAO Human Capital: Key Principles for Effective Strategic 

Workforce Planning, GAO-04-39 (Washington, D.C.: Dec. 11, 2003).

13See GAO-14-704G and GAO, Human Capital: A Guide for Assessing Strategic Training 

and Development Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: 

Mar. 1, 2004).

14To obtain contextual information related to the patient advocacy program, we reviewed 

VA analyses of data for fiscal years 2014 through 2017 that had been entered in PATS as 

of January 4, 2018. 

15See GAO-14-704G and GAO, Assessing the Reliability of Computer-Processed Data,

GAO-09-680G (Washington, D.C.: July 2009).
Page 6 GAO-18-356 VHA Patient Advocacy Program

We conducted this performance audit from February 2017 to April 2018 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

VHA’s patient advocacy program is intended to provide veterans with a 

means to provide feedback about health care services they receive at 

VAMCs. VHA sets forth minimum expectations for VAMCs’ administration 

of the program, including that veterans must have easy access to a 

patient advocate and must have their complaints addressed in a 

convenient and timely manner.

The patient advocacy program is administered at the VAMC level. Each 

of VA’s 170 VAMCs is responsible for making at least one patient 

advocate available to respond to veterans’ feedback, and for ensuring 

that feedback is recorded in PATS. VAMCs may designate other staff to 

assist patient advocates in responding to feedback, such as lead patient 

advocates and service-level advocates. Service-level advocates, such as 

nurses or administrative staff, are designated at some VAMCs to respond 

to veterans’ feedback before involving a patient advocate. All VAMC staff 

that have a designated role in the administration of the patient advocacy 

program are referred to as patient advocacy program staff. In addition to 

designating program staff, VAMCs may use a variety of methods to make 

veterans aware of the patient advocacy program, such as displaying 

signage on site and including information about the program on their 

websites. (See app. I for more information on the methods selected 

VAMCs used to make veterans aware of the program.)

Patient advocacy program staff enter veterans’ feedback in PATS using a 

report of contact (ROC) and assign one or more issue codes that 

generally describe the nature of the feedback, such as coordination of 

care. (See app. II for additional information on entering veterans’ 

feedback into PATS.) Each piece of feedback shared is categorized as 

either a request for information, compliment, or complaint. VHA’s 

handbook for the program specifies certain goals for data collection and 

resolution—specifically, that

Background

Administration of the 

Patient Advocacy Program
Page 7 GAO-18-356 VHA Patient Advocacy Program

• all complaints should be entered in PATS to enable a comprehensive 

understanding of veterans’ issues and concerns to, in turn, identify 

potential system-wide improvements; and

• responses should occur no later than 7 days after the complaint is

made.16

With this guidance, patient advocacy program staff use a variety of 

approaches for entering veterans’ feedback in PATS and closing it in the 

system once addressed. For example, when VAMCs have designated 

service-level advocates, the process for entering and closing feedback in 

PATS is generally different than the approach used by VAMCs that have 

only patient advocates. (See fig. 1.)

 16See VHA Handbook 1003.4.
Page 8 GAO-18-356 VHA Patient Advocacy Program

Figure 1: Process at Department of Veterans Affairs Medical Centers (VAMC) for Entering and Closing Veterans’ Feedback in 

the Patient Advocate Tracking System

Note: In this figure the term “veteran” includes veterans and their representatives, such as family 

members or friends, and the term “feedback” includes requests for information, compliments, and 

complaints. Patient advocates are VHA employees designated at each VAMC to receive feedback 

from veterans. Service-level advocates, such as nurses or administrative staff, are designated at 

some VAMCs to receive veterans’ feedback before involving a patient advocate. According to VHA’s 

handbook for the program, responses to complaints should occur as soon as possible, but no longer 

than 7 days after the complaint is made. VHA officials told us they interpret this goal to mean that 

complaints should be closed in PATS within 7 days. Other forms of feedback (requests for information 

and compliments) are not required to be entered in PATS but may be entered, as this information is 

useful in determining common areas of confusion that could be addressed in a proactive manner.
Page 9 GAO-18-356 VHA Patient Advocacy Program

Patient advocacy program staff at each VAMC are assisted by a VISNlevel coordinator who acts as a liaison between the VAMCs and VHA and 

is responsible for ensuring consistency in PATS data collection within the 

VISN. The VISN director is responsible for designating the coordinator 

and ensuring that each VAMC within the VISN has at least one patient 

advocate.

The VHA office responsible for overseeing the patient advocacy program 

changed as a result of CARA. From January 2011 to July 2017, the 

program was overseen by OPCC&CT under VHA’s Deputy Under 

Secretary for Health for Operations & Management.17 CARA included a 

provision for VHA to establish OPA to begin overseeing the program and 

specified that this office would report directly to the Under Secretary for 

Health, a higher-level office within VHA.18 Although OPCC&CT is no 

longer responsible for overseeing the program, it is to continue to play an 

advisory role to OPA during the initial phases of its work, according to 

OPCC&CT officials.

Many of OPA’s oversight responsibilities are specified in CARA including 

ensuring that patient advocates advocate on behalf of veterans, manage 

PATS, and identify trends in the data to determine whether there are 

opportunities for improving veterans’ health care. Also, OPA’s director is 

required to ensure that patient advocates receive relevant, consistent 

training across VAMCs. When establishing the office in July 2017, VHA 

officials wrote a memo indicating that OPA’s primary objectives were to 

implement a standardized policy for the patient advocacy program and to 

resolve any system-wide issues, such as concerns about care across 

VAMCs identified through veterans’ feedback. In addition, in August 2017, 

OPA began soliciting feedback from VAMCs on various aspects of the 

patient advocacy program to identify improvement priorities and best 

practices. By September 2017, OPA had identified an acting program 

director, established a workgroup (called the National Strategic 

Workgroup) to develop recommendations related to program 

 17VHA’s patient advocacy program was established in 1990. From the program’s inception 

to January 2011, the program was overseen by the National Veteran Service and 

Advocacy Program of the VISN Support Service Center.

18Pub. L. No. 114-198, § 924, 130 Stat. 767 (2016), codified at 38 U.S.C. 

§7309A(c)(2)(C).

Oversight of the Patient 

Advocacy Program
Page 10 GAO-18-356 VHA Patient Advocacy Program

administration, and finalized a charter that identifies workgroup 

deliverables.

VHA has provided limited guidance to VAMCs on the governance of the 

patient advocacy program. Specifically, VHA provided limited guidance on 

how to meet the program’s expectations that veterans have easy access 

to a patient advocate who will hear their complaints and address them in 

a timely manner. While VHA’s handbook for the program provides general 

information on the responsibilities of patient advocacy program staff, it 

does not specify the VAMC department to which patient advocates should 

report to help ensure VAMCs meet these expectations. According to VHA 

officials, the lack of specific guidance was intentional and due in part to 

VHA officials’ view that leadership at each VAMC is in the best position to 

understand the needs of veterans at their facilities, and therefore should 

have flexibility to make decisions about governance in response to those 

needs.

In addition to providing limited guidance to VAMCs, VHA’s patient 

advocacy program handbook is out of date and does not incorporate 

recent agency-wide changes, such as those made in response to VHA 

Strategic Plan FY 2013 – 2018 which identifies the goal of providing 

proactive, patient-driven health care.19 The handbook for the program 

was issued in 2005, expired in 2010, and as of January 2018, no updates 

had been released. In the absence of an updated document, VAMCs are 

still expected to follow the outdated handbook.20 However, the handbook 

does not identify the responsibilities of the current VHA office responsible 

for overseeing the program. Instead, it identifies the responsibilities of the 

VHA office that oversaw the program before OPCC&CT began 

overseeing the program in 2011.

In recent years, OPCC&CT reviewed the implementation of the patient 

advocacy program at some VAMCs and provided specific 

recommendations on how to change program governance to better reflect 

a more proactive patient advocacy program model. However, the 

recommendations from these reviews were provided only to some 

 19See Department of Veterans Affairs, Veterans Health Administration, VHA Strategic 

Plan FY 2013 – 2018.

20See Department of Veterans Affairs, Veterans Health Administration, Controlled 

National Policy/Directives Management System, VHA Directive 6330 (Washington, D.C.: 

June 24, 2016).

VHA Has Provided 

Limited, Outdated 

Guidance to VAMCs 

on the Governance of 

the Patient Advocacy 

Program
Page 11 GAO-18-356 VHA Patient Advocacy Program

VAMCs; guidance that could be applicable to all VAMCs was not added 

to the handbook.21 OPCC&CT officials stated that they did not update the 

handbook because they decided to instead spend time trying to 

understand recent feedback they received from VAMC officials and 

ensure that any updates would reflect system-wide shifts as a result of 

VHA’s strategic plan.

OPCC&CT’s limited and outdated guidance to VAMCs on the governance 

of the patient advocacy program is inconsistent with federal internal 

control standards for the control environment, which require agencies to 

establish an organizational structure, assign responsibility, and delegate 

authority to achieve agency objectives—key aspects of governance.22 To 

do so, an agency may develop an organizational structure that assigns 

responsibilities to discrete units and defines reporting lines at all levels of

the organization. Without providing specific, timely guidance to VAMCs on 

the governance of the patient advocacy program, the program is at risk of 

not meeting its minimum expectations.

In light of the limited and outdated guidance on the governance of the 

program, patient advocacy program staff at most of our selected VAMCs 

noted that the VAMC department to which patient advocates report can 

have a direct effect on the ability of staff to resolve veterans’ complaints. 

For example, patient advocates at one VAMC said because of the 

program’s position within the organization, they did not have the authority 

to ensure that VAMC officials external to the patient advocacy program, 

such as physicians, quickly engaged in responding to veterans’ 

complaints. In these cases, a patient advocate would contact the 

physician to resolve a complaint, but may not have received a response 

until the matter was brought to the attention of the physician’s 

supervisor—a reporting line that is outside of the patient advocacy 

program at this VAMC. Officials from several of our selected VAMCs and 

VSOs noted that the position of the patient advocacy program within 

VAMCs may not give patient advocates the authority to require VAMC 

 21According to an assessment OPCC&CT completed for one VAMC on the 

implementation of the patient advocacy program, for example, in a proactive model of the 

patient advocacy program, all staff take responsibility for the veteran’s experience and 

attempt to resolve complaints at the lowest level possible in the organization. To 

implement a proactive model, OPCC&CT recommended that this VAMC consider 

developing processes to increase the engagement of VAMC staff external to the patient 

advocacy program, such as physicians.

22GAO-14-704G.
Page 12 GAO-18-356 VHA Patient Advocacy Program

staff to respond to veterans’ complaints. They added that conflict-ofinterest concerns could arise when a veteran has a complaint about a 

VAMC for which the patient advocate works. (See app. III for additional 

information on the governance of the patient advocacy program at 

selected VAMCs.)

In VA’s written comments on a draft of this report, which are reproduced 

in Appendix IV, VA stated that it issued its new directive for the patient 

advocacy program that had been in development as we were conducting 

our review.23 While the updated directive specifies that a VAMC’s lead 

patient advocate should report to the facility director, it does not specify 

the VAMC department to which other patient advocacy program staff, 

including patient advocates who are not designated as lead patient 

advocates and service-level advocates, should report. In addition, OPA’s 

National Strategic Workgroup recently submitted recommendations to 

OPA on the governance of the patient advocacy program. OPA officials 

stated that they plan to prioritize the recommendations and elicit feedback 

from VISN directors on how to operationalize the recommendations. 

However, it is unclear whether OPA will provide additional guidance 

related to the governance of the program based on these 

recommendations, such as guidance on the VAMC department to which 

all types of patient advocacy program staff should report. Until actions to 

address the weaknesses we found are completed, guidance on the 

governance of the program will continue to be lacking.

VHA has provided limited guidance to VAMCs on the number and type of 

patient advocacy program staff needed to ensure that complaints from 

veterans are addressed in a convenient and timely manner. According to 

VHA’s existing handbook for the program, every VAMC should have at 

least one patient advocate and appropriate administrative, technical, and 

clerical support should be provided to allow for efficient performance of 

the responsibilities of program staff. OPCC&CT did not provide guidance 

on how VAMCs should determine the appropriate number of 

administrative, technical, and clerical staff or type of patient advocacy 

program staff, such as lead patient advocates and service-level 

advocates. According to officials, this was because no assessment was 

conducted to identify what staff resources would be needed to meet the 

 23See Department of Veterans Affairs, Veterans Health Administration, VHA Patient 

Advocacy, VHA Directive 1003.04 (Washington, D.C.: Feb. 7, 2018).

VHA Has Provided 

Limited Guidance to 

VAMCs on Staffing 

the Patient Advocacy 

Program
Page 13 GAO-18-356 VHA Patient Advocacy Program

expectations of the program. In the absence of such an assessment, 

OPCC&CT instead relied on each VAMC to determine what resources 

would be needed based on the facility’s size and services provided. 

However, VHA’s handbook for the program does not provide instruction 

for VAMC or VISN officials on how to determine the number and type of 

staff needed for the program. OPCC&CT officials added that budget 

constraints can also affect a VAMC’s ability to hire the appropriate staff 

for the program. (See app. III for additional information on the number 

and type of patient advocacy program staff at selected VAMCs.)

Officials at all but one of the selected VAMCs stated that program staff at 

their VAMCs had more work to do than they could handle. For example, 

VAMC officials cited backlogs in work, such as calls from veterans not 

being answered, messages not being responded to, voicemail boxes 

being full, and not all veterans’ feedback being entered into PATS. 

Officials from one VAMC we spoke with in July 2017 stated that due to 

workload demands and not enough patient advocacy program staff at 

their VAMC, they had roughly 300 unanswered phone calls at that time 

from veterans who want to provide feedback to a patient advocate.

Officials from several VSOs we spoke with stated that there is not enough 

patient advocate staff, adding that veterans reported that their calls to 

patient advocates were not answered, they were unable to reach an 

advocate, or their calls were not responded to in a timely manner.

The lack of staffing guidance is inconsistent with GAO’s Key Principles for 

Effective Strategic Workforce Planning, which states that workforce 

planning is essential to addressing an organization’s critical need to align 

its human capital program with its current and emerging mission and 

programmatic goals.24 Further, federal internal control standards require 

agencies to design control activities to achieve objectives, a key aspect of 

effectively staffing a program.25 Such control activities may include 

effectively managing the agency’s workforce, such as by continually 

assessing the knowledge, skills, and abilities of the workforce to achieve 

organizational goals.

 24See GAO Human Capital: Key Principles for Effective Strategic Workforce Planning,

GAO-04-39 (Washington, D.C.: Dec. 11, 2003). As part of the planning process, 

organizations should determine the skills and competencies that will be needed to achieve 

current and future programmatic results and develop strategies, such as policies and 

practices that are tailored to address the gaps in number, deployment, and alignment of 

human capital approaches.

25GAO-14-704G.
Page 14 GAO-18-356 VHA Patient Advocacy Program

The lack of guidance on staffing may impede VAMCs’ efforts to ensure 

that they have the appropriate number and type of staff to administer the 

patient advocacy program. The resulting misalignment of staff resources 

could have negatively affected VAMCs’ ability to achieve the program’s 

objectives, including addressing veterans’ complaints in a timely manner. 

For example, if there are not a sufficient number of patient advocates to 

respond to veterans’ phone calls in a timely manner, VAMCs may not be 

able to ensure that patient advocates can respond to veterans’ complaints 

within 7 days, as called for by VHA’s handbook for the program.

According to VHA officials, OPA analyzed feedback from VAMCs on the 

factors that should be considered in developing national guidelines for 

staffing, such as facility size and complexity level, and directed its 

National Strategic Workgroup to develop recommendations for 

determining the extent to which VAMCs have utilized various patient 

advocacy program staff, such as service-level advocates, by the spring of 

2018. However, OPA expects that these efforts will result in 

recommendations for consideration, and it is unclear what steps, if any, 

will be taken based on the recommendations. Until actions to address the 

weaknesses we found are completed, the lack of guidance for VAMCs on 

determining the appropriate number and types of staff will put the patient 

advocacy program at risk of being unable to address veterans’ complaints 

in a convenient and timely manner.
Page 15 GAO-18-356 VHA Patient Advocacy Program

VHA has recently developed a list of recommended training for patient 

advocates. In the spring of 2017, OPCC&CT officials updated a 

recommended training list for patient advocates developed before 2011 

when OPCC&CT began overseeing the patient advocacy program. The 

training list covers a wide variety of topics, including how to enter and 

examine trends in PATS data, as well as key responsibilities of patient 

advocates outlined in VHA’s handbook for the program. OPCC&CT 

officials stated that they would like to make the trainings required, but 

have not pursued this because of the lengthy process within VHA to 

designate required training for a specific group of staff. To update the list 

in 2017, OPCC&CT convened a workgroup (which included several 

patient advocates) to determine whether the old training list was 

sufficient, and the workgroup shared its suggested updates with VISNlevel coordinators for distribution to VAMCs in April 2017.

We found that OPCC&CT has not developed an approach to routinely 

assess the training needs of patient advocates. Rather, OPCC&CT 

officials stated that they relied on VAMC and VISN staff to conduct these 

assessments. However, VHA’s handbook for the program does not 

specify that VAMC or VISN officials are responsible for conducting routine 

assessments of patient advocates’ training needs. None of our selected 

VAMCs routinely conducted assessments of the training needs of patient 

advocates, such as assessing whether advocates were adequately 

trained to carry out their responsibilities. Officials from two VAMCs said 

they used ad hoc approaches to assess training needs. For example, one 

patient advocate supervisor stated that training is offered on an “as 

VHA Has 

Recommended 

Training for Patient 

Advocates, but Has 

Not Developed an 

Approach to 

Routinely Assess 

Their Training Needs 

or Monitored Training 

Completion

VHA Has Developed a 

Recommended Training 

List for Patient Advocates, 

but Has Not Developed an 

Approach to Assess Their 

Training Needs on a 

Routine Basis
Page 16 GAO-18-356 VHA Patient Advocacy Program

needed” basis in patient advocate meetings when a training need is 

identified.

The lack of an approach for routinely assessing the training needs of 

patient advocates is inconsistent with federal standards for internal 

control related to control activities.26 Under these standards relating to 

human capital, management ensures that training is aimed at developing 

and retaining employee knowledge, skills, and abilities to meet changing 

organizational needs. Management should also continually assess the 

knowledge, skills, and ability needs of a program so that the program is 

able to obtain a workforce that has the required knowledge, skills, and 

abilities to achieve organizational goals.

Without an approach for routinely assessing the training needs of patient 

advocates, VHA may not be able to clearly identify gaps in the knowledge 

and skills of these staff over time, which, in turn, could put the program at 

risk of not meeting its goals. For example, if there is a gap in 

understanding among patient advocates that all complaints should be 

entered into PATS, addressing veterans’ complaints may be delayed, if 

addressed at all, and opportunities to analyze complaint data for the 

purpose of identifying system-wide improvements may be missed.

According to VHA officials, OPA analyzed feedback from VAMCs on the 

training needs of patient advocates, including how to correctly enter data 

into PATS, and directed its National Strategic Workgroup to develop 

recommendations for assessing the training needs of patient advocates 

by the spring of 2018. OPA expects that these efforts will result in 

recommendations for OPA to consider, but it is unclear what steps, if any, 

will be taken based on the recommendations. Until actions to address the 

weaknesses we found are completed, the lack of routine assessments of 

training needs will continue to put the program at risk of staff not having 

the requisite skills and knowledge to carry out their duties.

VHA has not monitored the completion of training for patient advocates. 

Specifically, OPCC&CT officials said that they did not monitor the extent 

to which patient advocates completed the recommended training 

distributed in April 2017. Instead, these officials relied on patient advocate 

supervisors to monitor training completion. However, VHA’s handbook for 

 26GAO-14-704G.

VHA Has Not Monitored 

Training Completion for 

Patient Advocates
Page 17 GAO-18-356 VHA Patient Advocacy Program

the program does not specify that patient advocate supervisors are 

responsible for monitoring the completion of training for patient 

advocates.

Half of patient advocate supervisors at our selected VAMCs did not track 

the completion of patient advocacy training. Patient advocate supervisors 

said that they are able to track the completion of general VA employee 

training through VA’s Talent Management System. However, most 

training specific to patient advocacy were generally not included in this 

system during the period of our review. Officials from our selected 

VAMCs who did track patient advocacy training used various methods to 

record completion, such as keeping attendance lists for the training 

provided.

Taking steps to monitor training completion would be consistent with 

GAO’s Guide for Assessing Strategic Training and Development Efforts in 

the Federal Government which identifies components of the training and 

development process, including having agencies collect and monitor data 

corresponding to establishing training objectives.27 Monitoring training 

completion would also be consistent with federal standards for internal 

control related to control activities.28 Under these standards relating to 

human capital, management ensures that training is aimed at developing 

and retaining employee knowledge, skills, and abilities to meet changing 

organizational needs. Management also continually assesses the 

knowledge, skills, and ability needs of a program so that the program is 

able to obtain a workforce that has the required knowledge, skills, and 

abilities to achieve organizational goals—key components for monitoring 

training completion.

If patient advocates are not properly trained in how to use PATS to 

document and resolve complaints, tracking the status of complaints may 

be more difficult, which could increase the likelihood that they are not 

addressed in a timely manner, if at all. Further, CARA specifies that the 

director of OPA should ensure that patient advocates receive training 

specific to patient advocacy.29

 27GAO-04-546G.

28GAO-14-704G.

29Pub. L. No. 114-198, § 924, 130 Stat. 767 (2016), codified at 38 U.S.C. 

§ 7309A(c)(2)(C).
Page 18 GAO-18-356 VHA Patient Advocacy Program

According to VHA officials, OPA did not obtain information on whether 

patient advocates completed recommended training and did not identify 

an approach for monitoring training completion moving forward. Without 

monitoring training completion, there is an increased risk that patient 

advocates have not received the training they need to effectively fulfill 

their responsibilities such as advocating on behalf of veterans and 

consistently using PATS to document and resolve complaints.

VHA officials have not monitored PATS data-entry practices to ensure 

complaints were always entered into PATS and issue codes were 

assigned consistently to ROCs. OPCC&CT officials told us they did not 

monitor the data-entry practices of patient advocacy program staff to 

ensure that all complaints were entered into PATS, a key goal according 

to VHA’s handbook for the program.30 Rather, they relied on VISN and 

VAMC officials to ensure that program staff entered all complaints into 

PATS. Officials from two of the five VISNs we interviewed stated that they 

did not perform any audits or checks of the data entered into PATS by 

patient program staff at VAMCs.

 30See VHA Handbook 1003.4. The handbook states that requests for information and 

compliments may also be entered into PATS, as this information is useful in determining 

common areas of confusion that could be addressed in a proactive manner.

VHA Has Not 

Monitored Patient 

Advocacy Data-Entry 

Practices or 

Reviewed Patient 

Advocacy Data to 

Assess Program 

Performance and 

Identify System-Wide 

Improvements

VHA Has Not Monitored 

Whether Complaints Were 

Always Entered into PATS 

and Issue Codes Assigned 

Consistently
Page 19 GAO-18-356 VHA Patient Advocacy Program

We also found inconsistencies in the extent to which VAMC officials 

entered complaints into PATS, with complaints always entered into PATS 

at one of our selected VAMCs, while at other VAMCs some complaints 

were left unrecorded, according to officials. For example, at one VAMC, 

officials stated that over a third of the complaints received were not 

entered into PATS due to the competing workload demands of patient 

advocates. Similarly, at another selected VAMC, almost a quarter of the 

complaints received were not entered into PATS, according to patient 

advocates there who explained that they primarily used a document 

outside of PATS to record veterans’ feedback.

In addition, OPCC&CT officials told us they did not monitor whether 

patient advocates used a consistent practice to assign issue codes to 

veterans’ feedback recorded into PATS. Using a consistent data-entry 

practice is important to ensure that PATS data can be compared across 

VAMCs to better enable an accurate and comprehensive understanding 

of veterans’ issues and concerns, a goal of the patient advocacy program. 

OPCC&CT officials stated that they relied on VISN-level coordinators to 

monitor coding practices because VHA’s handbook for the program states 

that these coordinators should develop VISN-wide consistent approaches 

for entering complaints into PATS. VISN-level coordinators from two 

selected VISNs stated that they created a standard practice for assigning 

issue codes within a particular VISN; however, the coding practices 

differed between VISNs, making national level analysis difficult.

We also found inconsistencies in how VAMC officials coded specific 

veterans’ feedback. For example, patient advocates did not use 

consistent practices to code issues related to the Veterans Choice 

Program (Choice Program), one of the most common types of issues 

patient advocates told us they hear about from veterans.31 Officials from 

one of our selected VAMCs said they code feedback related to the 

Choice Program under a specific “request for information” issue code, 

regardless of whether the feedback was a request for information, 

compliment, or complaint. In contrast, officials at another VAMC stated 

that they typically code feedback related to the Choice Program as a 

complaint related to billing. (See app. II for additional information on dataentry practices at selected VAMCs.)

 31The Veterans Choice Program is a program that allows eligible veterans to receive 

health care from a community provider when faced with long wait times, lengthy travel 

distances, or other challenges accessing care at a VAMC.
Page 20 GAO-18-356 VHA Patient Advocacy Program

OPCC&CT’s lack of monitoring of PATS data-entry practices is 

inconsistent with GAO’s Assessing the Reliability of Computer-Processed 

Data which identifies the importance of consistent data-entry practices to 

ensure that data are reasonably complete and accurate.32 Further, federal 

standards for internal control related to information and communications 

require agencies to use quality information, such as relevant data from 

reliable sources, to achieve the agency’s objectives.33 Under internal 

control standards for control activities, management also is to monitor 

performance to achieve objectives. Without OPCC&CT monitoring dataentry practices, the patient advocacy program is at risk of not meeting its 

goal that all complaints are entered into PATS and there is an increased 

likelihood of VHA not having an accurate understanding of veterans’ 

complaints across VAMCs.

Moving forward, in fall 2017, OPA distributed meeting minutes to all VISN 

and VAMC directors stating that all veterans’ feedback should be 

consistently recorded in PATS. OPA officials also updated some of the 

issue codes in PATS in fall 2017 and added a code specifically for 

community care issues, such as issues related to the Choice Program. In 

addition, OPA officials stated that they plan to promote the consistent 

assignment of issue codes to veterans’ feedback through national 

training, but have not specified when this training will occur or if OPA staff 

will monitor patient advocates’ consistent assignment of issue codes or of 

data-entry practices generally. Until these actions are completed, 

however, the gaps in monitoring of PATS data-entry practices that we 

identified will continue to exist, putting the program at risk of incomplete 

or unreliable data that may not allow an accurate understanding of 

veterans’ complaints, critical to making system-wide improvements.

VHA officials have not systematically reviewed PATS data to assess 

program performance and identify potential system-wide improvements, 

goals of the patient advocacy program. Specifically, OPCC&CT officials 

stated that they reviewed PATS data in response to inquiries, but did not 

conduct systematic reviews of the data over time. For example, they did 

not track VAMC performance on responding to complaints in a timely 

manner or track the most common complaints across VAMCs to identify 

potential opportunities for system-wide improvements.

 32GAO-09-680G.

33GAO-14-704G.

VHA Has Not 

Systematically Reviewed 

PATS Data to Assess 

Program Performance and 

Identify Potential SystemWide Improvements
Page 21 GAO-18-356 VHA Patient Advocacy Program

OPCC&CT officials stated that they did not conduct systematic reviews of 

PATS data because VISN and VAMC officials were primarily responsible 

for these analyses. However, according to VHA’s handbook for the 

patient advocacy program, VHA officials have a responsibility to examine 

PATS data for trends across VAMCs and identify any areas for systemwide improvement. Officials stated that it was challenging to analyze 

PATS information included in narrative text, such as descriptions of 

veterans’ feedback.

Not reviewing PATS data is inconsistent with federal standards for 

internal control for monitoring which require agencies to establish and 

operate monitoring activities, such as assessing the quality of 

performance over time, and evaluate the results.34 Further, not 

conducting systematic assessments of PATS data made it difficult for 

OPCC&CT to determine program performance, such as whether the 

program was meeting its goal that all complaints are entered into PATS 

and responded to within 7 days. Officials explained that VHA interprets 

this goal to mean that complaints are closed in PATS within 7 days. 

According to VA, between FY 2014 and FY 2017 there were more than 

53,000 complaints per year open for greater than 7 days.35 If OPCC&CT 

officials had conducted systematic reviews of PATS data, they may have 

been able to identify that there were a significant number of complaints 

open for longer than 7 days and consider what actions should be taken, 

such as providing additional guidance to VAMCs on how to address 

complaints in a timely manner.

Furthermore, without systematically reviewing PATS data across VAMCs 

to identify potential system-wide improvements, OPCC&CT officials may 

have been unaware of important care issues across VAMCs. For 

example, patient advocates from several of our selected VAMCs stated 

that opioid prescription issues are among the most common complaints 

they received from veterans. If OPCC&CT officials were to have 

systematically reviewed PATS data across VAMCs to determine the 

prevalence of these types of complaints, they could have identified the 

need to address them on a national level and consider system-wide 

policies or guidance in response.

 34GAO-14-704G.

35According to VA, between FY 2014 and FY 2017, the total number of reports of contact 

entered in PATS ranged from about 360,000 to about 414,000 per year.
Page 22 GAO-18-356 VHA Patient Advocacy Program

According to VHA officials, OPA is in the process of identifying the data it 

needs to review on a routine basis, and directed its National Strategic 

Workgroup to identify program data that could be reviewed to assess 

program performance and identify potential system-wide improvements 

by the spring of 2018. However, OPA expects that these efforts will result 

in recommendations for OPA to consider, and it is unclear what steps, if 

any, will be taken based on the recommendations. Until actions to 

address the weaknesses we found are completed, the lack of a 

systematic review of PATS data will persist, putting the program at 

continued risk of missed opportunities for identifying and addressing 

weaknesses across VAMCs.

As one of the largest health care delivery systems in the nation, it is 

critically important for VHA to ensure that each veteran who receives 

health care services has easy access to an advocate who listens to that 

veteran’s feedback and responds in a timely manner. This is especially 

important given concerns about veterans’ ability to receive timely and 

quality care. However, VHA’s efforts to ensure that the patient advocacy 

program is meeting its goals—to identify potential system-wide 

improvements and respond to complaints within 7 days—have fallen 

short. OPCC&CT did not provide sufficient oversight to the program in the 

four key areas of governance, staffing, training, and data-entry practices, 

which has left the program at risk for not meeting its goals.

VHA’s newly established OPA has initiated plans to improve the patient 

advocacy program in these four areas; however, most of these plans 

center around a workgroup that will make recommendations for OPA to 

consider, and it is unclear what specific actions, if any, will be taken 

based on these recommendations. Further, documentation for several of 

OPA’s planned efforts has not been finalized. Unless specific actions to 

address the weaknesses we identified are completed expeditiously, the 

program is at risk of not meeting its goals, including addressing veterans’ 

complaints in a convenient and timely manner. Furthermore, without 

addressing the weaknesses we identified, OPA misses opportunities to 

review PATS data across VAMCs to identify potential system-wide issues 

that, if addressed, could significantly improve the experience of veterans. 

Such reviews are critical to ensuring that VHA is taking steps to both 

meet its goal in its strategic plan to provide veterans with timely and 

quality health care, and to address recent issues it has faced, such as 

veterans’ ability to access care in a timely manner.

Conclusions
Page 23 GAO-18-356 VHA Patient Advocacy Program

We are making the following six recommendations to the VHA 

Undersecretary for Health:

• provide updated guidance to VAMCs on the governance of the patient 

advocacy program, including clear definitions of reporting lines. 

(Recommendation 1)

• assess and provide guidance to VAMCs on appropriately staffing the 

patient advocacy program, including guidance on how to determine 

the appropriate number and type of staff. (Recommendation 2)

• develop an approach to routinely assess the training needs of patient 

advocates. (Recommendation 3)

• monitor the completion of training for patient advocates. 

(Recommendation 4)

• monitor PATS data-entry practices to ensure all complaints are 

entered into PATS and that veterans’ feedback is coded consistently. 

(Recommendation 5)

• systematically review PATS data to assess program performance and 

identify potential system-wide improvements. (Recommendation 6)

We provided a draft of this report to VA for comment. In its written 

comments, which are reproduced in Appendix IV, VA concurred with our 

recommendations and noted that it recently issued the new directive for 

patient advocacy that had been in development as we were conducting 

our review. The directive supersedes the outdated handbook for the 

patient advocacy program and describes certain aspects of program 

governance, including certain reporting lines, roles, and responsibilities. 

Accordingly, VA requested that we close our first recommendation related 

to governance. We revised our report to reflect the issuance of the new 

directive. However, we do not believe the directive fully implements our 

recommendation. While the updated directive specifies that a VAMC’s 

lead patient advocate should report to the facility director, it does not 

specify the VAMC department to which other patient advocacy program 

staff, including patient advocates who are not designated as lead patient 

advocates and service-level advocates, should report. Until VA specifies 

the reporting lines for these other patient advocacy program staff, our 

recommendation will remain open. In addition, VA stated in its written 

comments that OPA has efforts underway related to staffing, training, and 

PATS data entry and assessment and provided estimated completion 

Recommendations for 

Executive Action

Agency Comments 

and Our Evaluation
Page 24 GAO-18-356 VHA Patient Advocacy Program

dates for these efforts. We will monitor VA’s efforts to address our 

recommendations. VA did not provide technical comments.

We are sending copies of this report to the appropriate congressional 

committees, the Secretary of Veterans Affairs, the Under Secretary for 

Health, and other interested parties. In addition, the report will be 

available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact 

me at (202) 512-7114 or at draperd@gao.gov. Contact points for our 

Offices of Congressional Relations and Public Affairs may be found on 

the last page of this report. GAO staff who made key contributions to this

report are listed in Appendix V.

Debra A. Draper

Director, Health Care
Appendix I: Awareness of the Patient 

Advocacy Program

Page 25 GAO-18-356 VHA Patient Advocacy Program

Our eight selected Department of Veterans Affairs (VA) medical centers 

(VAMC) use a variety of methods to make veterans or their 

representatives aware of the patient advocacy program, including 

providing brochures on the program, displaying signage, and providing 

program information on the VAMC’s website.1 (See fig. 2 for examples of 

patient advocacy program signage at some of the VAMCs we visited.)

 1

The term “VAMC” refers to an individual VAMC and any of its associated facilities, such 

as community-based outpatient clinics.

Appendix I: Awareness of the Patient 

Advocacy Program
Appendix I: Awareness of the Patient 

Advocacy Program

Page 26 GAO-18-356 VHA Patient Advocacy Program

Figure 2: Examples of Patient Advocacy Program Signage at Selected Department of Veterans Affairs Medical Centers
Appendix II: Patient Advocate Tracking System 

(PATS) Data Entry and Management

Page 27 GAO-18-356 VHA Patient Advocacy Program

Patient advocacy program staff, such as patient advocates or servicelevel advocates who are designated to respond to veterans’ feedback, 

enter feedback from veterans or their representatives in the Veterans 

Health Administration (VHA) Patient Advocate Tracking System (PATS) 

by creating a report of contact (ROC). Each ROC includes basic 

information regarding the individuals involved, a description of the 

feedback provided by the veteran, and a description of the steps taken to 

resolve the issue. Patient advocacy program staff assign one or more 

issue codes that generally describe the nature of the feedback, such as 

“coordination of care.” (See figures 3 and 4.)

Appendix II: Patient Advocate Tracking 

System (PATS) Data Entry and Management
Appendix II: Patient Advocate Tracking System 

(PATS) Data Entry and Management

Page 28 GAO-18-356 VHA Patient Advocacy Program

Figure 3: Example of Information Patient Advocacy Program Staff at VAMCs Enter in the Patient Advocate Tracking System

(PATS) to Add a Report of Contact (ROC)

Note: In addition to contacting a patient advocate, veterans may use VA’s Inquiry Routing and 

Information System, an Internet-based public messaging system, to express feedback.
Appendix II: Patient Advocate Tracking System 

(PATS) Data Entry and Management

Page 29 GAO-18-356 VHA Patient Advocacy Program

Figure 4: Example of Information Patient Advocacy Program Staff at Department of Veterans Affairs Medical Centers (VAMC) 

Enter in the Patient Advocate Tracking System to Close a Report of Contact (ROC)
Appendix II: Patient Advocate Tracking System 

(PATS) Data Entry and Management

Page 30 GAO-18-356 VHA Patient Advocacy Program

In order to organize veterans’ feedback, VHA categorizes feedback as 

either requests for information, compliments, or complaints. Within each 

of these categories VHA defines specific issue codes for program staff to 

select from based on the description of the veteran’s feedback. (See table 

2.)

Table 2: Issue Codes Used to Describe Veterans’ Feedback in the Veterans Health Administration (VHA) Patient Advocate 

Tracking System (PATS), Fiscal Year (FY) 2017

Category Issue description 

Request for information Application for care/eligibility for medical benefits

Request for information Department of Veterans Affairs (VA) billing for service

Request for information Advance directives

Request for information Referral issues (internal/community)

Request for information Medical center regulations

Request for information Obtaining copies of medical records/completion of forms

Request for information VA regional office questions on compensation and pension

Request for information Legal issues

Request for information Patient rights and responsibilities

Request for information Other

Request for information Social services

Compliment Compliment received from patient/family

member/staff

Complaint Excessive wait at facility for a scheduled appointment

Complaint Excessive wait at facility for an unscheduled appointment

Complaint Excessive delay in scheduling or rescheduling appointment

Complaint Delay/postponement in scheduled test/procedures or surgery

Complaint Delay in receiving test results

Complaint Excessive wait for care (inpatient)

Complaint Excessive wait for equipment/wrong equipment sent

Complaint Delay ordering/renewing pain medications

Complaint Delay ordering/renewing other medications

Complaint Excessive wait in pharmacy

Complaint Excessive wait for pharmacy mailings

Complaint Phone calls/letters/secure messages not returned/or timely

Complaint Delay in receiving travel pay

Complaint Delay in receiving completed forms from provider

Complaint Appeal - denial of application

Complaint Appeal - tier level hierarchy
Appendix II: Patient Advocate Tracking System 

(PATS) Data Entry and Management

Page 31 GAO-18-356 VHA Patient Advocacy Program

Category Issue description 

Complaint Appeal - revocation/removal from program

Complaint All other clinical appeals (not caregiver related)

Complaint Dissatisfied with referral outcome

Complaint Patient perceives care is not coordinated/expectations not met

Complaint Inconsistency in information given to patient/family

Complaint Appointment date/time misunderstood/not communicated/wrong

Complaint Diagnosis/care/prevention

Complaint Purpose/side effects of medication

Complaint Emotional needs not met

Complaint Provider is difficult to understand

Complaint Lack of privacy

Complaint Complaints concerning canteen cafeteria/store/vending areas

Complaint Difficulty finding parking

Complaint Cleanliness/temperature/sensory concerns

Complaint Signage not clear (directional maps etc.)

Complaint Family not involved in patient’s care

Complaint Application for care/eligibility for medical benefits

Complaint Eligibility for health care/follow up/hospital/extended/community living center

Complaint Dental eligibility 

Complaint Ambulance/private hospital/private care payment eligibility

Complaint VA billing for service/pharmacy co-payment

Complaint Prosthetics eligibility

Complaint Travel eligibility 

Complaint Complementary and integrative health

Complaint Patient does not have one provider

Complaint Patient does not know who is their provider 

Complaint Hygiene, diet, feeding, therapy, ambulation needs (inpatient/health care) 

Complaint Problems with pain management 

Complaint Patient/family not included in planning care

Complaint Patient/family disagrees about decisions on care

Complaint Lack of confidence or trust in caregiver

Complaint Request for non-formulary medication

Complaint Complaints concerning medical records

Complaint Concerns with confidentiality/privacy in medical record

Complaint Medical center regulations

Complaint VA regional office and/or compensation and pension issues

Complaint Fiduciary issues
Appendix II: Patient Advocate Tracking System 

(PATS) Data Entry and Management

Page 32 GAO-18-356 VHA Patient Advocacy Program

Category Issue description 

Complaint Missing and/or damaged personal property

Complaint Allegations of negligence/malpractice

Complaint Allegations of abuse

Complaint Medication error

Complaint Issue related to safety

Complaint Patient not treated with dignity and respect/perceived rudeness

Complaint Perceived retaliation for expressing concerns

Complaint Lack of coordination between inpatient and outpatient care

Complaint Deceased patient issues by a family member 

Complaint Excessive delay of discharge/discharge too soon 

Source: GAO analysis of VHA documents. | GAO-18-356

Note: This table includes issue codes used by VHA patient advocacy program staff to document 

feedback recorded in PATS in fiscal year (FY) 2017. In October 2017, the Office of Patient Advocacy 

updated the list of issue codes in PATS for use during FY 2018. The updated list deletes some 

existing codes, adds new codes, and updates definitions for some existing codes.

The Comprehensive Addiction and Recovery Act of 2016 (CARA), 

includes a provision for every VAMC to display the purpose of the 

program, along with the contact information of a patient advocate at the 

facility, in as many prominent locations as deemed appropriate to be seen 

by the largest percentage of veterans.1 In September 2016, VHA Central 

Office sent a memo to Veterans Integrated Service Network (VISN) 

directors explaining this requirement and an Office of Patient Centered 

Care and Cultural Transformation (OPCC&CT) official obtained 

confirmation from all VHA facilities that this requirement was met in 

October 2016. Nevertheless, officials from two veterans service 

organizations (VSO) we interviewed stated they often encounter veterans 

who are not aware of the patient advocacy program.

According to VA, in fiscal year (FY) 2017, there were 268,114 veterans 

associated with ROCs entered in PATS.2 VA also reported that, in the 

same year, patient advocacy program staff entered 414,256 unique 

reports of contact in PATS. According to VA, from the unique reports of 

contact in PATS, program staff documented 473,564 issues, which 

 1

Comprehensive Addiction and Recovery Act of 2016, Pub. L. No. 114-198, § 922, 130 

Stat. 695, 766-767 (2016).

2

VHA officials stated that veterans’ feedback is not always entered in PATS, so the 

number of unique veterans associated with reports entered in PATS may not represent 

the total percentage of veterans who provide feedback to patient advocacy program staff.
Appendix II: Patient Advocate Tracking System 

(PATS) Data Entry and Management

Page 33 GAO-18-356 VHA Patient Advocacy Program

included (but were not limited to) 112,722 requests for information, 

35,839 compliments, and 325,003 complaints.3 See table 3 for the top 

five issues that patient advocacy program staff across VAMCs entered in 

PATS for FY 2017. According to VA, in FY 2017, a total of 1,391 program 

staff system-wide entered data in PATS. In the same year, according to 

PATS, veterans, rather than family members or friends, most often 

provided feedback to patient advocacy program staff.

Table 3: The Most Common Issues Entered in the Patient Advocate Tracking System (PATS), according to the Department of 

Veterans Affairs (VA), Fiscal Year 2017

Category Issue description Number of issues entered in PATSa

Request for Information Referral issues (internal/community) 54,903

Compliment Compliment received from patient, family member, and/or staff 

member

35,839

Complaint Patient/family disagrees about decisions on care 31,718

Complaint Phone calls/letters/secure messages not returned/answered in 

a timely manner

24,949

Complaint Patient perceives care is not coordinated/expectations not met 23,617

Source: VA. | GAO-18-356

Notes: Data presented in the table were provided by VA officials in January 2018 based on the 

number of issues entered in PATS in fiscal year 2017. According to Veterans Health Administration 

(VHA) officials involved in overseeing the patient advocacy program, patient advocates may interpret 

an issue description (designated by a specific issue code) differently. These differing interpretations 

may have contributed to the data shown in this table. In addition, VHA officials stated that veterans’ 

feedback is not always entered in PATS, so the number of issues shown in the table may not 

represent all feedback that patient advocates received from veterans.

a

Patient advocates or other VA officials can assign more than one issue code to each report of 

contact (ROC) entered in PATS. According to VAMC officials, multiple issue codes may be assigned 

to one ROC when a veteran describes multiple issues at the same time, or when there are multiple 

issue codes that describe a veterans’ complaint. Therefore, the number of issues may not be 

equivalent to the number of unique ROCs for a given year.

Our eight selected VAMCs varied in the number of patient level 

advocates and service-level advocates who had access to PATS, 

whether veterans’ feedback was recorded outside of PATS, and which 

issue code or codes were used to record feedback related to the 

Veterans Choice Program.4 (See table 4.)

 3

According to VA officials, these numbers are likely an underestimate of feedback 

provided by veterans, as not all feedback is entered in PATS.

4

The Veterans Choice Program is a program that allows eligible veterans to receive health 

care from a community provider when faced with long wait times, lengthy travel distances, 

or other challenges accessing care at a VAMC.
Appendix II: Patient Advocate Tracking System 

(PATS) Data Entry and Management

Page 34 GAO-18-356 VHA Patient Advocacy Program

Table 4: Patient Advocate Tracking System (PATS) Data-Entry Practices at Eight Selected Department of Veterans Affairs 

Medical Centers (VAMC), as of September 2017

VAMC

Number of 

patient 

advocates who 

entered data in 

PATS

Number of

service-level

advocates who

entered data in

PATS

Recorded 

feedback 

outside of 

PATS?

Description of issue code(s) used to record feedback 

related to the Veterans Choice Programc

A 2 0 Yes Ambulance, private hospital, private care, payment 

eligibilityd

B 6 3 Yes —e

C 2 n/ab Yes Ambulance, private hospital, private care, payment eligibility

D 1a 2 No VA billing for service;

d VA billing for service, pharmacy 

co-payment 

E 4 0 Yes —f

F 3 13 Yes Dissatisfied with referral outcome

G 5 n/ab Yes Referral issues (internal, community)g

H 1 10 No —h

Legend: n/a = not applicable, em dash (—) = a single issue code was not identified

Source: GAO summary of information from selected VAMCs. | GAO-18-356

Notes: The PATS is VA’s electronic system for documenting and tracking veterans’ complaints and 

other feedback. The term “VAMC” refers to an individual VAMC and any of its associated facilities, 

such as community-based outpatient clinics. Service-level advocates, such as nurses or 

administrative staff, are designated at some VAMCs to respond to veterans’ feedback before 

involving a patient advocate.

a

VAMC officials stated that in addition to one patient advocate, there are six quality management staff 

that serve as back up to the patient advocate and enter information into PATS.

b

This VAMC did not utilize designated service-level advocates as part of the patient advocacy 

program.

c

The Veterans Choice Program is a program that allows eligible veterans to receive health care from 

a community provider when faced with long wait times, lengthy travel distances, or other challenges 

accessing care at a VAMC.

d

In addition to using the issue codes listed, VAMC officials stated they created a special entry in the 

PATS facility drop-down menu to code Veterans Choice Program feedback. When officials enter a 

report of contact (ROC) in PATS, each VAMC has a drop-down menu of locations within their VAMC 

that they populate for each ROC. According to VA officials, this is one of the only categories within

PATS that VAMC employees are able to change.

e

VAMC officials stated that there was no single code or codes used to categorize Veterans Choice 

Program feedback, but added the Veterans Choice Program to the medical service drop-down menu 

within PATS.

f

VAMC officials stated that there was no single code or codes used to categorize Veterans Choice 

Program feedback.

g

This issue code is used for requests for information with no associated complaints.

h

VAMC officials stated they created a special entry in the PATS facility drop-down menu to code 

Veterans Choice Program feedback.
Appendix II: Patient Advocate Tracking System 

(PATS) Data Entry and Management

Page 35 GAO-18-356 VHA Patient Advocacy Program

Examples of methods that patient advocates and service-level advocates 

used at selected VAMCs to record veterans’ feedback outside of PATS 

included call logs and tracking spreadsheets. VAMC officials indicated 

that recording information outside of PATS helped them track their 

responses to veterans’ feedback. Some of the information recorded 

outside of PATS was additional information that is not required to be 

entered into PATS, such as requests for information.5

 5

VHA’s handbook sets a goal requiring that complaints are entered into PATS, but does 

not require that requests for information and compliments are entered. The handbook 

states that requests for information and compliments can be entered into PATS, as this 

information is useful in determining common areas of confusion that could be addressed 

in a proactive manner. See VA, VHA Patient Advocacy Program, VHA Handbook 1003.4 

(Sept. 2, 2005).
Appendix III: Approaches to the Governance 

and Staffing of the Patient Advocacy Program

Page 36 GAO-18-356 VHA Patient Advocacy Program

The eight Department of Veterans Affairs (VA) medical centers (VAMC) 

selected for our review used a variety of approaches to govern the patient 

advocacy program, resulting in differences in the number of positions for 

patient advocates and service-level advocates and the title of the 

positions. Service-level advocates, such as nurses or administrative staff, 

are designated at some VAMCs to respond to veterans’ feedback before 

involving a patient advocate. (See table 5.)

Table 5: The Number of Positions and Titles Used for Patient Advocates and Service-Level Advocates at Eight Selected 

Department of Veterans Affairs Medical Centers (VAMC), as of September 2017

VAMC

Patient advocates Service-level advocates

Number of positions Title Number of positions Title

A 2 Patient advocate 28 Service-level liaison

B 6 Patient experience liaison 29 Service-level advocate

C 2 Patient representative 0 Service line champion

D 1 Patient representative 16 Patient advocate liaison 

E 4a Patient advocate 57 Patient advocate liaison

F 3 Patient representative 86 Service-level advocate

G 9b Patient advocate 0 Service-level advocate

H 1 Patient representative 10 Service line patient experience 

liaison

Source: GAO summary of information from selected VAMCs. | GAO-18-356

Notes: The term “VAMC” refers to an individual VAMC and any of its associated facilities such as 

community-based outpatient clinics. Service-level advocates, such as nurses or administrative staff, 

are designated at some VAMCs to respond to veterans’ feedback before involving a patient advocate.

a

This VAMC also has four patient advocates who only handle complaints related to non-VA care. 

Since these patient advocates report outside of the VAMC’s patient advocacy program, we did not 

include them in the count of patient advocates. According to an official from this VAMC, these 

advocates do not enter veterans’ feedback in PATS.

b

This VAMC also has two assistant patient advocates who help the patient advocates respond to 

veterans’ feedback. These assistants are not included in the number of patient advocates.

Patient advocates reported to a variety of departments among our 

selected VAMCs. At two of the VAMCs, patient advocates reported to the 

customer or consumer relations department, while at three, patient 

advocates reported to the quality management department. In addition, 

the placement of the department that patient advocates reported to within 

the VAMC differed. For example, the patient advocate supervisor at one 

of the selected VAMCs said that patient advocates reported to the 

customer service manager, who did not report directly to the VAMC 

director. At another VAMC, the patient advocate reported directly to the 

VAMC director.

Appendix III: Approaches to the Governance 

and Staffing of the Patient Advocacy 

Program
Appendix III: Approaches to the Governance 

and Staffing of the Patient Advocacy Program

Page 37 GAO-18-356 VHA Patient Advocacy Program

In addition to the Veterans Health Administration (VHA) handbook for the 

patient advocacy program, all eight of our selected VAMCs developed 

their own policies for the administration of the program, and these policies 

varied. For example, while almost all of the policies specified the 

responsibilities with respect to the patient advocacy program of the 

service chiefs—officials who oversee the administration and operation of 

service lines such as primary care, these responsibilities varied. For 

example, two of the policies required service chiefs to incorporate 

veterans’ feedback into performance measures used for VAMC staff 

external to the patient advocacy program, such as physicians, while the 

other policies did not.

We also found variation between our selected VAMCs with respect to 

whether they had written descriptions of the service-level advocates’ 

roles. Of the six VAMCs that designated service-level advocates, three 

had written descriptions of their roles, while three did not. Further, among 

the VAMCs that had a written description of the role of a service-level 

advocate, the expectations for these advocates varied. For example, one 

VAMC’s written description specified that service-level advocates are 

expected to enter veterans’ feedback into PATS within 7 days of receiving 

the feedback. The written descriptions at the other two VAMCs did not 

specify this expectation.
Appendix IV: Comments from the Department 

of Veterans Affairs

Page 38 GAO-18-356 VHA Patient Advocacy Program

Appendix IV: Comments from the 

Department of Veterans Affairs
Appendix IV: Comments from the Department 

of Veterans Affairs

Page 39 GAO-18-356 VHA Patient Advocacy Program
Appendix IV: Comments from the Department 

of Veterans Affairs

Page 40 GAO-18-356 VHA Patient Advocacy Program
Appendix V: GAO Contact and Staff 

Acknowledgments

Page 41 GAO-18-356 VHA Patient Advocacy Program

Debra A. Draper, (202) 512-7114 or draperd@gao.gov

In addition to the contact named above, Hernán Bozzolo (Assistant 

Director), Rebecca Rust Williamson (Analyst-in-Charge), Jennie F. Apter, 

Q. Akbar Husain, and Emily Loriso made key contributions to this report. 

Also contributing were Julie Flowers, Jacquelyn Hamilton, and Vikki 

Porter.

Appendix V: GAO Contact and Staff 

Acknowledgments

GAO Contact

Staff 

Acknowledgments

(101674)
The Government Accountability Office, the audit, evaluation, and investigative 

arm of Congress, exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability of the 

federal government for the American people. GAO examines the use of public 

funds; evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through GAO’s website (https://www.gao.gov). Each weekday afternoon, GAO 

posts on its website newly released reports, testimony, and correspondence. To 

have GAO e-mail you a list of newly posted products, go to https://www.gao.gov

and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of production and 

distribution and depends on the number of pages in the publication and whether 

the publication is printed in color or black and white. Pricing and ordering 

information is posted on GAO’s website, https://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, 

Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at https://www.gao.gov.

Contact:

Website: https://www.gao.gov/fraudnet/fraudnet.htm

E-mail

Automated answering system: (800) 424-5454 or (202) 512-7470

Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400, 

U.S. Government Accountability Office, 441 G Street NW, Room 7125, 

Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707

U.S. Government Accountability Office, 441 G Street NW, Room 7814, 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Strategic Planning and 

External Liaison

Please Print on Recycled Paper.