text: Background VHA’s patient advocacy program is intended to provide veterans with a means to provide feedback about health care services they receive at VAMCs. VHA sets forth minimum expectations for VAMCs’ administration of the program, including that veterans must have easy access to a patient advocate and must have their complaints addressed in a convenient and timely manner. Administration of the Patient Advocacy Program The patient advocacy program is administered at the VAMC level. Each of VA’s 170 VAMCs is responsible for making at least one patient advocate available to respond to veterans’ feedback, and for ensuring that feedback is recorded in PATS. VAMCs may designate other staff to assist patient advocates in responding to feedback, such as lead patient advocates and service-level advocates. Service-level advocates, such as nurses or administrative staff, are designated at some VAMCs to respond to veterans’ feedback before involving a patient advocate. All VAMC staff that have a designated role in the administration of the patient advocacy program are referred to as patient advocacy program staff. In addition to designating program staff, VAMCs may use a variety of methods to make veterans aware of the patient advocacy program, such as displaying signage on site and including information about the program on their websites. (See app. I for more information on the methods selected VAMCs used to make veterans aware of the program.) Patient advocacy program staff enter veterans’ feedback in PATS using a report of contact (ROC) and assign one or more issue codes that generally describe the nature of the feedback, such as coordination of care. (See app. II for additional information on entering veterans’ feedback into PATS.) Each piece of feedback shared is categorized as either a request for information, compliment, or complaint. VHA’s handbook for the program specifies certain goals for data collection and resolution—specifically, that all complaints should be entered in PATS to enable a comprehensive understanding of veterans’ issues and concerns to, in turn, identify potential system-wide improvements; and responses should occur no later than 7 days after the complaint is made. With this guidance, patient advocacy program staff use a variety of approaches for entering veterans’ feedback in PATS and closing it in the system once addressed. For example, when VAMCs have designated service-level advocates, the process for entering and closing feedback in PATS is generally different than the approach used by VAMCs that have only patient advocates. (See fig. 1.) Patient advocacy program staff at each VAMC are assisted by a VISN- level coordinator who acts as a liaison between the VAMCs and VHA and is responsible for ensuring consistency in PATS data collection within the VISN. The VISN director is responsible for designating the coordinator and ensuring that each VAMC within the VISN has at least one patient advocate. Oversight of the Patient Advocacy Program The VHA office responsible for overseeing the patient advocacy program changed as a result of CARA. From January 2011 to July 2017, the program was overseen by OPCC&CT under VHA’s Deputy Under Secretary for Health for Operations & Management. CARA included a provision for VHA to establish OPA to begin overseeing the program and specified that this office would report directly to the Under Secretary for Health, a higher-level office within VHA. Although OPCC&CT is no longer responsible for overseeing the program, it is to continue to play an advisory role to OPA during the initial phases of its work, according to OPCC&CT officials. Many of OPA’s oversight responsibilities are specified in CARA including ensuring that patient advocates advocate on behalf of veterans, manage PATS, and identify trends in the data to determine whether there are opportunities for improving veterans’ health care. Also, OPA’s director is required to ensure that patient advocates receive relevant, consistent training across VAMCs. When establishing the office in July 2017, VHA officials wrote a memo indicating that OPA’s primary objectives were to implement a standardized policy for the patient advocacy program and to resolve any system-wide issues, such as concerns about care across VAMCs identified through veterans’ feedback. In addition, in August 2017, OPA began soliciting feedback from VAMCs on various aspects of the patient advocacy program to identify improvement priorities and best practices. By September 2017, OPA had identified an acting program director, established a workgroup (called the National Strategic Workgroup) to develop recommendations related to program administration, and finalized a charter that identifies workgroup deliverables. VHA Has Provided Limited, Outdated Guidance to VAMCs on the Governance of the Patient Advocacy Program VHA has provided limited guidance to VAMCs on the governance of the patient advocacy program. Specifically, VHA provided limited guidance on how to meet the program’s expectations that veterans have easy access to a patient advocate who will hear their complaints and address them in a timely manner. While VHA’s handbook for the program provides general information on the responsibilities of patient advocacy program staff, it does not specify the VAMC department to which patient advocates should report to help ensure VAMCs meet these expectations. According to VHA officials, the lack of specific guidance was intentional and due in part to VHA officials’ view that leadership at each VAMC is in the best position to understand the needs of veterans at their facilities, and therefore should have flexibility to make decisions about governance in response to those needs. In addition to providing limited guidance to VAMCs, VHA’s patient advocacy program handbook is out of date and does not incorporate recent agency-wide changes, such as those made in response to VHA Strategic Plan FY 2013 – 2018 which identifies the goal of providing proactive, patient-driven health care. The handbook for the program was issued in 2005, expired in 2010, and as of January 2018, no updates had been released. In the absence of an updated document, VAMCs are still expected to follow the outdated handbook. However, the handbook does not identify the responsibilities of the current VHA office responsible for overseeing the program. Instead, it identifies the responsibilities of the VHA office that oversaw the program before OPCC&CT began overseeing the program in 2011. In recent years, OPCC&CT reviewed the implementation of the patient advocacy program at some VAMCs and provided specific recommendations on how to change program governance to better reflect a more proactive patient advocacy program model. However, the recommendations from these reviews were provided only to some VAMCs; guidance that could be applicable to all VAMCs was not added to the handbook. OPCC&CT officials stated that they did not update the handbook because they decided to instead spend time trying to understand recent feedback they received from VAMC officials and ensure that any updates would reflect system-wide shifts as a result of VHA’s strategic plan. OPCC&CT’s limited and outdated guidance to VAMCs on the governance of the patient advocacy program is inconsistent with federal internal control standards for the control environment, which require agencies to establish an organizational structure, assign responsibility, and delegate authority to achieve agency objectives—key aspects of governance. To do so, an agency may develop an organizational structure that assigns responsibilities to discrete units and defines reporting lines at all levels of the organization. Without providing specific, timely guidance to VAMCs on the governance of the patient advocacy program, the program is at risk of not meeting its minimum expectations. In light of the limited and outdated guidance on the governance of the program, patient advocacy program staff at most of our selected VAMCs noted that the VAMC department to which patient advocates report can have a direct effect on the ability of staff to resolve veterans’ complaints. For example, patient advocates at one VAMC said because of the program’s position within the organization, they did not have the authority to ensure that VAMC officials external to the patient advocacy program, such as physicians, quickly engaged in responding to veterans’ complaints. In these cases, a patient advocate would contact the physician to resolve a complaint, but may not have received a response until the matter was brought to the attention of the physician’s supervisor—a reporting line that is outside of the patient advocacy program at this VAMC. Officials from several of our selected VAMCs and VSOs noted that the position of the patient advocacy program within VAMCs may not give patient advocates the authority to require VAMC staff to respond to veterans’ complaints. They added that conflict-of- interest concerns could arise when a veteran has a complaint about a VAMC for which the patient advocate works. (See app. III for additional information on the governance of the patient advocacy program at selected VAMCs.) In VA’s written comments on a draft of this report, which are reproduced in Appendix IV, VA stated that it issued its new directive for the patient advocacy program that had been in development as we were conducting our review. While the updated directive specifies that a VAMC’s lead patient advocate should report to the facility director, it does not specify the VAMC department to which other patient advocacy program staff, including patient advocates who are not designated as lead patient advocates and service-level advocates, should report. In addition, OPA’s National Strategic Workgroup recently submitted recommendations to OPA on the governance of the patient advocacy program. OPA officials stated that they plan to prioritize the recommendations and elicit feedback from VISN directors on how to operationalize the recommendations. However, it is unclear whether OPA will provide additional guidance related to the governance of the program based on these recommendations, such as guidance on the VAMC department to which all types of patient advocacy program staff should report. Until actions to address the weaknesses we found are completed, guidance on the governance of the program will continue to be lacking. VHA Has Provided Limited Guidance to VAMCs on Staffing the Patient Advocacy Program VHA has provided limited guidance to VAMCs on the number and type of patient advocacy program staff needed to ensure that complaints from veterans are addressed in a convenient and timely manner. According to VHA’s existing handbook for the program, every VAMC should have at least one patient advocate and appropriate administrative, technical, and clerical support should be provided to allow for efficient performance of the responsibilities of program staff. OPCC&CT did not provide guidance on how VAMCs should determine the appropriate number of administrative, technical, and clerical staff or type of patient advocacy program staff, such as lead patient advocates and service-level advocates. According to officials, this was because no assessment was conducted to identify what staff resources would be needed to meet the expectations of the program. In the absence of such an assessment, OPCC&CT instead relied on each VAMC to determine what resources would be needed based on the facility’s size and services provided. However, VHA’s handbook for the program does not provide instruction for VAMC or VISN officials on how to determine the number and type of staff needed for the program. OPCC&CT officials added that budget constraints can also affect a VAMC’s ability to hire the appropriate staff for the program. (See app. III for additional information on the number and type of patient advocacy program staff at selected VAMCs.) Officials at all but one of the selected VAMCs stated that program staff at their VAMCs had more work to do than they could handle. For example, VAMC officials cited backlogs in work, such as calls from veterans not being answered, messages not being responded to, voicemail boxes being full, and not all veterans’ feedback being entered into PATS. Officials from one VAMC we spoke with in July 2017 stated that due to workload demands and not enough patient advocacy program staff at their VAMC, they had roughly 300 unanswered phone calls at that time from veterans who want to provide feedback to a patient advocate. Officials from several VSOs we spoke with stated that there is not enough patient advocate staff, adding that veterans reported that their calls to patient advocates were not answered, they were unable to reach an advocate, or their calls were not responded to in a timely manner. The lack of staffing guidance is inconsistent with GAO’s Key Principles for Effective Strategic Workforce Planning, which states that workforce planning is essential to addressing an organization’s critical need to align its human capital program with its current and emerging mission and programmatic goals. Further, federal internal control standards require agencies to design control activities to achieve objectives, a key aspect of effectively staffing a program. Such control activities may include effectively managing the agency’s workforce, such as by continually assessing the knowledge, skills, and abilities of the workforce to achieve organizational goals. The lack of guidance on staffing may impede VAMCs’ efforts to ensure that they have the appropriate number and type of staff to administer the patient advocacy program. The resulting misalignment of staff resources could have negatively affected VAMCs’ ability to achieve the program’s objectives, including addressing veterans’ complaints in a timely manner. For example, if there are not a sufficient number of patient advocates to respond to veterans’ phone calls in a timely manner, VAMCs may not be able to ensure that patient advocates can respond to veterans’ complaints within 7 days, as called for by VHA’s handbook for the program. According to VHA officials, OPA analyzed feedback from VAMCs on the factors that should be considered in developing national guidelines for staffing, such as facility size and complexity level, and directed its National Strategic Workgroup to develop recommendations for determining the extent to which VAMCs have utilized various patient advocacy program staff, such as service-level advocates, by the spring of 2018. However, OPA expects that these efforts will result in recommendations for consideration, and it is unclear what steps, if any, will be taken based on the recommendations. Until actions to address the weaknesses we found are completed, the lack of guidance for VAMCs on determining the appropriate number and types of staff will put the patient advocacy program at risk of being unable to address veterans’ complaints in a convenient and timely manner. VHA Has Recommended Training for Patient Advocates, but Has Not Developed an Approach to Routinely Assess Their Training Needs or Monitored Training Completion VHA Has Developed a Recommended Training List for Patient Advocates, but Has Not Developed an Approach to Assess Their Training Needs on a Routine Basis VHA has recently developed a list of recommended training for patient advocates. In the spring of 2017, OPCC&CT officials updated a recommended training list for patient advocates developed before 2011 when OPCC&CT began overseeing the patient advocacy program. The training list covers a wide variety of topics, including how to enter and examine trends in PATS data, as well as key responsibilities of patient advocates outlined in VHA’s handbook for the program. OPCC&CT officials stated that they would like to make the trainings required, but have not pursued this because of the lengthy process within VHA to designate required training for a specific group of staff. To update the list in 2017, OPCC&CT convened a workgroup (which included several patient advocates) to determine whether the old training list was sufficient, and the workgroup shared its suggested updates with VISN- level coordinators for distribution to VAMCs in April 2017. We found that OPCC&CT has not developed an approach to routinely assess the training needs of patient advocates. Rather, OPCC&CT officials stated that they relied on VAMC and VISN staff to conduct these assessments. However, VHA’s handbook for the program does not specify that VAMC or VISN officials are responsible for conducting routine assessments of patient advocates’ training needs. None of our selected VAMCs routinely conducted assessments of the training needs of patient advocates, such as assessing whether advocates were adequately trained to carry out their responsibilities. Officials from two VAMCs said they used ad hoc approaches to assess training needs. For example, one patient advocate supervisor stated that training is offered on an “as needed” basis in patient advocate meetings when a training need is identified. The lack of an approach for routinely assessing the training needs of patient advocates is inconsistent with federal standards for internal control related to control activities. Under these standards relating to human capital, management ensures that training is aimed at developing and retaining employee knowledge, skills, and abilities to meet changing organizational needs. Management should also continually assess the knowledge, skills, and ability needs of a program so that the program is able to obtain a workforce that has the required knowledge, skills, and abilities to achieve organizational goals. Without an approach for routinely assessing the training needs of patient advocates, VHA may not be able to clearly identify gaps in the knowledge and skills of these staff over time, which, in turn, could put the program at risk of not meeting its goals. For example, if there is a gap in understanding among patient advocates that all complaints should be entered into PATS, addressing veterans’ complaints may be delayed, if addressed at all, and opportunities to analyze complaint data for the purpose of identifying system-wide improvements may be missed. According to VHA officials, OPA analyzed feedback from VAMCs on the training needs of patient advocates, including how to correctly enter data into PATS, and directed its National Strategic Workgroup to develop recommendations for assessing the training needs of patient advocates by the spring of 2018. OPA expects that these efforts will result in recommendations for OPA to consider, but it is unclear what steps, if any, will be taken based on the recommendations. Until actions to address the weaknesses we found are completed, the lack of routine assessments of training needs will continue to put the program at risk of staff not having the requisite skills and knowledge to carry out their duties. VHA Has Not Monitored Training Completion for Patient Advocates VHA has not monitored the completion of training for patient advocates. Specifically, OPCC&CT officials said that they did not monitor the extent to which patient advocates completed the recommended training distributed in April 2017. Instead, these officials relied on patient advocate supervisors to monitor training completion. However, VHA’s handbook for the program does not specify that patient advocate supervisors are responsible for monitoring the completion of training for patient advocates. Half of patient advocate supervisors at our selected VAMCs did not track the completion of patient advocacy training. Patient advocate supervisors said that they are able to track the completion of general VA employee training through VA’s Talent Management System. However, most training specific to patient advocacy were generally not included in this system during the period of our review. Officials from our selected VAMCs who did track patient advocacy training used various methods to record completion, such as keeping attendance lists for the training provided. Taking steps to monitor training completion would be consistent with GAO’s Guide for Assessing Strategic Training and Development Efforts in the Federal Government which identifies components of the training and development process, including having agencies collect and monitor data corresponding to establishing training objectives. Monitoring training completion would also be consistent with federal standards for internal control related to control activities. Under these standards relating to human capital, management ensures that training is aimed at developing and retaining employee knowledge, skills, and abilities to meet changing organizational needs. Management also continually assesses the knowledge, skills, and ability needs of a program so that the program is able to obtain a workforce that has the required knowledge, skills, and abilities to achieve organizational goals—key components for monitoring training completion. If patient advocates are not properly trained in how to use PATS to document and resolve complaints, tracking the status of complaints may be more difficult, which could increase the likelihood that they are not addressed in a timely manner, if at all. Further, CARA specifies that the director of OPA should ensure that patient advocates receive training specific to patient advocacy. According to VHA officials, OPA did not obtain information on whether patient advocates completed recommended training and did not identify an approach for monitoring training completion moving forward. Without monitoring training completion, there is an increased risk that patient advocates have not received the training they need to effectively fulfill their responsibilities such as advocating on behalf of veterans and consistently using PATS to document and resolve complaints. VHA Has Not Monitored Patient Advocacy Data-Entry Practices or Reviewed Patient Advocacy Data to Assess Program Performance and Identify System-Wide Improvements VHA Has Not Monitored Whether Complaints Were Always Entered into PATS and Issue Codes Assigned Consistently VHA officials have not monitored PATS data-entry practices to ensure complaints were always entered into PATS and issue codes were assigned consistently to ROCs. OPCC&CT officials told us they did not monitor the data-entry practices of patient advocacy program staff to ensure that all complaints were entered into PATS, a key goal according to VHA’s handbook for the program. Rather, they relied on VISN and VAMC officials to ensure that program staff entered all complaints into PATS. Officials from two of the five VISNs we interviewed stated that they did not perform any audits or checks of the data entered into PATS by patient program staff at VAMCs. We also found inconsistencies in the extent to which VAMC officials entered complaints into PATS, with complaints always entered into PATS at one of our selected VAMCs, while at other VAMCs some complaints were left unrecorded, according to officials. For example, at one VAMC, officials stated that over a third of the complaints received were not entered into PATS due to the competing workload demands of patient advocates. Similarly, at another selected VAMC, almost a quarter of the complaints received were not entered into PATS, according to patient advocates there who explained that they primarily used a document outside of PATS to record veterans’ feedback. In addition, OPCC&CT officials told us they did not monitor whether patient advocates used a consistent practice to assign issue codes to veterans’ feedback recorded into PATS. Using a consistent data-entry practice is important to ensure that PATS data can be compared across VAMCs to better enable an accurate and comprehensive understanding of veterans’ issues and concerns, a goal of the patient advocacy program. OPCC&CT officials stated that they relied on VISN-level coordinators to monitor coding practices because VHA’s handbook for the program states that these coordinators should develop VISN-wide consistent approaches for entering complaints into PATS. VISN-level coordinators from two selected VISNs stated that they created a standard practice for assigning issue codes within a particular VISN; however, the coding practices differed between VISNs, making national level analysis difficult. We also found inconsistencies in how VAMC officials coded specific veterans’ feedback. For example, patient advocates did not use consistent practices to code issues related to the Veterans Choice Program (Choice Program), one of the most common types of issues patient advocates told us they hear about from veterans. Officials from one of our selected VAMCs said they code feedback related to the Choice Program under a specific “request for information” issue code, regardless of whether the feedback was a request for information, compliment, or complaint. In contrast, officials at another VAMC stated that they typically code feedback related to the Choice Program as a complaint related to billing. (See app. II for additional information on data- entry practices at selected VAMCs.) OPCC&CT’s lack of monitoring of PATS data-entry practices is inconsistent with GAO’s Assessing the Reliability of Computer-Processed Data which identifies the importance of consistent data-entry practices to ensure that data are reasonably complete and accurate. Further, federal standards for internal control related to information and communications require agencies to use quality information, such as relevant data from reliable sources, to achieve the agency’s objectives. Under internal control standards for control activities, management also is to monitor performance to achieve objectives. Without OPCC&CT monitoring data- entry practices, the patient advocacy program is at risk of not meeting its goal that all complaints are entered into PATS and there is an increased likelihood of VHA not having an accurate understanding of veterans’ complaints across VAMCs. Moving forward, in fall 2017, OPA distributed meeting minutes to all VISN and VAMC directors stating that all veterans’ feedback should be consistently recorded in PATS. OPA officials also updated some of the issue codes in PATS in fall 2017 and added a code specifically for community care issues, such as issues related to the Choice Program. In addition, OPA officials stated that they plan to promote the consistent assignment of issue codes to veterans’ feedback through national training, but have not specified when this training will occur or if OPA staff will monitor patient advocates’ consistent assignment of issue codes or of data-entry practices generally. Until these actions are completed, however, the gaps in monitoring of PATS data-entry practices that we identified will continue to exist, putting the program at risk of incomplete or unreliable data that may not allow an accurate understanding of veterans’ complaints, critical to making system-wide improvements. VHA Has Not Systematically Reviewed PATS Data to Assess Program Performance and Identify Potential System- Wide Improvements VHA officials have not systematically reviewed PATS data to assess program performance and identify potential system-wide improvements, goals of the patient advocacy program. Specifically, OPCC&CT officials stated that they reviewed PATS data in response to inquiries, but did not conduct systematic reviews of the data over time. For example, they did not track VAMC performance on responding to complaints in a timely manner or track the most common complaints across VAMCs to identify potential opportunities for system-wide improvements. OPCC&CT officials stated that they did not conduct systematic reviews of PATS data because VISN and VAMC officials were primarily responsible for these analyses. However, according to VHA’s handbook for the patient advocacy program, VHA officials have a responsibility to examine PATS data for trends across VAMCs and identify any areas for system- wide improvement. Officials stated that it was challenging to analyze PATS information included in narrative text, such as descriptions of veterans’ feedback. Not reviewing PATS data is inconsistent with federal standards for internal control for monitoring which require agencies to establish and operate monitoring activities, such as assessing the quality of performance over time, and evaluate the results. Further, not conducting systematic assessments of PATS data made it difficult for OPCC&CT to determine program performance, such as whether the program was meeting its goal that all complaints are entered into PATS and responded to within 7 days. Officials explained that VHA interprets this goal to mean that complaints are closed in PATS within 7 days. According to VA, between FY 2014 and FY 2017 there were more than 53,000 complaints per year open for greater than 7 days. If OPCC&CT officials had conducted systematic reviews of PATS data, they may have been able to identify that there were a significant number of complaints open for longer than 7 days and consider what actions should be taken, such as providing additional guidance to VAMCs on how to address complaints in a timely manner. Furthermore, without systematically reviewing PATS data across VAMCs to identify potential system-wide improvements, OPCC&CT officials may have been unaware of important care issues across VAMCs. For example, patient advocates from several of our selected VAMCs stated that opioid prescription issues are among the most common complaints they received from veterans. If OPCC&CT officials were to have systematically reviewed PATS data across VAMCs to determine the prevalence of these types of complaints, they could have identified the need to address them on a national level and consider system-wide policies or guidance in response. According to VHA officials, OPA is in the process of identifying the data it needs to review on a routine basis, and directed its National Strategic Workgroup to identify program data that could be reviewed to assess program performance and identify potential system-wide improvements by the spring of 2018. However, OPA expects that these efforts will result in recommendations for OPA to consider, and it is unclear what steps, if any, will be taken based on the recommendations. Until actions to address the weaknesses we found are completed, the lack of a systematic review of PATS data will persist, putting the program at continued risk of missed opportunities for identifying and addressing weaknesses across VAMCs. Conclusions As one of the largest health care delivery systems in the nation, it is critically important for VHA to ensure that each veteran who receives health care services has easy access to an advocate who listens to that veteran’s feedback and responds in a timely manner. This is especially important given concerns about veterans’ ability to receive timely and quality care. However, VHA’s efforts to ensure that the patient advocacy program is meeting its goals—to identify potential system-wide improvements and respond to complaints within 7 days—have fallen short. OPCC&CT did not provide sufficient oversight to the program in the four key areas of governance, staffing, training, and data-entry practices, which has left the program at risk for not meeting its goals. VHA’s newly established OPA has initiated plans to improve the patient advocacy program in these four areas; however, most of these plans center around a workgroup that will make recommendations for OPA to consider, and it is unclear what specific actions, if any, will be taken based on these recommendations. Further, documentation for several of OPA’s planned efforts has not been finalized. Unless specific actions to address the weaknesses we identified are completed expeditiously, the program is at risk of not meeting its goals, including addressing veterans’ complaints in a convenient and timely manner. Furthermore, without addressing the weaknesses we identified, OPA misses opportunities to review PATS data across VAMCs to identify potential system-wide issues that, if addressed, could significantly improve the experience of veterans. Such reviews are critical to ensuring that VHA is taking steps to both meet its goal in its strategic plan to provide veterans with timely and quality health care, and to address recent issues it has faced, such as veterans’ ability to access care in a timely manner. Recommendations for Executive Action We are making the following six recommendations to the VHA Undersecretary for Health: provide updated guidance to VAMCs on the governance of the patient advocacy program, including clear definitions of reporting lines. (Recommendation 1) assess and provide guidance to VAMCs on appropriately staffing the patient advocacy program, including guidance on how to determine the appropriate number and type of staff. (Recommendation 2) develop an approach to routinely assess the training needs of patient advocates. (Recommendation 3) monitor the completion of training for patient advocates. (Recommendation 4) monitor PATS data-entry practices to ensure all complaints are entered into PATS and that veterans’ feedback is coded consistently. (Recommendation 5) systematically review PATS data to assess program performance and identify potential system-wide improvements. (Recommendation 6) Agency Comments and Our Evaluation We provided a draft of this report to VA for comment. In its written comments, which are reproduced in Appendix IV, VA concurred with our recommendations and noted that it recently issued the new directive for patient advocacy that had been in development as we were conducting our review. The directive supersedes the outdated handbook for the patient advocacy program and describes certain aspects of program governance, including certain reporting lines, roles, and responsibilities. Accordingly, VA requested that we close our first recommendation related to governance. We revised our report to reflect the issuance of the new directive. However, we do not believe the directive fully implements our recommendation. While the updated directive specifies that a VAMC’s lead patient advocate should report to the facility director, it does not specify the VAMC department to which other patient advocacy program staff, including patient advocates who are not designated as lead patient advocates and service-level advocates, should report. Until VA specifies the reporting lines for these other patient advocacy program staff, our recommendation will remain open. In addition, VA stated in its written comments that OPA has efforts underway related to staffing, training, and PATS data entry and assessment and provided estimated completion dates for these efforts. We will monitor VA’s efforts to address our recommendations. VA did not provide technical comments. We are sending copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, the Under Secretary for Health, and other interested parties. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or at draperd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in Appendix V. Appendix I: Awareness of the Patient Advocacy Program Our eight selected Department of Veterans Affairs (VA) medical centers (VAMC) use a variety of methods to make veterans or their representatives aware of the patient advocacy program, including providing brochures on the program, displaying signage, and providing program information on the VAMC’s website. (See fig. 2 for examples of patient advocacy program signage at some of the VAMCs we visited.) Appendix II: Patient Advocate Tracking System (PATS) Data Entry and Management Patient advocacy program staff, such as patient advocates or service- level advocates who are designated to respond to veterans’ feedback, enter feedback from veterans or their representatives in the Veterans Health Administration (VHA) Patient Advocate Tracking System (PATS) by creating a report of contact (ROC). Each ROC includes basic information regarding the individuals involved, a description of the feedback provided by the veteran, and a description of the steps taken to resolve the issue. Patient advocacy program staff assign one or more issue codes that generally describe the nature of the feedback, such as “coordination of care.” (See figures 3 and 4.) In order to organize veterans’ feedback, VHA categorizes feedback as either requests for information, compliments, or complaints. Within each of these categories VHA defines specific issue codes for program staff to select from based on the description of the veteran’s feedback. (See table 2.) The Comprehensive Addiction and Recovery Act of 2016 (CARA), includes a provision for every VAMC to display the purpose of the program, along with the contact information of a patient advocate at the facility, in as many prominent locations as deemed appropriate to be seen by the largest percentage of veterans. In September 2016, VHA Central Office sent a memo to Veterans Integrated Service Network (VISN) directors explaining this requirement and an Office of Patient Centered Care and Cultural Transformation (OPCC&CT) official obtained confirmation from all VHA facilities that this requirement was met in October 2016. Nevertheless, officials from two veterans service organizations (VSO) we interviewed stated they often encounter veterans who are not aware of the patient advocacy program. According to VA, in fiscal year (FY) 2017, there were 268,114 veterans associated with ROCs entered in PATS. VA also reported that, in the same year, patient advocacy program staff entered 414,256 unique reports of contact in PATS. According to VA, from the unique reports of contact in PATS, program staff documented 473,564 issues, which included (but were not limited to) 112,722 requests for information, 35,839 compliments, and 325,003 complaints. See table 3 for the top five issues that patient advocacy program staff across VAMCs entered in PATS for FY 2017. According to VA, in FY 2017, a total of 1,391 program staff system-wide entered data in PATS. In the same year, according to PATS, veterans, rather than family members or friends, most often provided feedback to patient advocacy program staff. Our eight selected VAMCs varied in the number of patient level advocates and service-level advocates who had access to PATS, whether veterans’ feedback was recorded outside of PATS, and which issue code or codes were used to record feedback related to the Veterans Choice Program. (See table 4.) Examples of methods that patient advocates and service-level advocates used at selected VAMCs to record veterans’ feedback outside of PATS included call logs and tracking spreadsheets. VAMC officials indicated that recording information outside of PATS helped them track their responses to veterans’ feedback. Some of the information recorded outside of PATS was additional information that is not required to be entered into PATS, such as requests for information. Appendix III: Approaches to the Governance and Staffing of the Patient Advocacy Program The eight Department of Veterans Affairs (VA) medical centers (VAMC) selected for our review used a variety of approaches to govern the patient advocacy program, resulting in differences in the number of positions for patient advocates and service-level advocates and the title of the positions. Service-level advocates, such as nurses or administrative staff, are designated at some VAMCs to respond to veterans’ feedback before involving a patient advocate. (See table 5.) Patient advocates reported to a variety of departments among our selected VAMCs. At two of the VAMCs, patient advocates reported to the customer or consumer relations department, while at three, patient advocates reported to the quality management department. In addition, the placement of the department that patient advocates reported to within the VAMC differed. For example, the patient advocate supervisor at one of the selected VAMCs said that patient advocates reported to the customer service manager, who did not report directly to the VAMC director. At another VAMC, the patient advocate reported directly to the VAMC director. In addition to the Veterans Health Administration (VHA) handbook for the patient advocacy program, all eight of our selected VAMCs developed their own policies for the administration of the program, and these policies varied. For example, while almost all of the policies specified the responsibilities with respect to the patient advocacy program of the service chiefs—officials who oversee the administration and operation of service lines such as primary care, these responsibilities varied. For example, two of the policies required service chiefs to incorporate veterans’ feedback into performance measures used for VAMC staff external to the patient advocacy program, such as physicians, while the other policies did not. We also found variation between our selected VAMCs with respect to whether they had written descriptions of the service-level advocates’ roles. Of the six VAMCs that designated service-level advocates, three had written descriptions of their roles, while three did not. Further, among the VAMCs that had a written description of the role of a service-level advocate, the expectations for these advocates varied. For example, one VAMC’s written description specified that service-level advocates are expected to enter veterans’ feedback into PATS within 7 days of receiving the feedback. The written descriptions at the other two VAMCs did not specify this expectation. Appendix IV: Comments from the Department of Veterans Affairs Appendix V: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, Hernán Bozzolo (Assistant Director), Rebecca Rust Williamson (Analyst-in-Charge), Jennie F. Apter, Q. Akbar Husain, and Emily Loriso made key contributions to this report. Also contributing were Julie Flowers, Jacquelyn Hamilton, and Vikki Porter.
target: VHA has designated patient advocates at each VAMC to receive and document feedback from veterans or their representatives, including requests for information, compliments, and complaints. In recent years, the importance of a strong patient advocacy program has taken on new significance given concerns with VHA's ability to provide veterans timely access to health care, among other issues. The Comprehensive Addiction and Recovery Act of 2016 included a provision for GAO to review VHA's patient advocacy program. This report examines the extent to which VHA has (1) provided guidance on the governance of the program; (2) provided guidance on staffing the program; (3) assessed the training needs of patient advocates and monitored training completion; and (4) monitored patient advocacy program data-entry practices and reviewed program data. GAO reviewed VHA and VAMC documents, including summaries of program data. GAO interviewed VHA officials about the program, as well as officials from a non-generalizable selection of eight VAMCs and five VISNs selected based on the volume of veteran complaints and other factors. GAO also compared VHA policies and practices to federal internal control standards. The Veterans Health Administration (VHA) provided limited guidance to Department of Veterans Affairs (VA) medical centers (VAMC) on the governance of its patient advocacy program and its guidance, a program handbook, has been outdated since 2010. VAMCs are still expected to follow the outdated handbook, which does not provide needed details on governance, such as specifying the VAMC department to which patient advocates should report. Officials from most of the VAMCs that GAO reviewed noted that the VAMC department to which patient advocates report can have a direct effect on the ability of staff to resolve veterans' complaints. The lack of updated and complete guidance may impede the patient advocacy program from meeting its expectations, to receive and address complaints from veterans in a convenient and timely manner. VHA also has provided limited guidance to VAMCs on staffing the patient advocacy program. VHA's handbook states that every VAMC should have at least one patient advocate and appropriate support staff; however, it did not provide guidance on how to determine the number and type of staff needed. Officials at all but one of the eight VAMCs in GAO's review stated that their patient advocacy program staff had more work to do than they could accomplish. This limited guidance on staffing could impede VAMCs' efforts to ensure that they have the appropriate number and type of staff to address veterans' complaints in a timely manner. Further, VHA has recommended training for patient advocates, but it has not developed an approach to routinely assess their training needs or monitored training completion. VHA officials stated that they relied on VAMC and Veterans Integrated Service Network (VISN) staff to conduct these activities. However, GAO found that for the eight VAMCs in its review, the training needs of patient advocates were not routinely assessed, and training completion was not always monitored. Without conducting these activities, VHA increases its risk that staff may not be adequately trained to advocate on behalf of veterans. Finally, VHA has not monitored patient advocacy program data-entry practices or reviewed the data to assess program performance. VHA officials stated that they relied on VISN and VAMC officials to ensure that all complaints were consistently entered into VHA's Patient Advocate Tracking System (PATS). However, GAO identified inconsistencies in the extent to which VAMC officials did so. VHA's lack of monitoring may pose a risk that not all complaints are entered into this tracking system—a goal of the program. Additionally, VHA officials stated they did not systemically review data in the system to assess program performance and identify potential system-wide improvements because VHA considered this the responsibility of VAMCs. As a result, VHA officials may miss opportunities to improve veterans' experiences. VHA is beginning to address many of these governance, staffing, training, and data issues, including directing a workgroup to provide recommendations by spring of 2018. However, because the recommendations will be advisory, and because program deadlines have slipped in the past, the nature and timing of the actions needed to resolve these issues remain unclear.