VA Health Care: Improvements Needed in Processes Used to Address Providers' Actions That Contribute to Adverse Events
Published: 2013-12-03T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Adverse events--clinical incidents that may pose the risk of injury to a patient as the result of a medical intervention, rather than the patient's underlying health condition--can occur in all health care delivery settings. VAMCs can use one or more of the protected (confidential and nonpunitive) and nonprotected processes to evaluate the role of individual providers in adverse events. GAO was asked to review the extent to which processes used to respond to adverse events are carried out across VAMCs. In this report, GAO examined (1) VAMCs' adherence to VHA's protected peer review process, and the extent to which VHA monitors this process, and (2) VAMCs' adherence to VHA's nonprotected processes and the extent to which VHA monitors these processes. To conduct this work, GAO visited four VAMCs selected for variation in size, complexity of surgeries typically performed, and location. GAO reviewed VHA policies and federal internal control standards and analyzed data from the four selected VAMCs. GAO also interviewed VHA and VA OIG officials, as well as officials from VISNs of the four selected VAMCs.
section_title: What GAO Found
paragraphs: The Department of Veterans Affairs (VA) medical centers GAO visited did not adhere to certain policy elements of the protected peer review process, and monitoring by VA's Veterans Health Administration (VHA) is limited. According to policy issued by VHA, protected peer review may be used by VA medical centers (VAMC) when there is a need to determine whether a provider's actions associated with an adverse event were clinically appropriate--that is, whether another provider with similar expertise would have taken similar action. Despite VAMC officials' general understanding of the protected peer review process, none of the VAMCs GAO visited adhered to all four protected peer review policy elements selected for review, including the timely completion of reviews, and the timely development of peer review triggers that signal the need for further review of a provider's care. Failure of VAMCs to adhere to the protected peer review policy elements may result in missed opportunities to identify providers who pose a risk to patient safety. Veterans Integrated Service Networks (VISN), responsible for oversight of VAMCs, monitor VAMCs' protected peer review processes through quarterly data submissions and annual site visits. A VHA official said that VHA monitors the process by reviewing and analyzing the aggregated quarterly data submitted by VAMCs through the VISNs. The VA Office of the Inspector General (OIG) also conducts oversight of the protected peer review process as part of a larger review of VAMCs' operations. While the VISNs and VA OIG have reviewed VAMCs establishment of peer review triggers to prompt further review of a provider's care, neither they nor VHA has monitored their implementation. As such, VHA cannot provide reasonable assurance that VAMCs are using the peer review triggers as intended, as a risk assessment tool. This weakens VAMCs' ability to ensure they are identifying providers that are unable to deliver safe, quality patient care.
VAMCs' adherence to the nonprotected focused professional practice evaluation (FPPE) process is unclear due to gaps in VHA's policy on documentation requirements, and VHA does not routinely monitor nonprotected processes. An FPPE for cause is a time-limited evaluation during which the VAMC assesses the provider's professional competence when a question arises regarding the provider's ability to provide safe, quality patient care. Information collected through the FPPE can be used to inform adverse actions, such as limiting the provider's scope of care. Although VAMC officials were generally aware of the FPPE process, there are gaps in VHA's policy regarding how these evaluations should be documented and what information should be included, which limited GAO's ability to assess VAMCs' adherence to the process. For example, one VAMC provided GAO with documentation labeled as an FPPE and identified by the service chief as an FPPE; however, the quality manager said a formal FPPE was not conducted and that the documentation was actually part of a protected peer review. These differing views illustrate that, even within the same facility, gaps in VHA's policy on documenting FPPEs create a lack of clarity and opportunities for misinterpretation and inappropriate use. Moreover, the gaps in VHA's policy may hinder VAMCs' ability to appropriately document the evaluation of a provider's skills, support any actions initiated, and track provider-specific incidents over time. There is no routine monitoring of FPPEs for cause by VHA, VISNs, or VA OIG.
section_title: What GAO Recommends
paragraphs: GAO recommends that VA take action to ensure VAMCs adhere to certain elements of the peer review policy, require VAMCs to report data on implementation of peer review triggers, and develop more specific policy to help guide the FPPE process, including documentation requirements. In its written comments, VA generally concurred with GAO's conclusions and recommendations.

section_title: Letter
paragraphs: The Department of Veterans Affairs’ (VA) Veterans Health Administration  (VHA) operates one of the largest health care delivery systems in the  United States. VHA’s health care system includes 152 VA medical  centers (VAMC) that offer a variety of inpatient and outpatient services,  ranging from routine examinations to complex surgical procedures.  According to VHA officials, these services are provided by a variety of  health care professionals, including over 39,000 physicians and dentists.  As in all health care delivery settings like VAMCs, adverse events— clinical incidents that may pose the risk of injury to a patient as the result  of a medical intervention or the lack of an appropriate intervention, such  as a missed or delayed diagnosis, rather than that patient’s underlying  medical condition—may occur, resulting in potential or actual harm to   In an analysis of VHA data published in 2011, for example,  patients.researchers found that there were 101 reported surgical adverse  events—events pertaining to incorrect surgical procedures, such as a  procedure performed on the wrong patient or with the wrong implant— across VAMCs from mid-2006 to 2009.
According to VHA officials, VHA requires adverse events to be reported to appropriate  VAMC officials through each VAMC’s incident reporting system, or through other methods  such as a phone call or email to the applicable medical center’s patient safety manager or  risk manager. In addition to identifying and reporting adverse events, VAMCs have a  quality management program that enables them to identify and respond to other quality-of- care issues that may not rise to the level of an adverse event. VHA uses the terms quality  management, quality improvement, and quality assurance interchangeably. provider’s professional misconduct or from potential system deficiencies  related to VHA policies or procedures.
Hereafter, we refer to peer review for quality management as protected peer review. proctoring a provider, major adverse actions, disciplinary actions, or  privileging actions.
Adverse events raise questions about the quality of care VAMCs provide  to veterans. You asked us to review any gaps in the processes that  VAMCs use for responding to adverse events and the extent to which the  processes are carried out across VHA’s health care system. In this report,  we examined  1.  VAMCs’ adherence to VHA’s protected peer review process and the  extent to which VHA monitors this process, and  2.  VAMCs’ adherence to VHA’s nonprotected processes and the extent  to which VHA monitors these processes.
To examine VAMCs’ adherence to protected and nonprotected  processes, we reviewed VHA’s policies on protected peer review and  FPPEs, and VA’s policy on AIBs. Additionally, we reviewed these  processes in the context of federal internal control standards for risk  assessment, information and communications, control activities, and  monitoring. We also visited four VAMCs, which were selected on the  basis of geographic variation using the location in one of VA’s 21  Veterans Integrated Service Networks (VISN) as a proxy, VAMC size  based on the number of unique patients served, and level of surgical  complexity (see table 1).
We also requested data from each VAMC on protected peer reviews,  FPPEs, and AIBs from fiscal years 2009 through 2011 in order to review  the four VAMCs’ adherence to the applicable policies and within the  context of relevant federal internal control standards. Specifically, we  reviewed VAMCs’ adherence to several elements of VHA’s protected  peer review policy for which VAMCs are required to collect data; these  selected elements include timeliness of peer review completion and  committee review of select peer review results. VHA requires VAMCs to  establish peer review or professional activity triggers that signal the need  to conduct a detailed assessment of a provider’s care, which can then  lead to an FPPE. We reviewed only whether VAMCs established peer  review triggers, since VAMCs collect peer review data. To assess the  reliability of the protected peer review data we analyzed, we reviewed  VHA’s protected peer review policy on data reporting requirements,  interviewed VHA and VAMC officials familiar with the data, and conducted  checks on the data, looking for missing values, outliers, or other  anomalies. We found some records that contained missing, incomplete,  or inaccurate data and excluded those data from our analysis. We  determined that the deletions had no material effect on our analysis and  that the data, as analyzed, were sufficiently reliable for our purposes.
GAO, VA Administrative Investigations: Improvements Needed in Collecting and Sharing  Information, GAO-12-483 (Washington, D.C.: Apr. 30, 2012). appeals has passed; we did not assess how long VAMCs should have  kept each of the investigation files.
To examine the extent to which VHA monitors VAMCs’ use of protected  and nonprotected processes to respond to adverse events, we  interviewed VHA officials from the Office of the Assistant Deputy Under  Secretary for Health (ADUSH) for Quality, Safety and Value, including the  Director of Risk Management and the Director of Credentialing and  Privileging; the Office of the Deputy Under Secretary for Health for  Operations and Management (DUSHOM); and the Office of the Medical  Inspector. We also interviewed officials from the VA Office of Inspector  General (OIG) Office of Healthcare Inspections, as well as the chief  medical officers of the four VISNs for the VAMCs we visited.
We limited our review to describing VHA’s protected and nonprotected  processes for responding to adverse events that involve individual  providers, excluding from our scope processes, such as root cause  analysis,processes issues. We focused our review on VHA physicians and  dentists, hereafter referred to as providers, because these are the primary  types of licensed independent providers at VAMCs. The findings of our  four site visits cannot be generalized to other VAMCs.   that focus primarily on events that involve systems or  We conducted this performance audit from July 2012 to December 2013  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: VAMCs conduct an initial review of cases that are identified as possible  adverse events to determine how best to respond and which process to  use to determine the facts of the case, such as protected peer review,  FPPE, or AIB. Because VAMCs generally have discretion in which of  these processes they choose to use to respond to an adverse event,  different VAMCs may choose different processes in response to  experiencing similar adverse events. Based on the nature of the adverse  event and the information gleaned through a particular review process, a  VAMC may decide to conduct multiple types of reviews, both protected  and nonprotected processes, as appropriate. Information collected  through protected review processes, including protected peer review,  cannot be used to inform adverse actions against a provider; information  collected through nonprotected processes, including FPPEs and AIBs,  can be used to support a VAMC’s decision to take adverse action against  a provider. According to VHA policy, VAMCs can use both protected and  nonprotected processes concurrently or consecutively as long as  protected and nonprotected processes and data collection are kept  separate. According to VHA officials, if a VAMC is using a protected  process to review an event and realizes that a nonprotected review may  be necessary, the protected process should be stopped and the VAMC  should start a nonprotected review. See figure 1 for an illustration of the  decision process a VAMC official might use when deciding how to  respond to an adverse event.
According to VHA’s protected peer review policy, peer review is required  under certain circumstances, such as a death that appears related to a  hospital-incurred incident or a complication from treatment and a suicide  within 30 days of a clinical encounter with a VA health care professional.Peer review may be considered in other circumstances, such as when  there is an unexpected or negative outcome. Once VAMC officials decide  to conduct a protected peer review, a peer reviewer is assigned to  evaluate the care delivered and the actions taken by the provider.peer reviewer makes an initial determination of whether the provider  should have taken different action when providing patient care and  preliminarily assigns one of the following three levels of care:
Level of care 1 – the most experienced, competent providers would  have managed the case in a similar manner;
Level of care 2 – the most experienced, competent providers might  have managed the case differently; or
Level of care 3 – the most experienced, competent providers would  have managed the case differently.
According to VHA’s peer review policy, the initial peer review should be  completed within 45 calendar days from determination of the need for  peer review. If the peer-reviewed case is assigned a level of care 2 or 3,  it must be referred to the VAMC’s peer review committee for further  review. After conducting a further review of the facts of the case, and  receiving further input from the provider under review, the peer review  committee either validates the initial level of care or assigns a higher or  lower level of care. The peer review committee’s level of care rating is  final and must be completed within 120 calendar days from determination  of the need for peer review. The peer review committee can also make  recommendations for nonpunitive, nondisciplinary actions, as appropriate,  such as reviewing and revising local policy, to improve the quality of care  delivered. The final level of care rating and any recommendations for  improvement are reported to the provider’s supervisor, who gives the  provider feedback that is based on the peer review committee’s findings.  According to VHA officials, VAMCs conduct approximately 23,000  protected peer reviews systemwide annually.
VHA also has a contract with an external organization that is used to audit  protected peer review. VAMCs may request external protected peer  review expertise if there are no qualified peers available at the VAMC.  According to a VISN official, external review may also be requested if the  VAMC needs to ensure an independent peer review, for example, if all  the providers in the same clinical specialty were involved in the event.  VHA also requires each VAMC to submit quarterly a sample of cases that  were recently peer-reviewed, for a secondary peer review by this external  organization.
According to VHA’s protected peer review policy, each VAMC is required  to develop peer review or professional activity triggers to signal the need  for further assessment of a provider’s clinical care. The triggers are  specific to an individual VAMC. If, after a detailed assessment, concerns  arise about a provider’s ability to deliver safe, quality patient care, then an  FPPE would be conducted. For example, if a provider meets a VAMC’s  peer review triggers by receiving three peer review level of care ratings of  3—meaning that the most experienced, competent providers would have  managed the cases differently—within a 12-month period, then VAMC  officials would be prompted to conduct a detailed assessment of the  provider’s care, and address concerns about the provider’s ability to  deliver safe, quality patient care by conducting an FPPE.
subsections: section_title: Nonprotected Processes
paragraphs: According to VHA policy, an FPPE may be used when a question arises  regarding a provider’s ability to provide safe, quality patient care, such as  whether documentation of patient encounters is late or insufficient, or  whether diagnoses are accurate. An FPPE may also be used when a  provider meets or exceeds a VAMC’s peer review triggers and a detailed  assessment indicates the need for further review. According to VHA and  VAMC officials, an FPPE begins by providing an opportunity for a  provider to improve his or her performance in area(s) of identified  concern. The FPPE is a time-limited period during which medical staff  leadership assesses the provider’s professional performance and ability  to improve. According to VAMC officials, if the provider’s performance  improves, VAMC officials may decide that the FPPE process is completed  and the provider may be returned to routine monitoring. If a provider’s  performance does not improve, then medical staff leaders may decide to  continue the FPPE for an additional period of time or determine if a  privileging action should be taken, such as reducing or revoking a  provider’s privileges. The FPPE may include a review of the provider’s  care, either through medical record review, direct observation, or  discussions with other individuals involved in the care of patients. FPPEs  conducted when a question arises regarding a provider’s ability to provide  safe, quality patient care are not conducted often, according to VHA and  VAMC officials. According to a VHA official, about 100 FPPEs of this kind  are conducted VHA-wide each year.
VA typically uses AIBs to examine nonclinical issues for which the facts  are in dispute, such as allegations of employee misconduct, according to  VHA and VAMC officials. These officials also said VAMCs may use AIBs  to investigate issues of an individual provider’s clinical competence, but  this is not typical. According to VA’s AIB policy, the convening  authority—typically the VAMC director—appoints members to an AIB and  defines the scope and authority of the investigation. The AIB collects  and analyzes evidence and develops a report, including findings and  conclusions. The VAMC director may use an AIB’s findings to inform  decisions of whether to take adverse action against a provider and, if so,  what type of action to pursue. In our 2012 report on AIBs, we found that  VAMCs and VISNs conducted a total of 1,136 investigations nationwide  from fiscal year 2009 through 2011, but because VA does not track the  types of matters investigated, it is unclear how many of these were  related to clinical competence.
section_title: VA and VHA Monitoring Responsibilities
paragraphs: There are several VA and VHA organizational components that are  involved in monitoring VAMCs’ adverse events and related processes.  VHA’s Office of the ADUSH for Quality, Safety and Value is responsible  for establishing VHA policies for protected peer review and FPPEs and  for providing guidance to VAMCs on using those processes. VHA’s Office  of the DUSHOM oversees the VISNs and provides the VISNs broad and  general operational direction and guidance; VISN directors are tasked  with oversight of the protected peer review process. The Office of the  Medical Inspector addresses health care problems to monitor and  improve the quality of care provided by VHA; veterans may report  problems with medical care received at VAMCs directly to the Office of  the Medical Inspector. The VA OIG Office of Healthcare Inspections  inspects individual health care issues and performs quality program  assistance reviews of VAMC operations. See figure 2 for a simplified  organizational chart of the relationship among these entities.
section_title: VAMCs Did Not Adhere to Certain Policy Elements of the Protected Peer Review Process, and Monitoring by VHA Is Limited
subsections: section_title: VAMCs Did Not Adhere to Certain Policy Elements of the Protected Peer Review Process
paragraphs: According to VHA’s protected peer review policy, and supported by  federal internal control standards for risk assessment and information and  communications, VHA’s protected peer review policy requirements should  ensure that identified patient safety risks are mitigated and lead to  organizational improvements and optimal patient outcomes. Additionally,  federal internal control standards state that agencies should have reliable  information relating to internal events to effectively run and control their  operations; this information should be identified, captured, and  communicated in sufficient detail and at the appropriate time to the right  people.
VAMC officials from all four sites we visited demonstrated a general  understanding of the process as described in the protected peer review  policy. For example, officials from each of the four VAMCs knew that peer  review was a protected nonpunitive process and officials from three of the  four VAMCs were able to describe the steps of the process. However, our  analysis of peer review data for fiscal years 2009 through 2011 provided  by the four VAMCs showed that none of the VAMCs adhered to all four  VHA protected peer review policy elements selected for review:   (1) completing the initial peer review within 45 calendar days,   (2) completing the final peer review within 120 calendar days, (3) sending  all initial level of care 2 and 3 peer reviews to the peer review committee,  and (4) developing peer review triggers. Additionally, these peer review  data included varying amounts of missing and inaccurate data, which  affected our ability to fully analyze these data. See table 2 for a summary  of the four VAMCs’ adherence to selected protected peer review policy  elements.
Completing initial peer reviews within 45 calendar days. In our  analysis of the timeliness policy elements for fiscal years 2009 through  2011, we found that VAMCs A and D completed 77 to 79 percent of initial  peer reviews within 45 calendar days. VAMC B completed 100 percent of  initial peer reviews within 45 calendar days for the 2 fiscal years for which  data were available—2010 and 2011. We could not determine the  completion percentage for VAMC C because the data provided by the  facility did not contain all of the information needed. According to a VAMC  C official, the peer review data provided to us was compiled just before  our site visit and was based on a review of past records; the missing data  elements could not be located during that review. Officials from VAMCs A  and D told us that it was difficult to get peer reviewers to abide by the   45-day rule, and, according to officials from VAMC D, the heavy workload  of those in charge of peer review tracking during our study period affected  their ability to ensure peer reviewers’ timely completion. According to the  official in charge of peer review tracking at VAMC D, a routine process is  now in place at this facility to remind peer reviewers to complete their  reviews after 30 of the 45 calendar days allotted have elapsed.
Completing the final peer reviews within 120 calendar days. VAMCs  A and D each completed 89 percent of final peer reviews within 120  calendar days for fiscal years 2009 through 2011; VAMC B completed   97 percent for fiscal years 2010 through 2011. As previously noted, we  were unable to determine the completion percentage for VAMC C. An  official at VAMC A told us that, in addition to the delays in completing the  initial peer reviews within 45 calendar days, the peer review committee  might not have always been able to review all peer-reviewed cases at its  monthly meeting if there had been a particularly large number of cases  sent to the committee that month.
Sending level of care 2 and 3 peer reviews to the peer review  committee. We found that VAMCs A and D sent 96 to 100 percent of  their initial level of care 2 and 3 peer reviews to the peer review  committee for fiscal years 2009 through 2011; VAMC B sent 99 percent   VAMC C sent 79 percent of its initial for fiscal years 2010 through 2011. level of care 2 and 3 peer reviews to the peer review committee. Officials  from VAMC C told us that a possible reason for the low adherence rate  could be that the peer review committee and one of the VAMC’s service  line committees tasked with sending level of care 2 and 3 cases to the  peer review committee for further review did not communicate well during  this time period. According to these officials, communication and  coordination between these two committees has recently improved.
Developing peer review triggers. We found that some VAMCs did not  develop peer review triggers in a timely way. VHA’s protected peer review  policy issued in January 2008 required that VAMCs develop criteria that  may define the need for further review or action. VHA’s 2010 update of  this policy specified that VAMCs were required to establish peer review or  professional activity triggers. The 2010 policy update provides one  example of what these triggers could include—three peer review level of  care 3 ratings for a provider within 12 consecutive months. Further,  federal internal control standards for risk assessment state that  management should identify internal risks and undertake a thorough and  complete analysis of the possible effects. Additionally, the standards state  that risk assessment should include establishment of criteria for  determining low, medium, and high risk levels.
We found that while all four VAMCs we visited developed peer review  triggers, two of the four VAMCs developed peer review triggers  approximately two years after the updated policy was issued. (See   table 3.)
According to some VAMC officials we spoke with, the delay in developing  peer review triggers was the result, at least in part, of a lack of guidance  from VHA. When asked about peer review triggers, officials from VAMCs  B and C told us they did not receive any assistance from VHA in  developing triggers and that the guidance provided by VHA was vague.  Officials from VAMCs A and C, the two VAMCs that did not initially  develop peer review triggers, told us that while they had informal triggers  that could prompt the need for a nonprotected review, such as an FPPE,  they had not formally documented the triggers.
When VAMCs fail to complete peer reviews in a timely manner and send  all level of care 2 and 3 initial peer reviews to the peer review committee,  they put patients’ safety at risk through potential exposure to substandard  care. VAMCs may fail to identify problematic providers in a timely manner  and take the appropriate actions. Additionally, by not submitting initial  level of care 2 and 3 peer reviews to the peer review committee for further  evaluation, VAMCs are not ensuring that the initial ratings assigned were  appropriate. Moreover, the delayed establishment of the peer review  triggers by some VAMCs may have resulted in missed opportunities to  identify providers who posed a risk to patient safety and to conduct an  FPPE, which would have allowed any warranted action to be taken  against the provider.
section_title: VHA and VA OIG Monitor VAMCs’ Protected Peer Review Process
paragraphs: Within VHA, the VISNs and the Office of Risk Management monitor  VAMCs’ protected peer review processes. Officials from the VISNs that  oversee the four VAMCs we visited told us they monitor VAMCs’  protected peer review processes through quarterly data monitoring and  annual site visits, as required by VHA’s protected peer review policy.  Officials from these VISNs said they monitor peer review data that  VAMCs are required to submit quarterly.submitted by VAMCs and reviewed by the VISN include, but are not  limited to, the number of peer reviews completed, the assigned level of  care ratings by the initial peer reviewer and by the peer review committee,   Data elements that are to be  the number of assigned level of care ratings changed to a higher or lower  level by the peer review committee, and the timeliness of the reviews.  After completing their review of VAMCs’ quarterly data, VISN officials are  to send aggregated peer review data to VHA’s office of the ADUSH for  Quality, Safety and Value through a shared electronic database.
In addition to quarterly review of the data, officials from all four VISNs  said they conduct annual site visits to the VAMCs within the VISN; these  site visits include a multifaceted review of VAMCs’ quality management  operations. However, we did not review documentation from the VISNs’  site visits; one VISN official told us that the VISN does not keep formal  records and that it is required only to attest to their completion. The scope  of VISNs’ annual site visits covers a broad variety of topics, including  VAMCs’ protected peer review processes, according to VISN officials. All  VISN officials we spoke with told us they typically chose which elements  of the peer review process to review based on the focus of recent  inspections by other entities, such as The Joint Commission and the VA  OIG. For example, officials we spoke with from one of the four VISNs  said they reviewed in a 2012 site visit whether the VAMCs implemented  the protected peer review policy elements for timeliness—specifically,  completing final peer reviews within 120 calendar days; VA OIG reported  in January 2011 that a VAMC within the VISN was not compliant with this  timeliness requirement.
VHA’s protected peer review policy requires VHA to conduct analysis of peer review  data findings submitted by each VISN and to disseminate those findings to the Under  Secretary for Health, VISNs, and other leadership. See VHA Directive 2010-025. rating level of care changed by the peer review committee. The Director  of Risk Management said that if an outlier is identified in the aggregated  data, it is brought to the attention of the relevant VISN officials. An official  from the Office of Risk Management reported that the office produces  quarterly reports from analysis of the VHA systemwide aggregated  protected peer review data and that these quarterly reports are shared  with each VISN. According to VHA’s fourth quarter report for fiscal year  2012, ratings for 22 percent of peer-reviewed cases were changed by the  VAMCs’ peer review committees. VHA’s analysis found that the peer  review committees were more likely to improve the peer review rating by  decreasing the assigned rating level, such as decreasing a level of care 3  to a level of care 1 or 2, or a level of care 2 to a level of care 1.
In addition to monitoring VISNs’ aggregated protected peer review data,  VHA’s Director of Risk Management said the office communicates  regularly with the Office of the DUSHOM, as required by VHA’s protected  peer review policy and supported by federal internal control standards for  information and communications. According to VHA’s protected peer  review policy, the DUSHOM’s responsibilities include establishing and  maintaining the peer review program in coordination with the ADUSH for  Quality, Safety and Value, and providing direction and guidance on data  elements that VAMCs must report through VISNs to VHA. Federal  internal control standards state that mechanisms should exist to allow the  easy flow of information down, across, and up through the organization,  and easy communications should exist between functional activities.  These standards also state that responsibility for decision-making should  be clearly linked to the assignment of authority, and individuals should be  held accountable accordingly.that the office periodically gives brief overviews of the office’s work,  including protected peer review findings, to the Under Secretary for  Health and to the DUSHOM.
The Director of Risk Management said  Beyond VHA monitoring of VAMCs’ protected peer review process, the  VA OIG also routinely reviews certain policy elements of the process  through its Combined Assessment Program, which reviews each VAMC  every 2 to 3 years. In the last 5 years, the VA OIG has reviewed  VAMCs’ compliance with several requirements, including (1) peer review  committee must submit quarterly reports to the medical executive  committee, (2) peer review committee must analyze protected peer  reviews for trends in follow-up items and recommendations, and   (3) protected peer reviews by the initial reviewer and the peer review  committee must be completed timely. The VA OIG has conducted at least  one Combined Assessment Program review since 2011 at each of the   See table 4 for a summary of protected peer  four VAMCs we visited.review monitoring activities conducted by VISNs, VHA’s Office of Risk  Management, and VA OIG.
According to officials from the Office of Risk Management, routine  monitoring of certain policy elements of VAMCs’ protected peer review  processes has enabled VHA to make changes in policy and improve  protected peer review monitoring activities. VHA’s Director of Risk  Management said that her office is responsible for making changes to  policy in response to VA OIG findings related to the protected peer review  process. For example, in a 2008 report on VISNs’ oversight of the  protected peer review process, the VA OIG found that VISNs failed to  substantially comply with the requirement to conduct periodic inspections;  the VA OIG recommended that VHA clarify its peer review policy to define  periodic site visits required of VISNs. VHA’s 2008 revision of the  protected peer review policy redefined site visits as annual to result in  more frequent monitoring by VISNs. VHA also improved peer review  data monitoring activities. According to the Director of Risk Management,  a national survey of VAMC risk managers was conducted in 2011 to  better understand how protected peer review data are used. Based on  survey data and a literature review, VHA officials said they determined  that VAMCs and VISNs needed additional data on the specific areas of  care that were being peer-reviewed as well as information about  significant problems identified and frequently cited issues. VHA officials  said that in fiscal year 2012, they expanded the required quarterly  protected peer review data set that VAMCs report to the VISN to include  the commonly peer-reviewed aspects of care, such as the choice of a  diagnostic test, performance of a procedure or treatment, and addressing  abnormal results of diagnostic tests.
section_title: VHA and VA OIG Monitoring of Peer Review Triggers Is Limited
paragraphs: According to VHA’s protected peer review policy, VISNs and the Office of  Risk Management play a role in monitoring VAMCs’ implementation of  protected peer review processes, which include peer review triggers. In  addition, federal internal control standards state that management needs  to comprehensively identify risks and analyze the possible effects; these  peer review triggers serve as part of VHA’s risk assessment tool to help  identify issues of risk to patient safety and improve the organization.
Officials of the VISNs for the four VAMCs we visited and the VA OIG told  us they have reviewed the establishment of peer review triggers by  VAMCs. However, VISNs, VHA, and VA OIG have not monitored whether  the triggers have actually been implemented. Officials we interviewed  from two of the four VISNs said their VISNs reviewed VAMCs’ peer  review triggers during the required annual site visits; an official from a  third VISN said that the VISN has confirmed that most VAMCs have  established peer review triggers. Officials from the fourth VISN told us  they have reviewed types of triggers in place at VAMCs, but they have  not confirmed that VAMCs have established peer review triggers. Officials  from all four VISNs told us they typically do not monitor whether VAMCs  have implemented the established peer review triggers, including  monitoring how many FPPEs for cause have been triggered. One VISN  official explained that the VISN cannot monitor every aspect of every  policy and regulation that govern VAMCs’ operations; therefore, they  choose to focus their monitoring efforts on the elements of particular  importance where other entities—such as the VA OIG and The Joint  Commission—have found evidence of noncompliance.
VHA’s Director of Risk Management told us that the office does not  monitor whether VAMCs have established peer review triggers. Further,  the official told us the office has not monitored how VAMCs have  implemented peer review triggers or tracked how many providers may be  exceeding the triggers and are subject to FPPEs for cause. The official  also noted that VHA has not asked VAMCs to document their peer review  triggers and has not asked the VA OIG to look specifically, during  Combined Assessment Program reviews, at whether VAMCs have  established triggers.
Officials from the VA OIG told us they did monitor VAMCs’ establishment  of peer review triggers as part of Combined Assessment Program reviews  conducted in fiscal year 2009. The OIG review found 97 percent  compliance with establishing peer review triggers across the 44 VAMCs  OIG officials visited; the four VAMCs we visited were not among the 44  VAMCs included in this review. According to VA OIG officials, they  decided not to review the establishment of peer review triggers in  subsequent Combined Assessment Program reviews because of the high  compliance rate in 2009; instead, subsequent Combined Assessment  Program reviews focused on requirements with which VAMCs had not  been in compliance, such as requiring the peer review committee to  submit quarterly reports on protected peer review to the medical  executive committee. VA OIG officials told us they have not reviewed  whether VAMCs have implemented the triggers.
Because neither VHA’s Office of Risk Management nor VA OIG review  whether peer review triggers have been implemented, VHA cannot  provide reasonable assurance that VAMCs are using the triggers as a risk  assessment tool as intended. Failure to do so weakens VAMCs’ ability to  ensure patient safety, and officials cannot be assured that the use of  these triggers meets the intended goal of identifying providers that are not  delivering safe, quality patient care.
section_title: VAMC’s Adherence to the FPPE Process Is Unclear Due to Gaps in Policy Addressing Documentation Requirements; VHA Does Not Routinely Monitor Nonprotected Processes
subsections: section_title: Gaps in Policy Create Lack of Clarity as to How VAMCs Are to Document the FPPE Process; Requirements for AIBs Generally Are Clear
paragraphs: FPPEs. According to federal internal control standards for control  activities, written documentation should exist for all significant events that  occur within an agency; this documentation should be readily available for  examination, and it should be complete and accurate in order to facilitate  tracing the event from initiation through processing to completion. In  documenting FPPEs, building strong and complete evidence on each  case is important to support the outcome of the evaluation, as well as to  track the identified area of concern over time.
VHA’s FPPE policy provides a general definition of an FPPE, that it can  be used for cause (when a question arises regarding a provider’s ability to  provide safe, quality patient care), that the criteria for the FPPE should be  defined by the VAMC in advance, and that the results of the FPPE must  be documented in the provider’s profile. However, there are gaps in  VHA’s policy regarding how these evaluations should be documented and  what information should be included, which limited our ability to assess  VAMCs’ adherence to the FPPE policy. Officials from two of the VAMCs  we visited told us there are no standardized guidelines on how the FPPE  process should be structured. According to the Director of Credentialing  and Privileging, VHA’s policy on FPPEs was intended to allow VAMCs  flexibility in the design of the evaluation to accommodate the variety of  ways that issues are identified and the types of issues that may be  addressed (see box for example of an FPPE).
Example of an FPPE at one VAMC we  visited   There are multiple ways that an FPPE may be  prompted, including multiple patient  complaints or exceeding a VAMC’s peer  review trigger. At one VAMC we visited, a  provider exceeded a peer review trigger— receiving two or more level of care 3 peer  review ratings within 6 months—which  prompted a detailed assessment, including a  retrospective evaluation of 25 percent of the  provider’s medical records over the previous  12 months. The medical records were  evaluated specifically for patient evaluations,  outcomes, and documentation. Upon  completion of the assessment, VAMC officials  determined that the provider’s documentation,  including patient discharge summaries,  patient transfer, medication review, and  disclosure notes, was inadequate. An FPPE  was initiated, for which the provider was  instructed on how to properly document these  types of summaries and notes. The evaluator  also recommended that the provider write the  notes for each patient seen before moving to  see other patients, instead of writing the notes  for all patients at the end of the day. VAMC  officials also initiated a performance  improvement plan for the provider, which  included monitoring his medical record  documentation over a 6-month period. When  officials determined that the provider had not  improved after this period, the FPPE was  extended to continue monitoring aspects of  his clinical care. The provider retired 6 months  later.
Officials from the VAMCs we visited were generally aware that FPPEs  can be used to address concerns about the quality of a provider’s care;  are time limited; and are not disciplinary, but could ultimately be used to  take adverse action against a provider, if necessary. While VAMC officials  were generally aware of the FPPE process, we found that the four  VAMCs we visited varied widely in their documentation of FPPEs,  attributable at least in part to the lack of specificity in VHA’s FPPE policy  regarding documentation requirements. In reviewing FPPEs conducted  between fiscal years 2009 and 2011 we found the following:
One of the four VAMCs provided a completed template for each of its  FPPEs, including the purpose of the FPPE, specifying what triggered  the review, the time period for review, comments by the evaluator, an  action plan based on the review, and evidence of concurrence with  the review by the applicable service chief. (See app. I for an example  of an FPPE template used at one of the VAMCs we visited.)
Two other VAMCs provided various combinations of documents as  evidence of their FPPEs, including professional standards board  minutes and emails and letters from evaluators. These documents  contained varying amounts of information detailing the circumstances  prompting the FPPE, comments from evaluators, and follow-up  actions, if any.
The fourth VAMC initially provided us with documentation of one  FPPE, including a completed template identifying the clinical service  involved, the method of evaluation, the evaluator’s findings, and the  service chief’s conclusions; and several documents, each with  focused professional practice evaluation labeled at the top, specifying  the medical records evaluated for the FPPE and the evaluator’s  comments on each case. The VAMC’s service chief told us that an  FPPE had been conducted for cause for this provider, but the VAMC’s  quality manager said a formal FPPE had not been conducted and that  the documentation we received was part of a protected peer review  process. This disagreement illustrates that even within the same  facility the interpretation of VHA’s policy on FPPEs differs, which can  lead to potentially inappropriate use.
Gaps in VHA’s policy on FPPE documentation requirements create a lack  of clarity and therefore may affect VAMCs’ ability to appropriately  document the evaluation of providers’ skills, support any actions initiated,  and track provider-specific FPPE-related incidents over time. For  example, if FPPEs are not well documented, VAMC officials may have  limited knowledge of the findings to proceed with any actions and limited  ability to track that such evaluations were conducted. As a result, if  another adverse event subsequently occurred involving the same  provider, the VAMC may not be aware of any prior findings. One VAMC  official stressed to us the importance of thorough documentation of an  FPPE, even if the determination is made that the provider delivered safe,  quality patient care and no adverse action is needed. Without adequate  documentation, a VAMC may conduct an FPPE that complies with VHA’s  policy, and determine that adverse action is needed on the basis of the  evaluation’s findings, but ultimately may be unable to take the action  because the documented evidence is insufficient. Officials at one VAMC  said they did not believe that the evidence gathered from an FPPE was  strong enough to hold up against a provider’s appeal of an adverse  action.
AIBs. Another type of nonprotected review that VAMC officials may  choose to address an adverse event is an AIB. In our review of VA’s  AIB policy, we found that the policy generally provides clear guidance on  the requirements, including documentation. For example, the policy  specifies steps on how VAMC officials determine the need for an AIB,  select the board members, and write the charge letter that convenes the  AIB, and also provides a number of templates and checklists, including  templates for the charge letter and investigative report, and checklists for  collecting evidence, conducting interviews, and writing the report.
When asked about AIBs, VAMC officials told us an AIB could be used  when (1) a case is egregious; (2) there is a previous incident involving the  same provider; (3) the VAMC needs or wants to take privileging or  disciplinary action; and (4) the facts of a case are in dispute or when there  is a high level of complexity and ambiguity in the case. For example, at  one VAMC we visited, an AIB was convened to investigate the facts and  circumstances regarding the care and treatment immediately preceding a  veteran’s death. The AIB was charged with investigating allegations that a  provider’s care deviated from standard practice procedures, including  prescribing inappropriate doses of medications, not ensuring appropriate  monitoring or appropriate interval of physical assessments, and providing  substandard documentation and communication.
section_title: VHA Does Not Routinely Monitor Nonprotected Processes
paragraphs: FPPEs. According to federal internal control standards for monitoring,  agencies should assess the quality of performance over time and provide  reasonable assurance that deficiencies are detected and promptly  resolved. Further, VHA’s credentialing and privileging policy states that  the DUSHOM is responsible for ensuring that VISN directors maintain an  appropriate credentialing and privileging process, which includes FPPEs,  consistent with VHA policy. The VISN Chief Medical Officer is  responsible for oversight of the credentialing and privileging process of  the VAMCs within the VISN.
Officials we interviewed from three of the four VISNs told us they do not  monitor FPPEs conducted in response to questions about a provider’s  ability to deliver safe, quality patient care. Officials from two of those three  VISNs said that they have examined a sample of FPPEs during annual  site visit reviews of VAMC operations; however, the sample of FPPEs  would include mostly FPPEs for newly hired providers, since there are  very few FPPEs conducted in response to questions that have arisen  about a provider’s ability to deliver safe, quality patient care. An official  from the fourth VISN said he monitors FPPEs during annual site visits to  VAMCs, including FPPEs conducted in response to questions that have  arisen about a provider’s ability to deliver safe, quality patient care.
Officials from the office of the DUSHOM and the Office of Credentialing  and Privileging said they do not monitor FPPEs conducted in response to  questions about a provider’s ability to deliver safe, quality patient care.  The Office of Credentialing and Privileging does not monitor these FPPEs  because there are so few of them that the cost of reviewing a process  that occurs so infrequently would outweigh the benefit. According to VHA  officials, if an FPPE leads to a proposed reduction or revocation of clinical  privileges, the Director of Credentialing and Privileging is frequently  consulted to ensure that appropriate due process is afforded the  provider.
Similar to the VISNs, VA OIG officials told us they do not monitor FPPEs  conducted when a question arises regarding a provider’s ability to deliver  safe, quality patient care. However, VA OIG officials said that, during site  visits for Combined Assessment Program reviews, they do review policy  elements of the process for FPPEs for providers newly appointed to the  VAMCs’ medical staff or for providers requesting new privileges.
Because none of these entities monitor FPPEs conducted when a  question arises regarding a provider’s ability to deliver safe, quality  patient care, VHA cannot be assured that the process is working as  intended or whether VAMCs need additional guidance or training about  the process.
According to a VA OIG official, these policy elements include whether there is evidence  that FPPEs for new providers or new privileges are initiated, completed, and reported to  the medical executive committee. requirements for VHA to monitor AIBs; however, federal internal control  standards for information and communications state that relevant,  reliable, and timely information is needed by an agency to achieve its  objectives and to control its operations. In our 2012 report on AIBs, we  found that VA does not collect and analyze aggregated data on AIB  investigations, and at that time, VHA officials told us that there were no  plans to do so. In our report, we recommended that VA establish a  process to collect and analyze aggregated data from AIB investigations  conducted within VHA. Having these types of data may provide VA with  valuable information to systematically gauge the extent to which matters  investigated by AIBs are occurring throughout VHA and to take corrective  action, if needed, to reduce the likelihood of future occurrences.  According to VHA officials, after our 2012 report was issued, VHA created  a workgroup to examine our recommendation and concluded that  monitoring AIBs was not necessary or warranted.
section_title: Conclusions
paragraphs: VAMCs’ adherence to policies on protected and nonprotected processes  for responding to providers’ actions that contribute to adverse events  helps ensure that quality care is provided to veterans and that safety risks  are minimized. Having clear and detailed guidance in policy for these  processes is critical to helping VAMC officials identify and address  adverse events, including providers’ contributing actions, in a timely and  appropriate manner. Although officials at the four VAMCs we reviewed  generally understood the protected peer review process, we found that  none of these four VAMCs adhered to all four VHA protected peer review  policy elements selected for review, such as completing peer reviews  within required time frames and sending the required peer-reviewed  cases to the peer review committee for further assessment. As such, VHA  may be missing opportunities for improvements both in the practice of  individual providers and organizationally. VHA also may be missing  opportunities to identify and intervene early with providers whose care  may pose a risk to patient safety if VAMC officials have not established or  implemented peer review triggers that would initiate a detailed  assessment of a provider’s care. Assisting with and monitoring VAMCs’  development and use of these peer review triggers will help VHA ensure  that the protected peer review process contributes to organizational  improvements and favorable patient outcomes, as intended by VHA  policy.
FPPEs and AIBs are nonprotected processes that VAMCs use to address  adverse events involving individual providers. However, gaps in VHA’s  FPPE policy on documentation requirements have created a lack of  clarity for VAMCs on how to appropriately document the process.  Inadequate documentation of the FPPE process may result in VAMC  officials being unable to take adverse action against a provider when  necessary. Providing more specific policy guidance for FPPEs would  better support VAMCs’ use of this process, including when officials  determine that they may need to take adverse action against a provider.  Although VHA officials reiterated that they do not have plans to collect  and analyze aggregated AIB data as we recommended in our 2012  report, we continue to believe that this is a potentially important quality  improvement tool for use by VHA.
section_title: Recommendations for Executive Action
paragraphs: To improve VHA’s use of the protected peer review and nonprotected  processes to respond to individual providers involved in adverse events  or when questions arise regarding providers’ ability to deliver safe, quality  patient care, we are making five recommendations.
To address protected peer review process requirements, we recommend  that the Secretary of Veterans Affairs direct the Under Secretary for  Health to ensure that VAMCs send all required initial peer reviews (level of care  2 and 3) to the peer review committee; ensure VAMCs’ peer review committees complete final peer reviews  within 120 calendar days; provide clear guidance and assistance on the purpose, development,  and implementation of peer review triggers; and  require VAMCs to periodically provide data on peer review triggers,  including the number of providers that have exceeded the triggers as  part of the protected peer review data VAMCs report to VISNs on a  quarterly basis.
To address the nonprotected FPPE process, we recommend that the  Secretary of Veterans Affairs direct the Under Secretary for Health to  develop more specific policy on the FPPE process, including  documentation requirements such as the FPPE’s purpose, time period  covered, evaluator’s assessment, and the summary of actions to be  taken.
section_title: Agency Comments and Our Evaluation
paragraphs: VA provided written comments on a draft of this report, which we have  reprinted in appendix II. In its comments, VA generally concurred with our  conclusions and our five recommendations, and described the agency’s  plans to implement each of our recommendations. VA also provided  technical comments, which we have incorporated as appropriate.
In response to our first and second recommendations that VA ensure that  all initial peer reviews (levels of care 2 and 3) be sent to the peer review  committee for review and that the committee’s reviews be completed  within 120 days, VA stated that it will provide refresher education to key  staff, such as chiefs of staff, risk managers, and VISN officials. VA  anticipates that its planned actions will be completed by December 31,  2013.
In response to our third recommendation that VA provide clear guidance  and assistance on the purpose, development, and implementation of peer  review triggers, VA stated that refresher education on this policy  requirement, which also encompasses professional activity triggers, was  communicated to key staff, such as chiefs of staff, risk managers, and  VISN officials, through a conference call. Additionally, VA stated that staff  in the VHA Risk Management Program, Office of Quality, Safety and  Value will be available to provide consultative assistance to facilities that  are unclear on how to implement this requirement. VA anticipates  completion of these activities by December 31, 2013.
In response to our fourth recommendation that VA require VAMCs to  periodically provide data on peer review triggers, VA concurred. VA  stated that VAMCs will be required to submit a deidentified, summary  report discussing trends and analysis of aggregate data on peer review  activity with their quarterly submission to the VISN. VA stated that this  new requirement will be included in the fiscal year 2014 revision of VHA  Directive 2010-025, Peer Review for Quality Management, which  establishes that the VAMC’s medical executive committee is responsible  for determining peer review or professional activity trigger levels. VA  anticipates completion of these activities by September 30, 2014. VA  disagreed with the latter part of our recommendation that the data  submitted should include the number of providers that have exceeded the  triggers. VA stated that reviewing aggregate data for the number of  providers who exceeded trigger thresholds would represent skewed data,  which VA officials believe is not reflective of the quality of care provided  by those providers submitted for triggered reviews. VA stated larger  facilities may appear to have artificially higher levels of providers referred  for detailed assessments than those of smaller facilities. We agree with  VA that VAMC clinical leadership, with VISN oversight, input, and  support, is the preferred means to handle trigger thresholds and data  analysis; however, VA did not specify what data on protected peer review  triggers that VAMCs would be required to report. We maintain that it is  important for VAMCs and the VISNs to review whether the peer review  triggers are implemented as intended. Part of this review should include  monitoring how many providers have exceeded the trigger thresholds.  Additionally, we believe that collecting and reporting such data will help  the VISNs and VHA ensure that the protected peer review process  contributes to organizational improvements and favorable patient  outcomes.
In response to our fifth recommendation that VA develop more specific  policy on the FPPE process, including documentation requirements such  as the FPPE’s purpose, time period covered, evaluator’s assessment,  and the summary of actions to be taken, VA stated that it will develop  guidance on the FPPE process that will begin with a description of the  process for a detailed assessment and define the FPPE for cause  process if an opportunity to improve is indicated. VA further noted that the  guidance will end with an overview of the adverse action process to be  initiated when the provider does not demonstrate adequate improvement  and a reduction or revocation of clinical privileges appears to be  indicated. VA anticipates completion of these activities by September 30,  2014. While we understand VA’s intention of expediting dissemination of  information about the FPPE process through guidance, we believe that it  is important for the guidance to be included in the next formal iteration of  VHA policy.
We are sending copies of this report to the appropriate congressional  committees, the Secretary of Veterans Affairs, the Under Secretary for  Health, and other interested parties. In addition, the report is available at  no charge on the GAO website at http://www.gao.gov.
If you or your staffs have any questions about this report, please contact  me at (202) 512-7114 or draperd@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made major contributions to this report  are listed in appendix III.
section_title: Appendix I: Example of a VA Medical Center’s (VAMC) Focused Professional Practice Evaluation (FPPE) Template
paragraphs: This FPPE form is a full recreation of a template used by one of the  VAMCs we visited.
section_title: Appendix II: Comments from the Department of Veterans Affairs
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Marcia Mann, Assistant Director;  Ashley Dixon; Mariel Lifshitz; Katie McConnell; Elizabeth Morrison;   Lisa Motley; Ann Tynan; and Michael Zose made key contributions to this  report.