VA Health Care: Actions Needed to Assess Decrease in Root Cause Analyses of Adverse Events
Released: 2015-08-28T00:00:00

Letter

Background

VHA’s National Patient Safety Improvement Handbook identifies key staff involved in the RCA process, establishes minimum requirements for conducting RCAs, and outlines the RCA process.

VISNs are regional systems of care that oversee the day-to-day functions of VAMCs that are within their network. Each VAMC is assigned to one of VA’s 21 VISNs.

Key VHA Staff Involved in the RCA Process

Within VHA, NCPS supports the RCA process VHA-wide as part of its broader efforts to reduce and prevent inadvertent harm to patients as a result of their care. NCPS staff categorize and analyze RCA data, and provide training and education for VAMCs on the RCA process. According to VHA policy, NCPS is also responsible for disseminating important information learned from RCAs to VAMCs. NCPS reports to the Assistant Deputy Under Secretary for Health for Quality, Safety, and Value, but also works with other VHA offices, including the Office of the Deputy Under Secretary for Health for Operations and Management, which directs operations at the VISN and VAMC levels. At the VISN level, patient safety officers may provide additional oversight of the RCA process and disseminate information from NCPS to the VAMCs within their networks. Each VAMC has a patient safety manager who facilitates the RCA process at the local level.

RCA Requirements

An RCA may be required by VHA policy if a VAMC’s initial review of an adverse event finds that there is a risk to the safety of veterans, based on the severity of the event and its likelihood of recurrence. VHA requires that each VAMC complete a minimum of eight RCAs each fiscal year, four of which must be on individual adverse events. The other four RCAs can be a combination of individual RCAs and aggregated RCAs, the latter of which review a group of similar adverse events to identify common causes and actions to prevent future occurrences. VHA requires that VAMCs conduct aggregated RCAs on three types of adverse events— falls, adverse drug events, and missing patients—to the extent that they occur in a given year. All RCA-related information is required to be entered into VHA’s centralized RCA reporting system—WebSPOT, a software application within VHA’s Patient Safety Information System. WebSPOT is the means by which RCA information is provided to NCPS and to VISN patient safety officers.

Information obtained through the RCA process is protected and confidential, according to federal law, and cannot be used to inform an adverse action or privileging action against a provider. Therefore, the RCA process is referred to as a protected process.

RCA Process

VAMCs use the RCA process to examine whether a systems or process issue caused an adverse event. Figure 1 provides an overview of the RCA process at VAMCs.

Adverse event occurs. The RCA process at a VAMC begins with the recognition of an adverse event. At the VAMC, the patient safety manager receives information from VAMC staff about an adverse event that occurs at the VAMC. To determine if an RCA is required, the patient safety manager evaluates the event using VHA’s safety assessment code matrix to score the severity of the event and its likelihood of recurrence on a scale of 1 (lowest risk) to 3 (highest risk). As directed by VHA policy, adverse events with a score of 3 always require an RCA. VAMCs have discretion in determining whether to conduct RCAs on adverse events with scores of 1 or 2.

VAMC conducts RCA. After determining the need for an RCA, the VAMC director convenes a multidisciplinary team of VAMC staff to identify root causes and actions to be taken with associated outcome measures. VHA policy states that those staff directly involved in the adverse event cannot participate on the RCA team; however, the RCA team may interview these staff as part of its investigation to obtain their perspectives on the event that occurred and suggestions for preventing its recurrence.

The RCA team is required to develop a report, which includes a description and flowchart of the adverse event, identifies one or more root causes, and includes actions to be taken with associated outcome measures. Actions describe VAMC-level changes to reduce or eliminate future occurrences of similar adverse events. Each action is also required to have at least one outcome measure—a specific, quantifiable, and time- bound means by which responsible staff can determine the extent to which the action has been taken to address the root cause. For example, in the case of an overdose of an anesthesia medication from a pump that held an unsafe amount of medication, the action might be to use a different type of pump that holds less medication and prevents an accidental overdose; an outcome measure might be to measure patient outcomes 1 year later to ensure that no such overdoses occurred.

Leadership reviews/approves. Upon completion of the RCA report, the RCA team presents its findings to VAMC leadership. The completed RCA report is required to be signed by the VAMC director within 45 days of the determination of the need for an RCA.signature is the date the RCA is considered complete. The patient safety manager then submits the completed report to NCPS through WebSPOT.

The date of the director’s VAMC implements RCA actions. After an RCA report is submitted to NCPS, patient safety managers follow up with VAMC staff on the implementation of identified actions, and, after implementation, evaluate the effectiveness of those actions in addressing the identified root causes.

Patient safety managers also update WebSPOT with the actual implementation date of each action. If a VAMC does not implement an action, the patient safety manager can indicate in WebSPOT that the action was not implemented and the reason why. VAMCs may not implement certain actions identified by the RCA team for several reasons, including funding constraints and other unforeseen complications, like building design limitations. After implementation, patient safety managers update WebSPOT to add any comments associated with implementation, as well as information about the effectiveness of each action in addressing identified root causes on a five-point scale from “much worse” to “much better.” In fiscal year 2014, VAMCs most commonly rated RCA actions as having made the related system or process “better” or “much better.”

Upon receipt of a completed RCA report, NCPS staff categorize key aspects, such as the type of adverse event, location of the event, corrective actions, and outcome measures. NCPS staff also categorize RCA actions according to an action strength hierarchy of stronger, intermediate, or weaker. (See table 1 for descriptions of stronger, intermediate, and weaker actions.) NCPS recommends using stronger or intermediate actions to the extent possible to improve the likelihood that actions will remove human error from processes and be more successful in addressing the root causes of an adverse event. About two-thirds (68 percent) of all actions resulting from RCAs in fiscal year 2014 were categorized as stronger or intermediate.

VAMCs Completed Fewer RCAs Each Year from Fiscal Year 2010 through Fiscal Year 2014, but VHA Has Not Analyzed the Reasons for the Decrease

Total completed RCAs (both individual and aggregated) at VAMCs decreased in each of the past 5 fiscal years. Overall, from fiscal years 2010 through 2014, the total number of RCAs completed at VAMCs decreased by 18 percent—from 1,862 in fiscal year 2010 to 1,523 in fiscal year 2014. (See fig. 2.) Individual RCAs accounted for 88 percent of the decrease during this time period.

VHA’s NCPS officials told us they are not certain why the number of completed RCAs has decreased over time, especially in light of an increase in reports of adverse events over the past 5 fiscal years. Specifically, our analysis of adverse event reports in WebSPOT shows that they increased by 7 percent in the past 5 fiscal years (from 109,951 in fiscal year 2010 to 117,136 in fiscal year 2014). An increase in reports does not necessarily mean that there should also be an increase in the number of RCAs conducted, as it is possible that the safety assessment code score was not high enough to require an RCA, giving the VAMC the discretion to address the adverse event through other available processes. However, NCPS officials told us they have not conducted an analysis to determine the contributing factors to the decrease. Without further analysis, it is unclear whether an increase in adverse event reports at the same time that the number of completed RCAs is decreasing is a cause for concern. NCPS’s lack of analysis is not consistent with federal internal control standards, which state that control activities should include comparisons and assessments of different sets of data so that analyses of the relationships can be made and appropriate actions taken.

NCPS officials told us they were aware of the decrease in completed RCAs, but have not conducted an analysis of the decrease because it is difficult to determine causal relationships between many possible contributing factors. Although they have not conducted an analysis, NCPS officials suggested possible contributing factors to the decrease in completed RCAs, including: (1) a change in the culture of safety at VAMCs; (2) VAMCs using alternative processes to address adverse events in place of RCAs; and (3) an increasing number of VAMCs conducting the minimum of four individual RCAs each fiscal year.

Change in the culture of safety at VAMCs. NCPS officials stated that they have observed a change in the culture of safety in recent years in which staff feel less comfortable reporting adverse Officials added that this change events than they did previously.is reflected in NCPS’s periodic survey on staff perceptions of safety; specifically, 2014 scores showed decreases from 2011 on questions measuring staff’s overall perception of patient safety, as well as decreases in perceptions of the extent to which staff work in an environment with a nonpunitive response to error. As previously noted, however, the number of adverse event reports has been increasing, despite NCPS officials’ observation of a change in the culture of safety.

VAMCs’ use of alternative processes. NCPS officials told us that VAMCs sometimes choose alternative processes, such as those based on Lean methods, to address adverse events when an RCA is not required. However, VHA is unaware how many VAMCs use these alternative processes. From fiscal year 2009 through fiscal year 2014, VHA trained over 20,000 staff on the use of Lean methods, but an official from the VA Center for Applied Systems Engineering—the VHA office that conducted the trainings—told us VHA has not conducted any follow-up to determine how these methods are being applied at VAMCs. The official added that, after training, it is up to VAMC leadership to implement Lean methods in their VAMCs, and that the Center for Applied Systems Engineering began working with NCPS about a year ago to begin aligning the RCA process with Lean methods. The lack of follow-up on the use of alternative processes is not consistent with standards for internal control. Without information on the extent to which VAMCs are using alternative processes like Lean methods in place of RCAs, NCPS has limited awareness of the extent to which VAMCs are addressing the root causes of adverse events.

Three of the four VAMCs in our review completed fewer RCAs in fiscal year 2014 compared to fiscal year 2010. Officials at one of these VAMCs told us the reason they had completed fewer RCAs was because the VAMC director supported the use of a Lean method to understand and act on the root cause of an adverse event when an RCA was not required. Officials at this VAMC also told us that they thought their Lean method was sometimes more appropriate for reviewing low-severity events because it yielded similar results to an RCA and allowed for a broader, more complete view of the issue being examined. NCPS officials told us they support VAMCs’ use of these alternative processes when appropriate, but acknowledged loss of information as the results of these processes are not required to be entered into WebSPOT, or otherwise shared with NCPS.

Increasing numbers of VAMCs conducting the minimum of 4 individual RCAs each fiscal year. NCPS officials told us they were aware that by setting a requirement in 2007 that VAMCs conduct a minimum of 4 individual RCAs each fiscal year, VAMCs that had previously completed many more than 4 might decrease the number of individual RCAs they completed over time. Our analysis of RCA data shows that from fiscal years 2010 through 2014, the number of VAMCs completing more than 4 individual RCAs declined by 8 percent (from 135 to 124 VAMCs). In addition, the number of VAMCs completing exactly 4 individual RCAs in this time period more than doubled, from 4 VAMCs in fiscal year 2010 to 10 VAMCs in fiscal year 2014. All 10 of these VAMCs completed more than 4 individual RCAs in fiscal year 2010, with totals ranging from 5 to 14 individual RCAs. Officials stated that the selection of 4 individual RCAs as a minimum (as well as the selection of 8 as a minimum total of individual and aggregated RCAs), was arbitrary but seemed reasonable. They expressed concern that raising the annual individual RCA minimum requirement may result in lower-quality RCAs.

Because NCPS has not conducted an analysis to understand the relationship between the decrease in RCAs and possible contributing factors, such as the increase in adverse event reports and use of alternative processes, it is unclear whether the decrease indicates a negative trend in patient safety at VAMCs or a positive one. For example, the decrease can indicate a negative trend of VAMCs not reporting severe adverse events that would require RCAs, or a positive trend reflecting fewer severe adverse events occurring. Moreover, without complete information on the extent to which VAMCs are using alternative processes to address the root causes of adverse events and the results of those processes, NCPS lacks important data that may be helpful in better identifying trends and system-wide patient safety improvement opportunities.

VHA Oversees the RCA Process by Monitoring VAMC Compliance, and Develops System- Wide Patient Safety Initiatives Informed by RCA Data

NCPS and VISN patient safety officers oversee the RCA process by monitoring each VAMC’s compliance with RCA requirements, including by reviewing RCA information in WebSPOT and conducting site visits.

Reviewing RCA information in WebSPOT. NCPS conducts quarterly reviews of RCA information in WebSPOT to monitor VAMCs’ progress toward meeting annual RCA requirements. NCPS monitors, for example, each VAMC’s progress toward completing the required number of individual and aggregated RCAs for the fiscal year. Our analysis of WebSPOT data shows that, from fiscal year 2010 through fiscal year 2014, almost all VAMCs completed the minimum number of RCAs required each year: an average of 98 percent of VAMCs completed four or more individual RCAs, and an average of 96 percent of VAMCs completed eight or more total RCAs. NCPS officials told us that their review of WebSPOT information also provides insight into the effectiveness of a VAMC’s RCA process. NCPS submits quarterly reports of VAMCs’ progress to the Deputy Under Secretary for Health for Operations and Management.

NCPS officials told us that when they find that a VAMC has not met the annual requirement for the number of completed RCAs, they may contact the VAMC’s patient safety manager to ask if barriers to the RCA process exist. Officials said that, in one such instance, the patient safety manager at a VAMC that had not completed the required number of RCAs told NCPS that the medical center director was not supportive of the RCA process. According to NCPS officials, in situations such as this they may then contact the VAMC’s leadership to remind them of the importance of completing RCAs and of the benefits to the entire system of having complete information in WebSPOT, and to offer their assistance. VISN patient safety officers we spoke with told us that they also monitor VAMCs’ compliance with RCA requirements through reviews of RCA information in WebSPOT, and by meeting with VAMC patient safety managers.

Conducting site visits to VAMCs. NCPS officials said they may conduct a site visit to provide consultation and feedback to a VAMC that appears to be encountering challenges in meeting RCA requirements, such as completing individual RCAs within 45 days. NCPS site visits can also include an examination of other aspects of the RCA process, including reviewing a sample of RCAs to examine the assignment of safety assessment scores, the strength of corrective actions, and the implementation status of the actions. Officials stated that the 12 to 20 site visits they conduct each year are the most valid way for them to verify the implementation of RCA actions because they provide NCPS with the ability to observe implemented activities and the effectiveness of RCA-based improvements. NCPS officials told us that they visit VAMCs at the request of the VAMC director or as participants in a visit made by other VHA offices, including the Deputy Under Secretary for Health for Operations and Management. In addition to NCPS, patient safety officers at three of the four VISNs in our review told us that they also conduct annual site visits to some or all VAMCs in their networks to assess implementation of RCA actions and to consult with VAMC patient safety managers.

In addition to monitoring compliance, NCPS uses RCA information to inform system-wide initiatives to improve patient safety. Not all initiatives are based solely on RCAs, but officials told us that RCAs are a contributing factor to NCPS’s larger patient safety improvement efforts. Officials told us that they focus their initiatives on problems that pose the greatest risk to patients or are the most prevalent in VA’s health care system, such as suicide. Officials explained that their choice of which initiative to pursue is determined by what will have the greatest impact on a problem. Examples of NCPS’s initiatives include Patient Safety Alerts and Advisories, topic summaries, and Clinical Team Training.

Patient Safety Alerts and Advisories. Patient Safety Alerts and Advisories are urgent notifications sent to VAMCs that contain a description of a safety issue, instructions for implementing actions to prevent recurrence of the problem, and due dates for completion of actions. NCPS officials told us that alerts and advisories can come from several sources, including reports from VAMCs, other VHA offices, and medical device manufacturers. Patient Safety Alerts and Advisories are developed by NCPS and then issued by the VHA Deputy Under Secretary for Health for Operations and Management. For example, VHA issued a Patient Safety Alert after a patient in a VAMC behavioral health unit hanged himself from an air conditioning vent. The RCA team recommended a structural change to the vents to prevent recurrence, which VHA then required to be implemented at all VAMCs. NCPS also tracks the date that VAMCs completed implementation of actions. From fiscal year 2010 through fiscal year 2014, NCPS has developed 57 alerts and 7 advisories.

Topic summaries. Officials told us that NCPS may issue an RCA topic summary if they identify a trend in adverse events or RCAs in WebSPOT. An RCA topic summary provides background context for the relevant adverse event, discusses root causes that were identified through the RCAs conducted, and describes corrective actions taken by VAMCs. For example, NCPS officials told us that after their review of RCAs identified a trend in adverse events caused by the misidentification of patients, they determined that system-wide improvements were needed. NCPS prepared topic summaries on misidentification related to specimens and transporting patients, as well as a guidance document on patient wristbands, which included best practices for VAMCs. NCPS officials told us topic summaries are distributed to VAMCs as part of the agenda for monthly conference calls that NCPS conducts with patient safety staff at VAMCs and VISNs, and that they are also made available through NCPS’s internal website and via e-mail. From fiscal year 2010 through fiscal year 2014, NCPS has issued 12 topic summaries.

NCPS may also determine the need for a topic summary on the basis of requests for WebSPOT searches from VAMC and VISN patient safety staff interested in knowing whether RCAs have been conducted for similar adverse events at other VAMCs.

NCPS officials estimated that they conduct about 200 such searches annually, and that these searches provide VAMC and VISN staff with information on similar adverse events, such as the corrective actions identified at other locations to address the adverse event. According to officials, NCPS may determine through these searches that several locations are encountering similar patient safety issues, prompting the preparation of a topic summary.

Clinical Team Training. NCPS implemented Clinical Team Training for surgical teams in 2007 following analysis of RCA information in WebSPOT that found communication failure to be a root cause or contributing factor in 75 percent of the more than 7,000 RCAs reviewed. The objective of Clinical Team Training is to enhance teamwork and overcome obstacles to effective communication across professional boundaries. The training curriculum includes 2 months of preparation by the VAMC; a day- long onsite learning session consisting of lectures, group interaction, and videos; and quarterly interviews of the clinical team to assess training implementation. One study found that surgical mortality decreased 11 percent more in VAMCs that received Clinical Team Training compared to those that had not NCPS officials told us they have expanded Clinical received it. Team Training beyond surgical teams, and have provided this training, for example, to teams in emergency departments, intensive care units, and inpatient behavioral health units.

Julia Neily et al., “Association between Implementation of a Medical Team Training Program and Surgical Mortality,” Journal of the American Medical Association, vol. 304, no. 15 (2010).

Conclusions

RCAs are an important tool for VAMCs to identify the systems or processes that contributed to an adverse event, and implement actions to address them. They are also an important contributor to NCPS initiatives to improve patient safety across VA’s health care system. It is unclear whether the 18 percent decrease in total RCAs completed from fiscal year 2010 to fiscal year 2014 is a negative trend reflecting less reporting of serious adverse events, or a positive trend reflecting fewer serious adverse events that would require an RCA. VHA has not, as would be consistent with federal internal control standards, conducted an analysis to determine the relationship between data showing a decrease in RCAs and factors that may be contributing to this trend, including VAMCs use of alternative processes, such as Lean methods, when RCAs are not required. Although the choice to use alternative processes may be appropriate, NCPS is not aware of the extent to which these processes are used, the types of events being reviewed, or the changes resulting from them. Without analyzing the reasons for declining RCAs, and understanding the extent that VAMCs use alternative processes and their results, NCPS has limited awareness of what VAMCs are doing to address the root causes of adverse events. Moreover, the lack of complete information may result in missed opportunities to identify needed system-wide patient safety improvements.

Recommendations for Executive Action

To ensure that appropriate steps are being taken to address the root causes of adverse events within VHA, the Secretary of Veterans Affairs should direct the Under Secretary for Health to take the following two actions:

Conduct an analysis of the declining number of completed RCAs within the VA health care system, including identifying contributing factors, and take appropriate actions to address them.

Determine the extent to which VAMCs are using alternative processes to address the root causes of adverse events when an RCA is not required, and collect information from VAMCs on the number and results of those alternative processes.

Agency Comments

We provided a draft of this report to VA for comment. In its written comments, reproduced in appendix I, VA generally agreed with our conclusions and concurred with our recommendations. In its comments, VA also provided information on an initial analysis it had conducted, as well as its plans for implementing each recommendation, with an estimated completion date of November 2015.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-7114 or at draperd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II.

Appendix I: Comments from the Department of Veterans Affairs

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the contact named above, Janina Austin, Assistant Director; Jennie F. Apter; Frederick K. Caison; Christine Davis; Kaitlin McConnell; Vikki L. Porter; Emily Wilson; and Malissa G. Winograd made key contributions to this report.