Department of Veterans Affairs 

Office of Inspector General 

Healthcare Inspection 

Evaluation of the Veterans Health 

Administration’s National Patient Safety 

Program 

Report No. 08-02075-148 June 18, 2009

VA Office of Inspector General 

Washington, DC 20420 
 

 

 

To Report Suspected Wrongdoing in VA Programs and Operations 

Telephone: 1-800-488-8244 between 8:30AM and 4PM Eastern Time,

 Monday through Friday, excluding Federal holidays

 E-Mail: vaoighotline@va.gov 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

Contents 

Page

Executive Summary ........................................................................................................i

Introduction ..........................................................................................................1

Summary.......................................................................................................................... 1 

Background...................................................................................................................... 1

Scope and Methodology .................................................................................................. 1 

National Patient Safety Program Overview..................................................................... 2

Inspection Results ...............................................................................................6 

Issue 1: Data Sources and Analyses ............................................................................... 6 

Issue 2: National Patient Safety Program Oversight ...................................................... 9

Issue 3: Evaluation and Improvement of Program Areas............................................. 11

Conclusions.................................................................................................................... 14

Recommendations.......................................................................................................... 14

Acting Under Secretary for Health Comments.............................................................. 14 

Assistant Inspector General Comments......................................................................... 15 

Appendixes 

A. Acting Under Secretary for Health Comments........................................................ 16

B. OIG Contact and Staff Acknowledgments............................................................... 20

C. Report Distribution................................................................................................... 21

VA Office of Inspector General 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

Executive Summary 

Introduction 

The VA Office of Inspector General (OIG), Office of Healthcare Inspections (OHI) 

completed an evaluation of the Veterans Health Administration’s (VHA’s) National 

Patient Safety (NPS) Program. The purposes of the evaluation were to determine 

whether VHA’s NPS Program (1) has been effective in accomplishing its stated goal of 

preventing inadvertent harm to patients receiving VHA care and (2) has provided 

efficient and effective coordination, oversight, and continuous improvement. 

Results and Recommendations 

We concluded that VHA took important, positive steps in 1998 when it expanded existing 

patient safety activities and created the National Center for Patient Safety (NCPS). 

VHA’s NPS Program has been the foundation for many national and international patient 

safety initiatives. We noted several opportunities to strengthen the NPS Program’s 

effectiveness, oversight, and continuous improvement. 

We recommended that the Acting Under Secretary for Health ensure that: 

• All relevant patient data sources be assessed for patient safety significance, 

coordinated across VHA’s quality and safety programs, and used to drive change. 

• Organized, coordinated oversight of the NPS Program is systematically provided 

by either the NCPS or another VHA entity. 

• VHA develops a plan to systematically review all aspects of the NPS Program for 

efficiency and effectiveness and make revisions as appropriate. 

Comments 

The Acting Under Secretary for Health concurred with the findings and 

recommendations. The implementation plan is acceptable, and we will follow up until all 

actions are complete. 

 (original signed by:)

JOHN D. DAIGH, JR., M.D. 

Assistant Inspector General for 

Healthcare Inspections 

VA Office of Inspector General i 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

VA Office of Inspector General 1 

Introduction 

Summary 

The VA Office of Inspector General (OIG), Office of Healthcare Inspections (OHI) 

completed an evaluation of the Veterans Health Administration’s (VHA’s) National 

Patient Safety (NPS) Program. The purposes of the evaluation were to determine 

whether VHA’s NPS Program (1) has been effective in accomplishing its stated goal of 

preventing inadvertent harm to patients receiving VHA care and (2) has provided 

efficient and effective coordination, oversight, and continuous improvement. 

Background 

The medical literature in the past several years has reported as many as 98,000 deaths 

annually due to errors in medical care, many of which are preventable. Preventable 

medical errors have been estimated to cost between $17 and $29 billion per year 

nationwide.1

 VHA responded to these numbers, as well as to intense media scrutiny 

following several high profile cases, by placing more emphasis on patient safety within 

its long-standing quality review and risk management programs. Several VHA offices 

have created programs to evaluate and seek to improve patient care and safety. These 

offices include the Deputy Under Secretary for Health for Operations and Management 

(DUSHOM), the Office of Quality and Performance, the Office of the Medical Inspector, 

and the National Center for Patient Safety (NCPS). Each of these offices has access to 

comprehensive patient databases and can obtain reports that assess performance against 

metrics, such as procedure complication rates, surgery waiting times, and patient 

satisfaction. Some specific programs have developed databases tailored for their patient 

care review needs, such as the National Surgical Quality Improvement Program (NSQIP), 

the Inpatient Evaluation Center, and the Cardiac Assessment Reporting and Tracking 

System. 

Although VHA has been recognized in the health care community worldwide as a leader 

in patient safety and has advocated a system-wide culture change, OHI and Government 

Accountability Office reviews have identified concerns with or suggestions for 

improvement in effectiveness and efficiency. 

Scope and Methodology 

For the purpose of this review, we defined the NPS Program as the activities described in 

the NPS Improvement Handbook2

 (the handbook), which are performed by the NCPS, 

Veterans Integrated Service Network (VISN), and VHA facility staff. To evaluate 

VHA’s NPS Program, we reviewed the handbook, reports, training materials, and other 

 

1

 Institute of Medicine, “To Err is Human: Building a Safer Health System,” November 1999, p. 1. 2

 VHA Handbook 1050.01, VHA National Patient Safety Improvement Handbook, May 23, 2008. 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

relevant documents. We visited the NCPS office July 9–10, 2008, and interviewed NCPS 

staff. We interviewed VA Central Office (VACO), VISN, and facility staff. Also, we 

reviewed patient safety review results and feedback gathered from VHA facilities during 

fiscal year (FY) 2007 Combined Assessment Program (CAP) reviews. As mentioned 

previously, other VHA programs and offices have data and activities relevant to patient 

quality and safety, and although we mention these occasionally, we did not review them 

in detail as part of this report. 

We conducted the review in accordance with Quality Standards for Inspections published 

by the President’s Council on Integrity and Efficiency. 

National Patient Safety Program Overview 

VHA has historically had a mechanism for reporting and reviewing unplanned patient 

incidents. In 1997, VHA established a patient safety oversight committee and an expert 

advisory panel and issued a patient safety handbook. In 1998, the panel met and issued a 

report recommending that a national patient safety program office be created and a 

program director be hired. In order to represent a significant change in the normal way of 

doing business with headquarters in VACO, the committee recommended that the 

program office be established outside of the Washington, DC, area. The NCPS office 

was established in Ann Arbor, MI, as a separate entity from the VHA office that was 

responsible for monitoring patient care quality. A Director was appointed who reported 

directly to the Under Secretary for Health. The goals for the new office were to change 

VHA’s culture to one of concern for patient safety, to prevent harm to patients under 

VHA care, and to involve staff in all VHA facilities. Since 2007, the NCPS Director has 

reported to the Associate Deputy Under Secretary for Health for Quality and Safety. The 

NCPS is a program office and has no line authority over staff in VISN offices or VHA 

facilities. 

In 1999, the revised handbook was issued, which established a policy for identifying, 

reporting, and mitigating vulnerabilities that may result in adverse patient events (such as 

falls and medication errors) and close calls. VHA facility staff are expected to identify 

and report actual adverse patient events and close calls. Facility patient safety managers 

(PSMs) prioritize these events and close calls for severity and probability using the safety 

assessment code (SAC). A root cause analysis (RCA) is a tool used by facility staff to 

determine the reasons why events occurred and to try to prevent future occurrences. The 

handbook describes two types of RCAs—aggregated and individual. Aggregated RCAs 

may be used for four events (falls, adverse drug events, parasuicides [actual or attempted 

suicides], and missing patients) for which data are gathered over time and evaluated 

annually. Individual RCAs are conducted for events with higher SAC scores. PSMs 

enter adverse event information into the NCPS’s patient safety information system 

database, known as SPOT (not an acronym). The NCPS has access to all reported patient 

adverse events, close calls, and RCAs across the VHA system. The SPOT database 

VA Office of Inspector General 2 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

contains only de-identified data—no patient or staff names and no social security 

numbers. 

Some of the accomplishments of the NCPS include: 

• Tools for uniform analysis of unsafe situations (RCAs) and Healthcare Failure 

Mode and Effects Analysis (HFMEA). 

• Development of approaches and tools to prevent unsafe situations (including falls 

prevention toolkit, a program to ensure correct surgery, and a series of cognitive 

aids for anesthesiology). 

• Development of patient safety alerts and advisories, which are forwarded to the 

DUSHOM for release to VHA facilities. 

• National education and training programs. 

• Internal and external websites and publications (newsletters and peer-reviewed 

journals). 

• Data sharing (facility, VISN, and VHA analysis; topic summaries; and tailored 

reports). 

• Collaboration with and dissemination of techniques to national and international 

health care organizations (including The Joint Commission, the National Patient 

Safety Foundation, and the World Health Organization). 

Also, the NCPS led and participated with other VHA program offices in the development 

of directives for the prevention of retained surgical items and for emergency airway 

management outside the operating room. Additionally, the NCPS created the Medical 

Team Training program, which is leading improvement efforts at all VHA facilities with 

surgical programs. 

In 2004, in response to safety vulnerabilities identified by the NCPS, VHA funded the 

Bar Code Expansion project to increase positive patient identification and decrease 

vulnerabilities in the labeling of all blood and laboratory specimens. This large multiyear project, which is managed by the Bar Code Resource Office, introduces handheld 

devices with integrated bar code scanners into patient care settings. In 2006, funds were 

provided to purchase the devices. In 2007, approval was given to purchase the software 

and hire personnel to support the project, and pilot testing began at 16 facilities that 

already had compatible wireless infrastructures. However, since the reorganization of 

information technology (IT) functions in VA, progress has virtually stopped. This is an 

important patient safety project and it should be kept on schedule until completion. 

National Center for Patient Safety Budget and Staff Resources. 

• Beginning in 1998, a program known as the Patient Safety Center of Inquiry 

(PSCI) received funding of $2 million. This program has provided funds to study 

topics such as home care safety, provider fatigue, and chemotherapy safety. 

VA Office of Inspector General 3 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

• In 1999, the NCPS Director received approval for 16 full-time employee 

equivalents (FTE) and an initial budget of $306,955. The NCPS FTE and budget 

have increased almost every year since the NCPS’s inception. 

• In 2000, the Patient Safety Reporting System (PSRS) was created and initially 

funded for $2 million. The PSRS is an independent incident reporting mechanism. 

The table below shows NCPS staffing and budget figures for the period FY 2000–2008. 

Three FTE under VA’s Office of IT are dedicated to NCPS’s data systems but are not 

included in the table. 

Table 1: National Center for Patient Safety Budget and Resources 

FY NCPS 

FTE 

NCPS Office 

Budget 

PSCI Budget PSRS Budget Total 

2000 24 $2.4 million $2 million $200,000 $4.6 million 

2001 31 $3.7 million $2 million $2 million $7.7 million 

2002 34.2 $3.6 million $2 million $3 million $8.6 million 

2003 34.2 $4.3 million $1.5 million $3 million $8.8 million 

2004 39.7 $5.8 million $1.5 million $1.4 million $8.7 million 

2005 39.7 $4.9 million $1.8 million $1.6 million $8.3 million 

2006 39.7 $6.5 million $1.4 million $1.4 million $9.3 million 

2007 48 $8.6 million $1.6 million $1.4 million $11.6 million 

2008 50 $10 million $1.5 million $1.4 million $12.9 million 

In addition to NCPS staff, each VHA facility is required to have at least one FTE PSM, 

and each VISN office has one FTE Patient Safety Officer (PSO). This totals 

approximately 180 FTE dedicated to patient safety. These positions report to the facility 

and the VISN, respectively, and are not accountable to the NCPS. 

National Center for Patient Safety Goal. The NCPS goal of preventing inadvertent 

harm to patients as a result of their care is laudable but difficult to measure. It appears 

that data from the entire U.S. health care system does not support a conclusion that 

VA Office of Inspector General 4 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

VA Office of Inspector General 5 

typical patient safety programs prevent harm to patients.3 Two reasons for this inability 

to state whether patients are safer are that attempts to address patient safety generally 

affect processes of care and that the data apply to small numbers of patients. A 

meaningful patient safety program should include defined program objectives, personnel, 

and a budget and should be monitored by regular progress reports to governance. The 

NCPS Director agreed that the goal is difficult to measure but cited the patient safety 

alerts as examples of how safety hazards had been identified and eliminated. We 

suggested that the program goals and objectives be modified to allow measurement and 

regular monitoring to determine if goals and objectives are being met. 

Patient Safety Culture. The NCPS has advocated the adoption of a culture of patient 

safety where prevention is emphasized over punishment to mitigate system vulnerabilities 

and reduce adverse events. Several authoritative sources confirm that the likelihood of 

enlisting employee cooperation in identifying and addressing safety concerns is higher 

when employees are not blamed.4,5

 While VHA directives provide methods to deal with 

employees who have committed intentional errors or unsafe acts, tension exists between 

the non-punitive culture and the expectation of accountability in government. 

 

3

 Peter J. Pronovost, MD, PhD, et al., “Tracking Progress in Patient Safety – An Elusive Target,” JAMA, Vol. 296, 

No. 6, August 9, 2006. 

4

 Institute of Medicine, p 2. 

5

 The Joint Commission, 2008 Comprehensive Accreditation Manual for Hospitals: The Official Handbook, PI-1. 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

Inspection Results 

Issue 1: Data Sources and Analyses 

To obtain the most comprehensive view of patient safety in VHA facilities, it is important 

to identify as many safety concerns as possible from all available sources. Many 

programs under the broad umbrella of quality and safety have the potential to identify 

safety issues and adverse events. At the facility level, the following programs comprise a 

partial list: 

• Patient incident reporting. 

• Patient advocate. 

• Peer review. 

• Tort claim information system. 

• Morbidity and mortality conferences. 

• NSQIP. 

• Infection control. 

While some facility staff may share data from these programs to identify patient safety 

issues and events, no such sharing is required. Most of these programs require facility 

data to be entered into databases or sent in reports that are available to the responsible 

program offices at the VACO level. If these databases were available to all relevant 

program offices for use in data analysis, it is possible that resulting actions could improve 

patient care quality and safety. However, quality and safety information is not always 

well coordinated among VHA entities, including the NCPS, but data outside its database 

is outside its sphere of control. 

The NCPS intranet website contains a variety of data that is available to VHA staff. 

Recently, the NCPS increased the types of trending summary reports available after 

receiving feedback that VACO, VISN, and facility staff have not consistently received 

information that they deem important. Specific examples of additional opportunities 

within the NCPS to improve adverse event identification and analyses follow. 

Adverse Event Identification. While VHA employees are expected to identify and 

report adverse events that they are aware of, reporting is voluntary, and there are 

generally no repercussions for not reporting adverse events. Many experts state that all 

employee reporting programs (voluntary and mandatory) result in substantial 

underreporting. The Institute for Healthcare Improvement (IHI) stated: 

Traditional efforts to detect adverse events have focused on voluntary 

reporting and tracking of errors. However, public health researchers have 

established that only 10 to 20 percent of errors are ever reported and, of 

those, 90 to 95 percent cause no harm to patients. Hospitals need a more 

VA Office of Inspector General 6 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

VA Office of Inspector General 7 

effective way to identify events that do cause harm to patients, in order to 

select and test changes to reduce harm.6

IHI explains that employee reporting alone is inadequate because it is highly subjective 

and often requires overburdened providers to complete time-consuming paperwork.7

 

Some health care organizations have implemented adverse event identification programs 

that do not rely on employee reporting. Several studies have shown that computer 

monitoring strategies have identified many times more potential adverse events than were 

reported through employee reporting mechanisms.8,9,10 While some of the events 

identified via these strategies were not confirmed as adverse events after review, the 

results provided a more complete picture of patient safety vulnerabilities that could be 

addressed. The NCPS has not endorsed any computer monitoring system. 

It is possible that VHA policies could be affected by the limited data captured by 

employees voluntarily reporting patient adverse events. A more robust safety concern 

identification system could provide more valid information on which to focus policy 

changes. 

Adverse Event Counts. Since 2000, PSMs have reported more than 400,000 aggregated 

review events and 154,000 safety reports or close calls. Facility staff have performed 

more than 10,000 RCAs. The graph on page 8 shows the frequency of the top 10 types of 

RCA events in the SPOT database. 

 

6

 Institute for Healthcare Improvement, “Introduction to Trigger Tools for Identifying Adverse Events,” 

http://www.ihi.org/IHI/Topics/PatientSafety/SafetyGeneral/Tools/IntrotoTriggerToolsforIdentifyingAEs.htm, 

accessed on May 14, 2008. 

7

 Ibid. 

8

 M. K. Szekendi, et al., “Active surveillance using electronic triggers to detect adverse events in hospitalized 

patients,” Qual Saf Health Care, Vol. 15, June 2006, pp. 184–190. 9

 David W. Bates, MD, MSc, et al., “Detecting Adverse Events Using Information Technology,” J Am Med Inform 

Assoc, Vol. 10, No. 2, March–April 2003, pp. 115–128. 10 C. W. Johnson, “How will we get the data and what will we do with it then? Issues in the reporting of adverse 

healthcare events,” Qual Saf Health Care, Vol. 12, December 2003, p. ii64. 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

Event Volume

Event Type

Inpatient Para- suicides

Communication of

Abnormal Results

Hospital Acquired

Infections

Outpatient Suicides

Missing Patients

Misidentification

High Alert Adverse Drug 

Events

Deaths Other Than

Suicide

Falls

Delays in Treatment/ 

Diagnosis/Surgery

700

600

500

400

300

200

100

0

123

143 140

253 251 248

323

351

601 616

Distribution of Root Cause Analysis Event Types Submitted to the National Center for 

Patient Safety for all of the Veterans Health Administration

July 1, 2005, through June 30, 2008

Aggregated RCAs address falls, adverse drug events, parasuicides, and missing patients. 

The highest type “delays in treatment/diagnosis/surgery” is also a frequent basis for tort 

claims and complaints to the OIG hotline. It would seem that this important event type 

could be considered for special review and analysis. NCPS staff told us that an analyst 

has been assigned to study this complex event type. 

Adverse Event Data Availability. The SPOT database contains large amounts of 

de-identified data. When VACO and oversight bodies have requested VHA data 

regarding adverse events, such as wrong site surgeries, they have expressed frustration at 

receiving selected data that does not always represent the entire spectrum of events and is 

difficult to track to specific facilities and patients. Facility PSMs have expressed 

frustration that they have had to create secondary, duplicative systems in order to capture 

the patient information needed for effective reviews and reports. Recently, the NCPS has 

identified a patient incident reporting system developed at a VHA facility that could be 

used VHA wide and might reduce the need for a duplicate information system. 

NCPS staff told us that they do not need patient-specific information because it is not 

relevant to understanding the adverse event or close call that occurred. They also told us 

that the identities of staff who report patient incidents must be kept confidential to 

prevent any repercussions for reporting. NCPS staff insist that other uses of SPOT data 

would be inappropriate. Nonetheless, we suggest that the NCPS, along with General 

VA Office of Inspector General 8 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

Counsel, re-evaluate the SPOT database and consider including patient and facility 

identifiers to improve utility for all users. 

Patient Safety Reporting System. This voluntary, confidential safety reporting program 

was initiated in 2000. It was patterned after the Aviation Safety Reporting System 

administered by the National Aeronautic and Space Administration. The confidentiality 

of PSRS information only allows VHA review of de-identified information about the 

reported safety issues. 

Approximately 750 issues were reported to the PSRS between FYs 2002 and 2007, and 

the frequency ranged from 84–196 reports per FY. The total cost of the PSRS program 

for the same 6-year period exceeded $11 million (range $1.4–$3 million per FY). The 

NCPS Director told us that the PSRS is a safety valve that permits individuals who are 

not willing to report by any other means to confidentially report events that would 

otherwise go unknown. However, employees who wish to report concerns anonymously 

have several hotline options that exist within VA (for example, the Office of Business 

Integrity and Compliance, the Office of Research Oversight, and the OIG), as well as 

options that exist outside of VA (such as The Joint Commission, the Nuclear Regulatory 

Commission, and the Occupational Safety and Health Administration). Patients who 

wish to report concerns also have options, including the VHA suicide hotline, the OIG 

hotline, and the Department of Defense hotline. We suggest that VHA re-evaluate the 

PSRS and consider eliminating it. 

Summary. Patient safety could be improved by better coordinating existing data sources 

in various programs, expanding the identification of patient events through the addition 

of automated systems, making appropriately identified data available for analysis, and 

using the data to drive change. High frequency event types should be given appropriate 

attention. We recommended that VHA ensure that all relevant patient data sources be 

assessed for patient safety significance, coordinated across VHA’s quality and safety 

programs, and used to drive change. 

Issue 2: National Patient Safety Program Oversight 

We found that although the NCPS monitors selected data elements within required 

processes, it does not provide comprehensive oversight of the NPS Program. The NCPS 

Director confirmed that his office is consultative and does not provide oversight. 

Although a patient safety oversight committee was in place prior to the creation of the 

NCPS, it ceased to exist in 2001. Specific examples of opportunities to improve 

oversight follow. 

Site Visits. NCPS staff conduct consultative site visits to each VHA facility every 

3 years. They told us that these visits serve the purpose of allowing for onsite evaluation 

of the facility’s patient safety program using a structured patient safety assessment tool. 

VA Office of Inspector General 9 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

NCPS staff provide consultations, follow-up on patient safety alerts, and review RCAs. 

NCPS staff told us that facility staff have expressed appreciation for these visits. 

However, prior to our review, the site visits were conducted inconsistently and did not 

result in documented corrective action plans when deficits were identified. These issues 

were corrected subsequent to our review, and in February 2009 a DUSHOM 

memorandum was issued requiring facility managers to submit formal abatement plans 

following NCPS site visits. Also, prior to our review, no attempt had been made to 

analyze the site visits’ results system wide to determine trends. Following our review, 

the NCPS presented trends observed during site visits to PSOs for discussion during a fall 

2008 conference. 

Many VISNs have created continuous readiness teams that visit facilities on a regular 

basis and include aspects of patient safety in their reviews. Site visits are also made 

through VHA’s Systematic Ongoing Assessment and Review Strategy program and 

include patient safety aspects. Several other VHA program offices conduct site visits for 

internal review, including the Office of Geriatrics and Extended Care, the Office of the 

Medical Inspector, NSQIP, and the Office of Research Oversight. Feedback from VHA 

facility staff is that too many internal teams conduct visits that can be disruptive to 

normal business. It is not clear that results from these visits are compared or trended to 

identify opportunities for improvement in programs or at facilities. 

Patient Safety Roles. The NCPS provides consultative services and has no line authority 

over PSOs or PSMs, who report to VISN and facility directors, respectively. The NCPS 

effects change through the DUSHOM’s office. Facility compliance with patient safety 

processes is monitored by PSOs. However, there is some duplication and lack of role 

clarity between NCPS staff and PSOs regarding several patient safety responsibilities 

(described in Table 2 on the next page). Efficiency could be improved and frustration 

reduced by better defining these roles. 

VA Office of Inspector General 10 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

Table 2: Comparison of Patient Safety Responsibilities of National Center for 

Patient Safety Staff and Patient Safety Officers 

Task NCPS Staff PSOs 

Site visits Every 3 years As needed 

Review all RCAs Yes Yes 

Track RCA completion Yes Yes 

Track RCA action item 

completion 

Review a sample of actions, 

issue reports regarding 

strength of action items. 

Closed actions are tracked 

and the data on open and 

closed actions is provided 

on the intranet site. 

Verify that actions are being 

implemented. 

Patient safety alerts and 

advisories 

Issue alerts and advisories, 

review at least two alert 

action plans on site visits. 

Track all alert action plan 

implementation. 

Provide training and 

consultation to facility staff 

Yes Yes 

Summary. It is expected that organized, coordinated oversight of VHA programs be 

provided to determine whether policies are effective and relevant or in need of revision. 

Currently, there appears to be redundancy and lack of role clarity between NCPS staff 

and VISN PSOs, with facility PSMs receiving mixed messages. We recommended that 

organized, coordinated oversight of the NPS Program be systematically provided by 

either the NCPS or another VHA entity. 

Issue 3: Evaluation and Improvement of Program Areas 

We found that the NCPS does not document systematic evaluation of required patient 

safety processes to determine if revision is needed. It is a general philosophy of any 

quality review activity to continually assess and seek to improve key processes. 

The NCPS Director told us that the NCPS has made the following changes in patient 

safety processes since 1999. 

• Strength of action reviews were incorporated into RCAs. 

• SPOT was revised 12 times. 

• Cause and effect diagrams have been added to enhance RCAs. 

• Cognitive aid books were widely distributed to assist PSMs and RCA teams. 

• An advisory committee comprised of PSMs and PSOs regularly updates the 

patient safety assessment tool (PSAT). 

VA Office of Inspector General 11 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

VA Office of Inspector General 12 

However, we did not find evidence of periodic, systematic evaluation. Examples of areas 

that would benefit from systematic assessment and possible revision follow. 

Cumbersome Processes and Content. In 1999, the NCPS developed several key 

processes and has since made several changes, as stated previously. However, feedback 

from PSOs and PSMs indicates that some items (such as RCAs, HFMEAs, and the 

PSAT) are overly cumbersome and would benefit from streamlining both the process and 

the content. 

Follow-Up of Action Items. Most RCAs result in action plans to prevent future similar 

occurrences. Facility PSMs are expected to follow all action items to full resolution. The 

NCPS selects a sample of RCAs across the VHA system and rates the actions as weaker, 

intermediate, or stronger. Between October 1, 2004, and June 5, 2008, the NCPS found 

52 percent of actions to be weaker, 25 percent to be intermediate, and 23 percent to be 

stronger. NCPS staff provide data about the trends in the sample of actions they review. 

They also provide consultation regarding action items, when requested. However, NCPS 

staff do not require resubmission of inadequate actions. Also, prior to our review, they 

did not intervene in real time to improve the strength of actions. With the Web-based 

SPOT, we were told that real time feedback is now possible. 

Joint Commission standards require facility managers to use the information from patient 

safety data analysis to implement changes and to evaluate these changes to determine 

whether they achieved the expected results. During FY 2007 CAP reviews, we found 

that facility managers did not consistently assure implementation of recommended 

corrective actions or evaluate the effectiveness of the interventions.11 While some 

facility managers had efficient corrective action tracking methods, others had none. We 

found inadequate implementation and evaluation of corrective actions in significant 

numbers of facilities in all four aggregate RCA areas. We have repeatedly recommended 

that facility directors effectively implement and evaluate corrective actions. 

The NCPS reviews all RCAs in the SPOT database and rates selected RCAs for strength 

of actions. Patient safety could be improved by ensuring that RCAs result in appropriate, 

adequate actions that are fully implemented. 

Safety Assessment Code Inter-Rater Reliability. SAC scoring determines the level 

and type of review required and is an important early step in adverse event evaluation. 

However, due to the variability of SAC scoring among facility staff, there is a chance of 

incorrectly scoring an adverse event, resulting in the inappropriate review process. 

During some FY 2007 CAP reviews, facilities corrected SAC scores, and RCAs were 

initiated that would not have been done based on the original SAC scores. Therefore, an 

 

11 Evaluation of Quality Management in Veterans Health Administration Facilities Fiscal Year 2007 (Report 

No. 07-00060-126, May 14, 2008), p. 8. 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

VA Office of Inspector General 13 

inter-rater reliability process that evaluates selected SAC scores would increase 

confidence that adverse events are being properly scored and reviewed. 

We did not find evidence of a system-wide inter-rater reliability process to evaluate the 

accuracy of adverse event SAC scores. While PSOs are tasked to verify the correct 

scoring of events, it would be useful to have guidance on how to conduct these 

evaluations consistently across the VHA system. Without an evaluation of the accuracy 

of SAC scoring, it is difficult to ensure that events receive the correct level of review to 

identify process or systems issues. 

Adverse Event Disclosure. The handbook states that VHA facility staff have an 

obligation to inform—or disclose to—patients about adverse events consequent to their 

VHA care (for example, as a result of significant medication errors).12 The routine 

disclosure of adverse events to patients has been VHA’s national policy since 1995.13,14 

We requested evidence from the NCPS about adverse event disclosure but were told that 

the NCPS does not review serious adverse events to determine whether patients were 

appropriately informed. The NCPS Director stated that the NCPS has no role in the 

disclosure of adverse events. 

During FY 2007 CAP reviews, we assessed adverse event disclosure and reported 

weaknesses.15 We reported that only 21 (54 percent) of 39 facilities had completed full 

disclosure. We found that adverse events reported through facility patient safety 

programs were the most likely to be considered for disclosure. We recommended that 

VHA reinforce the importance of compliance with this requirement. 

Barriers to disclosing adverse events include discomfort with conducting the 

conversations and differing interpretations of which events should be disclosed. A 

March 2006 consensus statement reiterated the importance of disclosure and sincere 

apology when patients have been injured while under medical care.16 More than 2 years 

after VHA provided new guidance, compliance continues to be below expectations. It is 

possible that the NCPS could assist in increasing compliance with this important yet 

challenging requirement. An NCPS staff member agreed and stated that the RCA process 

could be amended to include a notation about whether or not the adverse event was 

disclosed. 

Summary. As in all aspects of health care, standards and technologies change 

frequently. Periodic re-assessment of patient safety processes is necessary to keep 

 

12 VHA Handbook 1050.01, VHA National Patient Safety Improvement Handbook, May 23, 2008, p. 15. 13 Under Secretary for Health’s Information Letter, Disclosing Adverse Events to Patients, IL 10-2003-001, 

May 13, 2003. 

14 VHA Directive 2005-049, Disclosure of Adverse Events to Patients, October 27, 2005. 15 Evaluation of Quality Management in Veterans Health Administration Facilities Fiscal Year 2007, p. 4. 16 Massachusetts Coalition for the Prevention of Medical Errors, When Things Go Wrong: Responding to Adverse 

Events, A Consensus Statement of the Harvard Hospitals, Burlington, MA, March 2006. 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

current and to improve efficiency by eliminating steps that may no longer be necessary. 

We recommended that VHA develop a plan to systematically review all aspects of the 

NPS Program for efficiency and effectiveness and make revisions as appropriate. 

Conclusions 

We concluded that VHA took important, positive steps in 1998 when it expanded existing 

patient safety activities and created the NCPS. VHA’s NPS Program has been the 

foundation for many national and international patient safety initiatives. We noted 

several opportunities to strengthen the NPS Program’s effectiveness, oversight, and 

continuous improvement. 

Recommendations 

Recommendation 1. We recommended that the Acting Under Secretary for Health 

ensure that all relevant patient data sources be assessed for patient safety significance, 

coordinated across VHA’s quality and safety programs, and used to drive change. 

Recommendation 2. We recommended that the Acting Under Secretary for Health 

ensure that organized, coordinated oversight of the NPS Program is systematically 

provided by either the NCPS or another VHA entity. 

Recommendation 3. We recommended that the Acting Under Secretary for Health 

ensure that VHA develops a plan to systematically review all aspects of the NPS Program 

for efficiency and effectiveness and makes revisions as appropriate. 

Acting Under Secretary for Health Comments 

The Acting Under Secretary for Health concurred with the recommendations and provided 

implementation plans with target completion dates. During a recent international 

conference, quality and safety experts provided useful suggestions for initiatives to 

enhance the focus of the NPS program. The suggestions for enhancement in the NPS 

Program and sharing of data systems will be reviewed and appropriately incorporated in 

the strategic plan for quality and safety. Also, VHA will develop and implement a system 

to evaluate and assess the effectiveness of the NPS Program. This assessment system will 

be administered by the NCPS with oversight by the Office of Quality and Safety. Periodic 

reports of the NPS program will be made to the Under Secretary’s Coordinating Council 

for Quality and Safety. The full text of the comments is provided in Appendix A 

(beginning on page 16). 

VA Office of Inspector General 14 
Evaluation of the Veterans Health Administration’s National Patient Safety Program 

VA Office of Inspector General 15 

Assistant Inspector General Comments 

The Acting Under Secretary for Health’s comments and implementation plans are 

responsive to the recommendations. We will continue to follow up until all actions are 

completed. 
Evaluation of the Veterans Health Administration National Patient Safety Program

Appendix A 

Acting Under Secretary for Health Comments 

VA Office of Inspector General 16 

Department of 

Veterans Affairs Memorandum

Date: May 29, 2009 

From: Acting Under Secretary for Health (10) 

Subject: Healthcare Inspection – Evaluation of the Veterans Health 

Administration National Patient Safety Program 

To: Assistant Inspector General for Healthcare Inspections (54) 

1. I have reviewed the draft report, and I concur with the report and the 

recommendations. Thank you for the opportunity to review the draft report 

and for incorporating our technical comments in the final report. As noted in 

the report, VHA is recognized in the health care community worldwide as a 

leader in patient safety and as an advocate of system-wide culture change. The 

National Center for Patient Safety (NCPS) has made significant contributions 

to the field of patient safety and has been the foundation for many national and 

international initiatives; however, no program is without opportunity for 

improvement. 

2. I agree that patient data sources should be assessed for patient significance, 

coordinated across VHA’s quality and safety programs, and used to drive 

change. During a recent International Roundtable on Clinical Quality and 

Patient Safety, internal VA quality and safety experts and external consultants 

provided useful suggestions for initiatives to enhance the focus of the patient 

safety program. The suggestions for enhancement will be reviewed and 

appropriately incorporated in the strategic plan for quality and safety. 

3. In addition, I also agree with your assessment of the need for 

comprehensive oversight of patient safety programs. The VHA Office of 

Quality and Safety will collaborate with the Deputy USH for Health for 

Operations and Management to develop and implement a system to evaluate 

and assess the effectiveness of the National Patient Safety (NPS) Program as 

defined in Handbook 1050.01, VHA National Patient Safety Improvement 

Handbook. This assessment system will be administered by NCPS with 

oversight by the Office of Quality and Safety. Periodic reports of NPS 
Evaluation of the Veterans Health Administration National Patient Safety Program

VA Office of Inspector General 17 

programs will be made to the Under Secretary’s Coordinating Council for 

Quality and Safety. 

4. If you have any questions, please have a member of your staff contact 

Margaret M. Seleski, Director, Management Review Service (10B5) at 

(202) 461-8470. 

 (original signed by:) 

Gerald M. Cross, MD, FAAFP 

Attachment 
Evaluation of the Veterans Health Administration National Patient Safety Program

Acting Under Secretary for Health Comments 

to Office of Inspector General’s Report 

The following comments are submitted in response to the recommendations 

in the Office of Inspector General’s report: 

OIG Recommendations

Recommendation 1. We recommended that the Acting Under Secretary 

for Health ensure that all relevant patient data sources be assessed for 

patient safety significance, coordinated across VHA’s quality and safety 

programs, and used to drive change. 

Concur 

The Acting USH agrees that data streams must be available and utilized 

effectively to provide data and information which drives needed change. On 

January 21, 2009, the USH signed a memorandum ensuring that the National 

Center for Patient Safety (NCPS) had unencumbered access to all data from 

any VHA database, data source, or report. The NCPS is empowered to work 

directly with other program offices, the quality programs within the Office 

of Quality and Safety (the Office of Quality and Performance and the Office 

of Quality and Safety Analytics) as well as through the Under Secretary’s 

Coordinating Committee for Quality and Safety to coordinate patient safety 

initiatives and to drive organizational change. The USH recently sponsored 

an International Roundtable on Clinical Quality and Patient Safety during 

which internal VA quality and safety experts and external consultants 

provided useful suggestions for initiatives to enhance the focus of VHA’s 

patient safety program. We will be studying these recommendations and 

incorporating them as appropriate into our quality and safety strategic plan. 

Target date: In process November 2009

Recommendation 2. We recommended that the Acting Under Secretary 

for Health ensure that organized, coordinated oversight of the NPS Program 

is systematically provided by either the NCPS or another VHA entity. 

Concur 

The Acting USH concurs that there should be coordinated and 

comprehensive oversight of VHA’s Patient Safety programs. The VHA 

Office of Quality and Safety will collaborate with the Deputy USH for 

VA Office of Inspector General 18 
Evaluation of the Veterans Health Administration National Patient Safety Program

VA Office of Inspector General 19 

Operations and Management to develop and implement a system to evaluate 

and assess the effectiveness of National Patient Safety (NPS) programs as 

defined in Handbook 1050.01, VHA National Patient Safety Improvement 

Handbook, dated May 23, 2008. The NCPS will administer this assessment 

system with oversight by the Office of Quality and Safety. Periodic reports 

of NPS programs will be made to the USH’s Coordinating Council for 

Quality and Safety. 

Target date: In process November 2009

Recommendation 3. We recommended that the Acting Under Secretary 

for Health ensure that VHA develops a plan to systematically review all 

aspects of the NPS Program for efficiency and effectiveness and makes 

revisions as appropriate. 

Concur 

The NCPS will establish a plan for performing an annual systematic review 

of key national patient safety programs and as appropriate, for making 

revisions to address identified gaps and deficiencies. The NCPS will report 

its findings and coordinate programmatic initiatives through regular reports 

to the USH’s Coordinating Council for Quality and Safety. 

Target date: In process November 2009
Evaluation of the Veterans Health Administration National Patient Safety Program

Appendix B 

OIG Contact and Staff Acknowledgments 

OIG Contact Julie Watrous, RN, MS 

Director, Combined Assessment Program 

(213) 253-5134 

Acknowledgments Toni Woodard 

VA Office of Inspector General 20 
Evaluation of the Veterans Health Administration National Patient Safety Program

Appendix C 

Report Distribution 

VA Distribution

Office of the Secretary 

Veterans Health Administration 

Assistant Secretaries 

Deputy Under Secretary for Health for Operations and Management 

Associate Deputy Under Secretary for Health for Quality and Safety 

Office of Quality and Performance 

Director, National Center for Patient Safety 

Office of General Counsel 

Office of the Medical Inspector 

Veterans Integrated Service Network Directors (1–23) 

Non-VA Distribution

House Committee on Veterans’ Affairs 

House Appropriations Subcommittee on Military Construction, Veterans Affairs, and 

Related Agencies 

House Committee on Oversight and Government Reform 

Senate Committee on Veterans’ Affairs 

Senate Appropriations Subcommittee on Military Construction, Veterans Affairs, and 

Related Agencies 

Senate Committee on Homeland Security and Governmental Affairs 

National Veterans Service Organizations 

Government Accountability Office 

Office of Management and Budget 

This report is available at http://www.va.gov/oig/publications/reports-list.asp. 

VA Office of Inspector General 21