Report to the Secretary of Veterans 

Affairs 

United States Government Accountability Office

GAO

December 2004 VA PATIENT SAFETY 

PROGRAM 

A Cultural Perspective 

at Four Medical 

Facilities 

GAO-05-83 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

www.gao.gov/cgi-bin/getrpt?GAO-05-83.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Nancy 

Kingsbury at (202) 512-2700 or 

kingsburyn@gao.gov. 

Highlights of GAO-05-83, a report to 

Secretary of Veterans Affairs 

December 2004

VA PATIENT SAFETY PROGRAM

A Cultural Perspective at Four Medical 

Facilities 

GAO found progress in staff familiarity with and participation in the VA 

Patient Safety Program’s key initiatives, but these achievements varied 

substantially in the four facilities we visited. In our study conducted from 

November 2002 through August 2004, three-fourths of the clinicians across 

the facilities were familiar with the concepts of teams investigating root 

causes of unintentional adverse events and close calls. One-third of the staff 

had participated in such teams, and most who participated in these teams 

found it a positive learning experience. 

The cultural support clinicians expressed for the Program also differed. At 

three of four facilities, GAO found a supportive culture, but at one facility 

the culture blocked participation for many clinicians. Clinicians articulated 

two themes that could stimulate culture change: leadership actions and open 

communication. For example, nurses need the confidence to disagree with 

physicians when they find an unsafe situation. Although VA has conducted a 

cultural survey, it has not set goals or explicitly measured, for example, staff 

familiarity and mutual trust. 

Clinicians reported management practices at one facility that had helped 

them adopt the Program, including (1) story-telling techniques such as 

leaders telling about a case in which reporting an adverse event resulted in 

system change, (2) management efforts to coach staff, and (3) reward 

systems. 

The Patient Safety Program Process in the figure shows how ideally (1) 

clinicians have cultural support for reporting adverse events and close calls, 

(2) teams investigate root causes, (3) systems are changed, (4) feedback and 

reward systems encourage reporting, and (5) patients are safer. 

The Patient Safety Program Process 

Source: GAO. 

Supportive culture includes mutual trust and comfort in reporting

GAO focused on these safety activities

Program Reporting

Feedback/rewards

Modify program

System changes Safer patients

Evaluation

Root cause 

analysis

The Department of Veterans Affairs 

(VA) introduced its Patient Safety 

Program in 1999 in order to 

discover and fix system flaws that 

could harm patients. The Program 

process relies on staff reports of 

close calls and adverse events. 

GAO found that achieving success 

requires a cultural shift from fear of

punishment for reporting close 

calls and adverse events to mutual 

trust and comfort in reporting 

them. 

GAO used ethnographic techniques 

to study the Patient Safety Program 

from the perspective of direct care 

clinicians at four VA medical 

facilities. This approach recognizes 

that what people say, do, and 

believe reflects a shared culture. 

The focus included (1) the status of 

VA’s efforts to implement the 

Program, (2) the extent to which a 

culture exists that supports the 

Program, and (3) practices that 

promote patient safety. GAO 

combined more traditional survey 

methods with those from 

ethnography, including in-depth 

interviews and observation. 

What GAO Recommends

To better assess the adequacy of 

clinicians’ familiarity with, 

participation in, and cultural 

support for the Program, VA should 

(1) set goals, (2) develop tools for 

measuring goals by facility, and (3) 

develop interventions when goals 

have not been met. VA concurred 

with our recommendations and will 

develop an action plan. 
Page i GAO-05-83 VA Patient Safety Program 

Letter 1

Chapter 1 VA’s Patient Safety Program 2 

Scope and Methodology 5 

Background 10 

Chapter 2 Progress in Clinicians’ Familiarity with and 

Participation in the Program 15 

Facilities Shared Safety Hazards but Not Program Familiarity and 

Participation 15 

Summary 24 

Chapter 3 Measuring Cultural Support for the Program 25 

Varying Cultural Support 25 

Building a Supportive Culture 31 

Improving Assessment of, Familiarity with, Participation in, and 

Cultural Support for the Program 42 

Summary 42 

Chapter 4 Promoting Patient Safety 44 

Using Storytelling to Promote Culture Change 44 

Deliberate Teaching, Coaching, and Role Modeling 47 

Rewarding Close Call Reporting 48 

Summary 49 

Chapter 5 Conclusions and Recommendations 51 

Measuring Clinicians’ Familiarity with and Cultural Support for the 

Program 52 

Recommendations for Executive Action 53 

Agency Comments and Our Evaluation 53 

Appendix I Content Analysis, Statistical Tests, and Intercoder 

Reliability 54 

Content Analysis 54 

Ethnography 54 

Data Collection 54 

Data Analysis 56 

Significance Testing 57 

Contents 
Page ii GAO-05-83 VA Patient Safety Program 

Intercoder Reliability 59 

Appendix II A Timeline of the Implementation of VA’s Patient 

Safety Program 61 

Appendix III Semistructured Interview Questionnaire 63 

Appendix IV Comments from the Department of Veterans Affairs 66 

Appendix V GAO Contacts and Staff Acknowledgments 68 

GAO Contacts 68 

Staff Acknowledgments 68 

Glossary 69 

Tables 

Table 1: Familiarity with and Participation in the Patient Safety 

Program’s Initiatives at Four VA Facilities, 2003 18 

Table 2: Number of Root Cause Analyses at Four VA Facilities, 

Fiscal Years 2000–2003 23 

Table 3: Content Analysis: Achieving a Supportive Culture through 

Aspects of the Work Environment 32 

Table 4: Nonparametric Multiple Comparison Results 58 

Table 5: Intercoder Reliability Assessment Results 60 

Figures 

Figure 1: A VA Patient Safety Poster and Its Story 4 

Figure 2: Model of the Patient Safety Program at Four VA Medical 

Facilities 10 

Figure 3: Types of Adverse Event and Close Call Reporting at Four 

VA Facilities, June 2002 16 

Figure 4: Familiarity with and Participation in the Program by 

Facility 17 
Page iii GAO-05-83 VA Patient Safety Program 

Figure 5: Familiarity with VA’s Program Compared with Trust and 

Comfort in Reporting at Four Facilities 27 

Figure 6: Barriers to Staff Reporting Close Calls 29 

Abbreviations 

JCAHO Joint Commission on Accreditation of Healthcare 

 Organizations 

NASA National Aeronautics and Space Administration 

NCPS National Center for Patient Safety 

PSRS Patient Safety Reporting System 

RAP rapid assessment process 

RCA root cause analysis 

VA Department of Veterans Affairs 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
Page 1 GAO-05-83 VA Patient Safety Program 

December 15, 2004 

The Honorable Anthony J. Principi 

Secretary of Veterans Affairs 

Dear Mr. Secretary: 

This report on the Department of Veterans Affairs Patient Safety Program 

examines the Program’s status, the creation and implementation of a 

culture that supports close call and adverse event reporting, and practices 

that medical facility leaders have used to promote patient safety. In our 

study, we used ethnography, a social science method that includes 

qualitative and quantitative techniques developed within cultural 

anthropology for studying communities and organizations in natural 

settings. 

We include recommendations aimed at strengthening the Patient Safety 

Program by helping to build a more supportive culture and foster patient 

safety. 

We are sending copies of the report to appropriate congressional 

committees and others who are interested. We will also make copies 

available on request. If you have any questions about the report, please 

call me at (202) 512-2700. 

Sincerely yours, 

Nancy Kingsbury, Managing Director 

Applied Research and Methods 

United States Government Accountability Office

Washington, DC 20548 
Chapter 1: VA’s Patient Safety Program 

Page 2 GAO-05-83 VA Patient Safety Program 

At the end of the 20th century, a report that the Institute of Medicine 

issued estimated that up to 98,000 persons died each year from accidents 

in U.S. hospitals. Before the institute published this figure, the Department 

of Veterans Affairs (VA) had launched a Patient Safety Program that 

included teams investigating the root cause of medical close calls and 

adverse events and confidential staff reporting systems. The Program’s 

ultimate goal is to create a culture in which VA can discover and correct 

unsafe systems and processes before they harm patients. 

VA has indicated that it is attempting through the Patient Safety Program 

to introduce significant change in staff attitudes, beliefs, and behavior so 

that health care professionals will report events as part of their daily work. 

In testimony before the Congress in 2000, we suggested that the Program 

could be more successful if greater attention were paid to several 

leadership strategies the Institute of Medicine has outlined, such as 

making patient safety a more prominent goal and communicating the 

importance of patient safety to all staff.1

 In addition, we noted that 

“VA could also better ensure success if it prepared a detailed implementation plan that 

identifies how and when VA’s various patient safety Programs will be implemented, how 

they are aligned to support improved patient safety, and what contribution each Program 

can be expected to make toward the goal of improved patient safety.”2

One of the most challenging aspects of VA’s Patient Safety Program is 

creating an atmosphere in which employees are willing to reveal system 

problems and find system solutions to them. Traditionally, hospital 

employees have been held responsible for adverse patient outcomes, 

whether they stemmed from employees’ mistakes or the health care 

 1

Certain management practices are essential in creating safety within an organization and 

in the success of organizational change for improving patient safety: (1) balancing the 

tension between production efficiency and reliability (safety), (2) creating and sustaining 

trust throughout the organization, (3) actively managing the process of change, (4) 

involving workers in making decisions pertaining to work design and work flow, and (5) 

using knowledge management practices to establish the organization as a “learning 

organization.” (See Ann Page, ed., Keeping Patients Safe: Transforming the Work 

Environment of Nurses, Washington, D.C.: National Academies Press, 2004, pp. 3–4.) 

Throughout this report, we refer to the various patient safety initiatives under the National 

Center for Patient Safety (NCPS) as the Patient Safety Program, or the Program. The 

initiatives we studied included adverse event and close call reporting, root cause analysis 

(RCA), and the confidential reporting system to the National Aeronautics and Space 

Administration (NASA). 

2

GAO, Patient Safety Programs Promising but Continued Progress Requires Culture 

Change, GAO/T-HEHS-00-167 (Washington, D.C.: July 27, 2000), p. 3. 

Chapter 1: VA’s Patient Safety Program 
Chapter 1: VA’s Patient Safety Program 

Page 3 GAO-05-83 VA Patient Safety Program 

system. For example, a nurse might be blamed for administering the 

wrong medicine, even when the system was at fault, as when two 

medicines with similar names—one deadly, the other not—were stored on 

the same shelf in similar bottles. 

The poster and story in figure 1 show how complicated a day in the life of 

a healthcare provider can be. In this instance, a VA nurse recognized a 

potentially dangerous flaw in the system that could have caused 

unintentional harm to patients. In June 2002, she reported the close call, 

because she saw that the environment she worked in encouraged 

reporting, and she was then rewarded with a gift certificate. 
Chapter 1: VA’s Patient Safety Program 

Page 4 GAO-05-83 VA Patient Safety Program 

Figure 1: A VA Patient Safety Poster and Its Story 

High-risk industries such as nuclear power and aerospace have found that 

reliable safety organizations discover and correct system flaws. In 

effective safety cultures, frontline workers trust one another and report 

close calls and adverse events without fear of blame. Healthcare, which 

traditionally employs a culture of blame, must place a premium on 

Source: VA (poster). 

The Close Call Story behind the Poster:

We visited an intensive care unit and talked to the nurse 

who reported a close call of two look-alike drugs that 

were mixed together in the same drawer. She said she 

reached for liquid Tylenol and found potassium chloride 

concentrate also in the drawer. She told us that the two 

drugs were very different—one could kill you and the 

other is a mild analgesic. The two drugs were packaged 

similarly in containers with pull-off lids, and since the 

same drug company made both medications, the labels 

were similar. She told us she notified her supervisor and 

the pharmacy. Since the medical facility had a reward 

system for close calls, she received a gift certificate for 

the cafeteria, and later it was determined that this close 

call was the “pick of the month.” This meant that her unit 

received a plate of cookies. She said that she reported 

the close call not for the reward but because she is a 

professional. When one day a poster appeared in the hall 

alerting others to the two look-alike drugs, she wondered 

whether the other medical facilities were notified. She 

wondered whether she had made a difference in safety 

nationwide; a nurse rarely has that chance.
Chapter 1: VA’s Patient Safety Program 

Page 5 GAO-05-83 VA Patient Safety Program 

learning from staff reporting of adverse events and close calls.3 Experts in 

patient safety acknowledge that emphasis on culture is important in 

preventing medical adverse events and close calls and promoting patient 

safety.4

To describe the culture in VA’s medical facilities and to search for a 

deeper understanding of patient safety from the viewpoint of VA staff, we 

proposed to answer the following questions in the context of four VA 

medical facilities: 

1. What is the status of the Program’s implementation at four medical 

facilities? 

2. To what extent do the four sites we studied have a culture that 

supports the Program? What cultural changes can be stimulated? 

3. What practices in the four facilities promoted patient safety? 

To meet our study’s challenges, we used several methods from 

ethnography, and in certain cases we blended them with survey methods 

to provide in-depth knowledge of organizational culture from the 

perspective of VA’s frontline staff—its physicians, nurses, and others 

 3

See for example, Annick Carnino, “Management of Safety, Safety Culture and Self 

Assessment,” http://www.iaea.or.at/ns/nusafe/publish/papers/mng_safe.htm, (Feb. 19/2002); 

Columbia Accident Investigation Board, The CAIB Report, vol. 1 (Arlington, Va.: Aug. 26, 

2003). http://www.caib.us/ (Sept. 9, 2004) and Gaba, David “Structural and Organizational 

Issues in Patient Safety: A Comparison of Health Care to Other High-Hazard Industries,” 

California Management Review 43:1 (Fall 2000): 83-102.). A review of research on 

influences on collaboration also found that “mutual respect, understanding, and trust” 

appeared more often than any other factor to be a positive influence (see Paul Mattessich 

and others, Collaboration: What Makes It Work, 2nd ed. (St. Paul, Minn.: Amherst H. Wilder 

Foundation, 2001)). 

4

Highly effective safety organizations share the following characteristics: (1) 

acknowledgment of the high-risk, error-prone nature of the organization’s activities, (2) a 

blame-free environment in which individuals can report close calls without punishment, 

(3) an expectation of collaboration across ranks to seek solutions to vulnerabilities, (4) the 

organization’s willingness to direct resources toward addressing safety concerns, (5) 

communication founded on mutual trust, (6) shared perceptions of the importance of 

safety, and (7) confidence in the efficacy of preventive measures. (See M. D. Cooper, 

“Toward a Model of Safety Culture,” Safety Science 36 (2000): 111–36, and Lucian L. Leape 

and others, “Promoting Patient Safety by Preventing Medical Error,” JAMA 280:16 (Oct. 28, 

1988): 1444–47.) 

Scope and 

Methodology 
Chapter 1: VA’s Patient Safety Program 

Page 6 GAO-05-83 VA Patient Safety Program 

directly responsible for patient care.5 We intend this study to complement 

our earlier reports on organizational culture and changing organizations.6

We chose ethnography because several of its techniques and perspectives 

helped us study aspects of patient safety that would otherwise have 

remained overlooked or would not have been observed, such as informal 

mores, and to assist GAO in the development of new evaluation methods.7

These aspects were ethnography’s research traditions of (1) 

conversational interviews, enabling interviewers to explore a participant’s 

own view of and associations with an issue of interest, (2) the researchers’ 

observations of real processes to further understand the meaning behind 

patient safety from the natural environment of staff, and (3) systems 

thinking.8

 5

Ethnography is research carried out in a natural setting—such as a workplace—and using 

multiple types of data, both qualitative and qualitative. The approach embraces diverse 

elements that influence behavior. Most important, it recognizes that what people say, do, 

and believe reflect a shared culture—a set of beliefs and values—-that can be discovered 

by systematic study of their behavior. Ethnography produces a picture of social groups 

from their members’ viewpoint. (See Margaret D. LeCompte and Jean J. Schensul, 

Ethnographer’s Toolkit, vol. 1, Designing and Conducting Ethnographic Research 

(Lanham, Md.: Rowman & Littlefield, 1999).) Other ethnographers consider the 

multicultural image of organizations as leading to a consideration of culture’s cohesive, as 

well as divisive, functions. In this case, culture is defined as a learned way of coping with 

experience. Kathleen Gregory notes “More researchers have emphasized the homogeneity 

of culture and its cohesive functions.” However, she also describes a multicultural model 

that could be divisive in function among different occupational or ethnic groups. See 

Kathleen Gregory, “Native-View Paradigms: Multiple Cultures and Culture Conflicts in 

Organizations,” Administrative Science Quarterly 28 (1983): 359–76. 

6

GAO, Organizational Culture: Techniques Companies Use to Perpetuate or Change 

Beliefs and Values, GAO/NSIAD-92-105 (Washington, D.C.: Feb. 27, 1992); Weapons 

Acquisition: A Rare Opportunity for Lasting Change, GAO/NSIAD-93-15 (Washington, 

D.C.: Dec. 1, 1992); Managing in the New Millennium: Shaping a More Efficient and 

Effective Government for the 21st Century, GAO/T-OCG-00-9 (Washington, D.C.: Mar. 9, 

2000); Results-Oriented Cultures: Implementation Steps to Assist Mergers and 

Organizational Transformations, GAO-03-669 (Washington, D.C.: July 2, 2003); and HighPerforming Organizations: Metrics, Means, and Mechanisms for Achieving High 

Performance in the 21st Century Public Management Environment, GAO-04-343SP 

(Washington, D.C.: Feb. 13, 2004). 

7

One of the goals of the Center for Evaluation, Methods, and Issues in GAO’s Applied 

Research and Methods group is to find new tools for evaluation; one purpose in conducting 

this study was to see if ethnography was a practical tool for GAO to use in studying an 

organization’s culture. By statute, “[t]he Comptroller General shall develop and 

recommend to Congress ways to evaluate a program or activity the Government carries out 

under existing law.” See 31 U.S.C. §717(c) (2000). 

8

Regarding aspect no. 1, see James P. Spradley, The Ethnographic Interview (New York: 

Holt, Rinehart and Winston, 1997). 
Chapter 1: VA’s Patient Safety Program 

Page 7 GAO-05-83 VA Patient Safety Program 

Our study measures, at the facility level, the extent of familiarity with, 

participation in, and cultural support for the Program, and it complements 

a cultural survey VA conducted in 2000. VA expects to resurvey staff in the 

near future, using its past survey data as a baseline. VA’s original, 

nonrandom survey contained questions regarding shame and staff 

willingness to report adverse events when the safety of patients was at 

hazard during their care. The VA survey did not establish staff familiarity 

with key concepts of the Program, participation in VA safety activities, or 

the facilities’ levels of cultural support for the Program.9

We recognized that a tradition of fear of being blamed for adverse events 

and close calls might make staff reluctant to talk about their experience of 

potential harm to patients. Besides breaking through an emotional barrier, 

we wanted to understand the private views of staff on what facilitates 

patient safety. To achieve the informal, open, and honest discussions we 

needed, we conducted private, nonthreatening, conversational interviews 

with randomly selected clinicians and other staff in a judgmental sample. 

At each site, we chose one random and one judgmental (nonrandom) 

sample of staff to interview in a conversational manner, using similar 

semistructured questionnaires (see app. III). 

For the first sample, we interviewed a random selection of 10 physicians 

and 10 nurses at each of the four facilities. While this provided us with a 

representative sample of clinicians (physicians and nurses) from each 

facility, the sample size was too small to provide a statistical basis 

for generalizing our survey results to an entire facility. To give us a better 

understanding of the culture and context of patient safety beyond the 

clinicians involved in direct patient care at each facility, we also 

interviewed more than a hundred other staff in the four study sites, 

including medical facility leaders, Patient Safety Managers, and hospital 

employees from all levels—maintenance workers, security officers, 

nursing assistants, technicians, and service chiefs. (Appendix I contains 

more technical detail about our analysis.) 

 9

VA’s survey was a nonrandom survey sent to 6,000 clinicians; it provides a description of 

VA culture but not an adequate and reliable measure for generalizing at the facility level. 

Although NCPS asked each facility to use a random sample, NCPS staff acknowledged that 

in many cases this was not done. Furthermore, although the survey presented questions on 

cultural attitudes and beliefs, such as attitudes about punishment and shame for reporting 

adverse events, it did not address staff understanding of concepts such as close call 

reporting, root cause analyses (RCAs), confidential reporting systems, whether staff 

participated in RCA teams, or whether staff explicitly had mutual trust. 

Conversational Survey 

Interviews 
Chapter 1: VA’s Patient Safety Program 

Page 8 GAO-05-83 VA Patient Safety Program 

Reporting adverse events and close calls is a highly sensitive subject and 

can successfully be explored with qualitative methods that allow 

respondents to talk privately and freely. When staff did not recognize a key 

element of the Program, our interviewers explained it to them. (We were 

not giving the respondents a test they could fail.) Selecting clinicians 

randomly at each of four facilities, and asking some close-ended questions 

such as those expecting “yes” or “no” answers, allowed us to analyze and 

present some issues as standard survey data. This combined survey and 

ethnographic approach afforded us most of the advantages of standard 

surveys while establishing an environment in which the respondents could 

talk, and did talk, at length about the cultural context of patient safety in 

their own facilities. 

Clinicians responded to a standard set of questions, many open ended, 

such as, To what extent do you perceive there to be trust or distrust within 

your unit or team? Among the advantages these questions had were that 

they allowed the clinicians to discuss issues spontaneously and they 

allowed us to discover what facilitates trust from their point of view. Thus, 

if clinicians thought leadership was important, we had an opportunity to 

see this from their viewpoint rather than starting from the premise that 

leadership would be important to them. 

An important part of our approach was content analysis, which we used to 

analyze answers to both the standard and open-ended questions. Content 

analysis summarizes qualitative information by categorizing it and then 

systematically sorting and comparing the categories in order to develop 

themes and summarize them. We determined, by intercoder reliability 

tests, that our content analysis results were trustworthy across different 

raters. (See app. I.) 

We added another ethnographic technique in order to more completely 

understand the culture within each facility. Since responses to surveys are 

sometimes difficult to understand out of context, our in-depth 

ethnographic observations of the patient care process gave us a more 

complete picture of how the elements of the Patient Safety Program 

interacted. They also gave us a better understanding of VA’s medical 

facility systems. We observed staff in their daily work activities at each 

medical facility, which helped us understand patient safety in context. For 

example, we attended staff meetings where the Program was discussed 

and we attended RCA meetings, and we followed a nurse on her rounds. 

We took detailed field notes from our observations, and we analyzed and 

summarized our notes. 

Observation 
Chapter 1: VA’s Patient Safety Program 

Page 9 GAO-05-83 VA Patient Safety Program 

We reviewed files to examine data on adverse events, close calls, and RCA 

reports. We read files from administrative boards, reward programs, and 

patient safety committee minutes. And we interviewed high-level VA 

officials. 

Finally, our ethnographic research approach was systemic. This was to 

help us appreciate interactions between the elements of the Program and 

the facilities’ existing culture. Ethnography has traditionally been used to 

provide rich, holistic accounts of the way of life of a people or a 

community; in recent decades, it has also been used successfully to study 

groups in modern societies. A systems approach casts a wide net over the 

subject. In this case, we chose to study the Patient Safety Program in 

relation to other aspects of culture in VA’s medical facilities that might 

affect its adoption, such as the extent to which staff have mutual trust. 

We also developed a model, or flow chart, to guide our study of the 

Program and the culture of the facilities. The model, in figure 2, helped us 

conceptualize the important safety activities within the Program and 

analyze and present our results. We looked not only at the Program’s key 

elements, in the darkly shaded boxes in figure 2, but also at what 

surrounds them—the context of the medical facilities’ culture—and 

whether the culture supports the adoption of the Program. Our model 

illustrates that our primary focus was measuring clinicians’ supportive 

culture for reporting close calls and adverse events and their familiarity 

with and participation in reporting programs and RCAs. The model also 

depicts the interaction between clinicians’ receiving feedback and being 

rewarded and their desire to continue reporting close calls and adverse 

events. It also allows us to describe how clinicians’ reporting close calls 

and adverse events, and the subsequent investigation of the root causes of 

them, developed into system changes that in turn resulted in patients being 

safer. 

Systems Thinking 
Chapter 1: VA’s Patient Safety Program 

Page 10 GAO-05-83 VA Patient Safety Program 

Figure 2: Model of the Patient Safety Program at Four VA Medical Facilities 

We conducted the study at three medical facilities that VA had 

recommended as being well managed. We selected a fourth facility for 

geographic balance. Thus, the four facilities were in different regions of 

the country. Using rapid assessment techniques, we conducted fieldwork 

for approximately a week at each of two facilities, for 3 weeks at a third, 

and for 25 days at the fourth.10 We did our work from November 2002 to 

August 2004 in accordance with generally accepted government auditing 

standards. 

In 1998, in an influential editorial in the Journal of the American Medical 

Association, George Lundberg, the journal’s editor, along with Kenneth 

Kizer, then VA’s Under Secretary for Health, and other patient safety 

advocates and theorists, challenged the medical profession: 

“to make health care safe we need to redesign our systems to make error difficult to 

commit and create a culture in which the existence of risk is acknowledged and injury 

prevention is recognized as everyone’s responsibility. A new understanding of 

accountability that moves beyond blaming individuals when they make mistakes must be 

established if progress is to be made.”11

 10See James Beebe, Rapid Assessment Process (Lanham, Md.: Rowman & Littlefield, 2001). 

Before we began fieldwork, we also visited each facility and conducted numerous 

interviews for approximately 3 to 5 days in order to write our study protocol. 

11Leape and others, “Promoting Patient Safety by Preventing Medical Error,” p. 1444. 

Background 

The Patient Safety Goal 

Source: GAO. 

Supportive culture includes mutual trust and comfort in reporting

GAO focused on these safety activities

Program Reporting

Feedback/rewards

Modify program

System changes Safer patients

Evaluation

Root cause 

analysis
Chapter 1: VA’s Patient Safety Program 

Page 11 GAO-05-83 VA Patient Safety Program 

This vision of making patients safe through “redesign . . . to make errors 

difficult to commit” led to VA’s National Center for Patient Safety (NCPS), 

established to improve patient safety throughout the largest health care 

system in the United States.12 To transform the existing culture of patient 

care in VA’s medical facilities, VA’s leaders aimed to persuade clinicians 

and other staff in health care settings to adopt a new practice of reporting, 

free of fear and with mutual trust, identifying vulnerabilities, and taking 

necessary actions to mitigate risks. 

The Under Secretary had recognized risk to patients during care and that a 

focus on VA’s existing culture could improve patient safety. Related 

research shows that if complex decision making organizations are to 

change, they must modify their organizational culture.13 Traditionally, 

clinicians involved in an adverse event could be blamed or sued, but the 

roots of unintentional errors are now understood as originating often in 

the institutions and structures of medicine rather than in clinicians’ 

incompetence or negligence.14

Several contextual factors influence how the Patient Safety Program is 

experienced at the medical facilities we visited and show the increasingly 

complex world of patient care. Our study’s limitations meant that we could 

not study these factors, but health care facilities in general, as well as VA’s, 

are experiencing difficulty in hiring and retaining nurses, as well as 

potential staffing shortages. Patients admitted to VA medical facilities have 

more multiple medical problems that require more extensive care than in 

the past. VA’s eligibility reform allowed veterans without serviceconnected conditions to seek VA services, leading to a 70 percent increase 

in the number of enrolled veterans between 1999 and 2002. 

VA has provided funding of $37.4 million to NCPS for its Patient Safety 

Program operations and related grants and contracts for fiscal years 

 12VA’s health care system plays an important role in teaching physicians and nurses. It has 

193,000 full-time-equivalent employees. The 158 medical facilities are organized into 21 

regional networks. 

13GAO/NSIAD-92-105. 

14David M. Gaba, “Structural and Organizational Issues in Patient Safety: A Comparison of 

Health Care to Other High-Hazard Industries,” California Management Review 43 (2000): 

83–102. 

The Patient Safety Process 
Chapter 1: VA’s Patient Safety Program 

Page 12 GAO-05-83 VA Patient Safety Program 

1999–2004.15 In fiscal year 1999, NCPS defined three major initiatives: (1) a 

more focused system for mandatory close call and adverse event 

reporting, including a renewed focus on close calls; (2) reviews of close 

calls and adverse events, including RCAs, using interdisciplinary teams at 

each facility to discover system flaws and recommends redesign to 

prevent harm to patients; and (3) staff feedback on system changes and 

communication about improvements to patient safety.16

Starting with the NCPS program in 1999, reporting of close calls increased 

dramatically as their value for patient safety improvement was widely 

disseminated and increasingly recognized by VA personnel. A close call is 

an event or situation that could have resulted in harm to a patient but did 

not, either by chance or by timely intervention. VA encourages reporting 

close calls and adverse events, since redesigning system flaws depends on 

staff revealing them.17 VA’s Patient Safety Managers told us that only 

adverse events and not close calls were traditionally required to be 

reported to supervisors and then up the chain of command. 

Under the Program, staff also have optional routes for reporting—through 

Patient Safety Managers or a confidential system outside their facilities. 

Staff can now report close calls and adverse events directly to the 

facilities’ Patient Safety Managers. They, in turn, evaluate the reports, 

based on criteria for deciding which adverse events or close calls should 

be investigated further. NCPS also has a confidential reporting option—

the Patient Safety Reporting System (PSRS)—through a contract with the 

National Aeronautics and Space Administration (NASA). NASA has 

27 years of experience with a similar program, the Federal Aviation 

Administration’s Aviation Safety Reporting System. Under the contract 

with VA, NASA removes all identifying information and sends selected 

 15For fiscal year 2004, information was collected through August 4. 

16Efforts under NCPS that we did not study included prospective analysis of potential 

problems (such as reviewing contingency plans for failure of the electronic bar code 

medication administration system), safety protocols focused on surgery, and a system of 

technical alerts to warn clinicians of malfunctioning mechanical equipment. 

17The Patient Safety Program does not replace VA’s existing accountability systems, which 

include VA internal review boards, compromise or settlement of monetary claims, and 

referring possible criminal cases to the Department of Justice. See 38 C.F.R. §§14.560, 

14.561, 14.600 (2004). If an RCA team determines that a crime is suspected or has been 

committed, it initiates the review process by referring the matter to the facility director. 

Similarly, questions involving quality of performance are handled outside the Program. 

Close Call and Adverse 

event Reporting 
Chapter 1: VA’s Patient Safety Program 

Page 13 GAO-05-83 VA Patient Safety Program 

items of special interest to the NCPS. NASA also publishes a newsletter 

based on reports that have had their identifying information removed. 

Working on interdisciplinary teams of usually five to seven participants, 

staff focus on either one or a group of similar close calls or adverse events 

to investigate their causes. Then they search for system flaws and redesign 

patient care so that mistakes are harder to make. Under the Program, 

NCPS envisioned that these teams would be a key step to improving 

patient safety through system change and one of its primary mechanisms 

of introducing clinicians to the Program.18 In 1999, NCPS began RCA 

implementation.19 In this on-the-job training, Patient Safety Managers guide 

local interdisciplinary teams in studying reports of close calls or adverse 

events to identify and redesign system weaknesses that threaten patients’ 

safety. Teams are allowed 45 days to learn as much as possible from a 

close call or adverse event or a group of similar close calls or adverse 

events such as falls, missing persons, medication errors, and suicides 

called aggregated reviews. Within the given time period, teams are to 

develop action plans for system improvement. Personal experience on 

interdisciplinary RCA teams investigating close calls and adverse events at 

their home facilities is the clinicians’ key training experience. VA expected 

that the RCA experience would persuade staff that VA was changing its 

culture by encouraging a different approach to reporting. 

Staff need to receive proof that the Program is working by receiving timely 

feedback on their reporting. A feedback loop fosters and perpetuates close 

call and adverse event reporting.20 Without it, staff may feel the effort is 

not worth their time. NCPS built in feedback loops at several levels of the 

system. For example, individuals who report a close call or adverse event 

are supposed to get feedback from the RCA team on actions 

recommended as a result of their reports. Also, NCPS issues an online 

bimonthly newsletter that reports safety changes. 

 18All RCA material and findings are part of VA’s medical quality-assurance program. 

Records developed under the program are confidential, privileged, and subject to limited 

disclosure. See 38 U.S.C. §5705 (2000). 

19Only reported adverse events and close calls that meet certain criteria of seriousness and 

frequency are examined in RCAs. 

20John, Corrigan, and Donaldson, eds., To Err Is Human, p. 99. 

Root Cause Analysis 

Teams 

Feedback Mechanisms 
Chapter 1: VA’s Patient Safety Program 

Page 14 GAO-05-83 VA Patient Safety Program 

In chapter 2, we measure clinicians' familiarity and participation in the 

Program at the four facilities we visited. Chapter 3 is an examination of 

whether the culture at the four facilities supports the Patient Safety 

Program and chapter 4 provides examples of management practices that 

promote patient safety. We asked VA to comment on our report; VA’s 

comments are in appendix IV. Our response to their comments is in the 

conclusions located in chapter 5. VA also provided some additional 

comments to emphasize that it believes that it has taken steps to address 

the issue of mutual trust. VA describes those steps in the report on 

page 67. 
Chapter 2: Progress in Clinicians’ Familiarity 

with and Participation in the Program 

Page 15 GAO-05-83 VA Patient Safety Program 

In general, we found progress in clinicians’ understanding and 

participation in the Patient Safety Program. Three facilities had medium or 

higher familiarity with and participation in the Program’s core elements, 

and one had lower. At that facility, the staff were not following VA’s policy 

of reporting close calls and were not being educated in the benefits of 

doing so. Examining the data across our total random sample, we found 

that some clinicians were familiar with several core concepts of the 

Program and were unfamiliar with others—a picture NCPS officials said 

did not surprise them. 

About three-quarters of clinicians were familiar with the concept of RCAs 

(newly introduced in 2000) and the concept of the close call. About half 

the clinicians recognized the new confidential reporting process—another 

equally important program. One-third had participated in an RCA or knew 

someone who had. NCPS staff told us that participation in RCAs is crucial 

to culture change at VA, and clinicians who were on RCA teams indicated 

that they experienced the beginning of a culture shift.1

 Of the staff who 

had participated in RCAs, many indicated that it was a positive learning 

experience, but facilities varied in ensuring clinicians’ broad participation. 

VA has made progress in familiarizing and involving clinicians with the 

Program’s key concepts. But while the facilities we studied shared basic 

safety problems, three had made more progress than the fourth. First, all 

four experienced similar hazards to patient safety. Second, we report 

clinicians’ familiarity with and participation in the Program in two ways—

grouped first by facility and then across the four sites. 

The four facilities shared an overall pattern in the types of adverse events 

they reported, reflecting their common safety challenge. To establish the 

Program’s context, we asked at the four facilities to review documents 

related to close calls and adverse events reported over a one-month period 

(June 2002). All the facilities reported falls for this period, while two 

facilities or more recorded patients’ violence toward staff, patients’ 

suicides and suicide attempts, missing patients, and medication errors (see 

 1

For more on NCPS and its implementation of the Program, see the timeline in appendix II. 

Chapter 2: Progress in Clinicians’ Familiarity 

with and Participation in the Program 

Facilities Shared 

Safety Hazards but 

Not Program 

Familiarity and 

Participation 

Facilities’ Share Common 

Safety Reporting Pattern 
Chapter 2: Progress in Clinicians’ Familiarity 

with and Participation in the Program 

Page 16 GAO-05-83 VA Patient Safety Program 

fig. 3).2 Although our data reflect a limited time period, the highly 

overlapping types of reporting at the facilities parallel those found in the 

wider VA patient care system, as documented in an earlier review by the 

VA Medical Inspector.3

Figure 3: Types of Adverse Event and Close Call Reporting at Four VA Facilities, 

June 2002 

Note: Excludes reports in pharmacies, laboratories, and other areas of VA facilities that had separate 

reporting systems. Facilities with suicides not reported for June 2002 may have had suicides reported 

in other months. 

Staff at one facility had less familiarity with and participation in the 

Program than staff at the three others (see fig. 4).4

 In the interviews with 

the random sample, we found Facility D had lower familiarity with the 

Program’s concepts than the other facilities and lower participation in 

RCAs; this pattern was buttressed by additional interviews at Facility D. 

For example, the quality manager who supervised Patient Safety Managers 

at that facility did not realize that close call reporting was mandated, and 

the education officer who trained staff in patient safety told us that staff 

 2

Missing patients includes patients who have a pass to leave their unit and have not 

returned on time, as well as patients who leave without a pass. 

3

VA Office of Medical Inspector, VA Patient Safety Event Registry: First Nineteen Months 

of Reported Cases Summary and Analysis (Washington, D.C.: June 1997–Dec. 1998), p. 12. 

4

To measure how familiar the staff were with the Program’s core concepts, we calculated 

the average familiarity, grouped by facility, by combining answers for the series of 

questions noted in figure 4. More information about our methods is in appendix I; our 

questionnaire is in appendix III. 

Facilities’ Differences in 

Participation and 

Familiarity with the 

Program 

Source: GAO. 

VA medical facility 

Reported

Not reported

Reporting type A

Falls

Close call

Patient violence to staff

Medication error

Missing patient

Suicide or suicide attempt

BCD
Chapter 2: Progress in Clinicians’ Familiarity 

with and Participation in the Program 

Page 17 GAO-05-83 VA Patient Safety Program 

were generally not acquainted with the concept of reporting close calls. 

Because knowing that an initiative exists is often the first step to 

participation, the lower familiarity with the Program at Facility D in the 

fifth year of implementation was a likely impediment to the adoption of 

the Program there. 

Figure 4: Familiarity with and Participation in the Program by Facility 

Note: A summary code we created for each facility reflected a composite score for answers to five 

questions about familiarity with the key elements of and participation in RCAs: Do you know what a 

close call is? Do you know what the Patient Safety Reporting System is? Do you know what an RCA 

is? Have you participated in an RCA? Do you know anyone who has participated? Coders analyzed 

all answers for each individual random sample respondent with regard to expressions of mutual trust 

and comfort in reporting. Then they created a summary value rating of low, medium, or high for each 

individual. This summary rating was then tested through rater reliability, and the scores were 

determined acceptable. Individual summary ratings were averaged for each facility. In each key 

elements question, we let “yes” equal 2 and “no” equal 0, ensuring that an individual who knew each 

of the five elements would achieve a composite score of 10. Finally, we averaged composite scores 

to get an average score for each facility. Rather than display these numbers, we used a scale of high, 

medium, and low for 10, 5, and 0 and placed the answers accordingly. (Appendix I describes our 

methodology; appendix III reprints our questionnaire.) 

The four medical facilities we studied also varied in their adherence to 

close call reporting policies under the Program. We found three out of four 

facilities followed the policy of reporting close calls. One facility, in 

particular, showed a marked increase in the number of close calls in a 

short period of time; close call reports were rare in the 6 months before 

Differences in Facilities’ 

Adhering to Close Call 

Reporting Policy 

A B C D

Medical facilities

Source: GAO.

Low

Medium

High
Chapter 2: Progress in Clinicians’ Familiarity 

with and Participation in the Program 

Page 18 GAO-05-83 VA Patient Safety Program 

but numbered 240 in the 6 months after its leaders told staff patient safety 

was an organizational priority and introduced a simple reward system for 

close call reporting. However, one facility we visited was not reporting 

close calls in the Program’s fifth year. 

We looked at interview responses with randomly selected clinicians across 

all four facilities. We found that three-quarters of the clinicians knew the 

meaning of close call—that is, when a potential incident is discovered 

before any harm has come to a patient—but only half were aware of the 

option of reporting close calls and adverse events confidentially. (See 

table 1.) Close calls are presumed to occur more often than adverse 

events, and reporting them in addition to adverse events is central to the 

Program’s goal of discovering and correcting system flaws. Staff who do 

not recognize the close call concept cannot bring to light system flaws that 

could harm patients. Further, because changing from traditional blaming 

behavior to reporting without fear can take time, staff familiarity with the 

confidential reporting option is important. However, only half the 

clinicians surveyed at the four facilities knew that they could report 

adverse events or close calls confidentially under the NASA reporting 

contract. 

Table 1: Familiarity with and Participation in the Patient Safety Program’s Initiatives 

at Four VA Facilities, 2003 

Program 

Percentage 

of staff Indicator 

Root cause analysis 78 Familiar with the concept 

 35 Had participated 

 43 Knew someone who had 

participated 

Close call 75 Familiar with the concept 

Confidential report to NASA 51 Familiar with the program 

Source: GAO analysis. 

Note: Data, rounded to the nearest whole number, are from our interviews with 81 randomly selected 

VA physicians and nurses. If staff initially did not know of a concept, we explained it to them. If they 

then recognized it, we accepted their answer as “yes.” Therefore, when we state that they are familiar 

with it, this means they either knew the definition or recognized the term after an explanation. 

Clinicians who had participated in interdisciplinary RCA teams found that 

their participation enabled them to understand the benefits of using a 

systems approach rather than blaming individuals for unintentional 

adverse events and close calls. To understand the RCA process from close 

Familiarity with and 

Participation in the 

Program across Four 

Facilities 

Culture Shift through Root 

Cause Analysis 
Chapter 2: Progress in Clinicians’ Familiarity 

with and Participation in the Program 

Page 19 GAO-05-83 VA Patient Safety Program 

call reporting to RCA team analysis, we provide an example from 

fieldwork that shows how two misidentifications in a surgery ward led to a 

reexamination of the preoperative process in an RCA. (See “Developing 

Patient Safety from Examining Close Calls and an RCA.”) 

While examining how many RCAs were conducted from 2000 to 2003 at 

the four facilities, we found that the most active facility we studied had 

performed twice as many RCAs as the least active. The RCAs have the 

potential to promote a cultural shift from blaming staff for unintentional 

close calls and adverse events to a rational search for the root causes, but 

clinicians at the four facilities had inconsistent opportunities to participate 

in the Program. 

“Developing Patient Safety from Examining Close Calls and an RCA” 

illustrates an RCA team’s initial steps by following a series of events 

involving two close calls of mistaken identity in surgery at one facility. 

Developing Patient Safety from Examining Close Calls and an 

RCA 

The Patient Safety Manager had an unusual visit from the Chief Surgeon. 

He had come to report two recent instances of patients being mistakenly 

scheduled for surgery. The identity mix-ups had been discovered before 

the patients were harmed—a situation the surgeon recognized as fitting 

the Program’s mandate to report close calls in order to identify hazards in 

the system. After each close call, he had filled out a form and made a 

report to NCPS, which had called him back within 24 hours to ask for 

more information and to offer some reengineering suggestions. 

At the next weekly surgery preoperation meeting, the Chief Surgeon and 

his staff discussed their schedule and details of coming surgeries, using a 

matrix timetable projected for all to see. Then he discussed the two close 

calls. In both cases, the correct patient had come to the surgery 

preparation room, but the staff had been expecting someone else. In one 

case, the scheduling staff had confused two similar names. In the other 

case, the scheduling staff had, as usual, used the last four digits of the 

Social Security number to help identify the patient but had had two 

patients with the same last four digits. In the meeting’s discussion, the 

staff tried to understand how such mistakes could happen. 

Illustrating the Steps from 

Close Calls to RCAs 
Chapter 2: Progress in Clinicians’ Familiarity 

with and Participation in the Program 

Page 20 GAO-05-83 VA Patient Safety Program 

The Patient Safety Manager convened an expedited RCA team of three 

other VA staff to get at the root cause of such identification problems. 

She opened the meeting by saying, “If we don’t learn from this [close 

call], we’re all fools.” She announced that the RCA would be limited to 

two or three meetings rather than several weeks. After introductions, the 

staff members explained their role in scheduling and what happened in 

such cases. As they spoke, the staff tried to outline the scheduling 

process: what forms were completed, whether they were electronic or 

paper, how they moved from person to person, and who touched the 

forms. 

Several problems emerged. (1) Some VA patients might not always know 

their identity or surgical site because of illness or senility or both. Also, 

patients with multiple problems cannot always relay them to staff, 

because they may focus on one problem while the appointment 

scheduled is for another problem. (2) Two key VA staff may be absent at 

the same time and a substitute may make the error. (3) In one case, two 

patients’ names differed only by m and n. (4) A scheduler noted that 

scheduling is filled with interruptions and opportunities for confusion. 

For example, it is not uncommon that scheduled patients have 

overlapping numerals for the last four digits of their Social Security 

numbers. 

The RCA team’s next meeting was scheduled. In future meetings, the 

RCA team would consider various ways of preventing or minimizing 

similar events. 

Staff who had participated in RCAs told us that their experience was a 

valuable and convincing introduction to the Patient Safety Program. In lieu 

of giving clinicians formal training in the central concepts of the Program, 

NCPS expected to change the culture of patient care one clinician at a 

time by their individual experience in RCAs. NCPS intended that 

experience on multidisciplinary RCA teams investigating the underlying 

causes of reported close calls and adverse events at their home facilities 

would be clinicians’ key educational experience and that it would 

persuade them that VA was taking a different approach to reporting. All 

facilities are expected to perform RCAs, in which local interdisciplinary 

Clinicians’ Belief in RCAs 

as a Positive Learning 

Experience 
Chapter 2: Progress in Clinicians’ Familiarity 

with and Participation in the Program 

Page 21 GAO-05-83 VA Patient Safety Program 

teams study reports of close calls and adverse events in order to identify 

and redesign systems that threaten patients’ safety. 

Staff also reported that RCA investigations created a learning environment 

and were an excellent way to introduce staff to redesign systems to 

prevent harm to patients. Two doctors at one facility, for example, told us 

that the RCAs they participated in were a genuine “no blame learning 

experience” that they felt good about or found valuable. Two nurses at 

another facility reported being amazed at the change from a blaming 

culture to an inquiring culture as they experienced the RCA process. 

However, staff also told us that the RCA process took too much time or 

took time away from patient care. At another facility, where trust was low 

and only 5 of 20 clinicians had a positive view of reporting, each of those 

5 clinicians had a positive experience with RCAs under the new Program. 

“How Participating in RCAs Affects Clinicians’ Work” presents some 

clinicians’ own stories of their participation in RCAs. 
Chapter 2: Progress in Clinicians’ Familiarity 

with and Participation in the Program 

Page 22 GAO-05-83 VA Patient Safety Program 

How Participating in RCAs Affects Clinicians’ Work 

Physician 1: I participated in an RCA through my work in the blood 

bank. It taught me to look at errors systematically and not rush to blame 

individuals. But if an employee were eventually found responsible, then 

the Lab would hold that person accountable. [This example reflects the 

decision leaders must make between personal accountability and 

systemic change.] 

Physician 2: RCAs are a good thing. It’s fixing a potential disaster before 

it can coalesce and become a disaster. 

Nurse 1: I think RCAs are a good thing, because usually the problems are 

system problems. I think if you fix the system, you fix the problem. It 

seems to be that way in surgery. You try and concentrate on the things 

you can fix. 

Nurse 2: They used to have a process in psychiatry called “post mortem.” 

That process often led to the conclusion that a suicide could not have 

been prevented. By contrast, in the new RCA process, we look at how 

the RCA can promote system changes. 

Nurse 3: RCA does a good job of identifying not only the actual adverse 

event but also the contributing factors. This is very helpful because it 

allows us to better understand what to do about an adverse event. 

Nurse 4: RCA is a good system. It’s a good way to share information and 

avoid recurring error. 

Nurse 5: My general impression is that RCAs are great. They’re especially 

important when teams look for results and action items. 

Over the 4 years of the RCA implementation, the most active facility we 

studied (Facility A) had performed twice as many RCAs as the least active 

facility (Facility D). (See table 2.) The number of RCAs, similar to the 

number of close calls and adverse events, does not reflect the actual 

numbers of adverse events or close calls that occurred or how safe the 

facility is; rather, it reflects whether organizational learning is taking place, 

through increasing participation in a core Program activity. Similarly, 

NCPS staff recently reported to a facility leaders’ training session that 

Variation in Facilities’ RCA 

Activity 
Chapter 2: Progress in Clinicians’ Familiarity 

with and Participation in the Program 

Page 23 GAO-05-83 VA Patient Safety Program 

networks of their facilities varied fourfold in fiscal year 2002 with respect 

to number of RCAs conducted. Facility D’s director told us that NCPS had 

recently identified his facility as having too few RCA reviews. 

Table 2: Number of Root Cause Analyses at Four VA Facilities, Fiscal Years 

2000–2003 

Fiscal year Facility A Facility B Facility C Facility D

2000 10 9 8 1

2001 20 14 11 9

2002 13 9 8 5

2003 11 6 7 8

Total 54 38 34 23

Source: GAO analysis. 

Note: Includes only individual RCAs; excludes aggregate reviews. In 2002, VA began a program of 

aggregate RCAs, in which the most commonly reported events, such as patient falls, were grouped 

and analyzed quarterly. Thus, in 2003 we see a reduction in individual RCAs across these facilities. 

One facility was more successful than the three others at providing busy 

physicians with the opportunity to participate in RCA teams by adopting a 

mandatory rotation system. 

RCAs have been required under the Program since 2000. About threefourths of the respondents were familiar with the RCA concept. Seventyfive percent staff familiarity represents substantial learning, given when 

the concept was introduced. However, only about a third had participated 

in an RCA or knew someone who had. At one facility, we found broad 

participation by physicians because management required it. NCPS 

envisions RCA experience as central to changing to a culture of safety, but 

many VA clinicians (approximately 65 percent) at the facilities we studied 

had yet to participate in the nonblaming process that NCPS’s director told 

us he viewed as the most effective experience for culture change: “We 

don’t want professional root cause analysis people doing this stuff. Then 

you don’t change the culture.” 

We found a wide spectrum of methods being used to recruit physicians 

into RCA teams. One facility had broad physician participation in RCAs as 

its policy, and at another facility one unit had a rotational plan that 

encouraged its own clinicians to participate, in contrast to the whole 

facility. Administrators at three of the four had no policy across the facility 

to ensure physician participation on the teams. At two facilities, Patient 

Safety Managers told us it was difficult to get physicians to participate 

Inconsistent Opportunities 

to Participate in RCAs 
Chapter 2: Progress in Clinicians’ Familiarity 

with and Participation in the Program 

Page 24 GAO-05-83 VA Patient Safety Program 

because of their busy schedules. Understandably, most of the clinicians we 

surveyed had not served on RCA teams. 

We found progress but also variation in the range of clinicians’ familiarity 

with and participation in key elements of the Program. Looking facility by 

facility, we found one of the four facilities had lower familiarity and 

participation in the Program. Examining the clinicians across the random 

sample, we also found that about three-fourths were familiar with close 

call reporting but only half were familiar with a confidential reporting 

system. Focusing on RCAs, we found that about three-quarters of the 

sample knew the concept—that is, staff teams investigate the causes for 

accidents—while one-third had participated. Most of those who had 

participated thought that RCAs were promoting a culture shift by 

investigating adverse events and close calls in a no-blame atmosphere and 

redesigning systems so that future problems could be prevented. 

Summary 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 25 GAO-05-83 VA Patient Safety Program 

Cultural support for VA’s Patient Safety Program varied at the four 

facilities we studied. While clinicians we surveyed at three facilities had a 

more supportive cultural foundation for the Program, significantly lower 

levels of mutual trust and comfort in reporting limited the adoption of core 

Program activities at the fourth facility. Further, our analysis indicated 

that low trust and fear of punishment that characterize an unsupportive 

culture limit the adoption of the Program and constitute a feature held by 

clinicians that does not necessarily improve when they become familiar 

with the key concepts in the Program.1

The clinicians identified barriers to their participation in the Program. 

However, they fundamentally agreed on workplace conditions that can 

build the supportive culture and foster patients’ safety. Their most 

frequently articulated themes for building supportive culture were 

(1) effective leadership; (2) good two-way communication, including 

feedback on reports of adverse events and close calls; (3) their 

professional values; and (4) workflow.2

Clinicians at three of the four facilities had medium or higher cultural 

support for the Program. One facility had lower support, and many 

clinicians indicated that they would not report adverse events because 

they feared punishment.3

 This suggests that the Program will not succeed 

unless cultural support is bolstered. We explored the cultural support from 

these four groups in two ways: (1) by graphically comparing the groups’ 

levels of mutual trust and comfort in reporting close call and adverse 

events with their levels of familiarity with the Program and (2) by 

graphically demonstrating the barriers clinicians see as blocking their 

close call and adverse event reporting, in conjunction with some elements 

of basic familiarity with and cultural support for the Program. 

 1

We studied the attitudes, beliefs, and behavior of clinicians directly involved in patient 

care. Ethnographic studies of U.S. hospital workers other than clinicians reveal their 

unique perspectives. See, for example, Karen Brodkin Sacks and Dorothy Ramey, My 

Troubles Are Going to Have Trouble with Me (Brunswick, N.J.: Rutgers University Press, 

1984), and Karen Brodkin Sacks, Caring by the Hour: Women, Work, and Organizing at 

Duke Medical Center (Chicago: University of Illinois Press, 1988). 

2

For our purposes, workflow refers to the coordination of tasks within and across teams, 

and professional values refers to norms that are learned from formal and informal training 

and that are reinforced on the job. 

3

Cultural support is a composite measure of levels of mutual trust and comfort in reporting 

close calls and adverse events for each of four groups of clinicians. 

Chapter 3: Measuring Cultural Support for 

the Program 

Varying Cultural 

Support 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 26 GAO-05-83 VA Patient Safety Program 

In figure 5, we compare our findings on clinicians’ mutual trust and their 

comfort in reporting close calls and adverse events at the four facilities. 

The levels of these components of a supportive culture appeared to vary 

among the clinician groups.4

 For example, staff at Facility A had medium 

familiarity with the Program but had the lowest levels of comfort in 

reporting adverse events and close calls and mutual trust among the four 

facilities. Knowledge from specific safety training or RCA participation 

was not sufficient for them to readily change to safety practices under the 

Program if levels of comfort in reporting and mutual trust were not high 

enough. Figure 5 contrasts information on the supportive culture (mutual 

trust and comfort in reporting) with a measure of staff familiarity with the 

Program from figure 4. 

 4

In chapter 2, we described a scale of low, medium, and high familiarity with the Program 

that combined the answers to the following questions: Do you know what a close call is? 

Do you know what the Patient Safety Reporting System is? Do you know what an RCA is? 

Have you participated in an RCA? Do you know anyone who has participated? 

Clinicians’ Trust and 

Comfort in Reporting 

Varies by Facility 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 27 GAO-05-83 VA Patient Safety Program 

Figure 5: Familiarity with VA’s Program Compared with Trust and Comfort in 

Reporting at Four Facilities 

Note: We reviewed all coded expressions of mutual trust and comfort in reporting for each interview in 

the random sample, assessing the preponderance of expressions and creating a summary high, 

medium, or low value for each individual. Intercoder reliability testing found coding consistency 

acceptable. We averaged these scores for each facility. Finally, we created a summary code for each 

facility, reflecting a composite score, using five questions about familiarity with the key elements and 

participation in RCAs. Coders analyzed all answers for each individual random sample respondent 

with regard to expressions of mutual trust and comfort in reporting and then created a summary rating 

of low, medium, or high values for each individual. This summary rating was then tested through rater 

reliability, and the scores were determined acceptable. For each facility, the individual summary 

ratings were averaged. 

We assigned numeric values, as customary in quantifying verbal answers. For display and 

comparison purposes, we decided to let the maximum individual knowledge, trust, and comfort levels 

be 10. Thus, in each key elements question, we let “yes” equal 2 and “no” equal 0, ensuring that an 

individual who knew all of the five elements would achieve a composite score of 10. Finally, we 

averaged composite scores to get an average score for each facility. In the trust and comfort 

summary judgments, we let “high” equal 10, “middle” equal 5, and “low” equal 0. Rather than display 

these numbers, we used a scale of high, medium, and low for 10, 5, and 0 and placed their answers 

accordingly. (See app. I for more on our methodology.) 

Familiarity with Program

Mutual trust

Comfort in reporting

A B C D

Medical facilities

Source: GAO.

Low

Medium

High
Chapter 3: Measuring Cultural Support for 

the Program 

Page 28 GAO-05-83 VA Patient Safety Program 

Many staff at Facility A were afraid of being punished, and they mistrusted 

management and other work units. One staff member explained why staff 

would not report adverse events: “We have a culture of back-stabbing 

here. They are always covering themselves.” Many other staff members 

echoed this characterization of the atmosphere, linking the lack of cultural 

support to their decision not to perform the most basic of the Program’s 

activities. Staff at that facility needed a boost in supportive culture to fully 

implement the Program. In contrast, Facility D, with the least familiarity 

with the Program, had trust and comfort levels almost as high as any of the 

others, indicating that if the Program were to be pursued with greater 

vigor there, cultural support would not be a barrier to reporting close calls 

and adverse events. 

In interviewing clinicians, we found that barriers remain to reporting 

adverse events and close calls. Even for staff familiar with the concepts, 

reporting required overcoming numerous remaining obstacles. These staff 

indicated that reporting formally would be a time-consuming diversion 

from patient care or, worse, “an invitation to a witch hunt.” In figure 6, we 

display the cumulative effect of the barriers to reporting close calls that 

staff told us about, in conjunction with familiarity with and cultural 

support for the Program. 

Barriers to Reporting 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 29 GAO-05-83 VA Patient Safety Program 

Figure 6: Barriers to Staff Reporting Close Calls 

Note: We asked VA staff “Do you know what a close call is?” If they answered “No,” we explained it to 

them; if they recognized the concept, we accepted their answer as “Yes.” 

Clinicians told us about barriers to their participation in reporting, 

including (1) limited perceived value, (2) not knowing how to report, 

(3) not having enough time to report, (4) fearing traditional blame or 

punishment, (5) lacking trust that coworkers would not shame them, and 

(6) lacking knowledge of the confidential reporting option. Staff at all four 

sites reported such barriers in reporting both close calls and adverse 

events. We present some of their views in “Clinicians’ Barriers to 

Reporting Close Calls and Adverse Events.” 

Source: GAO. 

Limited 

perceived value

Unfamiliar 

with the 

confidential 

reporting Fear of system being 

blamed

Fear of 

being 

Not shamed

enough 

Do not time

know how

25% unfamiliar

with close calls

75% familiar 

with close calls
Chapter 3: Measuring Cultural Support for 

the Program 

Page 30 GAO-05-83 VA Patient Safety Program 

Clinicians’ Barriers to Reporting Close Calls and Adverse Events 

Nurse 1: Some clinicians feel comfortable reporting adverse events and 

close calls. I agree with the concept. It depends on the person. Some 

would feel it would be used against them. I’ve seen nonreporting, 

because, before, they got written comments such as “This is not a near 

miss.” “This is not a close call.” We get shut down instead of worked 

with. [By “shut down,” she meant that management told her it was not a 

close call and not to report it.] It happened to me. Management generally 

discourages and does not empower staff to feel comfortable reporting 

patient safety conditions. Instead, I reported and it was used against me. 

Physician 1: I can’t remember if I’ve written a close call. That does not 

happen here—only very, very rarely. Maybe I wrote one early on in my 

career, but I’m not sure. 

Physician 2: I thought I had a close call once and showed it to the chief 

of staff and he told me that it was not a close call. I’m unclear what the 

definition of a close call is. 

Physician 3: I know what a close call is in other settings, but not in the 

hospital setting. [Interviewer explains the definition.] They are not 

reporting on close calls in this hospital. 

Physician 4: Yes, I know what a close call is. I’ve not reported a close 

call, but if I were to, I would go to a nurse supervisor and tell her about it 

orally and have her report it. I would not use incident reports to report a 

close call—only actual events. 

Physician 5: I have not reported a close call. I’m removed from the 

nursing communications. 

Physician 6: I’m unsure if it is safe to report close calls without 

punishment. 

Nurse 2: If I saw a close call, I would go talk to the nurse who did it. 

Writing up a close call on someone would be cruel. I would not write up 

a close call or adverse event report on someone else. If something 

happened to the patient, I would write it up. Writing up another person 

would cause conflict. We need to help each other, and writing each other 

up is not considered helpful. 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 31 GAO-05-83 VA Patient Safety Program 

The themes for work conditions that promote a supportive culture for 

patient safety that clinicians articulated most often were (1) leadership, 

(2) communication, (3) professional values, and (4) workflow.5

A few strong patterns emerged from the clinicians’ responses to our openended interview questions about what affects trust and comfort in 

reporting close calls and adverse events. First, across the survey, the 

clinicians said their leaders’ actions were most likely to increase or 

decrease comfort and trust. Attributes of communication were the second 

most common aspect of their work that they said influenced their comfort 

and trust. Third, and somewhat less commonly, clinicians thought that the 

values and norms that they had developed in their professional training 

and that had been reinforced on the job influenced their culture, but they 

also thought that workflow could support or undercut trust generally. In 

their view, trust literally could be made or broken, depending on whether 

tasks shared between individuals or between units went smoothly and 

cooperation was maintained. Table 3 shows the results of our content 

analysis, listing the clinicians’ four top themes—leadership, 

communication, professional values, and workflow—and how many times 

we found these themes in our analysis. 

 5

Using content analysis, we grouped clinicians’ responses to open-ended questions in 

categories. We asked them a series of questions about trust, such as “To what extent do 

you perceive there is trust or distrust within your profession? Your team? And between 

your profession and other departments?” To measure comfort in reporting, we asked, “One 

of the goals of the Patient Safety Program is to create an atmosphere in which VA staff felt 

comfortable reporting adverse events and close calls without punishment or blame. To 

what extent do you think this is happening at your medical facility?” Many clinicians 

returned to the subject of trust and comfort in reporting adverse events and close calls 

spontaneously in the interviews, as when they answered questions like “What promotes 

patient safety?” and “What undercuts patient safety?” (More detail on our methodology is in 

app. I; our questions are in app. III.) 

Additional Steps to 

Stimulate Culture Change 

Building a Supportive 

Culture 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 32 GAO-05-83 VA Patient Safety Program 

Table 3: Content Analysis: Achieving a Supportive Culture through Aspects of the 

Work Environment 

 Culture element 

Aspects of work 

environment: four top 

themes 

Comfort in 

reporting Mutual trust 

Number of times 

theme appeared

in our analysis

Leadership 22 25 47

Communication 13 25 38

Professional values 15 8 23

Workflow 0 12 12

Source: GAO analysis. 

When we asked clinicians what affected a culture that supported comfort 

in reporting and trust among the different professions, departments, 

teams, and shifts they worked with, their most frequent answers were 

effective leadership and good two-way communication. Moreover, the 

clinicians told us that an unsupportive culture lacks these characteristics. 

Clinicians gave us these same answers, whether we asked about comfort 

in reporting or mutual trust. Further, we found that the culture of blame 

and punishment traditionally learned in medical training hampers close 

calls and adverse event reporting but that mutual trust is developed more 

by workplace conditions. 

Leadership’s role is important in fostering a supportive cultural 

environment for the Program. Clinicians reported examples of leaders 

facilitating comfort in reporting and mutual trust that enabled them to 

participate in the Program. But at several facilities we also heard about 

distrust of the Program that resulted from leaders’ action or lack of action. 

Clinicians told us that some VA leaders had not focused sufficiently on 

building the supportive culture that the Program requires. Staff reported 

that in order to trust, they needed information and needed to take part in 

decisions about their workplace and policies that affect their work. For 

example, clinicians told us that they wanted to be part of management’s 

decisions or, at the very least, to be informed about management’s 

decisions when a number of changes were being introduced, such as when 

medical supplies and software were purchased, clinicians were assigned 

temporary rotations, and performance measures were implemented. Their 

observations are in line with other studies that show that leaders’ making 

Effective Leadership 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 33 GAO-05-83 VA Patient Safety Program 

decisions without consulting frontline workers can cause serious 

problems of trust.6

In “Clinicians’ Perspectives on Leaders’ Supporting Trust,” we illustrate 

staff’s positive attitudes toward patient safety and how leadership is 

instrumental in developing mutual trust and comfort. 

Clinicians’ Perspectives on Leaders’ Supporting Trust 

Nurse 1: I asked my staff what the role of leaders should be so I could 

serve staff better. Many answered, “communication” and “knowing what is 

happening at the facility is important.” 

Physician 1: Leaders often bring up patient safety. They’re “taking a lead 

in making staff aware of patient safety.” At my facility, they hold staff 

meetings to review the patient safety goals of the Joint Commission on 

Accreditation of Healthcare Organizations (JCAHO). The chief of staff 

constantly brings up patient safety in meetings. The administration takes 

the lead, not only “talking the talk” but also “walking the talk.” 

Nurse 2: Trust is sustained, in part, because of weekly meetings with 

management, where they talk about patient safety. 

Physician 2: It’s leadership’s responsibility to communicate that staff are 

accountable for cooperation and coordination of patient care. 

Conversely, respondents said leaders’ actions can diminish clinicians’ 

comfort and trust, as summarized in “Clinicians’ Perspectives on Leaders’ 

Undercutting Trust.” Physicians and nurses at different facilities told us 

that trust is diminished when staff do not work in stable teams. Some of 

the policies that clinicians told us were obstacles to building a stable team 

include assigning floating or nonpermanent supervisory personnel, 

rotating physicians on and off the ward, and the monthly rotation of 

student nurses and doctors. 

 6

Page, ed., Keeping Patients Safe, pp. 3–4. 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 34 GAO-05-83 VA Patient Safety Program 

Clinicians’ Perspectives on Leaders’ Undercutting Trust 

Physician 1: For 20 years, there was nothing but “blame and train.” In 

the past, an adverse event or close call was associated with a person you 

had to blame, and the “fix” was to train them. 

Nurse 1: We have a panel of nurse managers who have discouraged 

adverse event reports for medication errors. I vow to encourage 

reporting errors without blame. We still have a way to go to be honest 

about reporting. 

Nurse 2: I know of instances when staff reported adverse events, they 

were transferred, so that does not make staff comfortable reporting 

them. There is no trust of management. 

Nurse 3: Decisions that affect our work are made without talking to staff 

or understanding our work situation. 

Physician 2: If you don’t know what’s going on, you invent it. 

Physician 3: The most critical change needed at this facility is in the 

area of leadership. Leaders are ineffective because they are not good at 

communication. We hear about reasons why we are blamed. This causes 

a feeling of distrust. 

Physician 4: Leadership has little grasp of patient care and, thus, policy 

directives have little impact. If we’re given a policy to spend a maximum 

of 20 minutes per patient, including completing records, I do what the 

patient needs. Management can just yell at me. 

Staff indicated that communication in the workplace affects trust and 

comfort in reporting. Further, they told us that communication is 

challenging, since it involves coordinating tasks with and between leaders 

and teams and their empowerment, all of which can be problematic in the 

medical setting. 

Some VA staff told us that unequal power relationships and hierarchical 

decision making are often obstacles to patient safety. They also elaborated 

on the kinds of communication that support patient safety, including 

empowering staff so that they can be heard. Traditionally, a nurse’s status 

is lower than a physician’s in hospitals, and some nurses could find it 

difficult to speak up in disagreement with physicians. For patients to be 

Communication 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 35 GAO-05-83 VA Patient Safety Program 

safe, however, nurses indicated they wanted to be empowered to openly 

disagree with physicians and other staff when they found an unsafe 

situation. For example, nurses told us that they had to speak up when they 

disagreed with the medication or dosage doctors had ordered. They also 

said that they had problems when physicians telephoned nurses and gave 

directions orally when policy stated that physicians’ orders must be 

written. 

The clinicians spoke to us about empowerment and their involvement or 

lack of involvement in decision making. “Clinicians’ Perspectives on How 

Communication Promotes Trust” gives some examples of what they told 

us about communication that they believed supports patient safety. 

Clinicians’ Perspectives on How Communication Promotes Trust 

Nurse 1: We interact with doctors and nurses in clinic. If something 

happens, we share with one another about how we might have done it 

differently. This goes on daily. 

Nurse 2: The director of the medical facility is a good communicator; he 

keeps us informed. He maintains a personal newsletter. Our nurse 

manager is well rounded and she listens. 

Nurse 3: Peers and coworkers communicating with one another 

supports patient safety. For instance, sometimes we have patients who 

have a history of violence. This information is reflected in the computer 

and comes up when they “chart them in,” but sometimes a nurse may still 

not know of such a history. Therefore, in the nurses’ reports, the history 

of violence and the need for caution is passed on. Extra information 

about the patients can also help them deescalate confrontations between 

patients. 

Physician 1: VA’s Computerized Order Entry system [a computerized 

method for ordering medications] promotes patient safety. Before, it was 

hard to read the physicians’ handwriting. The Computerized Order Entry 

at least eliminated the legibility problem. They do not have 

Computerized Order Entry at the university where I also work. VA also 

got rid of using Latin abbreviations. Now everything has to be written 

out. 

Physician 2: Open communication promotes team buy-in and therefore 

better customer service. 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 36 GAO-05-83 VA Patient Safety Program 

(continued from previous page) 

Physician 3: We have a good department because staff can 

communicate their complaints. 

Nurse 4: We do an RCA on our own close call or adverse event or those 

from other sources, and then we present the results to the staff. I 

brought a PowerPoint briefing to our staff meeting about another 

hospital’s wrong site surgery, so we could know what had happened. If 

JCAHO published an adverse event, I put it in our staff notes and have it 

discussed at the next staff meeting. 

Nurse 5: Management is more involved with the workers. It seems that 

they are listening more. 

Physician 4: Within the unit, we have good trust. Outside the unit, the 

administration has more trust and more communication. We’re in the 

loop more. In the clinic, we have good trust in nurse-to-doctor and 

doctor-to-doctor relationships and with leadership. 

Physician 5: I reported a close call recently and feared blame, but it was 

not that way at all. It was a learning experience for all who heard about 

it. I think it’s wonderful that VA has created this open atmosphere. 

Formerly, you might be a scapegoat, have backlash, and get a poorer 

rating. Today, we don’t feel we’re going to be punished. 

In “Clinicians’ Perspectives on How Faulty Communication Diminishes 

Trust,” we give clinicians’ examples of management’s undermining patient 

safety by deciding policies without consulting them, as when nurses were 

not included in decision making. Such policies sometimes proved 

dysfunctional or were ignored. 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 37 GAO-05-83 VA Patient Safety Program 

Clinicians’ Perspectives on How Faulty Communication 

Diminishes Trust 

Nurse 1: I have to double-check changes in supplies in order to 

safeguard patients, because Supply often sends ABC instead of XYZ. 

Since we’re not included in decisions about product changes, we’re 

forced to continually double-check Supply to keep patients safe. 

Nurse 2: We have poor communication between other units and the 

radiology unit. They send incontinent or violent Isolation [contagious] 

patients without notifying X-ray staff to be wary. 

Facility staff also wanted additional and more timely feedback on what 

happens to their reports of close calls, adverse events, and the results of 

RCAs. Some Patient Safety Managers often felt too busy to provide 

feedback to staff because their jobs included a number of activities, 

including facilitating RCAs. At one facility, Patient Safety Managers 

routinely reported system changes back to staff who made the reports, but 

at the other facilities, they did not have a routine way of doing this. Many 

staff at the four facilities told us that they did not know the 

recommendations of the RCA teams or the results of close call or adverse 

event reports. 

NCPS agrees that feedback to staff is necessary but inadequate, and it 

plans to focus on the need for feedback at facilities in the near future. 

NCPS’s Web site publicizes selected results of RCAs and alerts and system 

changes that result from reporting. Some of what VA’s leaders and 

frontline clinicians told us about the need for more feedback is presented 

in “Facility Staff Concerns about Limited Feedback.” 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 38 GAO-05-83 VA Patient Safety Program 

Facility Staff Concerns about Limited Feedback 

Nurse Manager: We do a good job of following up on close call or 

adverse event reports in my unit, but not as good a job following up on 

the recommendations from RCAs. I was able to implement the action 

items right away in my unit after I participated in an RCA on patients’ 

falls, but other nurse managers didn’t hear about the results from the 

RCA for 2 or 3 months. The RCA teams develop really good ideas, but we 

need follow-through to make sure everyone knows that this is what 

we’re going to do to change the system. Delays result from organizational 

routing and financial constraints. Even when the recommendation is 

signed, sometimes there’s a delay getting the information down to the 

nurse managers. 

Physician 1: There should be an annual report of actions taken as a 

result of reporting adverse events and close calls. For example, if three 

units have developed a different way of labeling medication that used to 

be labeled alike, then the rest of the staff should know about it. [This was 

a reference to medication that looks alike and confuses staff. One 

solution is for the pharmacy to buy the two medications from different 

manufacturers so that the labels will be different.] It makes people feel 

better to know the information they reported helped make things better. 

I’d make sure that the information on improved medical care gets 

reported back to the staff. 

Administrative Official: The distribution of RCAs has been limited to 

staff responsible for the action or system change, but in the future the 

results will be distributed more broadly. 

Physician 2: I haven’t heard any results from the RCAs. A pamphlet on 

the results would be a good idea. 

Note: “Administrative Official” is a title we used in this report to keep 

identity confidential. 

In addition, staff spoke to us frequently about workflow issues—how 

safely handing off tasks between shifts and teams required trust but could 

cause mistrust when the transition was not smooth or efficient. VA 

clinicians clarified for us that mutual trust could be either gained or lost 

between workers and units, depending on coordination. And they drew 

Workflow and Professional 

Training 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 39 GAO-05-83 VA Patient Safety Program 

conclusions about the importance of the quality and nature of workflow to 

patient safety. Clinicians also elaborated on aspects of the values they 

learned in training that did not facilitate a blame-free workplace. 

They indicated that shifting patient care between groups was an ongoing 

challenge to patient safety. For analysis purposes, we found these issues in 

continuity of care to be part of the larger problem of workflow, because 

they entailed the coordination of tasks and communication within and 

across teams. In the views of the clinicians at the facilities we studied, if 

staff, teams, or units begin to feel they cannot adequately communicate 

their patients’ needs for care because of workflow problems, then trust 

may be lost, in turn diminishing patient safety.7

 At one facility, where trust 

and comfort were lower than at the others, clinicians told us that 

workflow failures diminished trust and threatened patient safety. In 

“Clinicians on Workflow Problems and Patients’ Safety,” some physicians 

and nurses talk about these problems and how they tried to find solutions 

to promote patients’ safety. 

 7

The supportive culture necessary for patient safety is hard to achieve in a complex 

medical setting. According to the Institute of Medicine, when hospital staff are not fearful 

of reporting and when they have mutual trust, they cooperate better and are more 

successful at integrating their work tasks within and across teams. However, hospitals are 

complex social systems of numerous professions and work groups, and the work often 

involves high-risk tasks, making intrateam and interteam coordination difficult (see Page, 

ed., Keeping Patients Safe, pp. 3–4). Charles L. Bosk notes distrust between clinicians in 

different specialties, such as surgeons and radiologists or anesthetists and internists (see 

Bosk, Forgive and Remember: Managing Medical Error, Chicago: University of Chicago 

Press, 1979, p. 105)). 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 40 GAO-05-83 VA Patient Safety Program 

Clinicians on Workflow Problems and Patients’ Safety 

Nurse 1: Some units are less particular about paperwork and records 

than others, so when we transfer patients, their information is sometimes 

incomplete. Patients don’t come back to my unit as quickly from one unit 

as from other units, and sometimes their information is not available. 

Physician: Personnel tends to lose things, and this makes it hard to 

recruit new staff. 

Nurse 2: We often have difficulty getting the supplies we need. For 

example, it’s especially difficult to obtain blood on the night shift. 

Nurse 3: At the change of a shift, I had to discharge one patient and 

admit another. Since I couldn’t do both at the same time, I chose to admit 

but not to discharge. But my relief nurse expressed unhappiness about 

the situation, suggesting that I had left my work for another crew to do. I 

spoke with the relief nurse, and the problem of mistrust was resolved 

when everyone understood the work context better. When people 

communicate across shifts this way, they have a better understanding of 

and appreciation for one another. 

Nurse 4: I go to the ward before my shift starts to make sure the patients’ 

wounds have been properly dressed. I take dressings to homebound 

patients when they weren’t sent home with them. I cultivate motivated 

individuals from the ward staff, letting them see the procedures in the 

Dialysis Unit, and give them responsibility for those patients when 

they’re back on the ward and reward them. I stock snacks because feeble 

elderly patients are sent to Dialysis without breakfast, and then they’re 

expected to get to breakfast after their dialysis session and pay for their 

own meal. I see this situation as inherently unsafe, so I supply them with 

free snacks. 

The professional values physicians and nurses learned in their formal 

education or on the job can also be an obstacle to the Program, because 

these values do not always foster a nonpunitive atmosphere. Some of the 

values clinicians have been trained in run counter to the Program’s 

expectations for open reporting, as we show in “Clinicians’ Professional 

Values and the Patient Safety Program.” 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 41 GAO-05-83 VA Patient Safety Program 

Clinicians’ Professional Values and the Patient Safety Program 

Nurse 1: There is much trust within the nursing profession. We have to 

trust each other because of the critical nature of passing patients from 

one shift to another. 

Nurse 2: The only group I worry about is Clerical. Their work is frontline 

and high-stress, but it’s entry level, so they may have never worked in a 

hospital before. We have to double-check their work because there’s no 

system in the clinic to verify orders, as there is in the hospital. 

Nurse 3: We trust those we work with. The exception is Housekeeping. 

We have to continually call to complain about the cleanliness of the 

clinic. 

Nurse 4: Nurses have a value system in which we “eat our young,” which 

undercuts comfort in reporting errors. Traditionally, older nurses taught 

younger ones their way of doing things, and the younger ones were 

punished when they failed to do things that way. Now, we must allow 

nurses to do things a new way without punishment. 

Nurse 5: I keep hearing that we’re looking to learn and not blame. 

Nursing culture is a blaming culture, and [the Patient Safety Program] is 

helping to stop this. 

Nurse 6: The model in nursing is “a nun with a ruler.” 

Physician 1: The culture is changing, but it’s taking a while. I’m 

impressed with administration here that tries to say, “How can we learn 

from this?” 

Physician 2: To promote the Program, you have to have a change to a 

no-blame culture. 

Physician 3: Clinicians have to stop blaming each other and learn from 

their mistakes. 

VA clinicians explained that nurses see themselves as the patients’ first 

and last guard against harm during care. Nurses are expected to be doublechecking physicians’ orders, medicines, and dressings and, for example, 

preventing falls or suicide attempts. Generally speaking, in their traditional 

role, nurses feel personally responsible for patients’ welfare and are 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 42 GAO-05-83 VA Patient Safety Program 

designated to fulfill that role. They hold fast to protocols as safety devices, 

follow rules, and double-check work orders. Some spoke favorably of a 

bygone era when nurses could be counted on to back up one another, 

while many others thought this described their current work environment. 

In contrast, VA staff told us that physicians have thought of themselves as 

taking more original and independent actions but not as part of a 

multidisciplinary team. Their actions, based on traditional professional 

values, would thus undercut mutual trust. Physicians told us that patient 

safety would be improved if they were better trained to work on teams. 

Both nurses and physicians face many obstacles to improving patients’ 

safety in the increasingly complex and ever changing world of medicine. 

VA clinicians take seriously their mission as caretakers of the nation’s 

veterans, many of whom are older and have multiple chronic diseases, 

making these efforts to improve patient safety even more challenging. 

Many told us that they feel ethically and morally bound as frontline 

caretakers to keep their patients safe by reducing the number of adverse 

events and close calls. 

Although VA conducted a cultural assessment survey in 2000 and plans to 

resurvey VA staff in the near future, it has not measured staff familiarity 

with, participation in, and cultural support for the Program. For example, 

it did not ask about staff knowledge and understanding of key concepts 

(close call reporting, RCAs, and VA’s confidential reporting system to 

NASA) or RCA participation. Although the 2000 survey did describe some 

important attitudes about patient safety, such as shame and punishment 

related to reporting adverse events, it did not explicitly measure mutual 

trust among staff, a central theme of VA clinicians in describing what 

affected patient safety and a supportive culture. Finally, while NCPS staff 

asked each facility to administer the survey to a random sample, many 

facilities did not follow their directions. The VA survey may serve as a 

baseline measure of national local trends, but it could not be used to 

identify facility-level improvements or interventions.8

We found that three of the four facilities had a supportive culture that 

allowed staff to trust one another and feel comfortable reporting close 

 8

VA told us that despite the sample not being random, the NCPS did provide local results to 

facility directors in case the information was useful. 

Improving 

Assessment of, 

Familiarity with, 

Participation in, and 

Cultural Support for 

the Program 

Summary 
Chapter 3: Measuring Cultural Support for 

the Program 

Page 43 GAO-05-83 VA Patient Safety Program 

calls and adverse events. At the fourth site, clinicians told us their facility 

had an atmosphere of fear and blame that did not support the Program. 

Content analysis revealed the most frequent themes were effective 

leadership, good two-way communication, clinicians’ professional values, 

and workflow. 
Chapter 4: Promoting Patient Safety 

Page 44 GAO-05-83 VA Patient Safety Program 

Successful management actions at one facility had resulted in the most 

complete adoption of safety practices under the Program at the time of our 

study. These actions included (1) storytelling, a well-documented oral 

tradition in medicine, to show changes in norms and values; (2) teaching, 

coaching, and role modeling for open communication throughout the 

hierarchy; and (3) offering rewards for participation in close call reporting. 

Clinicians at that facility pointed to these practices, which facilitated 

patient safety and their adoption of the Program’s concepts and activities. 

The three other facilities used some or few of these practices; nonetheless, 

clinicians there proposed them as potentially good ways to improve 

patient safety. While our work reflects the clinicians’ views at the four 

facilities we studied, these findings correspond with other studies of 

organizations’ attempts to change culture.1

VA leaders at some facilities we studied showed staff they support the 

Program by telling stories. They used the stories to publicly demonstrate a 

changed and open atmosphere for learning from adverse events and close 

calls, for example. While leaders must still distinguish episodes that 

warrant professional accountability, they must fairly draw the line 

between system fixes and performance issues.2

 One way to do this is by 

repeating stories that demonstrate that VA leaders encourage a culture 

that supports the Program and an atmosphere of open reporting and 

learning from past close calls and adverse events. 

Leaders supported the Program by telling staff stories that demonstrated a 

systems change to safeguard patients after a medical adverse event was 

reported.3

 Storytelling has a long tradition in medicine as way of teaching 

 1

For example, Schein highlights practices that help leaders transmit culture to, and embed 

it in, the organization and help staff learn new practices from (1) how leaders react to 

critical incidents, organizational crises, and deliberate role modeling, teaching, and 

coaching and (2) criteria leaders use for allocating rewards and status. See Edgar H. 

Schein, Organizational Culture and Leadership (San Francisco, Calif.: Jossey-Bass, 1991). 

2

VA leaders told us that performance errors involve patterns of behavior that require 

disciplining physicians and other staff. For example, the same nurse giving out the wrong 

medicine three times in a month becomes a performance issue. 

3

Storytelling can be a way to implement system change. See, for example, Stephen 

Denning, The Springboard: How Storytelling Ignites Action in Knowledge-Era 

Organizations (Boston, Mass.: Butterworth-Heinemann, 2000); Ann T. Jordan, “Critical 

Incident Story Creation and Culture Formation in a Self-Directed Work Team,” Journal of 

Organizational Change Management 9:5 (1996): 27–35; and GAO/NSIAD-92-105. 

Chapter 4: Promoting Patient Safety 

Using Storytelling to 

Promote Culture 

Change 
Chapter 4: Promoting Patient Safety 

Page 45 GAO-05-83 VA Patient Safety Program 

newcomers about a group’s social norms.4 One leader shared with us the 

story he used to kick off VA’s Patient Safety Program. Each time he tells 

the story, he confirms the importance of changing VA’s culture and helps 

transform the organization because staff remember it. Instead of 

dismissing an employee who has reported not giving a patient the drug the 

patient was supposed to receive, the leader judged the adverse event to be 

a systems problem. In discussions with NCPS, the leader recognized that 

this story was an opportunity to show his staff that the facility was 

following the Program by taking a systems rather than a disciplinary 

approach and to highlight that reporting close calls and adverse events 

was critical in changing the patient care practice so that such problems 

would not recur. “Leaders’ Effective Promotion of Patient Safety in Staff 

Meetings” contains another example of storytelling to change 

communication practice. 

 4

For more on storytelling as a tradition in medicine, see Bosk, Forgive and Remember, pp. 

103–10. 
Chapter 4: Promoting Patient Safety 

Page 46 GAO-05-83 VA Patient Safety Program 

Leaders’ Effective Promotion of Patient Safety in Staff Meetings 

[The Administrative Official met with a unit leader and about 20 

physicians and residents.] 

Administrative Official: The Patient Safety Program includes close calls 

as reportable incidents. [That is, VA is accepting staff reports of close 

calls.] A culture change is needed at VA, brought about by sharing a 

vision of what is valuable to us. We also want to show that leadership 

endorses the Program. 

[He walked the meeting through an aviation example that showed that 

the first officer should have challenged the captain, raising parallels with 

failure to question authority—or to “cross-check”— at this facility. He 

asked the group how they challenged authority effectively. Finally, he 

introduced RCAs as a new type of system analysis. Physicians continued 

their discussion.] 

Physician 1: Cross-checking is more effective if it’s not hostile. 

Physician 2: There are fewer errors in medical settings where there’s a 

stable team, but recently VA has been trying to do things more quickly 

with fewer staff. 

Physician 3: Communication is a problem on my unit, where we have 

28 contract nurses. 

Physician 4: Could it be bad if one unit reported a lot of close calls? 

Physician 5: [in a leadership position]: VA has 50 years of being punitive. 

The Patient Safety Managers will be looking for patterns across a large 

number of reports, not seeking to blame individuals. 

Physician 6: Why can’t the reporting simply be open and the names of 

the reporters known? 

[Several members of the meeting talked about the fear of punishment 

that still existed.] 

Physicians 7 and 8: Are the forms discoverable? Can they be 

subpoenaed? Can the reports be anonymous? 
Chapter 4: Promoting Patient Safety 

Page 47 GAO-05-83 VA Patient Safety Program 

(continued from previous page) 

[In a subsequent interview, leaders told about how the Program was 

progressing.] 

Leader 1: We must change doing what you’re told without questioning 

orders. We tell nurses that it’s OK to challenge physicians in an 

atmosphere of mutual respect. We’re establishing it as a facility goal, 

keeping it on the front burner and keeping it a priority. 

Leader 2: Since leaders began visiting staff meetings to get the word out 

on close call reporting, we’ve noticed a change—a significant reduction 

in the fear of reporting close calls. Not all fear is gone, but the close call 

program is a success. 

Leader 3: Leadership raised safety consciousness with the close call 

airplane accident lesson. If it had been handed to us as just another 

memo, it might have been thrown away, but when leaders are there in 

person to answer questions, then it raises people’s awareness of patient 

safety. 

Physician 1: Leadership here went out and talked about patient safety. 

Their support and emphasis and bringing their level of importance to it 

made the Program happen. 

Staff at one facility told us that VA’s leadership supported the Program and 

the patient safety culture by teaching, coaching, and role modeling patient 

safety concepts to their staff in more than a hundred small meetings. VA’s 

leaders had a three-part agenda in their initial staff meetings. First, they 

taught a scenario in which two pilots failed to communicate well enough 

to avoid a fatal crash. The first officer did not cross-check and challenge 

an order from his captain to descend in a wind shear, resulting in the 

plane’s crashing and killing 37 people. Facility leaders depicted the strong 

parallels—-including the communication effects of unequal power 

relationships and hierarchical decisionmaking discussed earlier—-

between the pilots’ communication to save the plane and clinicians’ 

communications to save the patient. 

Second, they discussed the importance of communications in medical 

care, coaching lower-level staff to speak up when they saw adverse events 

and emphasized the importance of two-way communication. Finally, they 

Deliberate Teaching, 

Coaching, and Role 

Modeling 
Chapter 4: Promoting Patient Safety 

Page 48 GAO-05-83 VA Patient Safety Program 

introduced a new close call reporting program at the facility and modeled 

for staff that they supported this type of reporting in introducing the new 

Program and its elements. “Leaders’ Effective Promotion of Patient Safety 

in Staff Meetings” presents a portion of one such meeting and also 

interviews with VA staff when they discussed how the staff meetings had 

raised their consciousness about patient safety. 

“Leaders’ Effective Promotion” represents more than a hundred small 

meetings conducted at one facility that successfully demonstrated that 

patient safety was a priority for the organization. When top leaders 

attended staff meetings, staff listened to their message. It may be no 

coincidence that this facility had the highest rating for comfort in 

reporting, according to the findings of our survey. Many staff at this facility 

told us that because their top leaders spoke to them about the Program, 

they concluded that the Program and its culture change were a priority for 

their leaders. Midlevel staff also acknowledged progress but admitted to 

some remaining fear. 

Participants heard their leaders say that challenging authority—here 

called “cross-checking”—was important for patient safety. They were 

asked to compare their own communication patterns with the aviation 

crew’s communication in a similarly high-risk setting that depended on 

teamwork. The administrative official at the medical facility meeting, 

drawing an analogy between the aviation example and participants’ work, 

noted that an RCA had found that an adverse event could have been 

prevented if authority had been challenged. His message to the meeting’s 

participants was that VA’s leadership saw cross-checking as acceptable 

and necessary. 

The same facility that held small meetings for staff developed a close call 

reward system that reinforced the idea that reporting a close call not only 

did not result in punishment but was actually rewarded. Staff feared a 

negative atmosphere when the close call program was first established, 

with staff telling on one another, but this did not occur. The number of 

close calls at this facility was few before the reward program began. In the 

first 6 months of the program, 240 close calls were reported. While we 

were visiting the Patient Safety Managers, many staff called them to report 

close calls; each staff member was given a $4 cafeteria certificate. 

Patient Safety Managers at this facility told us that they rewarded 

reporting, no matter who reported or how trivial the report. The unit with 

the month’s best close call received a plate of cookies. The Patient Safety 

Rewarding Close Call 

Reporting 
Chapter 4: Promoting Patient Safety 

Page 49 GAO-05-83 VA Patient Safety Program 

Manager reported that a milestone had been reached when a chief of 

surgery reported a close call—a first for surgery leadership. “Rewarding 

Close Call Reporting” paraphrases leaders and clinicians on the success of 

the close call program at their facility. 

Rewarding Close Call Reporting 

Leader 1: With the close call program, the wards do not feel as secretive. 

VA leadership thought the new close call program might cause staff to 

turn on one another and begin to blame one another for reporting close 

calls, but this has not happened. 

Nurse 1: People are rewarded for reporting close calls and adverse 

events—and not punished. 

Nurse 2: I feel comfortable about reporting close calls and adverse 

events. When management first introduced the close call program, we 

thought everyone was going to tell on each other. If everyone starts to 

find out things about you, you could lose your job, because it could be on 

your record. You would have to ask yourself, “Is this something I would 

really want to tell someone about?” We thought it would be like “Big 

Brother Is Watching You.” But that is not what it’s like. I feel comfortable 

reporting close calls and adverse events. 

Administrative Official: To promote patient safety, we did a lot of 

reward and recognition to let staff know that what they have done 

[reporting close calls and adverse events] is important. 

Other facilities did not have as extensive a reward system. At one facility, 

the Patient Safety Manager had recently given a certificate to someone 

who had done a good job in describing an adverse event. However, at 

another facility, the quality manager who supervised Patient Safety 

Managers told us that she thought it improper to reward staff for 

reporting: She did not want to reward people for almost making a mistake. 

Clinicians in our interviews, however, pointed to the need to develop 

reward programs around patient safety. For example, one nurse said that 

if she were the director, she would call staff to thank them for reporting 

close calls and adverse events and would develop a reward system. 

We found that leaders used three management strategies at one facility 

that promoted the Program: (1) storytelling; (2) teaching, coaching, and Summary 
Chapter 4: Promoting Patient Safety 

Page 50 GAO-05-83 VA Patient Safety Program 

role modeling open communication in staff meetings; and (3) offering 

rewards for participation in close call reporting. These strategies changed 

clinicians’ attitudes and behavior, because they believed that the Program 

is an organizational priority, and they acted on this by reporting more 

close calls. An important part of the Program is encouraging close calls to 

surface so that safeguards can be established before patients are harmed. 
Chapter 5: Conclusions and Recommendations 

Page 51 GAO-05-83 VA Patient Safety Program 

Five years into VA’s Program to improve the safety of patients’ care at its 

medical facilities, we found progress at certain facilities but continuing 

barriers to the Program’s adoption at others. Having recognized the risks 

to patients that are inherent in medical care, VA seeks with its Program to 

identify and fix system flaws before they can harm patients. To 

successfully change its culture, VA acknowledges that it is necessary to 

change staff attitudes, beliefs, and behavior from those of fear of blame to 

open willingness to report close calls and adverse events. The fear is 

rooted in, and reinforced by, many years of professional training and 

experience in medical care settings. In the four facilities in which we 

studied the Program’s progress, we were able to measure significant 

differences in clinicians’ familiarity with and participation in the Program 

and the levels of cultural support for it. 

We conclude that progress in patient safety could be facilitated if VA’s 

program efforts focused on facilities where familiarity with the Program’s 

major concepts is low— concepts such as close call reporting, the NASA 

confidential reporting program, and RCAs—and on the facilities where 

participation in RCAs and levels of cultural support for the Program are 

low. VA may be able to use lessons learned by focusing on clinicians’ 

perspectives to prioritize future actions to further the goal of patient 

safety. 

VA should have tools available to determine which facilities face barriers 

to adopting the Program and, therefore, need assistance in stimulating 

culture change and promoting the Program. VA is to be commended for 

conducting a cultural survey that showed staff attitudes toward safety at 

the national level. However, since it was not a random survey, it was not 

effective in discerning staff attitudes at the local level. In addition, VA has 

not measured staff knowledge of the Program, staff participation in RCAs, 

or whether facility staff have enough mutual trust to support the Program. 

VA may be able to adapt measures we have suggested, such as adding to 

its survey some of our questions that focus on these issues, so as to 

identify facilities for specific interventions and assess the Program’s 

progress at the local and national levels. 

Chapter 5: Conclusions and 

Recommendations 
Chapter 5: Conclusions and Recommendations 

Page 52 GAO-05-83 VA Patient Safety Program 

Clinicians’ familiarity with the Program and opportunities to participate in 

RCAs could be measured at each facility in order to identify facilities that 

require specific interventions. Because low familiarity or participation can 

hinder the success of the Program, VA could attempt to measure and 

improve basic staff familiarity with the Program’s core concepts and 

ensure opportunities to participate in RCA teams. Our study developed 

measures of familiarity with and participation in the Program by analyzing 

responses from interviews of a small random sample of clinicians, and 

these could be further developed into useful measures in a larger study. 

These measures could also be developed into goals to be achieved 

nationally and, more importantly, locally for each facility. 

According to the clinicians we interviewed, the supportive culture of 

individual facilities plays a critical role in clinicians’ participation in the 

Program and warrants VA leadership’s priority. In one of the three 

facilities where staff had above average familiarity with the Program, staff 

told us that fear prevented them from fully participating in the Program. 

From the clinicians’ vantage point, their leaders need not accept given 

levels of mutual trust or comfort in reporting close calls and adverse 

events; instead, once facilities are identified as having low cultural support 

for the Program, that can be a starting point for change. In our 

conversational interviews with clinicians, they consistently pointed to 

specific workplace conditions that fostered their mutual trust and comfort 

in reporting. Notably, management can take actions to stimulate culture 

change by developing a work environment that reinforces patient safety. 

Drawing from their own experience, clinicians had views that were 

consistent with many studies of culture change in organizations, indicating 

that leaders’ actions and open communication are important in the 

transformation sought under the Program. 

We were able to directly observe practices that have convinced frontline 

workers that the Program is a priority for VA, that it is worth their while to 

participate in it, and that by doing so medical facilities are safer for 

patients. These practices included leadership’s demonstrating to staff that 

patient safety is an organizational priority—for example, by coaching and 

by communicating safety stories in face-to-face meetings with all staff—

and that the organization values reporting close calls because it rewards 

and does not punish staff for reporting them. 

Measuring Clinicians’ 

Familiarity with and 

Cultural Support for 

the Program 
Chapter 5: Conclusions and Recommendations 

Page 53 GAO-05-83 VA Patient Safety Program 

To better assess the adequacy of clinicians’ familiarity with, participation 

in, and cultural support for the Program, we recommend that the Secretary 

of Veterans Affairs direct the Under Secretary for Health to take the 

following three actions: 

1. set goals for increasing staff 

• familiarity with the Program’s major concepts (close call reporting, 

confidential reporting program with NASA, root cause analysis), 

• participation in root cause analysis teams, and 

• cultural support for the Program by measuring the extent to which 

each facility has mutual trust and comfort in reporting close calls and 

adverse events; 

2. develop tools for measuring goals by facility; and 

3. develop interventions when goals have not been met. 

We provided a draft of this report to VA for its review. The Secretary of 

Veterans Affairs stated in a December 3, 2004, letter that the department 

concurs with GAO’s recommendations and will provide an action plan to 

implement them. VA also commented that the report did not address the 

question of whether VA’s work in patient safety improvement serves as a 

model for other healthcare organizations. GAO’s study was not designed to 

evaluate whether VA’s program was a model, compared with other 

programs, but was limited to how the program had been implemented in 

four medical facilities. VA also provided several technical comments that 

we incorporated as appropriate. 

Recommendations for 

Executive Action 

Agency Comments 

and Our Evaluation 
Appendix I: Content Analysis, Statistical 

Tests, and Intercoder Reliability 

Page 54 GAO-05-83 VA Patient Safety Program 

To analyze the data we collected, we used content analysis, a technique 

that requires that the data be reduced, classified, and sorted. In content 

analysis, analysts look for, and sometimes quantify, patterns in the data. 

We conducted tests on clinicians’ responses to our key variables and 

found a number of significant differences. We also conducted intercoder 

reliability tests—that is, we assessed the degree to which coders agreed 

with one another. The tests showed that the consistency among the coders 

was satisfactory. 

Ethnography is a social science method, embracing qualitative and 

quantitative techniques, developed within cultural anthropology for 

studying a wide variety of communities in natural settings. It allowed us to 

study the Program in VA’s medical facilities. Ethnography is particularly 

suited to exploring unknown variables, such as studying what in VA’s 

culture at the four facilities affected the Program. In our open-ended 

questions, we did not supply the respondents with any answer choices. We 

allowed them to talk at length, and therefore the interviews lasted 

anywhere from a half hour to an hour or more. 

Ethnography is also useful for giving respondents the confidence to talk 

about sensitive topics. We anticipated that clinicians would find the study 

of VA’s medical facility culture, including staff views of close calls and 

adverse events, a sensitive subject. Therefore, we gave full consideration 

to the format and context of the interviews. Although ethnography is 

commonly associated with lengthy research aimed at understanding 

remote cultures, it can also be used to inform the design, implementation, 

and evaluation of public programs. Governments have used ethnography 

to gain a better understanding of the sociocultural life of groups whose 

beliefs and behavior are important to federal programs. For example, the 

U.S. Census Bureau used ethnographic techniques to understand 

impediments to participation in the census among certain urban and rural 

groups that have long been undercounted.1

We conducted fieldwork for approximately a week at each of two 

facilities, for 3 weeks at a third, and for 25 days at the fourth. Although 

ethnographers traditionally conduct fieldwork over a year or more, we 

 1

GAO, Federal Programs: Ethnographic Studies Can Inform Agencies’ Actions, 

GAO-03-455 (Washington, D.C.: March 2003). 

Appendix I: Content Analysis, Statistical 

Tests, and Intercoder Reliability 

Content Analysis 

Ethnography 

Data Collection 
Appendix I: Content Analysis, Statistical 

Tests, and Intercoder Reliability 

Page 55 GAO-05-83 VA Patient Safety Program 

used a more recent rapid assessment process (RAP). RAP is an intensive, 

team-based ethnographic inquiry using triangulation and iterative data 

analysis and additional data collection to quickly develop a preliminary 

understanding of a situation from the insider’s perspective.2

We drew two samples, one judgmental and one random. To understand 

how the Program was implemented at each medical facility, we conducted 

approximately a hundred nonrandom interviews with facility leaders, 

Patient Safety Managers, and a variety of facility employees at all levels, 

from maintenance workers, security officers, nursing aides, and 

technicians to department heads. This allowed us a detailed understanding 

of how the Program was implemented at each facility. 

To ensure that we represented clinicians’ views at all four facilities, we 

selected a random sample of 80, using computer-generated random 

numbers from an employee roster of clinicians, yielding 10 physicians and 

10 nurses at each facility.3

 While this provided us with a representative 

sample of clinicians (physicians and nurses) from each facility, the size of 

this sample was too small to provide a statistical basis for 

generalizing from our survey results to the entire facility or to all facilities. 

For both samples, we used a similar semistructured questionnaire (see 

app. III). It consisted of mostly open-ended questions and a few questions 

with yes-or-no responses. At every interview, we asked staff for their 

ideas, and we incorporated a number of their perspectives into this report. 

A hallmark of ethnography is its observation of behavior, attitudes, and 

values. Observation is conducted for a number of purposes. One is to 

allow ethnographers to place the specific issue or program they are 

studying in the context of the larger culture. Another, in our case, was to 

allow some facility staff to feel more comfortable with us as we 

interviewed them. Both purposes worked for us in this study. 

Because we had observed meetings and RCA teams at work, we could 

better understand respondents’ answers. Respondents noted how 

comfortable they were in talking to us and how different our 

conversational interviews were from other interviews they had 

experienced in the past. We observed staff in their daily activities. For 

 2

See James Beebe, Rapid Assessment Process. 

3

At one site, we interviewed 11 physicians, so our random sample actually consisted of 81 

staff. 
Appendix I: Content Analysis, Statistical 

Tests, and Intercoder Reliability 

Page 56 GAO-05-83 VA Patient Safety Program 

example, we accompanied a nurse while she administered medication 

using bar code technology that scans the medication and the patient’s 

wristband. We also observed staff at numerous meetings, including RCA 

team meetings, patient safety conferences, patient safety training sessions, 

staff meetings in which patient safety was discussed, and daily leadership 

meetings. 

Our methodology included collecting data from facility records. We 

examined all close calls and adverse events reported for a 1-month period 

and all RCA reports conducted at each facility, and we reviewed 

administrative boards and rewards programs. We read minutes from 

patient safety committees and other committees that addressed safety 

issues. 

Our data were mostly recorded, but some interviews were written, 

depending on respondents’ permission to record. Using AnnoTape, 

qualitative data analysis software, we coded the interviews for both 

qualitative and quantitative patterns, and we used the software to capture 

paraphrases for our analysis. 

We developed a prescriptive codebook to guide the coders in identifying 

interviews and classifying text relevant to our variables. After several 

codebook drafts, we agreed on common definitions and uses for the 

codes. In the content analysis of our random sample data, we looked for 

patterns, associations, and trends. AnnoTape allowed us to mark a digital 

recording or transcribed text with our codes and then sort and display all 

the marked audio or text bites by these codes. Because all the coders 

operated from a common set of rules, we achieved a satisfactory 

intercoder rater reliability score. AnnoTape also allowed us to record 

prose summaries of the interviews, some of which paraphrased what the 

clinicians said; the paraphrases we present in the report reflect the range 

of views and perceptions of VA staff at the four medical facilities. A rough 

gauge of the importance of their views is discernible in the extent to which 

certain opinions or perceptions are repeatedly expressed or endorsed. 

Using the statistical package SAS, we analyzed the variables with twochoice and three-choice answers and transferred them to an SAS file for 

quantitative analysis. Among the quantifiable variables were five yes-or-no 

questions asking about respondents’ familiarity with key elements of the 

Patient Safety Program. We created a new variable that reflected a 

composite familiarity score for the Program, using the five questions about 

familiarity with the key elements (the questions are listed in the note to 

Data Analysis 
Appendix I: Content Analysis, Statistical 

Tests, and Intercoder Reliability 

Page 57 GAO-05-83 VA Patient Safety Program 

fig. 4). We also assessed respondents’ levels of comfort in reporting close 

calls and adverse events and mutual trust among staff at each facility, 

based on each whole interview. We used these two assessments, rated 

high, middle, or low to characterize cultural support for the Patient Safety 

Program. 

In quantifying verbal answers for display and comparison purposes, we 

decided that the maximum individual familiarity, trust, and comfort levels 

should be 10. Thus, in each key elements question, we let “yes” equal 2 and 

“no” equal 0, ensuring that an individual who knew all of the five elements 

would achieve a composite score of 10. Finally, we averaged composite 

scores to get an average score for each facility. In the trust and comfort 

summary judgments, we let “high” equal 10, “medium” equal 

5, and “low” equal 0. Rather then display these numbers, we used a scale of 

high, medium, and low for 10, 5, and 0 and placed the answers accordingly. 

We were able to determine statistically significant differences in clinicians’ 

responses by facility and, unless otherwise noted, we report only 

significant results. 

First, we conducted a nonparametric statistical test, called Kruskal-Wallis, 

on all possible comparisons in the subset of variables that we report in our 

text.4

 Four of these variables were central to the report: comfort summary 

score, trust summary score, close call score, and root cause score. In the 

Kruskal-Wallis test, each observation is replaced with its rank relative to 

all observations in the four samples. Tied observations are assigned the 

midrank of the ranks of the tied observations. The sample rank mean is 

calculated for each facility by dividing its rank sum by its sample size. 

If the four sampled populations were actually identical, we would expect 

our sample rank means to be about equal—that is, we would not expect to 

find any large differences among the four medical facilities. The KruskalWallis test allows us to determine whether at least one of the medical 

facilities differs significantly from at least one other facility. This test 

showed that—for each of the comfort, trust and close call variables—at 

least one of the medical facilities differed significantly from at least one of 

the other medical facilities. 

 4

Rank sum tests such as Kruskal-Wallis are designed for situations in which the 

distributions of the populations that are the source of data are unknown. 

Significance Testing 
Appendix I: Content Analysis, Statistical 

Tests, and Intercoder Reliability 

Page 58 GAO-05-83 VA Patient Safety Program 

Next, we conducted a follow-up test to determine specifically which pairs 

of medical facilities were significantly different from other pairs on key 

variables. This follow-up test is a nonparametric multiple comparison 

procedure called Dunn’s test.5

 Our using Dunn’s test meant testing for 

differences between six pairs of medical facilities: A vs. B, A vs. C, A vs. D, 

B vs. C, B vs. D, and C vs. D. 

Table 4 presents the results of Dunn’s test, along with each facility’s 

sample rank mean and sample size. The pairs of facilities that are 

statistically significantly different from one another are in the far right 

column. Note that for the root cause characteristic, there are no 

statistically significant findings from the multiple comparison testing, 

which conforms to the results of the earlier Kruskal-Wallis test on root 

cause. 

Table 4: Nonparametric Multiple Comparison Results 

Characteristic Facility A Facility B Facility C Facility D 

 Statistically significant 

comparisona

Comfort 25.5 (20) 49.4 (20) 43.6 (19) 41.7 (20) A vs. B*** 

A vs. C*** 

A vs. D** 

Trust 28.8 (19) 44.4 (21) 46.3 (20) 41.7 (20) A vs. B* 

A vs. C** 

Close callb 38.5 (20) 49.2 (20) 42.7 (20) 26.3 (18) B vs. D*** 

C vs. D** 

Root causec 43.0 (20) 39.4 (21) 43.1 (20) 36.3 (19) None 

Source: GAO analysis. 

Note: Numbers are sample rank means and, in parentheses, sample sizes. 

a

Significance levels 0.0250, 0.0167, and 0.0083 are indicated by three, two, and one asterisks, 

respectively. These significance levels were determined by dividing overall significance levels 

0.15, 0.10, and 0.05, respectively, by 6, or the number of comparisons. 

b

A sum of scores on “Do you know what close call or near miss reporting is?” and “Do you know what 

the Patient Safety Reporting System to NASA is?”—a related subgroup of the knowledge questions. 

c

A sum of scores on “Do you know what an RCA is?” “Have you participated in an RCA?” and “Do you 

know anyone who has participated in an RCA?”—a related subgroup of the knowledge questions. 

 5

Dunn’s test is a multiple comparison procedure considered appropriate for use following a 

Kruskal-Wallis test. See Wayne W. Daniel, Applied Nonparametric Statistics (Boston: 

Houghton Mifflin, 1978), p. 212. 
Appendix I: Content Analysis, Statistical 

Tests, and Intercoder Reliability 

Page 59 GAO-05-83 VA Patient Safety Program 

Consistency among the three coders was satisfactory. We assessed 

agreement among the coders for selected variables for interviews with 

seven clinicians—that is, we assessed the extent to which they 

consistently agreed that a response should be coded the same. To measure 

their agreement, we used Krippendorff’s alpha reliability coefficient, 

which equals 1 when coders agree perfectly or 0 when coders agree as if 

chance produced the results, indicating a lack of reliability.6

 Our 

Krippendorff’s alpha values ranged from 0.636 to 1.000 for nine of the 

selected variables (see table 5). Compared with Krippendorff’s guidelines 

that alpha is at least 0.8 for an acceptable level of agreement and ranges 

from 0.667 to 0.8 for a tentative acceptance, we believe our overall our 

results are satisfactory. 

 6

The advantage of using Krippendorff’s technique is, among others, that it applies to any 

number of coders, any number of categories or scale values, any level of measurement, 

incomplete or missing data, and large and small sample sizes. 

Intercoder Reliability 
Appendix I: Content Analysis, Statistical 

Tests, and Intercoder Reliability 

Page 60 GAO-05-83 VA Patient Safety Program 

Table 5: Intercoder Reliability Assessment Results 

Variable Krippendorff’s alpha

Q2 Facility location 0.878

Q5 Respondent set 1.000

Q8 Respondent title 1.000

Q17 Change a 

Q18 Promotes safety a, b 

Q19 Undercuts safety a, b 

Q20 Close call recognition 0.796

Q21 PSRS 0.818

Q23 RCA recognition a, b 

Q24 RCA participation 0.808

Q25 RCA knows participant 0.636

Summary comfort score 0.757

Summary trust score 0.791

Source: GAO analysis. 

a

For this question, we consider Krippendorff’s alpha indeterminate: (1) the coders did not disagree 

(there was no variation) or (2) there was one disagreement among them but otherwise no variation. 

b

To calculate Krippendorff’s alpha, we used a computer program in N. Kang and others, “A SAS 

MACRO for Calculating Intercoder Agreement in Content Analysis,” Journal of Advertising 22:2 

(1993): 17–28. 
Appendix II: A Timeline of the 

Implementation of VA’s Patient Safety 

Program 

Page 61 GAO-05-83 VA Patient Safety Program 

This timeline highlights the training programs and other events NCPS 

completed between 1997 and 2004. 

Year Event 

1997 VA announces a special focus on patient safety 

VA drafts patient safety handbooka

VA develops Patient Safety Event Registryb

 

1998 Patient Safety Awards Program beginsc

Expert Advisory Panel is convened to look at reporting systems 

1999 Four Patient Safety Centers of Inquiry are funded 

NCPS is established and fundedd

VA informs Joint Commission on Accreditation of Healthcare Organizations that it will go beyond JCAHO’s sentinel 

event reporting system to include close calls 

VA pilots RCAs at six facilities 

Institute of Medicine issues To Err Is Human

2000 VA and NASA sign interagency agreement on the confidential Patient Safety Reporting System 

NCPS adverse event and close call reporting system established throughout VA 

NCPS trains clinical and quality improvement staff in patient safety topics, including the RCA process 

VA establishes Patient Safety Manager (hospital level) and Officer (network level) positions 

2001 RCA training continues 

Online and print newsletter Topics in Patient Safety begins publication 

RCA software is rolled out 

Facilities and networks are given the performance measure of completing RCAs in 45 days 

Healthcare Failure Mode and Effect Analysis (HFMEA), a proactive risk assessment tool is developed by VA and 

rolled out through multiple videoconferences 

2002 Aggregate RCA implementation is phased in over the yeare

New hires are trained in RCAs and Patient Safety Officers and Managers are given refresher training 

The Veterans Health Administration’s Patient Safety Improvement Handbook, 3rd rev. ed. (VHA 1050.1), is officially 

adopted 

Facilities are given a new performance measure, being required to conduct proactive risk assessment, using HFMEA 

to review contingency plans for failure of the electronic bar code medication administration system 

The American Hospital Association (AHA) sends Program tools developed by VA to 7,000 hospitalsf

Rollout of confidential reporting to NASA is largely complete 

2003 Facility directors receive a day of training to reinforce what they could do to improve the success of their patient 

safety programs 

Facilities are given a performance measure for timely installation of software patches to critical programs 

VA begins to provide training, funded by the Department of Health and Human Services, for state health departments 

and non-VA hospitals as the “Patient Safety Improvement Corps, an AHRQ/VA Partnership” 

Appendix II: A Timeline of the 

Implementation of VA’s Patient Safety 

Program 
Appendix II: A Timeline of the 

Implementation of VA’s Patient Safety 

Program 

Page 62 GAO-05-83 VA Patient Safety Program 

Year Event 

2004 Facility managers, for example, Nurse Executives and Chiefs of Staff, receive a day of patient safety training 

VA plans a patient safety assessment to document the Program’s progress 

Directors are given the performance measure of timely verification of radiology reports 

Source: NCPS and GAO. We updated the timeline at www.patientsafety.gov and revised it with input from NCPS. 

a

Revising VA’s patient safety handbook was one of the first tasks NCPS took on in 1999; it was finally 

published as Patient Safety Improvement Handbook, 3rd rev. ed. (VHA 1050.1) and officially adopted 

by VA in 2002. The handbook, now part of NCPS’s training material, is available at VA’s Web site. 

b

VA’s Safety Event Registry, developed in 1997, is an internal VA program for collecting data on 

adverse events. VA reports certain “sentinel events” to JCAHO. 

c

According to NCPS, the Patient Safety Awards Program, begun in 1998, is no longer active. 

d In the report, we consider that the Patient Safety Program began in 1999, when NCPS was 

established. 

e

Regularly held aggregate RCAs examined close call and adverse event reports that are grouped by 

commonly occurring events, such as falls. 

f

In 2002, AHA sent Patient Safety Program tools that VA had developed to 7,000 hospitals. The tools 

were videotapes about the Program and guides on how to conduct RCAs. AHA believed these tools 

would help non-VA hospitals develop their own Programs on patient safety. 

From 1999 through 2004, NCPS has conducted training in the Patient 

Safety Program. It was attended primarily by quality managers and Patient 

Safety Officers and Managers. Typically, the training lasted 3 days and 

included an introduction to the new Patient Safety Improvement 

Handbook and small group training in the RCA process. Trainees, 

especially Patient Safety Managers, were expected to take the Program 

back to their medical facilities, collect and transmit reported adverse 

events and close calls to NCPS, and guide clinicians in the RCA teams. We 

observed health fairs at several of the four facilities. 

Beginning in 2003, NCPS convened medical facility directors and other 

managers in 1-day sessions that introduced them to the systemic approach 

to improving patient safety, including a blame-free approach to adverse 

events in health care. 
Appendix III: Semistructured Interview 

Questionnaire 

Page 63 GAO-05-83 VA Patient Safety Program 

Appendix III: Semistructured Interview 

Questionnaire 
Appendix III: Semistructured Interview 

Questionnaire 

Page 64 GAO-05-83 VA Patient Safety Program 
Appendix III: Semistructured Interview 

Questionnaire 

Page 65 GAO-05-83 VA Patient Safety Program 
Appendix IV: Comments from the Department 

of Veterans Affairs 

Page 66 GAO-05-83 VA Patient Safety Program 

Appendix IV: Comments from the 

Department of Veterans Affairs 
Appendix IV: Comments from the Department 

of Veterans Affairs 

Page 67 GAO-05-83 VA Patient Safety Program 
Appendix V: GAO Contacts and Staff 

Acknowledgments 

Page 68 GAO-05-83 VA Patient Safety Program 

Nancy R. Kingsbury (202) 512-2700, kingsburyn@gao.gov 

Charity Goodman (202) 512-4317, goodmanc@gao.gov 

Additional staff who made major contributions to this report were Barbara 

Chapman, Bradley Trainor, Penny Pickett, Neil Doherty, Jay Smale, 

George Quinn and Kristine Braaten. Donna Heivilin, recently retired from 

GAO, also played an important role in preparing this report. 

Appendix V: GAO Contacts and Staff 

Acknowledgments 

GAO Contacts 

Staff 

Acknowledgments 
Glossary 

Page 69 GAO-05-83 VA Patient Safety Program 

A research and development arm of NCPS’s Patient Safety Program. The 

centers concentrate on identifying and preventing avoidable, adverse 

events, and each has a different focus. 

An event or situation that could have resulted in harm to a patient but, by 

chance or timely intervention, did not. It is also referred to as a “near 

miss.” 

Staff directly involved with patient care. 

An incident directly associated with care or services provided within the 

jurisdiction of a medical facility, outpatient clinic, or other Veterans 

Health Administration facility. Adverse events may result from acts of 

commission or omission. 

JCAHCO is an accrediting organization for hospitals and other health care 

organizations. 

A VA hospital and its related nursing homes and outpatient clinics. 

NCPS is the hub of VA’s Patient Safety Program, where approximately 30 

employees work, in Ann Arbor, Michigan. Other employees work in the 

Center of Inquiry in White River Junction, Vermont, and in Washington, 

D.C. 

PSRS, a confidential and voluntary reporting system in which VA staff may 

report close calls and adverse events to a database at the National 

Aeronautics and Space Administration. 

An interdisciplinary group that identifies the basic or contributing causes 

of close calls and adverse events. 

Glossary 

Center of Inquiry 

Close Call 

Frontline Staff 

Adverse Event 

Joint Commission on 

Accreditation of 

Healthcare Organizations 

Medical Facility 

National Center for Patient 

Safety 

Patient Safety Reporting 

System 

Root Cause Analysis Team 

(460511) 
The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To 

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Susan Becker, Acting Manager, BeckerS@gao.gov (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

GAO’s Mission 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Mail or Phone 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER