DOT/FAA/AM-17/13

Office of Aerospace Medicine

Washington, DC 20591

Crew Resource Management 

in Helicopter Air Ambulance 

Operations: 

A Literature Review

Crystal Rowley

Michelle R. Bryant

Civil Aerospace Medical Institute 

Federal Aviation Administration 

Oklahoma City, OK 73125

April 2017

Final Report
NOTICE

This document is disseminated under the sponsorship of 

the U.S. Department of Transportation in the interest of 

information exchange. The United States Government 

assumes no liability for the contents thereof.

___________

This publication and all Office of Aerospace Medicine 

technical reports are available in full-text from the 

Civil Aerospace Medical Institute’s publications website: 

http://www.faa.gov/go/oamtechreports
i 

Technical Report Documentation Page 

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.

DOT/FAA/AM-17/13

4. Title and Subtitle 5. Report Date

Crew Resource Management in Helicopter Air Ambulance Operations: 

A Literature Review

April 2017

6. Performing Organization Code

7. Author(s) 8. Performing Organization Report No.

Rowley C, Bryant MR

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)

FAA Civil Aerospace Medical Institute

P.O. Box 25082

Oklahoma City, OK 73125

11. Contract or Grant No.

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered

Office of Aerospace Medicine

Federal Aviation Administration

800 Independence Ave., S.W.

Washington, DC 20591 14. Sponsoring Agency Code

15. Supplemental Notes

Work was accomplished under approved task AM-B-HRR-521

16. Abstract

INTRODUCTION: Crew resource management (CRM) training has evolved from an aviation-specific

practice to a much broader-scoped practice involving other industries with team oriented objectives. 

Combined objective industries such as helicopter air ambulance (HAA) operations have the unique 

requirement to combine ground (dispatch) and air (flight crew) with traditional medical and aviation 

operations in ad hoc situations adding another dimension to the CRM implementation process. Despite the 

requirements for such training in HAA operations, evidence suggests that CRM training does not reach all 

team members equally. Thus, there are gaps in the current research both in theory and practice.

METHODS: Using a systematic computer query, 224 citations and 58 representative articles published 

from 1972 to 2015 were included in this review study. The results of the review are separated into five areas: 

(1) the definition of CRM and a description of implementation strategies across different industry types, (2)

a description of CRM training approaches, (3) an overview of the role CRM has played in aviation and

healthcare, (4) the role of dispatch and communication centers regarding CRM in HAA operations, and (5)

the gaps that that exist in training and implementation strategies in CRM as they relate to HAA operations.

The gaps were prioritized based on the estimated risk to HAA operations from lowest to highest.

RESULTS: The first priority was to develop a research plan to address pilots’ risk-based decision 

making processes. The second and third priorities were to develop a research plan to identify gaps in medical 

personnel’s knowledge regarding HAA operations, including their responsibilities in flight, and to determine 

whether these gaps play a role in the successful implementation of CRM principles. The fourth and fifth 

priorities were to develop a research plan to identify gaps in supporting personnel’s (i.e., dispatch, 

communication centers, etc.) knowledge regarding HAA operations and to determine whether these gaps 

play a role in the successful implementation of CRM principles.

DISCUSSION: Relevant CRM research from industries such as medical units, military units, and ground 

operations will be discussed. Cognitive and social skills associated with CRM as well as behavioral markers 

(e.g., communication, teamwork, and decision making) will also be shared and discussed.

17. Key Words 18. Distribution Statement

Aviation Crew Resource Management, CRM, Helicopter,

Helicopter Air Ambulance, Helicopter Emergency 

Medical Service, HAA, HEMS, Rotorcraft

Document is available to the public through 

the Internet:

http://www.faa.gov/go/oamtechreports/

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price

Unclassified Unclassified 27

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized 
ii

Contents 

Introduction ......................................................................................................................................1

What is CRM?..................................................................................................................................3

Skills Associated With CRM ........................................................................................................3

Behavioral Markers.......................................................................................................................5

Phases of CRM ................................................................................................................................6

Implementation .............................................................................................................................7

Trial Period ...................................................................................................................................8

Ending Period ...............................................................................................................................9

After-Action Period ....................................................................................................................10

Summary .....................................................................................................................................10

CRM Training Methods .................................................................................................................12

Background on CRM .....................................................................................................................14

CRM in Healthcare .....................................................................................................................14

CRM in Aviation ........................................................................................................................15

Dispatch and Communication Centers .......................................................................................15

Gaps Specific to HAA ................................................................................................................16

Recommended Next Steps ..........................................................................................................18

Conclusion .....................................................................................................................................19

References ......................................................................................................................................19
1

CREW RESOURCE MANAGEMENT IN HELICOPTER AIR AMBULANCE 

OPERATIONS: A LITERATURE REVIEW

INTRODUCTION 

On August 26th, 2011 a Eurocopter AS 350 B2 helicopter was dispatched to a hospital for 

medical transport of a patient to another hospital (NTSB/AAR-13/02). During patient pick-up, 

the pilot informed the company’s EMS communication center that he did not have enough fuel to 

complete the second leg of the mission. While en route to refuel, the helicopter ran out of fuel 

and crashed, resulting in the death of the pilot, flight nurse, flight paramedic, and patient on 

board. As a result of this incident, the NTSB recommended that the company revise procedures 

such that pilots are no longer solely responsible for non-routine operation decisions and, instead, 

are required to consult with the operation control center (OCC) for approval to accept, decline, or 

continue a mission when confronted with elevated risk situations (NTSB/A-13-015). This 

investigation highlighted the need to establish better communication procedures and practices 

directly related to Crew Resource Management principles specific to helicopter air ambulance 

(HAA) operations. 

The incident described above is not the first time CRM has been identified as a primary 

contributing factor of an HAA incident. As far back as June of 1998, a Helicopter Accident 

Analysis Team was constructed of government and industry helicopter experts to analyze 34 fatal 

helicopter accidents occurring between 1988 and 1992 from NTSB accident dockets (Table 1). 

Table 1. 

HAAT (1998) CRM Principles Attributed to 34 Fatal HAA Accidents 

CRM Cause Associated with Accident Number of 

Accidents 

Associated

Percentage of Accidents 

Associated with CRM 

Cause

Communication

Coordination with Ground Personnel 3 9%

Coordination with ATC 2 6%

Coordination with Other Pilots 3 9%

Judgment/Decision Making

Sense of Urgency Led to Risk Taking 10 29%

Diverted Attention/Distraction 7 21%

Flight Profile Unsafe for Conditions 17 50%

Poor CRM 6 18%

Perceptual Judgment Errors 7 21%

Procedural Errors 14 41%

Pilot Control/Vehicle Handling Deficiencies 8 24%

Use of Unauthorized Equipment 1 3%

Situation Awareness

Aircraft Position and Hazards 12 35%

Aircraft State 5 15%

Local and En Route Weather 8 24%
2

In addition, an NTSB special investigation report (NTSB/SIR-06/01) found that between 

January of 2002 and January of 2005, 55 HAA accidents occurred in the United States. These 

accidents resulted in 54 fatalities and 18 serious injuries, resulting in an increase in accidents 

from 3.53 accidents per 100,000 flight hours between 1992 and 2001 to 4.56 accidents per 

100,000 flight hours between 1997 and 2001 (NTSB/SIR-06/01). 

The number of HAA operations with critical or fatal outcomes rose steadily until 2008. To 

date, 2008 remains the single highest year of fatal helicopter accidents (Table 2; NTSB, 2009). In 

response to these accidents and the NTSB’s recommendations (NTSB, 2009), the Federal 

Aviation Administration (FAA) issued the HAA, Commercial Helicopter, and Part 91 Helicopter 

Operations rule (79 FR 9931) in 2014. This rule requires that all certificate holders have training 

in aeronautical decision making (ADM) and risk management, crew resource management and 

risk management as well as risk mitigation (79 FR 9931). 

Table 2. 

Number of Fatal Helicopter Accidents by Year and Resulting Fatalities 

Year Accidents Fatalities

2006 3 5

2007 2 7

2008 8 29

Between 2009 and 2014, the FAA focused on operational concerns with helicopter 

operations besides CRM (e.g., night operations, responding to inadvertent flight into 

deteriorating weather conditions; FAA, 2014). However, due to an increase in Helicopter Air 

Ambulance accidents (GAO, Aviation Safety, 2009) and NTSB recommendations, the FAA 

consolidated their HAA operations safety scope to require stricter flight rules and procedures, as 

well as improved communications and training and additional on-board safety equipment 

(79 FR 9931). In 2014, the FAA issued a fact sheet outlining safety measures that have been 

implemented for HAA operations (FAA, 2014) including commentary on the final rule 

(79 FR 9931) published February 21, 2014, requiring stricter safety procedures and equipment 

on board. 

Specific to CRM requirements, the FAA’s Advisory Circular (AC) 135-14B, Helicopter Air 

Ambulance Operations, recommends combined CRM training between air and ground crews to 

build “effective integration and coordination during routine flight operations as well as including 

issues such as the use of medical personnel to supplement flightcrew, as appropriate during 

emergency operations” (FAA, 2015). Two types of training, Air Medical Resource Management 

(AMRM) training and Judgment and Decision-making training, are outlined in sections 4-10 and 

4-11 in the AC. General training focused on safety culture and shared training are both included 

in AMRM training. According to the AC, shared training “provides a common language and 

understanding to enable appropriate safety communication, responsibility and authority, within 

both HAA operators and medical organizations (and others as appropriate)” (AC 135-14B). 

Section 4-11, Judgment and Decision-making Training, includes sections on topics that should 

be included, risk analysis, decision-making, management personnel involvement, as well as 
3

human factors that potentially affect HAA operations (AC 135-14B). This guidance makes up 

the bulk of the prescribed recommendations from the FAA to the operators. However, there are 

no guidance materials that specifically identify the knowledge gaps that may exist between 

medical crew knowledge and pilot knowledge while performing these operations.

Given the inception of the new rule in 2014, the FAA is now examining the effectiveness of 

Crew Resource Management training for HAA Operations. Therefore, in response to a request 

jointly made by the Flight Standards Division of the FAA (AFS-800) and the Air Transportation 

Division (AFS-200), this report summarizes the literature regarding Helicopter Crew Resource 

Management for Part 91 and Part 135 Operators including HAA Operations. This review will 

cover five broad areas: 

1. Define CRM and implementation strategies across different industry types. 

2. Describe CRM training approaches. 

3. Provide an overview of the role CRM has played in both the healthcare and aviation 

industries.

4. Describe the role of dispatch and communication centers with regard to CRM in 

HAA operations. 

5. Identify gaps that that exist in CRM in HAA operations. 

WHAT IS CRM? 

Crew Resource Management was identified as a means to train teams to work together. 

Specifically, it has been utilized to coordinate flight crews and to allow for cross-checking and 

back-up (FSF ALAR Briefing Note 2.2, 2009). Additionally, CRM has been implicated as a 

foundational element of both flight crew performance and interactions with automated systems 

across multiple domains. When flight crews and automation work together as a unit, the 

resources needed to achieve successful and safe flights are utilized efficiently (Kanki & 

Helmreich, 2010). Similarly, when crew members are trained to work together as a unit, flight 

safety is enhanced (Orasanu, 2010). 

CRM curriculum was introduced to achieve increased performance capabilities by utilizing 

team resources most effectively. Studies found that by emphasizing the cognitive and social 

skills required to manage team-oriented activities (such as flying aircraft), as opposed to the 

technical knowledge associated with performing the task (Royal Aeronautical Society, 1999), 

better team performance was possible (Helmreich, 1991). The premise of CRM stresses that 

strengthening communication skill between team members allows technical skills to be 

seamlessly integrated into an operation for an efficient use of resources with a greater chance of 

successful outcomes. 

Skills Associated With CRM 

Previous work has identified specific CRM components and their associated behavioral 

markers. The two fundamental CRM component groups are cognitive skills (e.g., decision 

making, situation awareness, and workload management), and social skills (e.g., leadership and 
4

teamwork; Flin & Martin, 2001). These categories are used fairly consistently throughout the 

literature (Table 3). For example, the Royal Aeronautical Society (1999) identified both 

cognitive and social skills as keystones to CRM in aviation. However, there are some differences 

in labeling among research studies, airlines, and fleets. This suggests a lack of uniformity in 

labeling skills versus behavioral markers. For example, the Airbus Flight Operations Briefing 

Notes (June, 2004) identifies 11 CRM skills including assertiveness, inquiry/advocacy, and time 

management. Similarly, the Flight Safety Foundation released a CRM Briefing Note (2009) 

identifying complacency/overconfidence, inadequate proactive flight management, and 

inadequate preparedness in addition to the 11 identified by Airbus in 2004. Though they are 

identified in these sources at a more fine-grained level, these skills do generally fall into either a 

cognitive or social skill (Table 3). 

Table 3. 

Similarities in CRM Categories Across References 

Study Cognitive Social

Royal Aeronautical Society, 

1999

Situational Awareness

Planning and Decision 

Making

Communications

Teamwork

Airbus Flight Operations 

Briefing Notes, 2004

FSF ALAR Briefing Note 2.2, 

Oct 2009

Inquiry and Advocacy

Procedures

Time Management

Error Management

Risk Management

Decision Making 

Spatial Disorientation

Leadership

Teamwork

Assertiveness

Interruptions/Distractions

Complacency/Overconfidence

Inadequate anticipation

Inadequate preparedness

Personal Factors

Absence of trained instructors 

and check airmen for CRM 

evaluations

ICAO-CAP 720 (Formerly, 

ICAO Digest No. 2), Aug 

2002 (CAA, 2002)

Situation Awareness

Problem Solving

Decision Making

Judgment

Stress Management

Critique

Leadership/Followership

Social Skills (i.e., listening, 

conflict resolution, and 

mediating).

Joint Aviation Authorities, 

JAR-OPS 3 Commercial Air 

Transport (Helicopters), July 

2007

(Weaver et al., 2010;Morey et 

al., 2002;

Haller et al., 2008)

Error Prevention and 

Detection

Stress, Stress Management, 

Fatigue and Vigilance

Information Acquisition

Situation Awareness

Workload Management

Decision Making

Communication

Co-ordination inside and 

outside the Cockpit

Leadership

Team Behavior

History of CRM, (FAA TV, 

2012)

Setting Priorities

Workload Management

Communications
5

Behavioral Markers 

Due to the diversity of literature sources and reporting methods, labeling behavioral markers 

that apply across the spectrum of CRM research has proved challenging. However, the most 

basic behavioral markers have been identified from the literature including communication, 

teamwork, and decision making (Helmreich, 1997). Subcategories of behavioral markers tend to 

vary based on organizational need, choice of measurement, and reporting method. When 

examining the lists contained in Table 3, the category “other” must be added to the behavioral 

markers identified to account for subcategories under the cognitive and social umbrellas (Figure 

1). However, other behavioral markers have been examined and found to be important indicators 

of performance. For example, Drury, Dorrian, Ferguson, and Thomas (2013) examined the 

heightened emotional activity (HEA) of 27 short-haul and long-haul pilots by measuring five 

HEA behavior markers: confusion, disagreement, stress, frustration, and unease. They found 

that pilots were able to identify when they were experiencing HEA reliably and validly 

throughout the task. They concluded that these behavioral markers were indicators of HEA for 

pilots, and they were aware of such HEA when performing tasks. While these findings suggest 

that pilots are aware of their HEA, the authors did not provide evidence to suggest that these 

HEA impaired performance. Nor did they identify whether there were potential differences in 

performance for those pilots who may not be aware of their HEA. 

Figure 1. Subcategories of Behavioral Markers Including Cognitive and Social Umbrella 

• Assertiveness

• Social Skills 

• Coordinating Inside and 

Outside the Cockpit

• Decision Making

• Problem Solving 

• Judgment 

• Critique 

• Error Prevention and 

Detection 

• Planning 

• Information Acquisition 

• Setting Priorities 

• Spatial Disorientation 

• Risk Management 

• Error Management 

• Time Management 

• Workload Management 

• Teamwork

• Leadership 

• Followership 

• Team Behavior

• Interruptions/ 

Distractions 

• Personal Factors 

• Complacency/ 

Overconfidence 

• Inadequate Anticipation

• Inadequate 

Preparedness 

• Absence of Trained 

Instructors 

COMMUNICATION TEAMWORK

DECISION

MAKING 

OTHER
6

Flin and Martin (2001) were concerned with the current implementation of CRM training 

including validity and reliability of such measures. One difference that study had from the Drury 

et al. (2013) study was that assessments were made of others, rather than a rating of one’s own 

behavior. This key difference means that while it may be possible for pilots to reliably and 

validly rate their own HEA depending on the operation type, workload, and stress level, it may 

not be possible to do the same for a team member. In addition, HEA is not considered to be a 

tenant of CRM, per se. Rather, it seems HEA may be more likely to have an effect on the 

behavioral markers of CRM. This distinction makes it even more important that measurement of 

performance be included in future studies of CRM. In fact, Flin and Martin (2001) suggested that 

if researchers provided more detail about the events and scenarios employed, it may be possible 

to reduce technical and non-technical behaviors associated with behavioral markers into a more 

condensed assessment. 

An FAA Advisory Circular (AC 120-51E Appendix 1), lists crew performance marker 

clusters. Behavioral markers are identified in three categories. Section 1, Communications 

Processes and Decision Behavior Cluster, includes behavioral markers for briefings (i.e. [2] The 

briefing is interactive and emphasizes the importance of questions, critique, and the offering of 

information). There are also behavioral markers related to inquiry, advocacy, and insertion, 

which, according to the authors, “relate to crewmembers promoting the course of action that they 

feel is best, even when it involves conflict with others.” Section 2, Team Building and 

Maintenance Cluster, includes markers for leadership/followership and concern for tasks, as well 

as markers that concern social relationships and group climate, such as crewmembers’ ability to 

remain calm under stressful conditions. Section 3, Workload Management and Situational 

Awareness Cluster, focuses on preparation, planning and vigilance, as well as workload 

distribution and avoiding distractions. 

More recent approaches to the study and implementation of CRM have consolidated the 

behavior markers depicted in Figure 1 with events and scenarios relevant to particular operations 

into a threat and error management model (TEM; Helmreich, 1998; Musson & Helmrich, 2004). 

Specific to aviation most CRM programs currently use TEM as a foundation for CRM training 

(Helmreich, Klinect, & Wilhelm, 1999). This model allows for prediction and prevention of error 

through the identification of threats present in an operational environment, as well as possible 

errors that may increase the risk of those threats (Chidester, 2016). Viewing aviation through the 

lens of TEM with CRM as an overlay has allowed for qualitative as well as practical 

improvements in flight safety (Huerta, 2014). 

PHASES OF CRM 

From the literature, CRM effectiveness was typically measured by examining one or more of 

four adoption phases: Implementation, Trial Period, Ending Period, and After-Action Period. For 

each adoption phase, studies gleaned best practices for a successful CRM program. These phases 

have been identified in previous works from the perspective of developing a measurement 

instrument for behavioral markers (Flin & Martin, 2001): scale development, scale trialing, 

training of assessors, and system evaluation. Each phase of adoption allows team members to 
7

familiarize themselves with CRM concepts and time to culturally adjust to maximize acceptance 

of the CRM program. In the following sections, each phase is defined and supporting literature 

was reviewed. 

Implementation 

The goal of the implementation phase is to create a CRM program, teach the principles of the 

program, and then implement the program into the work environment. Several studies examined 

the implementation phase in order to determine what processes were required to ensure 

successful adoption. For example, a study conducted by Kemper, van Dyck, Wagoner, Wouda, 

and de Bruijne (2014) focused on the implementation of CRM in three intensive care units 

(ICUs) of a hospital facility. Those personnel who were responsible for implementing the 

program were interviewed regarding the choices, rationales, and consequences that played a role 

in their approach. Personnel identified problems with communication, structural time allotted to 

the training phase, and a clear vision for implementation as obstacles to successful 

implementation. By addressing each of these problems, personnel across three different ICUs 

successfully launched several CRM initiatives, each using an implementation strategy tailored to 

their needs. This outcome highlighted how CRM training courses can be implemented in a way 

that best fits an organization and produces successful outcomes.

In a study conducted by Cahill, McDonald, and Losa (2014), current implementation 

practices across five European airlines were assessed. The researchers used three process 

mapping phases: (1) preflight, (2) flight execution, and (3) postflight, to identify areas in need of 

improved training. A process map is an account of a task from beginning to end and has the 

ability to visualize the process of that task. In this study, the operational flight process from start 

to finish was mapped by five different European airline personnel in a workshop setting. 

Researchers found that the process models held by workshop participants were focused mainly 

on the flight crew rather than the team as a whole (air traffic control, dispatch, and crew). As 

such, Cahill et al. proposed a new integrated model that focused on operational tasks of all team 

members. These results emphasize the need for a team focus during the implementation phase of 

a new CRM program that integrates the tasks of all members of a team. Such a focus should 

identify specific roles and responsibilities for individuals as well as the group as a whole to 

ensure a collective understanding of the team’s responsibilities (Cahill et al., 2014).

Recommendations. The two studies above represent two perspectives on the implementation 

phase of CRM programs: the mind-set and planning perspective, and the program model 

perspective. The mind-set and planning perspective describes the planning necessary for those 

personnel tasked with implementing the program within an organization. Six best practices were 

identified by Kemper, et al. (2014): 

1. Identify and solve problems.

2. Seek advice from other implementation experts.

3. Formulate end goals and evaluate them. 

4. Start directly after the CRM training. 
8

5. Schedule enough time in advance/planning for the implementation phase.

6. Be aware that CRM can easily lose momentum. 

The program model perspective describes the model by which a CRM program is developed 

for an organization. For example, does the CRM program focus on defining and understanding 

CRM practices? Does the CRM program identify the specific roles of the team and how their 

actions uniquely contribute to operational success? From this perspective, the following best 

practices were identified: 

1. Identify the specific roles of each individual expected to participate in the CRM 

process throughout the operation. 

2. Use explicit examples of good CRM practices when implementing a CRM program. 

Trial Period 

The trial period for CRM is defined as the period in which the program is in its first weeks 

following personnel training and implementation or when the CRM program is tested on smaller 

sample of the environment in which it is to be adopted. Trial period studies are an important part 

of the CRM phases because they can collect critical information ahead of a major 

implementation of CRM training. Findings from these studies shape the adoption, integration, 

and outcomes of each organization, which may utilize resources and tools offered to improve 

CRM in their industry. 

Young-Xu, Fore, Metcalf, Payne, Neily, and Sculli (2013) conducted a preliminary study to 

assess the degree to which the Boise VA Medical Center could improve both communication and 

situational awareness within nursing units. Prior to the study, it was identified that unclear 

protocols led to personnel failing to call in a specialist when patients met criteria for specialist 

attention. Since checklists have been shown increase performance by decreasing slips and lapses 

in procedures (Reason, 2008), Young-Xu et al. (2013) created a “read and do” checklist to

attempt to decrease the overall failure-to-rescue events within their smaller, 18-bed step-down 

intensive care unit. Findings showed that correct specialist calls were increased significantly 

from before the training (4%) to after the training (22%) with the use of the checklists. This trialperiod study showed how read-and-do checklists may improve outcomes in a larger nursing 

setting. The use of such trial-period studies emphasizes the ability to test CRM programs before 

widespread use is recommended. 

Other studies have focused on a trial period for determining training delivery type. For 

example, Clay-Williams, Greenfield, Stone, and Braithwaite (2014), evaluated the learning of 

participants in a modular-based CRM program (provided over the course of weeks or months) 

yields similar results to traditional all-day seminar environments. Participants were allowed to 

choose their module based on their perceived learning needs. Learners were provided a baseline 

attitude survey five minutes prior to workshops they attended, as well as a questionnaire to 

examine reactions to the training and to assess behaviors in the workplace within 15 minutes of 

attending each workshop. Results indicated that the workshop content and format were beneficial 

and met the needs of participants. While the authors did not conclude that these findings should 
9

be interpreted as generalizable to the wider population, this study represents one way researchers 

are utilizing the trial-period phase to determine best needs for an organization or process. By 

“testing out” these ideas, the CRM program becomes more robust and is more likely to meet the 

objectives and outcomes desired by an organization. 

Recommendations. The two studies referenced above represent two approaches that can be 

taken to the trial-period phase. The following recommendations were identified: 

1. Identify a subset of your population that accurately represents the broader body of 

team members. 

2. When implementing the trial-period phase of new CRM program, plan to measure the 

desired outcomes in the field once training is complete. 

Ending Period 

The ending-period phase is defined as the time at which an organization assesses the impact 

that CRM training has had on operations. This phase could be seen as a post-test follow up 

measure of performance once CRM training has completed. It is different from the trial period 

because it is assumed that the CRM program implemented is one that has had success in other 

organizations across the industry and possibly in other industries as well. It is unique from the 

after-action period, as it represents the general differences in performance from either 

implementing a different CRM program to replace an existing CRM program, or implementing a 

CRM program for the first time in the organization.

Sweeney, Warren, Gardner, Rojek, and Lindquist (2014) examined the effectiveness of a new 

CRM program through measurement of participants’ perceptions of quality of communication 

before and after the training program. Researchers found that by measuring the perceptions staff 

held before and after the study, they were able to assess the changes of these perceptions once 

training was complete. However, they point out that these measures were subjective and not 

followed up with objective measures (e.g., improved performance). 

Sandahl, et al. (2013) examined the start-to-finish results of a two-year case study involving 

CRM training. Their findings from the ending-period phase showed that without continuous 

organizational accommodations (e.g., debriefing and feedback), the positive communication 

increases found throughout the study would not continue. They also found that if resources were 

strained, positive CRM outcomes could not be sustained. Similarly, Morgan et al. (2015) found 

that by assessing ending-period outcomes they were able to identify increased teamwork ability 

but no change for performance-based outcomes. This distinction is important because it 

demonstrated that successful CRM adoption and outcomes cannot be used as a proxy for 

performance improvement. 

Recommendations. The studies referenced above represent different approaches for 

measuring the ending-period phase. As such, the following recommendations were identified: 

1. Plan to measure the outcomes of CRM adoption once training is complete. 

2. Plan to measure performance apart from CRM skills. 
10

3. Training phases of the CRM program should mirror operational constraints of the 

environment in which it will be implemented. 

After-Action Period 

The After-Action period is defined as the time at which an organization assesses the current 

CRM training effectiveness by examining the outcomes of operations and practices within the 

organization. These assessments can be done by examining accidents and mishaps, reports made 

by crew members, surveying personnel, and other methods. Crew Resource Management 

training programs should be constantly revised and updated as new information is compiled. 

Therefore, in the after-action period, it is essential that information gleaned during this time is 

analyzed in such a way as to identify gaps that exist in the current training so that improvements 

can be made. For example, in 2003, the Flight Safety Foundation summarized the final NTSB 

report on accident no. NYC99FA140 regarding an air ambulance crash in 1999. Based on their 

findings, a combination of weather and failed CRM incidents contributed to the crash. In 

response, the company made changes to their CRM training program to integrate line-oriented 

simulations to determine which crewmembers should receive additional training prior to flying. 

This example highlights how organizations can examine their CRM training programs internally 

or utilize external sources to assist with modifying the training program. 

In the medical field, O’Dea, O’Connor and Keogh (2014) sought to discover whether CRM 

in acute care domains would be beneficial. Using the Preferred Reporting Items for Systematic 

Reviews and Meta-Analyses (PRISMA), the authors examined 20 articles that met their 

standards for inclusion in a meta-analysis. Findings showed that CRM training participant 

reactions were highly positive with regard to training, and they believed it was relevant to 

improving teamwork as well as patient safety. The authors found an effect on participants’ 

knowledge of CRM, an effect on attitudes of CRM training, and an effect on behaviors related to 

good CRM practices. Though this study did not directly associate CRM training with better 

performance, their measurement of attitudes and behaviors are a benchmark for how these 

variables should be examined during the after-action period for any CRM training program. 

Recommendations. The studies referenced above represent different approaches for 

measuring the After-Action phase. As such, the following recommendations were identified: 

1. Organizations may utilize both internal and external reviewers to determine the broad 

implications and outcomes of CRM training within their company. 

2. Measurement of employee’s perception of CRM effectiveness is an important 

consideration in assessing the broad impact of a CRM training program. 

Summary 

The four core phases of CRM within an organization are Implementation, Trial Period, 

Ending Period, and After-Action Period. The purposes of each phase vary by the point in which 

they should be applied (i.e., development, adoption, and review) to ensure effectiveness of a 

CRM program within an organization as well as across the industry for which it was intended. 
11

Each phase represents a critical point at which the CRM program should be assessed. Following 

the best practices and recommendations outlined by the literature reviewed above provides a 

foundation for successful CRM programs. 

Table 4.

The Four Core Phases of CRM. 

Phases Definition Recommendations

Implementation

Create a CRM program, teach 

the principles of the program, 

then implement the program 

into the work environment.

 Assess the current safety 

culture.

 Identify and solve problems.

 Seek advice from other 

implementation experts.

 Formulate end goals and 

evaluate them.

 Start directly after the CRM 

training.

 Schedule enough time in 

advance/planning for the 

implementation phase.

 Be aware that CRM can 

easily lose momentum.

 Identify the specific roles of 

each individual expected to 

participate in the CRM 

process throughout the 

operation.

 Use explicit examples of 

good CRM practices when 

implementing a CRM 

program.

Trial Period

The first weeks following 

personnel training and 

implementation or when the 

CRM program is tested on 

smaller sample of the 

environment in which it is to 

be adopted.

 Identify a subset of your 

population that accurately 

represents the broader body 

of team members.

 When implementing the trialperiod phase of new CRM 

program, plan to measure the 

desired outcomes in the field 

once training is complete.
12

Ending Period

When an organization 

assesses the impact that CRM 

training has had on operations 

as compared to before CRM 

training was established.

 Plan to measure the outcomes 

of CRM adoption once 

training is complete.

 Plan to measure performance 

apart from CRM skills.

 Training phases of the CRM 

program should mirror 

operational constraints of the 

environment in which it will 

be implemented.

After-Action Period

When an organization 

assesses the broad CRM 

training program effectiveness 

by examining the outcomes of 

operations and practices 

within the organization.

 Organizations may utilize 

both internal and external 

reviewers to determine the 

broad implications and 

outcomes of CRM training 

within their company.

 Measurement of employees’

perception of CRM 

effectiveness is an important 

consideration in assessing the 

broad impact of a CRM 

training program.

CRM TRAINING METHODS 

Training courses vary within the industry based on several factors including aircrew, aircraft 

type, financial restraints, company culture, and course developer type (i.e., psychological basis or 

systems designs basis). In addition, training has been conducted using both classroom-based 

training courses and scenario based training courses with “real-world” assessments for both 

medical and aviation domains. In many cases, CRM training is implemented through a 

combination of classroom and simulation training. The medical literature suggests that some 

form of simulation should be woven into CRM courses to improve understanding of the 

principles in real time. The merits of whether these simulations should be traditionally simulated 

(that is, a predetermined fictitious scenario in a dedicated training space) or if training should 

occur “in-situ” (i.e., a predetermined fictitious scenario within the environment in which the 

team actually works) have been debated (Rall, Stricker, Reddersen, Dieckmann, & Conrad, 

2006). Nevertheless, evidence shows the use of simulation-based CRM training provided 

improved feedback, ability to integrate the curriculum, varied difficulty levels of the simulation, 

captured clinical variation, and individualized outcomes, all resulting in more valuable learning 

experiences for the participants (Issenberg, McGaghie, Petrusa, Gordon, & Scalese, 2005). 

Pietsch and Lischke (2014) identified simulator training as a valid method of providing 

coordinated CRM training for mountain rescue teams in Germany even when considering the ad 

hoc nature of team assembly. The authors noted 
13

“...the combined use of helicopters and qualified emergency medical care personnel 

offers substantial evidence to support the meaningfulness of intensive cooperation 

between mountain rescue and air rescue teams” (Pietch & Lischke, 2014). 

While simulation has enhanced the training (Foushee, 1984), the verdict is still out on 

whether simulation training alone is the best route. Several studies have focused on simulation 

vs. real world training methods. Droogh, Kruger, Ligtenberg, and Zijlstra (2012) examined 10 

intensivists and 10 ICU nurses as part of a mobile ICU (MICU) to determine whether training 

with a patient simulator would be a feasible alternative to traditional training methods for 

providing CRM training. Participants reported that the training is useful and that they were more 

confident in their skills after participating in the training (2012). 

A study conducted by Hänsel et al. (2012) examined the effectiveness of a 1.5-day CRM 

course on situational awareness among final-year medical students’ performance in sepsisresuscitation. The authors were concerned about the prevalence (62%–88%) of human error in 

critical incidents in anesthesia and aviation identified in Arnstein’s Catalogue of Human Error

(British Journal of Anaesthesia, 1997). Participants in the experimental conditions were trained 

using either (1) a CRM seminar course with no patient simulator or (2) with only the patient 

simulator. Results concluded that there was no difference in performance between experimental 

and the control group suggesting that simulator training was no better than a seminar based CRM 

training. 

Similarly, Clay-Williams, McIntosh, Kerridge, and Braithwaite (2013) were interested in 

whether simulation-based CRM training would be as effective as traditional classroom-based 

training. The randomized controlled trial included a total of 157 doctors, nurses, and midwives 

who were randomized into four groups including a control group, classroom training only, 

simulation training only, and classroom training followed by simulation training. The results 

concluded there was no statistical significance in performance and attitudes for participants in 

the simulation training-only group as compared to the combined simulation-classroom and 

classroom-only groups. 

Another review compared 12 studies regarding the effectiveness of simulation-based CRM 

versus other educational methods (Fung et al., 2015). The authors were interested in whether 

current research validated simulation-based CRM training as an acceptable alternative to any 

other education intervention. With the exception of one study (Shapiro et al., 2004), all of the 

studies reported significant improvement outcomes related to simulation-based CRM training. 

However, the authors point out that several limitations, including financial and risk of bias, 

should be considered (Fung et al., 2015). 

 While research has not found negative outcomes of CRM, there have been many where 

there was no difference in performance (e.g., Morgan et al., 2015). However, many of these same 
14

studies did not measure performance, had no pre-test comparison, and other similar issues. 

Specific areas in which CRM has shown positive outcomes are outlined in Table 5. 

Table 5. 

Specific areas in which CRM has shown positive outcomes. 

Area of Improvement Studies

Aeronautical decision-making Orasanu, 2010

Team decision-making

Mosier, 1991;

Orasanu, & Salas, 1993;

Cannon-Bowers & Salas, 1998

Conflict resolution Blake and Mouton, 1964;

Dowd, 2010

Coping with internal/external 

pressures

Helmreich, 2010

BACKGROUND ON CRM 

CRM in Healthcare 

Since 1994, the medical community has made strides to reduce the number and frequency of 

errors (Schenkel, 2008). Through this push for safer medical practices, the use of CRM training 

became progressively advanced in theory (see Helmreich & Foushee, 2010 for review) it also 

became more aggressively applied within medical operations (see Musson, 2009 for review). The 

medical community has shared both knowledge and experience regarding challenges and best 

practices with implementing CRM into the daily operation of aviation (Baker, Gustafson, 

Beaubien, Salas, & Barach, 2005). One example of a recent study that examined CRM issues 

identified deficiencies in medicine with frontline nursing in surgical and emergency room 

activities (Sculli, Fore, Neily, Mills, & Sine, 2011). Findings showed that by targeting leadership 

behaviors associated with CRM skills such as coordinated care and mutual respect (i.e., 

communication), overall perceptions of safety were improved. These perceptions have been 

found to be critical in the mitigation of errors (Helmreich, 1991). In a similar study (Young-Xu, 

Fore, Metcalf, Payne, Neily, & Sculli, 2013), a “read-and-do-checklist” was implemented after 

the Institute of Medicine (2000) advised the medical community to implement such procedures 

as have been used in aviation to enhanced operational safety in high stress situations. This 

checklist was provided to personnel with the purpose of improving failure-to-rescue rates in their 

18-bed cardiac step-down unit. Upon implementation of the checklist, the unit’s failure-torescue rates decreased from 25% before intervention to 12% after intervention. These findings 

are not unique throughout the CRM medical literature. Studies have found that implementing 

CRM techniques reduced observed errors (Morey et al., 2002), reduced length intensive care unit 

stays (Pronovost, 2003), improved perceptions of teamwork climate (Sexton, 2006), and reduced 

adverse labor and delivery outcomes (Mann, Marcus, & Sachs, 2006). 

As CRM gained traction in the aviation industry, other industries such as nursing and 

intensive care units have adopted CRM processes into their fields. For example, Sexton, 

Thomas, and Helmreich (2000) found in a study that compared 1,033 medical personnel to over 
15

30,000 aviation personnel found similarities in error reporting and hierarchical structure issues 

across domain. Sculli et al. (2011) also examined the use of CRM in aviation and drew 

similarities between high stress aviation situations that pilots face versus frontline nurses, who 

are generally the first on scene to treat and triage patients. Sculli et al. (2011) suggested 

implementing CRM systems at the frontline of occupations like these may help to alleviate 

threats to performance. 

CRM in Aviation 

Industry operations. The aviation community began implementing CRM when a series of 

aviation accidents were accredited with failures of CRM principles such as situational awareness 

(Florida Everglades crash, 1972, NTSB AAR-73-14). Shortly thereafter, in 1979, NASA held an 

industrial workshop deemed to be the foundation of CRM awareness and CRM principles. 

Psychologists, flight surgeons, academics, and airline pilots shared their experiences and 

knowledge regarding aviation and working toward managing the systems and workload more 

efficiently (Cooper, White, & Lauber, 1980). Smith (1979) examined crew behavior during a 

simulated emergency procedure. These findings revealed that certain crewmembers became 

overwhelmed by tasks necessary to address the emergency. By measuring workload, vigilance 

and decision-making in this study, Smith’s finding became the pinnacle for understanding group 

processes and CRM implementation needs within aviation. Since this time, human factors 

models have been introduced to the aviation industry with the purpose of improving 

communication both within flight crews and between flight and ground crews (Civil Aviation 

Authority, CAP 719, 2002). Since these findings were published, the FAA has released three 

Advisory Circulars (ACs; Dispatch Resources Management Training [DRMT] 121-32A; Crew 

Resource Management Training [CRMT] 120-51E, Air Medical Resource Management 

[AMRM] 00-64; and Helicopter Air Ambulance Operations [HAAO] 135-14B) in which CRM 

implementation requirements in various aviation operations are outlined.

Dispatch and Communication Centers 

Cumulatively, the role of CRM in the aviation and medical field has been studied 

extensively. What has not been studied with as much depth is the unique role that off-site team 

members play in the HAA operation. Studies have examined how air traffic controllers (ATCs) 

may benefit from CRM training (Moon, Yoo, & Choi, 2011), as well as how flight crews on 

board aircraft benefit from CRM (Wagener & Ison, 2014). Likewise, differences between CRM 

training for pilots versus flight nurses versus dispatch may vary widely depending on vocabulary, 

task requirements, expectations, and resources/capabilities of each entity. As the HAAO AC 

states, it is important to provide teams with “shared training [that] provides a common language 

and understanding.” The following papers reviewed the role of dispatches and communication 

centers in the CRM process, specifically where those roles had influence over HAA operations. 

Very little literature examines CRM as it applies to dispatch teams. Crew resource 

management is focused on the entire team and how it interacts during crisis; however, the focus 

has mainly been in the cockpit, neglecting the importance of effective dispatch. Grannan, 
16

Bastian, and McLay examined military dispatchers who were tasked with locating HAA air 

assets to respond to casualty situations (2015). To assist the dispatchers, Grannan et al. (2015) 

developed a binary linear programming (BLP) model to optimally choose among two different 

types of air assets. Grannan et al. introduced the BLP model that optimally locates two classes of 

air assets and assigns to casualty locations, such as military treatment facilities (MTF), using a 

dispatch preference list. The list is essentially an ordered ranking for each location. The BLP 

also balances workload and enforces contiguity among first assigned locations for air assets. 

The FAA has identified the aircraft dispatcher as an integral part of flight operations which 

can positively influence safety (FAA, AC 121-32A, pg. 5). Advisory Circular 121-32A, Dispatch 

Resource Management Training, was published in 2005 in order to integrate CRM with dispatch 

operations to form a formal dispatch resource management (DRM) training curriculum. 

According to the AC, DRM focuses on areas such as situational awareness and team building 

and optimizes communication between the various airline operations groups to form a cohesive 

team. DRM consists of three components: “initial indoctrination/awareness, recurrent practice 

and feedback, and continual reinforcement” (FAA, 2005). The FAA notes that most problems 

within flight and dispatch operations are the results of poor group decision-making, 

communication problems, inadequate leadership, and poor task/resource management (pg. 4). 

The focus of DRM training is to transition from the traditional focus on “how-to” operations to 

incorporate solid group relations between air and ground operations. While this AC provides 

guidance on the purpose and scope of dispatch CRM, there are still gaps that exist within HAA 

operations. For example, the composition of the dispatch team varies depending on number of 

aircraft and operation type. Dispatch teams can also be comprised of traditional communications 

centers who fall under FAA regulatory authority but may also be comprised of personnel who act 

on behalf of the air operator for certain duties (e.g., flight planning, deconfliction procedures). 

Currently, the FAA has no approach for identifying or assisting those dispatchers who perform 

duties outside of regulatory oversight. 

Gaps Specific to HAA 

Given the body of research that currently exists on CRM within both the medical and 

aviation field, this literature review identified only the key pillars of the concept. The focus of 

this research was placed on identifying where possible gaps may exist between the current 

understanding of CRM knowledge, the implementation of that knowledge, and the effectiveness 

of the principles in HAA operations. From this perspective, we were able to glean three areas of 

concern: the “mission mindset” issue, flight crew CRM, and dispatch CRM. 

The “Mission Mindset.” As previously indicated in Table 1, it has been shown that 10 of 34 

accidents investigated from 1988 to 1994 were associated with a “mission mindset” (HAAT, 

1998; e.g., a sense of urgency leading to risk-taking). Indeed, the structure of HAA operations 

lends itself to this mindset in several ways. First, the very nature of HAA operations is such that 

a patient is transported in one of a two ways: either from one hospital to another (hospital to 

hospital transfer), or from the scene of an incident/accident to a hospital (scene-call). A hospital 

to hospital transfer generally occurs when a patient is transferred to a receiving hospital because 
17

the sending hospital does not have adequate resources to meet the needs of a patient. When a 

scene-call occurs, the helicopter meets first responders and emergency response teams at a scene 

to transport a patient to a hospital after an accident/incident (e.g., skiing, hiking, car accident, 

etc). In both of these scenarios, a patient is moved via helicopter because time to transport is 

shortened both by providing a more direct route and because traffic encounters are greatly 

reduced (i.e., ground transport). Research has shown that in situations where time is a factor, 

stress becomes increased (Keinan, 1988).

When stress is increased, cognitive resources are reduced such that decision-making 

behaviors are impaired resulting in less risk-based consideration. It is common for pilots to 

experience reduced situational awareness, decreased decision-making capability, and diminished 

communication abilities in times of stress (Hancock & Desmond, 2001). Therefore, it is not 

surprising that a pilot’s perception of an HAA operation may be focused on “the mission”

resulting in reduced risk assessment and thus risky behaviors. For example, the scenario 

described at the beginning of this paper cited an incident in which the pilot informed the 

company’s EMS communication center that he did not have enough fuel to complete the second 

leg of the mission during patient pickup. While it is difficult to determine whether the “mission 

mindset” was primarily responsible for the subsequent fatal accident, it is clear that ensuring the 

aircraft has enough fuel to complete the operation is a necessary step in a pilot’s pre-flight 

checklist. This example highlights how the actions of a pilot may be influenced by perceptions of 

urgency and how this may contribute to greater risk taking. 

Crew CRM. The composite crew of an HAA varies dependent on patient needs. Typically, a 

crew consists of the pilot in control (PIC) and two medical crew members. The medical crew 

members can be made up of a flight nurse, flight paramedic, and/or a specialty nurse. When it is 

necessary to use large equipment to sustain the life of a patient, the crew can be comprised of 

just the PIC and one medical crew member. The variation of these crew characteristics 

introduces risk to scenarios where medical crew members are relied upon to assist with take-off 

and landing protocols. Indeed, the extent to which the crew members are trained in principles 

specific to aviation, flight, and CRM is unreported within the literature. The AMRM AC 

guidance suggests that medical crews be trained in accordance with the company’s needs and 

identifies topics to be covered with expected outcomes of the training. However, the appropriate 

level of overlap needed to bridge the possible knowledge gap between medical personnel and the 

PIC of the operation is not addressed. The ambiguity of this guidance works in favor of industry, 

as it allows for companies to individualize their training to best meet the needs of their unique 

structure. However, due to just how unique the structure of HAA operations is, it is possible that 

the medical crews are not experiencing domain-specific training in aviation that will improve 

their performance as flight crew or ensure flight safety. Incomplete training for medical crew and 

ground crew could lead to the transfer of inaccurate information and potentially increase risk to 

the operation. For example, medical crew using night-vision goggles without proper training can 

exacerbate the overall risk of the operation by the creation of new hazards. In addition, the 

CRMT AC highlights that having a shared vocabulary between crew members is an essential part 
18

of successful implementation of CRM principles. It is currently unclear if these common 

vocabularies are being obtained by HAA operation crew members or if the training they 

experience was created with such an objective in mind. Furthermore, the lack of uniformity with 

which CRM training has been conducted between the aviation industry and the health care 

industry makes HAA especially susceptible to risk. For example, the finding that most CRM 

training programs implemented in the aviation industry are based in a TEM approach (Heilmrich, 

Klinect & Wilhelm, 1999) is not a finding true of the health care industry. Differences such as 

these present risk in HAA operations because PIC and health care crew may not have a shared 

understanding of TEM practices or responsibilities. To this end, more research is needed to 

determine what, if any, knowledge gaps may exist between medical crew and the PIC, the extent 

to which these gaps may be closed, and if there are current training programs that address these 

issues successfully. 

Dispatch. Similar to the identified gap regarding medical crew knowledge and PIC 

knowledge, also unclear is the degree to which dispatch knowledge of flight-related tasks plays a 

role in successful CRM implementation. For example, it is important to know what the 

credentials of and training requirements are for current dispatch personnel. Each of these criteria 

should be addressed based on the task requirements of the dispatch personnel and what effect his 

has on flight safety. Additionally, it is important to know if there are existing training programs 

that address these issues successfully. Determining if there are current CRM related flight safety 

issues that dispatchers face while augmenting HAA operations would assist with enhancing 

training programs that currently exist while mitigating risks. 

Finally, with regard to both crew and dispatch gaps in the literature, one of the most glaring 

issues is the unknown quality and content provided by training programs across the industry. 

That is, to what degree do these training programs address the behavioral markers identified as 

the pillars of CRM? How has this training been implemented by organizations? What 

measureable changes have come about since the implementation of such programs? What 

perceptions do flight crews hold of the importance of CRM? Are those perceptions reinforced or 

challenged by training curriculum provided by their organizations? Each of these questions 

addresses some aspect of the overall quality of the CRM training programs and ultimately 

determines if knowledge will be transferred from training into field operations. Identifying 

components of successful CRM programs and how they impact flight safety within the flight 

crew team as well as those in augmenting roles is a natural next step to enhancing the safety of 

HAA operations with regard to current CRM concerns. 

Recommended Next Steps 

The following steps are recommended as a process for beginning to measure the impact 

of FAA published CRM guidance and rules in HAA operations. These recommendations address 

the current gaps identified from the literature review while assessing the current status of CRM 

training programs across the industry. 
19

 Develop a plan of research that assesses pilots’ risk-based decision making processes 

while engaged in HAA operations. 

 Assess stress experienced by pilots while conducting operations.

 Assess risk assessment by pilots while conducting operations. 

 Develop a plan of research that assesses the gaps that exist in medical personnel’s 

knowledge of aviation and flight safety amid medical responsibilities while involved in 

HAA operations.

 Determine whether gaps in medical personnel’s responsibilities and knowledge of 

aviation and flight safety play a role in the successful implementation of CRM principles. 

 Develop a plan of research that assesses the gaps that exist between supporting personnel 

(i.e., dispatch, communication centers, etc.) responsibilities and knowledge of aviation 

and flight safety amid dispatch responsibilities while augmenting HAA operations.

 Determine whether gaps in supporting personnel’s responsibilities and knowledge of 

aviation and flight safety play a role in the successful implementation of CRM principles. 

 Develop a plan to assess the organizational safety culture and its ability to embrace CRM 

principles. 

CONCLUSION 

HAA accidents are rising (NTSB/SIR-06/01) and have typically been associated with CRMrelated issues (79 FR 9931). It is the FAA’s responsibility to address these challenges by 

assessing the previously established rules and guidance by measuring industry practices of 

implementation. Following up published rules and guidance with measurement ensures a 

commitment to the safety of the National Airspace System by continuing to monitor progress 

toward our safety goals. From this review, it is clear that CRM principles are alive and well in 

the aviation and medical communities. However, it is unclear how well those behavioral markers 

are taught, trained, and then transferred to the operational environment. The assessment of such 

issues meets a research need that would gauge the effectiveness of current CRM guidance and 

rules within HAA operations while providing feedback and resources to industry regarding their 

current CRM training programs.

REFERENCES 

Airbus. (2004). Flight operations briefing notes: Human performance CRM aspects in 

incidents/accidents. Retrieved from http://www.skybrary.aero/bookshelf/books/171.pdf 

Arnstein, F. (1997). Catalogue of human error. British Journal of Anaesthesia, 79, 645–656.

Baker, D., Gustafson, S., Beaubien, J., Salas, E., & Barach, P. (2005). Medical teamwork and 

patient safety: The evidence-based relation. Agency for Healthcare Research and Quality, 

Rockville, MD. Retrieved from http://archive.ahrq.gov/research/findings/finalreports/medteam/index.html 
20

Cahil, J., McDonald, N., & Losa, G. (2014). A sociotechnical model of the flight crew task. 

Human Factors, 56(8), 1337–1363.

Cannon-Bowers, J. A., & Salas, E. (1998). Individual and team decision making under stress: 

Theoretical underpinnings. In J. A. Cannon-Bowers & E. Salas (Eds.), Making decisions 

under stress: Implications for individual and team training. (17–38). Washington, DC: 

APA Press. 

Clay-Williams, R., Greenfield, D., Stone, J., & Braithwaite, J. (2014). On a wing and a prayer: 

An assessment of modularized crew resource management training for health care 

professionals. Journal of Continuing Education in the Health Professions, 34(1), 56–67. 

Clay-Williams, R., McIntosh, C., Kerridge, R., & Braithwaite, J. (2013). Classroom and 

simulation team training: A randomized controlled trial. International Journal for Quality 

in Health Care, 25(3), 314–321.

Chidester, T. (2016). Creating a culture of safety. In K.J. Ruskin, M.P. Stigler, & S.H. 

Rosenbaum, Quality and Safety in Anesthesia and Perioperative Care. London: Oxford 

University Press. 

Civil Aviation Authority. (2002). CAP 719 – Fundamental human factors concepts. Previously 

ICAO Digest No. 1. Retrieved from http://publicapps.caa.co.uk/docs/33/CAP719.PDF 

Civil Aviation Authority. (2002). CAP 720 – Flight crew training: Cockpit resource management 

(CRM) and line-oriented flight training (LOFT). Previously ICAO Digest No. 2. 

Retrieved from http://www.skybrary.aero/bookshelf/books/890.pdf 

Cooper, G. E., White, M. D., & Lauber, J. K. (1980). Resource management on the flightdeck: 

Proceedings of a NASA/industry workshop. Moffett Field, Calif: NASA - Ames Research 

Center; 1980. NASA Conference Publication No. CP-2120.

Droogh, J. M., Kruger, H. L., Ligtenberg, J. J., & Zijlstra, J. G. (2012). Simulator-based crew 

resource management training for inter-hospital transfer of critically ill patients by a 

mobile ICU. The Joint Commission Journal on Quality and Patient Safety, 38(12), 554–

559.

Drury, D.A., Dorrian, J., Ferguson, S.A., & Thomas, M.J.W. (2013). Detection of heightened 

emotional activity in commercial airline crews: A reliability study. Aviation Psychology 

and Applied Human Factors, 3(2), 83–91. 

FAA TV. (2012, April 5). The History of CRM [Video file]. Retrieved from 

https://www.faa.gov/tv/?mediaId=447 

Federal Aviation Administration. (2004) Advisory circular: Crew resource management training

(AC No. 120-51E, Appendix 1). Retrieved from 

http://www.faa.gov/documentLibrary/media/Advisory_Circular/AC120-51e.pdf 
21

Federal Aviation Administration. (2005). Advisory circular: Dispatch resource management 

training (AC No. 121-32A). Retrieved from 

http://www.faa.gov/documentLibrary/media/Advisory_Circular/AC_121-32a.pdf 

Federal Aviation Administration. (2014). FAA Initiatives to Improve Helicopter Air Ambulance 

Safety [Fact sheet]. Retrieved from 

http://www.faa.gov/news/fact_sheets/news_story.cfm?newsId=15794 

Federal Aviation Administration. (2015). Advisory circular: Helicopter air ambulance operation

(AC No. 135-14B). Retrieved from 

http://www.faa.gov/regulations_policies/advisory_circulars/index.cfm/go/document.infor

mation/documentid/1027108 

Federal Register (2014). Helicopter air ambulance, commercial helicopter, and Part 91 helicopter 

operations. Final Rule [RIN 2120-AJ53] by FAA, 02/21/2014. Retrieved from 

https://www.federalregister.gov/articles/2014/02/21/2014-03689/helicopter-airambulance-commercial-helicopter-and-part-91-helicopter-operations 

Flin, R., & Martin, L. (2001). Behavioral markers for crew resource management: A review of 

current practice. The International Journal of Aviation Psychology, 11(1), 95–118.

Flight Safety Foundation. (2009). FSF ALAR briefing note 2.2: Crew resource management.

Retrieved from http://www.skybrary.aero/bookshelf/books/851.pdf 

Foushee, H. C. (1984). Dyads and triads at 35,000 feet: Factors affecting group processes and 

aircrew performance. American Psychologist, 39(8), 885–893.

Fung, L., Boet, S., Bould, M.D., Qosa, H., Perrier, L., Tricco, A.,...& Reeves, S. (2015). Impact 

of crisis resource management simulation-based training for inter-professional and 

interdisciplinary teams: A systematic review. Journal of Inter-professional Care, 29(3) 

1–12.

Grannan, B.C., Bastian, N.D., & McLay, L.A. (2015). A maximum expected covering problem 

for locating and dispatching two classes of military medical evacuation air assets. 

Optimization Letters, 9(8), 1511–1531.

Haller, G., Garnerin, P., Morales, M., Pfister, R., Berner, M., Irion, O.,...& Kern, C. (2008). 

Effect of crew resource management training in a multidisciplinary obstetrical setting. 

International Journal for Quality in Health Care, 20(4), 254–263. 

Hancock, P. A., & Desmond, P. A. (2001). Stress, workload and fatigue. Erlbaum, Mahwah, NJ. 

Hänsel, M., Winkelmann, A.M., Hardt, F., Gijselaers, W., Hacker, W., Stiehl, M.,...& Müller, 

M.P. (2012). Impact of simulator training and crew resource management training on 

final-year medical students' performance in sepsis resuscitation: A randomized trial. 

Minerva Anestesiologica, 78(8), 901–909. 
22

Helmreich, R. L. (1991). Outcomes of crew resource management training, The International 

Journal of Aviation Psychology, 1(4), 287–300. 

Helmreich, R. L. (1997). Managing human error in aviation. Scientific American, 276(5), 62–67. 

Helmreich, R. L. (1998). Error management as organisational strategy. Proceedings of the IATA 

Human Factors Seminar (pp. 1-7). Bangkok, Thailand: International Air Transport 

Association. 

Helmreich, R.L., Klinect, J.R., & Wilhelm, J.A. (1999). Models of threat, error, and CRM in 

flight operations. In Proceedings of the Tenth International Symposium on Aviation 

Psychology. Columbus, OH: The Ohio State University. 

Helmreich, R. L., Merritt, A. C., & Wilhelm, J. A. (1999). The evolution of crew resource 

management training in commercial aviation. International Journal Of Aviation 

Psychology, 9(1), 19. 

Helmreich, R.L., & Foushee, H.C. (2010). Why crew resource management? Empirical and 

theoretical bases of human factors training in aviation. In, E. Wiener, B. Kanki, & R. 

Helmreich (Eds.), Cockpit resource management (3–45). San Diego, CA: Academic 

Press. 

Helicopter Accident Analysis Team (HAAT). (1998). The final report of the Helicopter Accident 

Analysis Team. Retrieved from http://www.ihst.org/portals/54/industry_reports/HAATFinalReport.pdf 

Issenberg, S.B., Mcgaghie, W., Petrusa, E., Gordon, D., & Scalese, R. (2005). Features and uses 

of high-fidelity medical simulations that lead to effective learning: A BEME systematic 

review. Medical Teacher, 27(1), 10–28. 

Joint Aviation Authorities. (2007). JAR-OPS 3: Commercial Air Transportation (Helicopters).

Englewood, CO: Global Engineering Documents. 

Kanki, B., & Helmreich, R. (2010). Why CRM? Empirical and theoretical bases of human 

factors training. In, Kanki, B., Helmreich, R., & Anca, J. (Eds.), Crew resource 

management. Academic Press. Oxford, UK. 

Keinan, G. (1988). Training for dangerous task performance: The effects of expectations and 

feedback. Journal of Applied Social Psychology, 18(4), 355–373. 

Kemper, P.F., van Dyck, C., Wagner, C., & de Bruijne, M. (2014). Implementation of crew 

resource management: A qualitative study in 3 intensive care units. Journal of Patient 

Safety, 00(00), 1–9. 

Kemper, P.E., van Dyck, C., Wagner, C., Wouda, L., & de Bruijne, M. (2014). Barriers and 

facilitators for taking action after classroom-based crew resource management training at 

three ICUs. Joint Commission Journal on Quality and Patient Safety, 40(7), 311–318.
23

Mann, S., Marcus, R., & Sachs, B. (2006). Lessons from the cockpit: How team training can 

reduce errors on L&D. Contemporary OB/GYN, 51(1), 34. 

Meurling, L., Hedman, L., Sandahl, C., Felländer-Tsai, L., & Wallin, C.J. (2013). Systematic 

simulation-based team training in a Swedish intensive care unit: A diverse response 

among critical care professions. BMJ Quality & Safety, 22(6), 485–494. 

Moon, W. C., Yoo, K. E., & Choi, Y. C. (2011). Air traffic volume and air traffic control human 

errors. Journal of Transportation Technologies, 1, 47–53.

Morey, J., Simon, R., Jay, G., Wears, R., Salisbury, M., Dukes, K., & Berns, S. (2002). Error 

reduction and performance improvement in the emergency department through formal 

teamwork training; Evaluation results of the MedTeams project. Health Services 

Research Journal, 37(6), 1153–1581.

Morgan, L., Pickering, S.P., Hadi, M., Robertson, E., New, S., Griffin, D., Collins, G., RiveroArias, O., Catchpole, K., & McCulloch, P. (2015). A combined teamwork training and 

work standardization intervention in operating theatres: Controlled interrupted time series 

study. BMJ Quality & Safety, 24(2), 111–119. 

Mosier, K. (1991). Expert decision-making strategies. Paper presented at International 

Symposium on Aviation Psychology: 6th, April 29–May 2, 1991: Columbus, OH. 

Musson, D. (2009). Putting behavioural markers to work: Developing and evaluating safety 

training in healthcare settings. In, Flin, R., & Mitchell, L. (Eds.), Safer surgery: 

Analysing behaviour in the operating theatre. Ashgate. Farnham, UK. 

National Transportation Safety Board. (1972). Eastern Air Lines, Inc., L-1011, N310EA, Miami, 

Florida, December 29, 1972. Aircraft Accident Report AAR-73-14. Washington, DC: 

NTSB. 

National Transportation Safety Board. (2006). Aviation special investigation report: Special 

Investigation Report on Emergency Medical Services Operations. NTSB/SIR-06/01. 

Washington, DC: NTSB. 

National Transportation Safety Board (2009). Four safety recommendation letters concerning 

helicopter emergency medical services. Public Meeting of September 1, 2009. Retrieved 

from http://www.ntsb.gov/news/events/Pages/109.aspx 

National Transportation Safety Board. (2013). Crash following loss of engine power due to fuel 

exhaustion, Air Methods Corporation, Eurocopter AS350 B2, N352LN, Near Mosby, 

Missouri, August 26, 2011. Aircraft Accident Report AAR-13/02. Washington, DC: 

NTSB. 

National Transportation Safety Board. (2015). Safety Recommendation A-13-015. Washington, 

DC: NTSB. Retrieved from 

http://www.ntsb.gov/investigations/AccidentReports/_layouts/ntsb.recsearch/Recommen

dation.aspx?Rec=A-13-015 
24

O'Dea, A., O'Connor, P., & Keogh, I. (2014). A meta-analysis of the effectiveness of crew 

resource management training in acute care domains. Postgraduate Medical Journal, 

90(1070), 699–708. 

Orasanu, J., & Salas, E. (1993). Team decision making in complex environments. In, Klein, G., 

Orasanu, J., Calderwood, R., & Zsambok, C. E. (Eds.), Decision making in action: 

Models and methods (327–345). Norwood, NJ: Ablex. 

Orasanu, J. M. (2010). Flight crew decision-making. In, Kanki, B., Helmreich, R., & Anca, J. 

(Eds.), Crew resource management. Academic Press. Oxford, UK. 

Pietsch, U., & Lischke, V. (2014). Medical simulation training of helicopter-supported mountain 

rescue situations. AirRescue, 4, 16–19. 

Pronovost, P., Berenholtz, S., Dorman, T., Lipsett, P., Simmonds, T., & Haraden, C. (2003). 

Improving communication in the ICU using daily goals. Journal of Critical Care, 18(2), 

71–75.

Rall, M., Reddersen, S., Stricker, E., Zieger, J., Dieckmann, P., & Conrad, G. (2006). Evaluation 

of a mobile “in-situ” simulation training course for teams (paramedics and doctors) in 

wing-based emergency air rescue and intensive care using subjective pre-post 

competency ratings. Simulation in Healthcare, 1(2), 140.

Reason, J. (2008). The human contribution: Unsafe acts, accidents and heroic 

recoveries. Burlington, VT: Ashgate. 

Royal Aeronautical Society. (2009). Crew Resource Management: A paper by the CRM standing 

group of the Royal Aeronautical Society. Retrieved from 

http://www.skybrary.aero/bookshelf/books/232.pdf 

Salas, E., DiazGranados, D., Klein, C., Burke, C. S., Stagl, K. C., Goodwin, G. F., & Halpin, S. 

M. (2008). Does team training improve team performance? A meta–analysis. Human 

Factors, 50(6), 903–933. doi:10.1518/001872008X375009 

Sandahl, C., Gustafsson, H., Wallin, C., Meurling, L., Øvretveit, J., Brommels, M., & Hansson, 

J. (2013). Simulation team training for improved teamwork in an intensive care unit. 

International Journal of Health Care Quality Assurance, 26(2), 174 – 188. 

Schenkel, S. (2000). Promoting patient safety and preventing medical error in emergency 

departments. Academic Emergency Medicine, 7(11), 1204–1222. 

Sculli, G., Fore, A., Neily, J. Mills, P., & Sine, D. (2011). The case for training veterans 

administration frontline nurses in crew resource management. Journal of Nursing 

Administration, 41(12), 524–530.

Sexton, J. B., Makary, M., Tersigni, A., Pryor, D., Hendrich, A., Thomas, E.,...& Pronovost, P. 

(2006). Teamwork in the operating room: Frontline perspectives among hospitals and 

operating room personnel. Anesthesiology, 105, 877–884.
25

Sexton, J. B., Thomas, E. J., & Helmreich, R. L. (2000). Error, stress, and teamwork in medicine 

and aviation: Cross sectional surveys. BMJ, 320, 745–749.

Shapiro, M. J., Morey, J. C., Small, S. D., Langford, V., Kaylor, C., J., Jagminas, L.,... Jay, G.D. 

(2004). Simulation based teamwork training for emergency department staff: Does it 

improve clinical team performance when added to an existing didactic teamwork 

curriculum? Quality & Safety in Health Care, 13, 417–421. 

Smith, R. (1979). NASA technical memorandum 78482: A simulator study of the interaction of 

pilot workload with errors, vigilance, and decisions. NASA Ames Research Center, 

Moffett Field, CA. 

Sweeney, L., Warren, O., Gardner, L., Rojek, A., & Lindquist, D. G. (2014). A simulation-based 

training program improves emergency department staff communication. American 

Journal of Medical Quality, 29(2), 115–123. 

U.S. Government Accountability Office. (2009). Testimony before the subcommittee on aviation, 

committee on transportation and infrastructure, house of representatives: Aviation safety 

– Potential strategies to address air ambulance safety concerns (Publication No. GAO090627T). Retrieved from http://www.gao.gov/new.items/d09627t.pdf 

Wagener, F., & Ison, D. (2014). Crew resource management application in commercial aviation. 

Journal of Aviation Technology and Engineering, 3(2), 2–13. 

Weaver, S., Rosen, M., DiazGranados, D., Lazzara, E., Lyons, R., Salas, ...& King, H. (2010). 

Does teamwork improve performance in the operating room? A multi-level evaluation. 

The Joint Commission Journal on Quality and Patient Safety, 36(3), 133–142. 

Yang, J.H., Cowden, B.T., Kennedy, Q., Schramm, H., & Sullivan, J. (2013). Pilot perception 

and confidence of location during a simulated helicopter navigation task. Aviation Space 

Environment Medicine, 84(9), 952–960. 

Young-Xu, Y., Fore, A.M., Metcalf, A., Payne, K., Neily, J., & Sculli, G.L. (2013). Using crew 

resource management and a 'read-and-do checklist' to reduce failure-to-rescue events on a 

step-down unit. The American Journal of Nursing, 113(9), 51–57.