e 

 

U.S. Department 

of Transportation 

Federal Railroad 

Administration 

Rail Crew Resource Management (CRM): 

The Business Case for CRM Training in the 

Railroad Industry 

Office of Research and 

Development and Office of Safety 

Washington, DC 20590 

DOT/FRA/ORD-07/21

 

September 2007

Final Report 

 

This document is available to the 

U.S. public through the National

Technical Information Servic

Springfield, VA 22161. 
Notice 

This document is disseminated under the sponsorship of the 

Department of Transportation in the interest of information 

exchange. The United States Government assumes no liability

for its contents or use thereof. 

Notice 

The United States Government does not endorse products or 

manufacturers. Trade or manufacturers’ names appear herein 

solely because they are considered essential to the objective of 

this report. 
Form approved REPORT DOCUMENTION PAGE OMB No. 074-0188 

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, 

searching existing data sources, gather and maintaining the data needed, and completing and reviewing the collection of information. Send 

comments regarding this burden estimate or any other aspect of this collection if information, including suggestions for reducing this burden to 

Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 

22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0702-0288), Washington, D.C. 20503 

1. AGENCY USE ONLY (Leave Blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED 

September 2007 Research Report 

October 2004–January 2006 

4. TITLE AND SUBTITLE 5. FUNDING NUMBERS 

Railroad Crew Resource Management (CRM): The Business 

Case for CRM Training in the Railroad Industry 

6. AUTHOR(S) 

Stephen S. Roop, Curtis A. Morgan, Tobin B. Kyte, Winfred 

Arthur, Jr., Anton J. Villado, and Theodore Beneigh 

8. PERFORMING ORGANIZATION REPORT 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) NUMBERS 

Texas Transportation Institute 

The Texas A&M University System Final Report 

College Station, TX 77843-3135 

10. SPONSORING/MONITORING AGENCY 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) REPORT NUMBER 

U.S. Department of Transportation 

Federal Railroad Administration DOT/FRA/ORD-07/21 1120 Vermont Avenue, NW 

Washington, DC 20590 

11. SUPPLEMENTARY NOTES 

COTR: Scott Kaye, Federal Railroad Administration 

12a. DISTRIBUTION/AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE 

This document is available through National Technical 

Information Service, Springfield, VA 22161. 

13. ABSTRACT 

 Crew Resource Management (CRM) is a human factors training process that has been employed in the commercial 

aviation industry for over 25 years. During that time period, CRM has been credited with contributing to a marked 

decrease in human factors-caused accidents. Military teams, commercial shipping crews, surgical teams, nuclear power 

operators, and offshore drilling crews have all since employed forms of CRM training to address relative increases in 

human factors accidents compared to mechanical- or equipment-based accident causes. 

 This study uses utility analysis to quantify the anticipated benefits to the railroad industry if CRM training were to be 

more broadly adopted. The research team tested the utility analysis model using collected airline industry data and then 

applied it to actual and estimated data from the railroad industry. The study found that CRM training can be expected 

to have net positive benefits at both the industry and individual railroad level by reducing the overall costs associated 

with human factors accidents. This result was derived by taking into account mean values for the number of human 

factors accidents, number trained, reported costs of accidents, and costs of training. Additional benefits from improved 

crew coordination and cost savings from reduced litigation, while not quantified in this study, would add to the overall 

benefits of sustained railroad CRM training programs. 

14. SUBJECT TERMS 15. NUMBER OF PAGES 

Crew resource management, CRM, railroad safety, situational awareness, 126 organizational culture, teams, teaming, crew, training methods 

16. PRICE CODE 

17. SECURITY CLASSIF. 18. SECURITY 19. SECURITY CLASSIF. OF 20. LIMITATION OF 

CLASSIF. OF THIS ABSTRACT ABSTRACT 

PAGE 

UNCLASSIFIED UNCLASSIFIED UNCLASSIFIED 

NSN 7540-01-280-550 Standard Form 298 (Rec. 2-89) 

Prescribed by ANSI/NISO Std. 

239.18 

298-102 

 i 
 ii 
 iii 

Table of Contents 

List of Figures ................................................................................................................................. v

List of Tables ................................................................................................................................. vi

Acknowledgments......................................................................................................................... vii

Executive Summary ........................................................................................................................ 1

1.0 Introduction to Railroad CRM ................................................................................................. 3

1.1 Background and History of CRM Training ...................................................................... 3

1.2 CRM in the Railroad Industry .......................................................................................... 3

1.3 History of CRM Training in the Railroad Industry .......................................................... 4

1.4 Research Methodology ..................................................................................................... 9

2.0 Utility Analysis of CRM Training Effectiveness in Preventing Human 

Factors-Caused Accidents ..................................................................................................... 11

2.1 Overview of Utility Analysis Methodology ................................................................... 11

2.2 Obtaining or Estimating Airline Industry Utility Analysis Parameters ......................... 12

2.3 Airline Utility Analysis Parameters ............................................................................... 12

2.4 Airline Industry Utility Analysis Results ....................................................................... 16

2.5 Obtaining or Estimating Railroad Industry Utility Analysis Parameters ....................... 17

2.6 Railroad Utility Analysis Parameters ............................................................................. 18

2.7 Railroad Industry Utility Analysis Results ..................................................................... 20

2.8 Utility Analysis Results for a Typical Large Class I Railroad ....................................... 21

3.0 Applying Commercial Aviation’s Lessons Regarding CRM to the Railroad Industry ......... 23

3.1 Differences Between the Airline and Railroad Industries .............................................. 23

3.2 Comparison of Airline and Railroad Work Environments ............................................. 23

3.3 Scope of CRM Training in the Railroad versus the Airline Industry ............................. 32

4.0 Conclusions and Recommendations ...................................................................................... 39

4.1 Conclusions .................................................................................................................... 39

4.2 Recommendations .......................................................................................................... 40

References ..................................................................................................................................... 43

Acronyms ...................................................................................................................................... 51

Appendix A. History/Background of CRM Training ................................................................ A-1

Appendix B. Commercial Aviation Industry Interview Form ................................................... B-1

Appendix C. Airline Industry Numerical Calculation Explanations ......................................... C-1

Appendix D. Railroad Industry Interview Form ........................................................................ D-1

Appendix E. Railroad Industry Numerical Calculation Explanations ........................................ E-1

Appendix F. Sensitivity Analysis of Commercial Aviation Industry Utility.............................. F-1

Appendix G. Sensitivity Analysis of Railroad Industry Utility ................................................. G-1

Appendix H. Sensitivity Analysis of Typical Class I Utility ..................................................... H-1
 iv 
 v 

List of Figures 

Figure 1. Percentage of Primary Accident Causes 2000-2004 ...................................................... 5 

Figure 2. Train Accidents per Million Train-Miles 1980-2004 ..................................................... 6 

Figure 3. Track-Caused Accidents per Million Train-Miles 1980-2004 ....................................... 7 

Figure 4. Human Factors-Caused Accidents per Million Train-Miles 1980-2004 ........................ 7 

Figure 5. Broad Causes/Factors for Part 121 Accidents 1992-2001 .............................................. 8 
 vi 

List of Tables 

Table 1. Utility Analysis Results for Airline Industry Based on Average Parameter 

 Estimates ........................................................................................................................ 17

Table 2. Utility Analysis Results for Railroad Industry Based on Average Parameter 

 Estimates ........................................................................................................................ 21

Table 3. Utility Analysis Results for a Typical Large Class I Railroad Based on Average 

Parameter Estimates ....................................................................................................... 22

Table 4. Top Four Airline Pilot Tasks Groups and Corresponding Locomotive Engineer 

 and Conductor Task Groups .......................................................................................... 25

Table 5. Top Six Airline Pilot Work Activities and Corresponding Relative Importance 

 Scores for Locomotive Engineers and Conductors ........................................................ 26

Table 6. Top Six Airline Pilot Work Context Factors and Corresponding Relative 

 Importance Scores for Locomotive Engineers and Conductors .................................... 27

Table 7. Top Six Airline Pilot Knowledge and Corresponding Relative Importance 

 Scores for Locomotive Engineers and Conductors ........................................................ 29

Table 8. Top Six Airline Pilot Skills and Corresponding Relative Importance Scores for 

Locomotive Engineers and Conductors ......................................................................... 30

Table 9. Top Six Airline Pilot Abilities and Corresponding Relative Importance Scores for 

Locomotive Engineers and Conductors ......................................................................... 31
vii 

Acknowledgments 

The authors would like to thank Scott Kaye and Ralph Elston, Federal Railroad Administration 

(FRA) Office of Safety, for their support in researching Crew Resource Management (CRM) 

issues in the rail industry. Additionally, we would like to thank Dr. Thomas Raslear, FRA Office 

of Research and Development Human Factors Research Program, for his past support and the 

CRM subject matter experts at the participating airline companies and the railroad company that 

we visited for sharing their experience and information on their specific CRM training programs 

with us. We also would like to thank Les Olson, Jessica Franklin, and Sharon Gillespie, Texas 

Transportation Institute, and those in the Texas A&M community for their help and support in 

completing this project, including Sharon Thigpin and Marci Avery at the Texas A&M Research 

Foundation. 
viii 
1 

Executive Summary 

Purpose of this Report 

Crew Resource Management (CRM) is a human factors training process that has been employed 

in the commercial aviation industry for over 25 years. During that time period, CRM has been 

credited with contributing to a marked decrease in human factors-caused accidents. Military 

teams, commercial shipping crews, surgical teams, nuclear power operators, and offshore drilling 

crews have all since employed forms of CRM training to address relative increases in human 

factors-caused accidents compared to mechanical failure-caused accidents. The National 

Transportation Safety Board (NTSB) officially recommended CRM training for railroad crews in 

1999, and a few CRM programs in limited form have recently been introduced at several 

railroads. 

This study uses utility analysis, a statistical measure of the value gained by implementing a 

specific program, to quantify the anticipated benefits to the railroad industry if CRM training 

were to be more broadly adopted. Researchers tested the utility analysis model using collected 

airline industry data and then applied it to actual and estimated data from the railroad industry. 

The study found that CRM training can be expected to have net positive benefits at both the 

industry and individual railroad level by reducing the overall costs associated with human factors 

accidents. This result was derived by taking into account average values for the number of 

human factors accidents, number of employees trained, costs of accidents, and costs of training. 

Additional benefits from improved crew coordination and cost savings from reduced litigation, 

while not quantified in this study, would add to the overall benefits of sustained railroad CRM 

training programs. 

Methodology 

Procedures for quantifying and measuring the financial gains associated with the implementation 

of human resource management interventions, such as training, are not easy to apply, but the 

technology to do so is available and well developed in the field of industrial-organizational 

psychology. Specifically, this can be accomplished using utility analysis, a set of statistical 

procedures that allow the assessment of the dollar gains to an organization resulting from the use 

or implementation of a specified human resource management (HRM) intervention (Cascio, 

2000), in this case, CRM training. To accomplish this activity for the rail industry required that 

the team first apply the utility analysis model to the airline industry where CRM has been 

implemented for the longest time and with a known history. Researchers interviewed several 

airline industry subject matter experts (SMEs) to develop realistic estimates for several 

parameters, such as the number of personnel trained, cost per trainee, accident costs, and the 

duration of the training effect. Then, using derived or estimated values for the railroad industry, 

the utility analysis model was applied to estimate the potential benefits of implementing railroad 

CRM training on railroad human factors-caused accidents. 

The research team interviewed a small group of experienced railroad CRM SMEs to gather 

information it used to refine parameter values for the railroad industry. The railroad SMEs were 

also asked several questions in order to determine if differences existed in how a railroad 
2 

organization would quantify the benefits of CRM training compared with the airline industry 

data collected. The team used answers in the study to ensure that railroad estimates were both 

reasonable and based upon the costs seen in the limited application of CRM in the railroad 

industry to date. Finally, researchers examined parallels between tasks in the two industries in 

order to estimate whether the railroad industry could, in fact, anticipate similar benefits to those 

reported by the aviation SMEs. The full report includes detailed explanations of how data for the 

parameters were derived for both the aviation and railroad industries. 

CRM’s contribution to improvement in overall safety is generally accepted throughout the 

industries in which it is practiced; however, this analysis uses only the expected reduction in 

accident costs as its focus to calculate the net financial benefit of implementing CRM training 

within the rail industry. The team used the utility analysis model to calculate the estimated 

accident-related cost savings (in dollars) as a result of introducing a CRM training program. 

This approach was deemed to be the most conservative method available and excludes several 

additional latent (but more difficult to quantify) benefits identified in other industries. 

Researchers used a derivation of the general formula for assessing the utility of training (Cascio, 

2000; Schmidt, Hunter, & Pearlman, 1982), which builds directly on the general utility formula 

for assessing the payoff for selection programs. The utility analysis formula that was used in this 

study took into account the following parameters: 

• Accident-related cost savings (in dollars) after training for the specified duration of the 

training effect 

• Duration of the training effect (in years) 

• Number of persons trained 

• Estimated effectiveness of the CRM training program 

• Per trainee costs of accidents (in dollars) for the duration of the training effect 

• Per trainee cost of training program 

Findings 

Based upon several different calculations, the utility analysis determined that railroad CRM 

training can be expected to have positive financial benefits in terms of reduced accident costs 

alone. Sensitivity analyses of benefits at both the industry and typical Class I levels produced 

positive benefits when using the average (mean) values for each parameter. While a very small 

number of utility calculations using the worst-case scenarios of absolute minimum parameter 

values (shortest training effect duration, highest training costs, etc.) did not produce net benefits, 

all calculations using average (mean) or typical parameter values and above were encouraging. 

The model predicted a net annual benefit of between $4 million and $33 million for the railroad 

industry as a whole and a net utility of $400,000 to $6 million for a typical Class I railroad (using 

average values from the four largest U.S. railroads). These ranges were found by using average 

parameter values and varying the training effects, which the full report describes in detail. 

Additional side benefits to CRM training over and above accident cost savings, such as increased 

workplace safety, improved teamwork and efficiency, and avoided legal and post-accident 

hazardous material cleanup costs, have the promise to even further increase the overall benefits 

of implementing and maintaining an ongoing CRM training program. 
3 

1.0 Introduction to Railroad CRM 

1.1 Background and History of CRM Training 

CRM and closely related human factors training programs have been in use in aviation and other 

high-consequence, high-reliability industries for the past quarter century. The introduction of 

these programs has been linked to marked reductions in the number and types of accidents and 

incidents attributed to human error. CRM training typically consists of an ongoing training and 

monitoring process through which personnel are trained to approach their activities from a team 

perspective rather than from an individual perspective. The team concept allows more efficient 

use of the available assets (technical proficiency, personal experience and observations, 

knowledge of equipment or conditions, understanding of Federal rules or company procedures) 

to prevent errors or lessen the severity of errors once they occur. Although the primary goal of 

implementing CRM within an industry is to improve safety by preventing accidents and 

providing a safer work environment for railroad employees, it is widely accepted that a fully 

implemented CRM training program can positively affect the performance of individual crews, 

resulting in more efficient and less costly operations. In turn, cost savings from avoided 

litigation resulting from a reduction of personnel injuries, less frequent collisions with other 

vehicles, and reduced hazardous materials spills could also be quantified as expected benefits. 

NTSB and U.S. Department of Transportation (DOT) modal administrations (e.g., the Federal 

Aviation Administration (FAA) and the Maritime Administration (MARAD)) have expressed 

confidence in CRM as a training intervention to improve safety as evidenced by their 

recommendation and/or introduction of CRM training programs (NTSB, 1999a; DOT, 1999). In 

spite of this, little research has been conducted during the past quarter century of CRM training 

in commercial aviation to empirically demonstrate or prove that implementation of CRM, and 

only CRM, has directly resulted in improved safety statistics as shown by a reduction in human 

factor accidents (Salas, Burke, Bowers, & Wilson, 2001; Salas, Wilson, Burke, & Wightman, in 

press). Rather, its implementation has continued, and subsequently been mandated by FAA, 

based upon the positive benefits that are perceived to result from increased awareness of human 

factors causes of accidents and the effects that such awareness can have in preventing accidents. 

Appendix A includes a history of CRM training and its background in several industries. 

1.2 CRM in the Railroad Industry 

Based largely upon the successful implementation of CRM in the commercial aviation industry, 

military aviation, and the marine transportation industry, NTSB recommended that CRM be 

introduced in the railroad industry following its investigation of the 1998 accident in Butler, IN, 

between Norfolk Southern (NS) and Conrail (NTSB, 1999). In this accident, NTSB found that a 

lack of coordination, communication, and teamwork among the members of the NS crew was the 

primary cause of the crash. An examination of the factors involved showed that the human 

factors issues leading up to this crash were quite similar to those involved in numerous aviation 

and marine accidents previously investigated by NTSB. Because CRM had been attributed with 

reducing human factors-caused accidents in those industries, NTSB saw its potential as a 

countermeasure for similar rail accidents as well. 
4 

1.3 History of CRM Training in the Railroad Industry 

Several Class I and shortline railroads have already implemented CRM training programs. FRA 

Office of Research and Development sponsored a study of existing teams in the rail industry and 

an assessment of CRM at the North American Class I railroads, which the Texas Transportation 

Institute (TTI) completed in late 2003 (Morgan et al., 2003). This study found that several North 

American Class I railroads had implemented CRM training programs for train and engine 

personnel either before or as a direct result of the NTSB recommendation, but it also found that 

implementation was uneven throughout the industry and that CRM could potentially be used for 

reducing human factors-caused accidents in many other craft areas, such as dispatchers, 

engineering crews, and mechanical workers. 

The Association of American Railroads (AAR) and NS jointly developed a video-based CRM 

training course for the industry, which had been used at several railroads, while Union Pacific 

(UP) and Canadian Pacific (CP) had each developed its own, in-house CRM training programs. 

UP’s program was based upon one developed by Southern Pacific (SP) before the UP-SP 

merger, and CP’s program was based on the SP program, as well as some independently 

developed materials. Several other railroads desired to see a CRM training program that could 

be more broadly applied within the industry—not only to train engine crews—but also to 

maintenance-of-way and mechanical workers. 

FRA Office of Safety sponsored the development of a pilot CRM course of this type, which was 

tested with the involvement and assistance of the Burlington Northern Santa Fe (BNSF) and 

Kansas City Southern (KCS) Railways in 2004 and 2005 (Morgan et al., 2005). This effort 

produced a 1-day long, scenario-based CRM training course that introduces railroad personnel to 

the key principles of CRM and how CRM can be applied in the railroad operating environment. 

The course is designed with three separate tracks—one each for transportation, engineering, and 

mechanical crafts. The course materials were presented to the Federal Railroad Administraion 

(FRA) in late 2005 for review and potential use throughout the industry. Before such a broad 

implementation can be recommended by FRA or railroad management can implement such a 

program independent of FRA direction, a case for the efficacy and importance of investing in 

CRM training as a safety and operational improvement strategy must be made. It was for that 

purpose that this research project was carried out. 

1.3.1 Need for Railroad Industry CRM Training 

On May 16, 2005, FRA issued its Action Plan for Addressing Critical Railroad Safety Issues 

(Safety Action Plan). The plan states that “the great majority of train accidents are caused by 

track and human factors, and human factor accidents are growing in number” (FRA, 2005, p. 2). 

The plan also states: 

Two categories of accidents—those caused by defective track and those 

caused by human factors—comprise more than 70% of all train accidents 

and a very high percentage of serious train accidents are, accordingly, the 

major target area for improving the accident rate (FRA, 2005, p. 2). 
5 

As shown in Figure 1 from the Safety Action Plan, accidents with primary human factors causes 

constitute the largest category of train accidents, approximately 38 percent, of all train accidents 

over the last 5 years. Not only has the percentage of human factors-caused accidents increased, 

but the severity of these accidents has also increased. FRA issued a Safety Advisory on January 

10, 2005, “strongly urging all railroads to adopt revised procedures to guard against” human 

mistakes (FRA, 2005, p. 3). 

Source: FRA Action Plan for Addressing Critical Railroad Safety Issues, May 16, 2005. 

Figure 1. Percentage of Primary Accident Causes 2000-2004 

Safety data recently presented by AAR also point out several interesting trends in human factorscaused accidents. For example, Figure 2 shows the overall trend for train accidents from 1980 

and 2004. The decrease in the overall accident rate is large showing that, on a percentage basis, 

overall train accidents per million train-miles have dropped 62 percent since 1980 and 8 percent 

since 1990 (AAR, 2006). 

Comparison of the role of human factors accidents to track accidents in this overall downward 

trend is one of the most important aspects to observe. Figure 3 shows the trend over the same 

time period for track-caused accidents. It shows that track-caused accidents per million trainmiles have dropped 72 percent since 1980 and 17 percent since 1990. Figure 4 shows the trend 

in human factor-caused accidents. Human factors-caused accidents per million train-miles have 
6 

0

2

4

6

8

10

12

1980 1985 1990 1995 2000 2004

11.43

4.73 

4.33 

Sources: AAR analysis of data from: 

FRA Web site: http://safetydata.fra.dot.gov/Prelim/2004/r01.htm (preliminary 2004 data). 

 FRA, Railroad Safety Statistics Annual Report, 1997-2004, Tables 1-1, 1-2. 

 FRA, Accident/Incident Bulletin, 1980-1996, Tables 19, 36. 

Note: Excludes grade crossing accidents. 

Figure 2. Train Accidents per Million Train-Miles 1980-2004 

dropped 46 percent overall since 1980, but only 3 percent of this change has occurred since 

1990. Notice that, since 1996, the human factors-caused accident rate has been on an upward 

trend (AAR, 2006). 
7 

0

1

2

3

4

5

1.60 

1.33 

4.84 

1980 1985 1990 1995 2000 2004

Sources: AAR analysis of data from: 

FRA Web site: http://safetydata.fra.dot.gov/Prelim/2004/r01.htm & r02.htm (preliminary 2004 data). 

 FRA, Railroad Safety Statistics Annual Report, 1997-2004, Tables 1-1, 5-9. 

 FRA, Accident/Incident Bulletin, 1980-1996, Tables 19, 36. 

Note: Excludes grade crossing accidents. 

 Figure 3. Track-Caused Accidents per Million Train-Miles 1980-2004 

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

1980 1985 1990 1995 2000 2004

1.80 

1.74 

3.24 

Sources: AAR analysis of data from:

FRA Web site: http://safetydata.fra.dot.gov/Prelim/2004/r01.htm & r02.htm (preliminary 2004 data). 

 FRA, Railroad Safety Statistics Annual Report, 1997-2004, Tables 1-1, 5-9. 

 FRA, Accident/Incident Bulletin, 1980-1996, Tables 19, 36. 

Note: Excludes grade crossing accidents. 

Figure 4. Human Factors-Caused Accidents per Million Train-Miles 1980-2004 
8 

These graphs indicate that although the application of improved track technologies (e.g., new 

track steels, new attachment methods, etc.) to decrease a specific mechanical accident causal 

factor have continued to be effective, the rate of human factors-caused accidents is not 

decreasing at a similar rate—instead it has remained relatively stable. This has resulted in 

human factors-caused accidents becoming the largest single cause category while mechanical 

causes have continued to decrease. The same trends (drastically reduced mechanical- or 

equipment-caused accident rates and a relative leveling off of human factors-caused accident 

rates) were evident in the commercial airline industry in the late 1970s and early 1980s when 

CRM was developed to address this problem. Since that time, CRM training has become one of 

the most highly applied and accepted practices in the airline industry to address human factorscaused accidents. 

Figure 5 shows both primary and secondary causes and causal factors for commercial aviation 

(FAR Part 121 operations) over the 10-year period between 1992 and 2001, the latest year 

reported on NTSB’s aircraft accident statistics Web site (NTSB, 2006). The information 

indicates that personnel- or human factors-related causes are cited in 70-80 percent of the 

aviation accidents investigated by NTSB each year during that time period, while mechanical 

factors were only cited 20-30 percent of the time, and environmental factors were cited 

approximately 40 percent each year. This further reinforces the reasoning behind the airlines and 

the FAA’s decision to commit resources into improved training to decrease human factorscaused accidents. 

Figure 5. Broad Causes/Factors for Part 121 Accidents 1992-2001 
9 

1.4 Research Methodology 

One of the most difficult things to prove is that an accident that never occurred was prevented 

directly as the result of a given training intervention. This has been a dilemma in the case of 

recommending CRM training as a means to reduce accident rates. Although NTSB, the 

commercial aviation industry, the military, and other users of CRM all attribute the observed 

safety improvements and reductions in the number of human factor accidents which coincide 

with CRM’s introduction to CRM training, calculating or documenting specific accidents that 

have not occurred as a result of CRM training has proven problematic. Other safety initiatives 

have also been introduced during this time period; the use and fidelity of training simulators has 

improved; and technological advancements in training methods, such as computer-based training 

(CBT), have also played a role in reducing accidents through better training. 

Anecdotal reports of CRM saves where crew coordination, assertive communication, or 

improved situational awareness (all CRM core disciplines) has prevented a crash are abundant; 

however, no system is really in place to collect and document such occurrences. Close call 

reporting systems, like that in place in aviation and being developed for the railroad industry, 

may capture some of these. The practice of CRM principles or behaviors, however, may actually 

break the error chain long before any close call occurs. Documenting and demonstrating this 

improved operational side benefit remains one of the more difficult problems to address. 

Due to the absence of numerical data which can directly connect accident prevention with CRM 

training interventions, the research team had to seek another method to determine and document 

CRM’s potential for preventing accidents and the resultant costs in personnel and monetary 

assets within the railroad industry. In order to accomplish this, the team developed a utility 

analysis model for CRM training. Utility analysis is a series of statistical procedures used to 

evaluate the dollar gains accrued from the implementation or use of personnel interventions and 

systems over their associated costs (Cabrera & Raju, 2001; Cascio, 2000; Cascio & Aquinis, 

2005). Although this concept has been proven in industrial-organizational psychology and 

training specialist circles, to the research team’s knowledge, this concept has not yet been 

applied to assess the utility of CRM training in a specific industry or to a company within a 

specific industry. To accomplish this task for the rail industry, the team first had to apply the 

utility analysis model to the airline industry where CRM has been implemented for the longest 

time and with a known history and then had to draw parallels between the two industries in order 

to estimate whether similar benefits could be expected in the railroad industry. Chapter 2 

describes the results of this analysis. 

Chapter 3 discusses the similarities and differences between commercial aviation and the railroad 

industry. This includes detailed analysis of how job tasks between the two industries relate to 

one another, as well as a comparison of the organizational culture differences and training 

differences that exist between the two industries. Chapter 4 presents the conclusions and 

findings of the study and recommendations regarding implementation of CRM within the rail 

industry. 

11 

2.0 Utility Analysis of CRM Training Effectiveness in Preventing Human 

Factors-Caused Accidents 

2.1 Overview of Utility Analysis Methodology 

Procedures for quantifying and measuring the financial gains associated with the implementation 

of human resource management interventions, such as training, are not easy to implement, but 

the technology to do so is available and well developed in the field of industrial-organizational 

psychology. Specifically, this can be accomplished using utility analysis, a set of statistical 

procedures that allow the assessment of the dollar gains to an organization resulting from the use 

or implementation of a specified HRM intervention (Cascio, 2000), in this particular case, CRM 

training. Because the focus here is the estimation of the accident-related cost savings (in dollars) 

after implementing CRM training, researchers used a derivation of the general formula for 

assessing the utility of training (Cascio, 2000; Schmidt, Hunter, & Pearlman, 1982), which builds 

directly on the general utility formula for assessing the payoff for selection programs. The 

following presents the specific utility formula that was used here as Equation 1: 

 ΔU = (N × d ×SDy) − (N × C) (1) 

where: 

ΔU = accident-related cost savings (in dollars) after training, for the 

specified duration of the training effect 

N = number of persons trained 

 

d = effect of the CRM training program (which represents the 

standardized mean difference between the trained and untrained 

group on the specified criterion of interest; or the difference 

between the pre-training and post-training effect) 

 

SDy = per trainee costs of accidents (in dollars) for the duration of the 

training effect 

 

C = per trainee cost of training program 

 

In summary, utility analysis is a series of statistical procedures used to evaluate the dollar gains 

accrued from the implementation or use of personnel interventions and systems over their 

associated costs (Cabrera & Raju, 2001; Cascio, 2000; Cascio & Aquinis, 2005). The research 

team presents a detailed description of each of the parameter estimates in the utility analysis 

formula in Equation 1 and how each was obtained for the airline industry CRM utility analysis 

presented in this report. 
12 

2.2 Obtaining or Estimating Airline Industry Utility Analysis Parameters

The researchers used archival and published records where available, and SME ratings and 

judgments when they were not, to obtain the parameters for the utility analysis. The following 

sections will discuss details on how archival and published records were obtained and used for 

each specific parameter. SMEs were recruited by contacting a variety of major and regional 

airlines in the United States, as well as directly contacting known CRM researchers associated 

with research universities and academic institutes. A total of eight airline SMEs whose ratings 

and judgments were used to obtain the parameters for the utility analysis participated. 

Practitioner SMEs work (or have worked) in either the major or regional commerical airlines as 

CRM/human factors training developers or facilitators, while researcher SMEs have researched 

and published articles or papers regarding CRM training. Although most SMEs have also 

worked with CRM in other non-aviation settings, each of the interviewed SMEs has between 7 

and 24 years of experience in aviation human factors or CRM research, training development, 

and/or facilitation. In total, the SMEs that the team interviewed have published more than 150 

papers on CRM, human factors, and/or team training; several SMEs have graduate degrees in 

aviation or human factors-related disciplines (i.e., Aerospace Engineering, IndustrialOrganizational Psychology). 

2.3 Airline Utility Analysis Parameters 

The following presents a description of each parameter along with how its data were obtained for 

the airline industry. 

ΔU: Accident-related cost savings (in dollars) after training for the specified duration of 

the training effect 

This is the outcome or product of the utility analysis, and it represents, in dollars, the cost 

savings to be accrued from the implementation of CRM training. This utility estimate takes into 

account the effectiveness of CRM in impacting human factors-related/accident-related behaviors 

that are associated with accidents and, as a result, subsequently reducing accidents and their 

resultant costs. In addition, it also incorporates the following information: (a) how long the 

effect of training lasts; (b) the number of people to be trained; (c) the cost of accidents, which by 

definition also incorporates the number of accidents experienced; and (d) the cost of the CRM 

training program. The airline utility estimates presented here are at the level of the airline 

industry and not at the organization level. 

The duration of the training effect. This parameter represents how long the effect of training 

lasts. A best-case scenario is one where the effects of training last for a long time (i.e., long 

retraining interval); conversely, a worst-case scenario is one where the effect of training is very 

short-lived (i.e., short retraining interval). The duration of CRM training’s duration effect was 

obtained from (seven) SMEs who provided numeric estimates of how long they thought the 

effects of CRM training lasted (see Question 31 in Appendix B). The minimum, average, and 

maximum values of these numeric values were used to generate conservative, average or typical, 

and liberal utility estimates, respectively. 
13 

N: Number of persons trained 

This parameter represents the number of people trained per year. These were factually obtained 

using data from the Bureau of Labor Statistics (BLS) (U.S. Department of Labor [DOL], 2006) 

and the Bureau of Transportation Statistics (BTS) (DOT, 2004) for the airline utility analyses 

presented here. For more detailed information on how this was calculated, please see Appendix 

C. 

d: Effectiveness of CRM training 

This parameter represents the effectiveness of the CRM training program in reducing accidents. 

The effect size metric, d, can be described as the standardized mean difference between either: 

(a) the trained and untrained group or (b) the pre- and post-training period on the outcome of 

interest. In the case of this study, this is the number of human factors-caused accidents. It is 

important to emphasize that, as with any training intervention, discussions of the effectiveness of 

CRM must be within the context of the outcomes that it is conceptually expected to impact. 

Effectiveness must be described in terms of specific criteria (Arthur, Bennett, Edens, & Bell, 

2003). Thus the effectiveness criterion that researchers focused on in this project was human 

factors-caused accidents. Consequently, the collection of effectiveness estimates was based on 

the premise that CRM is intended to impact certain human factors behaviors, which are in turn 

related to the occurrence of accidents (Helmreich, Merritt, & Wilhelm, 1999). 

2.3.1 Methods of Calculating Training Effectiveness 

Direct empirical estimates of effectiveness: 

Estimates of training program effectiveness (also known as effect size) are typically obtained by 

implementing an evaluation study that permits an empirical assessment of the intervention’s 

effectiveness. The relationship between the intervention and the outcome variables of interest 

(i.e., program effectiveness) is thus operationalized as the effect size statistic, d—the 

standardized difference between the criterion means of the trained and untrained group. The 

effect size indicates (a) if a difference exists between the two groups (i.e., does the training 

program work?) and (b) how large it is (i.e, how well does it work?). The typical formula for 

computing d is presented in Equation 2: 

M M− C

d = E (2) S

W

where: 

d = effect of the CRM training program 

ME = mean of the trained group on the specified criterion of interest 

MC = mean of the untrained group on the specified criterion of interest 

SW = pooled, within group standard deviation 

For several reasons, it was not feasible for researchers to implement a primary evaluation study 

to obtain a direct estimate of the effectiveness of CRM training. A primary study would be one 

in which the effects of CRM were evaluated in a field (i.e., an airline) or laboratory setting (i.e., 

on a university campus using students and individuals from the local community). A study of 
14 

this nature was deemed beyond the scope of the present project for several reasons. First, a field 

study was unrealistic because it was unlikely that any airline would be willing to collaborate by 

allowing a portion of their pilots to forego or delay CRM training because doing so would likely 

seriously compromise their safety and related financial interests by placing employees and 

passengers at risk. Second, because FAA mandates CRM training, a study that required some 

pilots to forego CRM training would violate FAA regulations. Third, the use of a laboratorybased study, instead of a field study, to evaluate the effectiveness of CRM training was also 

considered to be beyond the scope of the project. Such a study would have to be designed so that 

the results could be generalized to the complex jobs of both the airline and railroad industries. 

Given the financial resources and timeframe within which this project had to be completed, such 

a methodology was not feasible. Consequently, researchers used other approaches to obtain 

these estimates. 

Indirect estimates: 

In the absence of evaluation studies that permit a direct empirical estimation of d, estimates of 

program effectiveness can be obtained indirectly. The following presents the approaches 

considered for this project. 

a) Meta analysis-based estimates. Where other evaluation studies are available, d can be 

estimated by cumulating the results of all the available studies using meta-analytic 

procedures (Arthur et al., 2003). In the present case, a detailed search of the extant 

literature and extensive consultation with researchers and experts in the field failed to 

obtain any previously conducted empirical studies that could be meta-analytically 

aggregated to obtain summary estimates of the effectiveness of CRM training. This 

failure to locate any primary formal evaluation studies is consistent with the previously 

noted observation that because of severe logistical and practical constraints, formal 

evaluation studies of CRM effectiveness are basically nonexistent (see Salas et al., 2001; 

Salas et al., in press). 

b) SME-based estimates. In absence of other available evaluation studies, SMEs can also 

be used to provide subjective estimates of the effectiveness of CRM training 

interventions for specified criteria. The research team used this approach to obtain the 

effect size estimates used in this project. First, by using a factual estimate obtained using 

the NTSB Accident and Incident Data System (NTSB, 2005), researchers calculated a 

current baseline of 40 CRM-related accidents over the past 5 years. For more detailed 

information on how this was calculated, please see Appendix C. Next, given a current 

baseline of 40 CRM-related accidents over the past 5 years, the SMEs were asked to 

estimate how many more accidents would have occurred if CRM training were not in 

place (see Question 36 in Appendix B). The research team also asked the SMEs to give 

upper and lower bound estimates. The numbers obtained from the SMEs were entered 

into Equation 2 to compute an effect size estimate for the effectiveness of CRM training. 

These effect sizes are referred to as SME-based, computed in the tables presenting the 

utility analysis results later in this chapter. 
15 

Researchers also obtained a second effect size estimate from the SMEs by asking them to 

indicate whether in their opinion, CRM had the following (see Question 37 in 

Appendix B): 

• No effect (a situation in which a group that was trained in CRM has the same 

number of accidents as one that was not trained [This was calculated using a d of 

0.]) 

• A small effect (a situation in which the trained group has about 3 less accidents 

than the untrained group [This reduction was calculated using a d of 0.20.]) 

• A moderate effect (a situation in which the trained group has about 8 less 

accidents than the untrained group [This reduction was calculated using a d of 

0.50.]) 

• A large effect (a situation in which the trained group has about 12 less accidents 

than the untrained group [This reduction was calculated using a d of 0.80.]) 

• A very large effect (a situation in which the trained group has about 15 less 

accidents than the untrained group [This reduction was calculated a d of 1.10.]) 

These effect sizes are referred to as SME-based estimates in the tables presenting the 

utility analysis results. 

c) Rule-of-thumb, literature-based effect size estimates. The team also used rules-ofthumb, literature-based effect size estimates in computing CRM’s utility. Specifically, in 

interpreting the magnitude of effect sizes, literature-based rules of thumb exist for 

effectiveness of training programs that are considered to be small (d = 0.20), medium (d

= 0.50), and large (d = 0.80) (Cohen, 1992). Again, these literature-based effect size 

estimates can be used to generate variable estimates of the utility of CRM training, 

assuming its effectiveness is weak, medium, or strong. This approach was used in 

conjunction with the SME-derived estimates to provide an additional range of boundary 

conditions concerning the utility of CRM in the absence of direct empirical estimates of 

program effectiveness. 

In summary, in the absence of direct empirical estimates of CRM effectiveness, the approach 

was to use multiple estimates of effectiveness that would collectively make for a more 

compelling case than any one estimate by itself since the resultant utility estimates would be 

based on a wide but reasonable derived range of effect size estimates. 

SDy: Per trainee costs of accidents (in dollars) for the duration of the training effect 

Researchers computed this parameter using the number of accidents per year, the cost per 

accident, the number of trainees, and the training effect duration. As previously noted, the 

training effect duration was obtained through SME judgments (see Question 31 in Appendix B) 

and so were the number (see Question 36 in Appendix B) and cost of accidents (see Question 38 

in Appendix B). As stated previously, however, the number of trainees was determined factually 

using data from the BLS (DOL, 2006) and the BTS (DOT, 2004). SDy was computed by first 

multiplying the number of accidents by the cost of an accident and then multiplying the product 

by the training duration effect (in years). This result, which represents the cost of accidents that 

would be incurred over the training duration effect, was then divided by the number of trainees 
16 

to obtain the costs of accidents (in dollars) for the duration of the training effect per trainee—that 

is, SDy. The following presents the specific formula that was used here as Equation 3: 

T × (# of accidents per year × cost per accident) SDy = (3) N

The duration of the training effect (in years) is denoted as T in Equation 3. This parameter 

represents how long the effect of training lasts. A best-case scenario is one where the effects of 

training last for a long time (i.e., long retraining interval); conversely, a worst-case scenario is 

one where the effect of training is very short-lived (i.e., short retraining interval). The duration 

of CRM training’s duration effect was obtained from (seven) SMEs who provided numeric 

estimates of how long they thought the effects of CRM training lasted (see Question 31 in 

Appendix B). The minimum, average, and maximum values of these numeric values were used 

to generate conservative, average or typical, and liberal utility estimates, respectively. 

C: Per trainee cost of CRM training 

The estimate of the total cost of CRM training per trainee was obtained through SME judgment 

(see Question 39 in Appendix B). The average of the SME estimates of the training cost per 

trainee represents a typical cost estimate and is presented in this chapter. 

2.4 Airline Industry Utility Analysis Results 

Table 1 presents the results of the airline utility analysis using average values for each of the 

parameters. The results indicate that implementing CRM training has positive benefits using 

average (or expected) values. Thus, in terms of a utility analysis framework, based on the data 

presented here, the benefits of CRM training to the airline industry conclusively outweigh its 

costs. Appendix F contains a sensitivity analysis of the effects within the aviation industry 

showing the results of the utility calculations using the minimum, mean, and maximum values 

for each of the parameters of interest. These levels represent the most conservative, average, and 

most liberal estimates of CRM effectiveness, respectively. 
Table 1. Utility Analysis Results for Airline Industry Based on 

Average Parameter Estimates 

17 

Based on 21 human error accidents/year with an average cost of $129m/accident . 

Number of persons trained (N) = 51,500C

Duration of training effect (days; T) = 384D

Per trainee cost of accidents in training effect duration (SDy) = $55,268.63 E

A B

Per trainee cost of training (C) = $794.00

Effect Size Magnitude and Source d Utility Utility (M$)F

Small effect 

SME-based estimate (minimum) 0.80 $2,236,176,391.37 $2,236m

SME-based, computed 1.40 $3,943,976,934.90 $3,944m

Literature-based effect size (small) 0.20 $528,375,847.84 $528m

Medium effect 

SME-based estimate (mean) 1.04 $2,919,296,608.78 $2,919m

SME-based, computed 1.41 $3,972,440,277.29 $3,972m

Literature-based effect size 

(medium) 

0.50 $1,382,276,119.61 $1,382m

Large effect 

SME-based estimate (maximum) 1.10 $3,090,076,663.13 $3,090m

SME-based, computed 2.23 $6,306,434,353.44 $6,306m

Literature-based effect size (large) 0.80 $2,236,176,391.37 $2,236m

AMean of responses to Question 36 (Appendix B). B

Mean of Question 38. CFactually obtained. 

DAverage of Question 31. E

Average of Question 39. FRounded to the nearest million. 

2.5 Obtaining or Estimating Railroad Industry Utility Analysis Parameters 

As with the airline industry, the research team used archival and published records where 

available, and SME ratings and judgments when they were not, to obtain the parameters for the 

railroad utility analysis. Appendix D presents the railroad SME semi-structured interview form 

that was used. Because of CRM’s current limited use in the rail industry (Morgan et al., 2003), 

the recruitment of railroad SMEs was more restricted than recruitment in the airline industry. 

Because railroad SMEs must be familiar enough with CRM to make valid judgments regarding 

the parameters used in the utility analysis, researchers’ recruiting efforts resulted in them being 

able to interview SMEs from only one railroad. This railroad was one of two railroads where 

CRM training has been implemented companywide among transportation employees for over 6 

years. Thus, unlike the airline SMEs, all three of the railroad SMEs were employed by the same 

company. One SME had over 10 years experience as a safety specialist on the railroad, and all 

have had at least 5 years experience in railroad CRM training development and/or facilitation. 
18 

Since the number of railroad SMEs were limited and they were all from the same organization, 

the input of the railroad SMEs was mainly used to gauge how a railroad company would value or 

quantify the effects of CRM compared to the effects observed at the airlines in this study. 

Because so few railroad CRM training programs have been widely implemented, the 

representativeness of the sample and the generalizability of the data collected should be carefully 

re-examined at some point in future research to draw conclusions regarding how strongly the 

experience of the aviation industry following CRM training is actually mirrored within the 

railroad industry. Chapter 3 discusses the projected effects and factors that may differentiate the 

two. 

2.6 Railroad Utility Analysis Parameters 

The following presents a description of each parameter along with how its data were obtained for 

the railroad industry. 

ΔU: Accident-related cost savings (in dollars) after training for the specified duration of 

the training effect 

As with the airline industry analyses, this parameter represents, in dollars, the cost savings to be 

accrued from the implementation of CRM training in the railroad industry. The utility estimates 

are again presented at the level of the railroad industry rather than an individual company. In 

addition to this analysis, however, the research team also presents benefit estimates for a typical 

large Class I railroad later in this chapter. 

The duration of the training effect. Only one of the three railroad SMEs interviewed provided 

an estimate of CRM training’s duration effect in the railroad industry (see Question 31 in 

Appendix D). He/she estimated this to be 180 days (6 months). Researchers subsequently used 

this as the conservative estimate of the length of the duration effect and used 270 days (9 

months) and 365 days (12 months) as the average and liberal estimate, respectively. Thus, again, 

the best-case scenario is one where the effects of training last for a long time (i.e., long retraining 

interval); conversely, a worst-case scenario is one where the effect of training is very short-lived 

(i.e., short retraining interval). 

N: Number of persons trained 

This parameter represents the number of people trained per year. These data were estimated 

based upon the number of employees reported by the rail industry. Specifically, data from BLS 

(DOL, 2006b) and the Surface Transportation Board (DOT, 2004b) indicated that approximately 

72,500 engineers and conductors (road, yard, and switch) were employed in the railroad industry 

in 2004. Appendix E contains a more detailed explanation of this calculation. 

d: Effectiveness of CRM training 

This parameter represents the effectiveness of the CRM training program in reducing accidents. 

For the railroad utility analysis, researchers used SME-based estimates because this was the only 

viable option of obtaining effect size estimates. As with the airline analysis, the SMEs were 

asked to indicate, whether in their opinion, CRM had (see Question 37 in Appendix D): 

• No effect (a situation in which a group that was trained in CRM has the same number 

of accidents as one that was not trained [This was calculated using a d of 0.]) 
19 

• A small effect (a situation in which the trained group has about 80 less accidents than 

the untrained group [This reduction was calculated using a d of 0.20.]) 

• A moderate effect (a situation in which the trained group has about 200 less accidents 

than the untrained group [This reduction was calculated using a d of 0.50.]) 

• A large effect (a situation in which the trained group has about 290 less accidents 

than the untrained group [This reduction was calculated using a d of 0.80.]) 

• A very large effect (a situation in which the trained group has about 360 less 

accidents than the untrained group [This reduction was calculated using a d of 1.10.]) 

Only one of the railroad SMEs that the research team interviewed was willing to provide an 

estimate of the effect size that CRM training has/would have on rail accidents. Aviation SMEs 

were much more willing to provide an estimate of CRM benefits. Much of this difference results 

from the relatively short history of CRM implementation in railroads compared to aviation. 

Without a long track record, it was difficult for railroad personnel to make such estimates. Once 

presented with the effect size estimates of others in both aviation and railroad industries, the 

hesitant railroad SMEs were willing to state their opinions of the estimates that were given—

generally agreeing that the estimates were in the range that they were considering. 

As with the airline industry analyses, the researchers also used rule-of-thumb literature-based 

effect size estimates in computing the CRM utility for the railroad industry. The airline industry 

section previously discussed these literature-based effect size estimates or rule-of-thumb values. 

In summary, in the absence of direct empirical estimates of CRM effectiveness in the railroad 

industry, the approach was again to use multiple estimates of effectiveness that would 

collectively make for a more compelling case than any one estimate by itself since the resultant 

utility estimates would be based on a wide but reasonably derived range of effect size estimates. 

SDy: Per trainee cost of accidents (in dollars) for the duration of the training effect 

This parameter was computed using the number of accidents per year, the cost per accident, the 

number of trainees, and the training effect duration. As previously noted, the training effect 

duration was obtained through SME judgments (see Question 31 in Appendix D). The number 

and cost of accidents, along with the number of trainees, however, were obtained from FRA, 

NTSB, and BLS numbers. The procedures used to estimate the number of trainees was 

previously described in the aviation section. The number of accidents per year was determined 

by computing the average of the number of human error accidents reported to FRA using the 

FRA’s (2004) accident database for the 5-year period 2000–2004. Appendix E contains a more 

detailed description of these calculations. The cost per accident was calculated by computing the 

average (and min and max) of the cost of accidents in 2000–2004 reported in FRA’s (2004) 

accident database. Appendix E contains a more detailed description of these calculations. The 

research team then divided this yearly cost average by the yearly number of accidents average 

respectively. SDy was then computed by first multiplying the number of accidents by the cost of 

an accident and then multiplying the product by the training duration effect (in years). This 

result, which represents the cost of accidents that would be incurred over the training duration 

effect, was then divided by the number of trainees to obtain the costs of accidents (in dollars) for 

the duration of the training effect per trainee—that is, SDy. See equation 3. 
20 

The duration of the training effect (in years) is denoted as T. Only one of the three railroad 

SMEs interviewed provided an estimate of CRM training’s duration effect in the railroad 

industry (see Question 31 in Appendix D). He/she estimated this to be 180 days (6 months). 

Researchers subsequently used this as the conservative estimate of the length of the duration 

effect and used 270 days (9 months) and 365 days (12 months) as the average and liberal 

estimate, respectively. Thus, again, the best-case scenario is one where the effects of training 

last for a long time (i.e., long retraining interval); conversely, a worst-case scenario is one where 

the effect of training is very short-lived (i.e., short retraining interval).

C: Per trainee cost of CRM training 

The estimate of the total cost of CRM per trainee was determined through SME judgment (see 

Question 39 in Appendix D). All three railroad SMEs provided estimates. Consequently, the 

average of the SME estimates was used to represent a typical estimate. 

2.7 Railroad Industry Utility Analysis Results 

Table 2 presents the results of the railroad utility analysis using average parameter values. The 

results indicate that the gain to the railroad industry from implementing CRM training is 

positive—ranging from $4 million to $33 million in cost savings for the industry as a whole. 

Appendix G is a sensitivity analysis of the results using the minimum, average, and maximum 

values for each of the parameters to represent the most conservative, average, and most liberal 

utility value of a CRM training program. The only scenario in which there were no accidentrelated cost savings was with the weakest effect size (shortest training effect period) under the 

most conservative scenario for all parameters (see Appendix G). Thus, in terms of a utility 

analysis framework, based on the data presented here, although the gains are not as large as those 

obtained for the airline industry, the results of the utility analysis indicate that the benefits of 

CRM training to the railroad industry generally outweigh its costs. This is especially true if the 

anticipated cost savings from improved operational efficiency and reduced litigation costs were 

to be included in addition to the benefits only from decreased accident costs. Additionally, 

accidents in which human factors is a secondary cause could be reduced, leading to further 

decreases in the number and severity of accidents and resulting in added benefits. 
21 

Table 2. Utility Analysis Results for Railroad Industry Based on 

Average Parameter Estimates 

Based on 1,032 human error accidents/yearA with an average cost of $63,512/accidentB

. 

Number of persons trained (N) = 36,250C

Duration of training effect (days; T) = 270D

Per trainee cost of accidents in training effect duration (SDy) = $1,337.51 

Per trainee cost of training (C) = $167.00E

Effect Size Magnitude and Source d Utility Utility (M$)F

Literature-based effect size (small) 0.20 $3,643,227.36 $4m

Literature-based effect size 

(medium) 

0.50 $18,188,693.40 $18m

SME-based estimate 0.50 $18,188,693.40 $18m

Literature-based effect size (large) 0.80 $32,734,159.44 $33m AFactually obtained, mean value. B

Factually obtained = mean cost of accidents ÷ mean number of accidents. 

CFactually obtained, training 50% in 1 year. DQuestion 31 (Appendix D). E

Mean of Question 39. FRounded to the 

nearest million. 

2.8 Utility Analysis Results for a Typical Large Class I Railroad 

Because the preceding railroad results were at the industry level, the research team also desired 

to present similar analyses at the organizational level. Consequently, researchers factually 

obtained data for the number of engineers and conductors from the Surface Transportation Board 

(DOT, 2006) and the number of accidents from FRA (2005) for four representative large Class I 

railroads, namely BNSF, CSX, NS, and UP. Appendix E contains a more detailed description on 

the calculation of the number of accidents. The number of engineers and conductors and the 

average number of accidents across these four railroads were then used to compute the utility 

gain in accident savings for a typical large Class I from implementing CRM training. These 

analyses modeled the same general boundary conditions and approach used for the industry level 

analyses. Table 3 presents the results of the typical large Class I analysis using average 

parameter values. These results show that, as would be expected, when disaggregated to the 

organizational level, the absolute dollar gain is smaller than the industry-level values. A 

sensitivity analysis of the results (presented in Appendix H) indicates that, for the small and 

medium effect sizes under the most conservative scenario, no utility gains exist based upon 

reduced accident costs alone. The gains under all the other scenario boundary conditions, 

however, are positive for this cost measure. Thus, as with the industry-level results, it is 

reasonable to conclude, and the results of the utility analysis suggest, that the typical large Class 

I railroad will accrue accident-related cost savings from the implementation of CRM training. 
22 

Table 3. Utility Analysis Results for a Typical Large Class I Railroad Based on 

Average Parameter Estimates 

Based on 189 human error accidents/yearA with an average cost of $63,947/accidentB. 

Number of persons trained (N) = 8,171C

Duration of training effect (days; T) = 270D

Per trainee cost of accidents in training effect duration (SDy) = $1,094.15 

Per trainee cost of training (C) = $167.00E

Effect Size Magnitude and Source d Utility Utility (M$)F

Literature-based effect size (small) 0.20 $423,506.24 $0.4m

Literature-based effect size 

(medium) 

0.50 $3,105,601.10 $3m

SME-based estimate 0.50 $3,105,601.10 $3m

Literature-based effect size (large) 0.80 $5,787,695.95 $6m AFactually obtained, mean value. B

Factually obtained = mean cost of accidents ÷ mean number of accidents. 

CFactually obtained, training 50% in 1 year. DQuestion 31 (Appendix E). E

Mean of Question 39. FRounded to the 

nearest million. 

This analysis used only accident-related costs of accidents with primary human factors causes. 

CRM training should also generate additional savings by improvements in the following areas, 

which were not a part of the analysis: 

• Reduced number of accidents with human factors as a secondary cause 

• Increased efficiency in operations 

• Increased overall productivity as the program expands to encompass all crafts 

• Avoided litigation costs due to reduced accidents 

• Reduced accident cleanup and remediation costs 

As a result, the positive benefit numbers shown in Table 3 should be viewed as the most 

conservative estimates of benefits that could be achieved from implementing CRM for the given 

parameters. 
23 

3.0 Applying Commercial Aviation’s Lessons Regarding CRM to the 

Railroad Industry 

3.1 Differences Between the Airline and Railroad Industries 

The utility analysis described in Chapter 2 reveals differences between the airline and railroad 

industries regarding accident-related cost savings that result from CRM training program 

implementation. Although the results generally suggest that the benefits of implementing CRM 

training outweigh the costs in both industries, the gains are smaller in the railroad industry. This 

can be attributed to differences between the industries in terms of the specific parameters used in 

each analysis. Specifically, the differences in overall utility are the result of CRM being 

estimated to have slightly less effect in the railroad industry, its effect estimated as not lasting as 

long, and it costs less per trainee to implement CRM in the railroad industry. In order to better 

understand why this might be the case, the following sections will review similarities and 

differences between cockpit and train road crews, as well as their respective industries. 

First, the research team examines why CRM should lead to a reduction in the number of human 

factor accidents in the railroad industry as had been attributed to CRM in aviation. This is 

accomplished by comparing and contrasting the types of tasks and work activities accomplished; 

work contexts; and the knowledge, skills, and abilities (KSAs) needed by cockpit and train crews 

to determine the level of similarity between the two compared crews. Furthermore, researchers 

review the causes of accidents in the railroad industry to better understand the need for a human 

factors training program like CRM. Second, researchers suggest how key differences in the 

scope of CRM training between the industries can explain the differences in effect size, effect 

duration, and cost of CRM training between the benefits calculated for the airline industry and 

that projected for the railroad industry. 

3.2 Comparison of Airline and Railroad Work Environments 

Although many types of teams exist in today’s work environment, crews are used to accomplish 

the most basic and fundamental tasks in high-reliability industries (i.e., aviation and railroad). 

By distinguishing crews from other types of teams, the team literature suggests that crews 

themselves are somewhat homogeneous. Webber and Klimoski (2004) define a crew as “a group 

of expert specialists each of whom have specific role positions, perform brief events that are 

closely synchronized with each other, and repeat these events across different environmental 

conditions” (p. 265). Researchers suggest, however, that understanding the tasks that specific 

teams accomplish is the key to determining how human resource initiatives, like team training, 

might differentially affect those teams (Arthur, Edwards, Bell, Villado, & Bennett, 2005). 

Similarly, understanding the types of behavioral skills needed for performance in specific crews 

is considered the theoretical driver of CRM training (Salas, Fowlkes, Stout, Milanovich, & 

Prince, 1999). 

With this in mind, the research team compared train and cockpit crews on several dimensions, 

including tasks, using the Occupational Information Network (O*NET) database. O*NET is 

DOL’s comprehensive job description database of worker attributes and job characteristics, as 

well as the Nation’s primary source of occupational information. Using O*NET (DOL, 2005), 

researchers also compared the types of broad work activities each crew performs, the KSAs 
24 

needed to perform those tasks and work activities, and finally each crew’s immediate work 

context. (Note: Although their jobs are quite different, O*NET combines railroad conductors 

with railroad yardmasters (code 53-4031.00) when describing the various work activities, tasks, 

KSA, and work contexts of these jobs. Furthermore, O*NET does not differentiate between 

conductors working within passenger service and those working with freight, even though 

distinctly different activities, tasks, KSAs, and contexts are associated with each (i.e., collecting 

passenger tickets). Thus, in order to get a more valid indication of a freight conductor’s job, the 

authors reviewed all the work activities, tasks, KSAs, and work contexts for O*NET code 53-

43031.00 and deleted the ones that were identified as solely yardmaster or passenger train 

conductor characteristics. The remaining characteristics were re-ranked to reflect the importance 

for the job of freight conductor specifically. The importance rankings in the following tables 

reflect this correction.) 

3.2.1 Work Tasks 

Table 4 compares the tasks of cockpit crewmembers (pilots, copilots, and flight engineers) to 

those of locomotive engineers and conductors as described by O*NET. Moreover, it compares 

the importance ratings of each task between the different occupations. Table 4 shows that a high 

degree of overlap exists in terms of the types of tasks each crew completes and the importance of 

those tasks. Specifically, the tasks of (1) communicating with crewmembers, (2) checking 

resources, (3) interpreting and choosing routes, and (4) coordinating and directing task activities 

with fellow crewmembers are the most important tasks for both cockpit and locomotive crews. 

3.2.2 Work Activities 

Table 5 compares the importance of the more broadly defined work activities, which are also 

similar between airline and train crews. According to the O*NET data, the field of “operating 

vehicles, mechanized devices, or equipment” is ranked as the most important work activity of 

both cockpit crews and locomotive engineers specifically. The table also shows similar 

importance rankings for (1) getting information; (2) inspecting equipment, structures, or 

material; (3) monitoring processes, materials, or surroundings; (4) documenting/recording 

information; and (5) identifying objects, actions, and events. 

3.2.3 Work Context 

Table 6 compares the work context in the cockpit and locomotive. Very relevant to CRM 

training is the high ranking from both crews in terms of importance of understanding the 

consequences of making an error. CRM is specifically implemented in high reliability industries 

because serious consequences of making an error exist. During the past several years, CRM 

researchers have suggested that CRM is an error countermeasure and that error management 

should be the leading justification for CRM initiatives (Helmreich, Merritt, & Wilhelm, 1999). 

Aside from the consequence of error, Table 6 shows that in both environments, crewmembers are 

responsible for others’ health and safety. Accuracy is important because they spend much of 

their time sitting, coordinating, and leading others, as well as using their hands to handle, control, 

or feel objects, tools, or controls.
25 

Table 4. Top Four Airline Pilot Tasks Groups and Corresponding 

Locomotive Engineer and Conductor Task Groups 

Work Task 

Airline Pilots, Copilots, and 

Flight Engineers

Importance Rank* & Task 

Statement 

Locomotive Engineer

Importance Rank* & Task 

Statement 

Locomotive Con

Importance Rank* & T

Communication 

1. Brief crews about flight details, 

such as destinations, duties, and responsibilities. 

4. Confer with flight dispatchers and 

weather forecasters to keep abreast of flight conditions. 

5. Contact control towers for takeoff 

clearances, arrival instructions, and other information using radio equipment. 

1. Confer with conductors or traffic 

control center personnel via radiophones to issue or receive information concerning stops, delays, or oncoming trains. 

1. Confer with engineers regarding tr

cargoes, and discuss alternative roobstructions occur. 

15. Receive instructions from dispatch

routes, timetables, and cargos.

Resource 

Checking 

2. Check passenger and cargo 

distributions and fuel amounts to ensure that weight and balance specifications are met. 

2. Inspect locomotives to verify 

adequate fuel, sand, water, and other supplies before each run, as well as to check for mechanical problems. 

10. Inspect each car periodically duri11. Inspect freight cars for complianc

and record car numbers and seal 

Interpreting and 

Choosing 

Route 

3. Choose routes, altitudes, and speeds 

that will provide the fastest, safest, and smoothest flights. 

3. Interpret train orders, signals, and 

railroad rules and regulations that govern the operation of locomotives. 

2. Confirm routes and destination inf5. Review schedules, switching order

records to obtain cargo loading anand plan work. 

Coordinate and 

Direct Task 

Activities 

6. 7. 

 Coordinate flight activities with 

ground crews and air-traffic control, and inform crewmembers of flight and test procedures. 

 Direct activities of aircraft crews 

during flight. 

 N/A 3. Direct engineers to move cars to fi

configurations, combining or sepabreak up trains. 

6. Signal engineers to begin train run

speed, using telecommunications e

7. Supervise and coordinate crew act freight and passengers, and provid porter, maid, and meal services to 

* The Importance Rank number associated with each statement in the table is the O*NET value (O*NET data online Janua

value does not represent one of the four tasks referred to in Section 3.2.1. 
26 

Table 5. Top Six Airline Pilot Work Activities and Corresponding 

Relative Importance Scores* for Locomotive Engineers and Conductors 

Work Activity 

Airline Pilots, Copilots, and 

Flight Engineers Importance Score 

(rank in parenthesis)* 

Locomotive Engineer 

Importance Score 

(rank in parenthesis)* 

Operating vehicles, mechanized 

devices, or equipment 100 (1) 96 (1) 

Getting information 96 (2) 79 (3) 

Inspecting equipment, structures, or 

material 92 (3) 92 (2) 

Monitoring processes, materials, or 

surroundings 92 (3) 75 (5) 

Documenting/recording information 83 (5) 50 (8) 

Identifying objects, actions, and 

events 83 (5) 62 (7) 

* The Importance Score value for each occupation in the table is the percent of O*NET survey respondents assigning this 

priority (O*NET data online January 15, 2006). The ranking in the parenthesis is the rank O*NET observed for the listeoccupation.
27 

Table 6. Top Six Airline Pilot Work Context Factors and Corresponding 

Relative Importance Scores* for Locomotive Engineers and Conductors 

Work Context 

Airline Pilots, Copilots, 

and Flight Engineers

Importance Score 

(rank in parenthesis)* 

Locomotive Engineer 

Importance Score 

(rank in parenthesis)* 

Loco

Im

(ran

Consequences of error 97 (1) 88 (1) 

Responsible for other’s 

health and safety 90 (2) 88 (1) 

Importance of being accurate 85 (3) 69 (5) 

Spend time sitting 80 (4) 69 (5) 

Spend time using hands to 

handle, control, or feel objects, 

tools, or controls 

80 (4) 88 (1) 

Coordinate or lead others 70 (6) 22 (31) 

* The Importance Score value for each occupation in the table is the percent of O*NET survey respondents assigning this 

priority (O*NET data online January 15, 2006). The ranking in the parenthesis is the rank O*NET observed for the listeoccupation.
28 

3.2.4 Knowledge, Skills, and Abilities 

Tables 7, 8, and 9 show the KSAs needed to perform the specified task and work activities 

specific to each crew. As the data in Table 7 show, the knowledge of transportation, public 

safety and security, geography, mathematics, and telecommunications are considered to be of the 

highest importance for both cockpit and train crews. The degree of overlap in the types of skills 

needed for both types of crews is also high. For example, as seen in Table 8, operation and 

control, coordination, operation monitoring, and active listening skills are valued as highly 

important in both crews. Finally, Table 9 shows that control precision, problem sensitivity, near 

vision, and oral expression are important abilities needed for both airline and locomotive crews. 

The table suggests, however, that spatial orientation and depth perception abilities, which are 

important in the cockpit, are less important in the locomotive. 
29

Table 7. Top Six Airline Pilot Knowledge and Corresponding 

Relative Importance Scores* for Locomotive Engineers and Conductors 

Airline Pilots, Copilots, and Flight Locomotive Engineer Locom

Engineers Knowledge Importance Score Im

Importance Score (rank in parenthesis)* (rank 

(rank in parenthesis)* 

Transportation 100 (1) 92 (1) 

Public safety and 65 (2) 54 (2) 

security 

Geography 65 (2) 42 (6) 

Physics 65 (2) 21 (12) 

Mathematics 60 (5) 50 (4) 

Telecommunications 60 (6) 54 (3) 

* The Importance Score value for each occupation in the table is the percent of O*NET survey respondents assigning this 

priority (O*NET data online January 15, 2006). The ranking in the parenthesis is the rank O*NET observed for the listeoccupation.
30

Table 8. Top Six Airline Pilot Skills and Corresponding 

Relative Importance Scores* for Locomotive Engineers and Conductors 

Skill 

Airline Pilots, Copilots, and Flight 

Engineers

Importance Score 

(rank in parenthesis)* 

Locomotive Engineer

Importance Score 

(rank in parenthesis)* 

Locom

Imp

(rank 

Operation and control 100 (1) 88 (1) 

Coordination 96 (2) 62 (6) 

Judgment and 

decisionmaking 96 (2) 62 (6) 

Operation monitoring 96 (2) 75 (5) 

Active listening 92 (5) 75 (4) 

System’s evaluation 81 (6) 46 (14) 

* The Importance Score value for each occupation in the table is the percent of O*NET survey respondents assigning this 

priority (O*NET data online January 15, 2006). The ranking in the parenthesis is the rank O*NET observed for the listeoccupation.
31

Table 9. Top Six Airline Pilot Abilities and Corresponding 

Relative Importance Scores* for Locomotive Engineers and Conductors 

Airline Pilots, Copilots, and Flight Locomotive Engineer Loco

Engineers Ability Importance Score Im

Importance Score (rank in parenthesis)* (ran

(rank in parenthesis)* 

Control precision 85 (1) 60 (1) 

Problem sensitivity 85 (1) 50 (3) 

Spatial orientation 85 (1) 40 (15) 

Near vision 80 (4) 45 (6) 

Oral expression 80 (4) 45 (6) 

Depth perception 75 (6) 40 (15) 

* The Importance Score value for each occupation in the table is the percent of O*NET survey respondents assigning this 

priority (O*NET data online January 15, 2006). The ranking in the parenthesis is the rank O*NET observed for the listeoccupation.
 32 

3.2.5 Analysis and Effect on CRM-Related Accidents 

Overall, the O*NET data indicate a great deal of overlap between cockpit and train crews in 

terms of the types of tasks and work activities that they perform, as well as the KSAs needed to 

accomplish those tasks. This parallels previous research on different types of teams in the 

industry (McGrath, 1984). What becomes apparent from reviewing these tables is that they 

reflect the importance of teamwork skills in performing the functions of cockpit and locomotive 

crews. Because the tasks and work activities involve coordinating and communicating, 

teamwork skills/abilities, such as coordinating, communicating, decisionmaking, active listening, 

and oral expression, are important. CRM’s goal to positively influence crewmembers’ attitudes 

toward teamwork and teamwork skills is in line with both cockpit and locomotive crews’ overall 

function. 

The impact of CRM on accidents, however, is dependent on the baseline of accidents caused by 

failures of the team without CRM training. If no (or few) accidents in an industry are caused by 

a lack of teamwork, then a training program aimed at improving teamwork will consequently not 

be effective at decreasing accidents. As described earlier in this report however, accidents 

resulting from lack of teamwork and other human factors causes are the most common type of 

accident in the railroad industry today. These accidents have specifically led to NTSB 

suggesting that CRM training be used to combat these accidents (NTSB, 1999a). This is similar 

to the airline industry in terms of the number of accidents attributed to lack of teamwork in the 

cockpit before CRM training (Foushee, 1984). This suggests that a training program aimed 

specifically at changing those behaviors (i.e., CRM) should have a positive effect on accidents in 

both the airline and railroad industries. The results of the utility analysis support this finding. 

3.3 Scope of CRM Training in the Railroad versus the Airline Industry 

Although CRM has a positive effect on accidents in both industries, the utility analyses suggest 

that the effect is not as large and does not last as long in the railroad industry, resulting in a 

smaller overall utility estimate in the railroad industry compared with that of the airline industry 

(despite the lower cost of CRM per trainee in the railroad industry). In order to explore why the 

effectiveness of CRM is smaller, lasts for a shorter duration, and the cost per trainee is less for 

railroad locomotive crews, the research team looked at differences between the industries in 

terms of the overall scope of CRM training in each industry. 

It is suggested that the overall pervasiveness or scope of CRM training can influence its 

effectiveness (FAA, 2004; Helmreich, Chidester, Foushee, Gregorich, & Wilhelm, 1990). In 

other words, differences between CRM training initiatives themselves can moderate CRM’s 

effect on different criterion measures. As described in the utility analyses, the effectiveness of 

CRM training (d), the duration of training effect (T), and the cost of accidents (reflected in SDy) 

were obtained from SMEs in both the railroad and airline industries using a structured interview 

form (see Appendices B and D). Recognizing that not all CRM training is the same within or 

between industries and that each SME has his/her own criteria for what constitutes CRM 

training, researchers asked each SME a variety of questions regarding his/her conceptualization 

of CRM training. The information gathered on the length of initial training, content, methods 

used, degree of practice and feedback, and degree of institutional support helped to determine 
 33 

each expert’s variance in what CRM training comprises. Understanding that experts used their 

specific conceptualizations of CRM when answering the questions on the structured interview 

form, researchers could see how these conceptualizations related to estimates of specific utility 

parameters derived from those questionnaires. Specifically, this allowed the research team to 

make judgments regarding how CRM training length, content, methods used, degree of practice 

and feedback, and institutional support were related to its effectiveness, the duration of that 

effect, and the cost of CRM training per trainee. 

The data suggest that although some variation exists in how aviation SMEs conceptualize CRM 

training, the largest difference is between the airline and railroad industries. The following 

sections will describe and discuss various differences in the scope of CRM training between the 

industries. To accomplish this, the research team compared and contrasted CRM in the aviation 

industry and railroad industry in three phases, which were (1) Initial Indoctrination/Awareness, 

(2) Recurrent Practice and Feedback, and (3) Continuing Reinforcement. Specifically, 

researchers suggest that differences between airline and railroad industry practices, in terms of 

the extent of recurrent practice, feedback, and continuing reinforcement of CRM objectives, can 

explain the differences in the effectiveness, duration, and cost of CRM between these industries. 

3.3.1 Indoctrination/Awareness Training 

CRM researchers assert that trainees must have the knowledge and understanding of concepts 

related to CRM before starting to practice CRM techniques in the field (Helmreich et al., 1990). 

Because trainees are most often unfamiliar with CRM and its concepts, the cognitive training 

methods through which knowledge is transferred to the trainees is of the utmost importance. 

Called initial indoctrination or awareness training, knowledge transfer normally takes place in a 

classroom setting, where the initial CRM concepts are taught (Salas et al., 2001). 

The aviation SMEs interviewed, as well as published reviews of aviation CRM, suggest that 

classroom/awareness training lasts between 1 to 2 1/2 days. Furthermore, SMEs state that CRM 

content domain includes topics such as coordination, situational awareness, communication, 

judgment and decisionmaking, threat and error management, assertiveness, stress and fatigue 

management, and command leadership. This parallels reviews of aviation CRM in the literature, 

as well as the authors’ firsthand experiences with aviation CRM courses, and indicates that the 

majority of CRM training teaches the content suggested by the FAA circular guidelines (FAA, 

2004). 

Similarly, information from SMEs indicate that little variance exists in terms of the methods 

used in aviation awareness training. As stated previously, these include classroom lectures but 

usually incorporate group exercises, video presentations, and role-play. Past reviews of CRM 

and firsthand experience with awareness training suggest that these methods are in fact used. 

Specifically, during classroom instruction, a heavy reliance exists on discussion and analysis of 

case studies of real life accidents caused by the failure of the crew to work together. Instructors 

of airline CRM training are actual pilots who have qualified to facilitate CRM training. This 

qualification entails participating in classes on how to facilitate CRM classes specifically. 

Before instructors can facilitate a class by themselves, they must be observed and signed off on 

by a qualified CRM facilitator. 
 34 

Railroad SMEs suggest that the CRM awareness and indoctrination training in the railroad 

industry is equivalent to that of the aviation industry in terms of length, content, and methods 

used. Specifically, SMEs state that initial railroad CRM lasts 1-2 days and covers content related 

to communication, human factors, conflict resolution, technical proficiency (as it relates to 

CRM), situational awareness, teamwork, coordination, and assertiveness. Like the aviation 

industry, awareness training takes place in a classroom setting, using group discussion, exercises, 

and case studies. A review of other railroad CRM programs suggests that the length, content, 

and methods used have been similar across the railroad industry (Morgan et al., 2003). Although 

different titles are sometimes given to identical topics, in general the same concepts are covered 

in the airline and railroad industries. The only difference is related to lines of authority and 

leadership, which results from different command structure in a locomotive cab than that which 

occurs in the cockpit. However, this difference seems negligible in terms of influencing the 

overall effectiveness, duration, or cost of CRM. The characteristics and qualifications of the 

facilitators themselves are also similar to the airline industry. They are former conductors or 

locomotive engineers who have gone through facilitation training. 

A more detailed review of railroad CRM training materials indicates that various modifications 

can and have been made to aviation-based CRM awareness programs, making them more 

applicable to the crews in the railroad industry, and thus equivalent to that in aviation (Morgan et 

al., 2003). TTI recently performed one of the most comprehensive developments, testing, and 

evaluation of a railroad-CRM training program (Morgan, Olson, Kyte, & Roop, 2005). TTI 

developed separate CRM training materials for various crews in the railroad industry, including 

transportation, engineering, and mechanical crews. Each CRM track has language, exercises, 

and case studies specific to each training audience. Moreover, the case studies involve railroad 

accidents (compared to aviation accidents), making this program more parallel to programs in the 

airline industry. CP, as well as a group including AAR, UP, and NS, have developed other CRM 

programs for locomotive crews, which are in use in some fashion in the railroad industry today 

(Morgan et al., 2003). 

In summary, CRM training materials have been adapted from the airline industry to the railroad, 

and many railroads have already implemented some sort of CRM training, although mostly on 

limited and short-term basis (Morgan et al., 2003). These programs are very similar to those of 

aviation in terms of content, methods used, and length of training. Therefore it is unlikely that 

differences found in the utility analysis regarding the effect of CRM, its duration, or the overall 

cost of CRM between the airline and railroad industries, are the result of differences in 

initial/awareness training. 

3.3.2 Practice and Feedback 

Practice and feedback is seen as a necessary component of skill acquisition (Cannon-Bowers, 

Rhodenizer, Salas, & Bowers, 1998; Pescuric & Byham, 1996). The practice and feedback 

phase of CRM training initiatives gives trainees an opportunity to perform work activities while 

obtaining feedback from the instructor and/or other trainees. The research team’s SMEs and the 

current literature suggest that the most common method for practicing and receiving feedback on 

CRM skills in the airline industry is in cockpit simulators using the Line-Oriented Flight 

Training program (LOFT) or through actual flight audits using Line Operations Safety Audit 

(LOSA). Originally developed by the U.S. Navy, LOFT incorporates a broad category of flight 
 35 

simulations that may be used for a number of different purposes (Guzzo & Dickson, 1996). In a 

typical LOFT simulation, the entire team that is involved in a mission follows procedures as if an 

actual flight was taking place. These procedures include several phases, such as pre-flight 

preparation, the use of different communication systems, and all necessary paperwork. 

Participants are presented with realistic problems and must utilize communication and 

decisionmaking strategies that are essential to CRM (Guzzo & Dickson, 1996). After the 

exercise is complete, the entire team is debriefed on their performance, including feedback on 

when they exhibited proper CRM skills or should have exhibited them. Training scenarios are 

developed specifically for these simulators to bolster these CRM competencies (Salas et al., 

2000). 

Compared to LOFT, LOSA involves trained observers riding along in actual aircraft cockpits to 

observe threatening situations encountered by the crews, errors made by the aircrew, and how 

crews handle the threatening situations (Helmreich, Klinect, Wilhelm, & Sexton, 2001). These 

observational data, along with pilot/co-pilot interviews, are put into a LOSA database, which can 

be used to study the actual degree to which pilots are employing certain CRM skills and their 

performance. The observers debrief the crewmembers and provide feedback to the pilots in 

terms of the CRM-related skills that they exhibited during the flight and the opportunities they 

had to do so but did not. 

Railroad SMEs suggest that less practice and feedback of CRM skills in their particular 

organization exist. Practice and feedback that occurs is during yearly on-the-job CRM 

training/assessments, which are similar to that of LOSA. Specifically, instructors or coaches 

observe road and yard crew’s CRM skills while riding along with them during normal 

operations. The instructors take notes on a standardized appraisal form; following these ridealongs, instructors provide feedback to the crewmembers. Like LOSA, its effectiveness is 

largely dependent on the skills, attitudes, and motivation of the trainers/mentors. These trainers 

are selected based both on their skills and abilities to be effective instructors. In addition, they 

receive special training as observers so that they can more readily identify all crew actions or 

inactions related to CRM skills. 

Compared to the airline SMEs, wide use of simulators as a context to allow practice and 

feedback was not currently part of the railroad SMEs’ conceptualization of implementing CRM 

training. In fact, simulators are not as critical a component of the training in the railroad industry 

as in aviation. Compared to the airline industry, relatively few simulators are in use in the 

railroad industry. As noted in TTI’s previous studies, the locomotive simulators that do exist are 

used largely to train only locomotive engineers by themselves rather than to train both an 

engineer and a conductor as a crew or in combination with dispatch training (Morgan et al., 

2003). 

The absence of practice and feedback in a simulated environment could account for industry 

differences in utility parameters. Many of the SMEs consider the practice and feedback phase of 

CRM training an essential component of CRM training and a key to its effectiveness. 

Specifically, the advantage of LOFT, or other simulator-based scenario exercises, is their 

usefulness in practicing highly important but very infrequent behaviors. For example, most 

commercial pilots will complete hundreds of hours of actual in-flight training without 
 36 

experiencing an engine fire. However, should this situation arise, a pilot must have the training 

necessary to respond appropriately. Simulators allow this situation to occur and closely mimic 

the conditions a pilot would face inside a real plane, without exposing the pilot to such a 

potentially dangerous flight condition. Studies show that simulators have proven to be highly 

effective in reducing human factor errors. In the aviation industry, a meta-analysis of flight data 

by Jacobs, Prince, Hays, and Salas (1990) found that combining simulator training with training 

on actual aircraft was more effective than training on the aircraft alone. The benefits of teaching 

infrequent behaviors using simulators are also apparent in the railroad industry.

In summary, the results of this research study suggest that the practice and feedback phase of 

CRM is more in depth in the airline industry than in the railroad industry. Although both have 

active ride-along programs that include feedback, the amount of practice and feedback in a 

simulated environment, which is considered to be an important component of effectiveness 

(Jacob et al., 1990), is far more extensive in the airline industry. It seems reasonable that this is 

one explanation for the smaller effect size (d) and duration of effect (T) parameters found in the 

railroad utility analyses. Furthermore, simulators are very expensive; thus their lower use as an 

overall part of the training program in the railroad industry could also account for the lower cost 

per trainee (C) found in that industry. 

3.3.3 Continuing Reinforcement 

Research has determined that training effectiveness depends not only on training content, 

methods used, the degree of practice and feedback, or the instructor’s ability, but also on factors 

beyond the immediate training environment (Helmreich et al., 1990). Taking a systems view of 

training helps trainers understand how organizational, situational, and trainee characteristics 

influence training effectiveness by their influence on pre-training and post-training motivation 

(Salas et al., 2000). The third phase of CRM training comes from this systems perspective, 

which looks at training in the larger context of the organization and its environment. Thus, this 

phase of training is not specific to the training program itself but instead represents variables 

related to the continuing reinforcement of CRM outside formal practice and feedback. Although 

this phase of training is separate from the training itself, it is considered an essential component 

because it is vital in making the awareness and practice/feedback phases of training come to 

fruition and affect job performance (Salas et al., 2000). 

Variables related to continuing reinforcement include a variety of post-training organizational, 

situational, and trainee characteristics that can influence the degree to which trainees are 

motivated to use what they learned in training on the job. For example, consistent with the 

general literature (e.g., Rodger & Hunter, 1991), airline SMEs participating in this study 

suggested that a high level of institutional support is necessary for CRM to have an effect on 

organizational performance measures. Other organizational characteristics included supervisor 

support and participation (Baldwin & Ford, 1988), a reward system that reinforces safety 

compliance along with on-time performance (Hackman, 1990), an environment conducive to the 

positive transfer of the training skills in the work space (Tracey, Tannenbaum, & Kavanagh, 

1995), and a continued commitment to use CRM to enhance an overall safety culture 

(Helmreich, in press). In summary, the relevant literature and the team’s SMEs suggested that 

for CRM to be effective, it needs to take place in an organization that has a good overall safety 

climate. 
 37 

SMEs participating in this study suggest that the transfer climate of CRM is positive in the 

airline industry. First, the SMEs suggest a great deal of upper management currently supports 

CRM in the airline industry. Interviews with railroad SMEs suggest that like the airline industry, 

quite a bit of institutional support for CRM exists within the railroad industry. Although it is 

difficult to conclude the exact safety climate of the particular railroad from which these SMEs 

were taken, the literature does suggest that some negative perceptions do exist between front-line 

employees and management in the railroad industry (FRA, 2001; 2002). For example, the 

confrontational relationship between labor and management may be the result of union-trade 

management disputes, as well as other related characteristics of the railroad industry, including 

its military rules-based roots and regulations regarding accident liability (i.e., Federal Employees 

Liability Act (FELA)) (FRA, 2001; 2002; TRB, 2006). Specifically, railroad experts from labor, 

government, consulting, academia, and the railroads themselves suggest that this negative 

climate can be a barrier to human-centered approaches to railroad safety (FRA, 2001; 2002). 

In summary, although researchers were unable to determine if this is the case in this railroad 

SMEs’ organization, a review of the literature suggests that further improvements leading to the 

growth of a more positive safety climate in the railroad industry may be necessary to encourage 

long-term reinforcement of CRM principles. Specifically, several indications exist that the 

safety climate in the railroad industry might limit the overall scope of CRM training in the 

industry. Strong resistance to some of the main CRM concepts was encountered in the authors’ 

firsthand experience with pilot training of CRM in the railroad environment. Specifically, 

evaluation and feedback from transportation and engineering trainees reflected their 

apprehension regarding CRM training effectiveness if first-line supervisors did not also 

participate in the new training and put its concepts into action (Morgan et al., 2005). As a result, 

the safety objectives set forth by CRM training would most likely be perceived as discordant by 

trainees working in what they observe to be a negative safety climate. As a result, the 

differences in the perceived safety climate could be one explanation for the smaller effect size 

(d) and duration of effect (T) found in the utility analysis. 

Based on the preceding review, the work activities, tasks, and the context in which work takes 

place are similar in both cockpit and train crews. Furthermore, the types of behaviors that are 

necessary to accomplish those work activities and tasks in those environments are similar. Like 

the airline industry before CRM, a large percentage of accidents in the railroad industry are the 

result of a failure of the team (Morgan et al., 2005). Thus, unfortunately, like the cockpit crews, 

simply being part of a train crew does not automatically lead to effective teamwork or behaviors 

necessary to accomplish the work activities and tasks in those environments. Differences 

between the railroad and airline industries in terms of the amount of practice and feedback and 

continuing reinforcement of CRM can, however, explain some of the differences in 

effectiveness, duration, cost, and overall accident-related cost savings found in the utility 

analyses described in Chapter 2 and evaluated further in Appendices C, E, F, G, and H. 

 39 

4.0 Conclusions and Recommendations 

4.1 Conclusions 

For this report, the research team chose to look at only one performance measure to assess the 

utility of CRM training to the railroad industry—potential for reduced accident costs. This 

approach to estimating the benefits of a CRM training program should be viewed as the most 

conservative manner to calculate such benefits. The reductions are based solely on the expected 

reduction in accidents whose primary cause is attributed to human factors and which have 

exceeded the level of reportable damage (approximately $6,700 or more). Due to limits in how 

accidents are reported and the number of accidents in which human factors is a secondary cause, 

the actual benefits of implementing CRM training programs may be much higher. Improved 

human factors performance can also prevent accidents related to mechanical failure in certain 

cases if improved coordination and teamwork are put into place as a result of CRM training. 

Although FRA’s current Safety Action Plan suggests several specific technology- and rule-based 

approaches to reduce the number and severity of human factors accidents occurring in the 

railroad environment today, the broader application of personnel safety training initiatives, such 

as CRM training to alter this trend, is also indicated and necessary. Research findings indicate 

that in addition to traditional technology-based and regulatory approaches, FRA should 

encourage railroads to more fully implement specific human factors training programs, such as 

CRM, which can improve compliance with existing or even improved procedures/rules. Beyond 

this, it can increase the coordination and efficiency of railroad operations by engendering 

improved teamwork across many different levels and crafts. The findings indicate that direct 

financial savings from reduced accident costs are likely following implementation of a CRM 

training program and that additional financial and safety benefits should accrue in addition to 

those from reduced accident costs alone. These benefits would be in terms of increased 

efficiency in crew operations, increased overall productivity at several levels as the program 

expands to encompass other crafts, avoided litigation costs due to the decrease in the number of 

accidents, and reduced accident cleanup and remediation costs. 

Several industry trends point toward continued increases in human factors-caused accidents 

unless many different mitigation methods are brought to bear on the problem. Among these are: 

• Increasing rail traffic on a relatively fixed capacity network 

• Increasing impacts of fatigue and limited rest cycles upon current railroad employees 

• Increased hiring and decreased average experience of railroad operating personnel (i.e., 

train crews, maintenance-of-way) as a result of both retirements and the rising demand 

for rail transportation 

• Increased demand for inland distribution by rail related to increased importation of 

foreign goods 

All of these problems stem from the recent increases in rail operations. More and more, railroad 

personnel will be required to meet growing freight transportation needs as railroad companies try 

to maintain freight market share. As a result, human factors-caused errors are not likely to 
 40 

decrease unless both technological breakthroughs and personnel training interventions are 

employed to limit accidents through better communication, coordination of efforts, use of 

available resources, and teamwork. 

Wider implementation of CRM training throughout the railroad organizations examined in this 

study should show positive benefits in the long term. It is important to remember that CRM 

training is a long-term training intervention. As the researchers’ examination of the airline 

industry has shown, short-term results in accident prevention that can be linked directly to CRM 

training are unlikely to occur. Over time, however, the cumulative effects should begin to be 

recognized based upon the experience of other industries with CRM as recommended by NTSB. 

4.2 Recommendations 

4.2.1 Training Methods Changes that Can Reinforce CRM Implementation 

The research team found that, despite the current differences between the railroad and airline 

industries, quantifiable utility exists to implement CRM training within the railroad industry. 

Researchers believe, however, that it is possible for the railroad industry to implement several 

initiatives to achieve even greater benefits from CRM training. As stated previously, the 

research team observed that rail simulators are used principally to train only locomotive 

engineers rather than to train an engineer and conductor as a crew, which would be more 

effective for teaching and reinforcing CRM skills. Training in this manner would add to the 

realism of the simulation and make it a forum where CRM skills can be practiced by participants 

and observed by the instructor. Interaction with other associated crafts can also be a part of such 

crew-based simulator use. For example, the use of CRM principles in the crew’s 

communications with a dispatcher can be evaluated if the instructor takes on the role of the 

dispatcher in providing input to the crew. Alternatively, an actual dispatcher or dispatcher 

trainee could be included in a simulator-training scenario that would evaluate both the crew (e.g., 

engineer and conductor) and the dispatcher. CRM behaviors and skills could be included as part 

of the evaluation and debriefed by the instructor at the end of each training exercise. 

Another possible solution to the availability problem is the use of lower fidelity simulators 

(Motowidlo, Hanson, & Crafts, 1997). Baker, Prince, Shrestha, Oser, and Salas (1993) were able 

to demonstrate that a tabletop computer-based simulator used for CRM training was accepted by 

aircrews as a format and acceptable as a method for training CRM skills. Bowers, Salas, Prince, 

and Brannick (1992) also found a low-fidelity simulation to be an acceptable, lower cost 

alternative to high-fidelity simulations for a helicopter flight task. Finally, low-fidelity 

simulations have been found to be accurate predictors of future job performance (Motowidlo, 

Dunnette, & Carter, 1990). This trend to lower fidelity simulation has begun even within the 

railroad industry as some large railroads have moved from the use of full-sized locomotive cab 

simulators to less complicated and less expensive console-based simulators using large computer 

monitors rather than projected images for visual cues. The lower cost of these simulators allows 

for more to be purchased and distributed throughout the railroads’ territory. 

In summary, several training opportunities are available to the railroad industry that might 

increase the scope of CRM training, including increased opportunity for practice and feedback of 
 41 

CRM skills, as well as continuing reinforcement (i.e., improved safety culture). Although the 

costs of CRM training would likely increase as a result of increased practice and feedback, these 

would most likely be offset by the increased effectiveness and duration of CRM training. These 

would likely result in even greater accident-related cost savings as a result of CRM training than 

this utility analysis suggests. 

4.2.2 Implementation of CRM 

CRM training has primary safety benefits that should be the most important reason for 

implementing CRM—regardless of cost; however, based upon the findings of this study that 

CRM training can also provide net positive financial benefits in even the most conservative case 

of reduced reportable accident costs alone, several railroads have and will continue to implement 

CRM training. Managers at those companies understand the benefits of human factors training 

and will support further growth of CRM as it proves its worth. As a result, FRA, as a proponent 

of CRM training, should continue to encourage railroads to begin or continue development of 

improved CRM training programs. This is the role that FAA took with CRM for many years in 

the airline industry—allowing acceptance by the individual airlines until CRM training proved 

itself and was widely accepted. FAA has, since 1998, issued Advisory Circulars, which outline 

the desired components of CRM that should be incorporated into each airline’s program. 

In the case of rail CRM, the introductory pilot rail CRM training materials developed by TTI and 

submitted to FRA in 2005 could be used as a basis for those railroads starting new programs, but 

eventually each railroad will need to modify the course to meet its own needs and focus on error 

types that it needs to address. Facilitators at each railroad will need to understand CRM 

principles and be able to get those points across to the students using pertinent examples from 

their own daily activities. This type of front-line acceptance and buy-in will be required for any 

program to be successful as will management support and reinforcement of CRM principles. 

4.2.3 Phase In of CRM Training 

Implementing CRM throughout the railroad industry is a massive undertaking that will take 

several years. As pointed out earlier in this report, phasing in of this program will consist of 

three phases—initial CRM awareness training, practice and feedback of CRM behaviors and 

skills on the job, and reinforcement of CRM principles throughout each of the organizations in 

which it is adopted. Without careful planning and coordination of these phases and development 

of support along each progressive step, CRM acceptance will struggle, and its effectiveness may 

be questioned. Because of this fact, railroad managers must look at the program requirements 

and determine how quickly and in what manner CRM training will be implemented at their 

railroad. For example, rather than training all of the company’s employees in 1 year, a longer 

period (e.g., 2 or 3 years) might be employed. During this time, accident rates for a territory that 

has received the training could be compared with one that has not. An immediate drop in the 

number of accidents may not occur until the program is more widely implemented. 

A system for evaluating rail CRM training program effectiveness and for making adjustments 

and necessary improvements to the training program will also need to be developed. Such a 

system would incorporate the lessons learned from other industries regarding program evaluation 

but should also take into account the distinctive characteristics of railroad operational safety. In 
 42 

addition to encouraging individual railroad companies to develop CRM training programs, FRA 

should also consider methods to provide adequate guidance and oversight to ensure 

standardization and nationwide implementation of CRM training.
 43 

References 

Arthur, W. Jr., Bennett, W. Jr., Edens, P. S., & Bell, S. T. (2003). Effectiveness of training in 

organizations: A meta-analysis of design and evaluation features. Journal of Applied 

Psychology, 88, 234-245. 

Arthur, W. Jr., Edwards, B. D., Bell, S. T., Villado, A. J., & Bennett, W. Jr. (2005). Team task 

analysis: Identifying tasks and jobs that are team based. Human Factors, 47, 654-669. 

Association of American Railroads (2006). U.S. Railroad Safety Statistics and Trends. 

Presentation by Peter W. French, January 17, 2006. Obtained via email from William M. 

Browder, Association of American Railroads Director of Operations. 

Aviation Information Resources, Inc. (2004). April 2004 Pilot Hiring Summary. Retrieved 

January 9, 2006, from http://www.jet-jobs.com/guestframe.html. 

Bailey, L. L., & Shaw, R. V. (1996). Flight inspection crew resource management training 

needs analysis. Washington, DC: U.S. Department of Transportation, Federal Aviation 

Administration, Office of Aviation Medicine. 

Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future 

research. Personnel Psychology, 41, 63-105. 

Baker, D., Prince, C., Shrestha, L., Oser, R., & Salas, E. (1993). Aviation computer games for 

crew resource management training. International Journal of Aviation Psychology, 3, 143-

156. 

Bowers, C., Salas, E., Prince, C., & Brannick, M. (1992). Games teams play: A method for 

investigating team coordination and performance. Behavior Research Methods, 

Instruments & Computers, 24, 503-506. 

Cabrera, E. F., & Raju, N. S. (2001). Utility analysis: Current trends and future directions. 

International Journal of Selection and Assessment, 9, 92-102. 

Cannon-Bowers, J. A., Rhodenizer, L., Salas, E., & Bowers, C. A. (1998). A framework for 

understanding pre-practice conditions and their impact of learning. Personnel Psychology, 

51, 291-320. 

Cascio, W. F. (2000). Costing human resources: The financial impact of behavior in 

organizations (4th ed.). Belmont, CA: South-Western College Publication. 

Cascio, W. F., & Aguinis, H. (2005). Applied psychology in human resource management (6th

ed.). Upper Saddle River, NJ: Prentice Hall. 
 44 

Cascio, W. F., & Ramos, R. A. (1986). Development and application of a new method for 

assessing job performance in behavioral/economic terms. Journal of Applied Psychology, 

71, 20-28. 

Clarke, S. (1999). Perceptions of organizational safety: Implications for the development of 

safety culture. Journal of Organizational Behavior, 20, 185-198. 

Cohea, W. W. (1997). Report on the joint industry project on human factors in offshore 

operations. Retrieved January 13, 2003, from Department of Interior, Minerals 

Management Services Web site: http://www.mms.gov/tarprojects/220.htm. 

Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159. 

Cooper, G. E., White, M. D., & Lauber, J. K. (1980). Resource management on the flight deck. 

Proceedings of a NASA/Industry Workshop. [NASA CP-2120]. Moffett Field, CA: 

NASA-Ames Research Center. 

Davies, J. M. (2001). Medical applications of crew resource management. In E. Salas, C. A. 

Bowers, & E. Edens (Eds.), Improving Teamwork In Organizations: Applications of 

Resource Management Training. Mahwah, NJ: Lawrence Erlbaum Associates. p 265-281. 

Federal Aviation Administration. (1989). Cockpit resource management training (Advisory 

Circular Publication No. 120-51A). Washington, DC: U.S. Department of Transportation. 

Federal Aviation Administration. (2004). Advisory Circular: Crew Resource Management 

Training (120-51E). Washington, DC: U.S. Government Printing Office. 

Federal Railroad Administration. (1999). Railroad Safety Statistics Annual Report. 

Washington, DC: U.S. Department of Transportation, Office of Public Affairs. 

Federal Railroad Administration. (2001). An Examination of Railroad Yard Worker Safety

(DOT/FRA/ORD-01-20). Washington, DC: U.S. Department of Transportation, Office of 

Research and Development. 

Federal Railroad Administration. (2002). Human-Centered Incident Investigation Methods for 

the Railroad Industry: Conference Summary (DOT/FRA/ORD-02-06). Washington, DC: 

U.S. Department of Transportation, Office of Research and Development. 

Federal Railroad Administration. (2004). Train Accident Data (by cause). Report generated 

October 15, 2005, from the Office of Safety Analysis: http://safetydata.fra.dot.gov/ 

OfficeofSafety/Query/default.asp?page=inccaus.asp. 

Federal Railroad Administration. (2005). Action plan for addressing critical railroad safety 

issues. Released May 16, 2005. Available at www.fra.dot.gov. 
 45 

Flin, R. (1995). Crew resource management for team in the offshore oil industry. Journal of 

European Industrial Training, 19, 23-27. 

Flin, R., & O’Connor, P. (2001). Applying CRM on offshore oil platforms. In E. Salas, C.A. 

Bowers, & E. Edens (Eds.), Improving Teamwork in Organizations: Applications of 

Resource Management Training. Mahwah, NJ: Lawrence Erlbaum Associates. Inc. 

Flin, R., O’Connor, P., Mearns, K., Gordon, R., & Whitaker, S. (2000). Factoring the human 

into safety: Translating research into practice (Report to Health and Safety Executive). 

University of Aberdeen. 

 

Foushee, H. C. (1984). Dyads and triads at 35,000 feet: Factors affecting group process and 

aircrew performance. American Psychologist, 39, 885-893. 

Gaba, D. M., Howard, S. K., Fish, K. J., Smith, B. E, & Sowb, Y. A. (2001). Simulation-based 

training in anesthesia crisis resource management (ACRM): A decade of experience. 

Simulation & Gaming, 32, 175-193. 

Gregorich, S. E., Helmreich, R. L., & Wilhelm, J. A. (1990). The structure of cockpit 

management attitudes. Journal of Applied Psychology, 75, 682-690. 

Grinde, T. A. (1994). Emergency resource management training. In Proceedings of the second 

international conference on health, safety, & the environment in oil and gas exploration 

and production (Vol, 2, pp. 413-417). Richardson, TX: Society of Petroleum Engineers. 

Guzzo, R. A., & Dickson, M. W. (1996). Teams in organizations: Recent research on 

performance and effectiveness. Annual Review of Psychology, 47, 307-338. 

Hackman, J. R. (Ed). (1990). Groups that work (and those that don’t): Creating conditions for 

effective teamwork. San Francisco, CA: Jossey-Bass Publishers. 

Halamek, L. P., Kaigi, D. M., Gaba, D. M., Sowb, Y. A., Smith, B. C., Smith, B. E., & Howard, 

S. K. (2000). Time for a new paradigm in pediatric medical education: Teaching neonatal 

resuscitation in a simulated delivery room environment. Pediatrics, 106, E45. Retrieved 

May 24, 2003, from http://www.pediatrics.org/cgi/content/full/106/4/e45. 

Harrington, D., & Kello, J. (1991). Systematic evaluation of nuclear operator team skills 

training. Paper presented at the American Nuclear Society, San Francisco, CA. 

Hazer, J. T., & Highhouse, S. (1997). Factors influencing managers’ reactions to utility analysis: 

Effects of SDy method, information frame, and focal intervention. Journal of Applied 

Psychology, 82, 104-112. 

Helmreich, R. (in press). Culture, Threat, and Error: Assessing System Safety. Safety in 

Aviation: The Management Commitment: Proceedings of a Conference. London: Royal 

Aeronautical Society. 
 46 

Helmreich, R. L., Chidester, T. R., Foushee, H. C., Gregorich, S., & Wilhelm, J. A. (1990). How 

effective is Cockpit Resource Management training: Exploring issues in evaluating the 

impact of programs to enhance crew coordination. Flight Safety Digest, 9, 1-17. 

Helmreich, R. L., Merritt, A. C., & Wilhelm, J. A. (1999). The evolution of crew resource 

management training in commercial aviation. International Journal of Aviation 

Psychology, 9, 19-32. 

Helmreich, R. L., Klinect, J. R., Wilhelm, J. A., & Sexton, J. B. (2001). The Line Operations 

Safety Audit (LOSA). In Proceedings of the First LOSA Week (pp. 1-6). Cathay City, 

Hong Kong, March 12-14, 2001. 

Howard, S. K., Gaba, D. M., Fish, K. J., Yand, G., & Sarnquist, F. H. (1992). Anesthesia crisis 

resource management training: Teaching anesthesiologist to handle critical incidents. 

Aviation, Space, and Environmental Medicine, 63, 763-770. 

Jacobs, J. W., Prince, C., Hays, R. T., & Salas, E. (1990). A meta-analysis of the flight simulator 

training research (Publication No. TR-89-006). Orlando, FL: Naval Training Systems 

Center. 

Jentsch, F., & Smith-Jentsch, K. A. (2001). Assertiveness and team performance: More than 

“just say no.” In E. Salas, C. A. Bowers, & E. Edens (Eds.), Improving teamwork in 

organizations: Applications of resource management training. Mahway, NJ: Lawrence 

Erlbaum Associates. 

Jha, A. K., Duncan, B. W., & Bates, D. W. (n.d.). Fatigue, sleepiness, and medical errors. 

Retrieved May 15, 2003, from Agency for Healthcare Research and Quality Web site: 

http://www.ahcpr.gov/clinic/ptsafety/pdf/chap46.pdf 

Lauber, J. K. (1987). Cockpit resource management: Background studies and rationale. In H. 

W. Orlady & H. C. Foushee (Eds.), Cockpit resource management training: Proceedings 

of the NASA/MAC workshop (NASA CP-2455; pp. 5-14). Moffett Field, CA: NASAAmes Research Center. 

Maersk Training Center Web site (n.d.). Retrieved February 11, 2003, from 

http://www.maersktrainingcentre.com/default.asp?page=2&Groupid=3. 

McGrath, J. E. (1984). Groups: Interaction and performance. Englewood Cliffs, NJ: PrenticeHall. 

Mearns, K., Flin, R., Fleming, M., & Gordon, R. (1997). Human and organizational factors in 

offshore safety. Suffolk, UK: HSE Books. 
 47 

Moore, W. H., & Bea, R. G. (1993). Management of human and organizational error in 

operational reliability of marine structures: Final joint industry project report, report No. 

HOE-93-1. Retrieved January 14, 2003, from U.S. Department of the Interior, Minerals 

Management Service Web site: http://www.mms.gov/tarprojects/167.htm. 

Morgan, C. A, Olson, L. E., Kyte, T. B., & Roop, S .S. (2003). Railroad crew resource 

management (CRM): Survey of teams in the railroad operating environment and 

identification of available CRM training methods. (Technical Report DFTR-091-

00030/474450-00001). College Station, TX: Texas Transportation Institute. 

Morgan, C. A, Olson, L. E., Kyte, T. B., & Roop, S. S. (2005). Railroad crew resource 

management (CRM): Report on pilot rail CRM training development and implementation. 

(Technical Report DFTR-091-00030/474450-00003). College Station, TX: Texas 

Transportation Institute. 

Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: 

The low-fidelity simulation. Journal of Applied Psychology, 75, 640-647. 

Motowidlo, S. J., Hanson, M. A., & Crafts, J. L. (1997). Low-fidelity simulations. In D. L. 

Whetzel & G. R. Wheaton (Eds.), Applied measurement methods in industrial psychology. 

Palo Alto, CA: Davies-Black Publishing. 

National Transportation Safety Board. (1999a). Safety Recommendation (NTSB/R/99/26). 

Washington, DC: U.S. Government Printing Office. 

National Transportation Safety Board. (1999b). Collision of Norfolk Southern Corporation 

Trains 255L5 with Consolidated Rail Corporation Train TV 220 in Butler, Indiana, on 

March 25, 1998 (NTSB/RAR-99/02). Washington, DC: U.S. Government Printing Office. 

National Transportation Safety Board. (2005). Accident and Incident Data System. Retrieved 

October 13, 2005, from the Federal Aviation Administration’s National Aviation Safety 

Data Analysis Center: http://www.nasdac.faa.gov/portal/page?_pageid=33,37327&_ 

dad=portal&_schema=PORTAL. 

National Transportation Safety Board. (2006). U.S. Air Carrier Operations Calendar Year 

2001: Annual Review of Aircraft Accident Data. (NTSB/ARC-06/01). Adopted by the 

NTSB February 14, 2006. Washington, DC: U.S. Government Printing Office. 

Naval Aviation Schools Command. (2003a). Retrieved from http://wwwnt.cnet.navy.mil/crm. 

Naval Aviation Schools Command. (2003b). OPNAV INSTRUCTION 1542.7C from 

http://wwwnt.cnet.navy.mil/crm/crm/opnavinst/1542C.doc. 

Naval Safety Center. (n.d). Helicopter GCT (ground crew coordination training). Retrieved 

August 21, 2002, from http://www.safetycenter.navy.mil/presentations/aviation/helogct.htm. 
 48 

Paton, D., & Flin, R. (1999). Disaster stress: An emergency management perspective. Disaster 

Prevention and Management, 8, 261-267. 

Pescuric, A., & Byham, W.C. (1996). The new look of behavior modeling. Training and 

Development, 50, 25-30. 

Pizzi, L., Goldfarb, N., & Nash, D. B. (n.d). Crew resource management and its applications in 

medicine. Evidence Report/Technology Assessment No. 43, Making Health Care Safer: A 

Critical Analysis of Patient Safety Practices, AHRQ Publication No. 01-E058 (Available 

online at http://www.ahcpr.gov/clinic/ptsafety/chap44.htm). 

Prince, C., Chidester, T. R., Bowers, C., & Cannon-Bowers, J. (1992). Aircrew coordination: 

Achieving teamwork in the cockpit. In R. W. Swezey & E. Salas (Eds.), Teams: Their 

training and performance. Norwood, NJ: Ablex Publishing. 

Risser, D. T., Rice, M. M., Salisbury, M. L., Simon, R., Jay, G. D., & Berns, S. D. (1999). The 

potential for improved teamwork to reduce medical errors in the emergency department. 

Annals of Emergency Medicine, 34, 373-383. 

Rogers, R., & Hunter, J. E. (1991). Impact of management by objectives on organizational 

productivity. Journal of Applied Psychology, 76, 322-336. 

Salas, E., Burke, C. S., Bowers, C. A., & Wilson, K. A. (2001). Team training in the skies: 

Does crew resource management (CRM) training work? Human Factors, 43, 641-674. 

Salas, E., Fowlkes, J. E., Stout, R. J., Milanovich, D. M., & Prince, C. (1999). Does CRM 

training improve teamwork skills in the cockpit? Two evaluation studies. Human Factors, 

41, 326-343. 

Salas, E., Prince, C., Bowers, C. A., Stout, R. J., Oser, R. L., & Cannon-Bowers, J. A. (1999). A 

methodology for enhancing crew resource management training. Human Factors, 41, 161-

172. 

Salas, E., Rhodenizer, L., & Bowers, C. A. (2000). The design and delivery of crew resource 

management training: Exploiting available resources. Human Factors, 42, 490-511. 

Salas, E., Wilson, K. A., Burke, C. S., & Wightman, D. C. (in press). Does CRM training work? 

An update, extension, and some critical needs. Human Factors, 48, 392-412.

Schmidt, F. L., & Hunter, J. E. (1983). Individual differences in productivity: An empirical test 

of estimates derived from studies of selection procedure utility. Journal of Applied 

Psychology, 68, 407-414. 

Schmidt, F. L., Hunter, J. E., McKenzie, R. C., & Muldrow, T. (1979). The impact of valid 

selection procedures on workforce productivity. Journal of Applied Psychology, 64, 609-

624. 
 49 

Schmidt, F. L., Hunter, J. E., & Pearlman, K. (1982). Assessing the economic impact of 

personnel programs on productivity. Personnel Psychology, 35, 333-347. 

Tracey, J. B., Tannebaum, S. I., & Kavanagh, M. J. (1995). Applying trained skills on the job: 

The importance of the work environment. Journal of Applied Psychology, 80, 239-252. 

Transportation Research Board. (2006). Railroad operational safety: Status and research 

needs. Washington, DC: National Academies of Science. 

U.S. Coast Guard Web site. (n.d.) Retrieved from http://www.uscg.mil/stcw/index.htm. 

U.S. Department of Labor (2005). O*NET version 8.0. Washington, DC. Retrieved January 15, 

2006, from the World Wide Web: http://online.onetcenter.org. 

U.S. Department of Labor. (2006). Career Guide to Industries, 2006-07 Edition, Air 

Transportation. Retrieved January 12, 2006, from the Bureau of Labor Statistics: 

http://www.bls.gov/oco/cg/cgs016.html. 

U.S. Department of Labor. (2006b). Occupational Outlook Handbook, 2006-07 Edition, Rail 

Transportation Occupations. Retrieved January 22, 2006, from the Bureau of Labor 

Statistics: http://www.bls.gov/oco/ocos244.htm. 

U.S. Department of Transportation. (1999). Human Centered Systems: The Next Challenge in 

Transportation. Washington, DC: U.S. Department of Transportation, Office of Research 

and Development. 

U.S. Department of Transportation. (2004a). Annual Employee Statistics by Labor Category 

2004 (P10). Retrieved January 12, 2006, from the Bureau of Transportation Statistics: 

http://www.bts.gov/programs/airline_information/number_of_employees/labor_category/ht

ml/2004.html. 

U.S. Department of Transportation. (2004b). Wage Statistics of Class I Railroads in the United 

States (Statement No. A-300). Retrieved January 25, 2006, from the Surface 

Transportation Board’s Office of Economics, Environmental Analysis, and Administration: 

http://www.stb.dot.gov/econdata.nsf/dc81d49e325f550a852566210062addf?OpenView& 

Start=1&Count=300&Collapse=1#1. 

U.S. Department of Transportation. (2006). Report of Railroad Employment–Class I Line-Haul 

Railroads (Statement No. M-350). Retrieved January 25, 2006, from the Surface 

Transportation Board: http://www.stb.dot.gov/econdata.nsf/322683bcf67f414385 

2566210062ac90?OpenView. 

U.S. General Accounting Office. (1997). FAA’s guidance and oversight of pilot crew resource 

management training can be improved: Report to congressional requesters. GAO RCED98-7. Washington, DC: U.S. Government Printing Office.
 50 

Von Thaden, T. L., & Steelman, K. S. (2005). Classifying crew performance failures in 

commercial aviation accidents: Can we get the numbers right? Proceedings of the Second 

Safety Across High-Consequence Industries Conference, St. Louis University. St. Louis, 

MO. 

Webber, S. S., & Klimoski, R. J. (2004). Crews: A distinct type of work team. Journal of 

Business and Psychology, 18, 261-279. 

Wiener, E. L., Kanki, B. G., & Helmreich, R. L. (Eds.) (1993). Cockpit resource management. 

San Diego, CA: Academic Press. 
 51 

Acronyms 

AAR Association of American Railroads 

BLS Bureau of Labor Statistics 

BNSF Burlington Northern Santa Fe 

BTS Bureau of Transportation Statistics 

CBT computer-based training 

CP Canadian Pacific 

CRM Crew Resource Management 

DOL Department of Labor 

DOT Department of Transportation 

FAA Federal Aviation Administration 

FELA Federal Employees Liability Act 

FRA Federal Railroad Administration 

HFACS Human Factors Analysis and Classification System 

HRM human resource management 

KCS Kansas City Southern 

KSAs knowledge, skills, and abilities 

LOFT Line-Oriented Flight Training 

LOSA Line-Oriented Safety Audit 

MARAD Maritime Administration 

NASA National Aeronautics and Space Administration 

NASDAC National Aviation Safety Data Analysis Center 

NS Norfolk Southern 

NTSB National Transportation Safety Board 

O*NET Occupational Information Network 

SMEs subject matter experts 

Safety Action Plan Action Plan for Addressing Critical Railroad Safety Issues 

SP Southern Pacific 

TTI Texas Transportation Institute 

TAMRF Texas A&M Research Foundation 

UP Union Pacific 
52 
 A-1 

Appendix A. 

History/Background of CRM Training 

Origins of CRM Training 

NOTE: The following history of CRM training development and implementation in several 

other industries is taken from the 2003 report on CRM conducted by TTI for FRA. Only minor 

changes to reflect new information have been made. 

Early Development 

Crew resource management (CRM) training was originally developed for use in the aerospace 

industry by the National Aeronautics and Space Administration (NASA) and commercial airlines 

as a means to decrease the number of airline crashes that were occurring as a result of 

breakdowns in crew performance. Aircrews made up of individuals with excellent individual 

aviation skills were often found by accident investigations to have made critical errors in 

teamwork that led to crashes. Investigations conducted into these crashes often included “pilot 

error” when listing out causal factors—both mechanical failures and human factors, that had lead 

up to the accident. 

Repeatedly, such pilot errors were found to be a result of the failure of the flight crew to work 

properly as a team in taking corrective actions or in making flight decisions rather than 

deficiencies in the technical skills of the pilots involved. Critical information known by one 

member of the crew was not passed to others—often due to social or positional barriers related to 

the culture of the airline organization or lack of assertiveness. Research into this problem 

showed that training for pilots and aircrew traditionally focused on technical aspects related to 

flying the aircraft (Prince, Chidester, Bowers, & Cannon-Bowers, 1992), but there was little or 

no training on how to interact as a crew. Later studies found that, in order for flight crews to 

exhibit exemplary performance, three elements were needed—technical competency, 

CRM/interpersonal skills, and an organizational context or culture that supports crew-based 

rather than individual decision making and that encourages dialogue and feedback among crew 

members regarding safety issues (Bailey & Shaw, 1996). 

Cockpit Resource Management 

CRM began as “cockpit resource management” rather than “crew resource management” 

because initially its application was limited only to those members of the aviation industry that 

were present in the cockpit and that made “safety of flight” decisions regarding the plane and its 

in-flight operations. This “flightdeck crew” that was the focus of early CRM training was 

usually limited to the positions of pilot, co-pilot, and when applicable, flight engineer. 

At most major airlines, individuals in each of these positions hold pilot’s licenses with either 

commercial or airline transport pilot ratings and have hundreds of hours of flight time 

experience. Each member of this crew has extensive knowledge of the aircraft’s flight 

characteristics, its performance capabilities, and the approved rules and procedures for dealing 

with a given problem (standardized by the use of checklists), but, in a critical or emergency 

situation, conflicting opinions among these highly-skilled and trained pilots, miscommunication, 
 A-2 

or failure to pass on vital information within the crew could still lead to selection of improper 

actions—resulting in a crash. 

Decisions based solely upon the seniority and experience of the plane’s captain (the pilot-incommand) and his/her perception of the problem often resulted in the failure of the flight crew as 

a whole to select the safest course of action to mitigate the problem. At times, this occurred 

despite the initial objection of other crewmembers, or the lack of assertiveness of junior 

crewmembers in making their objections known. When such input was given, senior pilots often 

disregarded it, instead counting upon their own perception, experience, and skills. 

Unfortunately, these are not always the most reliable sources of information in making flight 

decisions. 

The traditional airline culture supported the captain in this authoritarian position until several 

costly crashes in which this was a factor took place. Junior crewmembers lack of assertiveness, 

failure of the crew to recognize and correct a problem before it became too late, and last words 

such as “I knew you were going to do that” on cockpit voice recorders indicated that something 

had to change. Pilots and the airline industry needed a new way to approach their jobs which 

allowed for more team decision-making and encouraged communication between all members of 

the crew so that no essential information would be omitted that could be used to prevent a crash. 

Identified Need for CRM Training 

The initial need for CRM in the aviation industry was, and continues to be, supported by findings 

by the Federal Aviation Administration (FAA) and others that approximately 30-80% of all 

aviation accidents were due to human error (Lauber, 1987). For example, after studying 169 

accidents that had been investigated by the NTSB, the U.S. General Accounting Office (GAO) 

found that 30% were caused in part by pilot error (USGAO, 1997). In one-third of these 

accidents, the GAO determined that the pilots did not correctly utilize CRM principles. Finally, 

the GAO report concluded that CRM deficiencies contributed to 50% of the serious accidents 

where there was at least one fatality. 

Since the realization that such a significant percentage of aircraft accidents were due to human 

error, CRM training has continued to be widely adopted by the commercial aviation industry, the 

military, and other industries such as medicine, offshore oil production, nuclear power, and 

commercial shipping. Recent FRA statistics attribute human error as a causal factor in 37% of 

all train accidents not related to highway rail grade crossings. (FRA, 1999) These numbers point 

to the need in the rail industry to examine all possible methods to increase safety by predicting 

and preventing human error accidents. 

CRM and its Development 

Defining CRM 

CRM training was initially created to improve aviation safety by helping crewmembers to use all 

the resources available to them (Salas et al., 2000). “Crew resources” can include the training 

and experience of other members of the crew and all information related to the crew’s ability to 

function. Such information can be internal to the crew itself or available from someone outside 
 A-3 

the crew with which they can communicate such as air traffic controllers or aircraft maintenance 

specialists at their home base. Failing to consult or take advantage of these resources has been 

found to be a causal factor in too many accidents. 

Likewise, the performance capability of each flight crew member must also be evaluated based 

upon their physical, mental, or emotional condition at the time of the flight. This facet of crew 

resources can take into account stress, fatigue, illness, or lack of training in a specific area when 

evaluating the ability of an individual to perform certain tasks. Knowledge of these factors and 

learning to identify cues related to them can allow for better allocation of tasks within each crew. 

Keeping them in mind can prevent many errors before they happen. 

The Federal Aviation Administration (FAA) defined CRM in 1989 as “the utilization of all 

available human, informational, and equipment resources toward the goal of safe and efficient 

flight” (FAA 1989, p. 2). In the same document, CRM is further defined in this way: 

CRM is an active process by crewmembers to identify significant threats, to 

communicate, and to carry out a plan and actions to avoid or mitigate each 

threat. CRM also deals directly with the avoidance of human errors and the 

management and mitigation of those errors that occur. CRM reflects the 

application of human factor knowledge to the special case of flight crews and 

their interactions with each other, with other groups and with the technology

in the system (FAA, 1989, p. 2). 

This definition asserts that CRM is a process that crew members should use when in flight 

operations. The authors define this process based on the mitigation and avoidance of threats and 

human error. Alternatively CRM can be defined as a training method. Salas and his colleagues 

define CRM as a “family of instructional strategies that seek to improve teamwork in the cockpit 

by applying well-tested training tools (e.g., simulators, lectures, videos) targeted at specific 

content (i.e., teamwork knowledge, skills and attitudes)” (Salas, Prince, Bowers, Stout, Oser, & 

Cannon-Bowers, 1999, p. 136). Besides coming from a training perspective (compared to a 

process perspective), Salas et al. (1999) write about CRM as being really about teamwork. The 

differences in CRM definitions become apparent when one looks at the diverse skills taught in 

various CRM training programs as shown in Table A-1. 

Development of Commercial Aviation CRM Programs 

United Airlines was the first commercial airline to adopt CRM training for its cockpit crews and 

other companies soon followed suit as positive safety benefits became apparent. The NTSB and 

FAA took notice of the reduction of crashes and incidents as more and more airline companies 

implemented voluntary CRM training programs. Eventually, on March 19, 1998 the FAA 

officially changed its regulations to require CRM training of all airline personnel (Helmreich, 

Merritt, & Wilhelm, 1999). 

Encouraged by the success of CRM training for cockpit crews, commercial airlines began to 

apply it to other disciplines within the aviation industry. It is no longer just the members of the 

cockpit who are undergoing CRM training. Maintenance workers, flight attendants, and aircraft 

dispatchers are all now required to undergo CRM training. While specific statistics on airline 
 A-4 

implementation of CRM in other skill areas are scarce, those that can be found paint a favorable 

picture. 

For example, the U.S. Naval Safety Center cites that, after training two-thirds of Continental 

Airline’s maintenance personnel (approximately 1200 employees) in CRM principles, 

Continental saw maintenance ground damage costs drop by 66% and occupational injuries 

decrease by 27% (Naval Safety Center, n.d). This type of improvement points to the dramatic 

potential for improvements in safety that can accrue from the development of a CRM training 

program within an organization. 

Specialized Training Areas within CRM Training Programs 

In addition to this generational change, several specialized topic areas within CRM training have 

also emerged. These include traditional crew coordination training as described in Wiener, 

Kanki, & Helmreich (1993), attitude training (Gregorich, Helmreich, & Wilhelm, 1990), the 

proper use of assertiveness (Jentsch & Smith-Jentsch, 2001), the role of leadership within CRM 

(Helmreich et al., 1999), and team building (Helmreich et al., 1999). Within each of these areas, 

different methods for presenting and reinforcing CRM training have been developed. 

Differentiation of CRM Skills/ Learning Objectives in Different Applications 

The desired CRM skill sets or learning objectives may be expressed in different manners across 

varying groups within an industry, for CRM application in other industries, and for specific 

training programs. When CRM competencies are compared across training programs and 

industries, a similar set of key subject areas begins to emerge. This set includes subjects such as 

decision-making, assertiveness, crew coordination, leadership, teamwork, situational awareness, 

and active practice and feedback. While the main CRM concepts remain largely the same, these 

essential elements may be organized and presented in a manner that meets the interest of that 

particular industry. For example, the Naval Safety Center’s School of Aviation Safety organizes 

its CRM training program around seven skills: decision-making, assertiveness, mission analysis, 

communication, leadership, adaptability/flexibility, and situational awareness. For rail CRM 

training programs, the NTSB has recommended that training include crewmember proficiency, 

communication, situational awareness, and conflict resolution. (NTSB, 1999a) As shown in 

Table A-1, CRM programs in the medical arena include skills such as priority assessment, 

assertiveness, use of information, communication, leadership, avoidance of preoccupation, and 

situational awareness. The flexibility to present core CRM concepts in different manners allows 

for a more direct training focus within each industry and the ability to relate CRM to associated 

pre-existing training programs. 
 A-5 

Table A-1. Differing Terminology for Core CRM Skills by Industry 

Bridge Resource 

Management U.S. Navy Medical Fields NTSB Commercial Aviation 

Decision Making Decision Making Decision Making Priority 

Assessment 

Crewmember 

Proficiency 

Assertiveness Pilot Judgment Planning Assertiveness Assertiveness 

Mission Analysis Crew Coordination Stress and Fatigue 

Management Use of Information Crew Coordination 

Communication Communication Communication Communication Communication 

Leadership Leadership Error Management Leadership 

Adaptability/ 

Flexibility Teamwork Avoidance of Preoccupation Teamwork 

Situational 

Awareness 

Situational 

Awareness 

Situational 

Awareness 

Situational 

Awareness 

Active Practice 

and Feedback Relationship Issues Active Practice 

and Feedback 

Note: Shaded areas denote common skills listed across two or more industries. 

Adoption of CRM Training into Military Aviation 

The aviation branches of the U.S. military also took notice of the success that CRM training was 

having within civil aviation. Both the U.S. Air Force and the U.S. Navy began to develop and 

implement CRM training programs based upon those found at the airlines during the early 

1990’s. As in the airlines, the military’s motive for doing so was to reduce the number of 

accidents attributable to pilot error that were costing them millions of dollars in lost aircraft each 

year. The military’s initial CRM training programs began by training only the pilots of multipiloted aircraft and have since expanded to include other fields in military aviation such as other 

aircrew members, maintenance personnel, and air traffic controllers. 

While early training materials were adopted, almost wholesale, from the commercial aviation 

industry, in 1993 the U.S. Navy developed its own CRM training program for pilots that it called 

Naval Aircrew Coordination Training (ACT). ACT goals were to increase mission effectiveness, 

minimize preventable error, maximize aircrew coordination, and optimize risk management 

while ultimately reducing the mishap rate caused by human factors (Naval Aviation Schools 

Command, 2003a). The ACT program was ultimately based on a set of seven common 

behavioral skills that a 1991 Navy research and development effort had identified as central to 

aviation mishaps. In 1993, the Navy implemented an interim ACT program designed to address 

these seven skills to some degree while at the same time improving standardization among flight 

training. This interim program developed into the Integrated ACT program in 1995 when a more 

detailed program was developed. In 1998, the Navy upgraded its ACT program once again to 

include annual ground training and annual flight evaluation requirements regarding CRM (Naval 

Aviation Schools Command, 2003b). 
In its current format, outlined in official Navy documents as OPNAVINST 1542.7C, the Navy’s 

CRM initiative includes additional training components such as: 

• standardized data collection 

• crew feedback 

• performance measurement 

• an integrated CRM event-based curriculum (with an emphasis on situational awareness 

and decision making) 

• advanced flight instructor skills (with performance assessment and coaching) 

• decision skills training (with an emphasis on critical thinking) 

• Computer Aided Performance Assessment System (computer analysis of each 

crewmember’s performance in aircraft simulators); and 

• Tactical Integration/Instructional Systems Design (all designed to teach and reinforce 

CRM concepts and measure resulting behavioral changes). 

Instructors in the program focus upon operational risk assessment and management, aircraft 

flight control, communication skills, decision processes and skills, situational awareness, and 

both tactical and standard operating procedures. The Navy has seen positive effects from the 

implementation of its integrated CRM training programs. For example, U.S. Naval helicopter 

personnel trained in CRM were found to score 20% higher on behavioral evaluations and 

perform better on a written knowledge test and a scenario-based test than did another group who 

had not received CRM training (Salas, Fowlkes, Stout, Milanovich, & Prince, 1999). 

Generational Changes in Focus of CRM Training 

Since CRM training first began in the late 1970s (see Cooper, White, & Lauber, 1980), CRM 

within the commercial aviation industry has gone through several stages. Dr. Robert Helmreich 

and his colleagues, in their 1999 paper The Evolution of Crew Resource Management Training in 

Commercial Aviation, described these stages in the following manner: 

• 1st generation: Mainly a program for correcting for lack of junior officer assertiveness 

and pilot authoritarianism 

• 2nd generation: A more team-oriented program including decision-making training 

• 3rd generation: Recognition of human factors issues such as stress and situational 

awareness, extension of training to other teams in the airlines such as flight attendants, air 

traffic controllers, and maintenance personnel 

• 4th generation: CRM program customization by individual airlines to comply with FAA 

mandates regarding CRM training 

• 5th generation: Restored emphasis upon error management and safety 

• 6th generation: Identifying/preventing threats to safety at the earliest possible time and 

managing error. 

This generational perspective on CRM training is widely accepted throughout the commercial 

aviation industry and subsequent industries that have adopted CRM. 

 A-6 
 A-7 

Beyond Aircrews: The Proliferation of CRM Training 

Throughout the last decade, other industries such as medicine, military tank and shipboard 

operations, offshore oil exploration, commercial shipping, and nuclear power have applied CRM 

to crew training for their teams. First the work in these industries is such that in order for key 

components of the job to be performed individuals must work together in teams in order to 

perform those functions. Second, these industries involve high risk and high stress activities, just 

as the airline industry does. Similarly, the rail industry is made up of teams that must work 

together in a high risk, high stress environment. 

Examples of teams that also perform critical operations are hospital surgical teams, tank crews, 

petroleum and oil platforms crews, crews on a ship’s bridge, nuclear power plant control 

operators, and railroad operating crews. Because many of the incidents that occur to these teams 

are caused by human error, just like in the airline industry, CRM training has quickly moved 

outside the cockpit in an attempt to increase safety in these industries. Several of these programs 

are discussed below. 

Bridge Resource Management (Military and Commercial Shipping) 

In their 1993 book on cockpit resource management, Wiener, Kanki, and Helmreich note that, 

due to the number of merchant vessels lost at sea each year (an average of over one per day), 

commercial shipping is a prime area for human factors intervention through CRM training. The 

U.S. Coast Guard, in conjunction with the International Convention on Standards of Training, 

Certification, and Watchkeeping for Seafarers (STCW) has recommended the use of Bridge 

Resource Management (BRM) training as part of a category of safety training for all commercial 

seagoing vessels (United States Coast Guard Website, n.d.). Multiple-day BRM courses are now 

being offered by the Maersk Training Center, the U.S. Merchant Marine Academy’s Global 

Maritime and Transportation School, the Maritime Institute of Technology and Graduate Studies, 

Marine Safety International, Transport Canada, and other instructional facilities such as 

universities and maritime institutes. Such courses typically consist of both classroom instruction 

and bridge simulator training. 

The U.S. Navy has also recognized that human factors errors are a large contributor to the 

number of naval afloat mishaps (Naval Safety Center, 2002). Based upon work done by James 

Reason in 1997, the Navy classified four different types of human factors error that contribute to 

accidents. These include: 

• unsafe acts (including errors and violations); 

• unsafe supervision (including inadequate supervision and supervisor violations); 

• unsafe crew coordination (e.g., crew resource management problems); and 

• poor organizational influences (both external and internal). 

The Naval Safety Center concluded that 91%, 84%, 56%, and 38% of afloat mishaps could be 

related to each of these factors respectively. BRM training for shipboard crews, analogous to 

that used in naval aviation, has been one way to reduce these percentages and improve the safety 

of naval operations. 
 A-8 

One example of a successful commercial CRM training program in the marine industry is at the 

Maersk Training Centre in Svendbord, Denmark. Maersk’s five-day training program has 

different “tracks” which teach navigators and engineers. Skills such as communication, 

teamwork, leadership, and situational awareness are covered. Maersk’s use of carefully selected 

case studies and simulator environments make each course relevant to its target audience. For 

example, engineers practice CRM principles and behaviors in an engine room simulator. Also, 

engine-room case studies, taken from real-life situations, are used to make the concepts relevant 

to the shipboard engineer students. Maersk’s CRM courses are designed to help navigators and 

engineers better coordinate their tasks in both emergency and routine situations (Maersk Training 

Center Website, n.d.). 

Medical Sciences 

Human error is a primary cause of many adverse medical mistakes. For example, between 65-

70% of anesthesiology accidents or incidents are caused partly by human error (Howard, Gaba, 

Fish, Yang, Sarnquist, 1992; Pizzi, Goldfarb, Nash, n.d.). Hospital surgical and emergency 

rooms and other medical settings mirror that found on the flightdeck of an airliner in that a group 

of people has responsibility for human lives, must work with a complicated suite of 

technologically advanced instruments, and must make quick decisions—all with a very narrow 

margin of error. Years of research have shown that many of the same CRM training principles 

that worked well in aviation CRM, will transfer to medical situations. Understanding these 

principles could potentially help surgeons, anesthesiologists, and nurses to function in a team 

environment and to better identify, understand, learn from, and prevent medical errors. 

Although no specific program in the medical field is directly analogous to aviation CRM 

(Davies, 2001), training programs based on CRM type skills have recently appeared in the 

medical sciences at a slow but steady rate. One of the first applications of CRM principles in the 

medical arena was at the Palo Alto Veterans Affairs Medical Center in the Department of 

Anesthesiology Services. Howard et al. (1992), working at Stanford University, designed an 

Anesthesia Crisis Resource Management (ACRM) program that was rated highly by participants 

and increased resident anesthesiologist knowledge of ACRM principles (Howard et al., 1992). 

Other ACRM programs have since been developed and are currently taught at Harvard 

University, as well as the Universities of Copenhagen and Toronto (Davies, 2001). These 

programs are different than traditional CRM programs in that they are based around patient 

simulations, crisis situations, and the individuals trained in groups are specialists within one field 

(Davies, 2001). In addition to anesthesiology, researchers have also theorized how CRM 

principles could be used to train teams in operating, labor and delivery, emergency, and cardiac 

arrest response situations (Gaba, Howard, Fish, Smith & Sowb, 2001; Halamek, Kaegi, Gaba, 

Sowb, Smith, Smith & Howard, 2000; Risser, Rice, Salisbury, Simon, Jay & Berns, 1999). 

Learning content related to traditional CRM, for example fatigue, are also being assessed in the 

medical fields in terms of how it affects human error and decision-making (Jha, Duncan, & 

Bates, n.d.). 

Despite the need for CRM type skills in medicine, the medical sciences have been slow to 

embrace CRM training and the dispersion of general CRM principles in medicine has not 

progressed as fast as it did within the aviation industry. Davies (2001) suggests one of the 
 A-9 

reasons for this reduced integration has been that there is not a medical equivalent to a plane 

crash. That is, a mistake made by a single surgical team does not usually receive national media 

attention or calls for retraining and regulation like a single commercial airline crash may. 

Another reason for the slower acceptance of CRM training in medicine has been that the culture 

of health care professionals is such that medical errors are sometimes ignored (with the hope that 

they will not later be discovered), covered up, or blamed upon others (Davies, 2001). Despite 

the open learning environment that many teaching hospitals foster, doctors are seldom taught to 

learn from their mistakes, are taught to ignore the performance effects of stress and lack of sleep, 

and are not supported by their colleagues when admitting that they have erred (Davies, 2001). 

Offshore Oil & Gas Industries 

Many of the teams that work in the offshore oil and gas industries such as emergency response 

teams, crane operation teams, service vessel teams, well control teams, and control room teams 

have characteristics that are similar to teams in the aviation industry. Flin and O’Connor (2001) 

state that offshore workforces perform “complex and potentially hazardous operations in a 

constrained, isolated, and remote environment” (p. 218). Similar to other industries using team 

work structures in a similar environment, human error accounts for between 60 to 80 percent of 

failures in offshore systems (Moore & Bea, 1993; Cohea, 1997). Similarly, Mearns, Flin, 

Fleming, and Gordon (1997) reviewed 1268 offshore incidents between 1994 and 1996 and 

discovered that 46% of the human factor causes of these incidents were related to CRM 

principles. Although not titled CRM training, training programs utilizing CRM principles and 

targeting areas such as decision-making, communication, stress, and assertiveness have been in 

place on oilrigs for a number of years (Flin, O’Connor, Mearns, Gordon, & Whitaker, 2000). 

Control room operators, offshore installation managers, and regular offshore rig crews that are 

responsible for a considerable portion of accidents and fatalities in these dangerous operations 

have been trained using these programs which target potential human factors related failure. 

There are CRM training courses for offshore drilling teams that are offered by a number of 

different offshore companies and training organizations. For example, Norwegian offshore oil 

crews have been successfully trained in CRM for several years (Flin et al., 2000). These threeday courses are augmented by a two-day refresher course that is required every two years. Shell 

Expro has used CRM training as a part of their control room operators’ emergency response 

training (Flin, 1995). The Scandinavian oil company Elf Petroleum Norge has a CRM program 

titled “Emergency Resource Management” (Grinde, 1994). Additionally, Flin et al. (2000) 

designed a two-day course for oil and gas platforms that included lectures, group exercises, 

group discussions, questionnaires, and videos. Participants’ reactions to the course have 

generally been positive; most participants believed they would be able to make use of the 

information they learned in the course in their daily work. 

Nuclear Power Plants 

Because of the catastrophic loss of life that can result from poor decisions in the control rooms of 

these high-risk facilities, CRM’s overall goal of utilizing all available personnel, technical, and 

informational resources seems particularly relevant in this setting. Operators in the control room 

of nuclear power plants face stressors, which could lead to potential disasters on a daily basis. 

These stressors may emanate from environmental, organizational, or operational sources as 
 A-10 

varied as time pressure, accuracy of information, interagency liaisons, and equipment 

malfunctions (Paton & Flin, 1999). Fortunately, teams in nuclear power plants have utilized 

CRM training for over a decade (Harrington & Kello, 1991). For example, British Energy has 

designed a CRM course for its nuclear control room operators (Flin et al., 2000). 

CRM in the Rail Industry 

CRM training has also recently been applied within the rail transportation industry to reduce the 

number of accidents attributed to human factors. Because of the many parallel functions to those 

in the aviation industry, it would seem that CRM would be readily accepted within the rail 

industry, however, little action was taken to do so until the recommendations of an NTSB 

accident report regarding a rail accident in Butler, Indiana in 1998. This report resulted in 

recommendation from the NTSB that the Federal Railroad Administration (FRA), the 

Association of American Railroads (AAR), the American Short Line and Regional Railroad 

Association (ASLRRA), Norfolk Southern Railway (NS) and other railroads develop a “Train 

CRM” program. This recommendation was based largely upon the positive benefits that the 

NTSB had seen from CRM programs in other transportation industries. 

In response, the AAR and NS jointly developed a CRM training program that consisted mainly 

of video-based instruction that could be tailored and marketed to other AAR member railroads. 

Independent of this effort, the Southern Pacific Transportation Company (SP) had developed a 

rail CRM program based upon the one in use at US Air. CPR developed a CRM program based 

upon SP’s and its own research. Rail companies in Great Britain and other foreign rail 

companies have also explored the use of CRM as a means to prevent passenger rail disasters. A 

one-day, facilitated introductory railroad CRM training program was recently sponsored by the 

FRA and then developed and pilot tested by the Texas Transportation Institute at the BNSF 

Railway (Morgan et al, 2005). 
 B-1 

Appendix B. 

Commercial Aviation Industry Interview Form 

CREW RESOURCE MANAGEMENT

SUBJECT MATTER EXPERT

SEMI–STRUCTURED INTERVIEW FORM

Airline Version 

First, we would like to thank you for taking the time to meet with us. We anticipate that this 

interview will last approximately 2 hours. 

In 2000, the National Transportation Safety Board (NTSB) recommended that a Crew Resource 

Management (CRM) training program for the railroad industry be developed after several major 

rail accidents related to failures in crew coordination occurred. The NTSB stated that their 

recommendation was a direct result of the positive effects that aviation CRM had had in reducing 

aircraft accidents and incidents in commercial aviation. The railroad industry developed an 

initial CRM course for engineers and conductors; however, in many ways, CRM training in the 

rail industry today stands where CRM did in the early 1990s in the aviation industry—a few 

companies have voluntarily introduced it but a more clear understanding of its impact is needed 

before its use will become widespread. 

The Texas Transportation Institute (TTI), part of the Texas A&M University System, is involved 

in several research activities to aid the Federal Railroad Administration (FRA) in improving rail 

industry acceptance, understanding, and application of CRM training and other human factors–

based training. These activities have included both an assessment of CRM training programs 

currently in place at the major Class I railroads in the U.S. and development of a pilot course to 

expand CRM training to cover operations, dispatching, and maintenance personnel. 

This interview is part of a parallel research effort to make the business case for expanding the 

use of CRM and other human factors–related training within the railroad industry. As part of 

this effort, TTI is interviewing aviation CRM experts and program managers to determine the 

effect of CRM training on safety in the commercial aviation industry. We are not only seeking 

to develop materials that will show that CRM has had a measurable effect in accident prevention, 

but also to show that its implementation has provided positive business benefits. We are also 

aware that the Federal Aviation Administration (FAA) has made CRM mandatory training for 

the aviation industry which is not the case in the railroad industry. Management in individual 

railroad companies are facing the decision of whether or not to fully implement CRM training in 

a highly competitive industry that is facing rising gas prices, increasing demand, and limited 

resources. 

We appreciate your willingness to participate in this process as we seek to understand how CRM 

training developed in the commercial aviation industry. We will be summarizing the results of 

several interviews like this one; consequently, your answers will be confidential and will only be 

presented as part of an aggregate summary in reporting our findings to the FRA. 
 B-2 

Name of SME: ____________________________________________________________ 

Interview Facilitator: _______________________________________________________ 

Interview Method (circle one): Phone Face–to–face 

SME Background Information

Current Job Title 

Years as a CRM 

Facilitator 

Industries in which 

you have worked 

where you have been 

engaged in CRM 

related activities 

Years as a Safety 

and Human Factors 

Specialist 

CRM training is typically targeted at one of more of the following groups (a) Captains, (b) 

First Officers, (c) Second Officers, (d) Flight Attendants, and (e) Aircraft Maintenance/Ground 

Crew. 

In the context of CRM training, which of these groups are you familiar with? (Check all that 

apply.)

 Captains 

 First Officers 

 Second Officers 

 Flight Attendants 

 Aircraft Maintenance/ 

 Ground Crew 
SECTION A 

ORGANIZATION’S CRM BACKGROUND

1. Could you please describe what YOU mean or understand by the term CRM training, 

specifically within the context of [ORGANIZATION]. Does your conceptualization of CRM 

training include specific content areas (situational awareness, resource management, 

assertiveness, teamwork training, etc.)? (Ensure that you are clear as to whether or not their 

use or understanding of the term CRM includes specific content areas such as situational 

awareness, resource management, assertiveness training, teamwork training, etc. The 

answers to this question and the follow up questions will serve as the frame of reference for 

the rest of the questions. We will also content analyze the responses to this question to 

determine whether it is necessary to disaggregate them into clusters. Make sure we are 

differentiating initial from refresher training.)

Follow up: Does [ORGANIZATION] conduct CRM training as a stand–alone training 

program or is it integrated with other training programs/content? 

Follow up: If CRM is integrated with other training, are you able to distinguish CRM 

training from the other training topics or is the integration with the other training topics 

such that it is impossible to specifically distinguish CRM from other training content? (If 

CRM is indistinguishable from the other training content, have SME respond to the 

remaining questions assuming that their CRM training was a stand–alone training 

program or was distinguishable from other training content.) 

 B-3 
 B-4 

2. How long has your company been conducting CRM training? 

3. Was your CRM training implemented before CRM training was mandated by the FAA? If so, 

what was the motivation for implementing CRM training? (We need to recognize that there 

are some airlines that did not exist pre–CRM.) 

4. Who is receiving CRM training? (Answer to this question will determine the focus of 

subsequent questions.) 

5. Is CRM training company–wide, or only with certain groups/divisions? (That is, what % of 

the workforce is actually in the position to be trained?) 
 B-5 

6. In your opinion, how much institutional support (i.e., support from upper management) does 

CRM currently have in your organization? 

{ None at all ______________________________________________ 

{ Very little ______________________________________________ 

{ Some ______________________________________________ 

{ Quite a bit ______________________________________________ 

{ A great deal ______________________________________________ 

7. In your opinion, how much institutional support did CRM have at your organization when it 

was first implemented? 

{ None at all ______________________________________________ 

{ Very little ______________________________________________ 

{ Some ______________________________________________ 

{ Quite a bit ______________________________________________ 

{ A great deal ______________________________________________ 

8. In your opinion, if CRM training is to have an effect in the organization, how important is 

institutional support to its successful implementation? 

{ None at all ______________________________________________ 

{ Very little ______________________________________________ 

{ Some ______________________________________________ 

{ Quite a bit ______________________________________________ 

{ A great deal ______________________________________________ 
 B-6 

SECTION B 

CRM TRAINING, CONTENT, AND TEACHING METHODS

9. When is CRM training conducted? (For example, new hire orientation/within 1st year?) 

10. What content is covered in your CRM training? 

11. How long does the CRM training typically last? 

12. How is the CRM training conducted? (Incorporated into other training events [simulators, 

etc.], on–site facilitated training, computer–aided training, etc.) 
 B-7 

13. What are some of the teaching methods used in your CRM training? (That is, lecture–based, 

video, role–play, etc.)? 

14. What is a typical class size for CRM training? (Does it depend on training group?)

15. What is a reasonable number of trainees that can be included in a typical training class?” (Is 

there variance by airline size, location, unified training [at added travel costs] vs. localized 

training, etc?) 

16. Is the training group segmented by job function, training level, or other characteristics? (Is it 

captains only or mixed crew? Does typical makeup of class allow for interaction with other 

job functions [aircraft maintenance/ground crew, etc.]? If it does, what is the typical mix of 

trainees?) 
 B-8 

17. Who are the CRM training facilitators? (Is there variance by location, position, etc? How did 

the organization decide that these individuals would be facilitators?) 

18. Have there been any initiatives to require CRM training for everyone? If so how did you 

approach it? 

19. How many new employees do you typically hire each year for each of the positions listed 

below? 

Captains: ________ 

First Officers: ________ 

Second Officers: ________ 

Flight Attendants: ________ 

Aircraft Maintenance/ 

 Ground Crew: ________ 

20. How much turnover per year does the organization typically experience in the positions listed 

below? 

Captains: ________ 

First Officers: ________ 

Second Officers: ________ 

Flight Attendants: ________ 

Aircraft Maintenance/ 

 Ground Crew: ________ 
 B-9 

21. If it is not already done, is it possible to make CRM training part of the new hire orientation 

training? CRM may be a part of orientation training for captains, first officers, and second 

officers, however is it possible to make it part of new hire orientation for other positions 

(flight attendants, aircraft maintenance/ground crew)? What are some of the current pressures 

for having/or not having CRM training a part of orientation training? 

22. What is the content of your refresher training? (We may already have this information from 

previous questions.) 

23. What methods do you use for giving refresher training? (We may already have this 

information from previous questions.) 

24. How long does refresher training course last? (This value should be the length of time it takes 

to complete a refresher training course. We may already have this information from previous 

questions.) 
 B-10 

25. In your expert opinion, what are some of the outcomes or benefits of CRM training? 

26. How does your organization justify the cost of CRM training? 
SECTION C 

NUMBER OF EMPLOYEES TRAINED

ΔU = (N × d × SDy) – (N × C)

27. How many individuals (in each job category listed below) are you able to train during a 

single CRM training course? (This should reflect the number of individuals who are able to 

be trained in a single CRM training course.)

Captains: average: ________ min: ________ max: ________ 

First Officers: average: ________ min: ________ max: ________ 

Second Officers: average: ________ min: ________ max: ________ 

Flight Attendants: average: ________ min: ________ max: ________ 

Aircraft Maintenance/ 

 Ground Crew: average: ________ min: ________ max: ________ 

28. Typically, how man

(or MRM) training e

Captains: 

First Officers: 

Second Officers: 

Flight Attendants: 

Aircraft Maintenance/ 

 Ground Crew: 

y individuals (in each job category listed below) currently receive CRM 

ach year? 

________ 

________ 

________ 

________ 

________ 

 B-11 
 B-12 

29. Typically, how long does CRM training at [ORGANIZATION] take from the beginning of 

training until completion (in each job category listed below)? 

Captains: ________ 

First Officers: ________ 

Second Officers: ________ 

Flight Attendants: ________ 

Aircraft Maintenance/ 

 Ground Crew: ________ 

30. Typically, how man

each job category li

Captains: 

First Officers: 

Second Officers: 

Flight Attendants: 

Aircraft Maintenance/ 

 Ground Crew: 

y CRM training courses does [ORGANIZATION] conduct per year (in 

sted below)? 

________ 

________ 

________ 

________ 

________ 
 B-13 

SECTION D 

DURATION OF EFFECT OF CRM TRAINING

ΔU = (N × d × SDy) – (N × C)

31. In your opinion, how long after CRM training will individuals need a refresher course or 

retraining? (Make sure that this is differentiated from their mandated refresher 

interval/cycle. What we want is their opinion/estimate of how long the effects of CRM last 

in the absence of refresher training. That is, assuming CRM training changes behaviors, 

attitudes, and practices, how long do they think it takes before people revert back to their 

“old” ways and practices in the absence of refresher training?) 

32. Has there been an attempt to document or track how long the positive effects of CRM 

training last at [ORGANIZATION] or at any previous organizations that you may have 

worked for or know about? (If necessary, elaborate by drawing/describing a decay curve.)

33. Has [ORGANIZATION] collected any data on the effectiveness of CRM training programs 

or any other type of related training programs (e.g., situational awareness, assertiveness, 

teamwork training)?” If so, then what? 
34. We have been able to identify a few studies that have looked at how long the effects of CRM 

training last. Have you read or heard of any others? 

35. What is the time interval between your initial and refresher CRM training? 

Follow up: Is this period regulatory or based on company policy? 

 B-14 
 B-15 

SECTION E 

EFFECTS OF CRM TRAINING

ΔU = (N × d × SDy) – (N × C)

Please use the following definitions when answering questions in this section. 

U.S. air carriers: U.S. air carriers as defined by Title 14, Part 121 of the Code of Federal 

Regulations (CFR). In general these are major airlines and cargo carriers that fly large transport–

category aircrafts. This includes scheduled aircraft with 10 or more seats. 

Critical Event: Occurrence that directly causes an accident or incident. 

Accident: An occurrence associated with the operation of an aircraft, which takes place between 

the time any person boards the aircraft with the intention of flight and all such persons have 

disembarked, and in which any person suffers death1

, serious injury2, or in which the aircraft 

receives substantial damage3

. 

Incident: An occurrence other than an accident, associated with the operation of an aircraft, 

which affects or could affect the safety of operations. 

Accident = initial event with death, serious injury, or substantial damage. 

Incident = initial Event without death, serious injury, and/or substantial damage 

36. ATTENTION: In this question, we are asking about a certain subset of accidents. For this 

question we are only asking about accidents caused by either: 

(1) A critical event that was initiated by the cockpit crew’s ineffective communication, 

situational awareness, teamwork, and/or assertiveness, or 

(2) An unrelated critical event (mechanical problem, bird strike), yet the injuries, 

fatalities, or damage that resulted from the critical event were exacerbated by the 

cockpit crews ineffective communication, situational awareness, teamwork, and/or 

assertiveness.

 

1

 “Fatal injury” means any injury that results in death within 30 days of the accident. 2

 “Serious injury” means any injury that: (1) requires hospitalization for more than 48 hours, commencing within 7 days from the date the injury 

was received; (2) results in a fracture of any bone (except simple fractures of fingers, toes, or nose); (3) causes severe hemorrhages, nerve, 

muscle, or tendon damage; (4) involves any internal organ; or (5) involves second– or third–degree burns, or any burns affecting more than 5 

percent of the body surface. 

3

 “Substantial damage” means damage or failure that adversely affects the structural strength, performance, or flight characteristics of the aircraft, 

and that would normally require major repair or replacement of the affected component. None of the following is considered “substantial 

damage”: engine failure or damage limited to an engine if only one engine fails or is damaged, bent fairings or cowling, dented skin, small 

punctured holes in the skin or fabric, ground damage to rotor or propeller blades, and damage to landing gear, wheels, tires, flaps, engine 

accessories, brakes, or wingtips.
 B-16 

 Assume that as a whole, U.S. air carriers experience 40 of these types of accidents in a 

five–year period. Considering the impact of CRM training, in your opinion, how many more of 

these types of accidents would occur in this five–year period, if CRM training were not in place? 

Number of Accidents 

average: ________ 

Follow up: In your opinion, what would be an upper and lower bound estimate for the 

number of these types of accidents that CRM training prevents in a five–year period? 

Specifically, what are the highest and lowest reasonable estimates of how many more of 

these types of accidents you would expect if CRM training were not in place during these 

five years? 

Number of Accidents 

upper bound: ________ 

lower bound: ________ 

37. Like any training program, the effectiveness of CRM training at preventing accidents (as 

defined in Questions 36) could potentially range from no effect (i.e., CRM does not work nor 

make a difference) to a very large effect (CRM works very well and makes a big difference). 

Specifically, on the one hand it is possible that CRM training does not help to prevent 

accidents at all, while on the other hand, it could be that CRM training prevents a great 

number of accidents. 

Thus, the effects or contribution of CRM can range from (a) no effect, (b) a small effect, (c) a 

moderate effect, (d) a large effect, to (e) a very large effect. 

You can think of “no effect” as a situation in which a group that was trained in CRM has the 

same number of accidents as one that was not trained. 

A small effect would be one in which the trained group has about 3 less accidents than the 

untrained group. 

A moderate effect would be one in which the trained group has about 8 less accidents than 

the untrained group. 

A large effect would be one in which the trained group has about 12 less accidents than the 

untrained group. 

And a very large effect would be one in which the trained group has about 15 less accidents 

than the untrained group. 
 B-17 

Using the preceding estimates/approximations, in your opinion, how effective do you think CRM 

training is in reducing accidents? 

Effect of CRM Training on Accident 

Reduction

{ No effect 

{ Small effect 

{ Moderate effect 

{ Large effect 

{ Very large effect 

38. In your opinion, what is the total estimated cost4

 (in dollars) of an accident? 

Cost of a Single Accident 

average: ________ 

Follow up: In your opinion, what would be an upper and lower bound estimate for the 

cost of a “typical” accident? 

Cost of a Single Accident 

upper bound: ________ 

lower bound: ________ 

 

4 Includes but is not limited to costs incurred by the airline company as a result of (1) damage to the aircraft (including freight cargo), (2) third 

party property damage, injury, or death, (3) accident clean up, (4) liability payments to injured or killed airline employees, (5) liability payments 

to injured or killed passengers, (6) higher liability insurance premiums (cost of insurance coverage), and (7) any and all costs associated with 

loss of good will and customer trust and confidence. 
SECTION F 

COST OF CRM TRAINING

ΔU = (N × d × SDy) – (N × C)

39. What is the TOTAL cost of a single CRM training course? (Refer to response to Question 27 

to ensure that they are taking the total number trained into account in providing this 

estimate.) 

40. What factors are used to determine this cost (e.g., training materials, travel, lost work–time, 

instructor salary, training facilities, etc.)? 

41. Have any cost–benefit analyses been done regarding CRM at the company? 

Follow up: If it were not required by FAA regulations, what methods and/or arguments do 

you think management would find convincing to maintain or retain your CRM program? 

 B-18 
 B-19 

SECTION G 

SUMMARY AND WRAP UP

42. What are some issues not already mentioned that one faces when attempting to train airline 

employees in CRM principles? 
 C-1 

Appendix C. 

Airline Industry Numerical Calculation Explanations 

Calculation of Number of Persons Trained in CRM (N) in the Airline Industry per Year 

In order to get an assessment of the number of cockpit crewmembers trained in CRM each year, 

the authors used a variety of sources. Using data from BLS (DOL, 2006) and the BTS (DOT, 

2006), they determined about 70,500 pilots, co-pilots, and flight engineers are in the airline 

industry (Part 121; Air Carrier) today. The SME interviews indicated that as suggested by FAA 

(2004), the airline industry keeps CRM skills up to date using recurrent training. SME 

interviews suggest that on average this recurrent training takes place every 243 days. Thus, on 

average 47,000 current pilots, co-pilots, and flight engineers participate in recurrent CRM 

training every year. Furthermore, the airline industry has been hiring about 4,500 new pilots, copilots, and flight engineers a year; all of whom would attend initial CRM training (Aviation 

Information Resources, Inc., 2004). Therefore, on average, 51,500 pilots, co-pilots, and flight 

engineers attend CRM training every year. 

Calculation of the Number of CRM-Related Accidents in the Airline Industry over a 

5-Year Period 

In order to obtain a valid number of CRM-related accidents over the past 5-year period, the 

authors reviewed the National Aviation Safety Data Analysis Center’s (NASDAC) Accident 

Brief Reports for each accident that occurred in 2004 for Part 121 (Air Carrier) operations. They 

obtained these reports through the NTSB’s Accident and Incident Data System (NTSB, 2005). 

By analyzing the accident narratives contained in the preliminary, final, and/or cause reports, the 

authors determined if the accident was either (1) initiated by the cockpit crew’s ineffective 

communication, situational awareness, teamwork, and/or assertiveness or was (2) initiated by an 

unrelated event (mechanical problem, bird strike), yet the injuries, fatalities, or damage that 

resulted were exacerbated by the cockpit crew’s lack of CRM-type behaviors. The authors found 

eight accidents in the NTSB database that fell under these descriptions in the year 2004. They 

multiplied this number by 5 to get an estimate of the number of CRM-related accidents over a 5-

year period. This number is very similar to the number of CRM-related accidents in the airline 

industry found by Von Thaden and Steelman (2005) using the Human Factors Analysis and 

Classification System (HFACS).
 D-1 

Appendix D. 

Railroad Industry Interview Form 

CREW RESOURCE MANAGEMENT

SUBJECT MATTER EXPERT

SEMI–STRUCTURED INTERVIEW FORM

Railroad Version 

First, we would like to thank you for taking the time to meet with us. We anticipate that this 

interview will last approximately 2 hours. 

In 2000, the National Transportation Safety Board (NTSB) recommended that a Crew Resource 

Management (CRM) training program for the railroad industry be developed after several major 

rail accidents related to failures in crew coordination occurred. The NTSB stated that their 

recommendation was a direct result of the positive effects that aviation CRM had had in reducing 

aircraft accidents and incidents in commercial aviation. The railroad industry developed an 

initial CRM course for engineers and conductors; however, in many ways, CRM training in the 

rail industry today stands where CRM did in the early 1990s in the aviation industry—a few 

companies have voluntarily introduced it but a more clear understanding of its impact is needed 

before its use will become widespread. 

The Texas Transportation Institute (TTI), part of the Texas A&M University System, is involved 

in several research activities to aid the Federal Railroad Administration (FRA) in improving rail 

industry acceptance, understanding, and application of CRM training and other human factors–

based training. These activities have included both an assessment of CRM training programs 

currently in place at the major Class I railroads in the U.S. and development of a pilot course to 

expand CRM training to cover operations, dispatching, and maintenance personnel. 

This interview is part of a parallel research effort to make the business case for expanding the 

use of CRM and other human factors–related training within the railroad industry. As part of 

this effort, TTI is interviewing aviation CRM experts and program managers to determine the 

effect of CRM training on safety in the commercial aviation and railroad industries. We are not 

only seeking to develop materials that will show that CRM has had a measurable effect in 

accident prevention, but also to show that its implementation has provided positive business 

benefits. We are also aware that the Federal Aviation Administration (FAA) has made CRM 

mandatory training for the aviation industry which is not the case in the railroad industry. 

Management in individual railroad companies are facing the decision of whether or not to fully 

implement CRM training in a highly competitive industry that is facing rising gas prices, 

increasing demand, and limited resources. 

We appreciate your willingness to participate in this process as we seek to understand how CRM 

training developed in the commercial aviation and railroad industries. We will be summarizing 

the results of several interviews like this one; consequently, your answers will be confidential 

and will only be presented as part of an aggregate summary in reporting our findings to the FRA. 
 D-2 

Name of SME: ____________________________________________________________ 

Interview Facilitator: _______________________________________________________ 

Interview Method (circle one): Phone Face–to–face 

SME Background Information

Current Job Title 

Years as a CRM 

Facilitator 

Industries in which 

you have worked 

where you have been 

engaged in CRM 

related activities 

Years as a Safety 

and Human Factors 

Specialist 

CRM training is typically targeted at one of more of the following groups (a) Engineers, (b) 

Conductors, (c) Maintenance of Way employees (MOW), (d) Engineering employees (e) 

Dispatchers. 

In the context of CRM training, which of these groups are you familiar with? (Check all that 

apply.)

 Engineers 

 Conductors 

 Maintenance of Way employees (MOW) 

 Engineering employees 

 Dispatchers 
 D-3 

SECTION A 

ORGANIZATION’S CRM BACKGROUND

1. Could you please describe what YOU mean or understand by the term CRM training, 

specifically within the context of [ORGANIZATION]. Does your conceptualization of CRM 

training include specific content areas (situational awareness, resource management, 

assertiveness, teamwork training, etc.)? (Ensure that you are clear as to whether or not their 

use or understanding of the term CRM includes specific content areas such as situational 

awareness, resource management, assertiveness training, teamwork training, etc. The 

answers to this question and the follow up questions will serve as the frame of reference for 

the rest of the questions. We will also content analyze the responses to this question to 

determine whether it is necessary to disaggregate them into clusters. Make sure we are 

differentiating initial from refresher training.)

Follow up: Does [ORGANIZATION] conduct CRM training as a stand–alone training 

program or is it integrated with other training programs/content? 

Follow up: If CRM is integrated with other training, are you able to distinguish CRM 

training from the other training topics or is the integration with the other training topics 

such that it is impossible to specifically distinguish CRM from other training content? (If 

CRM is indistinguishable from the other training content, have SME respond to the 

remaining questions assuming that their CRM training was a stand–alone training 

program or was distinguishable from other training content.) 
 D-4 

2. How long has your company been conducting CRM training? 

3. What was your organization’s motivation for implementing CRM training? 

4. Who is receiving CRM training? (Answer to this question will determine the focus of 

subsequent questions.) 

5. Is CRM training company–wide, or only with certain groups/divisions? (That is, what % of 

the workforce is actually in the position to be trained?) 
 D-5 

6. In your opinion, how much institutional support (i.e., support from upper management) does 

CRM currently have in your organization? 

{ None at all ______________________________________________ 

{ Very little ______________________________________________ 

{ Some ______________________________________________ 

{ Quite a bit ______________________________________________ 

{ A great deal ______________________________________________ 

7. In your opinion, how much institutional support did CRM have at your organization when it 

was first implemented? 

{ None at all ______________________________________________ 

{ Very little ______________________________________________ 

{ Some ______________________________________________ 

{ Quite a bit ______________________________________________ 

{ A great deal ______________________________________________ 

8. In your opinion, if CRM training is to have an effect in the organization, how important is 

institutional support to its successful implementation? 

{ None at all ______________________________________________ 

{ Very little ______________________________________________ 

{ Some ______________________________________________ 

{ Quite a bit ______________________________________________ 

{ A great deal ______________________________________________ 
SECTION B 

CRM TRAINING, CONTENT, AND TEACHING METHODS

9. When is CRM training conducted? (For example, new hire orientation/within 1st year?) 

10. What content is covered in your CRM training? 

11. How long does the CRM training typically last? 

12. How is the CRM training conducted? (Incorporated into other training events [simulators, 

etc.], on–site facilitated training, computer–aided training, etc.) 

 D-6 
13. What are some of the teaching methods used in your CRM training? (That is, lecture–

based, video, role–play, etc.)? 

14. What is a typical class size for CRM training? (Does it depend on training group?)

15. What is a reasonable number of trainees that can be included in a typical training class?” (Is 

there variance by railroad size, location, unified training [at added travel costs] vs. localized 

training, etc?) 

16. Is the training group segmented by job function, training level, or other characteristics? (Is it 

Engineers only or mixed crew? Does typical makeup of class allow for interaction with other 

job functions [Transportation/MOW, etc.]? If it does, what is the typical mix of trainees?) 

 D-7 
17. Who are the CRM training facilitators? (Is there variance by location, position, etc? How did 

the organization decide that these individuals would be facilitators?) 

18. Have there been any initiatives to require CRM training for everyone? If so how did you 

approach it? 

19. How many new employees do you typically hire each year for each of the positions listed 

below? 

Engineers: ________ 

Conductors: ________ 

MOW employees: ________ 

Engineering employees: ________ 

Dispatchers: ________ 

20. How much turnover per year does the organization typically experience in the positions listed 

below? 

Engineers: ________ 

Conductors: ________ 

MOW employees: ________ 

Engineering employees: ________ 

Dispatchers: ________ 

 D-8 
 D-9 

21. If it is not already done, is it possible to make CRM training part of the new hire orientation 

training? CRM may be a part of orientation training for engineers and/or conductors, 

however is it possible to make it part of new hire orientation for other positions 

(MOW/Engineering/Dispatch)? What are some of the current pressures for having/or not 

having CRM training a part of orientation training? 

22. What is the content of your refresher training? (We may already have this information from 

previous questions.) 

23. What methods do you use for giving refresher training? (We may already have this 

information from previous questions.) 

24. How long does refresher training course last? (This value should be the length of time it takes 

to complete a refresher training course. We may already have this information from previous 

questions.) 
25. In your expert opinion, what are some of the outcomes or benefits of CRM training? 

26. How does your organization justify the cost of CRM training? 

 D-10 
 D-11 

SECTION C 

NUMBER OF EMPLOYEES TRAINED

ΔU = (N × d × SDy) – (N × C)

27. How many individuals (in each job category listed below) are you able to train during a 

single CRM training course? (This should reflect the number of individuals who are able to 

be trained in a single CRM training course.)

Engineers: average: ________ min: ________ max: ________ 

Conductors: average: ________ min: ________ max: ________ 

MOW employees: average: ________ min: ________ max: ________ 

Engineering employees: average: ________ min: ________ max: ________ 

Dispatchers: average: ________ min: ________ max: ________ 

28. Typically, how many 

(or MRM) training ea

Engineers: 

Conductors: 

MOW employees: 

Engineering employees: 

Dispatchers: 

individuals (in each job category listed below) currently receive CRM 

ch year? 

________ 

________ 

________ 

________ 

________ 
 D-12 

29. Typically, how long does CRM training at [ORGANIZATION] take from the beginning of 

training until completion (in each job category listed below)? 

Engineers: ________ 

Conductors: ________ 

MOW employees: ________ 

Engineering employees: ________ 

Dispatchers: ________ 

30. Typically, how many CRM training courses does [ORGANIZATION] conduct per year (in 

each job category listed below)? 

Engineers: ________ 

Conductors: ________ 

MOW employees: ________ 

Engineering employees: ________ 

Dispatchers: ________ 
 D-13 

SECTION D 

DURATION OF EFFECT OF CRM TRAINING

ΔU = (N × d × SDy) – (N × C)

31. In your opinion, how long after CRM training will individuals need a refresher course or 

retraining? (Make sure that this is differentiated from their mandated refresher 

interval/cycle. What we want is their opinion/estimate of how long the effects of CRM last 

in the absence of refresher training. That is, assuming CRM training changes behaviors, 

attitudes, and practices, how long do they think it takes before people revert back to their 

“old” ways and practices in the absence of refresher training?) 

32. Has there been an attempt to document or track how long the positive effects of CRM 

training last at [ORGANIZATION] or at any previous organizations that you may have 

worked for or know about? (If necessary, elaborate by drawing/describing a decay curve.)

33. Has [ORGANIZATION] collected any data on the effectiveness of CRM training programs 

or any other type of related training programs (e.g., situational awareness, assertiveness, 

teamwork training)?” If so, then what? 
34. We have been able to identify a few studies that have looked at how long the effects of CRM 

training last. Have you read or heard of any others? 

35. What is the time interval between your initial and refresher CRM training? 

Follow up: Is this based on company policy? 

 D-14 
 D-15 

SECTION E 

EFFECTS OF CRM TRAINING

ΔU = (N × d × SDy) – (N × C)

Please use the following definitions when answering questions in this section. 

Accident: Any collision, derailment, fire, explosion or other event involving the operation of ontrack equipment (standing or moving) that results in total damages to all railroads involved in the

event that is greater than $6,700 total damage to railroad on-track equipment, signals, track, track

structures and roadbed. 

Incident: An event involving the movement of on-track equipment that results in a reportable 

casualty but does not cause reportable damage above the threshold of $6,700. 

Primary Cause: 

Human Error by a Train/Switch/Yard Crew: An error defined by the FRA’s accident 

database under train operation/human factors (train/switch/yard crew). These include but are not

limited to... 

Human Error Examples

• Failure to properly secure/release hand brake on car(s) Use of brakes • Failure to apply sufficient number of hand brakes on car(s)

Employee physical • Impairment of efficiency or judgment because of drugs or alcohol 

condition • Employee asleep

Flagging, fixed, hand • Failure to give/receive/comply with hand signals 

and radio signals • Improper/Failure to comply with radio communications

• Failure to control shoving movement (man on or at leading end of 

movement) 

General switch rules • Absence of man on or at leading end of shoving movement 

• Cars left foul 

• Failure to couple

• Failure to stop train in clear 

Main track authority • Failure to comply with train order, track warrant, track bulletin, or 

timetable authority

Train handling/Train • Excessive buffing or slack action caused by train handling 

make-up • Improper use of independent brake

• Excessive coupling speed 

Speed • Failure to comply with restricted speed or its equivalent not in connection 

with a block or interlocking signal
 D-16 

• Switch improperly lined Use of switches • Switch not latched or locked

Cab signals • Failure to comply with automatic cab signals

36. Assume that with NO CRM training, in one year, U.S. railroads as a whole experience 1,000 

accidents/incidents in which human error by a train crew/switch crew/yard crew was 

determined to be the primary cause. Considering the impact of CRM training, in your 

opinion, how many of these accidents would have been prevented if CRM training were in 

place industry wide? 

Number of Accidents 

average: ________ 

Follow up: In your opinion, what would be an upper and lower bound estimate for the 

number of these types of accidents that CRM training would have prevented if CRM was 

in place industry wide? Specifically, what are the highest and lowest reasonable 

estimates of how many of these accidents you would expect to be prevented if CRM 

training were in place during this one year? 

Number of Accidents 

upper bound: ________ 

lower bound: ________ 
 D-17 

37. Like any training program, the effectiveness of CRM training at preventing accidents (as 

defined in Questions 36) could potentially range from no effect (i.e., CRM does not work nor 

make a difference) to a very large effect (CRM works very well and makes a big difference). 

Specifically, on the one hand it is possible that CRM training does not help to prevent 

accidents at all, while on the other hand, it could be that CRM training prevents a great 

number of accidents. 

Thus, the effects or contribution of CRM can range from (a) no effect, (b) a small effect, (c) a 

moderate effect, (d) a large effect, to (e) a very large effect. 

You can think of “no effect” as a situation in which a group that was trained in CRM has the 

same number of accidents as one that was not trained. 

A small effect would be one in which the trained group has about 80 less accidents than the 

untrained group. 

A moderate effect would be one in which the trained group has about 200 less accidents 

than the untrained group. 

A large effect would be one in which the trained group has about 290 less accidents than the 

untrained group. 

And a very large effect would be one in which the trained group has about 360 less

accidents than the untrained group. 

Using the preceding estimates/approximations, in your opinion, how effective do you think CRM 

training is in reducing accidents? 

Effect of CRM Training on Accident 

Reduction

{ No effect 

{ Small effect 

{ Moderate effect 

{ Large effect 

{ Very large effect 
 D-18 

38. In your opinion, what is the total estimated cost (in dollars) of one of these types of 

accidents? 

Cost of a Single Accident 

average: ________ 

Follow up: In your opinion, what would be an upper and lower bound estimate for the 

cost of a “typical” accident of this type? 

Cost of a Single Accident 

upper bound: ________ 

lower bound: ________ 
SECTION F 

COST OF CRM TRAINING

ΔU = (N × d × SDy) – (N × C)

39. What is the TOTAL cost of a single CRM training course? (Refer to response to Question 27 

to ensure that they are taking the total number trained into account in providing this 

estimate.) 

40. What factors are used to determine this cost (e.g., training materials, travel, lost work–time, 

instructor salary, training facilities, etc.)? 

41. Have any cost–benefit analyses been done regarding CRM at the company? 

Follow up: What methods and/or arguments do you think management would find 

convincing to maintain or retain your CRM program? 

 D-19 
SECTION G 

SUMMARY AND WRAP UP

42. What are some issues not already mentioned that one faces when attempting to train railroad 

employees in CRM principles? 

 D-20 
Appendix E. 

Railroad Industry Numerical Calculation Explanations 

Calculation of the Number of Engineers and Conductors in the U.S. Freight Railroad 

Industry 

In order to get an assessment of the number of engineers and conductors (train, yard, switch) in 

the U.S. freight rail industry, the authors used data from BLS Occupational Outlook Handbook 

(DOL, 2006b). The handbook states that 40,000 engineers and operators and 38,000 conductors 

and yardmasters were in the rail industry in 2004. Because yardmasters are not part of the 

current analysis, however, the authors subtracted 1,900 (5 percent) yardmasters from 38,000 

(conductors and yardmasters) to arrive at 36,100 conductors. Thus, in total 76,100 engineers and 

conductors were in the railroad industry. This does, however, include engineers and conductors 

in passenger rail service (i.e., Amtrak), which is not part of the current analysis. The Surface 

Transportation Board’s 2004 wage statistics state that Amtrak employs 3,538 train and engine 

employees (DOT, 2004b). Subtracting these employees from 76,000 results in about 72,500 

engineers and conductors (road, switch, and yards) in the U.S. freight railroad industry in 2004. 

Average Number and Costs of Accidents in the U.S. Freight Railroad Industry, Caused by 

a Human Factors Failure of the Engineer and/or Conductor (Train, Yard, Switch) 

In order to obtain an average number of reportable accidents in the U.S. freight railroad industry, 

caused by a human factors failure of the engineer and/or conductor (train, yard, switch), the 

authors used data from FRA’s accident database (FRA, 2004). The authors reviewed the specific 

human factors causes of the 1,310 human factors (primary cause) accidents that occurred in 2004 

and determined that 10 percent of these accidents could be attributed to railroad employees other 

than engineers and/or conductors (i.e., trainmasters). Next, the authors reviewed the human 

factors accidents from 2000-2003 and determined that about 10 percent of these accidents could 

also be attributed to non-train crew employees. The number of accidents per year was obtained 

by computing the average of the number of train crew-caused human error accidents over this 5-

year period. Furthermore, the cost of these types of accidents per year was obtained by 

computing the average of the cost of these accidents in 2000-2004 reported in the database 

(FRA, 2004) and then dividing this yearly cost average by the average yearly number of 

accidents found above. 

Number and Costs of Accidents in a Typical Large Class I Railroad, Caused by a Human 

Factors Failure of the Engineer and/or Conductor (Train, Yard, Switch) 

In order to obtain an average number of reportable accidents in a large Class I railroad caused by 

a human factors failure of the engineer and conductor (train, yard, switch), the authors used data 

from FRA’s accident database (FRA, 2005). The authors calculated the average number of train 

crew-caused (see footnote 4) human factors accidents in the four large Class I railroads in the 

year 2004. Similar to the rail industry as a whole, 90 percent of the total human factors accidents 

in this sample could be attributed to the train crews. Similar to 2004, the authors then adjusted 

the number of human factors accidents for the years 2000-2003 to reflect this 90 percent. They 

 E-1 
 E-2 

then divided the total number of these accidents associated with the railroads by 4 to get an 

average number of train crew-related human factor accidents in a typical large Class I railroad 

for each of the 5 years. The authors then averaged these 5 years to arrive at the average number 

of train crew human factors-caused accidents that occur at a typical large Class I railroad in 1 

year. Furthermore, the cost of these types of accidents per year was obtained by computing the 

average of the cost of these accidents in 2000-2004 reported in the database (FRA, 2004) and 

then dividing this yearly cost average by the average yearly number of accidents found above.
 F-1 

Appendix F. 

Sensitivity Analysis of Commercial Aviation Industry Utility 

Airline Industry Utility Analysis Sensitivity Analysis Results 

Tables F-1, F-2, and F-3 present sensitivity analyses of results of the airline utility analysis. The 

approach that the authors took was to generate three general sets of parameter estimates to 

represent what can be considered to be conservative (Table F-1), average (Table F-2), and liberal 

(Table F-3) utility estimates. Specifically, the conservative estimates represented a situation in 

which (a) the number of human error accidents was low, (b) the average cost of accidents was 

low, (c) the duration of training effect was short, (d) consequently, the cost of accidents in the 

training effect duration per trainee was low, and (e) the cost of training per trainee was high. 

Conversely, the liberal estimates represented a situation in which (a) the number of human error 

accidents was high, (b) the average cost of accidents was high, (c) the duration of training effect 

was long, (d) consequently, the cost of accidents in the training effect duration per trainee was 

high, and (e) the cost of training per trainee was low. The average scenario used estimates that 

were between the conservative and liberal extremes. 

Next, for each of the scenarios (i.e., conservative, average, and liberal), the authors used the 

various previously described effect size estimates to compute the utility that would be accrued to 

the airline industry given each specified level of CRM training effectiveness. The results 

presented in Tables F-1, F-2, and F-3 clearly indicate that even under the most conservative 

assumptions, with the weakest effect for CRM effectiveness, the gain to the airline industry from 

implementing CRM training is positive. Thus, in terms of a utility analysis framework, based on 

the data collected from the aviation SMEs and presented here, the benefits of CRM training to 

the airline industry conclusively outweigh its costs. 
 F-2 

Table F-1. Utility Analysis Results for Airline Industry 

Based on Conservative Parameter Estimates 

Based on 17 human error accidents/yearA with an average cost of $78m/accidentB

. 

Number of persons trained (N) = 51,500C

Duration of training effect (days; T) = 135D

Per trainee cost of accidents in training effect duration (SDy) = $9,462.03 

Per trainee cost of training (C) = $1,883.00E

Effect Size Magnitude and Source d Utility Utility (M$)F

Small effect 

SME-based estimate (minimum) 0.80 $292,861,116.44 $293m

SME-based, computed 1.40 $585,237,828.77 $585m

Literature-based effect size (small) 0.20 $484,404.11 $0.5m

Medium effect 

SME-based estimate (mean) 1.04 $409,811,801.37 $410m

SME-based, computed 1.41 $590,110,773.97 $590m

Literature-based effect size 0.50 $146,672,760.27 $147m (medium) 

Large effect 

SME-based estimate (maximum) 1.10 $439,049,472.60 $439m

SME-based, computed 2.23 $989,692,280.82 $990m

Literature-based effect size (large) 0.80 $292,861,116.44 $293m AMean of Question 36, Lower (Appendix B). B

Mean of Question 38, Lower. CFactually obtained. DMinimum of 

Question 31. E

Maximum of Question 39. FRounded to the nearest million. 
 F-3 

Table F-2. Utility Analysis Results for Airline Industry 

Based on Average Parameter Estimates 

Based on 21 human error accidents/yearA with an average cost of $129m/accidentB. 

Number of persons trained (N) = 51,500C

Duration of training effect (days; T) = 384D

Per trainee cost of accidents in training effect duration (SDy) = $55,268.63 

Per trainee cost of training (C) = $794.00E

Effect Size Magnitude and Source d Utility Utility (M$)F

Small effect 

SME-based estimate (minimum) 0.80 $2,236,176,391.37 $2,236m

SME-based, computed 1.40 $3,943,976,934.90 $3,944m

Literature-based effect size (small) 0.20 $528,375,847.84 $528m

Medium effect 

SME-based estimate (mean) 1.04 $2,919,296,608.78 $2,919m

SME-based, computed 1.41 $3,972,440,277.29 $3,972m

Literature-based effect size 

(medium) 0.50 $1,382,276,119.61 $1,382m

Large effect 

SME-based estimate (maximum) 1.10 $3,090,076,663.13 $3,090m

SME-based, computed 2.23 $6,306,434,353.44 $6,306m

Literature-based effect size (large) 0.80 $2,236,176,391.37 $2,236m AMean of Question 36 (Appendix B). B

Mean of Question 38. E

Average of Question 39. FRounded to the nearest million. 

CFactually obtained. DAverage of Question 31. 
 F-4 

Table F-3. Utility Analysis Results for Airline Industry 

Based on Liberal Parameter Estimates 

Based on 23 human error accidents/yearA with an average cost of $1,250m/accidentB. 

Number of persons trained (N) = 51,500C

Duration of training effect (days; T) = 1,095D

Per trainee cost of accidents in training effect duration (SDy) = $1,674,757.28 

Per trainee cost of training (C) = $250.00E

Effect Size Magnitude and Source d Utility Utility (M$)F

Small effect 

SME-based estimate (minimum) 0.80 $68,987,125,000.00 $68,987m

SME-based, computed 1.40 $120,737,125,000.00 $120,737m

Literature-based effect size (small) 0.20 $17,237,125,000.00 $17,237m

Medium effect 

SME-based estimate (mean) 1.04 $89,687,125,000.00 $89,699m

SME-based, computed 1.41 $121,599,625,000.00 $121,612m

Literature-based effect size 0.50 $43,112,125,000.00 $43,124m (medium) 

Large effect 

SME-based estimate (maximum) 1.10 $94,862,125,000.00 $94,862m

SME-based, computed 2.23 $192,324,625,000.00 $192,324m

Literature-based effect size (large) 0.80 $68,987,125,000.00 $68,987m AMean of Question 36, Upper (Appendix B). B

Mean of Question 38, Upper. CFactually obtained. DMaximum of 

Question 31. E

Minimum of Question 39. FRounded to the nearest million 
 G-1 

Appendix G. 

Sensitivity Analysis of Railroad Industry Utility 

Railroad Industry Utility Analysis Sensitivity Analysis Results 

Tables G-1, G-2, and G-3 present sensitivity analyses of the railroad utility analysis findings. 

Again, the authors generated three general sets of parameter estimates to represent conservative 

(Table G-1), average (Table G-2), and liberal (Table G-3) utility estimates. Specifically, the 

conservative estimates represented a situation in which (a) the number of human error accidents 

was low, (b) the average cost of accidents was low, (c) the duration of training effect was short, 

(d) consequently, the cost of accidents in the training effect duration per trainee was low, and (e) 

the cost of training per trainee was high. 

Conversely, the liberal estimates represented a situation in which (a) the number of human error 

accidents was high, (b) the average cost of accidents was high, (c) the duration of training effect 

was long, (d) consequently, the cost of accidents in the training effect duration per trainee was 

high, and (e) the cost of training per trainee was low. The average scenario used estimates that 

were between the conservative and liberal extremes. 

Next, for each of the scenarios (i.e., conservative, average, and liberal), the authors used the 

various previously described effect size estimates to compute the utility that would be accrued to 

the railroad industry given each specified level of CRM effectiveness. The results presented in 

Tables G-1, G-2, and G-3 clearly indicate that with the exception of only one, under all 

conditions the gain to the railroad industry from implementing CRM training is positive. The 

only scenario in which there were no accident-related cost savings was with the weakest effect 

size under the conservative scenario (Table G-1). Thus, in terms of a utility analysis framework, 

based on the data presented here, although the gains are not as large as those obtained for the 

airline industry, the results of the utility analysis indicate that the benefits of CRM training to the 

railroad industry generally outweigh its costs. 
 G-2 

Table G-1. Utility Analysis Results for Railroad Industry 

Based on Conservative Parameter Estimates 

A Based on 932 human error accidents/year with an average cost of $56,722/accidentB

. 

Number of persons trained (N) = 72,500C

Duration of training effect (days; T) = 180D

Per trainee cost of accidents in training effect duration (SDy) = $359.59 

Per trainee cost of training (C) = $175.00E 

Effect Size Magnitude and Source d Utility Utility (M$)F

Literature-based effect size (small) 0.20 $-7,473,427.28 $-7m

Literature-based effect size 0.50 $347,681.81 $0.3m (medium) 

SME-based estimate 0.50 $347,681.81 $0.3m

Literature-based effect size (large) 0.80 $8,168,790.89 $8m AFactually obtained, minimum value. B

Factually obtained = mean cost of accidents ÷ maximum number of 

accidents. CFactually obtained, training 100% of railroad work force in 1 year. DQuestion 31 (Appendix D). E

Maximum of Question 39. FRounded to the nearest million. 

Table G-2. Utility Analysis Results for Railroad Industry 

Based on Average Parameter Estimates 

Based on 1,032 human error accidents/yearA with an average cost of $63,512/accidentB. 

Number of persons trained (N) = 36,250C

Duration of training effect (days; T) = 270D

Per trainee cost of accidents in training effect duration (SDy) = $1,337.51 

Per trainee cost of training (C) = $167.00E

Effect Size Magnitude and Source d Utility Utility (M$)F

Literature-based effect size (small) 0.20 $3,643,227.36 $4m

Literature-based effect size 0.50 $18,188,693.40 $18m (medium) 

SME-based estimate 0.50 $18,188,693.40 $18m

Literature-based effect size (large) 0.80 $32,734,159.44 $33m AFactually obtained, mean value. B

Factually obtained = mean cost of accidents ÷ mean number of accidents. 

CFactually obtained, training 50% of railroad workforce in 1 year. DQuestion 31 (Appendix D). E

Mean of Question 

39. F

Rounded to the nearest million. 
 G-3 

Table G-3. Utility Analysis Results for Railroad Industry 

Based on Liberal Parameter Estimates 

Based on 1,156 human error accidents/yearA with an average cost of $70,354/accidentB. 

Number of persons trained (N) = 23,925C

Duration of training effect (days; T) = 365D

Per trainee cost of accidents in training effect duration (SDy) = $3,399.34 

Per trainee cost of training (C) = $150.00E

Effect Size Magnitude and Source d Utility Utility (M$)F

Literature-based effect size (small) 0.20 $12,677,094.80 $13m

Literature-based effect size 

(medium) 0.50 $37,075,862.00 $37m

SME-based estimate 0.50 $37,075,862.00 $37m

Literature-based effect size (large) 0.80 $61,474,629.20 $61m AFactually obtained, maximum value. B

Factually obtained = mean cost of accidents ÷ minimum number of 

accidents. CFactually obtained, training 33% of the railroad workforce in 1 year. DQuestion 31 (Appendix D). E

Minimum of Question 39. FRounded to the nearest million. 
G-4 
 H-1 

Appendix H. 

Sensitivity Analysis of Typical Class I Utility 

Typical Large Class I Railroad Utility Analysis Sensitivity Analysis Results 

Because the preceding railroad results were at the industry level, it might be informative to also 

present similar analyses at the organization level. Consequently, the authors factually obtained 

data for the number of accidents from FRA (2005) and the number of engineers and conductors 

from the Surface Transportation Board (DOT, 2006) for four representative large Class I 

railroads, namely BNSF, CSX, NS, and UP. Appendix E contains a more detailed description on 

the calculation of the number of accidents. The average number of accidents and the number of 

engineers and conductors across these four railroads were then used to compute the utility gain in 

accident savings to a typical large Class I from implementing CRM training. These analyses 

modeled the same general boundary conditions and approach used for the industry-level 

analyses. The results of the typical large Class I analyses, which are presented in Tables H-1, H2, and H-3, show that, as would be expected, when disaggregated to the organizational level the 

absolute dollar gain is smaller than the industry-level values. In addition, for the small and 

medium effect sizes under the conservative scenario, no utility gains exist. The gains under all 

the other scenario boundary conditions, however, are positive. Thus, as with the industry level 

results, it is reasonable to conclude that the results of the utility analyses suggest that the typical 

large Class I railroad will accrue accident-related cost savings from the implementation of CRM 

training. 

Table H-1. Utility Analysis Results for a Large Typical Class I Railroad 

Based on Conservative Parameter Estimates 

Based on 167 human error accidents/yearA with an average cost of $55,558/accidentB. 

Number of persons trained (N) = 16,342C

Duration of training effect (days; T) = 180D

Per trainee cost of accidents in training effect duration (SDy) = $279.99 

Per trainee cost of training (C) = $175.00E

Effect Size Magnitude and Source d Utility Utility (M$)F

Literature-based effect size (small) 0.20 $-1,994,741.24 $-2m

Literature-based effect size 0.50 $-572,078.11 $-0.6m (medium) 

SME-based estimate 0.50 $-572,078.11 $-0.6m

Literature-based effect size (large) 0.80 $800,585.02 $0.8m AFactually obtained, minimum value. B

Factually obtained = mean cost of accidents ÷ maximum number of 

accidents. CFactually obtained, training 100% of the railroad workforce in 1 year. DQuestion 31 (Appendix D). E

Maximum of Question 39. FRounded to the nearest million. 
 H-2 

Table H-2. Utility Analysis Results for a Typical Large Class I Railroad 

Based on Average Parameter Estimates 

A Based on 189 human error accidents/year with an average cost of $63,947/accidentB

. 

Number of persons trained (N) = 8,171C

Duration of training effect (days; T) = 270D

Per trainee cost of accidents in training effect duration (SDy) = $1,094.15 

Per trainee cost of training (C) = $167.00E

Effect Size Magnitude and Source d Utility Utility (M$)F

Literature-based effect size (small) 0.20 $423,506.24 $0.4m

Literature-based effect size 0.50 $3,105,601.10 $3m (medium) 

SME-based estimate 0.50 $3,105,601.10 $3m

Literature-based effect size (large) 0.80 $5,787,695.95 $6m AFactually obtained, mean value. B

Factually obtained = mean cost of accidents ÷ mean number of accidents. 

CFactually obtained, training 50% of railroad workforce in 1 year. DQuestion 31 (Appendix D). E

Mean of Question 

39. F

Rounded to the nearest million. 

Table H-3. Utility Analysis Results for a Typical Large Class I Railroad 

Based on Liberal Parameter Estimates 

A Based on 218 human error accidents/year with an average cost of $72,525/accidentB

. 

Number of persons trained (N) = 5,393C

Duration of training effect (days; T) = 365D

Per trainee cost of accidents in training effect duration (SDy) = $2,931.66 

Per trainee cost of training (C) = $150.00E

Effect Size Magnitude and Source d Utility Utility (M$)F

Literature-based effect size (small) 0.20 $2,353,140.00 $2m

Literature-based effect size 0.50 $7,096,275.00 $7m (medium) 

SME-based estimate 0.50 $7,096,275.00 $7m

Literature-based effect size (large) 0.80 $11,839,410.00 $11m AFactually obtained, maximum value. B

Factually obtained = mean cost of accidents ÷ minimum number of 

accidents. CFactually obtained, training 33% of the railroad workforce in 1 year. DQuestion 31 (Appendix D). E

Minimum of Question 39. FRounded to the nearest million.