May 2013

NASA/TM–2013-218000

Identification of Crew-Systems Interactions and 

Decision Related Trends

Sharon Monica Jones

Langley Research Center, Hampton, Virginia

Joni K. Evans

Analytical Mechanics Associates, Inc., Hampton, Virginia

Mary S. Reveley and Colleen A. Withrow

Glenn Research Center, Cleveland, Ohio

Ersin Ancel

National Institute of Aerospace, Hampton, Virginia

Lawrence Barr

Volpe National Transportation Systems Center, Cambridge, Massachusetts
NASA STI Program . . . in Profile

Since its founding, NASA has been dedicated to the 

advancement of aeronautics and space science. The 

NASA scientific and technical information (STI) 

program plays a key part in helping NASA maintain 

this important role.

The NASA STI program operates under the 

auspices of the Agency Chief Information Officer. 

It collects, organizes, provides for archiving, and 

disseminates NASA’s STI. The NASA STI 

program provides access to the NASA Aeronautics 

and Space Database and its public interface, the 

NASA Technical Report Server, thus providing one 

of the largest collections of aeronautical and space 

science STI in the world. Results are published in 

both non-NASA channels and by NASA in the 

NASA STI Report Series, which includes the 

following report types:

 TECHNICAL PUBLICATION. Reports of 

completed research or a major significant phase 

of research that present the results of NASA 

Programs and include extensive data or 

theoretical analysis. Includes compilations of 

significant scientific and technical data and 

information deemed to be of continuing 

reference value. NASA counterpart of peerreviewed formal professional papers, but 

having less stringent limitations on manuscript 

length and extent of graphic presentations.

 TECHNICAL MEMORANDUM. Scientific 

and technical findings that are preliminary or of 

specialized interest, e.g., quick release reports, 

working papers, and bibliographies that contain 

minimal annotation. Does not contain extensive 

analysis.

 CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees.

 CONFERENCE PUBLICATION. 

Collected papers from scientific and 

technical conferences, symposia, seminars, 

or other meetings sponsored or cosponsored by NASA.

 SPECIAL PUBLICATION. Scientific, 

technical, or historical information from 

NASA programs, projects, and missions, 

often concerned with subjects having 

substantial public interest.

 TECHNICAL TRANSLATION. 

English-language translations of foreign 

scientific and technical material pertinent to 

NASA’s mission.

Specialized services also include organizing 

and publishing research results, distributing 

specialized research announcements and feeds, 

providing information desk and personal search 

support, and enabling data exchange services.

For more information about the NASA STI 

program, see the following:

 Access the NASA STI program home page 

at http://www.sti.nasa.gov

 E-mail your question to help@sti.nasa.gov

 Fax your question to the NASA STI 

Information Desk at 443-757-5803

 Phone the NASA STI Information Desk at 

443-757-5802

 Write to:

 STI Information Desk

 NASA Center for AeroSpace Information

 7115 Standard Drive

 Hanover, MD 21076-1320
National Aeronautics and 

Space Administration

Langley Research Center 

Hampton, Virginia 23681-2199

May 2013

NASA/TM–2013-218000

Identification of Crew-Systems Interactions and 

Decision Related Trends

Sharon Monica Jones

Langley Research Center, Hampton, Virginia

Joni K. Evans

Analytical Mechanics Associates, Inc., Hampton, Virginia

Mary S. Reveley and Colleen A. Withrow

Glenn Research Center, Cleveland, Ohio

Ersin Ancel

National Institute of Aerospace, Hampton, Virginia

Lawrence Barr

Volpe National Transportation Systems Center, Cambridge, Massachusetts
Available from:

NASA Center for AeroSpace Information

7115 Standard Drive

Hanover, MD 21076-1320

443-757-5802
Crew Decision Making Trends and Issues

Page 1

1. INTRODUCTION

1.1 Purpose of Study

The goal of the Vehicle System Safety Technology (VSST) project within NASA’s Aviation 

Safety Program (AvSP) is to “enable a reduction in accidents and incidents through enhanced 

vehicle design, structure, systems, and operating concepts” (ref. 1). The VSST project is 

composed of three key Technical Challenges (TCs): 

(1) Improve Crew Decision-Making and Response in Complex Situations (CDM)

(2) Maintain Vehicle Safety between Major Inspections (MVS)

(3) Assure Safe and Effective Aircraft Control under Hazardous Conditions (ASC)

VSST project management uses systems analysis “to identify key issues and maintain a 

portfolio of research leading to potential solutions to these challenges”. To assist management 

in accomplishing these objectives, several systems analysis related milestones have been 

specified in the VSST project plan. The systems analysis milestone for fiscal year (FY) 2013 is 

focused on the CDM Technical Challenge. The goal of CDM is to “develop and demonstrate 

new capabilities that enable pilots to better understand and respond to complex situations”. The 

expected research outcome of the CDM TC is the set of five research products listed in Table 1. 
Crew Decision Making Trends and Issues

Page 2

Table 1. CDM List of Research Products

IMPROVE CREW DECISION-MAKING AND RESPONSE IN COMPLEX SITUATIONS (CDM)

CDM-1

Advanced Displays for 

Terminal Area and 

Surface Operations

Advanced displays and decision support concepts that improve 

situation awareness and provide hazard protection during 

terminal area and runway /surface current-day and NextGen 

operations 

CDM-2

Flight Deck 

Countermeasures for 

Spatial Disorientation 

and Loss-of-Energy State 

Awareness

Displays, decision support tools, and Crew Resource 

Management (CRM) concepts to help crews avoid, detect, 

mitigate, and recover from the loss-of-aircraft attitude and 

energy state awareness 

CDM-3

Crew System Monitoring 

And Management System 

(CSMMS)

These technologies offer the capability of identifying and 

quantifying the current physiological, workload, and attentional 

state of the crew and these data may be used to ensure and 

adapt to the crew’s and/or automation system’s readiness to 

respond in potentially hazardous situations.

CDM-4

Flight Deck Information 

Management Systems for 

Integrity and Awareness

The primary objective of this technology is the creation of 

integrated flight deck information management systems that 

assure the information needed by flight crews to make critical 

decisions is complete and not misleading.

CDM-5

Revised Pilot Proficiency 

Standards for Manual 

Handling and Automation 

Interactions

This technology will result in the creation of data and guidelines 

to support revised pilot proficiency standards and training 

associated with manual handling, automation interactions, and 

reverting from automated to manual handling.

As detailed in the VSST Project Plan, the CDM TC is described as follows:

“CDM research develops flight deck capabilities that enable pilots to make more 

informed decisions when confronted with complex situations. From a review of 

recent accidents, pilots are increasingly faced with complex, multi-faceted 

situations. Prior emphasis on responding to an individual failure with a targeted 

checklist may not always apply. Given the complexities of current-day operations 

and future trends toward greater levels of automation and information, 

technologies must be developed that help pilots assess these situations and 

execute an informed course of action. CDM research focuses on low altitude 

terminal area operations since this phase of flight is at greatest risk due to its 

operational complexity and time criticality. “ 

A summary of the goals, objectives and products for the CDM TC are graphically depicted using 

an objectives tree format (ref. 2) in Figure 1. 
Crew Decision Making Trends and Issues

Page 3

Figure 1. CDM Objectives Tree

The specific CDM related FY13 systems analysis milestone is stated below:

 

(1) “Deliver analysis of trends in aviation as related to crew systems interactions & decisions 

by reviewing the most current statistical and prognostic data available about accidents 

and incidents related to crew systems research areas

(2) Deliver assessment of future directions in aviation technology related to crew systems 

research areas through review of literature from academia, industry and other 

government agencies to establish requirements for future work in crew systems”

1.2 Overview of Study Contents

The expected outcomes for this study are addressed in sequential order. Outcome 1 is 

addressed in section 2, which contains a summary of statistical analyses of accident and 

incident data that have been conducted by NASA researchers. Outcome 2 is the focus of 

section 3, which is a summary of crew systems issues and future research needs that were 

derived from literature reviews, databases and aviation subject-matter experts. Finally, 

discussion and the conclusions that have been drawn are provided in section 4. 
Crew Decision Making Trends and Issues

Page 4

2. SUMMARY OF NASA STATISTICAL STUDIES

Systems analysis personnel within the NASA Aviation Safety Program have recently conducted 

several statistical analyses of accident and incident data (ref. 3 and 4). This section summarizes

the results of these analyses that are related to the CDM Technical Challenge. 

2.1 Analysis of NTSB/FAA Data

In 2011, a study (ref. 3) was conducted to identify trends in aviation accidents and incidents that 

were related to the three VSST Technical Challenges (TCs) as they were defined at that time:

1. Vehicle Health Assurance

2. Effective Crew-System Interactions and Decisions in All Conditions

3. Aircraft Loss-of-Control Prevention and Mitigation

For the “Vehicle Health Assurance” and “Aircraft Loss-of-Control Prevention and Mitigation”

TCs, historical aviation accidents were examined, using the National Transportation Safety 

Board (NTSB) Aviation Accident and Incident Data System (restricted to 1989-2008), and 

incidents, using the Federal Aviation Administration (FAA) Accident/Incident Data System 

(restricted to 1989-2006). The rationale was that these two TCs mapped easily to specific 

Commercial Aviation Safety Team/International Civil Aviation Organization (CAST/ICAO) (ref. 5)

categories of accidents and incidents. The “Vehicle Health Assurance” was mapped to 

system/component failures/malfunctions (SCFM) and non-impact fires, and “Aircraft Loss-ofControl Prevention and Mitigation” TC to the “loss of control-inflight” category. 

However, the “Effective Crew-System Interactions and Decisions in All Conditions” TC, which is 

now the current CDM TC, was mapped to select categories of causal factors because it could 

not be associated with a particular accident category. Given that the NTSB coding process does 

not use terms such as “crew-system interaction”, the Aviation Safety Analysis and Functionality 

Evaluation (ASAFE) program (ref. 6) was used because it contains a taxonomy that maps

specific causal factor codes used by the NTSB to more general error types based upon an 

expanded human factors analysis and classification system (HFACS) (ref. 7). For example, 

there are fourteen categories of errors specific to each of flight crew, ground personnel, air 

traffic control and maintenance. By using the ASAFE taxonomy, events with “flight crew 

decision errors” and “skill-based errors” could be selected and mapped to the “Effective CrewSystem Interactions and Decisions in All Conditions” TC. 

In order to better understand the ASAFE taxonomy with respect to the two types of errors 

chosen for this analysis, the NTSB database was searched. This search revealed that the most 

frequently cited flight crew skill-based errors were the following: 

• failure to maintain aircraft control or directional control 

• inadequate compensation for wind conditions

• inadvertent stall

• inadequate preflight preparation 

• inadequate visual lookout

In addition to the more generic errors of inadequate or poor pre-flight or in-flight planning or 

decisions, the most frequently cited specific decision errors were as follows: 

• selection of an inappropriate takeoff or landing site

• failing to refuel the aircraft
Crew Decision Making Trends and Issues

Page 5

• failing to perform an aborted landing or aborted takeoff

• selection of the wrong runway

• excessive airspeed

• inappropriate use of certain controls (usually brakes, carburetor heat or landing gear 

extension)

The accidents with flight crew skill-based errors and flight crew decision errors were selected if 

the error was considered either a cause or a factor (but not a finding) in the accident report. 

Causes are actions or events that are the direct source of the accident, while factors are actions 

or events that contributed to the accident (ref. 8). Each accident can have multiple causes and 

factors. Findings are actions or events that occurred in conjunction with the accident, but were 

not determined to have contributed to the accident.

The ASAFE taxonomy cannot be applied to the incident data because the level of causal factor 

coding is far less detailed. Each incident has been assigned a general cause in one of three 

categories: individual person (pilot, maintenance crew or ground crew), operational deficiency 

(mostly non-maintenance related system component failures) or environmental (weather or 

terrain conditions). Thus we are only able to report the proportion of incidents caused by pilot 

action or inaction, without specifying the type of pilot error.

Results

A summary of the accidents affected by the two selected types of pilot error can be found in 

Tables 2 and 3. Denominators are included in many lines for clarification, specifically with 

respect to fatal accident percentages. There are two types of fatal accident percentages, and 

both are of interest: the percentage of all fatal accidents that included a particular type of pilot 

error, and the percentage of accidents marked by the particular pilot error in which there was a 

fatality. The numerators in these percentages are identical (fatal pilot error accidents) but the 

denominators are different. 

More than half of the accidents and fatal accidents in Part 135 and Part 91 were caused in part 

by a flight crew skill-based error (Table 2), compared with less than one quarter of Part 121 

accidents. Part 121 accidents with skill-based errors also were less likely to be fatal (7%) than 

accidents in other operations (22%-30%). 
Crew Decision Making Trends and Issues

Page 6

Table 2. Summary of Flight Crew Skill-Based Error (SBE) Accidents by Operation

Category (1989-2008)

Decision errors (Table 3) were noted as the cause of fewer accidents overall (13%-30%) than 

skill-based errors, but for most operation categories, a slightly higher percentage of them 

include at least one fatality (13%-29%). The differences in percentages between operational

categories are less striking for decision errors than for skill-based errors.

Part 121 Part 135—

Scheduled

Part 135—

NonScheduledPart 91

Total Flight Hours 317,999,117 24,446,927 61,751,000 507,516,000

Total Accidents 738 219 1,202 26,922

Accidents with SBE 174 (23.6%) 123 (56.2%) 631 (52.5%) 17962 (66.7%)

SBE Accidents

per million flight hrs 0.547 5.031 10.218 35.392

Total Fatal Accidents 67 48 307 5214

Fatal SBE Accidents 

out of All Fatal 

Accidents

12/67 (17.9%) 29/48 (60.4%) 190/307 

(61.9%)

3892/5214 

(74.6%)

Fatal SBE Accidents 

out of All SBE 

Accidents

12/174 ( 6.9%) 29/123 (23.6%) 190/631 

(30.1%)

3892/17962 

(21.7%)

Total Fatalities 1956 309 745 10,109

Fatalities in accidents 

with a Skill-Based 

Error (SBE)

122 ( 6.2%) 150 (48.5%) 431 (57.9%) 7531 (74.5%)
Crew Decision Making Trends and Issues

Page 7

Table 3. Summary of Flight Crew Decision Error (DE) Accidents by Operation 

Category (1989-2008)

Figures 2-5 display the percentage of accidents in each time period with the two types of pilot 

error as a cause or factor in the accident for each of the operational categories. In Part 121 

accidents (Figure 2), decision errors have declined substantially over time, but skill-based errors 

have consistently been a factor in twenty to twenty-five percent of accidents.

Among Scheduled Part 135 accidents (Figure 3), the percentage of accidents with skill-based 

errors dropped during 1999-2003, but rose again in the latest time period, while decision errors 

reached a peak during 1999-2003.

Among Non-Scheduled Part 135 accidents (Figure 4), skill-based errors have declined, and the 

percentage of accidents with decision errors has remained fairly steady.

Among Part 91 accidents (Figure 5), there has been a decrease in the percentage of accidents 

with decision errors, but there has been little change in the percentage of accidents with skillbased errors.

Part 121 Part 135—

Scheduled

Part 135—

NonScheduledPart 91

Total Flight Hours 317,999,117 24,446,927 61,751,000 507,516,000

Total Accidents 738 219 1,202 26,922

Accidents with DE 96 (13.0%) 53 (24.2%) 363 (30.2%) 7306 (27.1%)

DE Accidents

per million flight hrs 0.302 2.168 5.878 14.395

Total Fatal Accidents 67 48 307 5214

Fatal DE Accidents 

out of All Fatal 

Accidents 

12/67 (17.9%) 14/48 (29.2%) 104/307 

(33.9%)

1694/5214 

(32.5%)

Fatal DE Accidents 

out of All DE 

Accidents

12/96 (12.5%) 14/53 (26.4%) 104/363 

(28.7%)

1694/7306 

(23.2%)

Total Fatalities 1956 309 745 10,109

Fatalities in accidents 

with Decision Errors 341 (17.4%) 52 (16.8%) 266 (35.7%) 3259 (32.2%)
Crew Decision Making Trends and Issues

Page 8

Figure 2. Percentage of Part 121 Accidents with the Specified Error Type

Across Four Time Periods.

Figure 3. Percentage of Scheduled Part 135 Accidents with the Specified Error 

Type Across Four Time Periods.
Crew Decision Making Trends and Issues

Page 9

Figure 4. Percentage of Non-Scheduled Part 135 Accidents with the Specified 

Error Type Across Four Time Periods.

Figure 5. Percentage of Part 91 Accidents with the Specified Error Type

Across Four Time Periods.

The incident data in general are not as detailed as the accident data, and that is particularly true 

with regard to causal factors. Only a single general cause is recorded for each incident, 
Crew Decision Making Trends and Issues

Page 10

although for some incidents the cause is shared (e.g., pilot and maintenance, pilot and ground 

crew). Table 4 shows the percentage of incidents for which the pilot was assigned blame, either 

solely or in part.

The percentage of incidents caused by any pilot error ranges from 13 percent (Part 121) to 56 

percent (Part 91). In general, pilot error rates for Part 91 are roughly double those for NonScheduled Part 135, which in turn are roughly double those for Scheduled Part 135. Part 121 

incidents invoving pilot error were always less frequent than the other flight operational 

categories with one exception; errors by ground crew and pilot, which are highest in Part 121. 

This is not surprising, since Part 121 flights are far more likely than those in other flight 

categories to require the assistance of ground crew.

Table 4. Summary of Incidents Caused by Pilot Error by Operation Category

(1989-2006)

Figures 6-9 display the percentage of incidents in each time period with the cause of the 

incident determined to be pilot error only or pilot error plus an additional factor (ground crew, 

maintenance or weather).

Among Part 121 incidents (Figure 6), the overall percentage of incidents for which pilot error 

was the cause has decreased since 1998. It should be noted that there was a slight increase in 

the 2004-2006 time period. The percentage of incidents in which the pilot shared blame with the 

ground crew, maintenance or weather has generally increased over time.

Among Scheduled Part 135 incidents (Figure 7), the proportion of incidents caused by pilot error 

alone has been substantially higher since 1998 than it was prior to 1998.

Among Non-Scheduled Part 135 incidents (Figure 8), the percentage of incidents with the single 

cause of pilot error has increased consistently until the last time period of 2004-2006, just as the 

percentage of pilot plus caused incidents has decreased

Incident Cause Part 121 Part 135—

Scheduled

Part 135—

NonScheduledPart 91

Total Incidents 7647 1921 2189 27,919

Pilot Error Only 714 ( 9.3%) 221 (11.5%) 470 ( 21.5%) 11961 (42.8%)

Pilot Error plus Ground 

Crew 167 ( 2.2%) 23 ( 1.2%) 16 ( 0.7%) 72 ( 0.3%)

Pilot Error plus 

Maintenance 79 ( 1.0%) 38 ( 2.0%) 57 ( 2.6%) 1142 ( 4.1%)

Pilot Error plus Weather 72 ( 0.9%) 41 ( 2.1%) 102 ( 4.7%) 2402 ( 8.6%)

Any Pilot Error 1032 (13.5%) 323 (16.8%) 645 (29.5%) 15577 (55.8%)
Crew Decision Making Trends and Issues

Page 11

Figure 6. Percentage of Part 121 Incidents with Pilot Error Across Four Time 

Periods (1989-2006)

Figure 7. Percentage of Scheduled Part 135 Incidents with Pilot Error

Across Four Time Periods (1989-2006)
Crew Decision Making Trends and Issues

Page 12

Figure 8. Percentage of Non-Scheduled Part 135 Incidents with Pilot Error

Across Four Time Periods (1989-2006)

Figure 9. Percentage of Part 91 Incidents with Pilot Error Across Four Time 

Periods (1989-2006)

Among Part 91 incidents (Figure 9), both pilot error percentages have remained relatively 

steady over time.
Crew Decision Making Trends and Issues

Page 13

2.2 NASA Analysis of ASRS Data

In 2011, AvSP systems analysis personnel completed a study (ref. 4) of Aviation Safety 

Reporting System (ASRS) data. The human factors related incidents were analyzed because 

the system analysis personnel believed these incidents were applicable to the “Effective CrewSystem Interactions and Decisions in All Conditions” (currently known as CDM) TC. In 

response to the systems analysis team’s request for incident reports involving human factor 

related incidents during the time period January, 1993 through January, 2011, the analysts at 

ASRS provide a dataset of 2,243 incidents for Part 121, 135, and 91 operations. The category 

of human factors is a new category added to the search criteria for the ASRS data and is only 

available from May, 2009, instead of January, 1993. The data prior to this most likely also has 

human factor information in it, but this information would require using a free text search string 

or by reading through each report. 

Information provided included the incident report number, date of incident, phase of flight, 

aircraft make/model, SCFM, and human factors. The human factor categories used in the 

analysis are the following, in alphabetical order:

1. Communication Breakdown

2. Confusion

3. Distraction

4. Fatigue

5. Human-Machine Interface

6. Other / Unknown

7. Physiological-Other

8. Situational Awareness

9. Time Pressure

10. Training / Qualification

11. Troubleshooting

12. Workload
Crew Decision Making Trends and Issues

Page 14

The human factors category can list more than one human factor involved for each incident 

report. A more in depth analysis of this raw data set was conducted to look at the conditions 

present when human factor incidents occurred. This information may be useful to researchers 

in determining areas of future research.

Results

The result of breaking the data into the human factor categories can be seen in Figure 10. In 

many of the reported incidents there were multiple human factors listed. There were 2,152 

reports that listed the specific human factor, with a total of 5,949 human factors listed in those 

2,152 reports. Situational Awareness was the tallest pole (with 1,309 incidents), followed by 

communication breakdown (844 incidents), confusion (757 incidents) human-machine interface 

(510 incidents), and distraction (506 incidents). Fatigue and physiological-other had the fewest 

reports with 95 and 84 incidents respectively.

Communic

ation 

Breakdown

Confusion Distraction Fatigue

HumanMachine 

Interface

Other / 

Unknown

Physiologic

al-Other

Situational 

Awareness

Time 

Pressure

Training / 

Qualificatio

n

Troublesho

oting

Sum 844 757 506 95 510 329 84 1309 324 402 378

0

200

400

600

800

1000

1200

1400

Incident Frequency

Human Factor Related Incidents

Human Factors

Figure 10. Human Factor Frequency for Human Factor Related Incidents

Forty four percent of the human factors related data, or 977 incidents, have an aircraft system 

component failure listed. The remaining 1266 human factor incidents did not list an SCFM. 

Figure 11 shows the results for the SCFM breakdown. The SCFM tall poles are navigation at 

155, propulsion systems at 139, and monitoring and management at 121. 
Crew Decision Making Trends and Issues

Page 15

Figure 11. SCFM Frequency for Human Related Incidents

The human factors per phase of flight were then broken down into FAR parts. Table5 displays

the results for FAR part 121. The categories with more than 60 incidents are highlighted for 

emphasis. The largest categories were in the parked phase of flight: communication breakdown 

with 171 incidents and situational awareness with 168. Of the Part 121 incidents, the parked 

phase of flight had the most tall poles, and the situational awareness human factor had the 

largest number of tall poles. The human factors of fatigue and physiological-other had the 

fewest incident reports for all phases of flight: for the different phases of flight, the final approach 

and landing categories had the fewest incident reports for all of the human factors categories.

Table 5. Frequency of Part 121 Human Factor per Phase of Flight for Human 

Factor Related Incidents

Human Factor Taxi Take 

Off

Initial 

Climb Climb Cruise Descent Initial Approach

Final 

Approach Landing Parked

Communication Breakdown 76 36 24 42 72 52 40 20 19 171

Confusion 67 41 32 58 50 57 37 27 21 104

Distraction 43 28 26 34 59 34 36 24 12 55

Fatigue 5 3 0 6 3 2 13 6 2 24

Human Machine Interface 16 43 28 52 54 39 32 24 11 26

Other / Unknown 8 12 13 24 44 29 18 11 16 30

Physiological-Other 4 6 4 9 18 10 6 3 7 12

Situational Awareness 101 79 47 77 113 84 74 42 46 168

Time Pressure 12 29 19 22 39 22 20 9 10 59

Training / Qualification 23 30 16 26 45 14 21 5 6 65

Troubleshooting 15 29 21 42 73 19 14 7 2 63

Workload 21 20 29 34 56 30 29 9 10 43

Number of Incident Reports 136 130 82 148 210 144 112 70 71 259

Phase of Flight - Part 121

67

8

33 29

72

90 80

27 40

76 83

22

121

155

28

139

54

4

0

20

40

60

80

100

120

140

160

180

Incident Frequency

Human Factor Related Incidents

Aircraft System Component Failure
Crew Decision Making Trends and Issues

Page 16

Table 6 contains the results of the FAR part 135 human factor per phase of flight analysis. 

Confusion during initial climb was the largest reported incident with 11, followed by situational 

awareness during initial climb and cruise, both with 9 incidents. The phase of flight with the 

most tall poles was the initial climb, and the category of human factors with the most tall poles 

was situational awareness. Many categories had no incidents or only one incident for part 135.

Table 6. Frequency of Part 135 Human Factor per Phase of Flight for Human 

Factor Related Incidents

Human Factor Taxi Take 

Off

Initial 

Climb Climb Cruise Descent Initial 

Approach

Final 

Approach Landing Parked

Communication Breakdown 2 2 7 2 2 2 1 3 1 3

Confusion 1 3 11 2 5 1 2 3 3 3

Distraction 1 2 5 4 4 0 1 1 2 2

Fatigue 0 0 0 0 0 2 0 0 0 1

Human Machine Interface 1 3 6 3 0 2 0 0 0 1

Other / Unknown 0 0 0 0 3 0 0 0 0 0

Physiological-Other 0 0 0 0 3 0 0 0 0 0

Situational Awareness 3 3 9 8 9 1 2 5 5 5

Time Pressure 1 1 4 3 3 0 1 1 1 3

Training / Qualification 0 0 4 1 1 0 0 0 0 0

Troubleshooting 1 2 2 3 1 0 0 0 0 2

Workload 1 1 8 2 4 2 1 1 2 0

Number of Incident Reports 3 7 17 8 12 2 2 6 5 7

Phase of Flight - Part 135

FAR part 91 human factors per phase of flight results are shown in Table 7. It had two tall poles 

both in situational awareness, one during cruise with 71 incidents and the other during landing

with 61 incidents. For human factors, there was a higher concentration of tall poles in 

situational awareness. In the area of phase of flight the tall poles were more heavily 

concentrated during cruise. Fatigue, physiological-other, and other/unknown had the fewest 

human factor incidents per phase of flight for part 91. The phase of flight for human factors with 

the fewest incidents was the parked phase of flight.
Crew Decision Making Trends and Issues

Page 17

Table 7. Frequency of Part 91 Human Factor per Phase of Flight for Human 

Factor Related Incidents

Human Factor Taxi Take 

Off

Initial 

Climb Climb Cruise Descent Initial 

Approach

Final 

Approach Landing Parked

Communication Breakdown 15 16 26 12 33 19 22 17 27 7

Confusion 20 12 27 13 24 17 18 20 20 6

Distraction 8 10 15 12 30 13 17 13 19 3

Fatigue 0 3 0 1 0 0 1 2 2 0

Human Machine Interface 6 9 25 11 28 12 20 10 19 1

Other / Unknown 3 3 3 4 11 6 3 3 12 0

Physiological-Other 1 2 1 0 2 1 1 1 1 0

Situational Awareness 31 33 40 22 71 28 35 27 61 9

Time Pressure 9 6 8 4 15 9 3 6 9 0

Training / Qualification 9 13 14 10 22 10 7 12 35 7

Troubleshooting 4 7 7 7 31 6 6 4 9 3

Workload 4 3 15 12 21 16 9 8 16 0

Number of Incident Reports 40 44 57 34 102 41 48 37 92 12

Phase of Flight - Part 91
Crew Decision Making Trends and Issues

Page 18

3. REVIEW OF CREW SYSTEMS ISSUES AND FUTURE RESEARCH 

NEEDS

Subject-matter experts and researchers have conducted a number of studies that included 

recommendations for improving the safety of the air transportation system. This section contains 

a review of these safety priority lists, information databases and other documented references 

pertaining to aviation crew systems issues and future research needs. 

3.1 CAST Safety Enhancements Reserved for Future Implementation (SERFIs)

The Commercial Aviation Safety Team (CAST) was formally established on June 23, 1998 (ref. 

9). Its mission is to provide government and industry leadership to develop and focus

implementation of an integrated, data-driven strategy to improve commercial aviation safety. 

The members of CAST are a variety of stakeholders, including representatives from 

government, industry, pilot groups, air traffic controllers, and others. The Joint Implementation 

Monitoring Data Analysis Team (JIMDAT) is a working group with CAST. The JIMDAT monitors 

the implementation of the Safety Enhancements (SEs) in the CAST plan and suggests 

modifications and changes to CAST (ref. 10).

In addition the current set of SEs in the CAST plan, there are number of SEs that could be 

added in the future. These SEs are referred to as “Safety Enhancements Reserved for Future 

Implementation” or SERFIs (ref. 11) for short. Table 8 contains a list of SERFIs that could 

possibly fall within the charter of the CDM TC. The SERFIs highlighted in the following table 

may not be currently funded by the NASA Aviation Safety Program. Detailed descriptions of 

each CDM related SERFI can be found immediately after Table 8.
Crew Decision Making Trends and Issues

Page 19

Table 8. CDM Related CAST SERFIs

SE # Title

LOOSEC NASA 

Role

22 Flight Deck Equipment Upgrades – Existing Type Designs (4-5) FAA AIR-1 S

35 Displays and Alerting Systems – Existing Designs AIA

36 Basic Airplane Design – Mode Confusion (1-2) NASA L

61 Aircraft/Vehicle Upgrade and Installation – Part 121/135 Cockpit 

Moving Map Display FAA AVR-1 S

62 Aircraft/Vehicle Upgrade and Installation – Vehicle Moving Map 

Display FAA ARP S

63 Aircraft/Vehicle Upgrade and Installation – Non-121 Aircraft 

Moving Map Display FAA AVR-1 S

71 Graphical Displays – Carry On NASA L

72 Graphical Displays – Panel Mounted – New Production NASA L

110 Cockpit Moving Map Phase 1 FAA AVR-1 S

111 Cockpit Moving Map Phase 1 & 2 FAA AVR-1 S

112 Cockpit Moving Map Phase 1, 2, & 3 FAA AVR-1 S

LEGEND

L

NASA has a Leading role: NASA is the Lead Overall Organization for Safety Enhancement 

Completion (LOOSEC), or the Lead Organization for Output Completion for one or more 

SE Outputs, or is assigned a specific action in an Output

S NASA has a Supporting role: NASA is shown as a contributing Resource in the SE or has 

Current Related Aviation Activities listed in the SE, but has no specific assigned actions

NASA is not listed anywhere in the SE as a contributing organization

SE falls within charter of CDM. Does not necessarily mean that work is currently funded 

by AvSP
Crew Decision Making Trends and Issues

Page 20

SE035: Displays and Alerting Systems – Existing Designs (R-D)

This safety enhancement outlines research and development (R&D) topics 

needed to evaluate display and alerting systems in existing type designs to 

provide: 

1. Graphic speed trend information 

2. A pitch limit indication 

3. Bank angle limits to buffet 

4. Barber poles and amber bands on primary airspeed indications 

5. Detection and annunciation of conflicting attitude, airspeed and altitude 

data information 

6. Detection and removal of invalid attitude, airspeed and altitude data 

information (i.e.. from an internal fault) 

7. Detection and removal of misleading attitude, airspeed and altitude data 

information (e.g., from an external sensor fault) to the extent feasible 

8. Information to perform effective manual recovery from unusual attitudes 

using chevrons, sky pointers, and/or permanent ground-sky horizon on all 

attitude indications 

9. Salient annunciation of autoflight mode changes and engagement status 

changes (e.g., blinking/colored/boxed mode information) 

10. Effective sideslip information and alerting of excessive sideslip (e.g., split 

trapezoid on attitude indicator) 

11. Clear annunciation of engine limit exceedances and significant thrust loss. 

SE022: Flight Deck Equipment Upgrades – Existing Type Designs

This safety enhancement ensures altitude awareness and accomplishment of 

checklist items. This will be accomplished through the development of guidelines 

and procedures for flight deck smart alerting system design and supporting 

operational procedures and training based upon: 

1. The installation of equipment to provide automatic aural altitude alert callouts on final approach or other such altitude alerting systems. 

2. The installation of automated or mechanical checklist devices to provide a 

positive means for checklist completion. 

3. Research and assessment of existing technology in flight deck smartalerting system design. 
Crew Decision Making Trends and Issues

Page 21

SE071: Turbulence Displays - Carry-on Equipment

The purpose of this Safety Enhancement is to provide improved, real-time 

turbulence information to pilots, ground operations personnel, and forecasters for 

turbulence avoidance decisions and for input to turbulence forecasts. This will be 

accomplished through the improvement of automated airborne turbulence

measurements; new flight deck displays of turbulence information; improved or 

new on-board look-ahead turbulence detection capabilities; and upgraded flight 

crew procedures for use of the improved information to avoid turbulence. 

SE071 assumes equipage with electronic flight bag (EFB) or other carry-on 

displays. The enhancement provides first-generation turbulence displays for 

these products via integration of up-linked information on existing carry-on 

displays.

SE063: General Aviation Aircraft Moving Map Display

The purpose of this Safety Enhancement is to reduce runway incursion incidents 

by improving pilot situational awareness using cockpit moving map technology in 

general aviation (GA) aircraft.

SE062: Airport Vehicle Moving Map Display

The purpose of this Safety Enhancement is to reduce runway incursion incidents 

by improving ground vehicle operator situational awareness using airport moving 

map technology.

SE061: Cockpit Moving Map Display

The purpose of this Safety Enhancement is to reduce runway incursions by 

improving pilot situational awareness using cockpit/vehicle moving map 

technology in commercial aircraft. The requirements for the moving map 

implementation are organized into four phases. Phase 1 will address 

development and installation of cockpit moving map (airport) displays with ownship position enabled by GPS. Phase 2 will add display functionality for datalinked traffic, ground and air, utilizing ADS-B and TIS-B. Phase 3 will add 

functionality for runway occupancy advisory systems. Phase 4 will add 

functionality for data-linked taxi routes and clearance limits. Each phase will also 

address heads-up guidance display systems (HUDs). Each phase will require the 

continuing development and certification of cockpit display equipment and the 

formation of standards, guidelines and procedures for use of the equipment

SE036: Basic Airplane Design – Mode Confusion (R-D)

This safety enhancement outlines research and development (R&D) topics aimed 

at reducing fatal accidents due to mode confusion leading to loss of control. 

Information regarding observed instances of flight crew mode confusion (by 

airplane model) should be disseminated to operator training departments, 

manufacturers, and the aviation human factors research community.
Crew Decision Making Trends and Issues

Page 22

SE112: Moving Map Display with Own-Ship, Traffic, and Runway 

Occupancy Advisory System

The purpose of this Safety Enhancement is to reduce runway incursions by 

improving pilot situational awareness using cockpit/vehicle moving map 

technology in commercial aircraft. The requirements for the moving map 

implementation are organized into four phases. SE112 builds upon SE110 and 

SE111 by adding functionality for runway occupancy advisory systems to Phase 

1 (own-ship position enabled by GPS) and Phase 2 (data-linked traffic enabled by 

ADS-B/TIS-B).

SE111: Moving Map Display with Own-Ship and Traffic Enabled by ADS-B/

TIS-B

The purpose of this Safety Enhancement is to reduce runway incursions by 

improving pilot situational awareness using cockpit/vehicle moving map 

technology in commercial aircraft. The requirements for the moving map 

implementation are organized into four phases. SE111 builds upon SE110 by 

adding display functionality for data-linked traffic, ground and air, utilizing ADS-B 

and TIS-B to Phase 1, which addresses development and installation of cockpit 

moving map (airport) displays with own-ship position enabled by GPS

SE110: Moving Map Display with GPS Own-Ship Position

The purpose of this Safety Enhancement is to reduce runway incursions by 

improving pilot situational awareness using cockpit/vehicle moving map 

technology in commercial aircraft. The requirements for the moving map 

implementation are organized into four phases. SE110 includes Phase 1 to 

address development and installation of cockpit moving map (airport) displays 

with own-ship position enabled by GPS.

SE072: Graphical Turbulence Displays – Panel Mounted – New Production

The purpose of this Safety Enhancement is to provide improved, real-time 

turbulence information to pilots, ground operations personnel, and forecasters for 

turbulence avoidance decisions and for input to turbulence forecasts. This will be 

accomplished through the improvement of automated airborne turbulence 

measurements; new flight deck displays of turbulence information; improved or 

new on-board look-ahead turbulence detection capabilities; and upgraded flight 

crew procedures for use of the improved information to avoid turbulence. 

SE072 recommends panel-mounted graphical turbulence displays for new 

production airplanes. 
Crew Decision Making Trends and Issues

Page 23

3.2 Flight Deck Automation Issues (FDAI) Study

The Flight Deck Automation Issues (FDAI) website (ref. 12) contains the results of a study 

conducted by principal investigators from Oregon State University, Research Integrators, Inc 

and Honeywell. The study (ref. 13) was funded by the FAA, Office of the Chief Scientific and 

Technical Advisor for Human Factors (AAR-100). The study, which was conducted over a ten 

year period (1997-2007), contained two phases: (1) identification of possible problems and 

concerns related to flight deck automation and (2) compilation of evidence related to flight deck 

automation issues. Two taxonomies of flight deck automation problems and concerns were 

developed based on the information obtained in phase one of the study. The “primary 

taxonomy” organizes the perceived problems and concerns into three major categories; reason 

for automation existence, automation design, and automation use. In the “alternative taxonomy”

of flight deck automation problems and concerns, the 2,428 citations of 114 issues were divided 

into five distinct categories (Figure 12). The two largest of these categories, “Pilot Centered 

Problems and Concerns” and “Automation-Centered Problem and Concerns” can be furthered 

divided into the subcategories shown in Figures 13 and 14. Of the 19 sub-categories identified, 

“pilot/automation interface” was the most frequently cited problem/concern (13%).

Figure 12. Percentage of Total Citations Related to the FDAI Alternative 

Taxonomy of Flight Deck Automation Problems and Concerns
Crew Decision Making Trends and Issues

Page 24

Figure 13. Percentage of the Total Citations Related to the FDAI Pilot-Centered 

Problems and Concerns Subcategories Taxonomy

Figure 14. Percentage of Total Citations Related to the FDAI AutomationCentered Problems and Concerns Subcategories Taxonomy
Crew Decision Making Trends and Issues

Page 25

3.3 NTSB Most-Wanted List

Every year the NTSB publishes a list of the most-wanted transportation safety improvements for 

various modes of transportation (e.g., highway). Although most of the research conducted in the 

NASA AvSP is directed toward commercial aircraft operations, a portion of the work in CDM 

may be applicable to the issue of “Improving General Aviation Safety”, which is currently on the 

NTSB’s Most Wanted List (ref. 14). Specific recommendations to address this Most Wanted 

issue include:

• “Adequate education and training and screening for risky behavior are critical to 

improving general aviation safety. For example, guidance materials should include 

information on the use of Internet, satellite, and other data sources for obtaining weather 

information

• Training materials should include elements on electronic primary flight displays, and 

pilots should have access to flight simulators that provide equipment-specific electronic 

avionics displays. 

• Knowledge tests and flight reviews should test for awareness of weather, use of 

instruments, and use of "glass" cockpits. 

• Need for a mechanism for identifying at-risk pilots and addressing risks so that both the 

pilot and passengers can safely fly.”

In addition, the NTSB also issues safety recommendations as a result of accident investigations 

and other safety concerns that arise. Some recent open recommendations that could possibly 

be related to CDM research are listed in Table 9.
Crew Decision Making Trends and Issues

Page 26

Table 9. Recent NTSB Recommendations Related to CDM Issues

Recommendation # Recommendation

A-11-092 Require principal operations inspectors to review flight crew training 

programs and manuals to ensure training in tailwind landings is (1) 

provided during initial and recurrent simulator training; (2) to the extent 

possible, conducted at the maximum tailwind component certified for the 

aircraft on which pilots are being trained; and (3) conducted with an 

emphasis on the importance of landing within the touchdown zone, 

being prepared to execute a go-around, with either pilot calling for it if at 

any point landing within the touchdown zone becomes unfeasible, and 

the related benefits of using maximum flap extension in tailwind 

conditions.

A-11-039 Require that role-playing or simulator-based exercises that teach first 

officers to assertively voice their concerns and that teach captains to

develop a leadership style that supports first officer assertiveness be 

included as part of the already required crew resource management 

training for 14 Code of Federal Regulations Part 121, 135, and 91 

subpart K pilots.

A-11-028 Actively pursue with aircraft and avionics manufacturers the 

development of technology to reduce or prevent runway excursions and, 

once it becomes available, require that the technology be installed.

A-10-037 Require all manufacturers of certified electronic primary flight displays to 

include information in their approved aircraft flight manual and pilot’s 

operating handbook supplements regarding abnormal equipment 

operation or malfunction due to subsystem and input malfunctions, 

including but not limited to pitot and/or static system blockages, 

magnetic sensor malfunctions, and attitude-heading reference system 

alignment failures.
Crew Decision Making Trends and Issues

Page 27

4. DISCUSSION AND CONCLUSIONS

The CDM TC, does not address a specific CAST/ICAO occurrence category, but instead 

addresses specific causal factors which lead to accidents: flight crew skill-based errors and 

flight crew decision errors. Skill-based errors were a causal factor in more than half of all Part 

135 accidents, two thirds of Part 91 accidents, and almost one quarter of Part 121 accidents. 

For all operational categories, accidents involving skill-based errors have not decreased 

appreciably during the study time period. Accidents involving decision errors have decreased 

for all operational categories except in Scheduled Part 135 operations. Even though more 

accidents are caused by skill-based errors, those accidents involving decision errors are more 

likely to be fatal than those accidents with skill-based errors. In Part 121 accidents, decision 

error resulted in more fatalities than those accidents with skill-based errors.

The human factor related incidents analysis of ASRS dataset resulted in 2,243 reports. It is 

important to note that the category of human factors in the ASRS data set has only been added 

since May, 2009 and that previous data within ASRS probably has human factor related 

incidents in it but that they are not categorized as such. A larger time frame would be helpful to 

better understand the causes of human error in the incident data. The data was analyzed in 

four ways: FAR part (121, 135, and 91), 10 phases of flight, 18 SCFM categories, and 12 

human factors. There were 2,243 phase of flight incident reports, 1,966 FAR part 121, 135, and 

91 incident reports, 977 SCFM listings, and 2,152 with specific human factors categories. When 

analyzing the human factors, FAR parts, and phase of flight, situational awareness was the top 

problem for all the FAR part categories. Communication breakdown and confusion also had 

higher frequencies than the other human factor categories. There was no one phase of flight 

that had the majority of problems. The fewest human factors were seen in fatigue and 

physiological-other. Analyzing human factors and SCFMs per FAR part also showed situational 

awareness as being the largest incident issue. In the future more analysis needs to be done in 

the area of situational awareness to determine what the biggest issues are in that human factors

category.

A review of crew systems related issues in safety priority lists, information databases and other 

document references identified a number of areas related to the CDM TC. Out of the 47 SERFIs 

that have been identified by the CAST, 11 of these Safety Enhancements could possibly fall 

within the charter of the CDM TC. In addition, many of the crew related issues identified by in 

the FDAI study and in published literature are closely related to the CDM goals and products 

(Figure 1). For example, the top three automation-center problems identified in the FDAI study, 

“pilot/automation interface”, “functionality (capabilities and limitations)” and “automation failure” 

are all being addressed by researchers within CDM. Although the current CDM products 

primarily focuses on crew related technologies and capabilities, modifications can be made in 

the future so that more of the CDM portfolio addresses the NTSB’s general aviation 

recommendations. 

One of the upcoming tasks of the AvSP systems analysis personnel is to develop a Bayesian 

model similar to the one created for loss of control (ref. 15, 16, 17), that examines the role of 

increasing complexity and reliance on automation in the future National Airspace System. This 

model will be used by AvSP management for identifying any gaps in its research portfolio and 

also to evaluate the impact of its research products on reducing the risk of an aviation accident 

in the future. The FDAI “alternative taxonomy” was selected as a foundation for organizing 

automation related issues in the “complexity and reliance on automation” Bayesian model
Crew Decision Making Trends and Issues

Page 28

currently under development. However, the FDAI “alternative taxonomy” was lacking some 

characteristics needed for the modeling the effort such as time-dependencies (e.g., latent vs. 

active causes), a further breakdown of automation-centered issues, etc. Therefore, the AvSP 

systems analysis personnel combined the FDAI “alternative taxonomy” with other related 

research pertaining to human factors, automation and flight deck design (ref. 18, 19, 20), to 

create the list of automation issues outlined in Table 10. 

Table 10. Summary of Automation Issues

Automation 

Centered Issues Operations Pilot Centered Issues

OrganizationCentered 

Issues

Automated 

Aircraft/ATC 

InteractionCentered 

Issues

Flight Crew 

Interface/Design

 Development 

process

 Pilot/automation 

interface

 Humancentered design

General 

Automation 

Design

 Functionality

 Complexity

 Levels of 

automation

 Automation 

failure

Underlying 

Philosophy of 

Automation

 Standardization

 Pilot/automation 

responsibility 

and authority

 Pilot role

Crew 

Performance

 Crew 

coordination

 Performance

 Use of 

automation

Awareness

 Situation/energy 

awareness

 Automation 

awareness

Workload

 Attention

 Workload

 Skill

 Confidence/trust

 Understanding 

of automation

 Crew personnel

factors

 Company 

automation 

philosophies, 

policies , and 

procedures

 Pilot selection, 

training, and 

evaluation

 ATC conflicts
Crew Decision Making Trends and Issues

Page 29

REFERENCES

1. Krasa, Paul; and Graves, Sharon: Vehicle Systems Safety Technology (VSST) Project Plan, 

National Aeronautics and Space Administration, February 1, 2013, internal paper.

2. Gibson, John E.; Scherer, William T.; and Gibson, William F.: How to Do Systems Analysis, 

John Wiley & Sons, Inc., Hoboken, New Jersey, 2007.

3. Evans, Joni K.: An Examination of Aviation Accidents and Incidents During The Years 1989-

2008 Associated With Technical Challenges Within The Vehicle Systems Safety Technologies 

(VSST) Project, July 2011, internal paper.

4. Withrow, Colleen A.; and Reveley, Mary S.: Analysis of Aviation Safety Reporting System 

Incident Data Associated with the Technical Challenges of the Vehicle Systems Safety 

Technology Project, June 2011, internal paper.

5. Commercial Aviation Safety Team/International Civil Aviation Organization (CAST/ICAO) 

Common Taxonomy Team: Aviation Occurrence Categories: Definitions and Usage Notes, 

version 4.1.5, April 2011. http://www.intlaviationstandards.org/Documents/

CICTTOccurrenceCategoryDefinitions.pdf.

6. National Aeronautics and Space Administration: Aviation Safety Analysis and Functionality 

Evaluation (ASAFE). https://asafe.larc.nasa.gov

7. Shappell, Scott A.; and Wiegmann, Douglas A.: The Human Factors Analysis and 

Classification System, U.S. Department of Transportation, Office of Aviation Medicine, 

DOT/FAA/AM-00/7, February, 2000.

8. National Aeronautics and Space Administration: Aviation Safety Analysis and Functionality 

Evaluation (ASAFE) – Definitions. https://asafe.larc.nasa.gov/DOC/definitions.html

9. Commercial Aviation Safety Team: The Commercial Aviation Safety Team, 2011. http://castsafety.org/.

10. Federal Aviation Administration: Fact Sheet – Commercial Aviation Safety Team, December 

1, 2011. http://www.faa.gov/news/fact_sheets/news_story.cfm?newsId=13257. 

11. Commercial Aviation Safety Team: CAST Safety Enhancements Reserved for Future 

Implementation, September 4, 2012. http://www.skybrary.aero/index.php/Category:CAST_

Safety_Enhancements_Reserved_for_Future_Implementation_%28SERFIs%29).

12. Research Integrations, Inc, Flight Deck Automation Issues (FDAI) Website, 

September 20, 2007. http://www.flightdeckautomation.com/. 

13. Funk, Ken; Lyall, Beth; and Riley, Victor: Perceived Human Factors Problems of Flightdeck 

Automation (Phase 1 Final Report). Corvallis: Oregon State University, 1996 URL: 

http://www.flightdeckautomation.com/phase1/phase1report.aspx.
Crew Decision Making Trends and Issues

Page 30

14. National Transportation Safety Board: NTSB Most Wanted List – Improve General Aviation 

Safety, November 14, 2012. http://www.ntsb.gov/safety/mwl5_2012.html. Accessed March 27, 

2013.

15. Ancel, Ersin; and Shih, Ann T.: The Analysis of the Contribution of Human Factors to the Inflight Loss of Control Accidents, in 12th AIAA Aviation Technology, Integration, and Operations 

(ATIO) Conference: Indianapolis, IN, Sept 17-19, 2012

16. Shih, Ann T.; Ancel, Ersin; and Jones, Sharon M.: Object-Oriented Bayesian Networks 

(OOBN) for Aviation Accident Modeling and Technology Portfolio Impact Assessment, in 

American Society for Engineering Management (ASEM) 33rd International Annual Conference, 

Virginia Beach, VA, Oct 17-20, 2012.

17. Luxhøj, James T.; Shih, Ann T.; Jones, Sharon M.; Ancel, Ersin; and Reveley, Mary: Safety 

Risk Knowledge Elicitation in Support of Aeronautical R&D Portfolio Management: A Case 

Study, in American Society for Engineering Management (ASEM) 33rd International Annual 

Conference, Virginia Beach, VA, Oct 17-20, 2012.

18. Rudisill, Marianne: “Line Pilots' Attitudes About and Experience with Flight Deck 

Automation: Results of an International Survey and Proposed Guidelines”, Proceedings of the 

Eighth International Symposium on Aviation Psychology, Columbus, OH: The Ohio State 

University Press, 1995. 

19. Federal Aviation Administration Human Factors Team: The Interfaces Between Flightcrews

And Modern Flight Deck System, June 18, 1996. 

http://www.faa.gov/training_testing/training/aqp/library/media/interfac.pdf

20. Sarter, N.B.; Woods, D.D.; and Billings, C.E.: “Automation Surprises”, Handbook of Human 

Factors & Ergonomics, second edition, G. Salvendy (Ed.), Wiley, 1997.
REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

2. REPORT TYPE 

Technical Memorandum

 4. TITLE AND SUBTITLE

Identification of Crew-Systems Interactions and Decision Related Trends 

5a. CONTRACT NUMBER

 6. AUTHOR(S)

Jones, Sharon M.; Evans, Joni K.; Reveley, Mary S.; Withrow, Colleen A.; 

Ancel, Ersin; Barr, Lawrence

 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

NASA Langley Research Center

Hampton, VA 23681-2199

 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration

Washington, DC 20546-0001

 8. PERFORMING ORGANIZATION

 REPORT NUMBER

L-20255

10. SPONSOR/MONITOR'S ACRONYM(S)

NASA

13. SUPPLEMENTARY NOTES

12. DISTRIBUTION/AVAILABILITY STATEMENT

Unclassified - Unlimited

Subject Category 03

Availability: NASA CASI (443) 757-5802

19a. NAME OF RESPONSIBLE PERSON

STI Help Desk (email: help@sti.nasa.gov)

14. ABSTRACT

NASA Vehicle System Safety Technology (VSST) project management uses systems analysis to identify key issues and maintain a portfolio of research 

leading to potential solutions to its three identified technical challenges. Statistical data and published safety priority lists from academic, industry and other 

government agencies were reviewed and analyzed by NASA Aviation Safety Program (AvSP) systems analysis personnel to identify issues and future research 

needs related to one of VSST's technical challenges, Crew Decision Making (CDM). The data examined in the study were obtained from the National 

Transportation Safety Board (NTSB) Aviation Accident and Incident Data System, Federal Aviation Administration (FAA) Accident/Incident Data System and 

the NASA Aviation Safety Reporting System (ASRS). In addition, this report contains the results of a review of safety priority lists, information databases and 

other documented references pertaining to aviation crew systems issues and future research needs. The specific sources examined were: Commercial Aviation 

Safety Team (CAST) Safety Enhancements Reserved for Future Implementation (SERFIs), Flight Deck Automation Issues (FDAI) and NTSB Most Wanted 

List and Open Recommendations. Various automation issues taxonomies and priority lists pertaining to human factors, automation and flight design were 

combined to create a list of automation issues related to CDM. 

15. SUBJECT TERMS

Aircraft accidents; Aircraft safety; Flight crews; Human factors; Statistical analysis

18. NUMBER

 OF 

 PAGES

35

19b. TELEPHONE NUMBER (Include area code)

(443) 757-5802

a. REPORT

U

c. THIS PAGE

U

b. ABSTRACT

U

17. LIMITATION OF 

 ABSTRACT

UU

Prescribed by ANSI Std. Z39.18

Standard Form 298 (Rev. 8-98)

3. DATES COVERED (From - To)

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

284848.02.01.07.04 

11. SPONSOR/MONITOR'S REPORT

 NUMBER(S)

NASA/TM-2013-218000

16. SECURITY CLASSIFICATION OF:

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, 

gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 

collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and 

Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person 

shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY)

01- 05 - 2013