Julia Pounds

Civil Aerospace Medical Institute 

Federal Aviation Administration 

Oklahoma City, OK 73125

Anthony S. Ferrante

Federal Aviation Administration 

Air Traffic Investigations Division

Washington, DC 20591

November 2003

Final Report

This document is available to the public

through the National Technical Information

Service, Springfield, Virginia 22161.

 

Office of Aerospace Medicine

Washington, DC 20591

FAA Strategies for Reducing 

Operational Error Causal Factors

DOT/FAA/AM-03/19
NOTICE

This document is disseminated under the sponsorship of

the U.S. Department of Transportation in the interest of

information exchange. The United States Government

assumes no liability for the contents thereof.
i

������������������������������������

��������������� ����������������������������� ����������������������������

������������������

������������������������� ����������������

������������������������������������������������������������� ��������������

����������������

����������������������� ���������������������������������

���������������������������������������

��������������������������������������������� ���������������������������

���������������������������

�

��������������������������������������

����������������

������������������������

�

�������������������������

����������������������������

���������������������

���������������������������������������� ���������������������������������������

�����������������������������

��������������������������������

����������������������������

��������������������� ����������������������������

������������������������

������������������������������������������������������

��������������

��������������������������������������������������������������������������������������������������������������

��������������������������������������������������������������������������������������������������������������

�����������������������������������������������������������������������������������������������������������

�������������������������������������������������������������������������������������������������������������

�������������������������������������������������������������������������������������������������������������

��������������������������������������������������������������������������������������������������������������������

���������������������������������������������������������������������������������

��������������� ����������������������������

�������������������������������������

�������������������������������������

��������������

���������������������������������������������������������

��������������������������������������������

����������������

���������������������������������������� �������������������������������������� ������������� ����������

������������� ������������� ���

����������������� ������� ������������������������������������������

1

FAA STRATEGIES FOR REDUCING OPERATIONAL ERROR CAUSAL FACTORS

Introduction

The US Federal Aviation Administration (FAA) 

oversees the largest, safest, and most complex aviation 

system in the world, relying on a workforce of highly 

trained air traffic control specialists who interact with 

an environment of radar, computers, and communication facilities to maintain the safety and efficiency of 

the system. In fiscal year (FY) 2000 alone, the US air 

traffic system handled 166,669,557 operations. Calculated as a percent of facility activities, the operational 

error (OE) rate per 100,000 activities increased from 

.60 in calendar year (CY) 1999 to .69 in CY00 and .74 

in CY01, then declined by 11% to .66 in CY021 (FAA, 

2003a). Although air traffic declined after the events of 

September 11, 2001, the OE rate reflects the continuing 

need to identify mitigation strategies.

The FAA has historically tried to understand and 

prevent the incidence of OEs. To accomplish this, an 

elaborate and detailed incident reporting system evolved 

to capture causal factors related to their occurrence. The 

data were intended to supply information about where 

to target intervention strategies, with several initiatives 

focusing on one critical component of the system—the air 

traffic control specialist (ATCS). Because the controller is 

the closest person to the air traffic situation and the last 

point of prevention, much attention was logically focused 

on controller performance. Indeed, a controller’s action or 

inaction provides a channel through which pre-existing 

system vulnerabilities can be manifested. As Fisher (2002) 

noted, human error is the mechanism that translates 

this potential for making a mistake into an occurrence. 

Thus, performance of the controller will always be at 

the sharp end of the operational system (Dekker, 2002) 

in the complex and multifaceted environment of ATC. 

Whereas past initiatives focused on remedial training and 

targeted deficient performance, several recent initiatives 

have focused on human performance within, and as it 

interacts with, the larger ATC system, and viewed the 

human element as a fundamental part of this complex 

environment.

Initiatives

In recent years, the FAA Air Traffic Evaluations and 

Investigations Staff began several programs that focused 

more attention on skill building and performance maintenance rather than on remedial training. This approach is 

based on the philosophy of adult education and individual 

responsibility for maintenance of best performance rather 

than viewing training from a directive “schoolhouse” approach, often disparagingly referred to as the “blame and 

train” method. Initiatives included fielding automation 

to re-create traffic situations, developing safety metrics, 

analyzing incident data to identify performance enhancement opportunities, and sponsoring research to further 

develop the capability to identify causal factors.

Incident Re-creation

As computer capabilities increased, the idea that 

computer processing could be harnessed to re-create 

operational errors became a reality. During the 1990s, 

the FAA Civil Aerospace Medical Institute (CAMI) 

collaborated with the FAA Air Traffic Evaluations and 

Investigations staff and Atlanta air route traffic control 

center (ARTCC) to develop automation to graphically 

re-create radar data that were routinely recorded by en 

route air traffic control facilities. 

Referred to as the Systematic Air Traffic Operations 

Research Initiative (SATORI), it was developed, tested, 

and fielded to all en route facilities and regional quality 

assurance offices with the goal of gaining “a better understanding of the interaction between the various elements 

of displayed information, verbal interactions, and the 

control actions taken by air traffic control specialists” 

(Rodgers & Duke, 1993, pg. 1). SATORI is still currently 

in operational use and enables its users to re-create segments of operational traffic in a format similar to what 

was displayed to the ATCS, for example, showing relative 

location and separation, speeds, and headings of aircraft. 

Among other things, SATORI can display full and limited 

data blocks, beacon targets, and conflict alerts. Video and 

audio are synchronized, and the air traffic situation can 

be displayed in four dimensions. 

1

Calculations of rates use 15 decimal places but are rounded to 2 places 

for the table on page 6 of the FAA Administrator’s Fact Book.
2 

At en route facilities, SATORI systems enable the 

facility quality assurance staff to re-create OE situations 

for the controllers involved (see Figure 1).

It is important to note that the system was not intended to be used to “call” an OE; that is, to identify 

when an OE occurred. An OE was first determined to 

have occurred and then SATORI was used to review the 

situation. Systems located at regional and headquarters 

offices enable further incident review. FAA Order 7210.56 

provides guidance for the use of all replay tools. 

As technological advances are made, the ATC system must 

adjust to these changes and ensure that radar reduction tools 

are used correctly and consistently throughout the system 

in order to provide the most accurate re-creation possible 

(FAA Order 7210.56, pg. 1-4).

After an OE is identified (or “called”), SATORI recreations are useful in determining aspects of controller 

and/or pilot performance involved in the event. Re-creations can also be used to review peak periods of traffic 

flow (“pushes”) and the effects of weather on traffic flow. 

Viewing the re-creation also helps to target specific skill 

enhancement programs for those employees involved in 

the event. In addition to helping with the identification 

of performance issues, re-creations of randomly selected 

traffic samples not related to OE situations have also 

been productively used to assess controllers’ technical 

proficiency in relation to training.2

The first of its kind to harness the capability of computers to re-create air traffic situations, SATORI re-creations 

have also been used to provide assistance to other agencies’ investigations of incidents involving aircraft. These 

have included the National Transportation Safety Board 

(NTSB), US Department of Justice, and the National 

Aeronautics and Space Administration (NASA).

In addition to performance management and investigation, re-creations of traffic samples not related to OEs 

have been used in human factors research at CAMI. 

For example, traffic samples were used to study sector 

complexity, controller workload, and performance (Manning et al., 2001; Mills, Pfleiderer, & Manning, 2002). 

TRACON SATORI, a prototype system for the terminal 

environment, was used to examine how controllers use information about aircraft relative position to maintain “the 

picture” of the traffic situation (Pounds, in review).

��������� ���������������������

2

 Technical performance issues consist of areas of knowledge and 

application that might benefit from training. These issues are not 

necessarily areas of deficiency. An employee may demonstrate overall 

acceptable technical proficiency but might benefit from technical 

training in the application of a particular skill or task (FAA Order 

7210.56, pg. 3-1).
3

Used effectively, the capability to re-create traffic situations can bring about beneficial changes in procedures, 

airspace, and future ATC systems. SATORI re-creations 

are currently being used in diverse ways to enhance system performance. It is anticipated that next generation 

re-creation tools currently being developed will continue 

to provide added value. 

Calibrating Incident Severity 

The FAA Evaluations and Investigations Staff monitors the frequency of operational errors to determine the 

system vulnerabilities contributing to each incident so 

that they can be identified and reduced. Once a relevant 

separation standard is violated by ATC, an OE is recorded. Every violation of separation standards provides 

an important opportunity for lessons learned and system 

improvement, although not all operational errors share the 

same characteristics. Separation standards and procedures 

differ depending upon, for example, the type of airspace, 

weather conditions, type of aircraft, and altitude. 

A study conducted by Rodgers and Nye (1993) investigated whether the number of aircraft being worked 

by the controller or the traffic complexity at the time 

of the OE was related to its severity, defined in terms 

of vertical and horizontal separation between aircraft. 

Three categories were created by assigning a maximum 

of ten possible points each to the horizontal and vertical 

separation reported.3 Based on the total point value, OEs 

were partitioned into categories of major (20 points), 

moderate (14-19 points), and minor (13 or less points) 

severity.4

Results of the Rodgers and Nye study were counterintuitive and demonstrated that neither number of aircraft 

nor traffic complexity was significantly related to major, 

moderate or minor OE severity, although the analyses 

revealed ways that the reporting process, and thus the 

resultant data, could be improved. The authors recommended gathering more normative rather than descriptive 

data, increasing the reliability of the reported data, and 

using a re-creation capability to permit investigators to 

review the dynamics of the air traffic situation. 

The U.S. Department of Transportation, Office of 

Inspector General (December 2000) recommended that 

the FAA Air Traffic Investigations Division tackle the 

problem of modeling and defining severity for OEs in 

flight to describe the degree that the applicable separation standard was violated. The purpose was to group 

airborne OEs as low, moderate, or high severity and thus 

be able to focus resources on the most severe events and 

to identify factors related to specific categories of events. 

Data about systemic causes of OEs could then be used 

to more explicitly direct action towards prevention of 

future occurrences. 

The FAA Air Traffic Evaluations and Investigations Staff 

developed a classification system to distinguish between 

OEs on these dimensions. Classification categories were 

developed to reflect the operational environment. In the 

first version of the index, an OE was classified on each 

dimension according to its severity—the extent to which 

separation distance was reduced—as low (39 points and 

below), moderate (40-89 points), or high (90 points 

and above) using a 100-point scale. Objective distances 

were used to minimize subjective interpretations of the 

data. Actual radar data from numerous operational errors 

representing en route facilities nationwide were used to 

test the adequacy of the categories and the classification 

methodology. The components of the model included 

elements associated with loss of standard separation, such 

as the relationship of the aircraft in conflict to one another 

(e.g., converging versus diverging courses), closure rate, 

and level of ATC involvement – whether the event was 

a “controlled” or “uncontrolled” OE. The point distributions for en route radar OEs are shown in Table 1. A 

similar table was developed for OEs in terminal and en 

route airspace with single-site radar.

The initial step in calculating error severity was to 

determine the lateral and vertical proximity between the 

involved aircraft. The horizontal and vertical distances 

were defined as the minimum separation based on the radar 

data just prior to aircraft divergence. Situations with faster 

aircraft closure rates, coupled with converging, opposite 

direction flight paths were assumed to present a greater 

threat than slower closure rates and diverging flight tracks. 

The idea was that head-on encounters with high rates 

of speed, coupled with minimum radar data separation 

distance prior to divergence should account for the greatest 

total point value in the model. An assumption was made 

that aircraft not crossing each other’s paths greatly reduces 

the threat to safety, and consequently, does not receive 

severity index points (see Table 1). In addition, if aircraft 

are not converging, closure rates become less significant 

when the other parameters are properly factored. 

This model does not imply in any way that any minimum separation less than those required by FAA Order 

7110.65 is acceptable from an operational perspective. 

The main purpose of the ATC system is to preserve safety. 

To learn from the occasional shortfall in the ATC system, 

the controller’s action prior to the OE was also included 

in the assessment of the total severity of an event. A “controlled” OE was defined as an OE where the controller 

was aware of the impending conflict and took corrective 

3

 Vertical separation was subdivided depending upon whether the 

incident occurred below or above 29,000 feet (FL290).

4

 Of the 1053 OEs in their sample, only 15 (.01%) were coded as 

“major” severity.
4 

action to increase separation. An “uncontrolled” OE was 

defined as an OE where the controller was unaware of 

the conflict, took no corrective action, and/or became 

aware of the conflict but did not have enough time to 

effectively mitigate the loss of separation. 

The severity model assigns a total of 100 points. Component categories are used to allot point values corresponding with their relative significance during the event. Table 

1 shows the distribution of points possible for terminal 

and en route facilities with single site radar. For example, 

horizontal separation of less than one-half mile in airspace 

requiring at least 3 miles horizontal separation would be 

assigned a point value of 25; closure rate of 700 knots 

and greater would be assigned 10 points. These values 

would be summed with points from the other appropriate categories. The remainder of the 100 points left after 

the assigned points have been subtracted is the “safety 

factor.” Figure 2 shows a hypothetical example of how 

the points in each category of the severity index can be 

used graphically to display the various elements of one 

OE. A different table is used for en route OEs requiring 

5-miles' separation.

This severity index is an algorithmic approach, based 

on objectively measurable and observable variables in 

the operational environment. As the method was further 

developed, the initial three categories were split into four: 

High Severity (A); Moderate Severity, uncontrolled (B); 

Moderate Severity, controlled (C); and Low Severity (D). 

The four category model made it conceptually similar to, 

and potentially confusable with, less objective methods 

being used by other groups to classify incident types such 

as runway incursions.

Data-driven Focus on Performance 

Analysis of OE data by the FAA Office of Investigations revealed several recurring causal factors, for example, 

readback/hearback errors and position relief briefings. As 

a result, interventions were initiated to address them. A 

series of videos was also produced to communicate several 

types of complex system vulnerabilities.

Readback/hearback errors. Readback/hearback errors 

occur if the pilot incorrectly repeats back to the controller the instruction or information just received from the 

controller and if the controller fails to catch the pilot’s 

incorrect response. As an example, a readback/hearback 

error would be noted when the controller instructs “Piper 

123 climb and maintain 1 – 0 thousand,” but the pilot 

repeats back to the controller “Roger. Piper 123 climbing to 1 – 2 thousand.” If the controller does not catch 

the pilot’s error in the response, it would be classified as 

a readback/hearback error. Although these are human 

errors, they do not necessarily lead to an OE. 

���������������������������������������������������������������������������������������������

�������������������� ������� �����������������������

������������������������������

�������

������������������� ��� ����������������� ���

��������������������� ��� ��������������������� ���

��������������������� ��� ���������������������� ���

��������������������� ��� ��������������������� ���

��������������������� �� ����������������������� ��

��������������������� �� ������������������������� ��

������������� ������� ����������������������

���������������������

�������

���������������������� ��� ����������������� ���

����������������������� �� ��������������������� ���

����������������������� �� ���������������������� ���

�������������������� �� ������������������������� ���

������������� ������� ����������������������� ��

������������������������������ ��� ������������������� �������

����������������������������� ��� �������������� ���

������������ ��� ������������������������ ���

��������������������������� �� ��������������������������� ��

�����������������������������������������������������������������������������������������������������

��������������������������������������������������������������������������������������������������
5

Awareness programs targeting readback/hearback errors were developed by headquarters staff as well as by 

regional and facility groups. Facility programs included 

“tape talks” where voice recordings for the ATCS on 

a position were reviewed by the ATCS and/or facility 

staff specialists to assess communication performance. 

The Air Traffic Investigations Division Staff produced a 

video — “Preventing Readback Errors” – highlighting 

how different influences, such as ambient noise and distraction, can contribute to the occurrence of this type of 

error. The video was sent to all facilities as a mandatory 

briefing item. Based on its initial success, this awareness 

program became an annual event with January inaugurated as Hearback – Readback Awareness Month, which 

included constant emphasis on good communications 

skills, random tape monitoring to highlight examples of 

correct phraseology, and positive coaching.

Position relief briefings. Position relief briefings take 

place when one controller assumes (takes over) responsibility for a position from another controller, transferring responsibility from one controller to the other. The 

position relief briefing is a standard operating process 

designed to optimize transfer of responsibility while at 

the same time minimizing the additional workload associated with the task of transferring duty. The relieving 

controller previews the position and then indicates to 

the controller being relieved that the verbal briefing may 

begin. A checklist covers items to be noted prior to the 

relieving controller assuming responsibility for the position. The relieving controller observes the position and 

then the controller being relieved points out any abnormal 

items, traffic situation, and any other issues of concern 

using the checklist. Thorough coverage of the checklist 

items is meant to ensure that the relieving controller 

“has the picture” of the situation. That is, the controller 

being relieved ensures that the relieving controller sees 

all relevant information, understands the situation, and 

is aware of any potential conflicts or problems. After the 

relieving controller assumes responsibility for the position, the controller being relieved observes the overall 

position operation to determine if assistance is needed 

and provide or summon it as appropriate.

Trends in the OE data showed that OEs frequently 

occurred within 10 minutes of a controller taking over a 

position. Figure 3 shows the distribution of 1,056 OEs 

between 1997 and 2000, and the number of OEs that 

occurred within 5, 10, and 20 minutes of the controller 

taking over a position. As a percent of the total number 

of OEs (Figure 4), approximately 9% occurred within 

the first 5 minutes, 18% within the first 10 minutes, and 

35% within the first 20 minutes on position. 

An initiative to address this problem required all 

managers to validate position relief checklists, as well 

as provide a capability for recording the briefing. Shift 

���������

�������������������

�������������

������������

���

������������

�������������

������������

��

�������������

���������

��

�������������

���

���������

����������

����������

���

�����������

�����������

������������

��

���������������

��������

���������

�������������������

�������������

������������

���

������������

�������������

������������

��

�������������

���������

��

�������������

���

���������

����������

����������

���

�����������

�����������

������������

��

���������������

��������

������������������������������������������������������������������������������

������������������������
6 

supervisors were required to ensure the use of a position 

relief briefing checklist and, where available, ensure that 

position relief briefings were recorded. Controllers were 

trained and encouraged to accept position responsibility 

only after they were fully aware of the traffic, and the 

relieving controller and the controller being relieved were 

to establish an appropriate overlap period to complete 

the transfer of responsibilities. Some facilities mandate a 

specific amount of time that the relieved controller shall 

remain at the position. Other facilities adopted this as 

a good practice and permit the overlap period to vary 

depending upon traffic demands. The intent in this latter practice is that basing the overlap period on traffic 

demand reduces the likelihood of potential distractions 

associated with multiple personnel remaining on the same 

operational position unnecessarily. 

Video briefing materials. The Air Traffic Investigations 

Division Staff produced four videotapes to focus awareness 

��

��

��

��

��

��

��

���

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

�

��

��

�

��

�

�

�

�

�

��

��

��

��

���

���

���

���

���

���

��������� ������� ���������� �������� �������� �������� �������� �������� �������� �������� �������� ���������

����������������

��������� ������� �������� �������� ������������

��������������������

���������������

��������

�����������������������������������������������������������������������������

��������������������������������������������������

��

��

���

���

���

���

���

���

���

���

���

��������������������

����

����

��

��

����

��

��

����

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

���

�������������������

�������������������������������������������������������������������������������

��������������������������������������������������
7

on different types of causal factors found to be related 

to OEs. 5 Copies of these were sent to all FAA air traffic 

facilities to be used for briefing materials.

Accidents often occur after a series of inconsequential 

events that create links in a chain during which individuals have had a number of opportunities to intercede and 

break the chain. The “Break the Chain” video illustrates 

how events, if uninterrupted, can culminate in an accident 

and how attending to details can help break the chain of 

events and prevent accidents and incidents. The episode 

portrayed in the video is fictional but was compiled from 

actual events. A small business-owned aircraft is carrying 

a group of business executives to a meeting. The flight 

departs later than expected due to the passengers’ late 

arrival and delays in air traffic services. The original plan 

was to depart ahead of developing stormy weather; however, the delay and the quickly developing storm result 

in moderate icing conditions. An aircraft mechanic, the 

passengers, both pilots, an airway facility staff specialist, 

and multiple air traffic control specialists (both terminal 

and en route) all either contributed in some way to the 

chain of events or failed to interrupt the sequence when 

they had the opportunity. 

“Collision Course: What are the odds?” depicts an 

actual event, demonstrating how rare and improbable 

events can and do occur. Individually, each event would 

be fairly benign. However, as part of a chain of events, 

seemingly inconsequential errors can set in motion a series 

of events that cannot be undone. Outwardly insignificant 

errors committed, for the most part, by controllers in two 

air route traffic control centers, put two large jet aircraft 

on a converging course and that information could not 

be relayed from the ground to the aircraft because radio 

communications were lost. The video demonstrates 

how the failure to accomplish routine procedures such 

as switching an aircraft to another frequency, becoming 

momentarily distracted, or being complacent about an 

evolving situation can potentially result in large and unexpected adverse outcomes, although a midair collision 

was avoided in the event depicted.

“Consequences of Simple Omissions” is a compilation 

of actual events. This video illustrates examples of how 

small omitted actions, lack of attention, failure to follow operational practices, compounded by poor facility 

practices, and lack of self discipline or professionalism 

resulted in incidents ranging from operational errors to 

fatal accidents. The video advocates adhering to individual professionalism and remembering the importance 

of maintaining standards and accuracy for safety. 

The video titled “Preventing Readback Errors,” mentioned earlier, focused on communication strategies to 

reduce misunderstandings between controllers and pilots. Strategies include reducing the complexity of each 

communication and chunking information within the 

communication to facilitate understanding. Information 

is presented in a humorous manner to illustrate strategies 

to overcome common blocks to good communication.

Identifying OE Causal Factors 

The need for a formal reporting process was recognized 

early in the FAA’s evolution and by 1965, FAA Order 

8020 had established the ATS System Error Reporting 

Program. Early on, recommendations were developed 

regarding the conduct of incident investigations and the 

use of the resulting information, many of which were 

incorporated into the Order (O’Connor & Pearson, 

1965). For instance, O’Connor and Pearson suggested 

that any reporting system should be based on a system 

view, including controller performance, the influence of 

personal capacities and skills, the design and operation 

of the system, and modifiers of the working environment 

such as supervision, operating procedures, health, morale, 

and work schedules. Notably, recommendations made 

by O’Connor and Pearson remain as relevant today as 

they were then.

Air traffic control has become an increasingly complex 

system involving men and equipment in a continuous and 

dynamic decision-making function. Future developments 

point to the rising use of complex equipment, including 

high-speed computers, as aircraft speeds and the system’s 

load continue to rise. By projecting current trends, it can 

be anticipated that future system changes involving equipment, personnel, and/or procedures point to the need for 

longer and longer lead times as complexity grows. The 

above considerations point to the need for close scrutiny 

of the ongoing system failures and/or incidents in order to 

provide the most accurate feedback information for system 

correction or modification (pg 1).

Additionally, O’Connor and Pearson recommended 

an approach to system error evaluations, asserting that 

“the man-machine system will never be perfectly reliable, 

efficient, and error free because of the inherent limitations and idiosyncrasies of the human component” (pg. 

1) and that a deliberate and objective process would be 

better than “shooting in the dark for sources of error and 

possible solutions” (pg. 1). 

Later, other changes were periodically made to the 

incident reporting form. For example, in 1997 a version 

was fielded having three additional causal factors under 

the category of Inappropriate use of Displayed Data. This 

category was associated with the controller’s use of the 

5

 Ms. Christine Soucy, a safety investigator with AAT-200 wrote and 

helped direct the videos for this program.
8 

radar display and situation awareness associated with 

use of the radar data. The three added items—failure to 

detect displayed data, failure to comprehend displayed data, 

and failure to project future status of displayed data—were 

included to address the controller’s situation awareness. 

This change proved initially successful, with subsequent 

OE data showing that use of the Inappropriate use of 

Displayed Data - Other category decreased and the data 

that would otherwise have been attributed to this Other

category distributed across these three new categories 

(Rodgers, personal communication). Although this 

change brought finer detail to the description of the 

OE, the issue of reporting reliability raised by Rodgers 

and Nye (1993) was still an issue.

Currently, the FAA Air Traffic Investigations Division 

oversees and coordinates the OE reporting process governed by the FAA Air Traffic Quality Assurance Order 

7210.56 (FAA, 2002). The order “provides specific guidance on investigation, reporting, and recording types of 

incidents that impact the quality of air traffic services” 

(pg. i). Specifically, section 5-1-2 (pg. 5-2) stipulates: 

5-1-2. SUSPECTED EVENT 

a. In order to maintain an effective Air Traffic System, 

it is imperative that we identify all deficiencies within our 

system and take appropriate corrective actions necessary to 

fix any associated problems. Operational errors and deviations are reported for just that reason, so those problems 

(either systemic or individual) can be corrected to enhance 

system integrity. The identification of operational errors 

and deviations without fear of reprisal is an absolute requirement and is the responsibility of all of us who work 

within our system.

b. Accordingly, it remains Air Traffic Policy that any 

employee who is aware of any occurrence that may be an 

operational error, deviation, or air traffic incident (as defined 

in paragraph 4-1-1, Definitions), immediately report the 

occurrence to any available supervisor, controller-in-charge 

(CIC) or management official.

To develop information so that data-driven decisions 

about causal factors and intervention strategies could 

be made, the Office of Investigations determined that a 

method for identifying causal factors related to human 

performance was needed—a method that viewed human 

performance as one among several potential points of 

system vulnerability. Resulting information about human 

factors could then be proactively used to mitigate the 

potential for future incidents. This effort was responsive 

to goals in the FAA’s 1999 Strategic Plan, including the 

goal to “eliminate accidents and incidents caused by 

human error” (FAA, 1999). The FAA’s National Aviation Research Plan for 1999 also echoed the intended 

outcome of developing enhanced measures of human 

performance and increased understanding of factors that 

lead to performance decrements. 

An initial effort was undertaken in coordination with 

the FAA’s Civil Aerospace Medical Institute (CAMI) to 

determine whether retrospective analysis of existing 

OE reports could extract additional useful information 

by using a human factors approach (Pounds & Isaac, 

2001; Pounds & Scarborough, 2002). The retrospective 

analyses relied on data from existing OE reports that 

were based on standardized procedures specified by FAA 

Order 7210.56. Outcomes from this work suggested that 

standard reporting procedures did not require facilities to 

collect the type of data necessary to perform a thorough 

human factors analysis. That is, although the report forms 

captured descriptive data about the OE, little information 

was collected about events and causal factors preceding 

and during the loss of separation. 

As these retrospective analyses were being completed, 

the FAA Office of Aviation Research entered into a collaborative agreement with Eurocontrol and signed Action Plan 12 (AP12): Management and Reduction of 

Human Error in ATM. The initial goal of this project 

was to examine two existing techniques for identifying 

human error and to determine whether they could be 

harmonized into one technique.6 If so, the FAA and 

Eurocontrol member states would be able to use the 

technique retrospectively to examine existing incident 

reports for information related to human factors trends 

in the data, and to leverage this information to develop 

and share mitigation strategies. 

As AP12 activities progressed, ATC subject matter 

experts from both FAA and Eurocontrol judged that the 

harmonized technique – JANUS – also showed promise 

as a supplement to existing reporting processes (Pounds 

& Isaac, 2001). That is, rather than serving merely as a 

retrospective data mining tool, the new technique might 

also have value if integrated directly into existing OE 

reporting processes. Based on this hypothesis and after 

successful harmonization (Pounds & Isaac, 2001), the 

course of AP12 was modified to include further refinement and testing of the technique. Validation activities 

posed unique challenges to both the FAA and Eurocontrol. 

Based on discussions of these differences, it was decided 

6

 The two techniques used for the harmonization were the Human 

Factors Analysis and Classification System (HFACS; Shappell & 

Wiegmann, 2000) and the model for Human Error in Air Traffic 

Management (HERA; EATMP, 2003). The Human Factors Analysis 

and Classification System (HFACS), a human factors taxonomy for 

analysis of existing aviation accident databases, was originally developed 

for the US Navy to investigate military aviation accidents. HFACS 

is currently being used by the FAA to analyze civil aviation accident 

databases. HERA is a model of human error for air traffic control 

that was developed as a tool to increase the effectiveness of human 

error identification. 
9

that parallel and complementary approaches be used 

based on the particular requirements of each to conduct 

the validation activities. Thus the FAA and Eurocontrol 

developed the technique to reflect their individual system 

needs, resulting in two structurally parallel techniques.

The FAA method is a structured interview process that 

leads the analyst through a series of questions designed 

to identify the mental error and the contextual conditions surrounding it. Questions related to the controller’s 

perceptions, memory, and decision-making processes that 

lead to execution of a plan are included. The method of 

using the technique considers the dynamic ATC traffic 

environment and treats each OE as a potential chain of 

events (or human errors) that result in the final loss of 

separation.

This line of work expanded on efforts by the Air 

Traffic Evaluations and Investigations Staff to improve 

causal factor information. For an inexperienced analyst 

of an OE, the technique is a potential aid to ask the right 

questions for eliciting causal factors and relevant human 

factors. For an experienced analyst, the technique ensures 

that he or she considers a broad range of causal factors 

rather than relying on experience and focusing on “the 

usual suspects.” Figure 5 illustrates the general categories 

of contextual conditions included in the technique. 

A research project to test and validate the JANUS 

technique was conducted in collaboration with the Air 

Traffic Evaluations and Investigations Staff and the National Air Traffic Controllers Association. A parallel project 

was conducted by Eurocontrol. The research proposed to 

answer several basic questions related to validity: Does 

the technique work? How well does it work? Is it better than the current method? Is it ready for operational 

implementation? Will the results from the technique help 

to improve safety management? 

Validation activities also posed unique challenges to 

both the FAA and Eurocontrol. Based on discussions 

of the organizational differences, it was decided that a 

harmonized approach could be defined that would allow 

the other organization to leverage the potentially complementary work, findings, and lessons learned while also 

meeting the particular requirements of each organization. 

Methodologies for testing the technique were adapted 

to accommodate the respective organizations’ testing 

environments, including use of operational resources 

and test constraints. 

Results from the validation activities conducted to 

date by both organizations suggest that information provided by the JANUS technique will help improve safety 

management through the more effective identification of 

������� ������������������������� ������� ������������������������� �������������� ��������������

������������������ ������������������

�����������������

��������������

�����������������

��������������

������������� �������������

������� �������

�������������

������

�������������

������������

���������

������������

���������

���������������� ����������������

����������� �����������

����������������

��������������

����������������

��������������

�����������

���������� ����������

������������

��������������� ��������������� �����������

����������

�����������

������������

������������������������������������������������������������������������������

���������������������������
10 

human factors associated with OEs. However, a definitive 

answer can only be scientifically determined by a longitudinal analysis of data gathered using this technique. The 

information can then be evaluated for its usefulness as 

strategies are identified with which to mitigate the potential for future operational errors. Indeed, the ancient 

Roman symbol of Janus, after which the harmonized 

technique was named, was attributed with the ability to 

look back so that an understanding of past events could 

lead to insight about future events. 

Keeping The Mental Edge

The National Air Traffic Professionalism (NATPRO) 

project is an example of how information identified by 

OE analysis can be turned into strategy and skill enhancement.7

 NATPRO is a new training approach sponsored by 

the FAA Air Traffic Investigations Division. Rather than 

relying solely on knowledge-based training, this approach 

integrates the concept of “performance coaching,” using 

an awareness seminar coupled with a practicum.

Facility personnel first participate in the coaching 

clinic developed by Breedlove (2003) that covers the 

knowledge and critical skills of effective coaches. The 

coaches then conduct the seminar and practicum at their 

facility. The coaches help participants understand how 

the seminar concepts relate to performance by pointing 

out the connection between the seminar material and 

the practicum experience when participants practice the 

activities. For example, the coach might try to distract the 

participant during practice to demonstrate the influence 

of environmental distractions on performance. 

The initial NATPRO program is focused on cognitive 

skills related primarily to visual attention, such as detection of information, focusing on relevant information 

and multitasking. The practicum includes activities to 

exercise the mind and improve concentration through 

distributed practice. The basic concept is not only limited to concentration. The NATPRO program concept is 

designed to also include other ATC skills such as auditory 

attention, decision making, and planning. 

The seminar on cognitive skills was developed to increase an individual’s understanding of the mental process 

of concentration, factors that affect mental processes, and 

how mental skills relate to performance. Although the 

skills are generic, we hope that by improving general skills, 

individuals will demonstrate a corresponding improvement in controller performance. Once armed with the 

knowledge provided by the seminar, participants then 

experience their own strengths and limitations during 

the Practicum. Interactive web-based computerized skill 

challenges permit participants to gain insight about their 

own skills. By testing themselves against the computer 

and experiencing how performance can vary in relation 

to factors such as distraction, fatigue, boredom, and so 

on, participants gain increased understanding of their 

own performance and identify strategies to improve it. 

Putting the challenges in a web-based application gives 

individuals and teams the opportunity to compete for 

high scores, should they desire to do so. Although the 

competitive aspect is available to participants, it is not 

required. It is included to enhance the experience for 

those participants who want to engage in interpersonal 

competition.

Admittedly, the successful transfer of skills from this 

program to actual air traffic control performance will be 

difficult to measure objectively. There are several measures 

that may be partially influenced as a result of this training, 

including a reduction in the number of operational errors; 

however, because the air traffic system is complex, there 

are many factors not related to controller preparation and 

awareness that also affect the number of operational errors. Other less tangible measures may include an increase 

in efficiency and higher job satisfaction. Success of the 

program could be captured by evaluating data from several 

sources. Initially, we plan to compare the practicum skill 

levels and subjective evaluations from participants before 

and after participating in the program. However, whether 

skill enhancement realized in the NATPRO program will 

translate to working traffic cannot be objectively evaluated 

at this time. Measurement of performance using high-fidelity air traffic simulations or medium fidelity standalone 

simulation tools would supply more substantive evidence 

for evaluating the effectiveness of the program.

If this type of program is successful, it would demonstrate a more cost-effective, personally rewarding delivery 

of training. The NATPRO approach may improve air 

traffic safety and efficiency by increasing the controller’s 

attention and perception skills, sustaining a highly skilled 

work force. Although targeting ATC performance, the 

skills themselves are generic. Participants would be expected to also benefit by transfer of skill to other activities 

beyond their professional ATC work environment. 

Conclusion

A philosophy of individual leadership combined with 

joint accountability guides these efforts and others, all 

part of the FAA’s ongoing efforts to maintain and enhance 

aviation safety by identifying system vulnerabilities and 

developing mitigation strategies. Air traffic control is a 

high-demand, high-consequence activity, and controllers 

7

 The NATPRO project was developed by Mr. Randall Breedlove 

with assistance from Mr. Jimmy Mills (ASO-150) and the Air Traffic 

Southern Region Learning Council. 
11

must rely on their mental skills to successfully orchestrate 

air traffic. The initiatives described here reflect a humancentered approach but situates the human as the crux of 

a larger, more complex, dynamic system of other people 

and computers in a fast-paced environment of radar 

information, communications, and aircraft movement. 

Although any single initiative may not be a solution in 

itself—no silver bullet that will immediately eliminate all 

human error and every operational error—mitigation and 

reduction is a journey of many single steps that progress in 

the same direction toward prevention of these events.

References

Breedlove, R. (2003). NATPRO. Creating the cognitive 

advantage for high performance ATC professionals. 

Unpublished materials.

Dekker, S. (2002). The field guide to human error investigations. Aldershot, UK: Ashgate. 

EATMP. (2003). Short report on human performance 

models and taxonomies of human error in ATM 

(HERA). HRS/HSP-002-REP-02. Brussels: EUROCONTROL.

Federal Aviation Administration. (1999). National Aviation Research Plan. Washington, DC: Author.

Federal Aviation Administration. (1999). Strategic Plan. 

Washington, DC: Author.

Federal Aviation Administration (2002). Air Traffic Quality Assurance (FAA Order 7210.56). Washington, 

DC: Author.

Federal Aviation Administration. (2003a). Administrator’s 

Fact Book. Washington, DC: Author.

Fisher, B. (2002), Study looking at runway incursions identifies contributing factors and recommends solutions. 

ICAO Journal, 57(1), pp. 13, 27-28.

Manning, C.A., Mills, S.H., Fox, C., Pfleiderer, E., & Mogilka, 

H.J. (2001). Investigating the validity of performance and 

objective workload evaluation research (POWER). Washington, DC: Federal Aviation Administration Office of 

Aviation Medicine. DOT/FAA/AM-01/10.8

Mills, S.H., Pfleiderer, E.M. & Manning, C.A. (2002). 

POWER: Objective Activity and Taskload Assessment in 

En Route Air Traffic Control. Washington, DC: Federal 

Aviation Administration Office of Aviation Medicine. 

DOT/FAA/AM-02/2.8

O’Connor, W.F., & Pearson, R.G. (1965). ATC System Error 

and Controller Proficiency. Washington, DC: Federal 

Aviation Agency Office of Aviation Medicine.

Pounds, J. (in review). Using static and relational information 

to maintain the “picture” in ATC.

Pounds, J., & Isaac, A. (2001). Preliminary Test of Two 

Models for Human Factors Analysis of Operational 

Errors. Proceedings of the 11th International Symposium on Aviation Psychology, Columbus, Ohio, 

pp. 1-5. [CD-ROM]. 

Pounds, J., & Isaac, A. (2001). Development of an FAAEUROCONTROL approach to the analysis of human 

error in ATM. Washington, DC: Federal Aviation 

Administration Office of Aviation Medicine. DOT/

FAA/AM-02/12.8

Pounds, J., & Scarborough, A. (2002). Retrospective human factors analysis of US runway incursions. Paper 

presented at ICAO NAM/CAR/SAM Runway 

Safety / Incursion Conference. Mexico City.

Rodgers, M. D. & Duke, D. A. (1993). SATORI: Situation 

assessment through the re-creation of incidents. Washington, DC: Federal Aviation Administration Office 

of Aviation Medicine. DOT/FAA/AM-93/12.8

Rodgers, M.D., Mogford, R.H., & Strauch, B. (2000). 

Post-hoc analysis of situation awareness in ATC 

incidents and major aircraft accidents. In M.R. 

Endsley & D.J. Garland, (Eds.), Measurement and 

analysis of situation awareness in complex systems. 

Mahwah, NJ: Erlbaum.

Rodgers, M.D. & Nye, L.G. (1993), ‘Factors associated 

with the severity of operational errors at air route 

traffic control centers.’ In M.D. Rodgers (Ed.), 

An examination of the operational error database for 

air route traffic control centers. Washington, DC: 

Federal Aviation Administration Office of Aviation 

Medicine. DOT/FAA/AM-93/22.8

Shappell, S.A., & Wiegmann, D.A. (2000), The Human 

Factors Analysis and Classification System–HFACS. 

Washington, DC: Federal Aviation Administration 

Office of Aviation Medicine. DOT/FAA/AAM00/07.

8

8 This publication and all Office of Aerospace Medicine technical 

reports are available in full-text from the Civil Aerospace Medical 

Institute’s publications Web site: http://www.cami.jccbi.gov/aam400A/index.html