Examining ATC 

Operational Errors 

Using the Human 

Factors Analysis and 

Classification System

Alfretia Scarborough

Larry Bailey

Julia Pounds

Civil Aerospace Medical Institute

Federal Aviation Administration

Oklahoma City, OK 73125

December 2005

Final Report

DOT/FAA/AM-05/25

Offi ce of Aerospace Medicine

Washington, DC 20591
NOTICE

This document is disseminated under the sponsorship of

the U.S. Department of Transportation in the interest of

information exchange. The United States Government

assumes no liability for the contents thereof.
i

Technical Report Documentation Page 

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No. 

DOT/FAA/AM-05/25 

4. Title and Subtitle 5. Report Date 

Examining ATC Operational Errors Using the Human Factors Analysis and December 2005 

Classification System 6. Performing Organization Code 

 

7. Author(s) 8. Performing Organization Report No. 

Scarborough A, Bailey L, Pounds J 

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS) 

FAA Civil Aerospace Medical Institute 

P.O. Box 25082 11. Contract or Grant No. 

Oklahoma City, OK 73125 

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered 

Office of Aerospace Medicine 

Federal Aviation Administration 

800 Independence Ave., S.W. 

Washington, DC 20591 14. Sponsoring Agency Code 

15. Supplemental Notes 

Work was accomplished under approved task AM-B-06-HRR-524 

16. Abstract 

In the literature of aviation accidents and incidents, human error has been recognized as the predominant factor 

contributing to aviation mishaps. Consequently, a number of human error models and taxonomies have been 

adapted to study the unique characteristics of flying an aircraft. However, relatively few attempts have been made 

to apply the same tools toward understanding the human factors causes of air traffic control (ATC) operational 

errors (OEs). An operational error is an occurrence attributable to an element of the air traffic system in which 

aircraft separation minima are not maintained. As a first attempt to systematically examine the underlying 

human causes of OEs, we report on the results of a study that consisted of three phases: (1) conducting a 

literature review to identify candidate error models and taxonomies, (2) selecting an appropriate error model or 

taxonomy for use in the ATC environment, and (3) applying the selected error model, or taxonomy, to a subset 

of the items identified by the FAA as OE causal factors. The results of our study revealed that, of the models and 

taxonomies examined, the Human Factors Analysis and Classification System (HFACS) was the taxonomy most 

readily adapted for use in an initial examination of ATC OEs. Causal factor items from 5,011 OE reports were 

classified using the HFACS taxonomy. Most items were classified as decision errors and skill-based errors. 

Additional research is needed to develop a more comprehensive understanding of the factors that contribute to 

ATC decision errors and skill-based errors.

17. Key Words 18. Distribution Statement 

Air Traffic Control, Operational Errors, Human Error Document is available to the public through the 

Defense Technical Information Center, Ft. Belvior, VA 

22060; and the National Technical Information 

Service, Springfield, VA 22161 

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price 

Unclassified Unclassified 35 

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized 

iii

ACKNOWLEDGMENTS

The authors thank Carol Manning, Ph.D., for her patience as this paper evolved and Scott Shappell, Ph.D., 

for his unrelenting support for this manuscript. Our air traffi c experts, Don Ellis and Roger Becker, were 

invaluable sources of ATC information. Without their subject-matter expertise and their patience this report 

could not have been written. Kathy Wade and Roni Anderson, the Civil Aerospace Medical Institute’s great 

library staff, were very patient with us during the literature search process. Finally, the authors appreciate the 

improvements to this manuscript suggested by our other reviewers.

1

EXAMINING ATC OPERATIONAL ERRORS USING

THE HUMAN FACTORS ANALYSIS AND CLASSIFICATION SYSTEM

INTRODUCTION

The Federal Aviation Administration (FAA) is charged 

with maintaining the National Airspace System (NAS)—a 

vast and increasingly complex transportation system. 

Highly skilled air traffi c control specialists (ATCSs) 

interact with a matrix of radars, computers, and communication systems to ensure the safe and effi cient 

operation of aircraft. ATCSs follow established rules 

and procedures to separate aircraft; when separation is 

not maintained an operational error (OE) is recorded. 

OEs are defi ned as:

An occurrence attributable to an element of the air traffic system in which: (1) less than the applicable separation 

minima results between two or more aircraft, or between 

an aircraft and terrain or obstacles (e. g. , operations below 

minimum vectoring altitude (MVA); equipment / personnel on runways), as required by FAA Order 7110. 65 or 

other national directive; or (2) an aircraft lands or departs 

on a runway closed to aircraft operations after receiving 

air traffic authorization. (3) an aircraft lands or departs on 

a runway closed to aircraft operations, at an uncontrolled 

airport and it was determined that a NOTAM regarding 

the runway closure was not issued to the pilot as required 

(pg. 5-1, FAA Order 7210. 56). 

This report summarizes the fi ndings of a study to examine the underlying human causes associated with ATC 

OEs. The study involved three phases. First, a literature 

search was conducted to identify aviation-related human 

error models and taxonomies. Second, candidate error 

models and taxonomies were evaluated to determine their 

relative strengths and weaknesses regarding their application to ATC OEs. Third, the selected candidate was used 

to examine the items reported as causal factors on archival 

OE reports. We hoped that this method would help to 

clarify the underlying human errors in OEs. 

This study supports the FAA’s National Aviation Research Plan for developing enhanced measures of human 

performance and increasing the understanding of factors 

that lead to performance decrements (FAA, 1999a). 

It also supports the FAA’s Strategic Plan to eliminate 

accidents and incidents caused by human error (FAA, 

1999b). In addition, this study is responsive to the Safety 

Strategic Objective identifi ed in the DOT Strategic Plan 

for 2003-2008 (DOT, 2003) and the FAA’s Flight Plan 

for 2004-2008 (FAA, 2003) to reduce operational errors 

and runway incursions. 

BACKGROUND

The FAA’s air traffi c control (ATC) safety program relies 

on the timely and accurate recording and transmission of 

data about incidents in the U. S. airspace that “adversely 

affect the capabilities of ATC facilities to provide safe, 

orderly, and expeditious movement of air traffi c” (FAA 

Form 7210-3, 2002, p. 4-1). To achieve these results, 

the FAA established a means for documenting these 

data. For instance, when established separation standards 

are not maintained, OEs are reported to the FAA’s Air 

Traffi c Service Offi ce of Evaluations and Investigations 

(AAT-20). The existing process for investigating human 

performance factors associated with OEs involves a voluminous amount of descriptive data but provides little 

information about the underlying causes and remedies 

(Pounds & Scarborough, 2000). 

We conducted a literature review to identify human 

error models and taxonomies that captured important 

elements of the cognitive processes of the human operator, 

the actions of the operator, and the environment in which 

the operator performs. These three domains extended from 

the earlier work of Lewin’s (1951) fi eld theory in which a 

person’s behavior (i. e. , task performance) was postulated 

to be a function of the qualities of both the person and 

environment in which the behavior occurred. 

In conducting the literature review, it became clear that 

the use of the term “model” and “taxonomy” sometimes 

were used interchangeably. However, for the purpose 

of this review, a taxonomy is defi ned as a classifi cation 

system that organizes data into meaningful categories 

(Kirwan, 1992). In contrast to a taxonomy, a model not 

only describes the categories within the system, but it also 

indicates the manner by which the various components are 

affected by each other (Shorrock, Kirwan, Isaac, Anderson, 

& Bove, 1999). While a taxonomy has descriptive power, 

a model has both descriptive and predictive power. This 

distinction will be used later when evaluating the strength 

and limitations of a given model or taxonomy. 

Review of Human Error Models and Taxonomies

Our literature review was focused on identifying aviation-related human error models and taxonomies that 

could be applied to the ATC environment. A search of 

six aviation technical report databases was conducted to 

identify the relevant literature. The six databases included: 

(1) the European Organization for the Safety of Air 

Navigation Report Database, (2) the Aviation Research 
2

Laboratory Institute of Aviation Report Database, (3) 

the W. J. Hughes Technical Center Reports Database, (4) 

the Civil Aerospace Medical Institute (CAMI) Technical 

Report Database, (5) the National Aeronautics and Space 

Administration (NASA) Technical Report Database, and 

(6) the National Transportation Safety Board (NTSB) 

Report Database. From these sources, articles and book 

chapters were reviewed to determine if a given human 

error model or taxonomy was described in suffi cient 

detail so that an independent analysis could be made on 

the relative strengths and limitations associated with an 

operator’s: (1) cognitive process, (2) task-related behaviors, 

and (3) environmental conditions (i.e., organizational 

context). 

Ten candidate taxonomies/models were selected, 

based on the completeness with which they addressed 

each of the three categories listed above (See Table 1). 

The authors rated the ten candidates on eight dimensions adapted from Kirwan (1992) that were designed 

for evaluating human error taxonomies (See Table 2). 

The eight dimensions included: comprehensiveness, 

accuracy, consistency, theoretical validity, auditability, 

resource usage, utility, and acceptability. Each of the 

dimensions and the corresponding rating scale that we 

developed are described in the following section. It should 

be mentioned that although the work of Kirwan (1992) 

was used as a guide, considerable adjustments were made 

to the defi nitions and rating scales to refl ect the specifi c 

needs of this project. 

Evaluating Human Error Models and Taxonomies

Comprehensiveness refers to the extent to which a content 

domain is sampled. In this project, the three domains of 

interest refl ected an operator’s: (1) cognitive process, (2) 

task-related behaviors, and (3) environmental conditions, 

such as organizational context and external factors (e.g., 

government regulations). A three-point rating scale was 

derived for judging the degree to which each of the three 

domains was adequately sampled by a model or taxonomy. 

Since the authors were not experts in the fi eld of human 

error theory, they relied on the rationale presented in the 

source documents for justifying the domain sampling. 

A value of “1” meant that a rationale was presented for 

only one dimension (e.g., person), a value of “2” meant 

that a rationale was presented for two dimensions (e.g., 

person and task), and a value of “3” meant that a rationale 

was presented for all three dimensions (i.e., person, task, 

and environment). 

Accuracy assesses the degree to which critical operator 

errors are captured by the error model or taxonomy. Since 

error reduction was the goal of this program of research, 

it is important that the criteria by which error reduction are judged are clearly identifi ed. Three factors were 

selected for rating accuracy: the identifi cation of system 

errors (such as those detected at the level of the National 

Airspace System), the identifi cation of operator errors 

(such as the errors committed by a given ATCS), and an 

empirical validation that the operator errors affected the 

system errors (such as how the number of ATC OEs affect 

the system outcomes of safety effi ciency and/or effectiveness). A three-point rating scale was employed to indicate 

the presence of one, two, or all three factors. 

Consistency refers to the degree to which different users 

of the model or taxonomy arrive at the same results when 

evaluating the same material (i.e., accident reports). In 

statistical terms consistency as used here is associated with 

inter-rater agreement. However, before inter-rater agreement can be assessed, it is important that the categories of 

the model or taxonomy are unambiguous and mutually 

exclusive. Once again, a three-point hierarchical rating 

scheme was employed to assess consistency: A “1” was assigned when mutually exclusive categories were defi ned, a 

“2” was assigned when an empirical measure of inter-rater 

agreement was provided for a single study, and a “3” was 

assigned when empirical measures of inter-rater agreement 

were provided across multiple studies using different raters. 

It should be noted that since this dimension is associated 

with the calculation of a statistic, all things being equal, 

the error model or taxonomy with the highest measure 

of inter-rater agreement would be chosen. 

Theoretical validity, as used here, addresses the three 

phases of theory development: description, prediction, 

and explanation (Kaplan, 1964). In the description phase, 

the phenomenon of interest is defi ned in a way that allows 

measurement. Given that error taxonomies are developed 

for the purpose of identifying error types and tracking 

changes across time, taxonomies tend to reside within the 

descriptive phase of theory development. The prediction 

phase is associated with identifying important variables 

that predict the criterion of interest. This requires that 

both the predictor and criterion are operationally relevant. 

Given that models propose a casual sequence, they are 

well suited for a test of their predictive validity. 

The last phase of theory development deals with 

the power of a theory to explain why the phenomenon 

occurs (i.e., an empirical validation of the causal chain 

of events). However, instead of using just words, when 

an error occurs the theory can be used to explain both 

the necessary and suffi cient conditions surrounding the 

event. By necessary, we mean that when the elements of 

the predictors (i.e., hypothesized causes) are absent, the 

criterion of interest -errors- is also absent. By suffi cient, 

we mean that when the elements of the predictors are 

present, the criterion of interest is also present (Swartz, 

1997). Since it is diffi cult to manipulate variables in 

an operational environment, theoretical validity at the 
3

Table 1. Error Models and Taxonomies. 

Name Source Description 

Error Taxonomy 

 Human Factors 

 Analysis and 

 Classification System 

 (HFACS) 

Wiegmann & Shappell (2003) Identifies and organizes latent errors 

using a hierarchical structure involving 

organizational influences, unsafe 

supervisory actions, preconditions for 

unsafe acts, and unsafe acts. 

 Violation Taxonomy Mason (1997) Identifies the main organizational 

factors which might promote violations, 

and management strategies that could 

help to eliminate or reduce these factors 

by addressing the motives behind them. 

 A Human Error 

 Taxonomy based on 

 Cognitive Engineering 

 & Social & Occupational

 Psychology 

Bagnara et al. (1991) Identifies four categories of conditions 

affecting the state of a human system: 

human performances, decision making, 

socio-organizational conditions, and 

external situations. 

 Latent Error Model of 

 Accident Causation 

Reason (1990) Identifies four fundamental elements of 

all organizations that must work 

together harmoniously if efficient and 

safe operations are to occur: corporate 

decision makers, line management, 

psychological precursors of unsafe acts, 

and unsafe acts. Within each of the 

elements are latent (hidden) and active 

failures that represent vulnerability 

points for the creation of unsafe acts. 

 Taxonomy for 

 Describing Human 

 Malfunctions 

Rasmussen (1986) Identifies factors that shape 

performance, the contextual factors 

associated with error, characteristics of 

the task being performed, and the 

classification of error modes. 

 Pyramid Model Isaac (1995) Classifies human error within an air 

traffic management environment, 

composed of three levels: the top level 

(representing individual factors), the 

middle level (representing task 

characteristics), and the bottom level 

(representing the organizational 

influences). 

(Continued)
4

Table 1. Continued

Name Source Description 

Error Model 

 Aviation Safety and 

 Human Reliability 

 Analysis Method 

 (ASHRAM) 

Miller and Forester (2000) Allows aviation researchers to 

analyze aviation mishaps that 

involve human errors in ways 

that account for the operational 

context, crew expectations, 

training, airframe-related 

human-system interfaces, and 

crew resource management. 

A Technique for Human Error 

Analysis (ATHEANA) 

Cooper et al. (1996) Performs a human reliability 

analysis in the context of 

probabilistic risk assessment. 

ATHEANA is based on an 

understanding of why humansystem interaction failures 

occur as opposed to behavioral 

and phenomenological 

description of operator 

responses.

A Technique for Human Error 

Assessment (THEA) 

Pocock, Wright, & Harrison 

(1999) 

Used by interactive system 

designers and engineers to help 

anticipate human-machine 

interaction failures. The 

technique employs a cognitive 

error analysis based on an 

underlying model of human 

information processing. 

A Dynamic Reliability Technique 

for Error Assessment in ManMachine System (DREAMS) 

Cacciabue (1993) Identifies the origin of human 

errors in the dynamic 

interaction of the operator and 

the plant control system. 

Human error probabilities are 

combined with the probabilities of system failures in 

order to obtain an overall 

probabilistic safety assessment 

(PSA) for the whole plant. 
5

Table 2. Ratings of Error Taxonomies and Models Name Source Com.* Acc. Con. Th.V. Aud. Res. Utility Accept. Error Taxonomy  HFACS Wiegmann & Shappell 

(2003)

3 2 3 1 3 2 2 3 

 Violation Taxonomy Mason (1997) 2 2 1 1 2 2 2 2  A Human Error  Taxonomy based on  Cognitive Engineering  & Social &Occupational  Psychology 

Bagnara, et al. (1991) 2 2 1 1 2 2 2 2 

 Latent Error Model of  Accident Causation 

Reason (1990) 3 2 1 1 2 2 2 3 

 Taxonomy for  Describing Human  Malfunctions 

Rasmussen (1986) 2 2 1 1 2 2 2 3 

 Pyramid Model Isaac (1995) 3 2 1 1 2 2 2 1 Error Model  ASHRAM Miller and Forester (2000) 3 2 1 2 3 1 2 3  ATHEANA Cooper, et al. (1996) 3 2 1 2 3 1 2 2  THEA Pocock, Wright, & Harrison 

(1999)

2 2 1 2 2 1 2 2 

 DREAMS Cacciabue (1993) 2 2 1 2 2 1 2 1 

* Com = Comprehensiveness , Acc. = Accuracy, Con. = Consistency, Th.V. = Theoretical Validity, Aud. = Auditability  Res. = Resource Usage, Utility = Utility, Accept. = Acceptability. 
6

explanation phase is seldom achieved in total. Using a 

three-point hierarchical scale, a “1” was assigned when 

the defi nitions were empirically measured, a “2” was assigned when predictive power was demonstrated, and a 

“3” was assigned when a causal sequence was tested for 

necessary and suffi cient conditions. 

Auditability refers to the ease by which another person 

can illustrate and discuss the rationale behind the results 

of the classifi cations or model predictions. It is important 

to note that the ratings associated with auditability were 

dependent on the information reported in the available 

literature and do not necessarily refl ect the ratings that 

would be achieved if the authors had access to the “user” 

manuals. Once again, a 3-point hierarchical scale was used 

to assess this dimension. A rating of “1” was assigned to 

sources that provided just a conceptual view of an error 

model or taxonomy. A rating of “2” was assigned to 

sources that provided steps or checklists that should be 

followed when using a specifi c error model or taxonomy. 

A rating of “3” was assigned to sources that not only 

provided checklists but also incorporated examples from 

case histories to illustrate the kinds of judgments required 

of the user of a given model or taxonomy. 

Resource Usage is defi ned by the amount of time, 

money, and human resources necessary to develop and 

maintain the infrastructure to support the error model or 

taxonomy. Since none of the material reviewed provided 

quantifi able information on resource usage, a three-point 

subjective scale was developed based on how much effort 

would be required to convert the FAA OE database so 

that it would be compatible with a given error model or 

taxonomy. The three levels included: large amount of 

adaptation required (rating of “1”), moderate amount 

(rating of “2”), and minimal (rating of “3”). 

Utility is based on the needs of the user. In this case, 

the needs of the user are two-fold, to identify and reduce 

the number of OEs. Utility differs from auditability in 

that the latter is concerned with being able to document 

what was done, and the former is concerned with whether 

what was done helped to reduce errors. A three-point 

scale was developed based on the ability of an error taxonomy or model to specify error reduction measures. A 

rating of “1” was given when errors were identifi ed, but 

no guidance was provided for reducing them. A rating 

of “2” was used when general error-reduction guidelines 

were presented. Finally, a rating of “3” was used when 

guidelines were provided for reducing specifi c classes or 

types of errors. 

Acceptability depends on how referent groups have 

judged the value of a given model or taxonomy. The two 

referent groups of interest in this study are academicians 

and safety practitioners. Aspects of both referent groups 

were integrated into a three-point hierarchical scale. A 

rating of “1” was given if the error model or taxonomy 

received a favorable review in the scientifi c literature about 

human error. A rating of “2” was granted if in addition to 

receiving a favorable review, the error model or taxonomy 

was also successfully implemented in an error reduction 

program within a command and control environment 

(e.g., a nuclear power plant or a military operations control 

center). If the command and control environment was 

aviation related, the rating was increased to a “3.” 

Using the above rating scheme, 4 error models and 6 

error taxonomies were rated by the fi rst two authors, as 

shown in Table 2. The numerical ratings associated with 

the 8 dimensions refl ected the consensus of the authors. 

Although an argument could be made about any given 

rating, the overall trend revealed the following:

1. There was little if any variability in the ratings associated with: accuracy, consistency, auditability, and 

utility. Thus, each of the 10 articles included:

a. System and operator error identifi cation. 

b. Definitions of mutually exclusive categories 

(HFACS also included measured inter-rater reliability). 

c. User outline/checklist describing the steps to follow 

for implementing an error model or taxonomy. 

d. General error reducing guidelines. 

2. Rating variability for theoretical validity and resource 

usage was explained by the type of technique used (i.e., 

error model vs. error taxonomy). 

a. All the error models had higher theoretical validity ratings when compared with all the error 

taxonomies. It should be noted, however, that the 

higher ratings for the error models was due to their 

potential for testing predictive validity rather than 

empirically demonstrating the predictive power of 

a given technique. 

b. All the error taxonomies were rated as requiring 

fewer resources to implement as compared to all 

the error models. Again, the source documents did 

not provide empirical data for direct comparisons. 

That being the case, the ratings refl ected our best 

judgments based on the available materials. 

3. The dimensions of completeness and acceptability 

were the only two dimensions whose rating variability 

proved to be source specifi c. That being the case, these 

two dimensions provided the most useful information with regard to selecting a given error model or 

taxonomy for use in a study of the underlying human 

factor causes of OEs. 

a. HFACS and Reason’s Latent Error Model were 

the two highest-rated error taxonomies. 

b. ASHRAM emerged as the highest-rated error 

model. 
7

Given that error models and taxonomies differed 

on their relative strengths and limitations, the authors 

decided to select one candidate from each method for 

further examination. HFACS was selected to represent 

the error taxonomies and ASHRAM was selected to 

represent the error models. HFACS was selected not 

only because of the results of Table 2 but also because 

its framework incorporates the work of Reason (1990) 

and Rasmussen (1986), both of which were highly rated 

taxonomies. Similarly, ASHRAM was selected because 

it adapted the ATHEANA model (also highly rated) for 

use in the aviation industry. A synopsis of both HFACS 

and ASHRAM is presented next. 

The Human Factor Analysis and Classifi cation 

System (HFACS)

The Human Factors Analysis and Classifi cation System 

(HFACS; Wiegmann & Shappell, 2003) classifi es human errors and the causal factors associated with them. 

Originally developed for aviation accidents and mishaps, 

HFACS has been adapted to other domains, such as 

mechanical and medical errors (ibid). HFACS has strong 

linkages to Reason’s (1990) model of latent and active 

errors (see Figure 1) as they relate to organizational infl uences, unsafe supervision, pre-conditions for unsafe acts 

of operators (i.e., factors that affect an individual’s mental 

and physical behavior), and the unsafe acts themselves. 

Active errors are related to the performance of operators 

in complex systems that have an immediate impact on 

the system, in contrast to latent (or hidden) errors of 

designers and managers. Whereas the active errors of the 

operator attract the most attention during an accident 

investigation, Reason (1990) noted that the latent errors 

of the organization, supervisor, and the preconditions of 

unsafe acts, may pose the greatest risk to system safety. 

This is because latent errors, unless they are discovered, 

will continue to remain in the system even after rectifying 

errors committed by an individual operator. 

Figure 1 illustrates the concept of latent errors. Each 

of the four layers has “holes” (i.e., latent errors), which 

represent potential defi cits or vulnerabilities in system 

safety. Under normal conditions, the holes are out of 

alignment, which means a defi cit in one layer of the 

system is compensated by some protection in another 

layer. For example, when a pilot catches the mistake of 

a co-pilot or an aircraft mechanic. However, when the 

holes are in alignment, it means that a defi cit in one 

layer of the system is exacerbated by a defi cit in another 

layer. An example of multiple latent errors in alignment 

is a situation where a supervisor discovers that a pilot is 

not following the Federal Aviation Regulations (FARs) 

or standard operating procedures (SOPs) and elects not 

to do anything about it (i.e., Unsafe Supervision). The 

pilot continues to follow bad practices (i. e. , Precondition) until one day he commits several “Unsafe Acts” by 

fl ying under visual fl ight rules (VFR) into instrument 

meteorological conditions (IMC), that results in a controlled-fl ight-into-terrain (CFIT) accident. 

The causal sequence depicted by the arrow in Figure 1 

suggests that an unsafe act is the fi nal failure in a series of 

defi cits that begins at the organizational level. However, 

when conducting accident investigations, the starting point 

begins with the unsafe act committed by the operator. By 

asking “why” an unsafe act was committed, investigators 

can follow the accident chain of events through each of 

the four levels and uncover the direct and indirect infl uences on an operator’s actions. As one might imagine, the 

discovery of latent errors requires diligent search. 

Figure 2 shows how HFACS extended Reason’s model 

by systematically identifying the vulnerability points 

(i.e., latent errors) at each level in the system, and then 

demonstrating how this analysis provided additional information about the human factor causes associated with 

aviation accidents. (See Wiegmann & Shappell, 2003, for 

a thorough description of the causal categories associated 

with each error level.) For instance, organizational infl uences are vulnerable to errors associated with resource 

management, organizational climate, and organizational 

processes. Resource management includes the management of human resources, fi nancial allocations, equipment, 

and facilities. Organizational climate is broken down by 

its management structure, policies and procedures, and 

the underlying culture. Organizational processes are the 

operations, procedures, and safety oversight that exist 

within the organization. 

A more complete illustration of HFACS is shown in 

Figure 3. 

Error vulnerabilities at the supervisory level manifest 

themselves in the form of unsafe supervision. For example, 

when supervisors fail to provide guidance or accurate 

information to the workforce in a timely fashion, correct 

known problems, or willfully allow personnel to violate 

safety-related rules and regulations, this creates a potentially 

hazardous environment for the operation. 

Preconditions of unsafe acts include environmental 

factors, personal factors, and the physical, mental, and 

physiological condition of the operator. Examples of 

the latter include physical fatigue, mental fatigue, and 

effects of over-the-counter medications. Whereas environmental factors represent those forces outside of the 

individual that affect performance (such as the weather 

and cockpit displays), personal factors refer to events that 

are brought on by the operator. These events include the 

use or misuse of crew resources and factors that relate to 

personal readiness, such as staying up late the night before 

an early departure. 
8

Organizational

Influences

Unsafe

Supervision

Preconditions

For

Unsafe Acts

Unsafe

Acts

Figure 1. Reason’s Latent Error Model of Accident Causation 

Organizational

Influences

Unsafe

Supervision

Preconditions

For

Unsafe Acts

Unsafe

Acts

Errors

Violations

Environmental Factors

Personal Factors

Conditions of Operator

Inadequate Supervision

Planned Inappropriate Operations

Failure to Correct Known Problem

Supervisory Violations

Resource Management

Organizational Climate

Organizational Process

Figure 2. The Human Factors Analysis and Classification System (HFACS) 

Source: Adapted from Wiegmann & Shappell (2003) 
9

©Shappell & Wiegmann, 2002

Figure 3. The Human Factors Analysis and Classification System (HFACS)

Errors

UNSAFE

ACTS

Perceptual

Errors

Skill-Based

Errors

Decision

Errors Routine Exceptional

Violations

Inadequate

Supervision

Planned

Inappropriate

Operations

Failed to

Correct

Problem

Supervisory

Violations

UNSAFE

SUPERVISION

Resource

Management

Organizational

Climate

Organizational

Process

ORGANIZATIONAL

INFLUENCES

PRECONDITIONS

FOR

UNSAFE ACTS

Condition of 

Operators

Physical/

Mental

Limitations

Adverse 

Mental 

States

Technological 

Environment

Physical 

Environment

Personal 

Readiness

Crew Resource 

Management

Personnel 

Factors

Adverse 

Physiological 

States

Environmental 

Factors

Errors

UNSAFE

ACTS

Perceptual

Errors

Skill-Based

Errors

Decision

Errors Routine Exceptional

Violations

Inadequate

Supervision

Planned

Inappropriate

Operations

Failed to

Correct

Problem

Supervisory

Violations

Inadequate

Supervision

Planned

Inappropriate

Operations

Failed to

Correct

Problem

Supervisory

Violations

UNSAFE

SUPERVISION

Resource

Management

Organizational

Climate

Organizational

Process

Resource

Management

Organizational

Climate

Organizational

Process

ORGANIZATIONAL

INFLUENCES

PRECONDITIONS

FOR

UNSAFE ACTS

Condition of 

Operators

Physical/

Mental

Limitations

Adverse 

Mental 

States

Technological 

Environment

Physical 

Environment

Personal 

Readiness

Crew Resource 

Management

Personnel 

Factors

Adverse 

Physiological 

States

Environmental 

Factors
10

Unsafe acts are operator actions involved in producing 

an error. The erroneous actions are classifi ed as either 

unintentional errors or willful violations. The subdivisions 

of unintentional errors are an adaptation of Rasmussen’s 

Skills, Rules and Knowledge (SRK) error model (Rasmussen, 1986). The subdivisions include decision errors, 

skilled-based errors, and perceptual errors. In contrast to 

errors, violations are willful deviations from known rules 

and procedures. A more comprehensive overview of the 

HFACS categories appears in Appendix A. 

Aviation Safety and Human Reliability Analysis 

Method (ASHRAM)

The Aviation Safety and Human Reliability Analysis 

Method (ASHRAM) developed by Miller and Forester 

(2000) belongs to a class of probability risk assessment 

models (PRA) that incorporates the reliability of human 

performance, commonly referred to as a Human Reliability Assessment (HRA). ASHRAM is an extension of 

A Technique for Human Error Analysis (ATHEANA) 

originally developed for the U. S. Nuclear Regulatory 

Commission (NRC). In their report on the development 

of ASHRAM, the authors noted that, although there 

were similarities between the nuclear power and aviation 

industries, there were enough differences to warrant the 

development of a HRA method unique to aviation. 

The reported similarities between the nuclear power 

and aviation industry included:

1. Highly technological systems

2. High consequences of failure

3. Very few signifi cant failure events

4. Governmental regulation of hardware and operations

5. Small, highly qualifi ed crew in control

6. Simulators used in crew training

7. Dependence upon displays for information about the 

environment. 

Differences between the two industries are shown in 

Table 3. Appendix B provides additional background on 

PRAs, HRAs, and the development of ATHEANA. 

As shown in Figure 4, ASHRAM is a causal model 

of a pilot’s cognitive processes, which consists of the 

error forcing context, error mechanisms, and unsafe or 

contributory actions. The Error Forcing Context (EFC) 

is divided into two categories, those that infl uence the 

pilot and those that infl uence the aircraft. Both types of 

infl uences serve as input to the pilot’s environmental perceptions. Pilot EFCs consist of six channels of input: (1) 

information from maps and manuals, (2) environmental 

cues (i. e. , things happening outside of the aircraft) such 

as traffi c (TF) and weather (WX), (3) data link (wireless transfer of information such as the listing of pilot 

reports [PIREPs] and electronic communications from 

air traffi c control, ATC), (4) displays and controls (fl ight 

and systems instrumentation, control yoke, trim wheels, 

switches and cockpit computer input devices), (5) radio 

communications from ATC, other planes, and intercom 

with other crew members, and (6) verbal communications 

with other crew members. The design of the aircraft and 

the laws of physics as well as the control actions of the 

pilot determine aircraft EFCs. 

As with ATHEANA, the basic cognitive model detailing the mechanisms of pilot error is based on the work of 

Woods, Johannesen, Cook, & Sarter (1994), Woods & 

Paterson (2000), Roth, Mumaw, & Lewis (1994), Mumaw 

and Roth (1992), and Reason (1990). The model consists 

of three classes of cognitive function that are interactive 

and non-sequential: (1) Environmental Perception, (2) 

Reasoning and Decision Making, and (3) Action. These 

error producing mechanisms are further augmented by 

operator conditions such as stress and fatigue and the 

training received concerning aircraft design and operating procedures. 

Environmental perceptions consist of perceptual 

processes, attention, detection, recognition, monitoring, 

interpretation of environmental cues, and the overall 

understanding of the state of the aircraft/environmental 

system. Although the elements that comprise environmental perceptions are similar to the elements comprising 

what is commonly referred to as “situation awareness” 

(SA), Miller and Forester (2000) note that there is enough 

controversy over the defi nition of SA that it became 

necessary to avoid that term. For more information 

about SA and the controversy surrounding it, the reader 

is referred to Endsley (1995), Flach (1995), and Sarter 

and Woods (1991). 

Reasoning and decision making, as defi ned by Miller 

and Forester (2000), is the cognitive or thinking process 

that includes awareness and deduction of unsafe or dangerous conditions, remembering situation-specifi c training, 

deciding to follow recommended procedures, planning 

fl ight navigation, diagnosis of trouble symptoms, deciding 

how to respond to situations, problem solving, and novel 

or creative use of existing tools or symptoms. 

The pilot’s actions are the control inputs to the airframe, 

operation of control hardware in the cockpit, communications to crew and passengers, and any other overt 

physical behaviors. As shown in Figure 4, these actions 

may produce unsafe or contributory actions. 

A pilot’s unsafe actions are the overt acts of commission or omission that are taken by the pilot and/or crewmembers that lead to a degradation in safety. However, 

Miller and Forester (2000) emphasize that the individuals 

involved may not know they are committing an unsafe 

act, and instead may be performing actions that make 

sense to them at the time. The latter then are referred to 
11

Table 3. Summary of the Major Differences Between Nuclear Power and Aviation as Reported in Miller, 

D., & Forester, J. (2000). 

Topics Nuclear Power Commercial Aviation 

Licensing agency NRC Federal Aviation Administration 

Potential accident consequences Extremely High; thousands of 

lives 

Very High; hundreds of lives 

Incentives to operate 

w/inadequate safety 

Power grid needs, profits of 

utility 

Meet schedule, passenger 

frustration, airline profits 

Reports of errors and near 

misses—human error 

probabilities (HEPs) available 

The industry has developed an 

HEP databank, called 

NUCLARS, but participation has 

been minimal 

There are several databases of 

near misses and accidents, some 

are privately owned by airlines, 

others are public—no known 

banks include HEPs 

Contact with help in emergencies Shift Technical Advisor, Incident 

team at Emergency Operations 

Center, Technical Evaluation 

Center 

Radio to radar centers, tower, 

airlines, manufacturer 

Normal operations Continuous, mostly supervisory 

control, with periods of direct, 

manual control 

Each flight is a discrete event and 

is dependent on crew for 

initiating and orchestrating. 

During cruise supervisory control 

is used. 

Minimum elements needed for 

mission: critical functions 

Fuel, cooling, pressure control, 

power conversion systems, crew, 

safety systems 

Flight controls, thrust, cabin 

pressure or supplemental oxygen, 

navigation information, pilot, 

communication w/destination, 

flyable weather 

Physical inertia High, with a few notable 

exceptions, such as large loss-ofcoolant accidents (LOCAs), 

changes in physics take place 

slowly—minutes and hours 

Low, changes happen rapidly—

seconds and minutes 

Speed of system response Relatively slowly, except large 

LOCA – slow feedback from 

inputs 

Relatively fast—rapid feedback 

from inputs 

Feedback from system to control 

inputs 

Remote reports and 

instrumentation; mostly discrete 

readouts, but some integrated 

displays, mixture 

“Seat-of-pants”, real-time visual, 

aural, kinesthetic, also 

instrumentation; mostly discrete 

readouts, but some 

 of electromechanical and 

electronic

integrated displays, mixture of 

electromechanical and electronic 

Emergency operation written 

guidance 

Written, symptom-based 

procedures 

“Manual decision-making,” and 

checklists for most critical flight 

operations 

Accident sequences A few major decisions and 

actions can cover several hours 

Many decisions and actions can 

cover only a few minutes 

Transient conditions Difficult to integrate all info to 

construct valid mental model 

Easier to construct mental model 

from discrete displays 

Activation of emergency 

subsystems 

Automatic, in most cases, with 

crew notification 

Pilots are in the loop—get 

warning displayed and have to 

initiate safety system response 
12

Environmental

Perception

Reasoning &

Decision/Making Displays Action Controls Aircraft

Condition

Radio Comm.

Crew Comm.

Radio Comm.

Crew Comm.

Data Link

Envir. Cues

Maps/Manuals

Weather

Traffic

Physics.

Design Procedures

Training

Design Factors and

Procedural Factors

Operator Factors

Stress, fatigue, skills, knowledge, 

Tendencies & rules

Error-Forcing Context

Error Mechanisms Unsafe or Contributory

Actions

Figure 4. ASHRAM (Miller & Forester, 2000) 

as contributory actions. For example, the assumption that 

an aircraft was running out of fuel (due to the erroneous 

belief that both right and left tanks had been used) may 

lead to a forced landing on inappropriate terrain. 

Selecting and Testing the Candidate

As can be seen from the previous discussions, both 

HFACS and ASHRAM are well suited for the study of 

pilot errors. Thus, the authors chose between the two 

techniques based on which one (1) could be more readily 

used for the study of OEs and (2) would be best suited for 

a retrospective analysis of archival OE data. Both HFACS 

and ASHRAM employ retrospective analysis techniques 

in determining the underlying causes of human factors 

errors among pilots. In fact, a number of the elements 

associated with ASHRAM’s error mechanisms shown in 

Figure 4 have a parallel in HFACS due to the common 

infl uence of Reason’s (1990) model of human error. 

In the end, the choice was based on the time it would 

take to develop materials from each of the two techniques 

for use in our study. Of the two candidates, we determined 

that HFACS could be used “as is.” Thus, the HFACS 

taxonomy was selected. Later, in the discussion section, 

we will return to the issue of how HFACS and ASHRAM 

might be integrated so that an analysis could be conducted 

to assess risk factors associated with OEs. 

METHOD

Three steps were used to study the underlying human 

factors causes of ATC OEs. 

First, two coders were trained to use of the HFACS 

methodology. Second, items from the FAA’s Air Traffi c 

Control Operational Error/Deviation Report Form 7210-

3 (FAA, 2002) were mapped onto HFACS categories. 

Third, once a reliable mapping was completed, the 

results were used to describe the pattern of human factors causes of ATC OEs reported for the fi ve-year period 

from 1998–2002. 

Coders

Two ATC SMEs were selected for their experience 

with operations in both terminal and en route facilities. 

Each SME had more than 15 years of combined ATC 

experience in these types of facilities. In addition, both 

SMEs possessed specialized knowledge about the air traffi c control quality assurance process and experience in 

identifying the causal factors associated with OEs. Both 

SMEs had job-related experience using Form 7210-3 while 

working at ATC facilities. They also obtained HFACS 

experience in a previous study requiring that they use 

HFACS subcategories to classify narrative information 

from ATC OEs. 
13

Materials

Operational Error/Deviation Report (Form 7210-3). 

Block 68 of Form 7210-3 lists potential OE causal factors 

(See Appendix C). The causal factors are divided into 6 

major categories: (1) Data Posting, (2) Radar Display, (3) 

Aircraft Observation (Towers only), (4) Communication 

Error, (5) Coordination, and (6) Position Relief Briefi ng. 

Each main category is further divided into clusters of items 

that provide greater specifi cation about the causal factor 

category. Each cluster also has an item labeled “other” 

but this item was not used in this study. Overall, a total 

of 48 items from Block 68 were used. 

The Human Factors Analysis and Classifi cation System 

(HFACS). Coders were provided copies of the defi nitions 

for each HFACS category and subcategories as shown in 

Appendix A. HFACS categories used for the classifi cation task included unsafe operator acts, preconditions 

for unsafe acts, unsafe supervision, and organizational 

infl uences. 

Procedures

The procedures were divided into three phases, a training phase for the SME coders, a mapping phase, and a 

classifi cation phase conducted by the researchers. 

Training Phase

Each SME received a blank copy of the causal factors 

listed on FAA Form 7210-3 in Block 68 (see Appendix 

C) and the HFACS categories and defi nitions (see Appendix A). After reviewing the materials the SMEs were 

asked if they needed further clarifi cation. The SMEs 

were then provided with separate offi ce space and asked 

to associate each item in Block 68 of Form 7210-3 with 

an appropriate HFACS category. Once they completed 

their tasks they were brought together to discuss any 

problems they encountered.The SMEs were provided with 

an opportunity to change their results before returning 

all material to the researchers. 

Mapping Phase

One week after training, the SMEs returned to repeat 

the same task they accomplished during the training 

phase. For this activity the SMEs worked independently 

and were not permitted to discuss their results until they 

were turned in to the researcher. These procedures were 

necessary to calculate a measure of inter-rater agreement 

so that comparisons could be made with results reported 

in Wiegmann & Shappell (2003). After the researchers 

had collected the materials, any differences between the 

two SMEs were discussed until a consensus was reached 

as to the “right answer.” These results were then used to 

conduct the classifi cation phase. 

Classifi cation Phase

The authors constructed a Block 68 by HFACS frequency matrix using a sample of convenience consisting 

of 10,754 causal factors extracted from 5,011 FAA OE 

reports for the period 1998 to 2002. The frequencies 

associated with each category listed in Block 68 of Form 

7210-3 were then associated with the appropriate HFACS 

category as identifi ed in the mapping phase. 

RESULTS

Matching Phase

Table 4 shows the results of the matching phase during 

which the two SME coders associated each of the 48 items 

from Block 68 of Form 7210-3 with a corresponding 

HFACS category. Coeffi cient Kappa was computed to 

test the extent to which the two SMEs agreed on their 

associations. The resulting kappa of .96 indicated a high 

level of agreement (Fleiss, 1981). This value is somewhat 

higher than those reported by Wiegmann and Shappell 

(2003). In those studies kappa ranged from a low of .60 

to a high of .95. 

Of the 48 Block 68 items, 62% (n=29) were associated 

with the HFACS subcategory of skill-based errors and 

38% (n=18) were associated with the HFACS subcategory 

of decision errors. It is interesting to note that the SMEs 

were unable to fi nd any Block 68 items that corresponded 

to preconditions for unsafe acts, unsafe supervision, or 

organizational infl uences. This was because there was no 

place on Form 7210-3 to record this information. 

Table 4 also demonstrates that certain item categories 

from Block 68 were more often associated with a given 

HFACS subcategory than others. For example, most of 

the items comprising the data-posting category were associated with skill-based errors. In contrast, for the tower 

observations and position debriefi ngs categories, the items 

were predominately associated with decision errors. Items 

in the radar display and coordination clusters had a mixed 

association with skill-based and decision errors. 

Classifi cation Phase

Of the 10,754 causal factors extracted from 5,011 OE 

reports, 1350 (13%) causal factors were listed as “other.” 

Since no additional information was provided as to what 

“other” might be, this information could not be classifi ed. 

Thus these data were dropped from further analyses. The 

remaining 9,404 causal factors were categorized according 

to the mapping framework identifi ed by the SMEs in the 

matching task. The results are displayed in Table 5. 

Table 5 is a Block 68 by HFACS category matrix in 

which row totals are the total number of causal factors 

associated with a given Block 68 category, and column 
14

(Continued) 

Table 4. Form 7210-3 Causal Factors and Associated HFACS Subcategories 

Causal Factors Associate HFACS 

Subcategories 

Data Posting Category 

 Computer Entry Cluster 

 Incorrect input Skill-Based Error 

 Incorrect update Skill-Based Error 

 Premature termination of data Decision Error 

 Input/Update not made Skill-Based Error 

 Flight Progress Strip Cluster 

 Not updated Skill-Based Error 

 Interpreted incorrectly Skill-Based Error 

 Posted incorrectly Skill-Based Error 

 Updated correctly Skill-Based Error 

 Premature removal Skill-Based Error 

Radar Display Category 

 Misidentification Cluster 

 Failure to reidentify aircraft when the accepted target 

 identity becomes questionable 

Skill-Based Error 

 Overlapping data blocks Skill-Based Error 

 Acceptance of incomplete or difficult to correlate 

 position information 

Decision Error 

 Inappropriate Use of Displayed Data Cluster 

 MODE C Decision Error 

 BRITE Decision Error 

 Conflict alert Decision Error 

 Failure to detect displayed data Skill-Based Error 

 Failure to comprehend displayed data Skill-Based Error 

 Failure to project future status of displayed data Skill-Based Error 

Aircraft Observation (Towers Only) Category 

 Actual Observation of Aircraft Decision Error 

 Improper Use of Visual Data Cluster 

 Landing Decision Error 

 Taking Off Decision Error 

 Ground Operation Decision Error 

 Taxiing across runway Decision Error 

 Holding in position for takeoff Decision Error 

 Phraseology Skill-Based Error 

 Transposition Skill-Based Error 

 Misunderstanding Skill-Based Error 

 Read-back Cluster 

 Altitude Skill-Based Error 

 Clearance Skill-Based Error 

 Identification Skill-Based Error 

 Other 

 Acknowledgment Skill-Based Error 
15

Table 4. Continued 

Communication Error Category 

Associate HFACS 

Subcategories 

Coordination Category 

 Area of incident Cluster 

 Intra-sector/position Skill-Based Error 

 Inter-sector/position Skill-Based Error 

 Inter-facility Skill-Based Error 

 Facility type 

 Failure to utilize/comply with precoordination information Skill-Based Error 

 Improper use of Information 

 Exchanged in Coordination Cluster 

 Aircraft identification Decision Error 

 Altitude/Flight level Decision Error 

 Route of flight Decision Error 

 Speeds Decision Error 

 APREQs Decision Error 

 Special instructions Decision Error 

 Failure to Coordinate between Ground and 

 Local Control Cluster 

 Crossing active runway Skill-Based Error 

 Vehicle, equipment, or personnel on active runway Skill-Based Error 

 Use of other than active runway for arrival 

 and departures 

Skill-Based Error 

 Runway closure Skill-Based Error 

Position Relief Briefing Category 

 Employee did not use position relief checklist Decision Error 

 Employee being relieved gave incomplete briefing Skill-Based Error 

 Relieving employee did not make use of pertinent data 

 exchanged at briefing 

Decision Error 
16

Table 5. Frequencies of OE Causes by HFACS Subcategories* 

HFACS Categories

OE Causes V PE SBE DE PR CRM PML APS AMS SV FC PIO IS OP OC RM 

Row

Total

Data Posting 0 0 556 27 0 0 0 0 0 0 0 0 0 0 0 0 583 Computer Entry 0 0 167 27 0 0 0 0 0 0 0 0 0 0 0 0 194 Incorrect input 0 0 69 0 0 0 0 0 0 0 0 0 0 0 0 0 69 Incorrect update 0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 0 32 Premature  termination date 0 0 0 27 0 0 0 0 0 0 0 0 0 0 0 0 27 Input/Update not  made 0 0 66 0 0 0 0 0 0 0 0 0 0 0 0 0 66 Other 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 NA Flight Progress Strip 0 0 389 0 0 0 0 0 0 0 0 0 0 0 0 0 389 Not updated 0 0 107 0 0 0 0 0 0 0 0 0 0 0 0 0 107 Interpreted  incorrectly 0 0 186 0 0 0 0 0 0 0 0 0 0 0 0 0 186 Posted incorrectly 0 0 41 0 0 0 0 0 0 0 0 0 0 0 0 0 41 Updated  incorrectly 0 0 44 0 0 0 0 0 0 0 0 0 0 0 0 0 44 Premature  removal 0 0 11 0 0 0 0 0 0 0 0 0 0 0 0 0 11

Other 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 NA

Radar Display 0 0 5260 250 0 0 0 0 0 0 0 0 0 0 0 0 5510

Misidentification 0 0 212 27 0 0 0 0 0 0 0 0 0 0 0 0 239

Failure to 

 reidentify aircraft 0 0 22 0 0 0 0 0 0 0 0 0 0 0 0 0 22

Overlapping data 

 blocks 0 0 190 0 0 0 0 0 0 0 0 0 0 0 0 0 190

Acceptance of

 incomplete  information 0 0 0 27 0 0 0 0 0 0 0 0 0 0 0 0 27

Other 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Inappropriate Use of 

 Displayed Data 0 0 5048 223 0 0 0 0 0 0 0 0 0 0 0 0 5271

MODE C 0 0 0 121 0 0 0 0 0 0 0 0 0 0 0 0 121

 BRITE 0 0 0 44 0 0 0 0 0 0 0 0 0 0 0 0 44 Conflict alert 0 0 0 58 0 0 0 0 0 0 0 0 0 0 0 0 58

Failure to detect 0 0 1447 0 0 0 0 0 0 0 0 0 0 0 0 0 1447Failure to 

 comprehend 0 0 1205 0 0 0 0 0 0 0 0 0 0 0 0 0 1205

Failure to project 

 future status 0 0 2396 0 0 0 0 0 0 0 0 0 0 0 0 0 2396

 Other 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 NAAircraft Observation (Towers Only) 0 0 0 721 0 0 0 0 0 0 0 0 0 0 0 0 721

Actual Observation 

 of Aircraft 0 0 0 277 0 0 0 0 0 0 0 0 0 0 0 0 277

Improper Use of 

 Visual Data 0 0 0 444 0 0 0 0 0 0 0 0 0 0 0 0 444

Landing 0 0 0 141 0 0 0 0 0 0 0 0 0 0 0 0 141
17

Table 5 Continued. 

OE Causes V PE SBE DE PR CRM PML APS AMS SV FC PIO IS OP OC RM 

Row

Total

 Taking Off 0 0 0 173 0 0 0 0 0 0 0 0 0 0 0 0 173 Ground Operation 0 0 0 130 0 0 0 0 0 0 0 0 0 0 0 0 130 Taxing across  runway 0 0 0 85 0 0 0 0 0 0 0 0 0 0 0 0 85 Holding for  takeoff 0 0 0 45 0 0 0 0 0 0 0 0 0 0 0 0 45 Other 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 NACommunication Error 0 0 1486 0 0 0 0 0 0 0 0 0 0 0 0 0 1486 Phraseology 0 0 141 0 0 0 0 0 0 0 0 0 0 0 0 0 141 Transposition 0 0 223 0 0 0 0 0 0 0 0 0 0 0 0 0 223 Misunderstanding 0 0 242 0 0 0 0 0 0 0 0 0 0 0 0 0 242 Readback 0 0 698 0 0 0 0 0 0 0 0 0 0 0 0 0 698 Altitude 0 0 382 0 0 0 0 0 0 0 0 0 0 0 0 0 382 Clearance 0 0 155 0 0 0 0 0 0 0 0 0 0 0 0 0 155 Identification 0 0 161 0 0 0 0 0 0 0 0 0 0 0 0 0 161 Other 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 NA Acknowledgment 0 0 182 0 0 0 0 0 0 0 0 0 0 0 0 0 182 Other 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 NACoordination 0 0 741 220 0 0 0 0 0 0 0 0 0 0 0 0 961 Area of incident 0 0 457 0 0 0 0 0 0 0 0 0 0 0 0 0 457 Intra-sector /  position 0 0 227 0 0 0 0 0 0 0 0 0 0 0 0 0 227 Inter-sector /  position 0 0 230 0 0 0 0 0 0 0 0 0 0 0 0 0 230 Inter-facility 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Facility type 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 NA Failure to utilize /  comply with  pre-coordination 0 0 185 0 0 0 0 0 0 0 0 0 0 0 0 0 185 Improper use of  information 0 0 0 220 0 0 0 0 0 0 0 0 0 0 0 0 220 Aircraft ID 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 28 Altitude/Flight  level 0 0 0 87 0 0 0 0 0 0 0 0 0 0 0 0 87 Route of flight 0 0 0 46 0 0 0 0 0 0 0 0 0 0 0 0 46 Speeds 0 0 0 7 0 0 0 0 0 0 0 0 0 0 0 0 7 APREQS 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 24 Special  Instructions 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 28 Other 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 NA Ground and Local  Control Failure 0 0 99 0 0 0 0 0 0 0 0 0 0 0 0 0 99 Crossing active  runway 0 0 44 0 0 0 0 0 0 0 0 0 0 0 0 0 44 Vehicle or  personnel on  active runway 0 0 25 0 0 0 0 0 0 0 0 0 0 0 0 0 25
18

Table 5 Continued. 

OE Causes V PE SBE DE PR CRM PML APS AMS SV FC PIO IS OP OC RM 

Row

Total

Use of other than 

 active runway for  arrival and  departures 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 0 15

Runway closure 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 0 15

 Other 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 NAPosition Relief Briefing 0 0 50 93 0 0 0 0 0 0 0 0 0 0 0 0 143

Employee did not 

 use checklist 0 0 0 19 0 0 0 0 0 0 0 0 0 0 0 0 19

Incomplete briefing 0 0 50 0 0 0 0 0 0 0 0 0 0 0 0 0 50Relieving employee 

 did not make use of  pertinent data 0 0 0 74 0 0 0 0 0 0 0 0 0 0 0 0 74

 Other 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 NACol.Total 0 0 8093 1311 0 0 0 0 0 0 0 0 0 0 0 0 9404 *HFACS Codes V – Violations PE – Perceptual Errors SBE – Skilled-Based Errors DE – Decision Errors PR – Personal Readiness CRM – Crew Resource Mangement PML – Physical/Mental Limitations APS – Adverse Physiological States AMS – Adverse Mental States SV – Supervisory Violations FC – Failed to Correct Problem PIO – Planned Inappropriate Operations IS – Inadequate Supervision OP – Organizational Process OC – Organizational Climate RM – Resource Management 
19

totals are the total number of causal factors associated with 

a given HFACS category. Cell totals are the frequency 

with which a given Block 68 category was associated with 

a given HFACS category. 

By examining the row totals we can see that of the 

9,404 causal factors extracted, 5,510 (59%) were associated with the radar display, 1,486 (16%) were associated 

with communications, 961(10%) were associated with 

coordination, 721(8%) were associated with aircraft observation, 583(6%) were associated with data posting, and 

143(1%) were associated with position relief briefi ngs. 

When the same Table 5 information is examined by 

column totals we see that 8,093 (86%) cases were classifi ed as skill-based errors and 1,311 (14%) cases were 

classifi ed as decision errors. It is interesting to note that 

these results are similar to those reported for aviation 

accidents. For the period 1990-2000,Wiegmann and 

Shappell (2003) reported that 80% of general aviation 

accidents involved skill-based errors and 20% involved 

decision errors. 

The cell values shown in Table 5 represent the interaction of row and column totals for each of the six Block 

68 categories. First, of the 583 data posting errors, 556 

(95%) were classifi ed as skill-based errors and 27 (5%) 

were classifi ed as decision errors. Second, of the 5,510 

OEs related to the radar display, 5,260 (95%) were classifi ed as skill-based errors and 250 (5%) were classifi ed as 

decision errors. Third, for OEs related to problems with 

the Tower controller’s observation of aircraft, all 721 of 

them (100%) were classifi ed as decision errors. Fourth, 

for OEs reported as communication errors, all 1,486 of 

them (100%) were classifi ed as skill-based errors. Fifth, 

of the 961 OEs reported as involving coordination, 741 

(77%) were classifi ed as skill-based errors and 220 (23%) 

were classifi ed as decision errors. Sixth, of the 143 OEs 

related to position relief briefi ngs, 50 (35%) were classifi ed as skill-based errors and 93 (65%) were classifi ed 

as decision errors. 

DISCUSSION

The goal of this project was to systematically examine 

the underlying human factor causes of ATC OEs. This 

was achieved by using the HFACS framework for the 

ATC environment. One of the immediate benefi ts from 

classifying OEs into HFACS categories was the realization that the majority (86%) of OEs were classifi ed as 

skill-based errors and not decision errors. The ramifi cations of this fi nding affect not only the kind of training 

that controllers should receive following an OE, but also 

the expectations about the impact that ATC decision 

support tools will have on lowering the number and 

severity of OEs. 

As previously discussed, skill-based errors tend to be the 

result of habitual actions associated with an individual’s attention, memory, and/or execution technique. Rather then 

being executed by conscious thought, skill-based actions 

(along with their corresponding errors) tend to be executed 

with little conscious effort. From a training perspective, 

then, it is important to discover what skill-based actions 

are being executed erroneously so that the individual 

can be made aware of the problem. Thus, a skill-based 

training environment (ideally one that simulates actual 

job conditions) is necessary to practice new skills so that 

they can be executed without much conscious thought. 

For example, if an OE were the result of a controller’s 

perceptual scan, then an appropriate training environment would be one that consists of scanning activities. 

Similarly, if a loss of memory were implicated in an OE, 

then an appropriate training environment would be one 

that consists of job-related memory activities. In both 

cases the training activities should provide increasing 

levels of diffi culty until the prescribed skills are performed 

routinely under actual job conditions. 

In contrast to skill-based errors, decision errors 

are the result of intentional behaviors that proceed as 

planned, yet the plan itself proves to be inadequate for 

the situation at hand. In HFACS, there are three types 

of decision errors: procedural errors, poor choices, and 

problem-solving errors; all of which require conscious 

effort to execute. From a training perspective, then, 

the challenge is to provide exercises that will improve a 

person’s ability to make better decisions within the time 

constraints imposed by the job setting. For example, assume that an OE was the result of an ATCS issuing an 

altitude change to resolve a potential confl ict, but in so 

doing created a loss of separation with another aircraft. 

After investigating the OE, it may be determined that the 

ATCS would benefi t from real-time training activities that 

required him/her to predict the consequences of various 

actions (i.e., speed, heading, and altitude changes) used 

to prevent a potential loss of separation. 

Although fewer causal factors were classifi ed as decision 

errors than skill based errors, considerable attention has 

been placed on using technology to aid the controller’s 

decision process. For example, decision support tools 

now exist that provide controllers with the capability 

of modeling the best route through a sector or within a 

sector so as to minimize interference with other aircraft 

routes in adjacent sectors and maximize the ability to 

detect potential confl icts. One might expect that with 

improved decision support tools there would be fewer 

OEs. It should be noted, however, that by design decision 

tools are directed at aiding controller decisions. Thus, 

the expected reduction in OEs should come from those 

errors that are the result of poor decisions. However, if 
20

the majority of OEs result from skill-based errors (at least 

based on the results found in this analysis), it would seem 

that greater attention should be placed in this area. 

One area of training that has recently been developed 

to improve ATC skills is entitled The National Air Traffi c 

Professionalism (NATPRO) project (Pounds, in review). 

NATPRO training is expected to improve air traffi c safety 

and effi ciency by increasing the controller’s attention 

and perception skills. The training consists of a knowledge-based seminar followed by a practicum designed to 

enhance a specifi c skill, for example, detection of relevant 

information while scanning. Although the skill itself is 

generic, the specifi c job application is associated with radar 

scanning in the en route and terminal radar environments 

and scanning for aircraft, vehicles and pedestrians from 

the tower. The strength of NATPRO training is that participants receive immediate feedback on their performance 

relative to their individual baseline performance and can 

request feedback about their performance relative to the 

performance of other participants. 

Returning to the system level of analysis, one of the 

major contributions of HFACS is the labeling (thus 

removing the mystery) of the more common types of 

latent errors (Reason, 1990). However, it is one thing 

to know that a particular latent error exists within the 

system, but that does not tell us the probability that a 

given latent error might occur. It would be useful to 

develop probability estimates for the occurrence of each 

type of latent error so that an overall OE risk assessment 

could be assigned, e.g., to a given ATC facility, a specifi c 

sector, or particular runway. Considering that ASHRAM 

was derived from human reliability risk assessment, the 

thought of integrating HFACS into an ASHRAM-like 

framework has great appeal. 

The existing OE investigation process could be further 

refi ned to better identify human error mechanisms related 

to OEs. A revised Form 7210-3 could then be used to 

record unsafe or contributing actions associated with 

OEs. Finally, for each ATC position (e.g., radar control 

or local control) the context could be specifi ed. Having 

all three components in place, the fi nal step would be to 

derive an overall risk assessment described previously for 

a given facility, sector, runway, and so on. Although the 

mathematics involved in computing a risk assessment 

are beyond the scope of this paper, the HRA literature 

provides the necessary guidance for completing such a 

project (c. f. Swain & Guttmann, 1983). 

CONCLUSION

HFACS proved to be a useful taxonomy for classifying 

the causal factors associated with OEs. A greater percentage were classifi ed as skill-based errors as compared to 

decision errors. In addition, our results demonstrated that 

the "causal factors" listed in the current OE reporting 

system is lacking in information concerning organizational 

factors, unsafe supervisory acts, and the preconditions 

of unsafe acts. It is recommended that greater attention 

be placed on developing a more comprehensive human 

factors assessment of OE causes across all levels. 

As with any study, when interpreting these results, 

one should consider the quantity and quality of the 

OE data available. Any post hoc analysis depends on 

the comprehensiveness and accuracy of the data. In this 

study we did not distinguish between facility types, i.e., 

Air Traffi c Control Towers (ATCTs), Terminal Radar 

Approach Control facilities (TRACONs), or Air Route 

Traffi c Control Centers (ARTCCs). Also, we did not 

examine the causal factors based on who was deemed to 

be primarily responsible for the OE versus who played a 

contributing role. We also used only a subset of the data 

from the 7210-3.

Since the nature of this study was to identify a candidate taxonomy or model and then to test the strongest 

candidate using OE data, the above limitations do not 

weaken the conclusion that HFACS, or some variation 

of it, could profi tably be incorporated into the OE reporting process. However, for those who wish to draw 

additional conclusions from the material presented, the 

above limitations should be considered. 

REFERENCES

Bagnara, S., Di Martino, C., Lisanti, B., Mancini, G., & 

Rizzo, A. (1991). A human error taxonomy based on 

cognitive engineering and social and occupational 

psychology. In G. Apostolakis (Ed.), Probabilistic 

safety assessment and management, pp. 513-518. New 

York: Elsevier Science Publishing Co., Inc. 

Cacciabue, P.C., Cacciabue, A. & Vivalda, C. (1993). A 

dynamic reliability technique for error assessment 

in man-machine systems. International Journal of 

Man-Machine Studies, 48: 403-428. 

Cooper, S.E., Ramey-Smith, A.M., Wreathall, J., Parry, 

G.W., Bley, D.C., and Luckas, W.J., Taylor, J.H., 

Barriere, M.T. (1996). A technique for human error 

analysis (ATHEANA). NUREG/CR-6350. Washington DC: U.S. Nuclear Regulatory Commission. 
21

Department of Transportation (2003). Draft Safety 

Strategic Plan. Washington, DC: Author. 

Dougherty, E.M. (1990). Human reliability analysis—

where shouldst thou turn? Reliability Engineering 

& System Safety, 29, 283-299. 

Endsley, M.R. (1995). Towards a theory of situation 

awareness in dynamic systems. Human Factors, 

37(1): 32-64. 

Federal Aviation Administration (1999a). National aviation research plan. Washington, DC: Author. 

Federal Aviation Administration (1999b). Strategic plan. 

Washington, DC: Author. 

Federal Aviation Administration (2002). Air traffi c quality assurance (Order 7210.56). Washington, DC: 

Author. 

Federal Aviation Administration (2003). Federal Aviation 

Administration’s Flight Plan 2004-2008. Washington, DC: Author. 

Flach, J.M. (1995). Situation awareness: Proceed with 

caution. Human Factors, 37(1):149-157. 

Fleiss, J.L. (1981). Statistical methods for rates and proportions. New York: Wiley. 

Hollnagel, E. (2002, March 9). Human reliability analysis. 

Accessed from www.ida.liu.se/~eriho/ResearchInterests_O.htm

Isaac, A. (1995). An investigation into co-ordination and 

communication errors in the Australia ATM system. 

Unpublished report for the Australian CAA.

Kaplan, A. (1964). The conduct of inquiry: Methodology 

for behavioral sciences. San Francisco: Chandler 

Publishing. 

Kirwan, B. (1992). Human error identifi cation in human reliability assessment. Part 2: Detailed comparison of techniques. Applied Ergonomics, 23 (6): 

371-381. 

Lewin, K. (1951). Field theory in social sciences. New 

York: Harper. 

Mason, S. (1997). Procedural violations – Causes, costs, 

and cures. In F. Redmil & K. J. Rajan (Eds.). Human factors in safety critical systems (pp. 287-318). 

Butterworth-Heinemann: London. 

Miller, D., & Forester, J. (2000). Aviation safety human reliability analysis method (ASHRAM). 

SAND2000-2955. Albuquerque, NM: Sandia 

National laboratories. 

Mumaw, R.J. and Roth, E.M. (1992). How to be more 

devious with a training simulator: Redefi ning scenarios to emphasize cognitively diffi cult situations. 

In 1992 Simulation Multi Conference: Nuclear 

Power Plant Simulation and Simulators. 

Pocock, S., Wright, P., Harrison, M. (1999). THEA – A 

technique for human error assessment early in design. Paper presented at the RTO HFM Workshop 

on “The human factor in system reliability – Is 

human performance predictable?” Siena, Italy. 

Pounds, J., Thompson, D., Jack, D.G., & Breedlove, R. 

(in review). Final report on the limited deployment of 

the National Air Traffi c Professionalism (NATPRO) 

program at Miami ARTCC.

Pounds, J., & Scarborough, A. (May, 2000). Air traffi c 

error analysis methods. Presentation at the quarterly 

meeting of regional air traffi c quality assurance 

managers, Norfolk, VA. 

Rasmussen, J. (1986). Information processing and human-machine interaction: An approach to cognitive 

engineering. New York: Elsevier Science Publishing 

Co., Inc. 

Reason, J. (1990). Human error. New York: Cambridge. 

Roth, E.M., Mumaw, R.J. , & Lewis, P.M. (1994). An 

empirical investigation of operator performance 

in cognitively demanding simulated emergencies, 

NUREG/CR-6208, Westinghouse Science and 

Technology Center, Pittsburg, PA. 

Sarter, N.B., & Woods, D.D. (1991) Situation awareness: 

A critical but ill-defi ned phenomenon, International 

Journal of Aviation Psychology, I (1): 45-57. 

Shorrock, S.T., Kirwan, B., Isaac, A., Anderson, H., & 

Bove, T. (1999). Technical review of human performance models and taxonomies of human error in air 

traffi c management. Unpublished manuscript. 

Swain, A.D., & Guttmann, H. E. Handbook of human 

reliability analysis with emphasis on nuclear power 

plant applications. NUREG/CR 1278. Albuquerque, NM: Sandia National laboratories (1983). 
22

Swartz, N. (1997, March 4). The concepts of necessary 

and suffi cient conditions. Accessed from www.sfu.

ca/philosophy/swartz/conditions1.htm

Wiegmann, D.A., & Shappell, S.A. (2003). A Human 

Error Approach to Aviation Accident Analysis: The 

Human Factors Analysis and classifi cation System. 

Burlington: Ashgate Publishing Company. 

Woods, D.D., & Paterson, E. S. (2000). How unexpected 

events produce an escalation of cognitive and 

coordinative demands, in P.A. Hancock and P.A. 

Desmond (9th eds.) Stress, Workload, and Fatigue, 

Hillsdale, NJ: Lawrence Erlbaum. 

Woods, D.D., Johannesen, L.J., Cook, R.I. & Sarter, 

N.B. (1994). Behind human error: Cognitive systems, 

computers, and hindsight, Crew System Ergonomics Information Analysis Center (CSERIAC), The 

Ohio State University, Wright-Patterson Air Force 

Base, Columbus, OH. 

Yellin, Joel (1976). The reactor safety study (RSS) 

(WASH-1400). Nuclear Regulatory Commission: 

Washington, DC. 
A1

APPENDIX A 

HFACS Categories and Definitions (Wiegmann & Shappell, 2003). 

Unsafe Acts – Performed by the operator 

 Errors – The mental or physical activities of individuals that fail to achieve their 

intended outcome 

Skill Based – Basic skills that occur without significant conscious 

thought

Decision – Intentional behavior that proceeds as planned, yet the plan 

itself proves inadequate or inappropriate for the situation. This is 

sometimes referred to as “honest mistakes” 

Perceptual – Occurs when one’s perception of the situation differs from 

reality usually because of degradation of sensory input 

 Violations – Willful disregard for the rules and regulations 

Routine – Habitual by nature and often tolerated by governing authority 

Exceptional – Isolated departures from authority not necessarily 

indicative of an individual’s behavior pattern, nor condoned by 

management 

Preconditions for Unsafe Acts - Addresses the underlying causes for Unsafe Acts 

 Environmental Factors – Surrounding conditions affecting the operator 

Physical Environment – Refers the operational environment and the 

ambient environment 

Technological Environment – Encompasses a variety of issues including 

the design of the equipment and controls, display/interface 

characteristics, checklist layouts, task factors and automation 

 Condition of Operators – Factors within the individual that interferes with 

optimal performance 

Adverse Mental States – Preexisting mental states (such as mental 

fatigue, personality traits, pernicious attitudes, and misplaced motivation) 

affecting performance 

Adverse Physiological States - Those medical or physiological conditions 

that preclude safe operations
A2

Physical/Mental Limitations – Those instances when operational 

requirements exceed the capability of the individual 

 Personnel Factors – Factors that individuals do to themselves to create 

preconditions for unsafe acts 

Crew Resource Management – Addresses factors that lead to poor 

coordination among personnel 

Personal Readiness – Addresses situations when individuals fail to 

prepare physically or mentally for duty 

Unsafe Supervision – Supervisory actions that influence the conditions of the operator 

and the type of environment in which they operate 

 Inadequate Supervision – Deals with lack of guidance and oversight in day-today operations 

 Planned Inappropriate Operations – Occurs when individuals are put in an 

unacceptable risk due to the operation tempo and/or scheduling of work 

 Failure to Correct Problem – Those instances when efficiencies among 

individuals, equipment, training, or other related safety errors are “known” to the 

supervisor, yet are allowed to continue unabated 

 Supervisory Violations – Those instances when existing rules and regulations are 

disregarded by supervisors 

Organizational Influences – Fallible decisions of upper-level management affecting 

supervisory practices as well as the conditions and actions of the operator 

 Resource Management – Encompasses the realm of corporate-level decisionmaking regarding the allocation and maintenance of organizational assets (such 

as personnel, money, equipment, and facilities) 

 Organizational Climate – The working atmosphere within the organization 

which includes culture, policies, and structure 

 Organizational Process – Refers to corporate decisions and rules that govern the 

everyday activities within the organization. This includes the establishment/ use 

of standard operational procedures, and formal methods for maintaining 

oversight of the workforce 
B1

APPENDIX B 

Probability Risk Assessment, Human Reliability Analysis and ATHEANA 

A Technique for Human Error Analysis (ATHEANA) belongs to the class of human reliability 

analysis (HRA) models used in probabilistic risk assessments (PRAs) to assess the “implications of 

various aspects of human performance on risk” (Cooper, et. al., 1996, p. 1-1). Closely tied to the nuclear 

industry, the concept of PRA was developed as a means to: (1) identify potential areas of risk and indicate 

how those risks could be mitigated, and (2) quantify the overall risk potential of a given plant (Reason, 

1990). Both criteria were necessary to achieve public and government support for the building and 

operating of a nuclear power plants. 

As a guideline, the basic steps for conducting a PRA were formally proposed in a 1975 

publication of the U.S. Reactor Safety Study known as WASH-1400 (Yellin, 1976). Reason (1990) 

summarized the PRA steps reported in WASH-1400 as: 

a. Identify the sources of potential hazard. 

b. Identify the initiating events that could lead to this hazard. 

c. Establish the possible sequences that could follow from various initiating events using event 

trees.

d. Quantify each event sequence. This involves data or judgment about two things: (1) the 

frequency of the initiating event, and (2) the probability of failure on demand of the relevant 

safety systems. 

e. Determine the overall plant risk. This will be a function of the frequency of all possible 

accident sequences and their consequences (p. 219) 

Although PRAs provided essential information for assigning risk to a proposed nuclear power 

plant, they failed to adequately address the human element association with system failures (Reason, 

1990). Specifically, the means by which probabilities were assigned to potential human failures lacked 

methodological rigor. Hence a need arose to incorporate a Human Reliability Analysis (HRA) into PRAs. 

HRA is defined as a set of methods that assess the probability that a person will correctly perform 

some system-required activity during a given time period without performing any extraneous activity that 

can degrade the system (Hollnagel, 2002). One of the more widely used methods of conducting a HRA is 

the Technique for Human Error Rate Predictions, THERP (Swain & Guttmann, 1983). The procedures 

used by THERP are similar to those used in a standard PRA and include: 

a. Identify the system functions that may be influenced by human error. 

b. Conduct a detailed task analysis of the related human operations. 

c. Estimate the relevant error probabilities using a combination of subject matter experts 

(SMEs) and empirical data. 

d. Estimate the effects of human errors on system failure events usually by integrating HRA 

with PRA. 

Although widely used, Reason (1990) notes that THERP is often criticized for its simplistic view 

of the human operator. THERP, as with other HRA techniques, assumes that the human operator behaves 

as any other piece of equipment. That is, the operator either performs correctly or does not. This 

assumption is embedded in the basic formula used to calculate the probability of a specific erroneous 

action (PEA):

PEA = HEPEA* PSF Wk C

N

k

¦ k  

*

1

where: HEPEA = the human error probability of a erroneous action 
B2

 PSF = a performance shaping conditions 

 W = weight of a PSF 

 C = constant 

 N = number of PSFs 

As Hollnagel (2000) explains, the above equation contains two fundamental assumptions, neither 

of which is reasonable: 

1) First, that the probability of failure can be determined for specific types of actions 

independently of any context. 

2) Second, that the effects of the context are additive, which is the same as saying that the 

various performance conditions (such as interface quality, stress, level of training, complexity 

of task, etc.) do not influence one another (p. 2). 

Further criticisms of HRA techniques emerged throughout the 1980s. The major criticisms were 

summarized by Doughtery (1990) and latter elaborated upon by Hollnagel (2002) as follows: 

a. Existing empirical data are insufficient to support quantitative predictions of human 

performance in complex systems. This problem had actually been recognised by HRA 

practitioners since the early 1960s. As alternatives to empirical data, HRA often relied on 

either expert judgement or data from simulator studies. 

b. Expert judgements can be used in lieu of empirical data, but there is a lack of agreement 

about the use of expert judgement methods. The methods neither have satisfactory betweenexpert consistency, nor produce accurate predictions. 

c. Data from simulator studies can be used instead of empirical data, but the calibration to real 

life situations or ability to generalize is inadequate. The veracity and validity of simulator 

data have not yet been convincingly demonstrated. 

d. The accuracy of predictions from HRA methods is debatable and generally unproven, 

particularly for non-routine tasks. Benchmark studies usually produce divergent results, for 

many different reasons. 

e. The psychological realism in most HRA methods is inadequate, and the assumptions about 

human behaviour are often highly questionable from a psychological point of view. 

f. The treatment of important Performance Shaping Factors is inadequate. In particular, there is 

too little emphasis on PSFs relating to management, organisation, culture, etc. (p. 2) 

Given the mounting criticisms surrounding the method of calculating human reliability 

probabilities, researchers began developing more comprehensive HRA models, such as: A Technique for 

Human Error Assessment Early in Design, THEA (Pocock, Wright, and Harrison, 1999); Dynamic 

Reliability Technique for Error Assessment in Man-Machine Systems, DREAMS (Cacciabue, 1993); and 

the Technique for Human Error Analysis, ATHEANA (Cooper et al., 1996). The latter will be discussed 

in more detail. 

The goal of the ATHEANA project was to develop an improved method for HRA that would 

“allow for a more realistic assessment and representation of the human contribution to plant risk, and 

thereby increase the utility of PRA” (Cooper et al., 1996, p. ix). Specifically, ATHEANA was developed 

to address deficiencies in HRA approaches by: 

1. Addressing errors of commission and dependencies 

2. Representing more realistically the human-system interactions that have played important 

roles in accident response, as evidenced by operating experience, and 

3. Integrating recent advances in psychology with engineering, human factors and PRA 

disciplines (Cooper et al., 1996, p. 1-2). 
B3

The above were accomplished through an integration of retrospective analysis of past operating 

events and the prospective analysis in support of PRA. This can be seen in the six elements comprising 

ATHEANA as shown below. The six elements fall under three broad categories consisting of: 

1. Error Forcing Context (EFC) 

a. Performance Shaping Factors (PSF) 

b. Plant Conditions (PC) 

2. Human Error 

a. Error Mechanisms 

b. Unsafe Actions 

3. PRA Model 

a. Human Failure Events (HFEs) 

b. Scenario Definitions 

EFC represents the interaction of PC and PSF. Typically EFCs are unanalyzed plant conditions 

(similar to Reason’s latent errors) that serve to shape an operator’s response to an event. Examples of 

unanalyzed plant conditions are the effects of a history of false alarms associated with a system 

component, shutting off safety functions during various phases of system shut down, and unusual or 

incorrect equipment configuration that is integrated into the operating system. Examples of performanceshaping factors include such factors as procedures, training, communications, supervision, staffing, 

organizational factors, stress and environmental conditions. 

ATHEANA defines human error as the “divergence between an action actually performed and the 

action that should have been performed” (Cooper et. al., 1996, p.2-9). Although the term “action” implies 

a behavioral act (i.e. unsafe actions), ATHEANA also allows for error mechanisms (i.e., psychological 

mechanisms such as judgment and decision making) to be incorporated into the definition of human error. 

The PRA model is based on probability estimates for the frequency of occurrence of specific 

failure points associated with a given accident scenario. HFEs represent the failure of a function, system 

or component as the result of an unsafe action by the human operator (e.g. errors of commission and 

errors of omission) that places the operation in a worse condition. The scenarios that are used to develop 

probability estimates usually consist of event tree and fault tree sequences that model a particular chain of 

events. The level of detail presented depends of the function, systematic, or component level of analysis. 

C1

APPENDIX C

Form 7210-3

Final Operational Error/Deviation Report Report Number

Part II. FACILITY MANAGER ACTION 

67. Select the category of the operational error/deviation. (more than one category may be possible) 

Procedural Equipment ATCS Manager/Supervisor/Other Personnel 

 

68. Causal Factors No Yes (employee) 

A B C D E 

A. Data Posting 

(1) Computer Entry 

Incorrect input 

Incorrect update 

Premature termination of data 

Input/Update not made 

Other (explain) 

(2) Flight Progress Strip 

Not updated 

Interpreted incorrectly 

Posted incorrectly 

Updated incorrectly 

Premature removal 

Other (explain) 

B. Radar Display 

(1) Misidentification 

Failure to reidentify aircraft when the accepted target identity becomes questionable 

Overlapping data blocks 

Acceptance of incomplete or difficult to correlate position information 

Other (explain) 

(2) Inappropriate Use of Displayed Data 

MODE C 

BRITE 

Conflict alert 

Failure to detect displayed data 

Failure to comprehend displayed data 

Failure to project future status of displayed data 

Other (explain) 

C. Aircraft Observation (Towers Only) 

(1) Actual Observation of Aircraft 

(2) Improper Use of Visual Data 

Landing 

Taking Off 

Ground Operation 

Taxiing across runway 

Holding in position for takeoff 

Other (explain) 
C2

No Yes (employee) 

A B C D E 

D. Communication Error 

(1) Phraseology 

(2) Transposition 

(3) Misunderstanding 

(4) Read back 

Altitude 

Clearance 

Identification 

Other (explain) 

(5) Acknowledgement 

(6) Other (explain) 

E. Coordination 

(1) Area of Incident 

Intra-sector/position 

Inter-sector/position 

Inter-facility 

Facility type: , level: , and facility ID: 

(2) Failure to utilize/comply with precoordination information 

(3) Improper use of information exchanged in coordination 

Aircraft Identification 

Altitude/Flight Level 

Route of Flight 

Speeds 

APREQs 

Special Instructions 

Other (explain) 

(4) Failure to coordinate between ground and local control 

Crossing active runway 

Vehicle, equipment, or personnel on active runway 

Use of other than active runway for arrival and departures 

Runway closure 

Other (explain) 

F. Position Relief Briefing 

(1) Employee did not use position relief checklist 

(2) Employee being relieved gave incomplete briefing 

(3) Relieving employee did not make use of pertinent data exchanged at briefing 

(4) Other (explain) 

Page 8 
C3

FORM 7210-3 DEFINITIONS 

SECTION A: DATA POSTING

A data posting error is any error of calculation, omission, or incomplete data, erroneous entries, handling, or subsequent revisions

to this data. This includes errors in posting and recording data. It does not include errors involved in receiving, transmitting,

coordinating, or otherwise forwarding this information. If one of the causal factors listed does not adequately describe the factor 

involved, list the factor under "Other" and provide a brief explanation. 

SECTION B: RADAR DISPLAY

a. Misidentification 

Radar misidentification means a failure to properly identify the correct target and includes subsequent errors committed after the 

original identification was properly accomplished. Indicate the listed item(s), which most closely describes the reason for 

misidentification. If one of the causal factors listed does not adequately describe the factor involved, list the factor under "Other"

and provide a brief explanation. 

b. Inappropriate Use of Displayed Data 

A data or display information error occurs due to a failure to maintain constant surveillance of a flight data display or traffic 

situation and to properly use the information presented by the display or situation. If one of the causal factors listed does not

adequately describe the factor involved, list the factor under "Other" and provide a brief explanation. 

SECTION C: AIRCRAFT OBSERVATION (Towers Only)

An aircraft observation error means a failure to maintain constant surveillance of aircraft and the movement area, and to properly 

react to, interpret, or otherwise utilize, in a timely manner, the information being viewed. If one of the causal factors listed does 

not adequately describe the factor involved, list the factor under "Other" and provide a brief explanation. 

SECTION D: COMMUNICATIONS ERROR

A communications error is a causal factor associated with the exchange of information between two or more people (e.g., pilots 

and specialists). It refers to the failure of human communication not communications equipment. 

a. Phraseology 

Use of incorrect or improper phraseology. 

b. Transposition 

An error due to transposition of words, numbers, or symbols by either oral or written means. This involves writing/saying one 

thing while thinking/hearing something else. 

c. Misunderstanding 

The failure to communicate clearly and concisely so that no misunderstanding exists for any actions contemplated or agreed 

upon.

d. Read back 

The failure to identify improper or incorrect read back of information. 

e. Acknowledgment 

The failure to obtain or give an acknowledgment for the receipt of information. 

f. Other

If the causal factors listed above do not adequately describe the factor involved, list the factor and provide a brief explanation.
C4

SECTION E: COORDINATION

Any factor associated with a failure to exchange requirement information. This includes coordination between individuals, 

positions of operation, and facilities for exchange of information such as APREQ's, position reports, forwarding of flight data,

etc. If one of the causal factors listed does not adequately describe the factor involved, list the factor under "Other" and provide a 

brief explanation. 

SECTION F: POSITION RELIEF BRIEFING

Relief briefing errors are special errors of both communication and coordination, which occur as the result of position relief.

They include such things as failure to give a relief briefing, failure to request a briefing, incomplete or erroneous briefing, etc. If 

one of the causal factors listed does not adequately describe the factor involved, list the factor under "Other" and provide a brief 

explanation.