Developing Temporal 

Markers to Profile 

Operational Errors

Julia Pounds1

Mark D. Rodgers2

Deborah Thompson3

Daniel G. Jack4

1

 Civil Aerospace Medical Institute

Oklahoma City, OK 73125

Federal Aviation Administration

2

 CSSI, Inc.

Washington, DC 20024

3

 University of Oklahoma

Norman, OK 73069

4

 Xyant Technology, Inc.

Norman, OK 73072

August 2006

Final Report

DOT/FAA/AM-06/20

Office of Aerospace Medicine

Washington, DC 20591
NOTICE

This document is disseminated under the sponsorship 

of the U.S. Department of Transportation in the interest 

of information exchange. The United States Government 

assumes no liability for the contents thereof.

___________

This publication and all Office of Aerospace Medicine technical reports are 

available in full-text from the Civil Aerospace Medical Institute’s publications 

Web site: 

www.faa.gov/library/reports/medical/oamtechreports/index.cfm
i

Technical Report Documentation Page 

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No. 

DOT/FAA/AM-06/20 

4. Title and Subtitle 5. Report Date 

Developing Temporal Markers to Profile Operational Errors August 2006 

6. Performing Organization Code 

 

7. Author(s) 8. Performing Organization Report No. 

Pounds J,1 Rodgers MD,2 Thompson D,3 Jack D4 

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS) 

 

11. Contract or Grant No. 

1

FAA Civil Aerospace Medical Institute 

P.O. Box 25082 

Oklahoma City, OK 73125 

3

University of Oklahoma 

Norman, OK 73069 

2

CSSI, Inc. 

Washington, DC 20024 

4

Xyant Technology, Inc. 

Norman, OK 73072 

 

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered 

 

 

 

Office of Aerospace Medicine 

Federal Aviation Administration 

800 Independence Ave., S.W. 

Washington, DC 20591 14. Sponsoring Agency Code 

15. Supplemental Notes 

Work was accomplished under approved task AM-BHRR524 

16. Abstract 

A commonly held view is that system and human vulnerabilities, whether they emerge at a common moment or 

over a situation, can form links in a chain of events resulting in an air traffic operational error (OE). However, 

this truism has not led to the development of better techniques for profiling this progression. If we generally 

accept that OEs evolve over time, then OEs have temporal characteristics. By better understanding these 

temporal characteristics, we will better understand how vulnerabilities become links in a chain so that 

resources can be allocated effectively to develop mitigation strategies. Two activities were conducted to develop 

the temporal markers (TMs) framework. First, air traffic subject matter experts identified objective (or calculable) 

points that can be identified a traffic situation. These points could differ at the point in time they occur, but they 

would occur in some form in most, if not all, OEs. A list of TMs was generated and tested using archival en 

route OEs. Using the framework is straightforward. This preliminary evidence suggests that temporal profiling 

could help to discover trends across OEs that are not currently being systematically examined. A larger set of OEs 

could be tested with this method to accomplish this. Ultimately, the technique should be used for all OEs in the 

national airspace so that a TM database could be developed for OE trend analysis. 

 

17. Key Words 18. Distribution Statement 

 

 

Air Traffic Control, Timeline, Chronology, Operational 

Error, Vulnerabilities, Human Error, Incident Investigation

Document is available to the public through the 

Defense Technical Information Center, Ft. Belvior, VA 

22060; and the National Technical Information 

Service, Springfield, VA 22161 

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price 

Unclassified Unclassified 17 

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized 

1

Developing Temporal Markers to Profile Operational Errors

Introduction

Researchers and operational personnel alike generally 

accept the view that air traffic operational errors1 (OEs), 

like many types of events in complex systems, evolve 

over time (e.g., Dekker, 2002; Dörner, 1996; Harry 

and Schroeder, 2000; Klein, Orasanu, Calderwood, and 

Zsambok, 1993; Perrow, 1999; Petersen, 1996; Phimister, Bier, and Kunreuther, 2004; Reason, 1997; Sagan, 

1993; Salas and Klein, 2001). If we generally accept 

that OEs evolve over time, then we can expect OEs to 

have temporal characteristics. However, the truism that 

events in complex systems have a temporal component 

has not yet led to development of better techniques for 

profiling the temporal progression of evolving events such 

as OEs. This project was conducted to begin to fill this 

gap. Temporal information may give OE analysts another 

means to identify and evaluate a variety of potential OE 

reduction strategies.

A commonly held view is that system and human 

vulnerabilities can converge at one moment or may be 

distributed over the course of an evolving situation. 

In either case, they can contribute to, or cumulate in, 

unsuccessful outcomes. Many researchers have devoted 

attention to identifying and analyzing these precursor 

and error-producing human and system conditions (e.g., 

Bier, 1998; Krause, S. S., 2003; Petersen, 1996; Phimister, 

Bier, and Kunreuther, 2004; Rasmussen, 1980; Reason, 

1990, 1997). On the other hand, the study of temporal 

characteristics in this context has received little attention 

and is generally conducted relative to its perception and 

related subjective experiences (e.g., Block, 1990; Gibbon 

and Allan, 1984).

Although many types of data are currently reported 

after an OE, by identifying the temporal characteristics 

of OEs, we should be able to understand OEs from a 

different perspective. Presently, over 200 elements of 

information are recorded as part of the Federal Aviation 

Administration’s (FAA’s) OE reporting process. In addition 

to a description of when and where the event occurred, 

1

 An operational error is defined as an occurrence attributable to an 

element of the air traffic system in which (1) less than the applicable 

separation minima results between two or more aircraft, or between 

an aircraft and terrain or obstacles (e.g., operations below minimum 

vectoring altitude (MVA); equipment / personnel on runways), as 

required by FAA Order 7110.65 or other national directive; or (2) an 

aircraft lands or departs on a runway closed to aircraft operations after 

receiving air traffic authorization, or (3) an aircraft lands or departs 

on a runway closed to aircraft operations, at an uncontrolled airport 

and it was determined that a NOTAM regarding the runway closure 

was not issued to the pilot as required (FAA Order 7210.56, 2002).

these encompass characteristics of the controllers and 

supervisors involved, training, past performance of personnel (i.e., involvement in previous incidents), equipment 

and aircraft. Items also cover data posting (e.g., computer 

entries and flight progress strips), controller’s use of radar 

displays to maintain knowledge of the traffic situation, 

communications, coordination, transfer of position 

responsibilities (i.e., position relief briefings), and in air 

traffic control towers, the observation of aircraft. 

A chronology (timeline) of the OE is routinely developed by quality assurance personnel at air traffic facilities 

during the FAA’s OE reporting process as part of the Final 

Operational Error or Deviation Report (Form 7210-3; 

FAA, 2002). Directions for completing the Final OE/D 

Report are: 

Explain, in chronological order, each factor relevant to 

the incident. Tell a detailed story, describing the pertinent 

actions of all those involved (e.g., controllers by position, 

supervisors, aircraft, etc.). It should be apparent what actions (or lack of) contributed to or caused the incident. 

Include any explanations necessary from previous blocks. 

(pg. 7)

Two examples of timelines from our archive are shown 

below, although the amount of content and format of 

timelines included in the Final Report varies widely 

across facilities.

Example 1: 

1356:24 DAL1906 checked on frequency at FL330.

1358:42 R63 cleared DAL1906, due to ZID Letter of 

Agreement, direct Pocket City, then MOSEY5 

arrival to CVG. R63 cleared DAL1906 to turn left 

heading 3-5-0, when able direct Pocket City.

1358:52 DAL1906 acknowledged the heading of 3-5-0 

direct Pocket City and MOSEY.

1401:16 AAL2736 advised R63 that his TCAS had activated advising him to climb due to traffic at his 

altitude.

Example 2:

1931:35 When DAL0025 reported on frequency climbing 

to 17,000 feet, Specialist “A” cleared the aircraft to 

climb and maintain FL220 and to proceed direct 

Birmingham (BHM).
2

1932:14 When DAL1347 reported on frequency, Specialist “A” cleared the aircraft to climb and maintain 

17,000 feet.

1933:12 Separation was lost.

1933:20 Specialist “A” cleared DAL1347 to level off at 

present altitude. DAL1347 did not respond until 

the third call at which time the aircraft was already 

level at 17,000 feet.

1933:37 Specialist “A” instructed DAL0025 to expedite 

climb to FL220.

1933:47 Specialist “A” cleared DAL 1347 to descend and 

maintain 16,000 feet.

Various stakeholder groups have long monitored incoming OE reports so that strategies could be developed 

to reverse human performance errors, such as readback/

hearback errors, lack of positive control and misapplication 

of procedures. Several analyses incorporated a temporal 

dimension. For example, air traffic operations tracked the 

number of OEs relative to the controller’s time on position 

and number of aircraft being handled by the controller 

(Pounds and Ferrante, 2005). Researchers examined 

whether information about the stabilization of controllers’ scanning patterns during the first 10-20 minutes on 

position could be used to inform our understanding of 

how controllers establish situation awareness during that 

time (Smolensky, 1993). Although much attention has 

been devoted to understanding the transfer of position 

responsibilities and despite an ongoing interest in time as 

a factor in relation to various air traffic operations, there 

is currently no way to identify and analyze information 

about how an OE evolves temporally. 

One means for understanding the evolution of an 

OE might be to look for events that could be identified 

temporally, which all OEs have in common or which 

could potentially occur in every OE. We called these 

events temporal markers (TMs). These points may differ 

in when they occur during the OE timeline, but they 

occur in some form in most, if not all, OEs. TMs could 

also be used to compute time intervals. We called these 

time intervals calculated between two TMs the Temporal 

Marker Time Spans (TMTSs).

This idea led us to first identify an exhaustive list of 

TMs. The list was then organized into a framework and 

tested using a convenience sample of OEs to see whether 

using TMs would yield new information about OEs. This 

report details how this was accomplished, the resulting 

framework of TMs, examples of new information and 

examples of difficult questions that could be answered 

by this type of analysis.

Step 1 – Identifying the TM Framework

Method

In Step 1, we first had to define (operationalize) what 

we meant by temporal markers and determine whether any 

events in OEs matching the definition could be identified 

using available materials generated from previous OE 

investigations. In Step 2, we used the list of temporal 

markers identified in Step 1 to analyze a convenience 

sample of OEs. A short summary completes each section, 

although the principal discussion of the results is reserved 

for the conclusion of the paper.

We defined a temporal marker (TM) as an objective (or 

calculable) point in time that can be identified during the 

unfolding of the OE. Temporal markers must be identifiable (or calculated) from data sources available in the AT 

facility. The range of possible TMs was bounded only by 

“the context of an operational error.” These event points 

could occur at different times during an OE, but they 

would occur in some form in most, if not all, OEs. 

Participants

Three retired FAA air traffic control specialists (ATCSs) 

with experience in OE investigations were recruited as 

ATC subject matter experts (SMEs) to assist in the development of a comprehensive list of TMs. Their collective 

experience included 87 years of facility management 

and operational supervision in en route and terminal 

operations. 

Materials

A convenience sample of six OEs from one Air Route 

Traffic Control Center (ARTCC) was used for Step 1. 

These were selected because both the Final OE/D Report 

(FAA Form 7210-3) and the Systematic Air Traffic Operation Research Initiative (SATORI) re-creation for each 

were available from an existing archive. SATORI (Rodgers 

and Duke, 1993) was developed, tested and fielded to all 

en route facilities and regional quality assurance offices 

with the goal of gaining “a better understanding of the 

interaction between the various elements of displayed 

information, verbal interactions and the control actions 

taken by air traffic control specialists ” (pg. 1). Information 

about the SATORI system’s capabilities first appeared in 

ATCA’s Journal of Air Traffic Control (Rogers and Duke, 

1993). SATORI is still currently in operational use and 

enables its users to re-create segments of operational traffic 

in a format similar to what was displayed to the ATCS, for 

example, showing relative location and separation, speeds 

and headings of aircraft. Among other things, SATORI 

can display full and limited data blocks, beacon targets 

and conflict alerts. Video and audio are synchronized and 

the air traffic situation can be displayed in four dimensions. At en route facilities, SATORI systems enable the 
3

facility quality assurance staff to re-create OE situations 

for the controllers involved.

The SMEs were provided with our definition of a 

temporal marker and a list of some TM examples, such 

as the time when the aircraft entered the controller’s assigned airspace, the time when the aircraft checked on the 

controller’s frequency and the time when the controller 

issued the first control instruction to the pilot.

Procedure

The SMEs convened as a group on several occasions 

to create an exhaustive list of TMs. An iterative process 

was used to develop, test and modify the list. At the first 

meeting, the principal investigator for the project discussed 

the project goals with the SMEs. The SMEs then observed 

Satori re-creations and reviewed the final reports from 

three OEs to familiarize themselves with the materials 

for the project. They were then asked to generate a list 

of events that, from their experience, were common to 

all OEs. From these, the SMEs then drafted their first 

TM list using the TM definition. 

At the second meeting, the list was reviewed, discussed 

and edited. Once this draft list was judged by the SMEs 

to be complete, it was tested using the three remaining 

OEs. To test the list, the SMEs first viewed the Satori

playbacks for the three OEs and noted any TMs on the 

list that needed to be modified or any that needed to be 

added to or deleted from the list. The group convened a 

third time to review and finalize the list. 

Results

The list of TMs that the SMEs developed and concurred with included identifiable, objective event points 

that occur during the evolution of an OE, reflecting 

several types of activities and demonstrating what kinds 

of information can be extracted from data available at 

facilities. TMs enable OEs to be described in more detail 

than is generally the case. 

Table 1 shows the final list organized as a general 

framework. Columns in the table represent the numbering system (TM #), TM definitions2

, sources where a 

time stamp for the TM can be found (Data Source) and 

one or more columns for the aircraft of interest (A/C #). 

Column headings in Table 1 show how the analyst would 

set up the framework to examine TMs in an OE with 

two aircraft in conflict and another aircraft of interest. 

Each column is discussed below as it would be used by 

an analyst. 

2 Throughout the framework, controllers are described as appropriate 

according to their relationship with each other or with the aircraft, i.e., 

receiving, relieving, transferring. This helps the analyst to remember 

relative relationships in the OE.

Aircraft Numbering Methodology

The analyst first numbers each aircraft to permit TMs 

to be identified for each aircraft as needed. Aircraft are 

designated systematically as A/C1, A/C2, A/C3 and so 

on. The number assigned to the aircraft is based on the 

sequence of its entry into the controller’s airspace where 

the OE occurred.—whether by (a) coordination accomplished between controllers using the land line, (b) using 

automated handoff and acceptance, or (c) pilot initiated 

contact with ATC. The first aircraft discussed (or flashed) 

between the handoff and receiving controllers, or that 

contacted the controller directly (e.g., for VFR service 

in the airspace), is labeled as A/C1. The second aircraft 

discussed (or flashed) between the handoff and receiving 

controllers, or that contacted the controller directly, is 

labeled as A/C2 and so on. All aircraft of interest as elements of the OE situation are assigned a number using 

this method. In addition to an aircraft that was being 

transferred between handoff and receiving controllers, an 

aircraft might also have contacted the controller directly, 

for example, for VFR service through the airspace, to 

request flight following, or to request an IFR clearance 

after departing an uncontrolled airport where the center 

provides departure services.

An example of how aircraft are numbered is as follows. 

Consider a situation in which an aircraft (Cessna1234) 

departed an uncontrolled airport. This airport did not 

have an affiliated Terminal Radar Approach Control 

(TRACON) facility so ATC services for pilots arriving 

and departing this airport were provided by the center 

controller working the center’s Alpha sector. 

•	 The pilot of Cessna1234 contacted the Alpha sector 

controller at 0800 and requested IFR clearance through 

Alpha sector. 

•	 Alpha sector controller issued the pilot of Cessna1234 

the appropriate clearances. 

•	 The Alpha sector controller also verbally accepted 

a handoff of another aircraft (MEL0098) from an 

adjoining sector controller at 0805. 

•	 Another aircraft (HRZ990) was flashed to the Alpha 

sector controller, who then clicked on the flashing 

datablock and accepted the handoff of HRZ990 at 

0806.

•	 The pilot of HRZ990 requested to remain in level 

flight at 35,000 feet (to transition Alpha sector).

•	 Cessna1234 and MEL0098 lost separation at 0811. 

•	 To reestablish separation between Cessna1234 and 

MEL0098, the Alpha sector controller instructed 

MEL0098 to climb to 35,000 feet.

•	 At 0812, separation was lost between HRZ990 and 

MEL0098.
4

Table 1. Framework of temporal markers for an OE situation having three aircraft of interest, including 

clarification for some temporal markers (in italics). 

TM # a Temporal Markers for Center and Terminal (Radar) Data 

Source b

A/C

1

A/C

2

A/C

3

1 Time position relief briefing completed. 4

2 Time relieving controller assumed position. 5,4 

3 Time controller being relieved left position. 8

* TM#1: The time when completion is acknowledged by the receiving controller. 

* TM#3: Position is considered “left” at the time the relieved controller unplugged from the position. 

4_1a Time of 1st Inter/Intra facility coordination initiated on the involved 

A/C. 6,4 

4_1b Time of 1st Inter/Intra facility coordination completed on the involved 

A/C. 6,4 

* TM#4a and 4b: land line coordination. 

5a Time radar identification initiated by transferring controller (Handoff, 

Point out)*. 6,1,4 

5b Time radar identification completed by receiving controller (Handoff, 

Point out)*. 6,1,4 

* TM# 5a and 5b: These two TMs may be the same controller if initial identification is being established. 

* TM# 5a and 5b: If no landline coordination takes place, this would be the first coordination for the aircraft 

completed electronically. 

6 Time A/C checked in on receiving controller's frequency. 4

7 Time two-way communications established between pilot and 

receiving controller. 4

8 Time A/C entered receiving controller's airspace. 1,6 

 

9_1a Time of 1st ATC instruction by receiving controller. 4

* TM# 9_1a: may occur at the same time as TM#7.

9_1b Time pilot verbally responded to 1st ATC instruction from receiving 

controller. 4

 

10 Time of first point where positive separation between A/C was not 

ensured. 1,2,6,9 

*TM#10 is the first time when the aircraft were set on conflicting headings. The “first point” was defined 

irrespective of the location of the aircraft — 

1. the aircraft could be in the previous sector, e.g., if separation was not guaranteed before the aircraft 

was “shipped” to the next controller. 

2. the aircraft could be in the receiving controller’s sector after the controller takes the handoff of the 

aircraft.

This can occur as a result of either (for example) – 

1. controller action or inaction to ensure separation 

2. pilot actions or inactions (e.g., on wrong frequency in new sector) 

11a Time of last ATC instruction by receiving controller prior to loss of 

separation. 4

11b Time pilot acknowledged last ATC control instruction by receiving 

controller prior to loss of separation. 4

 

12 Time conflict alert activated. 6,1 

 

13_1a Time of controller's 1st instruction to avoid separation error. 4
5

Table 1 (continued). 

13_1b Time pilot verbally responded to controller's 1st instruction to avoid 

separation error. 4

 

14 Time separation lost. 7,2,1 

 

15_1a Time of controller's 1st attempt to reestablish separation. 4

15_1b Time pilot verbally responded to controller's 1st attempt to reestablish 

separation. 4

 

16 Time separation reestablished. 2,1 

a. Left column shows sequencing codes. Shading indicates element groups. 

b. Data source codes are listed with the relevant TM. Data sources are listed in Table 2. 

To maintain sequence information, Cessna1234 would 

be designated as A/C1 and MEL0098 would be designated 

as A/C2. In cases where a third aircraft (e.g., HRZ990) 

influenced the events leading to an OE, it would be 

designated as A/C3, and TMs would be identified for it 

also if, for example, the analyst wanted to specify its role 

in the event. In this example, A/C3 lost separation with 

A/C2 subsequent to when separation was lost between 

A/C1 and A/C2 (e.g., the controller created a second OE 

while reestablishing separation after the first OE). 

In another case, perhaps A/C3 required special handling or multiple communications with the Alpha sector 

controller prior to when separation was lost between 

A/C1 and A/C2. Examining TMs for an A/C3 or A/C4 

would shed light on situations where other aircraft were 

involved but did not lose separation or were suspected 

of being a contributing factor to the OE. This method 

of examining traffic events could be used with as many 

aircraft as the situation warranted.

No explicit rules were developed during this project 

about what circumstances dictate that the analyst should 

use more than two aircraft. A useful guide for this decision 

would be to include other aircraft when the information 

would contribute to a better understanding of the OE. 

Columns as many A/C as needed would be used by the 

analyst, and times corresponding to each TM for the A/C 

would be entered in the appropriate column. Using this 

method, the framework permits more than two aircraft 

to be tracked if the analyst determines that information 

about the third aircraft is needed to better understand 

how the OE developed. 

Temporal Markers for Center and Terminal (Radar)

This column of the framework describes each TM 

identified by our SMEs. Several of the TMs are annotated for clarification. Each annotation is shown in italics 

beneath the TM to which it refers. 

Data Source

The SMEs also identified the source(s) of the data 

needed to identify each TM and these are listed in the 

adjacent Data Source column. These codes represent 

the sources where one would find the information to 

identify when that particular TM occurred. The data 

sources corresponding to the codes are listed in Table 2. 

Only one TM requires that the data be obtained from 

facility personnel (TM #3: “Time controller being relieved 

left position.”). The rest are obtained using data from 

Table 2. Sources of data to find times to complete the Temporal Markers in Table 1. 

Code Data Source

1 SATORI or RAPTOR 

2 NTAP (National Track Analysis Program) 

3 CDRs (Continuous Data Recording) 

4 Audio tapes 

5 SISO (sign in-sign off /out records) 

6 DART (Data Analysis Reduction Tool) 

7 OEDP (Operations Error Detection Program, also known as the “snitch patch”) 

8 Personnel interviews 

9 Flight progress strips 
6

automation and tools used in en route facilities, such 

as the National Track Analysis Program (NTAP), flight 

progress strips and audio tapes. 

TM # 

The left-hand column in Table 1 lists the alphanumeric 

code assigned to each type of TM. Because the primary 

goal of this project was to identify temporal characteristics 

in an OE as it evolved over time, we first assigned each 

type of TM a sequencing code. (The obvious choice for 

a numbering method was to use the common method of 

assigning the first TM as #1, the second TM as #2 and 

so on.) This is the first character of the TM# code and 

indicates the relative order for that type of TM within the 

general sequence of TMs that make up the framework. 

Thus, it was reasonable to assign the first TM of the 

framework to the first moment when a controller assumed 

position responsibility. However, transfer of position is a 

complex, multi-step procedure and encompasses several 

sub-tasks, so these steps were designated as TM#1, TM#2 

and TM#3 as follows. 

•	 TM#1: Time verbal position relief briefing completed.

The time when the verbal briefing was completed was 

defined as the moment when the receiving controller 

acknowledged that it was completed (FAA, 2004). By 

that time, the relieving controller has asked any questions necessary to ensure a complete understanding of 

the operational situation and these have been completely 

answered by the controller being relieved. 

•	 TM#2: Time relieving controller assumed position. 

The time when the relieving controller takes responsibility for the position was defined as (a) when the relieving 

controller makes a statement or otherwise indicates to 

the controller being relieved that position responsibility 

has been assumed and (b) the controller being relieved 

releases the position (FAA, 2004).

•	 TM#3: Time controller being relieved left position. 

The time when the controller left position was defined 

as the time when the controller unplugged his/her headset from the position. Generally, the time at which the 

controller is relieved of position responsibility is formally 

defined as the time when the controller being relieved (a) 

signs off the position in accordance with existing directives or (b) otherwise indicates that the relief process is 

complete (FAA, 2004). In some situations, the controller 

being relieved may remain for a brief period and continue 

to observe and assist with traffic demands. During certain 

times, such as those of high workload or complex traffic 

situations, the controller being relieved sometimes stays 

at the position to assist the relieving controller after the 

transfer of position responsibility has been accomplished. 

This overlap can be mandated, encouraged, or dispensed 

with, depending upon the facility or circumstances. 

Therefore, for our definition of TM#3, we considered the 

relieved controller to have left the position when he/she 

unplugged his/her headset from the position.

One number is not sufficient to distinguish all of 

the types of TMs which were identified. Some TMs 

tend to occur together and certain types of TMs tend 

to be repeated throughout the course of an OE. Thus, 

additional codes were needed for each TM and so two 

additional codes were systematically assigned following 

an underscore after the TM sequence number. We put an 

underscore after the first number to distinguish it from 

the next characters in the TM# string if that character 

was also a number.

The numbers and letters after the underscore indicate 

which TMs in an OE are repeated and which are related. 

Repeated instances of a TM are signified by adding a 

number series (i.e., “1,” “2,” “3,” and so on) after the 

underscore to indicate the TM’s sequence relative to that 

type. Thus the “1” after the underscore represents the first 

specific instance of that TM type. A “2” in that position 

after the underscore represents the second instance of 

that TM type. 

The second character after the underscore is a letter. 

Letters were used to signify one element of a TM pair 

when two TM types are sequential, related activities. The 

letters “a” and “b” are used to indicate that two TMs tend 

to occur coupled, such as when an activity is initiated 

(i.e., “a”); its completion is also included (i.e., “b”). That 

is, if one occurs the other will also usually occur as shown 

in the example below.

TM # Temporal Markers for Center and Terminal 

(Radar)

4_1a Time of 1st Inter/Intra facility coordination initiated 

on the involved A/C. 

4_1b Time of 1st Inter/Intra facility coordination 

completed on the involved A/C. 

The shaded rows in Table 1 highlight the TMs that 

typically occur together, either as groups of TM sequences 

(i.e., TM#1, TM#2, TM#3) or as pairs, shown by the 

“a” and “b” (e.g.., TM#4_1a, TM#4_1b). Using TM# 

as an example, the “4” indicates that this TM is fourth 

in the TM framework’s sequence. The “1” following the 

underscore indicates that this is the first occurrence of 

this pair. The “a” and “b” indicate that this is a pair of 

TMs that can be expected to occur together. These are 

collectively interpreted as the first occurrence of the TM#4 

pair. This coding scheme is useful when some types of 

TMs can clearly be grouped (e.g., the position relief) or 
7

separated into specific elements (e.g., a controller-pilot 

communication exchange). For example, a clearance issued by the receiving controller to the pilot can be divided 

into two elements: the controller’s communication and 

the pilot’s response. Coding for this event is shown by 

TM#9, which indicates that this TM pair is the 9th type 

of TM in the framework, the first occurrence of this type 

(controller-pilot communication). The “a-b” designation 

distinguishes (a) the controller’s instruction to the pilot 

from (b) the pilot’s response to the controller.

1. Time of 1st ATC instruction by receiving controller

(TM 9_1a). 

2. Time pilot verbally responded to 1st ATC instruction 

from receiving controller (TM 9_1b). 

As described above, some TMs may occur once for an 

OE (e.g., TM#1-3; the relieving controller took position 

responsibility after a briefing from the controller being 

relieved). Other types can be repeated (e.g., Time of 1st / 

2nd / 3rd /etc. controller-pilot communications) and, thus, 

instances of a type may be repeated during the course of 

an OE’s evolution. All TMs that are relative to aircraft can 

apply to each aircraft of interest. Therefore, no character 

was included in the temporal marker code to specify individual aircraft. The aircraft columns in the framework 

are used to distinguish TMs for the separate aircraft.

Although listing every communication with the 

involved aircraft might at first seem unwieldy or unnecessary, the cost-benefit tradeoff favors including them. 

The current requirement (FAA, 2002) for examining the 

unfolding of an OE is to examine the interval beginning 

five minutes prior to the involved aircrafts’ initial contact 

with the controller having the OE (in our example, the 

Alpha sector controller) and concluding five minutes 

after separation is lost between the two aircraft in the 

OE. Gathering the data and transcribing the voice tapes 

to obtain a timeline of the event is work intensive for the 

facility staff. One benefit from including all communications with the aircraft in the TM framework is that 

the TM structure, by providing a standardized format 

for the timeline (which must be transcribed anyway), 

would provide elaborative information about exchanges 

and activities between the controller and the aircraft 

involved with the OE beyond that which is currently 

available. This would be valuable both for analysis of the 

individual OE case and when summarized at facility- or 

system-wide levels. 

Discussion

Using TMs

Once the TMs in the template are populated with 

data from the OE, sets of TMs can be extracted from 

the framework to examine specific types of situations and 

activities. For example, our SMEs pointed out that there 

is a point in every OE when separation is not ensured for 

two aircraft, and if nothing interceded to change their 

trajectories, they would eventually lose separation. These 

two concepts need to be clearly distinguished: (1) when 

separation is not ensured versus (2) when separation is 

lost. They might seem similar in meaning but, in reality, 

they can differ markedly. The first, “when separation is 

not ensured,” means that the aircraft are on courses that 

will lead to an eventual loss of separation if nothing else 

happens in the meantime, such as another action by a 

controller or a pilot. Two aircraft can be assigned headings 

and/or altitudes that will eventually result in an OE several 

minutes (or miles) before the OE. Thus, events might 

be set in motion, and an OE could essentially begin to 

develop in another sector prior to the one where it finally 

takes place. On the other hand, “loss of separation” means 

that the aircraft were already too close at the time and 

were in a conflict situation where less than minimum 

separation between them existed. Thus, “separation is 

not ensured” can be thought of as the projection of the 

current traffic situation into the future, whereas “loss of 

separation” is the outcome. 

Analysts might want to look upstream from when the 

OE occurred to see whether earlier actions could be taken 

to avoid the OE. Because TMs can be uniquely assigned 

to each aircraft and because TMs are independent of the 

controller’s airspace boundaries, this TM could occur 

before one or both aircraft are in the receiving controller’s 

airspace (where the OE occurred), even though they may 

have already been handed off to the receiving controller 

(who “got the deal”) with a TM sequence as follows.

1. TM#10: Time of first point where positive separation between A/C (e.g., A/C1 and A/C2) was not 

ensured.

2. TM#12: Time conflict alert activated.

3. TM#14: Time separation lost.

4. TM#16: Time separation reestablished.

Depending upon when TM#10 occurred, different 

strategies for mitigating future OEs of the same type are 

revealed. For example, controllers are expected to demonstrate good control judgment and to maintain awareness and separation of traffic. The controller using good 
8

control judgment would not place aircraft in hand-off 

status in conflict or in a situation leading to possible loss 

of separation and would ensure that separation standards 

are maintained at all times. This includes ensuring that 

aircraft entering and departing his/her sector airspace are 

not in conflict or about to lose separation. The controller 

would also use procedures that do not place workload and 

stress on other controllers or facilities. Consider the different mitigating strategies suggested by two situations.

Variation A – TM#10 occurred prior to the handoff

The controller is required to “transfer control of an 

aircraft only after eliminating any potential conflict 

with other aircraft for which you have separation responsibility” (FAA Order 7110.65, 2004, Para 2−1−15), 

“[e]nsure that, prior to transferring communications: 

potential violations of adjacent airspace and potential 

conflicts between aircraft in their own area of jurisdiction 

are resolved” (FAA, 2004, Para 5−4−5) and “[r]estrictions 

issued to ensure separation are passed to the receiving 

controller” (FAA, 2004, Para 5−4−5).

However, this only implies that potential conflicts 

between aircraft in the receiving area of jurisdiction are 

resolved prior to transferring communications. In other 

words, the transferring controller should not have aircraft 

established on a conflicting course when transferring them 

to the receiving controller. If a controller were to transfer 

aircraft to another sector controller without ensuring that 

their separation would still be established for the receiving 

controller, the aircraft would be on a conflicting course but 

not in conflict when handed off. One might say, in other 

words, that an OE would be a “set up” for the receiving 

controller, who would receive a “dirty” traffic situation. 

In these cases, the receiving controller inherits a potential 

OE situation that was set-up by the previous controller 

and ultimately may “get a deal” unless he/she realizes 

after taking the handoff that the aircraft are on a conflicting course and takes steps to ensure their separation. 

Mitigation, in these cases, could target the procedures, 

controller performance and other factors related to the 

controller who handed off the aircraft, rather than (or 

in addition to) those of the controller in whose airspace 

the OE occurred.

Variation B – TM#10 occurred after the handoff 

After receiving aircraft, the controller is required to 

maintain appropriate separation standards between them 

(FAA, 2004). When a controller accepts the handoff of 

aircraft that have separation ensured, the traffic situation 

is handed off “clean,” and the receiving controller must 

maintain separation standards between them. If the aircraft are then set on a conflicting course (either by the 

controller’s action or inaction) and lose separation, then 

the receiving controller still “has the deal.” However, in 

this case, the aircraft were not on a conflicting course 

or in conflict when handed off but they lost separation 

while in the receiving controller’s airspace. In other words, 

the situation was handed off “clean,” but the receiving 

controller made it “dirty.”

To better understand OEs, analysts would like to 

know more about how events unfold, e.g., how often 

do certain types of events happen and does one type 

make an OE more likely to occur than another? This 

temporal knowledge could then be combined with other 

information (such as aircraft types, weather, procedures) 

to identify strategies to improve the system. 

Summary of Step 1

The TM framework developed in Step 1 provides a 

method to structure an OE. By using objective temporal 

information to populate the framework, the result then 

can be used in different ways, depending upon the issue 

of interest. For example, in a case analysis of a single OE 

the framework could be used to clarify temporal relationships between a number of aircraft and the controller. 

Further, the organizing structure of the TM framework 

permits analysts to compare TMs across many OEs to 

identify possible systemic concerns.

In addition to identifying temporal markers, the 

sequenced nature of the framework could also be used 

to calculate intervals between TMs (a Temporal Marker 

Time Span; TMTS) for other patterns of interest, such as 

the TMTS between first point where positive separation 

was not ensured and time that separation was lost. With 

this in mind, the framework was tested to determine 

whether calculating these intervals would yield additional 

information.

Step 2 – Testing the TM Profile

Method

Participants

The three SMEs from Study 1 also participated in 

this activity.

Materials

A convenience sample of 29 OEs was used, which did 

not include the 6 OEs used in Study 1. All of the OEs 

came from one ARTCC. These were selected because both 

the final report (FAA Form 7210-3) and the SATORI 

re-creation were available for each in an existing archive. 

A voice transcript was developed for each OE from its 

SATORI re-creation so that the SMEs could use a hard 

copy of the voice recording. We provided each SME with 

copies of the TM framework, the final OE reports and 

the voice transcripts of each OE.
9

Procedure

The SMEs were instructed to fill out the TM framework for each OE using available information. The SMEs 

denoted aircraft identified from the OE materials as A/C1, 

A/C2, etc. according to the procedure described previously. The SMEs’ task was to identify from the available 

materials the specific times for each TM for the aircraft 

that lost separation. If the TM was the type that recurred 

(time of 1st ATC instruction by receiving controller to 

A/C1, A/C2, etc.), then they identified the times for each 

recurrence by using the notation method developed in 

Step 1 of the project to expand the TM framework (i.e., 

TM#9_1a, TM#9_2a, TM#9_3a and so on) to indicate 

temporal sequence. 

The SMEs worked independently to complete the 

TM framework for each OE. When the frameworks were 

completed, the TM times were compared for inconsistencies between SMEs. The SMEs were then reassembled 

to resolve the discrepancies. Their consensus generated 

the final dataset of TMs. 

Results

The TM framework can be used to temporally profile one OE (i.e., for a case analysis) or several (i.e., for 

a trend analysis). The dataset of TMs identified by the 

SMEs was used to calculate time spans between TMs 

(TMTSs) for each OE. Averages across OEs were then 

calculated for several TMTS to determine if this approach 

would reveal more about OEs beyond currently available 

information. 

Examples are shown in Table 3 through Table 6. In 

each table, the first time used to calculate the TMTSs is 

the TM in the table’s caption. The second time used to 

calculate a TMTS is the TM shown in the body of the 

table (i.e., a, b, c, d). The number is the calculated time 

spans between the time of the first TM and second TM 

(in hours : minutes : seconds). The tables show the average 

time span and the median3 for the intervals. Minimum 

and maximum intervals were calculated for some examples 

to determine if they added additional information. The 

3 The median is the midpoint value where half the data values are 

above it and half are below it.

median value’s similarity to the average, minimum and 

maximum values suggests how the value of interest is 

distributed in the sample. For example, in Table 3, the 

median values of each TMTS are closer to the average 

values than to either the minimum or maximum values. 

However, Table 4 illustrates how the median TMTS can 

be lower than the average. This suggests that the analyst 

should examine the individual OEs to determine what 

situation resulted in this data. 

Table 3 shows some calculated time spans, beginning 

with the time the controller takes position responsibility. For example, row a) in Table 3 describes when the 

relieving controller first issued a control instruction to 

one of the aircraft that was later involved in the OE. The 

average TMTS (0:23:51) shows this action was taken 

within about 20 min of the controller taking over the 

position. Row d) of Table 3 shows that the average time 

interval from the time the controller assumed position 

responsibility to the time separation was lost averaged 

about 28 min (0:28:30) in this sample of OEs. These 

values are similar to the national data for the controller’s 

average time on position prior to an OE. However, the 

intervals ranged from 1 min 38 sec to 1 hr 27 min and 

32 sec in this sample.

Table 4 describes three TMTS to profile a communications pattern between the controller and one of the aircraft 

involved in the OE (A/C1). Row a) shows the average 

time between the receiving controller’s first instruction to 

A/C1 and the point at which separation was not ensured 

to have been relatively short (0:01:31). In some OEs in 

this sample, separation was not ensured between A/C1 

and A/C2 prior to the relieving controller’s first communication to A/C1, indicated by the negative value for at 

least one OE in the sample (-0:05:29). Negative values 

indicate that the second TM used in the TMTS analysis 

preceded the first TM in the OE.

Table 5 shows the average TMTS (0:01:30) from the 

time of the controller’s last instruction to A/C1 until separation was lost. Interestingly, for at least one OE in this 

sample, the interval from the controller’s last instruction 

to the aircraft to the separation loss was 8 min and 50 

sec, although most TMTS were very short, as indicated 

by the low median value (0:00:53) for this TMTS.

Table 3. TMTS from (TM#2) the time the relieving controller assumed position to — 

Average Median Min Max 

a) (TM#9_1a) Time of 1st ATC instruction by receiving 

controller to A/C1. 0:23:51 0:20:44 0:00:38 1:17:25 

b) (TM#10) Time of first point where positive separation 

between A/C was not ensured. 0:25:23 0:21:14 0:00:38 1:17:25 

c) (TM#11a) Time of the last instruction by receiving 

controller to A/C1 prior to loss of separation. 0:26:59 0:21:41 0:00:38 1:27:30 

d) (TM#14) Time separation lost. 0:28:30 0:22:38 0:01:38 1:27:32 
10

Table 4. TMTS from (TM#9_1a) 1st ATC instruction by receiving controller to A/C1 to — 

Average Median Min Max 

a) (TM#10) Time of first point where positive separation 

between A/C was not ensured. 0:01:31 0:00:53 -0:05:29 * 0:07:38 

b) (TM#11a) Time of the last instruction by receiving 

controller to A/C1 prior to loss of separation. 0:03:08 0:02:23 0:00:00 0:10:05 

c) (TM#14) Time separation lost. 0:04:38 0:04:19 0:00:31 0:10:07 

 * Note. The negative value indicates that TM#10 preceded TM# 9_1a in at least one OE. 

Table 5. TMTS from (TM#11a) Time of the last instruction by receiving controller to A/C1 prior to loss of 

separation to — 

Average Median Min Max 

a) (TM#14) Time separation lost. 0:01:30 0:00:53 0:00:01 0:08:50 

Table 6. TMTS from (TM#10) Time of first point where positive separation between A/C was not ensured to — 

Average Median Min Max 

a) (TM#11a) Time of the last instruction by receiving 

controller to A/C1 prior to loss of separation. 0:01:36 0:00:24 -0:05:55 * 0:10:05 

b) (TM#14) Time separation lost. 0:03:07 0:01:59 0:00:20 0:10:07 

 * Note. The negative values indicate that the second TM preceded the first TM in the OE. 

Table 7. TMTS from (TM#2) the time the relieving controller assumed position to — 

A/C1 (n OEs=29) A/C2 (n OEs=20) 

Average Min Max Average Min Max 

a) (TM#9_1a) Time of 1st instruction by 

receiving controller. 0:23:51 0:00:38 1:17:25 

b) (TM#10)* Time of 1st point where positive 

separation between aircraft is not ensured. 0:25:23 0:00:38 1:17:25 0:25:23 0:00:38 1:17:25 

c) (TM#9_1a) Time of 1st instruction by 

receiving controller. 0:28:1 0:07:44 1:26:53 

d) (TM#11a) Time of the last instruction by 

receiving controller prior to loss of 

separation. 

0:26:59 0:00:38 1:27:30 

e) (TM#14)* Time separation was lost. 0:28:30 0:01:38 1:27:32 0:28:30 0:01:38 1:27:32 

f) (TM#11a) Time of the last instruction by 

receiving controller prior to loss of 

separation. 

 0:31:03 0:09:49 1:26:53 

* Note. Shaded rows indicate TMs that involve both aircraft. 
11

Table 6 shows two TMTS calculated, beginning with 

the time when separation between the aircraft was not 

ensured (i.e., if nothing intervened to change the trajectory 

of either aircraft, they would eventually lose separation). 

As shown in rows a), the average interval between that 

time and the controller’s last instruction to one of the 

aircraft was 1 min and 36 sec. However, looking at the 

minimum and maximum values shows that, in at least 

one OE in this sample, the controller’s last instruction 

to one of the aircraft involved in the OE occurred 5 

min and 55 sec before separation was not ensured. At 

the other extreme, in at least one OE in this sample, the 

controller’s last instruction to one of the aircraft involved 

in the OE occurred 10 min and 05 sec after separation 

was not ensured. 

If data from more OEs were available for TM profiling, other analyses could be conducted. For example, 

OEs could be grouped by other causal factors of interest, 

e.g., failure by the controller to detect displayed data, the 

controller’s time on position, combined positions, weather 

conditions and traffic complexity factors. TMTS could 

then be calculated for these groups to determine whether 

their TMTS profiles differed. 

OE Profiling

The TM framework can be used for case analysis to 

understand the temporal evolution of one OE (e.g., patterns of communications between the controller and the 

aircraft) or for trend analysis to understand patterns across 

a group of OEs (e.g., factors related to “dirty” situations 

handed off to receiving controllers). By generating TMTS 

profiles between events that occurred during the OE relative to the interaction of the controller and each aircraft 

(A/C1, A/C2, A/C3, etc.), the results can be compared 

side-by-side as in Table 7, which shows average TMTSs 

between some of the TMs in this set. For example, the 

controller’s first instruction to A/C1, on average, was 

0:23:51 min after taking position but before the first 

point where no positive separation was ensured, which 

occurred 0:25:23 min after taking position. A/C2 was 

given its first control instruction, on average, approximately 5 min later, which occurred 0:28:18 min after 

taking position but after the first point where positive 

separation was not ensured.

The controller’s last instruction to A/C1, on average, 

was 0:26:59 min after taking position and before separation was lost (at 0:28:30 min after taking position). 

The controller’s last instruction to A/C2, on average, 

was 0:31:03 min after taking position. However the last 

instruction “prior” cannot also occur “after.” This is logically inconsistent and is a statistical artifact of using small 

samples. Analysts should look instead at other statistics 

such as the median value or mode.4 This suggests that 

TMs should either be used for case analysis of individual 

traffic situations or with large samples. Future work will 

tease apart the aspects of this phenomenon as it pertains 

to OE profiling.

Summary of Step 2

In this activity, we demonstrated how time intervals 

calculated from TMs could be used to profile the temporal 

evolution of an OE. The project relied on a convenience 

set of archival OE reports. Issues of missing data and 

small sample sizes related to using archival OE materials were identified as well as future projects for this line 

of research.

Project Summary

We identified a set of TMs that could potentially occur 

in every OE. Moreover, all but one TM can be identified 

using sources of objective data available at the AT facility. Subjective assessments by the facility’s staff are not 

needed to use this method of OE analysis. 

In Step 1 of this project, we developed a framework 

of TMs to organize types of objective, time-based information about OEs that are routinely available at ATC 

en route facilities but which have not previously been 

systematically organized in this manner. 

In Step 2, we tested the framework developed in 

Step 1 by using it to organize information from a small 

convenience sample of archival reports of en route OEs. 

TMs were identified for each OE and several TMTSs 

were calculated to demonstrate the utility of the methodology. Results demonstrated how intervals between 

TMs could provide additional information about how 

OEs evolve over time. Which two TMs are selected to 

calculate a TMTS depends on which dimensions of the 

OE interest the analyst.

Any trends in TMs and TMTSs which appear in OEs 

could help the OE analyst to detect potential system 

vulnerabilities. For example, one might want to identify 

trends in the following TMs and TMTSs. 

Temporal Markers, such as:

•	 the point when the two aircraft were first set up on a 

conflicting course (that is, separation was not ensured 

if no one interceded and intervening actions would 

be required to maintain separation). 

•	 the time when the receiving controller established 

communication with the aircraft. 

4 The mode is the value in the sample that occurred most 

frequently.
12

Time Spans, such as: 

•	 the time interval from when the controller assumed 

the position relative to when separation was not 

ensured.

•	 the time interval between when separation was not 

ensured until separation was lost. 

•	 the time interval between the onset of evasive actions 

and when separation was lost.

While this project was conducted to explore the 

possibility of using objective, temporal data to better 

understand OEs, several interesting questions emerged 

from this small sample. One trend worth exploring is 

whether there is a high frequency of situations (a) where 

controllers “set up” an OE for the next position by handing off a “dirty” situation to the receiving controller or (b) 

whether most OEs are received “clean” but the receiving 

controller then makes it “dirty” by either ensuing actions 

or inactions. Finding that aircraft are handed off with 

confliction could have several implications for mitigation 

and reduction. If the TM profile of an OE shows that the 

first point where aircraft were set on a conflicting course 

occurs at a particular time (such as prior to the handoff, 

within the first 5 min of the controller’s time on position, 

prior to establishing communications with the aircraft), 

the circumstances at that time could be examined. 

Analysts could examine other interesting issues - for 

instance, whether aircraft involved in OEs have other 

temporal characteristics in common, such as time from 

checking on frequency to first control instruction, differences in profiles for OEs occurring during the first 

few minutes after a position transfer or OEs with similar 

causal factors. 

In sum, using TMs and TMTSs to analyze OEs would 

permit analysts to address several interesting types of 

questions, such as the following. 

•	 Handoff situations: Did the responsible controller set 

up the conflict or inherit it? The controller handing 

off an aircraft must resolve any conflicts prior to the 

handoff to the receiving controller. 

•	 Recovery strategies: Did the aircraft come to the receiving controller already in a confliction pattern (“dirty”) 

so that the controller had to become a “fixer?” If the 

conflict was set up before the receiving controller accepted the aircraft, strategies of the receiving controller 

to fix separation could be identified, contributing conditions could be identified, handoff procedures could 

be reviewed, performance of the handoff controller 

could be evaluated, etc..

•	 Performance and other factors: Did the aircraft situation come to the controller “clean” and the situation 

became “dirty” from either actions or inactions by the 

controller? If the conflict was set up after the controller 

accepted the aircraft, then the receiving controller did 

not ensure separation and the reasons for that situation 

could be examined.

•	 Controller-pilot factors: When was controller-pilot 

communication established, if at all? TM profiles 

might show that aircraft involved in OEs were more 

(or less) likely to have communications established 

between controller and pilot.

•	 Communication loads: What was the pattern of controller-pilot communications before, during and after 

the OE? For example, the number of communications 

between the controller and the pilot(s) might be an 

indicator of workload or of controller efficiency. TM 

profiles might show that OEs differ from non-OE 

situations by having a higher (or lower) rate of pilotcontroller communications.

This approach to analysis could be extended to a system-wide application. Other interesting questions and 

temporal trends are likely to be identified as larger data 

samples are analyzed. These trends may provide guidance 

for the display of additional information and formatting 

and may have implications for the design of enhanced 

decision support capability. 

Limitations

Convenience sample 

As noted earlier, the examples in this paper were based 

on a set of TMs from a small sample of ARTCC OEs. 

Inferences from small samples should be made carefully 

until the method can be tested on a larger set of OEs. 

For example, the average times calculated from a small 

number of cases makes generalization beyond the sample 

to other OEs difficult without verifying the results with 

a separate sample of OEs. Other measures, such as the 

median (midpoint) value, should also be examined to 

adjust for effects of outliers (i.e., extraordinary minimum 

or maximum values) on the average value. 

The small sample constrained us to look at the TMs 

and TMTSs only in a general way. Much more information would be available to identify meaningful trends if a 

larger set of OEs was analyzed using for this type of profile 

analysis. Trends evident in larger samples of OEs would be 

considered more robust, leading to greater confidence in 

mitigation strategies based on the results. A larger set of 

OEs would permit a better understanding of whether it 

makes sense to average over OEs without grouping them 

based on other similarities, such as causal factors. 

The ARTCC operational errors we used were selected 

because the event chronology and a SATORI re-creation 

were available for each OE. However, the SMEs judged 

that the list would also apply to OEs in Terminal Radar 

Approach Control facilities, since they are also radar 
13

environments, and possibly to OEs at air traffic tower 

facilities. This was not verified in this study in part because 

OE re-creations were not available. 

Missing data 

In this sample, data (and consequently, perhaps an 

important TM) could be missing for different reasons. 

The temporal event might have occurred during the OE 

but the original report omitted the necessary information, 

and it was not available from the SATORI for the event. 

Thus, the corresponding TM could not be identified 

later, resulting in missing data for that TM in that OE. 

Alternatively, a temporal event might not have occurred 

during the OE, resulting in missing data for that TM in 

that OE. In our sample, we cannot distinguish between 

these two types of missing data. Issues of missing data 

are not typically considered when researchers analyze OE 

data, but when relying on archival data only those TMs 

can be identified for which information is available. 

Future Work

Other event types

Because we wanted to use events in OEs that could be 

identified objectively as temporal markers, the result was 

that most TMs focused on the interactions between the 

controller and aircraft or between two or more aircraft. 

Future work to test the TM set will determine whether 

other types of events should be included as TMs - expedited maneuvers, altitude changes, heading changes and 

so on. Our definition of temporal markers constrains this 

possibility somewhat because any TM candidate must 

satisfy the general condition that these events may differ 

at the point in time they occur, but they occur in some 

form in most, if not all, OEs and must be “objective or 

calculable.” This is an important characteristic to enable 

comparison between OEs.

Large samples

To fully understand how to best exploit this method of 

analysis, it needs to be applied to a large set of OEs. For 

example, a large ATC facility could apply this Temporal 

Marker framework to all of its OEs for a year and profile 

OEs both individually and overall for both case and trend 

analysis. We could then gain a better understanding of 

OE profiling and be better able to evaluate whether 

controllers and management gained additional insights 

about OEs from this type of information. 

National trends

Currently, the chronologies developed at facilities 

after OEs are not required to be as detailed as the TM 

framework nor are they as highly structured. The TM 

framework could help to structure the OE chronologies 

if it were used as a tool to complete the final OE report. 

This would provide more complete archival information 

for OE analysis. A national database of TMs could be 

analyzed for trends in TMs and TMTSs to expand our 

knowledge about how OEs evolve so that vulnerable 

situations could be identified and mitigated.

Re-creations 

To exercise this approach to structuring objective events 

in OEs, a comprehensive re-creation capability is needed 

that would provide seamless visualization of traffic over 

time and across facility types. Currently the ARTCCs 

have the SATORI system, TRACONs have RAPTOR 

replays and some larger towers have the potential to replay 

traffic using the Airport Movement Area Safety System 

(AMASS). These are separate systems and have different 

levels of usability and timeliness of turnaround. Tower 

facilities would present the greatest challenge for this 

approach because they currently have limited capability 

to re-create traffic situations. An end-to-end integrated 

capability would enable a larger temporal understanding 

of the evolution of OEs (and potentially other types of 

events) that is not currently available.

Conclusions

In sum, we believe that using the TM framework will 

give operational personnel and researchers another tool 

to identify information about factors leading to OEs. 

By profiling OEs in this manner, this method provides 

an objective method for identifying structure in high 

consequence, complex, dynamic events. This approach 

provides a means to view OEs in a new light that has the 

potential to directly lead to mitigation activities and to 

ensure continual improvement in our understanding of 

OEs and the OE analysis process. 

References

Bier, V. (1998). Proceedings of workshop on accident 

sequence precursors and probabilistic risk analysis. College 

Park, MD: University Printing Services.

Block, R.A. Ed. (1990). Cognitive models of psychological time. Hillsdale, NJ: Erlbaum.

Dekker, S. (2002). The field guide to human error 

investigations. Burlington, VA: Ashgate.

Dörner, D. (1996). The logic of failure. Recognizing 

and avoiding error in complex situations. Reading, MA: 

Perseus Books. 

Federal Aviation Administration. (2002). Air traffic quality assurance. FAA Order 7210.56. Washington, 

DC: Author.
14

Federal Aviation Administration. (2004). Air traffic 

control. FAA Order 7110.65. Washington, DC: Author.

Gibbon, J., & Allan, L. Eds. (1984). Timing and 

time perception. New York: New York Academy of Sciences.

Harry, M., & Schroeder, R. (2000). Six sigma. 

New York: Currency. 

Klein, G.A., Orasanu, J., Calderwood, R., & 

Zsambok, C.E. (1993). Decision making in action: Models 

and methods. Norwood, NJ: Ablex. 

Krause, S.S. (2003). Aircraft safety: Accident investigations, analysis & applications (2nd ed.). New York: 

McGraw-Hill.

Perrow, C. (1999). Normal Accidents. Princeton, 

NJ: Princeton University Press. 

Petersen, D. (1996). Human error reduction and 

safety management (3rd ed.). New York: Van Nostrand 

Reinhold.

Phimister, J.R., Bier, V.M., & Kunreuther, H.C. 

Eds. (2004). Accident precursor analysis and management.

Washington, DC: National Academies Press.

Pounds, J., & Ferrante, A.S. (2005). FAA strategies for reducing operational error causal factors. In 

B. Kirwan, M. Rodgers, & D. Schäfer (Eds.), Human 

factors impacts in air traffic management (pp. 87-105). 

Burlington, VT: Ashgate. 

Rasmussen, J. (1980). What can be learned from 

human error reports? In K. Duncan, M. Gruneberg, & D. 

Wallis (Eds.), Changes in working life. London: Wiley.

Reason, J. (1990). Human error. New York: Cambridge University Press. 

Reason, J. (1997). Managing the risks of organizational accidents. Brookfield, VT: Ashgate. 

Rodgers, M.D., & Duke, D.A. (1993). SATORI: 

Situation assessment through the re-creation of incidents.

Washington, DC: Federal Aviation Administration Office 

of Aviation Medicine. DOT/FAA/AM-93/12. 

Rogers, M.D., & Duke, D.A. (1993). SATORI: 

Situation assessment through the re-creation of incidents.

Journal of Air Traffic Control, October-December, 10-

14.

Sagan, S.D. (1993). The limits of safety. Princeton, 

NJ: Princeton University Press.

Salas, E., & Klein, G. (2001). Linking expertise 

and naturalistic decision making. Mahwah, NJ: Lawrence 

Erlbaum.

Smolensky, M.W. (1993). Toward the physiological measurement of situation awareness: The case for eye 

movement measurements. Proceedings of the Human 

Factors and Ergonomics Society 37th Annual Meeting, 

Seattle, Washington. Santa Monica, CA: Human Factors 

and Ergonomics Society.