Logistic Regression Analysis of 

Operational Errors and Routine 

Operations Using Sector Characteristics

Elaine M. Pfleiderer

Cheryl L. Scroggins

Carol A. Manning

Civil Aerospace Medical Institute

Federal Aviation Administration

Oklahoma City, OK 73125

February 2009

Final Report

DOT/FAA/AM-09/4

Office of Aerospace Medicine

Washington, DC 20591

OK-09-0434
NOTICE

This document is disseminated under the sponsorship 

of the U.S. Department of Transportation in the interest 

of information exchange. The United States Government 

assumes no liability for the contents thereof.

___________

This publication and all Office of Aerospace Medicine 

technical reports are available in full-text from the Civil 

Aerospace Medical Institute’s publications Web site: 

www.faa.gov/library/reports/medical/oamtechreports


Technical Report Documentation Page 

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No. 

DOT/FAA/AM-09/4 

4. Title and Subtitle 5. Report Date 

Logistic regression analysis of operational errors and routine operations February 2009 

using sector characteristics 6. Performing Organization Code 

 

7. Author(s) 8. Performing Organization Report No. 

Pfleiderer EM, Scroggins CL, Manning CA 

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS) 

FAA Civil Aerospace Medical Institute 

P.O. Box 25082 11. Contract or Grant No. 

Oklahoma City, OK 73125 

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered 

Office of Aerospace Medicine 

Federal Aviation Administration 

800 Independence Ave., S.W. 

Washington, DC 20591 14. Sponsoring Agency Code 

15. Supplemental Notes 

Work was accomplished under approved task AM- B07-HRR-522 

16. Abstract 

Two separate logistic regression analyses were conducted for low- and high-altitude sectors to determine whether 

a set of dynamic sector characteristics variables could reliably discriminate between operational error (OE) and 

routine operation (RO) traffic samples. OE data were derived from SATORI re-creations of OEs occurring at the 

Indianapolis Air Route Traffic Control Center between 9/17/2001 and 12/10/2003. RO data were extracted 

from System Analysis Recordings (SARs) taped between 5/8/2003 and 5/10/2003. Dynamic sector 

characteristics submitted as potential predictors were: Average Control Duration, Number of Handoffs, Number 

of Heading Changes, Number of Intersecting Flight Paths, Number of Point Outs, and Number of 

Transitioning Aircraft. In the low-altitude sector model, backward stepwise elimination reduced the variables to 

the Number of Intersecting Flight Paths, the Number of Point Outs, and the Number of Handoffs with 75% 

overall classification accuracy. In the high-altitude sector model, backward stepwise elimination reduced the 

variables to the Number of Intersecting Flight Paths, the Number of Heading Changes, the Number of 

Transitioning Aircraft, and Average Control Duration with 79% overall classification accuracy. Classification 

rates achieved through the use of the selected sector characteristics support the assumption that elements of the 

sector environment contribute to the occurrence of OEs. Continued investigations along these lines may 

highlight complexity factors that should be addressed to ensure that separation is maintained. 

 

17. Key Words 18. Distribution Statement 

 

 

Air Traffic Control, Sector Characteristics, Complexity, 

Operational Errors, Logistic Regression Analysis

Document is available to the public through the 

Defense Technical Information Center, Ft. Belvior, VA 

22060; and the National Technical Information 

Service, Springfield, VA 22161 

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price 

Unclassified Unclassified 21 

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized 

iii

Contents

Logistic Regression Analysis of Operational Errors and Routine Operations Using Dynamic

Sector Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Predictor Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Exclusion of Static Predictor Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Data Sources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Transitioning Aircraft and Heading Change Adjustments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Preliminary Logistic Regression Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Logistic Regression Analysis: Low-Altitude Sectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Logistic Regression Analysis: High-Altitude Sectors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Discussion. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10

Low-Altitude Sector Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

High-Altitude Sector Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Future Research. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Appendix A: Indianapolis Air Route Traffic Control Center Traffic Samples. . . . . . . . . . . . . . . . . . . . . . . . . . . A1



Executive Summary

Althougha considerable amountofresearchhasfocused

on the relationship between sector characteristics and

controller workload or perceived complexity, relatively

few studies have examined the relationship between

sector characteristics and the occurrence of operational

errors (OEs). In many early studies of OE causal factors,

examinations of sector characteristics were limited to

purely theoretical relationships or to traffic counts and

altitude transitions of the involved aircraft. When sector

characteristics were examined, they consisted primarily

of ratings rather than objective measures. Most of this

work has been conducted without reference to routine

operations (ROs). Yet, for every OE that occurs in a

sector, there are hundreds (possibly thousands) of hours

in which an OE did not occur. To truly understand the

environmental and contextual factors that contribute to

OEs, it is necessary to identify what was different about

the sector environment at the time the OE occurred.

In the present study, OE and RO traffic samples were

compared using logistic regression analysis. Two separate

logistic regression analyses were conducted for low- and

high-altitude sectors. The OE sample was drawn from

119 OEs occurring in the Indianapolis en route airspace

from 9/17/2001 through 12/10/2003. Of these, 40 occurred in the low-altitude sectors and 79 occurred in

the high-altitude sectors. The RO traffic samples were

recorded between 5/8/2003 and 5/10/2003. These

data were processed in 5-minute intervals using custom

software designed to calculate objective measures from

routinely recordedNationalAirspaceSystem(NAS)data. 

This produced a total of 2763 RO traffic samples. The

79 high-altitude OE traffic samples were combined with

79 randomly-selected high-altitude RO traffic samples

to produce a total of 158 traffic samples for the highaltitude sector analysis

In logistic regression analysis of the low-altitude sector samples, Average Control Duration, the Number of

Handoffs,theNumberofHeadingChanges,theNumber

of Intersecting Flight Paths, the Number of Point Outs,

andtheNumber ofTransitioningAircraftweresubmitted

asthe initialset of predictors. Backward stepwiseelimination reduced the variables in the final low-altitudesector

model to the Number of Intersecting Flight Paths, the

Number of Point Outs, and the Number of Handoffs. 

In the low-altitude sector model, each intersecting flight

path increased the likelihood that the traffic sample was

an OE by 189%,each point out increased thelikelihood

by 57%, and each handoff increased the likelihood that

the traffic sample was an OE by 19%. Of the 40 ROs in

thelow-altitudesample,32(80%)were correctly classified

and 8 (20%) were misclassified as OEs. Of the 40 OEs

in the sample, 28 (70%) were correctly classified and 12

(30%)weremisclassifiedasROs. Overall,thelow-altitude

model had 75% classification accuracy.

In logistic regression analysis of the high-altitude sector samples, Average Control Duration, the Number of

Handoffs,theNumberofHeadingChanges,theNumber

of Intersecting Flight Paths, the Number of Point Outs,

and the Number of Transitioning Aircraft were submitted as the initial set of predictors. Backward stepwise

elimination reduced the variables in the final high-altitude sector model to the Number of Intersecting Flight

Paths, the Number of Heading Changes, the Number of

Transitioning Aircraft, and Average Control Duration. 

In the high-altitudesector model,each one-unit increase

in the Number of Intersecting Flight Paths increased the

likelihood that a traffic samplewas an OE by 100%,each

one-unit increase in the Number of Heading Changes

increased the likelihood by 36%, every Transitioning

Aircraft increased the likelihood by 27%, and each onesecond increase in Average Control Duration increased

thelikelihood by 1%. Of the 79 ROs in the high-altitude

sample, 64 (81%) were correctly classified and 15 (19%)

were misclassified as OEs. Of the 79 OEs in the sample,

60 (76%) were correctly classified and 19 (24%) were

misclassified as ROs. Overall, the high-altitude model

had 79% classification accuracy.

The results of the logistic regression analyses indicate

that sufficient models may be constructed from sector

characteristics variables. Overall classificationaccuracybetween75-79% isremarkableformodels constructedsolely

of environmental and contextual factors. After all, other

factors (e.g., human elements, organizational influences)

also contributeto the occurrence of OEs. Unfortunately,

all thelogistic regression modelswere better at classifying

ROs than OEs. Classification of OEs ranged from as low

as 70% in the low-altitude sector sample to 76% in the

high-altitude sample. Although this level of accuracy

would be unacceptable for most automation tools, it is

unrealistic to expect definitive results from one or two

analyses. Moreover,thesector characteristic variablesused

in these analyses do not represent an exhaustive list of all

the potential predictors of OEs.

Although logistic regression cannot be used to identify

causal factors directly (i.e., prediction is not the same as

causation), thelogistic regression coefficients do provide

information about the likelihood of an OE relativeto the

predictors in the model. Thus, the results have immediate heuristic value in that they invite questions about

how the dynamic predictors interact with static sector
vi

characteristics. Dynamic elements lend themselves to

automationapplications,butstatic characteristicsmustbe

addressedforsectorrestructuring. Thedynamicpredictors

that make up thelogistic regression models are indicants

of conditions that discriminate between OEs and ROs. 

These indicantsmay be used to reveal aspects ofthesector

environment that might be altered to reducethe number

of OEs. For example, the combination of the Number

of Point Outs and the Number of Handoffs in the lowaltitude sector model may indicate that the location of

sector boundaries increases coordination workload and

complexity. On the other hand, the combination of the

Number of Point Outs and the Number of Intersecting

Flight Paths may point to problems with the orientation

of traffic paths relative to those boundaries. The combination of the Number of Heading Changes and the

Number of Transitioning Aircraft in the high-altitude

sectors is suggestive of traffic complexity in high-altitude

sectors adjacent to low-altitude arrival or departure sectors. Average Control Duration may be a function of the

size of high-altitude sectors.

Given the sample size and consequent restriction of

the predictor set, there is no guarantee that these results

will generalize to other samples. Multiple studies, with

samplessizesthat allowfor a more inclusivelist of predictors, must be conducted at a number of facilities before

such models might be reliable enough for practical applications. Nevertheless, the methodology of comparing

OE and RO traffic samples is promising. Continued

investigations along theselinesmay highlight complexity

factors that must be addressed to ensure that separation

is maintained.
Logistic Regression Analysis of Operational Errors and Routine

Operations Using Dynamic Sector Characteristics

Environmental Systems(ACES), average complexity and

densityratingsdrawnfromthe1995annualfacilityreview,

and complexity factor ratings provided by one Airspace

and Procedures specialist from each area in the facility. 

Several statistical techniques were employed to predict

OE occurrence in the Atlanta sectors. In a linear multiple

regression analysis, ratings of radio frequency congestion

and the effects of restricted airspace accounted for 31%

of the variance in the number of OEs per sector. The

same two predictors generated a function that classified

OE frequency groups (i.e., sectors with no OEs, fewer

than four OEs, and sectors with four or more OEs) with

58%accuracy inadiscriminantfunctionanalysis. Ratings

of weather influences, radio frequency congestion, and

complexityweresignificantly different in aMANOVAof

OE frequency groups. The authors stated that the overall

Hotellings F(16, 68) = 1.60, p = .094 of the MANOVA

was acceptable dueto theexploratory nature of thestudy

and emphasis on data exploration. Sector size failed to

emerge as a significant contributor in any of these models, but a t-test between the combined no- and low-OE

groups and the group consisting of sectors with four or

more OEs was statistically significant (p<.05).

In both the Grossberg (1989) and Rodgers, Mogford,

and Mogford (1998) studies, sector characteristics were

evaluated against the number of OEs per sector. Therisk

of OE occurrence increases as a function of the number

of hours a sector is in operation (i.e., exposure increases

the risk of occurrence). If OE incidence is not adjusted

for this, the internal validity of the analysis may be compromised. However, there is no evidence to suggest that

these adjustments were made in either study.

Another disadvantage ofOE incidence as a dependent

variable is that it restricts the number of independent

variables that may be analyzed without detriment to

external validity. With most statistical analyses, the risk

that results will fail to generalize increases as the number

of independent variables approachesthe number of cases. 

In spite of this, Rodgers, Mogford, and Mogford (1998)

submitted 28 predictors for stepwise linear multiple

regression analysis of OE incidence in 45 sectors at the

Atlanta ARTCC (i.e., a ratio of 1.6 cases foreach predictor). A correlation matrix of the predictors revealed 26

comparisons with r ≥ .60. The authors relied on tests

for tolerance in the stepwise procedure to eliminate

redundancies and reduce the number of variables in

the predictor set. However, in stepwise linear multiple

In the current Air Traffic Control (ATC) system, an

OperationalError(OE)occurswhenthere is a violationof

aircraftseparationminima as defined by FederalAviation

Administration regulations (FAA, 2007). Although the

FAA has called for a reinvention of the National Airspace

System (NAS), it is reasonable to assume that future

systems will also includesome kind of aircraft separation

standards. As changestoprocedures andairspacestructure

are being considered, it becomes increasingly important

to understand the environmental and contextual factors

(often referred to as sector characteristics) that contribute to the loss of separation to ensure that safety is not

compromised.

A considerable amount of research has focused on the

relationship between sector characteristics and controller

workload or perceived complexity (e.g., Buckley, DeBarysche, Hitchner, & Kohn, 1983;Chatterji & Sridhar,

2001;Christien & Benkouar, 2003;Kopardekar &Magyarits, 2003; Laudeman, Shelden, Branstrom, & Brasil,

1998;Mogford,Murphy, & Guttman, 1994;Robertson,

Grossberg, & Richards, 1979). However, relatively few

studies have examined the relationship between sector

characteristics and the occurrence of OEs. In many early

studies of OE causal factors,examinations of sector characteristicswerelimited to purely theoreticalrelationships

(e.g.,Arad, 1964; Schmidt, 1976) orto traffic counts and

altitudetransitionsofthe involvedaircraft(e.g.,Kershner,

1968; Schroeder, 1982; Spahn, 1977).

Grossberg (1989) expanded on this by collecting

ratings from 97 controllers and supervisors regarding

various aspects of the sector environment. He found

that the highest-rated sector complexity factors in the

Chicago Air Route Traffic Control Center (ARTCC)

were control adjustments (e.g., merging, spacing, and

speed changes), climbing and descending flight paths,

and mix of aircraft types. An index based on thesefactors

was significantly correlated (r = .74) with the number of

OEs occurring in 27 Chicago sectors over 21 months in

1987 and 1988.

Rodgers, Mogford, and Mogford (1998) evaluated

the relationship between sector characteristics and the

incidence of OEs at the Atlanta ARTCC. OE data were

extracted from Facility Operations and Administration

reports for 85 OEs occurring over a three-year period

from 1992 to 1995. Sector characteristics variables were

derived froma number ofsources and included static sector characteristics from the facility’s Adaptation Control
regression, a minimum of 40 cases for each predictor is

necessary to ensure the solution will generalize to other

samples (Tabachnick & Fidell, 2006).

Perhaps the biggest drawback to OE frequency as a

dependent measure is that it affords no comparison with

routine operations (ROs). Yet, for every OE that occurs

in a sector, there are hundreds (possibly thousands) of

hours during which an OE did not occur. Variables that

correlate with sector OE frequency do not describe what

was different about the sector at the time of the OE. 

To truly understand the environmental and contextual

factors that contribute to OEs, it is necessary to identify

what was different about the sector environment at the

time the OE occurred.

Pfleiderer and Manning (2007) conducted an investigation to determine whether logistic regression analysis

of objective sector characteristics could discriminate

between OE and RO traffic samples. Prior to this study,

investigations ofthe contributionofsector characteristics

to the occurrence of OEs relied heavily on attributed

causalfactors and subjectiveratings. Although interviews

and ratings about the importance of complexity factors

are beneficial in the early stages of research, practical

prediction models must eventually be calculated from

objectivemeasures. After all, it isthe actual characteristics

of the sectors that must be addressed when developing

strategies to reduce OEs rather than opinions or beliefs

about those characteristics. Two separate logistic regression analyses were performed for high- and low-altitude

sector samples at the Indianapolis ARTCC (ZID). In

the high-altitude sector sample, a logistic regression

model comprising the Number of HeadingChanges, the

Number of Transitioning Aircraft, and Average Control

Duration was able to accurately classify 80% of the

OE and RO traffic samples. In the low-altitude sector

sample, variables included in the final model were the

Number of Point Outs, the Number of Handoffs, and

the Number of Heading Changes. This model was able

to accurately classify 79% of the low-altitude OE and

RO traffic samples.

Unfortunately, the Pfleiderer and Manning (2007)

study was flawed. Available traffic data consisted of OEs

from9/17/2001to12/10/2003andROsfrom2/25/2005

to 3/3/2005. Clearly, the time differential between the

OE and RO traffic samples was a confounding influence

because it represented an uncontrolled, systematic difference between the two groups. 

A second problem with the Pfleiderer and Manning

(2007)design involvedpairingOEandROtraffic samples

(by sector, day of week, and time of day). Logistic regression analysis assumes that all cases are independent

of one another. Violation of this assumption results in

over-dispersion, which produces an inflatedType I error

rate for tests of predictors (Tabachnick & Fidell, 2006). 

Although there is no reason to assume that errors would

be correlated between the OE and RO traffic samples

(particularly in light of the disparate time frames from

which thesamples were drawn), neither is thereevidence

to the contrary. Random selection of RO traffic samples

would have guaranteed that the assumption of independence had been met.

Another possiblesource of inflation fortests of predictors involved the variables Number of Heading Changes

and Number ofTransitioning Aircraft. When controllers

become aware that an OE is developing, they may issue

clearances in an attempt to resolve the situation. Inclusion of altitude or heading changes made in response to

such clearances would inflate the contributions of the

Number ofTransitioning Aircraft (based on the number

of altitude changes) andtheNumberofHeadingChanges

as predictors of OEs. The choice of processing interval

(four minutes prior to loss of separation and one minute

after) may have magnified this effect.

In the present study, OE and RO traffic samples are

againcomparedusinglogistic regressionanalysis,butsome

important modifications were made to the design. Both

the OE and RO traffic samples used to build the logistic

regression model were from 2003, thus eliminating the

confound of the previous analysis. A randomly-selected

sub-sample of the RO traffic data was designated for

model building. The remaining RO cases and traffic

samples from OEs occurring in the ZID airspace from

2001-2002 were set aside for cross validation. No attempt was made to match the RO traffic samples to the

OE samples, thus meeting the assumption of independence and eliminating the potential for Type I errors for

tests of predictors. The Number of Heading Changes

and Number of Transitioning Aircraft were adjusted to

eliminate changes made in responseto control actions to

avoidan imminentOE(described indetail intheMethod

section) to guard against inflated contributions by these

two predictors. In addition, the processing interval for

the OE traffic samples was changed to include only the

five minutes prior to initial loss of separation.

Unfortunately, restricting the OE traffic samples to

a single year reduces the sample size. This prohibits

separation of low- and high-altitude sector samples for

individual analyses,eventhoughthere isreasonto suspect

theyconstituteheterogeneoussub-samples. Tobeginwith,

OE incidence differs between strata. Of 119 OEs in the

full sample, 40 occurred in low-altitude sectors, and 79

occurred in the high-altitude sectors. Inclusion of sector

strata as a binary categorical variable would provide no

information beyond this fact. Second, separate logistic

regression analyses of thelow- and high-altitudesamples

produced two very different models in the Pfleiderer and
Manning (2007) study. It is possible that combining

low- and high-altitude sector samples would produce

a model that fits the high-altitude sectors poorly and

the low-altitude sectors not at all. Cross validation can

be used to determine whether the 2003 data are similar

enough to the 2001-2002 data to justify pooling the

OE samples. If OE classification is similar in the modelbuilding and cross-validation results, this suggests that

the model works equally well for both samples and that

they may be similar enough to be pooled. If that is the

case, the OE data will be pooled, and separate logistic

regression analyses will be conducted for the low- and

high-altitude sector samples.

Thesector characteristicsusedas independent variables

in this study are based on measures that have previously

demonstrated a relationship with workload, complexity,

or the occurrence of OEs. These variables are described

in detail in the following paragraphs.

Predictor Variables

Average Control Duration. Aircraft control duration

is influenced by a number of factors, including aircraft

performance characteristics, Traffic Management Initiatives (TMI), and sector size – all of which have been associated with sector workload or complexity (Grossberg,

1989; Mogford, Murphy, & Guttman, 1994; Pfleiderer,

Manning,&Goldman,2007). AverageControlDuration

is the mean of the durations (in seconds) of all aircraft

controlled by the sector within a processing interval. 

Control time occurring before or after the interval was

not included in the calculations.

Number of Handoffs. Althoughtraffic countremainsthe

best single predictor of the number of OEs on a national

level, previous research suggests that it is not an effective

predictor of OEs at thesector level (e.g. Schroeder, 1982;

Schroeder, Bailey, Pounds, & Manning, 2006; Spahn,

1977). Aside from doubts about its effectiveness as a predictor,perhapsthebiggestdrawbackto traffic count isthat

it tends to be highly correlated with other traffic-related

measures. As a result, traffic counts create redundancies

thatmay overshadowmoreeffectivepredictors. Handoffs,

however, are correlated with the number of aircraft in

the sector, but they are not synonymous with it. In the

ZID traffic samples used in this study (described in the

Method section), the total number of aircraft and the

total number of handoffs had a Spearman’s correlation of

rs = .66 (N = 2763). Handoff counts capture elements of

communicationworkloadandrequiredattention. Handoff

initiates (i.e., outbound handoffs from the current sector

to another sector or facility) are generally associated with

the issuance of a frequency change clearance. Handoff

accepts (i.e., inbound handoffs from another sector or

facility to the current sector) are accompanied by an

eventual(but not necessarily concurrent) verification that

aircraft coming intothesector aretunedtothe appropriate

frequency. Handoff counts alsoprovide informationabout

coordination and required procedures. Although most

handoffs are fairly automatic, some require coordination

with other sectors. In some instances, aircraft must comply with altitude or other restrictions before they can be

handed off or a handoff can be accepted. Handoffs may

also reflect the impact of sector geography. According to

Couluris and Schmidt (1973), the number of handoffs,

coordination, andpointouts “resultfrom,or are influenced

by, the existence and design (shape) of the sectors. The

additional work created can be thought of as the cost of

sectorization” (p. 657). The Number of Handoffs is the

total number of handoff initiates and handoff accepts

occurring within the 5-minute processing interval.

Number of Heading Changes. Heading changes have

demonstrated a relationshipwith controllerratings of activity (e.g., Laudeman et al., 1998),workload (e.g., Stein,

1985), and complexity (e.g., Kopardekar & Magyarits,

2003). Heading changes are involved with a number

of procedures such as merging and spacing, Standard

Terminal Arrival Routes (STARs), Standard Instrument

Departure Routes (SIDs), and holding. The Number

of Heading Changes is a count of all turns in excess of

10 degrees per 12-second radar update that continue in

the same direction for at least three updates. Heading

changes made in an attempt to avoid an imminent OE

were excluded from the counts for OE traffic samples. 

The criteria for these exclusions are described in the

Method section.

Number of Intersecting Flight Paths. This factor was

one of the highest rated complexity factors in the highaltitude and super high-altitude sectors in the Pfleiderer,

Manning, and Goldman (2007)study, inwhich a sample

of 32 controllers and 4 supervisors from ZID provided

ratings for a set of 22 sector complexity factors. In addition, the Number of Intersecting Flight Paths was

associated with a component (i.e., composite factor

score) that demonstrated a reliable relationship with

the number of OEs in the ZID sectors. A similar factor

(several traffic flows converging at the same point) was

among the top-rated complexity factors in an investigation of Maastricht airspace conducted by Eurocontrol

(2006). The Number of Intersecting Flight Paths is the

maximum number of flight pathsthat might beexpected

to intersect, irrespective of altitude, within a 10-minute

projected time frame given the current speed and trajectory of the aircraft. Projections were calculated at each

12-second radar update within each minute of data. The

length and slope of the projected paths were based on

the distance and angle of the current and previous radar

position coordinates.
Number of Point Outs. Coordination between controllers was one of theevents selected by Schmidt (1976) for

his Control Difficulty Index even though he considered

it to be one of the most difficult to process – with good

reason. Coordination is not often recorded. However,

point out entries represent one of the few instances in

which coordination between sectors is recorded. The

Number of Point Outs is the total number of point out

entriesmadeby theRadar andRadarAssociate controllers

during the 5-minute processing interval.

Number of Transitioning Aircraft. The amountof climb-

ing and descending traffic has long been recognized as

a contributor to the difficulty of working a sector (e.g.,

Arad, 1964). Grossberg (1989) observed that one of the

factorsmostoften identifiedasbeing responsibleforsector

complexity in the Chicago ARTCC was climbing and

descending flight paths. More recently, Kopardekar and

Magyarits (2003) found that the number of descending

aircraft and the number of altitude changes greater than

750 feet per minute both contributed significantly to the

explanation of variance in a linear regression model of

subjective complexity ratings collected attheFortWorth,

Atlanta, Cleveland, and Denver ARTCCs. The Number

ofTransitioningAircraftrepresentsthenumber of aircraft

makingoneormore altitude changesduring the5-minute

processing interval. To be counted as a change, altitude

must increase or decrease by a minimum of 200 feet per

12-second radar update and must continue to change

in the same direction for at least three updates. Altitude

changes resulting from last-minute clearances made in

an attempt to avoid the OE were excluded from counts

for the OE sample. The criteria for these exclusions are

described in the Method section.

Exclusion of Static Predictor Variables

Sector complexity factors are generallydescribed intwo

ways:static anddynamic (Mogford,Guttman,Morrow,&

Kopardekar,1995). Thepredictor variablesjustdescribed

represent dynamic sector characteristics because they

change over time. Static sector characteristics are those

that change infrequently or not at all and are generally

related to airspace design (e.g., size of thesector, number

of shelves or tunnels). Static sector characteristics may

vary between sectors, but they do not varywithin sectors. 

Consequently, the variance of static variables would be

seriously limited in the present study because multiple

OEs occurred in many of thesamesectors in thesample. 

Even if static sector characteristics were related to OEs,

it is unlikely this relationship would be detected. For

this reason, predictors were restricted to dynamic sector

characteristics. These were submitted to logistic regression analysisto determinethe degreetowhich they could

discriminate between OE and RO traffic samples.

Method

Data Sources

Traffic Samples. Alltraffic sampleswere initiallyderived

from System Analysis Recordings (SARs) generated by

en route Host Computer Systems. The Host features

data reduction programs that generate text reports of

selected subsets of SAR data. The information used to

calculate the predictor variables was extracted from Log

and Track reports produced by one of these programs,

the Data Analysis and Reduction Tool (DART). Log

reports include controller entries and information sent

to the radar display and the auxiliary text display (e.g.,

data blocks and list items). Track reports contain detailed

information (e.g., altitude, heading, ground speed, and

position) from the Host computer’s internal radar track

database.

OE traffic samples were derived from Systematic Air

TrafficOperationsResearchInitiative(SATORI;Rodgers

& Duke, 1993) files. SATORI files contain reconfigured

DART information(i.e.,log andtrack reports ina slightly

different format). SAR data are difficult to obtain from

facilities because they require a prohibitive amount of

storagespace. SATORIre-creationsrequirelessspace and

so these files are often the only traffic data saved after an

OE. Therefore, the primary constraint on the size and

range of the data set was the availability of SATORI recreations. SATORI data meeting processing criteria (i.e.,

five minutes prior to the initial loss of separation) were

only availablefor 119 OEs occurring in the ZID airspace

from9/17/2001through12/10/2003. OEtraffic samples

from 2003 (n = 48) were used for model building, and

the remaining OE traffic samples (n = 71) were used for

cross validation.

The RO traffic samples were derived from ZID SAR

data recorded on 5/8/2003 (15:55 to 17:05, 18:55 to

20:10, and 20:50 to 22:15 ZULU), 5/9/2003 (0:00 to

1:10 ZULU), and 5/10/2003 (11:20 to 12:40 ZULU). 

DART log and track text reports were first encoded into

database files and then processed in 5-minute intervals

using custom software designed to calculate objective

measures from routinely recorded NAS data. This produced a total of 2644 RO traffic samples. A randomlyselected set (n = 48) were used for model building, and

the remaining samples (n = 2596) were set aside for

cross validation. Random selection was accomplished

by first sorting traffic samples by date, time, and sector

number, then assigning each sample a random number

(generated by the Microsoft Excel RAND function), and

then resorting the samples according to this value. The

first 48 cases (equal to the number of OE samples) were

assigned to the model-building sample. A list of the ZID

sectors and thenumber of associated traffic samples inthe
model-building and cross-validation samples is provided

in Table A1 of Appendix A.

In the model-building sample, the number of samples

in theROandOEgroups isequivalent. This is important

because widely disparate group size produces logistic regressionmodelsthatfavor(i.e., over-represent)thelargest

group. Equal group size also ensures that classification

accuracy in excess of 50% represents improvement over

chance. On the other hand, group size does not matter

in the cross-validation sample becausethese cases are not

used to build the model. The existing model is simply

being applied to the sample.

Transitioning Aircraft and Heading Change Adjustments

Altitude and heading changes made in an attempt to

avoid an imminent OE were excluded from the analysis

using summary incident narrativesfrom the Operational

Error Investigation form and lists of computer-detected

changes. The summary incident narratives described

events leading up to the loss of separation. The lists included the aircraft identifier, start time, stop time, and

direction of all computer-detected altitude and heading

changes used in the calculation of the Number of Transitioning Aircraft and Number of Heading Changes.

The time at which the controller became aware of a

potential OE and any action taken to avoid loss of separation were usually described in the narrative. The time

and content ofthe clearances obtained fromthe narrative

were then compared with the list of computer-detected

changes for the OE traffic sample. If the listed clearance

matched one of the changes (i.e., matched by aircraft

identifier, time, and direction), it was marked for exclusion. In theeventthatthe controllerwas not awarethat an

OE was about to occur or the clearance information was

not contained in the narrative, all altitude and heading

changes occurring after the conflict alert warning (i.e.,

the alert just prior to theloss of separation) were marked

for exclusion. The exclusions were then subtracted from

the summary measures (i.e., Number of Transitioning

Aircraft and Number of Heading Changes). Altitude

changes were evaluated by aircraft rather than tabulated

changes. The value of the Number of Transitioning Aircraft remained unaltered if an aircraft with one excluded

altitude change had other changes not associated with an

OE. If an aircraft made only one change or all altitude

changes were associated with the OE, then the aircraft

was subtracted from the total value of the Number of

Transitioning Aircraft.

Results

Preliminary Logistic Regression Analysis

Stepwise elimination was employed because such

methods are extremely valuable in the exploratory stages

of research when the focus is on identifying predictors. 

Backward elimination was used for selection because it is

less prone to omit useful variables, since all variables are

in the model at the beginning of the process (Menard,

1995). Thelikelihood-ratiotest,whichcomparesthefitof

the model with and withouteach predictor atevery step,

was the selection criterion used because it is more rigorous than other methods (Norušis, 1990; Pampel, 2000;

Tabachnick & Fidell, 2006). Hosmer and Lemeshow

(2000)recommend using criterion levels as high as .15 or

.20 to ensurethat allrelevant variables are included in the

logistic regression model. However, a more conservative

criterion level of .10 was used in this analysis.

Descriptive statistics for the model-building sample

are shown in Table 1. Descriptive statistics for the cross-

validation sample are shown in Table 2. It is clear that

several of the variables were not normally distributed. 

In the model-building sample, the distribution of the

Number of Point Outs differed from normal by nearly

10 standard deviations in skewness and 14 standard de-

viations in kurtosis. The Number of Intersecting Flight

Paths deviated by 14 standard deviations in skewness

and by more than 40 standard deviations in kurtosis. In

the cross-validation sample, departures from normality

in skewness ranged from 13 (Average Control Duration)

to more than 91 (Number of Point Outs) standard de-

viations. Departures from normality in kurtosis ranged

from just under 4 (Number of Transitioning Aircraft)

Average Control Duration (seconds) 182.05 50.91 -1.19 2.09

Number of Handoffs 4.54 3.05 .72 .02

Number of Heading Changes 2.19 2.19 1.27 1.28

Number of Intersecting Flight Paths .56 .97 3.54 19.67

Number of Point Outs 1.13 1.76 2.44 6.67

Number of Transitioning Aircraft 3.54 2.29 .54 -.27

1

SE Skewness = .246;

2

SE Kurtosis = .488

Table 1. Descriptive Statistics: Model-building Sample (N = 96)

Variable Mean SD Skew.1 Kurtosis2
to more than 362 (Number of Point Outs) standard

deviations. Beyond theoretical issuessurrounding the use

of frequency data in parametric statistics, such extreme

departures from normality confirm the choice of a nonparametric statistic.

Though logistic regression is a “distribution-free”

statistic, it is not assumption free. As with other forms

of regression, multicollinearity among the predictors

can lead to biased estimates (Menard, 1995). None of

the Spearman’s correlation coefficients shown in Table 3

are of sufficient magnitude to suggest multicollinearity

in the predictor set. Tolerance values were >.45 for all

predictors, far in excess of the <.20 that would indicate

multicollinearity. Correlations of the predictors in the

cross-validation sample are shown in Table 4. Although

all ofthe coefficients arestatistically significant(no doubt

due to the size of the sample), the pattern of associations

is similar to that of the model-building sample.

The Model χ2 represents the difference between the

-2 log likelihood for the model containing the predictor

variables and that of the model with the constant only. 

Thus, the Model χ2

 is analogous to the overall F in linear

regressionbecause itteststhenullhypothesisthatthe coefficientsfor allpredictors inthemodelequalzero (Norušis,

1990). The final logistic regression model for this sample

generated a Model χ2

(3, N=96) = 51.67, p <.01, indicating significantly improved prediction over the model

with the constant only. The Hosmer-LemeshowTest is a

particularly robust measure of fit. Cases are divided into

deciles of predicted probabilities, and then observed and

expected probabilities are compared within each decile. 

As with most goodness-of-fit measures, a non-significant

result is desirable. The Hosmer-Lemeshow χ2 (7, N=96)

= 6.10, p =.53 for this sample indicated that the model

fit the data reasonably well. 

Table 3. Correlation̅ Matrix: Model-building Sample (N = 96)

1 2 3 4 5 

1 Average Control Duration 

2 Number of Handoffs -.29** 

3 Number of Heading Changes .23* .33** 

4 Number of Intersecting Flight Paths .40** .26* .56** 

5 Number of Point Outs .07 .08 .22 .17 

6 Number of Transitioning Aircraft .03 .38** .58** .38** .22* 

̅ Spearman’s rho; **p < .01; *p < .05

Table 4. Correlation̅ Matrix: Cross-validation Sample (N = 2667)

1 2 3 4 5 

1 Average Control Duration 

2 Number of Handoffs -.10** 

3 Number of Heading Changes .26** .29** 

4 Number of Intersecting Flight Paths .27** .26** .33** 

5 Number of Point Outs .06** .23** .26** .19** 

6 Number of Transitioning Aircraft .13** .42** .45** .33** .23** 

̅ Spearman’s rho; **p < .01; *p < .05

Average Control Duration (seconds) 173.13 53.60 -.63 .90

Number of Handoffs 3.75 2.70 1.01 1.82

Number of Heading Changes 1.45 1.57 1.58 3.77

Number of Intersecting Flight Paths .26 .52 2.10 5.18

Number of Point Outs .87 1.59 4.32 34.39

Number of Transitioning Aircraft 2.95 2.23 .80 .36 1

SE Skewness = .047;

2

SE Kurtosis = .095

Table 2. Descriptive Statistics: Cross-validation Sample (N = 2667)

Variable Mean SD Skew.1

 Kurtosis2
Logistic regression coefficients (B), standard errors,

estimated odds ratios (Odds), 95% confidence intervals

for the odds ratios (95% CI), and significance values

for the likelihood-ratio tests are provided in Table 5. 

Note that neither the logistic regression coefficients

nor standard errors are inflated. The absence of large

coefficients indicates a sufficient ratio of cases to predictors. Variables included in the model were the Number

of Intersecting Flight Paths, the Number of Heading

Changes, and the Number of Handoffs. In this sample,

every one-unit increase in the Number of Intersecting

Flight Paths increased the likelihood by a multiplicative

factor of 4.94 (i.e., increased the odds by e4.94). In other

words,each intersectingflightpathdetected increasedthe

likelihood that the traffic sample was an OE by 394%. 

Every heading change increased the likelihood by 58%,

and each handoff increased the likelihood by 28% that

a traffic sample was an OE.

Classificationaccuracy inthemodel-building sample is

shown inTable 6. In thissample, 41 (85%) ofthe 48ROs

were correctly classified, and 7 (15%) were misclassified

as OEs. Of the 48 OEs in the model-building sample,

34 (71%) were correctly classified and 14 (29%) were

misclassified as ROs. Overall, classification reached 78%

accuracy in the model-building sample. This represents

28% improvement overpriorprobabilities(i.e.,thenumber that would be correctly classified by chance). 

Classification accuracy in the cross-validation sample

is shown inTable 7. Keep in mind that relative group size

does not matter in the cross-validation sample because

these cases are not being used to build the model. The

existing model is simply being applied to the sample. 

Of the ROs in the cross-validation sample, 1829 (71%)

were correctly classified and 767 (29%) were misclassified as OEs. Of the OEs in the cross-validation sample,

51 (72%) were correctly classified and 20 (28%) were

misclassified as ROs. Overall classification accuracy was

71% in the cross-validation sample.

Although the loss of classification accuracy when the

model was applied to the RO samples was somewhat

disappointing, the question of whether the samples

were similar enough for pooling applied only to the OE

samples. The similarity between classification accuracy

forthemodel-building (71%) andcross-validation(72%)

OE samples suggested they were homogenousenough to

justify pooling for separate logistic regression analyses of

the low- and high-altitude sectors. 

In the pooled sample of OEs, 40 occurred in the

low-altitudesectors and 79 occurred in the high-altitude

sectors. The 40 low-altitude OE traffic samples were

Number of Intersecting Flight Paths 1.60 .50 4.94 1.86 13.12 .000

Number of Heading Changes .46 .21 1.58 1.05 2.37 .019

Number of Handoffs .25 .11 1.28 1.03 1.59 .019

Constant -3.10 .71 .05 

Table 5. Logistic Regression Summary (N = 96)

Variable B S.E. Odds 95% CI p

Predicted

Routine Operation Operational Error Total 

Routine Operation 41 (85%) 7 (15%) 48 

Observed

Operational Error 14 (29%) 34 (71%) 48 

Table 6. Classification: Model-building Sample (N = 96)

Table 7. Classification: Cross-validation Sample (N = 2667)

Predicted

Routine Operation Operational Error Total 

Routine Operation 1829 (71%) 767 (29%) 2596 

Observed

Operational Error 20 (28%) 51 (72%) 71 
combined with 40 randomly-selected low-altitude RO

traffic samples to produce a total of 80 traffic samples for

thelow-altitudesector analysis. The 79 high-altitudeOE

traffic sampleswere combinedwith79randomly-selected

high-altitudeROtraffic samplesto produce a total of 158

traffic samplesforthe high-altitudesector analysis. Backward stepwiseelimination using thelikelihood-ratio test

as selection criterion (p =.10) wasemployed for both the

low- and high-altitudesector logistic regression analyses. 

Appendix A contains a list of the ZID sectors and the

number of low-altitude (Table A2) and high-altitude

(Table A3) traffic samples associated with each.

Logistic Regression Analysis: Low-Altitude Sectors

Asshown inTable 8,theNumber of HeadingChanges

is highly correlatedwith both theNumber ofIntersecting

Flight Paths (rs

=.62) and the Number of Transitioning

Aircraft (rs

=.68). The association between Number of

Transitioning Aircraft and the Number of Handoffs

(rs=.61) is also of sufficient magnitude for concern. 

Nevertheless, Tolerance values were > .40 for all predictors, indicating there is no reason to suspect bias due to

multicollinearity in the low-altitude sector sample.

The logistic regression model for the low-altitude

sample generated a Model χ2

(3, N=80) = 23.82, p <.01,

indicating significantly improved prediction over the

model with the constant only. The non-significant

Hosmer-Lemeshow χ2 (8, N=80) = 1.61, p=.99 for the

low-altitude sample signifies that the model fit the data

well. Logistic regression coefficients (B), standard errors,

estimated odds ratios (Odds), 95% confidence intervals

for the odds ratios (95% CI), and significance values

for the likelihood-ratio tests for the low-altitude sector

sample are provided in Table 9. As with the full sample,

neither the logistic regression coefficients nor standard

errors are inflated, indicating a sufficient ratio of cases

to predictors.

In the low-altitude sample model, the Number of Intersecting Flight Paths had the highest odds ratio (2.89),

followed by the Number of Point Outs (1.57), and the

Number of Handoffs (1.19). In other words, each intersecting flight path increased thelikelihood thatthetraffic

sample was an OE by 189%, each point out increased

the likelihood by 57%, and each handoff increased OE

likelihood by 19%. However,the confidence intervalsfor

the Number of Point Outs and the Number of Handoffs

suggest that the parameter estimates for these predictors

might not generalize to another sample.

Classification accuracy in the low-altitude sample

(Table 10) was similar to that of the combined sample. 

Of the 40 ROs in the low-altitude sample, 32 (80%)

were correctly classified and 8 (20%) were misclassified

as OEs. Of the 40 OEs in the sample, 28 (70%) were

correctly classified and 12 (30%) were misclassified as

ROs. Overall, the low-altitude model had 75% classification accuracy.

Logistic Regression Analysis: High-Altitude Sectors

Although several ofthe Spearman’s correlationsshown

inTable11arestatistically significant,noneof coefficients

are of sufficient magnitude to suggest multicollinearity. 

Accordingly,Tolerance values were high (.56 and above)

for all predictors in this sample.

The logistic regression model for the high-altitude

sample generated a Model χ2

(4, N=158) = 73.01, p

<.01, indicating significantly improved prediction over

the model with the constant only. The non-significant

Hosmer-Lemeshow χ2 (8, N=158) = 3.33, p=.91 for

the high-altitude sample verified that the model fit the

data. Logistic regression coefficients (B), standard errors,

estimated odds ratios (Odds), 95% confidence intervals

for the odds ratios (95% CI), and significance values

for the likelihood-ratio tests for the high-altitude sector

sample are provided inTable 12. As with thefull sample,

neither the logistic regression coefficients nor standard

errors are inordinately large, indicating a sufficient ratio

of cases to predictors.

In the high-altitude sample model, the Number of

Intersecting Flight Paths had the highest odds ratio

(2.00), followed by the Number of Heading Changes

(1.36), the Number ofTransitioning Aircraft (1.27), and

Average Control Duration (1.01). In other words, each

one-unit increase in the Number of Intersecting Flight

Paths increased the likelihood that a traffic sample was

Table 8. Correlation̅ Matrix: Low-Altitude Sectors (N = 80)

1 2 3 4 5 

1 Average Control Duration 

2 Number of Handoffs -.27* 

3 Number of Heading Changes .21 .40** 

4 Number of Intersecting Flight Paths .33** .32** .62** 

5 Number of Point Outs -.14 .17 .25* .06 

6 Number of Transitioning Aircraft .22 .61** .68** .56** .08 

̅ Spearman’s rho; **p < .01; *p < .05
Number of Intersecting Flight Paths 1.06 .36 2.89 1.43 5.83 .00

Number of Point Outs .45 .24 1.57 .98 2.52 .04

Number of Handoffs .17 .11 1.19 .97 1.46 .10

Constant -1.94 .57 .14 

Table 9. Logistic Regression Summary: Low-Altitude Sectors (N = 80)

Variable B S.E. Odds 95% CI p

Predicted

Routine Operation Operational Error Total 

Routine Operation 32 (80%) 8 (20%) 40 

Observed

Operational Error 12 (30%) 28 (70%) 40 

Table 10. Classification: Low-Altitude Sectors (N = 80 )

Table 11. Correlation̅ Matrix: High-Altitude Sectors (N = 158)

1 2 3 4 5 

1 Average Control Duration 

2 Number of Handoffs -.14 

3 Number of Heading Changes .36** .42** 

4 Number of Intersecting Flight Paths .49** .22** .51** 

5 Number of Point Outs .11 .24** .21** .24** 

6 Number of Transitioning Aircraft .16* .43** .53** .39** .19* 

̅ Spearman’s rho; **p < .01; *p < .05

Number of Intersecting Flight Paths .69 .28 2.00 1.16 3.45 .01

Number of Heading Changes .31 .15 1.36 1.01 1.83 .03

Number of Transitioning Aircraft .24 .11 1.27 1.03 1.57 .02

Average Control Duration .01 .01 1.01 1.00 1.03 .01

Constant -4.43 1.11 .01 

Table 12. Logistic Regression Summary: High-Altitude Sectors (N = 158)

Variable B S.E. Odds 95% CI p
10

an OE by 100%, each one-unit increase in the Number

of Heading Changes increased the likelihood by 36%,

every Transitioning Aircraft increased the likelihood by

27%, and each one-second increase in Average Control

Duration increased the likelihood by 1%.

Classification accuracy in the high-altitude sample

(Table 13)wasslightly betterthan that ofthelow-altitude

sample. Of the 79 ROs in the high-altitude sample, 64

(81%) were correctly classified and 15 (19%) were misclassified as OEs. Of the 79 OEs in thesample, 60 (76%)

were correctly classified, and 19 (24%)were misclassified

as ROs. Overall, the high-altitude model had 79% classification accuracy. This represents a 29% improvement

over prior probabilities (i.e., the number that would be

correctly classified by chance).

Discussion

The results of the logistic regression analyses indicate

that a sufficient model may be constructed from sector

characteristics variables. Overall classificationaccuracybetween75-79% isremarkableformodels constructedsolely

of environmental and contextual factors. After all, other

factors (e.g., human elements, organizational influences)

also contributeto the occurrence of OEs. Unfortunately,

all thelogistic regression modelswere better at classifying

ROs than OEs. Classification of OEs ranged from as low

as 70% in the low-altitude sector sample to 76% in the

high-altitude sample. Although this level of accuracy

would be unacceptable for most automation tools, it is

unrealistic to expect definitive results from one or two

analyses. Moreover,thesector characteristic variablesused

in these analyses do not represent an exhaustivelist of all

the potential predictors of OEs.

One of the most unexpected findings was the uniqueness of the preliminary analysis model. The Number of

Handoffs had an odds ratio of 1.28, and yet this variable

failed to demonstrate a similar level of influence in the

low-altitudemodel andwas conspicuously absentfromthe

high-altitudemodel. Therelative influenceoftheNumber

of Handoffs when low- and high-altitude sectors were

combined iseither an indictment of the use of regression

techniques in general (i.e., they tend to capitalize on the

unique characteristics of the sample) or an indication

that traffic count becomes more salient as the level of

analysis changes. Most dynamic variables correlate, to

a greater or lesser degree, with traffic count. Therefore,

traffic count may emerge as unique characteristics of the

low- and high-altitude sectors are obscured. That traffic

count remains the single best predictor of OEs at the

national level, yet fails to predict well at the sector level,

may be an example of this phenomenon. Consequently,

differences between the low-altitude, high-altitude, and

combined models might have implications for the suitability of applying policies based on evaluations made at

the national level at the sector level.

Low-Altitude Sector Model

The most influential variable in the low-altitude

sector model was the Number of Intersecting Flight

Paths (Odds = 2.89), followed by the Number of Point

Outs, and the Number of Handoffs (Odds = 1.19). 

Aside from the addition of the Number of Intersecting

Flight Paths, this model was similar to the low-altitude

sector model in Pfleiderer and Manning (2007). In the

previous study, the most influential predictor was the

Number of Point Outs (Odds = 3.30), followed by the

Number of Handoffs (Odds = 1.54), and the Number

of Heading Changes (Odds = 1.49). Doubts about the

validity of the Number of HeadingChanges as a genuine

predictor in the previous study were dueto concerns that

much of its effect in the low-altitude model was related

to clearances made in an attempt to resolve the OE. As

anticipated, the Number of Heading Changes was not

included in the low-altitude sector model once this variable was adjusted. Both the Number of Point Outs and

the Number of Handoffs lost a considerable amount of

reliability, as evidenced by the range of the confidence

intervals. On the other hand, the fact that the Number

of Point Outs and the Number of Handoffs were also

included in the previous model (based on a completely

differentsample ofROs)somewhat beliesthe contention

that these parameter estimates might not generalize to

another sample.

Predicted

Routine Operation Operational Error Total 

Routine Operation 64 (81%) 15 (19%) 79 

Observed

Operational Error 19 (24%) 60 (76%) 79 

Table 13. Classification: High-Altitude Sectors (N = 158)
11

On the whole, classification accuracy in the low-altitude samples was inferior to that of the previous study. 

Although classification of the RO samples was improved

(80% as opposed to 77% in the previous analysis), OE

classification was not. The current model, as compared

to 82% accurate classification in the previous analysis,

accurately classified only 70% of the OE samples.

The predictive strength of the Number of Point Outs

and the Number of Handoffs in the Pfleiderer and Manning (2007) results suggested that coordination played

a primary role in the development of OEs in the ZID

low-altitude sectors. This impression was bolstered by

the Pfleiderer et al. (2007) data, in which controllers

and supervisors at ZID rated coordination as one of the

primary sources of complexity in low-altitude sectors. 

Consequently, the emergence of the Number of Intersecting Flight Paths as the most influential predictor in

the current low-altitude logistic regression model was

surprising, because controller and supervisor ratings for

this complexity factor were moderate in thelow-altitude

sectors. The results of the logistic regression analysis

suggest that coordination may be a contributing factor, but converging traffic patterns might be of greater

consequence.

High-Altitude Sector Model

TheNumber ofIntersecting Flight Pathswasthemost

influential predictor in the high-altitude sector model

(Odds = 2.00), followed by the Number of Heading

Changes. As anticipated, there was a reduction in the

relative influence of the Number of Heading Changes

after adjustments were made to exclude changes made

in response to clearances to avoid an imminent OE (i.e.,

the estimated odds ratio was reduced from 2.28 in the

previousstudy to1.36 inthepresentstudy). Nevertheless,

it remained a significant predictor in the high-altitude

model. Otherelements ofthemodelwerenearly identical

to those of Pfleiderer and Manning (2007). The Number

of Transitioning Aircraft had an estimated odds ratio of

1.26 in the previous study and increased to 1.27 in the

present one. AverageControl Duration had an estimated

odds ratio of 1.02 in the previous study and decreased

to 1.01 in the present one. Classification accuracy in the

high-altitudesamplewas alsorelatively consistentbetween

the two studies. Both models correctly classified 81% of

the ROs. However, 79% of the OEs were correctly classified in the previous analysis, whereas only 76% were

correctly classified in this one.

The influence of the Number of Intersecting Flight

Pathswasnosurprise inthissample,becauseZIDcontrollers and supervisors rated this complexity factor one of

the most influential in the high- and super high-altitude

sectors (Pfleidereret al., 2007). The Number of Heading

Changes remained influential, despite changes to the

processing interval and adjustments to the variable itself. 

This is consistent with Laudeman et al. (1998), in which

heading changes received the highest beta weight in a

linear multipleregression analysis of controller ratings of

activity in four sectors at the Denver ARTCC. In their

discussion,the authors attributedthe influenceofheading

changes to the “significant arrival traffic in all the sectors

that were observed” (p. 7). Arrival and departure traffic

complexity is generally considered to be a low-altitude

phenomenon, but this perception may be inaccurate. In

the present study, the Number of Heading Changes was

extremely influential inthehigh-altitudemodelbutfailed

to be included in the low-altitude sector model. 

The third most influential factor in the high-altitude

logistic regression analysiswastheNumber ofTransition-

ingAircraft. Climbinganddescendingtraffichaslongbeen

recognized as a contributor to the difficulty of working

a sector (e.g., Arad, 1964; Grossberg, 1989; Kopardekar

& Magyarits, 2003). This finding is also consistent with

Pfleiderer et al. (2007), in which the complexity factor

Climbing and Descending Traffic received the highest

complexity rating for the high-and super high-altitude

sectors.

The fourth variable included in the high-altitude

sample model was Average Control Duration. Odds ratios in logistic regression are an indication of effect size. 

The closer the odds are to zero, the smaller the effect

size. Consequently, a 1.01 odds ratio suggests that the

effect for Average Control Duration is small. However,

it is important to remember that logistic regression

coefficients (i.e., the natural logs of the odds ratios) are

not standardized. Average Control Duration is based on

the number of seconds each aircraft was controlled by

the sector. Thus, a minimal change in Average Control

Duration produced a relatively large 1% change in the

likelihood that a traffic sample was an OE.

Future Research

Although logistic regression cannot be used to identify

causal factors directly (i.e., prediction is not the same as

causation), thelogistic regression coefficients do provide

information about thelikelihood of an OE relativeto the

predictors in the model. Thus, the results have immediate heuristic value in that they invite questions about

how the dynamic predictors interact with static sector

characteristics. Dynamic elements lend themselves to

automationapplications,butstatic characteristicsmustbe

addressedby sectorrestructuring. Thedynamicpredictors

that make up thelogistic regression models are indicants

of conditions that discriminate between OEs and ROs. 
12

These indicantsmay be used to reveal aspects ofthesector

environment that might be altered to reducethe number

of OEs. For example, the combination of the Number

of Point Outs and the Number of Handoffs in the lowaltitude sector model may indicate that the location of

sector boundaries increases coordination workload and

complexity. As Couluris and Schmidt (1973) observed,

handoffs andpoint outs “resultfrom, or are influencedby,

the existence and design (shape) of the sectors” (p. 657). 

On the other hand, the combination of the Number of

Point Outs and the Number of Intersecting Flight Paths

may point to problems with the orientation of traffic

paths relative to those boundaries. The combination

of the Number of Heading Changes and the Number

of Transitioning Aircraft in the high-altitude sectors is

suggestive of traffic complexity in high-altitude sectors

adjacent to low-altitude arrival or departure sectors. Average Control Duration as a predictor of OEs may be a

function of the size of high-altitude sectors.

Becauseoftheresearchthatremainstobe accomplished,

these results must be viewed as preliminary. Given the

sample size and consequent restriction of the predictor

set, there is no guaranteethat theseresults will generalize

to othersamples. Multiplestudies,with samplessizesthat

allow for a more inclusivelist of predictors, must be conducted at a number of facilities beforesuchmodelsmight

bereliableenoughforpractical applications. Nevertheless,

themethodologyof comparingOEandROtraffic samples

is promising. Continued investigations along these lines

may highlight complexity factors that must be addressed

to ensure that separation is maintained.

References

Arad, B.A. (1964, May). The control load and sector

design. Journal of Air Traffic Control, 5, 12-31.

Buckley, E.P., DeBarysche, B.D., Hitchner, N., & Kohn,

P. (1983). Methods and measurements in real-time 

air traffic control system simulation. (Report No. 

DOT/FAA/CT-TN83/26). Atlantic City, NJ:

Federal Aviation Administration.

Chatterji, G.B., & Sridhar, B. (2001). Measures for air

traffic controller workload prediction. Proceedings 

of the 1st Aircraft, Technology, Integration, and Operations Forum. Los Angeles: American Institute

of Aeronautics & Astronautics. Available online

at: www.aiaa.org.

Christien, R., & Benkouar, A. (2003). Air traffic complexity indicators and ATC sectors classification. 

Proceedings of the 5th Eurocontrol/FAA Air Traffic 

Management Research and Development Conference,

Budapest, Hungary. Available online at: http://

atm2003.eurocontrol.fr.

Couluris, G.J., & Schmidt, D.K. (1973), Air traffic

control jurisdictions of responsibility and airspace

structure. Proceedings of the Conference on Decision 

and Control, 4th Annual Symposium on Adaptive 

Processes (pp. 655-658). New York: Institute of

Electrical and Electronics Engineers.

Eurocontrol (2006). A complexity study of the Maastricht 

upper airspace centre. (EEC Report No. 403). 

Brétigny-sur-Orge, France: Eurocontrol Experimental Centre.

FederalAviationAdministration(2007). Air traffic control. 

(Order No. 7110.65R). Available online at: www.

faa.gov/ATpubs.

Grossberg, M. (1989, April). Relation of sector complexity to operational errors. Quarterly report of 

the Federal Aviation Administration’s Office of Air 

Traffic Evaluations and Analysis. Washington, DC:

Federal Aviation Administration.

Hosmer, D.W., & Lemeshow, S. (2000). Applied logistic 

regression (2nd ed.) New York: Wiley.

Kershner, A.M. (1968). Air traffic control system error 

data for 1965 and 1966 as related to age, workload, 

and time-on-shift of involved controller personnel

(Report No. NA-68-32). Atlantic City, NJ: FAA

NAFEC.

Kopardekar, P., & Magyarits, S. (2003, June), Measurement and prediction of dynamic density. Proceedings 

of the 5th Eurocontrol/FAA Air Traffic Management 

Research and Development Conference, Budapest,

Hungary. Available online at: http://atm2003.

eurocontrol.fr.

Laudeman, I.V., Shelden, S.G., Branstrom, R., & Brasil,

C.L. (1998). Dynamic density: An air traffic management metric. (NAS/TM-1998-112226). Moffett

Field, CA: NASA Ames.

Menard, S. (1995), Applied logistic regression, Sage Uni-

versity Papers Series on Quantitative Applications

in the Social Sciences, No. 106. Thousand Oaks,

CA: Sage.
13

Mogford, R. H., Guttman, J. A., Morrow, S. L., &

Kopardekar, P. (1995). The complexity construct 

in air traffic control: A review and synthesis of the 

literature. (Report No. DOT/FAA/CT-TN95/22). 

Atlantic City, NJ: FAA William J. HughesTechnical Center.

Mogford,R.H.,Murphy,E.D., & Guttman,J.A. (1994). 

Using knowledgeexploration toolsto study airspace

complexity in airtraffic control. International Journal of Aviation Psychology, 4(1), 29-45.

Norušis, M.J. (1990), Advanced statistics student guide. 

Chicago: SPSS.

Pampel, F.C. (2000), Logistic regression: A primer. Sage

University Papers Series on Quantitative Applications in the Social Sciences, No. 132. Thousand

Oaks, CA: Sage.

Pfleiderer,E.M.,&Manning,C.A. (2007). Prediction and 

classification of operational errors and routine operations using sector characteristics variables. (Report

No. DOT/FAA/AM-07/18). Washington, DC:

FAA Office of Aerospace Medicine.

Pfleiderer, E.M., Manning, C.A., & Goldman, S.M. 

(2007). Relationship of complexity factor ratings with 

operational errors. (Report No. DOT/FAA/AM07/11). Washington,DC: FAAOffice ofAerospace

Medicine.

Robertson, A., Grossberg, M., & Richards, J. (1979). 

Validation of air traffic controller workload models. 

(Report No. DOT/FAA/RD-79/83). Cambridge,

MA: U.S. Department ofTransportation Research

and Special Programs Administration – Volpe National Transportation System Center.

Rodgers, M.D., & Duke D.D. (1993), SATORI: Situation Assessment Through the Re-creation of Incidents. 

(ReportNo. DOT/FAA/AM-92/12). Washington,

DC: FAA Office of Aviation Medicine.

Rodgers,M.,Mogford,R., &Mogford, L. (1998). The relationship of sector characteristics to operational errors 

(ReportNo. DOT/FAA/AM-98/14). Washington,

DC: FAA Office of Aviation Medicine.

Schmidt, D.K. (1976), On modeling ATC workload

and sector capacity. Journal of Aircraft, 13(7),

531-537.

Schroeder, D.J. (1982, October). The loss of prescribed

separation between aircraft: How does it occur?

Proceedings of the Behavioral Objectives in Aviation 

Automated Systems Symposium, Society of Automotive 

Engineers (SAE), P-114, 257-269.

Schroeder, D., Bailey, L., Pounds, J., & Manning, C. 

(2006). A human factors review of the operational 

error literature. (Report No. DOT/FAA/AM06/21). Washington,DC: FAAOffice ofAerospace

Medicine.

Spahn, M.J. (1977). The human element in air traffic 

control: Observations and analyses of the performance 

of controllers and supervisors in providing ATC 

separation services. Supplement 2: Analysis of the 

System Effectiveness Information System (SEIS) data 

base. (Report No. MTR-7655, Supplement 2). 

McLean, VA: METREK Division of the MITRE

Corporation. 

Stein, E. (1985), Air traffic controller workload: An examination of workload probe. (Report No. DOT/FAA/

CT-TN84/24). Atlantic City, NJ: FAA William J. 

Hughes Technical Center.

Tabachnick,B.G., & Fidell, L.S. (2006). Using multivariate statistics (5th ed.). New York: HarperCollins.

A1

Appendix A

Indianapolis Air Route Traffic Control Center Traffic Samples

Table A1. Model-building and Cross-validation Samples

Number of Associated Traffic Samples 

Model-building Sample Cross-validation Sample 

Sector

Number 

Sector

Name 

Sector

Strata

Routine 

Operation 

Operational 

Error

Routine 

Operation 

Operational 

Error

18 Nabb LO 2 3 71 4 

19 New Hope LO 1 2 75 1 

20 Lexington LO 2 3 43 1 

21 London LO 3 0 73 1 

24 Parkersburg LO 0 0 76 1 

25 Hazard LO 0 1 60 0 

26 River LO 1 1 74 1 

30 Columbus LO 0 2 76 2 

31 Lytle LO 0 1 75 3 

32 Rosewood LO 0 1 70 1 

33 Muncie LO 3 0 73 1 

34 Shelbyville LO 3 1 73 3 

35 Terre Haute LO 0 0 62 3 

66 Madison HI 1 3 75 3 

69 Pike LO 2 1 74 2 

76 Batesville SH 2 0 63 1 

78 Springfield HI (IM) 1 1 75 1 

79 Bobcat HI (IH) 0 4 73 1 

80 King HI 1 1 73 2 

81 Pocket City HI 3 1 72 3 

82 Louisville HI 2 3 72 5 

83 Falmouth HI 3 1 74 1 

84 Rebel HI 2 0 74 0 

85 Charleston HI 2 1 74 4 

86 Beckley HI 1 3 75 2 

87 Appleton HI 3 3 73 2 

88 Dayton HI 2 2 72 6 

89 Indianapolis HI 1 2 75 5 

91 Impel SH 1 2 62 1 

92 Mystic SH 1 1 51 2 

93 Dacos SH 1 0 75 2 

94 Somerset SH 0 0 74 2 

95 Henderson SH 0 2 76 1 

96 Bluefield SH 0 1 74 0 

97 Lockbourne SH 3 0 73 1 

98 Patterson SH 1 0 48 0 

99 Wabash SH 0 1 68 2 

TOTAL 48 48 2596 71 
A2

Table A2. Low-Altitude Sector Sample

Sector Number of Associated Traffic Samples 

Number 

Sector

Name 

Sector

Strata Routine Operation Operational Error 

18 Nabb LO 0 7 

19 New Hope LO 4 3 

20 Lexington LO 1 4 

21 London LO 2 1 

24 Parkersburg LO 4 1 

25 Hazard LO 2 1 

26 River LO 2 2 

30 Columbus LO 3 4 

31 Lytle LO 4 4 

32 Rosewood LO 3 2 

33 Muncie LO 0 1 

34 Shelbyville LO 2 4 

35 Terre Haute LO 6 3 

69 Pike LO 7 3 

TOTAL 40 40 

Table A3. High-Altitude Sector Sample

Sector Number of Associated Traffic Samples 

Number 

Sector

Name 

Sector

Strata Routine Operation Operational Error 

66 Madison HI 6 6 

76 Batesville SH 3 1 

78 Springfield HI (IM) 5 2 

79 Bobcat HI (IH) 5 5 

80 King HI 4 3 

81 Pocket City HI 1 4 

82 Louisville HI 2 8 

83 Falmouth HI 1 2 

84 Rebel HI 4 0 

85 Charleston HI 1 5 

86 Beckley HI 1 5 

87 Appleton HI 4 5 

88 Dayton HI 3 8 

89 Indianapolis HI 2 7 

91 Impel SH 2 3 

92 Mystic SH 3 3 

93 Dacos SH 6 2 

94 Somerset SH 3 2 

95 Henderson SH 6 3 

96 Bluefield SH 7 1 

97 Lockbourne SH 5 1 

98 Patterson SH 1 0 

99 Wabash SH 4 3 

TOTAL 79 79