Relationship of 

Complexity Factor 

Ratings With 

Operational Errors

Elaine M. Pfleiderer

Carol A. Manning

Scott M. Goldman

Civil Aerospace Medical Institute

Federal Aviation Administration

Oklahoma City, OK 73125

May 2007

Final Report

DOT/FAA/AM-07/11

Office of Aerospace Medicine

Washington, DC 20591
NOTICE

This document is disseminated under the sponsorship 

of the U.S. Department of Transportation in the interest 

of information exchange. The United States Government 

assumes no liability for the contents thereof.

___________

This publication and all Office of Aerospace Medicine 

technical reports are available in full-text from the Civil 

Aerospace Medical Institute’s publications Web site: 

www.faa.gov/library/reports/medical/oamtechreports/index.cfm
i

Technical Report Documentation Page 

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No. 

DOT/FAA/AM-07/11 

4. Title and Subtitle 5. Report Date 

Relationship of Complexity Factor Ratings With Operational Errors May 2007 

6. Performing Organization Code 

 

7. Author(s) 8. Performing Organization Report No. 

Pfleiderer EM, Manning CA, Goldman SM 

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS) 

FAA Civil Aerospace Medical Institute 

P.O. Box 25082 11. Contract or Grant No. 

Oklahoma City, OK 73125 

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered 

Office of Aerospace Medicine 

Federal Aviation Administration 

800 Independence Ave., S.W. 

Washington, DC 20591 14. Sponsoring Agency Code 

15. Supplemental Notes 

Work was accomplished under approved task AM-BHRR-522. 

16. Abstract 

This study is an examination of the extent to which objective static sector characteristics and controller ratings of 

static and dynamic sector complexity factors contributed to the occurrence of operational errors (OEs) at the 

Indianapolis air route traffic control center (ZID). A multiple regression model of the relationship between a 

combination of static sector characteristics (sector altitude strata and sector size) resulted in a modest prediction 

of the variance in OE incidence (R = .70, R2

 = .49). Sector size was negatively related to OEs, indicating that 

smaller sectors were associated with more OEs. Sector strata were positively related to OEs, indicating that higher 

altitude sectors were associated with more OEs. Principal Components Analysis (PCA) of the complexity ratings 

produced four components with eigenvalues >1.00, accounting for 62% of the variance in the data. Components 

were used as predictors in a multiple regression analysis of the number of OEs in the ZID sectors. Only 

Component 1 (climbing and descending aircraft in the vicinity of major airports) and Component 2 (services 

provided to non-towered airports) contributed significantly to the total proportion of variance explained by the 

model (R = .78, R2

 = .61). Component 2 shared an inverse relationship with the number of OEs, indicating that 

the complexity related to providing services to non-towered airports is associated with fewer OEs. These results 

will be used to guide the choice of objective measures for further analysis of the influence of static and dynamic 

sector characteristics in the occurrence of OEs. 

 

17. Key Words 18. Distribution Statement 

 

 

Air Traffic Control, Operational Errors, Complexity, Static 

Sector Characteristics, Dynamic Sector Characteristics, 

Subjective Ratings

Document is available to the public through the 

Defense Technical Information Center, Ft. Belvior, VA 

22060; and the National Technical Information 

Service, Springfield, VA 22161 

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price 

Unclassified Unclassified 18 

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized 

iii

Executive Summary

Extensive research has focused on the behavioral and 

organizational aspects of operational error (OE) occurrence. While recognizing that the human component of 

OEs is extremely important, it is equally important to 

examine contextual and environmental factors. In this 

study, we analyzed the extent to which controller ratings 

of static and dynamic sector complexity factors related 

to the occurrence of OEs at the Indianapolis air route 

traffic control center (ZID). OE information was derived 

from final reports for 247 errors that occurred between 

1/15/2001 and 5/28/2005. Thirty-six ZID volunteers 

(32 controllers and 4 operational supervisors) rated the 

importance of 22 static and dynamic complexity factors 

for each sector on which they were certified. 

Principal Components Analysis (PCA) of sector complexity ratings produced four components that accounted 

for approximately 62% of the variance in the dataset. The 

basic theory behind PCA is that variables cluster together 

into “components” that reflect underlying dimensions in 

the data. In this PCA, the pattern of variables associated 

with Component 1 described climbing and descending 

aircraft in the vicinity of major airports. Component 2 

variables involved ATC services provided to non-towered 

airports. Component 3 comprised variables associated with 

military operations and special use airspace. Component 

4 described the effects of inclement weather on ATC 

operations. These results were comparable in many ways 

to the PCA of sector complexity factors at the Atlanta 

ARTCC conducted by Rodgers, Mogford, and Mogford 

(1998). Specifically, Components 1 (major airports) and 

3 (military activity and Special Use Areas [SUAs]) of the 

two analyses were strikingly similar, suggesting that these 

dimensions may be common to more than one facility.

A multiple regression analysis was conducted to examine 

the relationship between the four dimensions revealed by 

the PCA and the number of OEs. Only Component 1 

(major airports) and Component 2 (non-towered airports) 

explained a significant amount of the variance in OEs in 

the ZID sectors (R = .78, R2 = .61). Component 1 was 

positively associated with the number of OEs (i.e., higher 

scores were related to a higher number of OEs), whereas 

Component 2 had a negative relationship (higher scores 

were related to fewer OEs). The relationship between 

Component 2 and the incidence of OEs reminds us that 

sector complexity does not always produce a negative 

outcome. Indeed, a certain degree or type of complexity may actually be associated with a reduction in the 

number of OEs. The fact that Component 3 failed to 

contribute significantly to the prediction of OEs in this 

analysis does not mean that military airspace or SUAs 

do not make a sector more difficult to work or increase 

the likelihood of an OE. It simply means that subjective 

ratings of these factors failed to predict OEs. Similarly, the 

inability of Component 4 to contribute significantly to 

the regression model may reflect the intermittent nature 

of this dynamic event, or it may simply be an artifact of 

the way the variable was measured. In other words, the 

presence of inclement weather might be highly correlated 

with the occurrence of OEs but the component scores 

based on subjective ratings of variables associated with 

inclement weather were not.

We believe it is imprudent to make strong recommendations based on the results of analysis of subjective 

ratings. Practical prediction models must be calculated 

from objective measures because the actual characteristics 

of the sectors must be addressed when developing strategies to reduce OEs. However, these results represent a 

necessary and important step toward understanding how 

static and dynamic sector characteristics combine to create sector complexity, and how that complexity relates to 

the occurrence of OEs. Subjective information about the 

importance of complexity factors will guide our choice 

of objective measures in future analyses and may also 

be used to weight their importance, thus enabling us to 

make recommendations for reasonable, effective changes 

to the current system.

1

Relationship of Complexity Factor Ratings

With Operational Errors

The Federal Aviation Administration (FAA) is currently implementing strategic safety initiatives aimed at 

reducing operational error rates (FAA, 2005). Toward 

that goal, a considerable amount of research has focused 

on behavioral and organizational aspects of operational 

error (OE) occurrence. While recognizing that the human component of OEs is extremely important, it is also 

important to remember that air traffic controllers do not 

operate in a void. Logic dictates that environmental and 

contextual factors contribute to the development of at 

least a portion of these errors. Otherwise, the frequency 

of OEs would be relatively equal in all sectors. This 

simply is not the case. Some sectors are more prone to 

OEs than others.

The idea that sector characteristics might contribute 

to the occurrence of OEs is not new. Environmental 

and contextual factors affecting controller workload and 

performance – often referred to as sector complexity 

– have been the focus of numerous studies (e.g., Arad, 

1964; Arad, Golden, Grambart, Mayfield, & Van Saun, 

1963; Buckley, O’Connor, & Beebe, 1969; Davis, Danaher, & Fischl, 1963; Grossberg, 1989; Hurst & Rose, 

1978; Kirwan, Scaife, & Kennedy, 2001; Kopardekar 

& Magyarits, 2003; Laudeman, Shelden, Branstrom, 

& Brasil, 1998; Masalonis, Callaham, & Wanke, 2003; 

Mogford, Murphy & Guttman, 1994; Rodgers, Mogford, 

& Mogford, 1998; Schmidt, 1976; Stager, Hameluck, & 

Jubis, 1989; Stein, 1985). Even so, a single set of reliable 

general complexity factors has remained elusive. This may 

be partially due to the complicated interaction between 

static and dynamic sector characteristics. Static sector 

characteristics are usually related to airspace design and 

change infrequently or not at all. Dynamic sector characteristics are those that fluctuate, such as traffic volume or 

weather. Mogford, Guttman, Morrow, and Kopardekar 

(1995) observed that “a given level of traffic density and 

aircraft characteristics may create more or less complexity 

depending on the structure of the sector” (p. 3). Buckley, 

DeBarysche, Hitchner, and Kohn (1983) concluded that 

traffic characteristics and sector geometry were “important 

factors in determining the results which will occur in a 

given experiment, but they interact in a complex way. 

The nature and extent of this interaction depends upon 

the measures involved” (p. 73).

Perhaps one of the greatest challenges to the study 

of environmental factors in the development of OEs is 

that sectors are almost as unique as the people who work 

them. For example, Grossberg (1989) found that the 

highest-rated complexity factors in the Chicago air route 

traffic control center (ARTCC) were control adjustments 

involved in merging and spacing aircraft, climbing and 

descending aircraft, mix of aircraft types, frequent coordination, and amount of traffic. An index based on these 

factors was significantly correlated with the number of 

OEs. In the Jacksonville airspace, Mogford and coworkers 

(1994) found complex routings, spacing and sequencing 

for departures and arrivals, and frequency congestion 

to be most predictive of a subjective complexity index. 

Comparing their results with Grossberg’s, they concluded 

that complexity factors that were salient in one facility 

might not be applicable to another. Although this observation may be valid, another interpretation is that different 

complexity factors were found to be predictive because 

the two studies used different criteria. In other words, 

numbers of OEs and the value of a subjective complexity 

index may not be comparable criterion measures.

This brings us to an important point: It is extremely 

difficult to compare the results of the studies cited because of the variety of methods and measures used to 

assess complexity. Some of the studies compared sector complexity factors with OEs, some with subjective 

workload measures, and some with subjective complexity 

ratings. On the basis of a comprehensive review of the 

literature that spanned more than 40 years of research and 

identified in excess of 100 complexity factors, Hilburn 

(2004) concluded that “despite the breadth and depth 

of previous work done into identifying ATC complexity 

factors, a good deal of work remains. Nobody, it seems, 

has yet managed to construct a valid and reliable model 

of ATC complexity that [1] moves substantially beyond 

the predictive value of simple traffic density alone, and 

[2] is sufficiently context-free” (p. vi). In his conclusions, 

Hilburn makes an excellent case for the use of non-linear 

models of complexity to circumvent the context-specific 

shortcomings of linear models. The only drawback to 

this argument is that it fails to recognize that sometimes 

the context is the factor of interest. This is not to imply 

that Hilburn was ignoring the importance of context 

or the inevitability of contextual influences in airspace 

complexity. Rather, it emphasizes the fact that development of a “context-free” model is not always a desirable 

goal in research.

In spite of the “embarrassment of riches” represented 

by the literature, one precept is evident: It is extremely 
2

important to compare complexity factors in as many 

environments as possible. The Sector Characteristics 

and Operational Errors (SCOpE) project is an extension of a study conducted by Rodgers, Mogford, and 

Mogford (1998) that examined the relationship between 

sector complexity factors and the occurrence of OEs at 

the Atlanta ARTCC (ZTL). Specifically, the SCOpE 

project was initiated to compare and contrast the results 

of selected analyses from the 1998 study with similar 

analyses conducted using data from the Indianapolis 

ARTCC (ZID). The methodology of developing a regression model at one facility and applying the derived 

regression weights to another facility has met with limited 

success (e.g., Laudeman et al., 1998; Masalonis et al., 

2003). The advantage of the SCOpE paradigm is that it 

employs discrete models, thus enabling us to collect a set 

of general factors (i.e., factors that may reliably predict 

OEs at more than one facility) while documenting differences between facilities. After all, facility differences 

often represent important environmental and contextual 

elements as well.

The first phase of this project (Goldman, Manning, 

& Pfleiderer, 2006) compared a set of ZID static sector 

characteristics with those identified by Rodgers et al. 

(1998) at ZTL. With some exceptions, many of the static 

environmental and contextual factors that predicted OEs 

at ZTL also predicted the occurrence of OEs at ZID. In 

both studies, sector altitude strata, sector size, and number 

of major airports explained a significant proportion of 

the variance in the number of OEs per sector. However, 

some factors that were significantly correlated with OEs 

at ZTL failed to predict them in the ZID sample.

The second phase of the SCOpE project considers 

the relationship between a set of subjective static and 

dynamic complexity factor ratings and ZID OEs. In 

their analysis of sector characteristics and OEs, Rodgers 

et al. (1998) collected subjective complexity ratings and 

combined them with objective static sector characteristics 

in a Principal Components Analysis (PCA) to identify 

and describe the dimensions represented by these different types of variables. In the present study, subjective 

complexity ratings provided by ZID controllers will be 

examined in a series of discrete analyses to evaluate their 

relationship with OEs at ZID.

Method

Participants

Participants were 37 volunteers from ZID. Of these, 

32 were Certified Professional Controllers (CPCs), 4 

were operations supervisors, and 1 was a developmental 

controller who had completed Radar Associate training 

on all sectors in his area of specialization but was not yet 

certified on the corresponding radar positions. To guarantee that all participants were treated fairly and ethically, 

the experimental protocol and materials were cleared 

through the FAA’s institutional review board. Treatment 

of participants also met with guidelines established by 

the American Psychological Association. Volunteers were 

assured complete anonymity and reminded of their right 

to terminate participation at any time.

The mean age of the volunteer participants was 42 

years (SD = 6 years). Participants had been certified to 

control traffic for an average of 15 years (SD = 7 years), 

had been working at an ARTCC facility for a mean of 

17 years (SD = 7 years), and had been working at their 

current facility for an average of 16 years (SD = 8 years). 

Four had previous experience in the Terminal Radar Approach Control (TRACON) environment, and six had 

previously worked at an Airport Traffic Control Tower 

(ATCT). ZID is divided into seven areas of specialization, 

each comprising either five or six sectors. All areas were 

reasonably well represented by the sample of volunteer 

controllers and supervisors.

Materials

Complexity Factor Questionnaire (Complexity-Q). 

“Complexity-Q” refers to an automated experimental 

protocol software program and the questionnaire it was 

designed to administer. The Complexity-Q program is 

divided into four sections (i.e., Work Experience, Tutorial, Demonstration, and Questionnaire). Each section is 

described separately in the following paragraphs.

The Work Experience section recorded biographical 

data about participants’ work experience, collected information about the sectors on which they were certified, 

and generated a random-ordered list of sectors to be 

included in the questionnaire (based on input from the 

participant). It also randomized the presentation order 

of the complexity factor list and recorded the order of 

both lists in the output.

The Tutorial section consisted of a Microsoft PowerPoint slide presentation (automatically opened by the 

Complexity-Q program) that explained the purpose of 

the study, provided participants with an operational definition of sector complexity (i.e., “the static and dynamic 

characteristics that increase the level of difficulty involved 

in working traffic in a sector”), familiarized them with 

the basic structure of the questionnaire, and provided 

instructions about the functionality of interface elements 

(e.g., buttons, sliders, and bars). The Demonstration 

section was an extension of the Tutorial that provided 

participants with an opportunity to practice using the 

elements described therein.
3

The Complexity Factor Questionnaire followed the 

same basic structure for each sector on which the participants were certified. They were asked to provide a general 

“Complexity Rating” for a sector using a slider object 

with an underlying scale ranging from 0 to 100. The end 

points of the slider were labeled “Low” and “High” with 

visual anchors set at 10-point intervals. However, the 

slider was not restricted to these anchors, thus affording 

raters maximum response flexibility.

Once the participants entered an overall complexity 

rating, they were presented sequentially with a series of 

22 complexity factors and asked to indicate the level of 

influence each factor had on the complexity of the sector. 

The “Factor Rating” was made using the same slider and 

scale as the general complexity rating. If the participants 

were unsure about the meaning of the factor, a detailed 

description could be obtained by moving the mouse 

over the “Complexity Factor” label. The list of factors 

and their descriptions was initially derived from the 19 

complexity factors identified by Mogford et al. (1994). 

Subject Matter Experts (SMEs) from the facility and the 

FAA Academy provided additional factors prior to data 

collection. The Complexity-Q factors and their descriptions are provided in Table 1. Note that only two extra 

factors were added by the SMEs, yet there are 22 factors 

in the list. The “mix of aircraft with different performance 

characteristics” and “VFR versus IFR traffic” factors were 

combined in the original list but were separated into two 

distinct factors for this study.

After participants entered factor ratings for all 22 

factors, they were given the opportunity to enter any 

complexity factors they believed were not included in 

the list. If a participant added a complexity factor, the 

new factor was added to that particular participant’s list 

for all subsequent sectors. Thus, whenever participants 

entered a new complexity factor to the list, they had to 

rate that factor for all remaining sectors.

Procedure

Testing took place from 6/13/2005 to 6/17/2005 in a 

classroom at ZID. The Complexity-Q automated protocol 

was administered on laptop computers arranged around 

a large table to provide participants with as much privacy 

as possible. Participants were first given informed consent 

forms to read and sign. Once their written consent was 

obtained, they were shown the basic structure of the 

Complexity-Q interface, and the “Work Experience” 

section was brought up on the screen. Participants were 

requested to complete this section and then move through 

all subsequent sections in the order they appeared on 

the main interface (i.e., Tutorial, Demonstration, and 

Questionnaire). They were encouraged to ask questions 

about the interface or content of the Complexity-Q at any 

time during the automated protocol. Most participants 

completed the protocol in 40 minutes.

Measures

In addition to subjective Complexity-Q factor ratings 

provided by controllers, OE and sector characteristics 

information were collected from data provided by facility management. The following sections describe these 

variables and their sources.

Operational Error (OE) Data. The OE database consisted of information extracted from electronic records 

of the Final Operational Error/Deviation Report (FAA 

Form 7210-3) for 247 OEs occurring in ZID airspace 

from 1/15/2001 through 5/28/2005. Variables obtained 

from the final OE reports included the date and time 

of the OE and the number of controlled aircraft in the 

sector at the time the OE occurred. OEs were tallied for 

each sector in the ZID airspace.

Sector Characteristics Data. Sector altitude strata (super 

high-, high-, intermediate high-, intermediate-, or lowaltitude) were obtained from the facility’s Adaptation 

Control Environmental System (ACES) sector description 

file and verified with sector maps. The number of associated airports and the number of airports for which the 

sector provided approach services were derived from sector 

descriptions included in the center’s Standard Operational 

Procedures (SOPs). Staff from the facility’s airspace office 

clarified and augmented this information.

Results and Discussion

Descriptive Statistics 

The sample consisted of 181 complexity ratings 

provided by CPCs (n = 169) and operations supervisors 

(n = 12). Prior to the analysis, sector-by-sector comparisons were made between the mean ratings provided by 

CPCs and those provided by supervisors to determine 

whether the two sets of observations were comparable. 

On the average, supervisors’ ratings were less than two 

standard deviations from those of controllers, indicating 

that supervisor and CPC ratings were similar enough to 

constitute a homogenous sample. Conversely, data from 

the single developmental controller were excluded from the 

analysis. Supervisors and CPCs had experience working 

both the Radar Associate and Radar positions, whereas 

the developmental controller did not. This difference in 

experience gave rise to a discernable pattern of rating differences, suggesting that the developmental was sampled 

from a different population. Table 2 lists descriptive statistics for the Complexity-Q ratings. Though many of the 

distributions approximated normality, there were some 

notable exceptions. The distribution of the VFR versus 

IFR traffic, Shelves/Tunnels, Foreign aircraft/pilots, and 
4

Table 1. Complexity-Q Complexity Factors and Descriptions*

Complexity Factor Description 

 Climbing and descending traffic Climbing and descending aircraft are those that are transitioning 

altitudes, including departure and arrival traffic, aircraft that require 

different altitudes to alleviate conflictions due to crossing traffic or 

other problems, and aircraft requesting altitude changes due to 

turbulence, pilot preference, etc. 

 Mix of aircraft with different 

performance characteristics 

Extent to which the mix of props, turboprops, and jets impacts the 

controller.

VFR versus IFR traffic Extent to which differences in controlling VFR and IFR traffic, or VFR 

pilots encountering IFR conditions impacts controller workload. 

 Number of intersecting aircraft flight 

paths

The number of converging flight paths due to airways, arrival routes, 

frequent requests for direct routings; number of airways coming into 

same NAVAID; number of routes converging on a STAR, etc. 

 Number of multiple functions controller 

must perform 

Set of related tasks or services required in this sector (e.g., approach 

control, terminal feeder, en route, and in-trail spacing). 

 Traffic volume Extent to which the number of aircraft relative to the amount of 

available airspace impacts the controller. 

 Amount of military or other special 

traffic

Number of special missions (e.g., military, NASA, flight inspection, 

Lifeguard).

 Number of required procedures that 

must be performed (i.e., crossing 

restrictions in LOAs) 

A group of tasks, or a specific task, required by regulation or 

direction. A procedure mandates controller actions and must be 

performed regardless of other required tasks. 

 Amount of coordination/ interfacing 

required

Coordination with adjacent sectors, approach control, other en route 

centers, military facilities, etc. 

 Major airports (inside and outside 

sector boundaries) that might influence 

the number of procedures used, etc. 

Extent to which the controller's work is affected by the concentration 

of flights into one area due to the orientation of the sector relative to 

one or more major airports. 

 Extent operations are affected by 

weather 

Presence of weather conditions that necessitate requests for 

deviations, route changes, etc. 

 Relative frequency of complex routings Frequency of aircraft that are not on a published route structure, such 

as vectors or direct routings, etc. 

 Special Use Areas (Restricted areas, 

warning areas, and military operating 

areas) and their associated activities 

Extent to which SUA reduces the amount of airspace for nonparticipating aircraft, create obstructions to flight routes, increase the 

likelihood of conflictions due to reroutes, create situations requiring 

special handling and monitoring, etc. 

 Size of sector airspace The volume of airspace contained within the lateral and horizontal 

boundaries of the sector and the extent to which it impacts the 

controllers' ability to handle traffic volume, deal with special 

conditions (e.g., weather), and conflict resolution. 

 Requirement for longitudinal spacing/ 

sequencing

Combining aircraft from several streams into one stream. 

 Adequacy of radio/radar coverage Radio: Extent to which insufficient radio coverage results in the use of 

alternate communication techniques, such as pilot-to-pilot relays. 

Radar: Extent to which lack of radar results in use of non-radar 

procedures.

 Amount of radio frequency congestion Extent to which radio frequency congestion limits the controller's 

ability to utilize the frequency for issuing instructions to aircraft. 

 Traffic Management Initiatives Extent to which Traffic Management Initiatives impact operations 

(e.g., miles that must be made up, vectoring required to meet 

initiatives).

 Terrain/Obstructions Extent to which the terrain/obstructions (e.g., mountainous areas) 

adds complexity. 

 Shelves/Tunnels Extent to which shelves and/or tunnels add to complexity of the 

sector (e.g., are the lateral boundaries of the sectors above or below 

aligned or are they different?).

 ̧Foreign aircraft/pilots with English as a 

second language 

Extent to which communications with foreign aircraft/pilots with 

English as a second language increases the difficulty of 

communications.

 ̧Non-towered airports Extent to which providing service to non-towered airports increases 

the complexity of the sector. 

* Complexity factors and descriptions adapted from Mogford et al. (1994) except where indicated ( ̧)
5

Table 2. Complexity-Q Descriptive Statistics (N = 181)

Variable Mean SD Skew.1

 Kurtosis2

Climbing and descending traffic 77.71 20.94 -0.86 0.00 

Mix of aircraft types 63.75 22.22 -0.62 0.30 

VFR versus IFR traffic 17.75 28.15 1.28 0.08 

Number of intersecting flight paths 70.94 23.26 -0.79 -0.02 

Number of multiple functions 68.92 21.18 -0.64 0.05 

Traffic volume 73.38 21.14 -0.82 0.29 

Amount of military or other special traffic 35.66 25.59 0.49 -0.73 

Number of required procedures 69.67 22.43 -0.71 0.11 

Coordination/ interfacing required 68.75 21.82 -0.61 -0.08 

Number of major airports 68.19 26.53 -0.84 -0.17 

Extent operations are affected by weather 75.52 22.71 -0.90 0.10 

Relative frequency of complex routings 48.75 27.15 -0.04 -1.04 

Special Use Areas 48.06 29.09 0.06 -1.15 

Size of sector airspace 52.20 27.17 -0.11 -0.87 

Longitudinal spacing/ sequencing 72.20 25.44 -1.02 0.37 

Adequacy of radio/radar coverage 34.03 32.49 0.56 -1.01 

Amount of radio frequency congestion 60.50 25.99 -0.55 -0.51 

Traffic Management Initiatives 70.56 27.40 -0.90 -0.08 

Terrain/Obstructions 11.85 23.98 1.97 2.54 

Shelves/Tunnels 24.68 30.84 1.06 -0.11 

Foreign aircraft/pilots 18.22 19.93 1.46 1.43 

Non-towered airports 16.66 29.24 1.55 0.86 

1

SE Skew. = .181 2SE Kurt. = .359

Non-towered airports complexity ratings all had extreme 

positive skews. The distribution of Terrain/Obstructions 

ratings was both positively skewed and leptokurtotic. 

Such deviations are understandable, considering that 

these complexity factors only apply to some sectors (e.g., 

low-altitude sectors, sectors with shelves or tunnels). 

On the other hand, the Extent operations are affected 

by weather and Requirements for longitudinal spacing/ 

sequencing were given high ratings in almost every sector. 

Consequently, the distributions of these variables were 

significantly negatively skewed.

Given the scale of some of the deviations, it is comforting to know that assumptions regarding normality are 

not required when PCA is used descriptively. However, 

it is important to remember that PCA is sensitive to the 

magnitudes of correlations. To the extent that normality 

fails, the solution may be degraded (Tabachnick & Fidell, 

1989). Table 3 contains a Pearson’s correlation matrix of 

all the Complexity-Q factors. Note that variables with 

non-normal distributions achieved a significant degree of 

association with several others in the dataset, suggesting 

that the deviations were not severe enough to prevent a 

satisfactory PCA solution.

Principal Components Analysis

PCA is a statistical technique often used to describe 

relationships between complex sets of variables. Components extracted by PCA contribute to our understanding of a phenomenon by consolidating variables into 

parsimonious groups. In orthogonal rotation, loadings 

represent the correlations between a variable and a component. Variables with stronger loadings are generally 

considered to be more representative of a component’s 

underlying processes. Thus, we can use PCA not only to 

identify the number and nature of unique dimensions 

described by the Complexity-Q factors but also to what 

extent each variable relates to them. More importantly, 

we can use component scores output from the PCA to 

analyze the relationship between the complexity factors 

and the number of OEs in each sector.

For the Complexity-Q analyses, the number of sectors was reduced from 40 to 37 due to the combination 

of some sectors deemed appropriate by ZID personnel. 

With only 37 sectors in the sample and 22 complexity 

factors, the case-to-predictor ratio is unacceptable for most 

multivariate analyses. Component scores (computed by 

weighting variable scores using regression-like coefficients) 
6

Table 3. Correlation Matrix of Complexity-Q Factors (N = 181) 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 

1 Climbing/descending 2 Mix of aircraft types .56** 3 VFR versus IFR traffic .02 .27** 4 Intersecting flight paths .51** .34** -.35** 5 Multiple functions .56** .46** .01 .40** 6 Traffic volume .54** .24** -.31** .53** .42** 7 Military/Special traffic .26** .32** .20** .08 .30** .05 8 Required procedures .48** .39** -.14 .48** .58** .42** .19* 

9 Coordination .55** .46** .10 .40** .65** .42** .31** .55** 

10 Major airports .57** .38** -.16* .46** .50** .50** .16* .64** .46** 11 Weather .27** .14 -.14 .34-** .34** .39** .22** .45** .20** .33** 

12 Complex routings .23** .17* -.32** .39** .30** .36** .31** .42** .35** .33** .34** 13 Special Use Areas .37** .35** .14 .11 .33** .15* .62** .24** .35** .24** .30** .22** 

14 Size of sector airspace .35** .43** -.08 .44** .37** .21** .31** .40** .39** .40** .22** .32** .34** 15 Spacing/ sequencing .36** .16* -.35** .43** .30** .61** .14 .43** .28** .53** .38** .39** .12 .24** 16 Radio/radar coverage .12 .32** .47** -.07 .22** -.14 .42** .14 .30** .13 .13 -.01 .29** .24** -.12 17 Radio freq. congestion .37** .49** .10 .27** .57** .38** .29** .38** .52** .38** .20** .24** .42** .40** .28** .30** 18 TMI .31** .09 -.48** .42** .33** .57** .08 .47** .19* .48** .44** .47** .16* .23** .59** -.15* .34** 19 Terrain/Obstructions -.03 .20** .81** -.29** -.03 -.34** .19* -.08 .06 -.14 -.10 -.22** .11 -.02 -.32** .51** .05 -.41** 20 Shelves/Tunnels .24** .30** .24** .05 .18* -.11 .31** .19* .16* .21** .12 .07 .28** .34** .08 .31** .17* .05 .21** 21 Foreign aircraft/pilots .18* .21** -.10 .26** .25** .24** .40** .28** .23** .32** .39** .33** .44** .33** .23** .21** .29** .33** -.07 .17* 22 Non-towered airports .03 .25** .80** -.31** .03 -.31** .15* -.12 .11 -.11 -.15* -.32** .04 -.07 -.31** .50** .02 -.49** .83** .23** -.10 

**p < .01; *p <. .05 
7

can be substituted for individual Complexity-Q ratings, 

thereby reducing the number of predictors without losing 

information about their interrelationships.

Prior to the analysis, the Kaiser-Meyer-Olkin (KMO) 

measure of sampling adequacy was examined to test 

whether partial correlations among the variables were 

small. KMO values of .6 and above are required for a 

good solution. A KMO of .87 was produced by the set of 

variables selected. A PCA with Varimax rotation converged 

in nine iterations and produced four components with 

eigenvalues > 1. These components accounted for approximately 62% of the variance in the dataset. As shown in the 

rotated component matrix in Table 4, all but one of the 

variables had a loading of .50 or greater with at least one 

of the components. Only Relative frequency of complex 

routings failed to demonstrate a strong relationship with 

any one dimension, with respective loadings of .26, -.35, 

.35, and .39 on the four extracted components.

Component 1 had an eigenvalue of 4.93 and accounted for approximately 22% of the variance in the 

dataset. The variables with high loadings on this component seem to describe activity related to climbing 

and descending aircraft in the vicinity of major airports. 

When considering these kinds of flights, it is easy to see 

how the variables that describe this component relate to 

one another. For example, arrival and departure traffic 

associated with Major airports (.68) would tend to increase the Number of climbing and descending aircraft 

(.79). Airspace around major airports tends to have more 

Intersecting flight paths (.63). Traffic volume (.59) also 

tends to be higher proximal to major airports. Increased 

traffic directly impacts the Amount of radio frequency 

congestion (.58) and in some sectors would increase the 

Mix of aircraft types (.69). The Amount of coordination 

required (.74), Number of multiple functions (.73), and 

Number of required procedures (.66) represent tasks the 

controller must perform in complex airspace that often 

surrounds larger airports and would become more exigent in conjunction with the other factors. Perhaps the 

association of the Size of sector airspace (.51) represents a 

relationship between the size of the sector and the impact 

of these activities. In other words, their effects may be 

mediated by the amount of time available for resolution 

or completion.

Component 2 had an eigenvalue of 3.67 and accounted 

for approximately 17% of the variance. This component 

comprises complexity issues associated with low-altitude 

sectors that provide approach services into Non-towered 

airports (.91), the variable with the highest loading on 

this component. In the Indianapolis airspace, 11 sectors 

Table 4. Principal Components Analysis Rotated Component Matrix

Component

Variable 1 2 3 4 

Climbing and descending traffic .79 

Mix of aircraft types .69 

VFR versus IFR traffic .90 

Number of intersecting flight paths .63 

Number of multiple functions .73 

Traffic volume .59 .51

Amount of military or other special traffic .76 

Number of required procedures .66 

Coordination/ interfacing required .74 

Number of major airports .68 

Extent operations are affected by weather .74

Relative frequency of complex routings 

Special Use Areas .72 

Size of sector airspace .51 .56 

Longitudinal spacing/ sequencing .54

Adequacy of radio/radar coverage .60 

Amount of radio frequency congestion .58 

Traffic Management Initiatives .59

Terrain/Obstructions .88 

Shelves/Tunnels .53 

Foreign aircraft/pilots .60 

Non-towered airports .91 

* Component loadings < .50 not shown. 
8

provide approach services to airports without towers. 

Radar coverage does not always reach to the ground, so 

a loading of .60 for the Adequacy of radio/radar coverage 

may reflect difficulties associated with this factor. Aircraft 

flying VFR would also be present in these low-altitude 

sectors, thus increasing the ratio of VFR versus IFR 

traffic (.90), the second-highest loaded variable. The 

complexity factor Terrain and other obstructions (.88) 

is exclusively low-altitude and is relevant in sectors that 

provide approach services. In contrast, sectors with nontowered airports do not have a high mix of aircraft types 

(as most aircraft have lower performance characteristics), 

have low traffic volume, and limited frequency congestion. 

Moreover, these sectors have limited multiple functions, 

coordination, procedures, complex routings, spacing and 

sequencing, traffic management initiatives, and shelves/

tunnels. Thus, it makes sense that complexity ratings for 

those factors did not load on Component 2.

Component 3 had an eigenvalue of 2.78 and accounted 

for approximately 13% of the variance in the dataset. 

Variables associated with this component are primarily 

related to Military operations (.76) and other Special 

Use Area (SUA) restrictions (.72). In the ZID airspace, 

tunnels are associated with military operations. This might 

account for the relationship of Shelves/Tunnels (.53) with 

this dimension. Restrictions due to military operations 

would reduce the amount of usable airspace. Thus, a 

.56 loading of Size of sector airspace on this dimension 

makes sense. However, it is unclear why Foreign aircraft/

pilots would be associated with this component to such 

a high degree (.60). Perhaps the unifying theme of this 

dimension is that each of these factors warrants special 

consideration or attention, and Foreign aircraft/pilots 

fall into this category.

Component 4 had an eigenvalue of 2.35 and accounted 

for approximately 11% of the variance. The variables 

most strongly associated with this component relate to 

difficulties associated with inclement weather, as evidenced 

by the two highest-loaded variables, Extent operations 

are affected by weather (.74) and Traffic Management 

Initiatives (.59). Requirements for longitudinal spacing/ 

sequencing (.54) and problems associated with Traffic 

volume (.51) are also magnified by inclement weather.

The test for a successful PCA is simple: A good PCA 

makes sense. That certainly appears to be true of the 

components extracted from the Complexity-Q ratings. 

In this particular application, the component scores 

can be used to empirically test some of the assumptions 

made during component interpretation. For example, 

if Component 2 really addresses complexity factors 

endemic to low-altitude sectors that provide services to 

uncontrolled airports, then Component 2 scores should 

be significantly higher in those sectors. If Component 3 

relates to problems associated with military airspace and 

SUAs, then factor scores should be higher in sectors that 

have this kind of airspace.

To gain a clearer understanding of how the components 

relate to sectors in different altitude strata, a one-way 

analysis of variance was conducted for each of the 

components, using mean component scores as the 

dependent variable. The results demonstrated that mean 

component scores were significantly different between at 

least two of the sector groups for Component 1 [F(2,34) = 

7.85, p < .01], Component 2 [F(2,34) = 27.87, p < .01], 

and Component 4 [F(2,34) = 6.57, p< .01].

When viewing the results from post hoc comparisons 

(Tables 5-7), keep in mind that component scores are 

composite variables calculated from individual responses 

weighted by the component loadings. Consequently, the 

means no longer resemble the original 100-point scale 

ratings. Post hoc tests revealed that high-altitude sectors 

had significantly greater mean Component 1 scores than 

super high-altitude sectors (see Table 5). Although mean 

Component 1 scores appeared to be greater for highaltitude vs. low-altitude sectors, this difference was not 

statistically significant.

As anticipated, low-altitude sectors had significantly 

higher mean Component 2 scores than other types of 

sectors (Table 6). The mean of Component 2 scores for 

all low-altitude sectors was .93. However, the complexity 

factors most closely associated with Component 2 better 

describe low-altitude sectors that provide services to 

Table 5. Component Score Descriptive Statistics and Post hoc Comparisons†:

Component 1 by Sector Altitude Strata 

Strata Low High Mean SD

Low -.08 .61 

High p =.05 .46 .51 

Super High p =.21 p =.00 -.50 .62 

† Tukey HSD
9

non-towered airports. Mean Component 2 scores in 

these sectors (1.41) were considerably higher than mean 

Component 2 scores in low-altitude sectors that do not 

provide such services (-.21). Low-altitude sectors had 

significantly lower Component 4 scores than high- and 

super high-altitude sectors. This suggests that difficulties 

related to inclement weather were rated lower in lowaltitude sectors than in the higher altitudes. As with 

Component 2, there were no significant differences 

between high- and super high-altitude sectors for 

Component 4 scores (Table 7).

Component 3 scores did not vary significantly between 

sectors with different altitude stratum. This is not 

surprising, because the variables most strongly associated 

with this component (e.g., special routes, restricted areas) 

are found at all altitudes. On the other hand, Component 

3 scores should be higher in sectors subject to these kinds 

of restrictions. Based on information derived from sector 

descriptions in the SOPs, sectors with restricted areas, 

military training routes, aerial refueling routes, and/or 

MOAs were grouped for comparison with other sectors. 

Of the 37 sectors sampled, 26 met one of more of these 

criteria. Mean Component 3 scores were slightly higher in 

these sectors (.02) than in others (-.09), but this difference 

was not significant; t(35) = -6.12, p = .55.

Comparison of PCA Results with Rodgers, Mogford, and 

Mogford (1998)

Although differences in measures and methodologies 

between the present study and Rodgers et al. (1998) render 

direct comparisons problematic, there were similarities 

between some of the components that are worthy of note. 

Table 8 contains loadings for the first and third extracted 

components from both studies. The correspondence 

between these components is readily apparent, particularly 

with regard to Component 1. Rodgers and coworkers 

named their first extracted component Traffic Activity 

because the variables associated with it “appeared to be 

related to traffic volume and activities associated with 

managing aircraft” (p. 13). For the same reason, the name 

might accurately describe our Component 1.

Size of sector airspace was positively associated with 

both Component 1 and Component 3 in our model but 

had only a small negative loading on Component 1 and a 

small positive loading on Component 3 in Rodgers et al. 

(1998). In their analysis, Size of sector airspace was strongly 

associated with Component 2 (not shown). This is hardly 

surprising as their Component 2 was named “Size” and 

consisted almost entirely of static sector characteristics 

excluded from our analysis. By the same token, none of 

the components extracted in the Rodgers et al. analysis 

could have corresponded with our “Non-towered airport” 

Component 2 because they did not analyze Non-towered 

airports, VFR versus IFR traffic, or Terrain and other 

obstructions as complexity factors.

The third extracted component in both analyses was 

related to Military operations and other Special Use Area 

restrictions. In the Rodgers et al. (1998) study, Radio and 

radar coverage had a prominent negative loading, which 

was inconsistent with our findings. While recognizing 

that this variable “did not seem to be directly related to 

military functions,” they suggested that it might have been 

“conceptually associated with military airspace [because] 

areas that have poor radio and radar coverage, or are 

Table 6. Component Score Descriptive Statistics and Post hoc Comparisons†:

Component 2 by Sector Altitude Strata 

Strata Low High Mean SD

Low .93 .93 

High p =.00 -.54 .23 

Super High p =.00 p =.93 -.62 .15 

† Tukey HSD

Table 7. Component Score Descriptive Statistics and Post hoc Comparisons†:

Component 4 by Sector Altitude Strata

Strata Low High Mean SD

Low -.54 .80 

High p =.02 .11 .50 

Super High p =.01 p =.81 .27 .32 

† Tukey HSD
10

Table 8. Comparative Component 1 and Component 3 Loadings 

Component 1 Component 3 Variable Current

study 

Rodgers et al. 

(1998)

Current

study 

Rodgers et al. 

(1998)

Climbing and descending traffic .79 .84 .11 -.10 

Mix of aircraft types .69 .18 .31 .07 

VFR versus IFR traffic .01 NA .09 NA 

Number of intersecting flight paths .63 .45 .08 .34 

Number of multiple functions .73 .83 .18 .30 

Traffic volume .59 .63 -.15 -.14 

Amount of military or other special traffic .11 .13 .76 .68 

Number of required procedures .66 .85 .14 .17 

Coordination/ interfacing required .74 .48 .20 .33 

Number of major airports (Hubbing) .68 .55 .11 -.38 

Extent operations are affected by weather .18 .66 .21 .11 

Relative frequency of complex routings .26 .55 .35 .06 

Special Use Areas .20 .10 .72 .48 

Size of sector airspace .51 -.29 .56 .24

Longitudinal spacing/ sequencing .43 .57 -.02 -.32 

Adequacy of radio/radar coverage .14 .12 .45 -.59 

Amount of radio frequency congestion .58 .62 .32 -.09 

Traffic Management Initiatives .33 NA .08 NA 

Terrain/Obstructions -.02 NA .11 NA 

Shelves/Tunnels .21 .09 .53 .44 

Foreign aircraft/pilots .08 NA .60 NA

Non-towered airports .04 NA .02 NA 

controlled by the military, are relatively inaccessible to 

FAA ATC” (p. 13). This explanation could just as easily 

apply to our positive loading of .45.

In our analysis, Relative frequency of complex routings 

was not associated with any one component, but in the 

Rodgers et al. (1998) study, it had a definitive Component 

1 loading (.55). This probably reflects disparate sample 

characteristics. The only discernable divergence between 

ratings provided by supervisors and controllers in our 

sample was that supervisors consistently rated the Relative 

frequency of complex routings higher than controllers 

did. Unfortunately, we cannot make any inferences about 

the comparative influence of the Relative frequency of 

complex routings in ZID and ZTL because of these 

sample differences.

In spite of the dissimilarities noted, it is the 

similarities that are most striking. Considering statistical 

differences (i.e., rotated vs. un-rotated matrix, four vs. 

six extracted components), sample differences (i.e., 

ratings provided predominantly by controllers vs. ratings 

only provided by supervisors), variable set differences 

(i.e., 22 complexity factors vs. 16 complexity factors 

with variables derived from other sources), and airspace 

differences (i.e., ZID vs. ZTL), the obvious parallels 

between Components 1 and 3 and the dimensions they 

describe are remarkable.

Multiple Regression Analysis of Complexity Factor Scores 

With Operational Errors

As previously mentioned, one of the primary benefits 

of using PCA in this application is consolidation of the 

individual complexity factors into a reduced number of 

components. The method used to compute component 

scores (i.e., weighting individual responses according to 

that variable’s relationship with the dimension) tends to 

produce variables with normal distributions. Moreover, 

orthogonal rotation methods make it virtually impossible 

for the components, as a predictor set, to suffer from 

multicollinearity. Therefore, there is little question as to 

their appropriateness for multiple regression analysis. In 

this sample, the distribution of the number of OEs per 

sector had a mean of 6.68, with a standard deviation 

of 4.23 (Skewness = .61, SE Skewness = .388; Kurtosis 

= -.67, SE Kurtosis = .759). This was less than two 

standard deviations from normal in skewness and less 

than one standard deviation from normal in kurtosis. 

Consequently, both the predictors (the component scores) 

and the criterion (the number of OEs) met assumptions 
11

of normality. No univariate or multivariate outliers were 

detected. Studentized1 residuals plotted against predicted 

values were randomly distributed in a horizontal band 

around zero, indicating that the assumption of linearity 

and the assumption of equality of variance were met. Visual 

examinations of the cumulative probability plot of the 

observed distribution of residuals against that expected of 

a normal distribution demonstrated that the assumption 

of normally distributed errors was also met.

Standard multiple regression of the extracted 

complexity components on the number of OEs per 

sector produced a multiple R = .78 (R2 = .61) that was 

significantly different from zero, F(4,32) = 12.72, p <.01. 

Note that in this analysis the number of ZID sectors has 

been reduced from 40 to 37 due to sector combinations 

that were recommended by ZID personnel to facilitate 

the administration of the Complexity-Q questionnaire. 

As shown in Table 9, Components 1 and 2 contributed a 

significant amount of unique information to the model, 

whereas Components 3 and 4 did not.

Conclusions

The present study is the second phase of a multi-year 

project (called SCOpE) to better understand the contribution of sector complexity measures to the occurrence 

of OEs in en route air traffic control. To do this, we 

plan to measure complexity by examining both static 

and dynamic measures as well as subjective and objective measures. The static/dynamic distinction involves 

relatively unchanging characteristics of the sector (such 

as size, strata, length of jet ways and airways, number of 

airports, required procedures, number of shelves) and 

more transitory aspects of the traffic situation (such as 

amount of traffic, weather, aircraft mix, climbing and 

descending aircraft). The subjective/objective method 

incorporates both controllers’ opinions about the importance of complexity factors and impartial information 

extracted from environmental data.

1

 Studentized residual: The residual divided by an estimate of its 

standard deviation that varies from case to case, depending on the 

distance between the case value of the independent variable and the 

mean of the independent variable (Norušis, 1990).

Because limited data were available during the first year 

of the project and because of the lengthy processing time 

required to produce the objective dynamic data obtained 

during the second year, we have not yet been able to evaluate the effectiveness of the two measurement methods. 

In this study, we looked at subjective ratings for a set of 

combined static and dynamic sector characteristics. Next 

year, we will complete the project by adding objective 

dynamic sector information to the analysis.

In the first two years of the SCOpE project, we progressively analyzed the relationship between objectively 

measured static sector characteristics and OE occurrence in 

isolation. This does not mean that we believe they function 

in isolation. As Buckley et al. (1983) concluded, traffic 

density and sector geometry “interact in a complex way” 

(p. 73). The modest prediction (43%) of OE occurrence 

produced by the static variables sector size, sector strata, 

and number of major airports in the Goldman et al. 

(2006) study was to be expected because the corresponding objective dynamic variables with which they interact 

were missing from the equation. However, the analyses of 

static characteristics provided us with a baseline by which 

to evaluate the objective dynamic measures.

The controllers’ subjective ratings provided a perspective about the importance of complexity factors not 

available from the objective data. Furthermore, the PCA 

of subjective complexity ratings was extremely beneficial 

because the extracted components provided a complexity 

“profile” of the ZID sectors as perceived by the controllers 

who rated them. Four components were extracted that 

described different dimensions of complexity common 

to ZID sectors.

We interpreted the meaning of the components based 

on the loadings of the individual complexity factors. By 

examining the relationship between objective measures 

and the subjectively derived component scores, we were 

able to discern at least one dimension that was reliably 

associated with a particular type of sector (i.e., Component 

2, that described sectors providing services to non-towered 

airports). Thus, we identified specific complexity factors 

that are particularly relevant in these sectors.

Table 9. Multiple Regression Analysis: Complexity Component Scores on Number of 

Operational Errors (N = 37)

Variable B SE B t ȕ

Component 1 4.62 .69 6.67 .75** 

Component 2 -1.18 .51 -2.33 -.26* 

Component 3 1.54 .95 1.62 .19 

Component 4 -.33 .73 -.45 -.05 

** p = <.01; * p = <.05
12

Other complexity components were more difficult 

to describe. Component 1 appeared to be related to 

activities associated with the control of climbing and 

descending aircraft in the vicinity of major airports, and 

Component 3 seemed to be related to military operations 

and special use airspace. However, we were unable to 

verify our interpretations by comparing the component 

scores with external objective measures. Nevertheless, the 

concordance of our Components 1 and 3 with the first 

and third components in Rodgers et al. (1998) suggests 

that these dimensions, though difficult to describe, may 

be common to more than one facility. The number of 

possible predictors in the scientific literature and the lack 

of concordance between studies make the discovery of 

two potentially stable dimensions a major finding.

We then conducted a multiple regression analysis 

that used the four complexity components to predict 

OE occurrence. Only Component 1 (major airports) 

and Component 2 (non-towered airports) explained a 

significant amount of the variance in OEs in the ZID 

sectors (R = .78, R2 = .61). Component 1 was positively 

associated with the number of OEs (i.e., higher scores were 

related to a higher number of OEs), whereas Component 

2 had a negative relationship (higher scores were related to 

fewer OEs). Considering that the component scores were 

based on subjective ratings of sector complexity factors 

and did not include any variables explicitly describing 

individual behavioral or organizational aspects associated with OEs, 61% explained variance is impressive. 

However, components were constructed from “humanweighted” complexity factors, and so it is possible that 

the human element was not entirely missing from the 

equation. The relationship between Component 2 and 

the incidence of OEs reminds us that sector complexity 

does not always produce a negative outcome. Indeed, a 

certain degree or type of complexity may be related to a 

reduction in OEs.

The insight gained by analysis of the subjective ratings provided by ZID personnel (i.e., the identification 

of complexity dimensions and the variables most closely 

associated with them) will guide our efforts as we move 

on to the next phase of this project. Certainly complexity 

characteristics associated with Components 1 and 2 will 

be the subject of close scrutiny. Of course, complexity 

factors associated with the other dimensions will also 

be of interest. The fact that Component 3 failed to 

contribute significantly to the prediction of OEs in this 

analysis does not mean that military airspace or SUAs 

don’t make a sector more difficult to work or increase the 

likelihood of an OE. It simply means that subjective ratings of these factors failed to predict OEs. Similarly, the 

inability of Component 4 to contribute significantly to 

the regression model may reflect the intermittent nature 

of this dynamic event, or it may simply be an artifact of 

the way the variable was measured. In other words, the 

presence of inclement weather might be highly correlated 

with the occurrence of OEs but the component scores 

based on subjective ratings of variables associated with 

inclement weather were not.

The next phase of the SCOpE project involves analyzing objective measures that correspond to the subjective 

ratings of dynamic complexity factors examined in the 

present study. While static complexity factors (measured 

both objectively and subjectively) have so far had limited 

value in providing useful information about OE occurrence, objective dynamic measures should be much more 

descriptive and relevant.

Practical prediction models (linear or otherwise) must 

eventually be calculated from objective measures because 

the actual characteristics of the sectors must be addressed 

when developing strategies to reduce OEs. Nevertheless, 

subjective information about the importance of complexity factors can guide the choice of objective measures for 

analysis and may also be used to weight their importance. 

A potential application for the information gained from 

this study is a tool that could be used by airspace designers 

to evaluate the potential OE risk associated with different 

design concepts.

References

Arad, B.A. (1964, May). The control load and sector 

design. Journal of Air Traffic Control, 12-31.

Arad, B.A., Golden, B.T., Grambart, J.E., Mayfield, 

C.E., & Van Saun, H.R. (1963). Control load, 

control capacity and optimal sector design. (Report 

No. RD64-16). Atlantic City, NJ: Federal Aviation 

Administration. 

Buckley, E.P., DeBarysche, B.D., Hitchner, N., & Kohn, 

P. (1983). Methods and measurements in real-time 

air traffic control system simulation. (Report No. 

DOT/FAA/CT-TN83/26). Atlantic City, NJ: 

Federal Aviation Administration.

Buckley, E.P., O’Connor, W.F., & Beebe, T. (1969). 

A comparative analysis of individual and system 

performance indices for the air traffic control system. 

(Report No. NA-69-40). Atlantic City, NJ: Federal 

Aviation Administration.

Davis, C.G., Danaher, J.W., & Fischl, M.A. (1963). The 

influence of selected sector characteristics upon ARTCC 

controller activities (Contract No. FAA/BRD-301). 

Arlington, VA: Matrix Corporation.
13

Federal Aviation Administration (2005). Federal Aviation 

Administration Flight Plan 2005-2009. Available 

online at: www.faa.gov/about/media/Flight_Plan_

rev020905.

Goldman, S., Manning, C., & Pfleiderer, E. (2006). 

Static sector characteristics and operational errors. 

(Report No. DOT/FAA/AM-06/1). Washington, 

DC: Federal Aviation Administration Office of 

Aerospace Medicine.

Grossberg, M. (1989, April). Relation of sector complexity to operational errors. Quarterly report of 

the Federal Aviation Administration’s Office of Air 

Traffic Evaluations and Analysis. Washington, DC: 

Federal Aviation Administration.

Hilburn, B. (2004). Cognitive complexity in air traffic 

control – A literature review. (EEC Note No. 04/04). 

Brétigny-sur-Orge, France: EUROCONTROL.

Hurst, M.W., & Rose, R.M. (1978). Objective job difficulty, behavioral response, and sector characteristics in air route traffic control centers. Ergonomics, 

21(9), 697-708.

Kirwan, B., Scaife, R., & Kennedy, R. (2001). Investigating complexity factors in UK air traffic management. Human Factors and Aerospace Safety 1(2), 

125-144.

Kopardekar, P., & Magyarits, S. (2003, June). Measurement and prediction of dynamic density. Paper 

presented at the 5th Eurocontrol/FAA Air Traffic 

Management Research and Development Conference. Budapest, Hungary.

Laudeman, I., Shelden, S., Branstrom, R., & Brasil, C. 

(1998). Dynamic Density: An air traffic management 

metric. (Report No. NAS/TM-1998-112226.) Moffett Field, CA: NASA Ames.

Masalonis, A., Callaham, M., & Wanke, C. (2003, June). 

Dynamic Density and complexity metrics for realtime traffic flow management. Paper presented at 

the 5th Eurocontrol/FAA Air Traffic Management 

Research and Development Conference. Budapest, 

Hungary.

Mogford, R.H., Guttman, J.A., Morrow, S.L., & 

Kopardekar, P. (1995). The complexity construct in air 

traffic control: A review and synthesis of the literature.

(Report No. DOT/FAA/CT-TN95/22). Atlantic 

City, NJ: Federal Aviation Administration.

Mogford, R.H., Murphy, E.D., & Guttman, J.A. (1994). 

Using knowledge exploration tools to study airspace 

complexity in air traffic control. International Journal of Aviation Psychology, 4(1) , 29-45.

Norušis, M.J. (1990). SPSS Advanced statistics student 

guide. Chicago: SPSS.

Rodgers, M., Mogford, R., & Mogford, L. (1998). The 

relationship of sector characteristics to operational 

errors. (Report No. DOT/FAA/AM-98/14). Washington, DC: Federal Aviation Administration Office 

of Aviation Medicine.

Schmidt, D.K. (1976). On modeling ATC workload 

and sector capacity. Journal of Aircraft, 13(7), 

531-537.

Stager, P., Hameluck, D., & Jubis, R. (1989). Underlying 

factors in air traffic control incidents. Proceedings of 

the Human Factors Society 33rd Annual Meeting, 

43-46.

Stein, E. (1985). Air traffic controller workload: An examination of workload probe. (Report No. DOT/FAA/

CT-TN84/24). Atlantic City, NJ: Federal Aviation 

Administration.

Tabachnick, B.G., & Fidell, L.S. (1989). Using multivariate statistics (2nd ed.). New York: HarperCollins.

A-1

APPENDIX A 

Principal Components Rotated Component Matrix 

Component 

Variable 1 2 3 4 

Climbing and descending traffic .79 .02 .11 .11

Mix of aircraft types .69 .22 .31 -.13

VFR versus IFR traffic .01 .90 .09 -.17

Number of intersecting flight paths .63 -.40 .08 .14

Number of multiple functions .73 .07 .18 .22

Traffic volume .59 -.28 -.15 .51

Amount of military or other special traffic .11 .19 .76 .19

Number of required procedures .66 -.08 .14 .37

Coordination/ interfacing required .74 .13 .20 .12

Number of major airports .68 -.12 .11 .32

Extent operations are affected by weather .18 -.03 .21 .74

Relative frequency of complex routings .26 -.35 .35 .39

Special Use Areas .20 .10 .72 .20

Size of sector airspace .51 -.20 .56 -.15

Longitudinal spacing/ sequencing .43 -.32 -.02 .54

Adequacy of radio/radar coverage .14 .60 .45 .06

Amount of radio frequency congestion .58 .08 .32 .15

Traffic Management Initiatives .33 -.47 .08 .59

Terrain/Obstructions -.02 .88 .11 -.09

Shelves/Tunnels .21 .16 .53 -.16

Foreign aircraft/pilots .08 -.10 .60 .45

Non-towered airports .04 .91 .02 -.16