Relationship of the 

Aircraft Mix Index With 

Performance and Objective 

Workload Evaluation 

Research Measures and 

Controllers’ Subjective 

Complexity Ratings

Elaine M. Pfl eiderer

Civil Aerospace Medical Institute

Federal Aviation Administration

Oklahoma City, OK 73125

August 2005

Final Report

DOT/FAA/AM-05/16

Offi ce of Aerospace Medicine

Washington, DC 20591
NOTICE

This document is disseminated under the sponsorship of

the U.S. Department of Transportation in the interest of

information exchange. The United States Government

assumes no liability for the contents thereof.
i

Technical Report Documentation Page 

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No. 

DOT/FAA/AM-05/16 

4. Title and Subtitle 5. Report Date 

Relationship of the Aircraft Mix Index With Performance and Objective August 2005 

Workload Evaluation Research (POWER) Measures and Controllers’ 

Subjective Complexity Ratings 

6. Performing Organization Code 

 

7. Author(s) 8. Performing Organization Report No. 

Pfleiderer EM 

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS) 

FAA Civil Aerospace Medical Institute 

P.O. Box 25082 11. Contract or Grant No. 

Oklahoma City, OK 73125 

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered 

Office of Aerospace Medicine 

Federal Aviation Administration 

800 Independence Ave., S.W. 

Washington, DC 20591 14. Sponsoring Agency Code 

15. Supplemental Notes 

Work was accomplished under approved task AM-B05-HRR-522. 

16. Abstract 

Aircraft mix (i.e., the mix of aircraft with different performance characteristics in a sector) has been repeatedly 

cited as a complexity factor in en route air traffic control. However, scant attention has been directed to a 

statistical examination of this relationship. The present study is the third in a series of investigations designed to 

define, quantify, and assess the validity of aircraft mix as a contributor to traffic complexity. Eighteen 30-minute 

samples of System Analysis Recording data were collected from the Fort Worth and Atlanta en route centers. 

Performance and Objective Workload Evaluation Research (POWER) measures and the Aircraft Mix Index 

(Pfleiderer, 2003a) were computed in 6-minute intervals for each of the 36 samples. Principal Components 

Analysis of the combined data sets produced four components with eigenvalues >1 accounting for approximately 

71% of the variance. The Aircraft Mix Index was most closely associated with Component 1, which was 

composed of variables generally associated with traffic complexity. These variables were used as predictors against 

a criterion of controllers’ subjective “Complexity” ratings in multiple regression analyses of low- and highaltitude sector samples. The Aircraft Mix Index failed to contribute significantly to the explained variance in the 

both the low-altitude (R=.69; R2

=.47) and high-altitude (R=.57; R2=.33) sector models. In the aggregate, the 

results suggest that although aircraft mix appears to be associated with traffic complexity, it may not be as 

influential as other complexity factors in the en route environment. 

17. Key Words 18. Distribution Statement 

Aircraft Mix, Air Traffic Control, Airspace Complexity, 

Workload

Document is available to the public through the 

Defense Technical Information Center, Ft. Belvior, VA 

22060; and the National Technical Information 

Service, Springfield, VA 22161 

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price 

Unclassified Unclassified 18

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized 

1

RELATIONSHIP OF THE AIRCRAFT MIX INDEX WITH PERFORMANCE AND

OBJECTIVE WORKLOAD EVALUATION RESEARCH MEASURES AND

CONTROLLERS’ SUBJECTIVE COMPLEXITY RATINGS

Aircraft mix has been proposed as one of the traffi c 

characteristics contributing to sector complexity in en 

route air traffi c control (e.g., Christien & Benkouar, 2003; 

Federal Aviation Administration [FAA], 1984; FAA, 1999; 

Grossberg, 1989; Histon, Aigoin, Delahaye, Hansman, & 

Puechmorel, 2001; Mogford, Murphy, Roske-Hofstrand, 

Yastrop, & Guttman, 1994; Robertson, Grossberg, & 

Richards, 1979). However, its reputation has primarily 

been based on anecdotal evidence with little attention 

focused on examining this relationship statistically. The 

present study is the third in a series of investigations 

designed to defi ne, quantify, and assess the validity of 

aircraft mix as a contributor to traffi c complexity in en 

route air traffi c control.

First in the series was an investigation of the salient 

features of aircraft mix as it relates to aircraft performance 

characteristics (Pfl eiderer, 2000). For this analysis, 30 

Certifi ed Professional Controllers (CPCs) from several 

en route centers across the United States provided average 

speed, climb, and descent rate estimates for a sample of 

30 distinct aircraft types. A matrix of squared Euclidean 

distances was derived from summary estimates (i.e., means 

of speed, climb, and descent rates) and used to construct 

a multidimensional scaling (MDS) model of controllers’ 

perceptions of the aircraft performance characteristics of 

the aircraft. Interpretation of the two-dimensional MDS 

model suggested that Dimension 1 was related to engine 

type, and Dimension 2 was associated with weight class. 

The results were interpreted as evidence of performancebased prototypes (for further explanation, see Pfl eiderer, 

2000). However, it was also evident from the position of 

the elements (i.e., aircraft types) in the derived stimulus 

space that it might be possible to develop a measure of 

aircraft mix using these two easily obtained variables.

Pfl eiderer (2003a) continued that line of investigation 

in a study designed to determine whether controllers’ 

perceptions of aircraft performance were comparable to 

the actual recorded performance of aircraft in a sample 

of live air traffi c data. In general, controllers’ perceptions 

of aircraft performance characteristics were similar to the 

actual performance of the aircraft in the recorded data. 

However, weight class was far less salient as a separate 

dimension in the model derived from the matrix of System 

Analysis Recording (SAR) data than in the model based 

on controller estimates. The relationship between weight 

class and engine type in the SAR data model was a clear 

reminder that weight class is a correlate of engine type (i.e., 

most piston-driven aircraft are Small, most turboprops 

are Large, all Heavy aircraft are jets). This result led to the 

conclusion that engine type alone was an appropriate and 

suffi cient dimension upon which to base the calculation 

of the Aircraft Mix Index. (For a complete description of 

Aircraft Mix Index calculations, see Appendix A.)

Based on the apparent success of the fi rst phase of the 

investigation, a second phase was initiated that focused 

on testing the Aircraft Mix Index for its ability to discriminate between altitude strata. After all, if the index 

had suffi cient variability and precision, it should be able 

to discriminate between high- and low-altitude sectors. 

This was based on the assumption that high-altitude sectors should have a lower incidence of aircraft mix due 

to the relatively low service ceilings of some aircraft, 

whereas low-altitude sectors should have a much higher 

incidence of aircraft mix because all aircraft must climb 

and descend through low-altitude airspace at some point 

in their fl ight. For this analysis, the Aircraft Mix Index 

was calculated in 15-minute intervals for all active sectors within a 1-hour sample of air traffi c data recorded 

at the Kansas City en route center (15 high-altitude and 

13 low-altitude sectors). As anticipated, values of the 

Aircraft Mix Index tended to be higher in low-altitude 

sectors than in high-altitude sectors. A comparison of the 

two groups using the distribution-free Mann-Whitney 

U statistic (Mann & Whitney, 1947) revealed that the 

Aircraft Mix Index was reliably different between highand low-altitude sectors.

Because the Aircraft Mix Index was able to discriminate 

between sector strata, it passed the “minimum test” to 

be considered as a possible addition to the suite of Performance and Objective Workload Evaluation Research 

(POWER) variables. POWER refers to a set of measures 

developed for quantifying en route air traffi c controller activity and task load (for a detailed description of 

POWER measures and methodology, see Mills, Pfl eiderer, 

& Manning, 2002). In the fi rst phase of the present study, 

I conducted an evaluation of the relationship between the 

Aircraft Mix Index and existing POWER measures using 

Principal Components Analysis (PCA). PCA is a statistical 
2

technique often used to reveal patterns of correlations 

among variables. Values in the component loading matrix 

produced by PCA represent the correlation of individual 

variables with the underlying dimension the component 

describes. If the Aircraft Mix Index was redundant with 

traffi c volume (as opposed to describing some aspect 

of the complexity associated with that traffi c) it should 

load onto the same component as the total number of 

controlled aircraft. On the other hand, if the Aircraft 

Mix Index provided information about the complexity 

associated with the presence of aircraft with different 

performance characteristics then it should load onto a 

component with others that relate to traffi c complexity. 

Moreover, its loading should be of suffi cient magnitude 

to suggest that this is a reliable relationship.

Though PCA offers insight into the relationship of the 

Aircraft Mix Index relative to the other POWER variables, 

it cannot tell us whether the information it provides is 

unique. More importantly, it does not address the larger 

question regarding the relative contribution of aircraft 

mix to traffi c complexity. Therefore, contingent upon 

the results of the PCA, a multiple regression analysis was 

conducted using a subjective criterion of “Complexity” 

provided by controllers from the each of the en route 

centers sampled. The predictor variable set consisted 

of those variables identifi ed by the PCA as being most 

closely related to the “Complexity” dimension/construct. 

The results of the multiple regression analysis should 

tell us whether aircraft mix (as measured by the Aircraft 

Mix Index) contributes a signifi cant amount of unique 

information to the prediction of controllers’ perceptions 

of sector complexity.

METHOD

System Analysis Recording data and subjective complexity ratings were generously provided by researchers 

associated with the Dynamic Density project (e.g., 

Kopardekar & Magyarits, 2003). Traffi c samples selected 

for the analyses were collected at the Fort Worth and 

Atlanta en route centers. The Fort Worth data consisted 

of samples from six high-altitude and three low-altitude 

sectors. The Atlanta data were from fi ve high-altitude and 

four low-altitude sectors. Two 30-minute samples were 

collected from each of the selected sectors (a total of 36 

samples). Traffi c sample descriptions and sector maps are 

provided in Appendix B.

Three controllers individually viewed Systematic Air 

Traffi c Operations Research Initiative (SATORI; Rodgers 

& Duke, 1993) re-creations and rated the complexity of 

the traffi c situation on a scale from one to seven (lowest to 

highest) at 2-minute intervals throughout the 30-minute 

sample time frame. For the current study, means of the 

individual controller ratings were averaged over 6-minute 

intervals to create a total of 180 observations.

POWER measures were computed in 6-minute intervals for each of the traffi c samples, producing a total of 

180 observations for each POWER measure. Variables 

selected for the PCA (shown in Table 1) consisted of fi ve 

POWER measures that have consistently demonstrated 

a relationship with controller activity and task load (e.g., 

Manning, Mills, Fox, Pfl eiderer, & Mogilka, 2001; Mills, 

Pfl eiderer, & Manning, 2002), and fi ve thought to relate 

to traffi c complexity. The selected Traffi c Complexity/

Proximity variables are relatively new additions to the 

Table 1. POWER Variables Selected for Principle Components Analysis (PCA)

POWER Variable 

Number of Controlled Aircraft 

Number of R-side Entries 

Number of R-side Entry Errors 

Number of RA-side Entries 

Number of RA-side Entry Errors 

Controller Activity/Task load 

Aircraft Mix Index 

Mean Lateral Distance 

Mean Vertical Distance 

Number of Altitude Changes 

Number of Heading Changes 

Traffic Complexity/Proximity 
3

POWER suite of measures. (Consequently, this analysis 

also represents a serendipitous opportunity to examine 

whether or not these variables do, in fact, appear to describe a separate dimension.) The reasons for including 

the Aircraft Mix Index have already been described in 

some detail, but the rationale behind the other variables 

in this group deserves some explanation.

There is little doubt that the number of aircraft within 

a sector affects controller workload. It is also doubtful 

that this measure alone suffi ciently captures all aspects 

of the complexity associated with that traffi c (Hilburn 

& Flynn, 2004; Mogford et al., 1994). One of the traffi c 

complexity issues that should be addressed is the relative 

position of the aircraft. For the suite of POWER measures, we have opted to incorporate summary measures 

of aircraft proximity (i.e., Mean Lateral Distance, Mean 

Vertical Distance). Though not as elegant as some measures (e.g., clustering techniques developed by Delahaye 

& Puechmorel, 2000), they do have the advantage of 

refl ecting the dimensions controllers use to evaluate 

aircraft separation.

The number of climbing and descending aircraft is 

well established as a contributor to traffi c complexity 

(e.g., Christien & Benkouar, 2003; Histon et al., 2001; 

EUROCONTROL, 2002b, as cited in EUROCONTROL, 2004; Grossberg, 1989; Robertson, Grossberg, 

& Richards, 1979; Stein, 1985). The number of altitude 

changes provides more information than a count of the 

number of aircraft in transition. Altitude changes have 

been shown to correspond well with the number of 

altitude clearances and may provide some indication of 

the amount of workload associated with monitoring the 

response to and the effectiveness of the issued clearance 

(Pfl eiderer, 2003b).

As shown in Figure 1, heading changes have the potential to profoundly impact the complexity of an air traffi c 

situation, whether they occur as part of the scheduled 

fl ight plan or in response to a clearance. It is not surprising, therefore, that heading changes have been shown to 

contribute signifi cantly to sector complexity (e.g., Laudeman, Sheldon, Branstrom, & Brasil, 1998). It should be 

noted that POWER only counts heading changes greater 

than or equal to 10o that persist for a minimum of 36 

seconds. These computer-detected heading changes have 

been shown to correspond well with the number of issued 

heading clearances (Pfl eiderer, 2003b).

RESULTS AND DISCUSSION

Principal Components Analysis

Descriptive Statistics

As shown in Table 2, the distributions of most of the 

variables selected for the PCA approximated normality. 

However, distributions of the Aircraft Mix Index, R-side 

Entry Errors, and Radar Associate Entry Errors deviated 

signifi cantly. Although assumptions regarding normality 

are not generally in effect when PCA is used descriptively, 

in this particular application it is important to remember 

that PCA is sensitive to the sizes of correlations. To the 

extent that normality fails, the solution may be degraded 

and this should be considered when interpreting the results 

of the analysis (Tabachnik & Fidell, 1989).

Preliminary Comparative PCA

Because the data were sampled from different populations (i.e., en route centers), preliminary comparative 

analyses were conducted to determine whether the two 

data sets were similar enough to justify combining them. 

A. Traffic Pattern Prior to Heading Changes B. Traffic Pattern After Heading Changes

Figure 1. Potential Effect of Heading Changes on the Complexity of Traffic Patterns

(Adapted from Hilburn & Flynn, 2004 after van Gent, Hoekstra, & Ruigrok,1997) 
4

This was necessary because the number of observations 

available from the individual facility data sets is somewhat small for a successful PCA. Indeed, according to 

Comrey’s (1973) guidelines for sample size in PCA, a 

sample size of 90 (i.e., that seen if the sites were analyzed separately) would fall somewhere between “poor” 

and “very poor.” Though it is certainly undesirable to 

overanalyze any set of data, the best (and possibly the 

only) way to determine whether or not two data sets are 

similar enough for a combined analysis is to conduct 

separate PCAs and compare the extracted components. 

For these analyses, a maximum of two components (i.e., 

the hypothesized number of dimensions in the selected 

variables) was chosen as the criterion for component 

extraction. This decision was based on the fact that we 

were only interested in the similarities between the two 

data sets and not in the subtle differences that might 

exist between them. In other words, we simply needed 

to know if comparable analyses would yield comparable

results. Specifying the maximum number of components 

that might be extracted ensured that the analyses were 

comparable.

In the Fort Worth sample, the two extracted components accounted for approximately 54% of the variance. 

Component 1 had an eigenvalue of 3.00 and accounted for 

approximately 30% of the variance, whereas Component 

2 had an eigenvalue of 2.40 and accounted for approximately 24% of the total variance.

In the sample of data from the Atlanta en route center, 

the two extracted components accounted for approximately 48% of the variance. Component 1 had an eigenvalue 

of 2.68 and accounted for approximately 27%, whereas 

Component 2 had an eigenvalue of 2.10 and accounted 

for approximately 21% of the total variance.

Though the two samples did not produce identical 

results, the extracted components were reasonably similar. As shown in Table 3, the fi rst extracted component 

in the Fort Worth sample (ZFW-1) was comparable to 

the second extracted component in the Atlanta sample 

(ZTL-2). Component ZFW-2 was similar to ZTL-1 

except for lateral and vertical distances. (Please note that 

the order of extraction is somewhat irrelevant. Varimax 

rotation tends to reapportion variance among components so they become relatively equal in importance.) 

The recommended procedure for statistically comparing 

the pattern and magnitude of component loadings is the 

Pearson product-moment correlation. The correlation 

of ZFW-1 and ZTL-2 yielded a coeffi cient of r = .79. 

The correlation of ZFW-2 and ZTL-1 produced an r

= .83. Both correlations were signifi cant at the <.01 

level, indicating that the data were similar enough for 

a pooled analysis.

Pooled Sample PCA

Principal components analysis is generally used in the 

exploratory stages of research when the exact number 

and nature of the dimensions are not known. Although 

the selected variables were hypothesized to represent 

elements of two dimensions, extraction of one or more 

additional components would not be entirely unexpected. 

Therefore, a minimum eigenvalue of 1.00 (as opposed to 

a specifi ed number of components) was selected as the 

criterion for component extraction, thus allowing for 

true exploration of the data.

Varimax rotation was selected for the analysis because 

it increases the interpretability of the solution. As the 

name suggests, varimax (variance maximizing procedure) 

simplifi es components by maximizing the variance of 

Table 2. Principal Components Analysis Descriptive Statistics (N = 180)

Variable Mean S.D. Skew.1

 Kurtosis2

Aircraft Mix Index 10.85 14.78 2.23 6.62

Mean Lateral Distance (nm) 48.65 13.84 .25 -.48

Mean Vertical Distance (ft/100) 54.59 17.77 .74 .53

Number of Altitude Changes 6.86 3.71 .37 -.13

Number of Heading Changes 4.41 3.03 .71 -.14

Number of Controlled Aircraft 14.82 4.02 .25 .41

Number of R-side Entries 33.99 11.74 .31 -.46

Number of R-side Entry Errors 2.23 2.35 1.78 4.13

Number of RA-side Entries 6.64 5.97 .98 .41

Number of RA-side Entry Errors .66 1.21 3.04 12.31

1

S.E. Skew. = .181; 2S.E. Kurt. = .360 
5

the loadings within components. Simply stated, varimax 

makes small loadings smaller and large loadings larger. 

This simplifi es interpretation of the components by making it more obvious which variables are associated with 

them (Tabachnik & Fidell, 1989).

Table 4 contains a correlation matrix of the variables 

selected for analysis. It is clear that many of the variables 

are related, though the magnitudes are moderate. Nevertheless, it is encouraging that the Aircraft Mix Index 

appears to be more closely associated with the “Complexity/Proximity” variables than with others in the matrix.

PCA with varimax rotation converged in eight iterations and produced four components with eigenvalues > 

1. These components accounted for approximately 71% 

of the variance in the data set. As shown in the rotated 

component matrix in Table 5, all variables loaded onto at 

least one component with a loading of .40 or greater.

Component 1 had an eigenvalue of 2.40 and accounted 

for approximately 24% of the variance in the data set. 

Without exception, the variables associated with this 

component were selected to represent various aspects of 

traffi c complexity. Values in the loading matrix describe 

the correlation of each variable with the underlying 

dimension the component represents. Notice that the 

Aircraft Mix Index has one of the highest loadings on 

this component.

Component 2 had an eigenvalue of 2.06 and accounted 

for about 21% of the variance. The two variables with 

the highest loadings, the number of controlled aircraft 

and the number of Radar controller computer entries, 

are straightforward activity measures. Radar controller entry errors tend to increase as controller activity 

increases. Therefore, a conservative loading of .54 on 

this component makes sense within the context of the 

other variables.

Component 3 had an eigenvalue of 1.35 and accounted 

for approximately 13% of the variance in the data set. 

Generally, components described by only two variables 

are considered to be unreliable and are not interpreted. 

However, this component has emerged in a number of 

analyses (e.g., Mills, Pfl eiderer, & Manning, 2002) suggesting that it is, in fact, reliable. The extraction of “Radar 

Associate Activity” as a separate component may refl ect 

the unique relationship that Radar Associate Entries and 

Errors share with other activity measures. When activity 

is relatively low, a Radar controller working alone has 

time to make entries on the Radar Associate’s console 

(because some entries can only be made from that console). As the traffi c situation becomes more demanding, 

the Radar controller no longer has time to make entries 

from the RA-side console. During peak hours, a Radar 

Associate controller is assigned to the sector and entries 

made from the RA-side console become more frequent. 

It is probably the distinctive “J-shaped” distribution of 

RA-Entries and their relationship to RA-Entry Errors 

that distinguishes these variables as a separate component. 

Therefore, Component 3 might be viewed as a subset of 

general activity.

Component 4 had an eigenvalue of 1.27 and accounted 

for approximately 13% of the variance. The same caveat 

regarding two-variable components applies to Component 

4, only in this case it may be more justifi ed. Components 

defi ned by only two variables may be reliable if the variables 

are highly correlated with one another (i.e., r >.70) and 

are relatively uncorrelated with others in the variable 

set. These variables fail to meet the fi rst criterion in that 

Table 3. Preliminary Comparative PCA – Rotated Component Matrices*

Variable Component Component 

ZFW-1 ZFW-2 ZTL-1 ZTL-2 

Aircraft Mix Index .78 .14 -.34 .75

Lateral Distance (nm) -.73 .29 .54 -.44

Vertical Distance (ft/100) .73 -.28 -.66 .24

Altitude Changes .86 .06 .19 .75

Heading Changes .58 .15 .33 .74

Number of Aircraft -.05 .68 .73 .30

R-side Entries -.35 .63 .71 .25

R-side Entry Errors .15 .73 .39 -.02

RA-side Entries -.14 .68 .60 .12

RA-side Entry Errors .23 .57 .39 .00

* Varimax rotation converged in 3 iterations for both analyses. 
6

Table 4. Correlation Matrix (N -= 180)

Aircraft 

Mix Index 

Lateral 

Distance 

Vertical

Distance 

Altitude 

Changes 

Heading 

Changes 

Controlled

Aircraft 

R-side 

Entries

R-side 

 Entry 

Errors

RA-side 

Entries

Lateral Distance -.34**

Vertical Distance .41** -.40** Altitude Changes .38** -.42** .34**

Heading Changes .31** -.24** .07 .48**

Controlled Aircraft -.04 .35** -.14 .16* .30** 

R-side Entries -.17* .25** -.33** .08 .09 .54**

R-side Entry Errors -.05 .06 -.07 .12 .04 .40** .39** R-side Entries -.06 .17* -.26** -.01 .15 .26** .14 .05 RA-side Entry Errors .05 .04 -.01 .11 .04 .08 .15* .35** .39** 

**p < .01; *p<. .05
7

the bivariate correlation between them (r = .35) is less 

than .70. As shown in the matrix in Table 4, bivariate 

comparisons between each of these variables with others 

in the set resulted in a considerable number of signifi cant 

correlations. Consequently, this component fails to meet 

criteria that would indicate it might be reliable. It is also 

important to note that the distributions of both these 

variables are severely positively skewed and leptokurtotic. 

Therefore, it is likely that the communality described 

by this component refl ects a similarity of distribution 

rather than of meaning. On the other hand, data entry 

errors tend to increase with the number of entries. As 

such, Component 4 might also be viewed as a subset or 

correlate of general activity.

The results of the PCA demonstrate that the Aircraft 

Mix Index was consistently associated with other variables thought to relate to traffi c complexity. Moreover, 

the magnitude of its loading (.71) suggests that this is a 

reliable relationship. This leads us to the next phase of 

the experiment: Multiple regression analysis using the 

variables associated with the “Complexity” dimension 

(Aircraft Mix Index, Mean Lateral Distance, Mean Vertical Distance, Number of Altitude Changes, and Number 

of Heading Changes) to predict controllers’ subjective 

“Complexity” ratings.

Multiple Regression Analysis

Perhaps the most important assumption of a regression 

analysis is that the observations are sampled from the 

same population. Although the preliminary comparative PCA indicated that the data from the two facilities 

were similar enough to justify pooling, the results of 

comparisons of the Aircraft Mix Index in high- and 

low-altitude sectors in a previous study (i.e., Pfl eiderer, 

2003b) suggested that high- and low-altitude sectors 

might constitute heterogeneous samples. Therefore, 

initial data screening was conducted by visually examining scatterplots of each predictor variable against the 

criterion with observations color-coded according to 

altitude strata. It was immediately apparent that highand low-altitude sectors should be analyzed separately. 

Unfortunately, splitting the sample resulted in a sample 

size of 70 for the low-altitude sectors (i.e., a 14:1 case 

to independent variable ratio). Ideally, we would want a 

ratio of 20 cases for every predictor to ensure suffi cient 

statistical power to detect small effect sizes and to accommodate measurement error. Nevertheless, a 14:1 ratio 

exceeds the absolute minimum requirement of fi ve cases 

for every predictor (Tabachnik & Fidell, 1989).1

Descriptive Statistics

Descriptive analyses were conducted separately for 

the high- and low-altitude samples. The criterion variable (i.e., Complexity ratings provided by controllers) 

originally consisted of discrete values representing anchor 

points along an underlying continuum of the controllers’ perceptions of traffi c complexity. However, the 

Complexity ratings used in this analysis represent the 

means of ratings taken every 2 minutes, summarized over 

6-minute intervals. As such, these ratings were normally 

distributed in both sample sets (see Table 6).

Table 5. Principal Components Analysis Rotated Component Matrix

 Component 

Variable 1 2 3 4 

Aircraft Mix Index .71 

Mean Lateral Distance (nm) -.67 

Mean Vertical Distance (ft/100) .61 

Number of Altitude Changes .80 

Number of Heading Changes .64 

Number of Controlled Aircraft .85 

Number of R-side Entries .79 

Number of R-side Entry Errors .54 .69

Number of RA-side Entries .89 

Number of RA-side Entry Errors .47 .79

* Component loadings < .40 not shown.

1

 Please note that splitting the sample into high- and low-altitude sectors was not an option for the principal components analysis. Simply 

stated, if N=90 was insuffi cient for PCA then N=70 was even more so.
8

In the low-altitude sector sample, the distribution of 

the Aircraft Mix Index deviated by as much as fi ve standard 

deviations in both skewness and kurtosis. Square root 

transformation of the Aircraft Mix Index in this sample 

reduced deviations to less than one standard deviation 

from normal. In the high-altitude sector sample, transformation of the Aircraft Mix Index was contraindicated 

because transformation would create unacceptable deviations in a distribution that was acceptable in its natural 

state. (Indeed, all the selected variables were normally 

distributed in the high-altitude sample.)

Correlations. Tables 7 and 8 contain matrices of Pearson’s product-moment correlations for all variable pairs 

in each of the sub-samples. In the low-altitude sample 

(Table 7), the Aircraft Mix Index exhibits a signifi cant 

relationship with both the criterion and several of the 

predictors. However, the Aircraft Mix Index failed to 

demonstrate a signifi cant bivariate association with any 

of the other variables in the high-altitude sector sample 

(Table 8).

Tests of Assumptions

Multicollinearity. Multicollinearity refers to a very 

strong linear relationship between sets of predictor 

variables that renders the regression coeffi cients unstable 

(Fox, 1991; Tabachnik & Fidell, 1989). It is important 

to note that it is not the bivariate correlations among 

the predictors that creates multicollinearity, but rather 

the multiple correlation of the regression of a particular 

predictor on the others. Therefore, the best way to test 

for multicollinearity in the predictor set is to conduct a 

series of regressions with each of the predictors taking 

turns as the criterion and examining the squared multiple 

correlations for perfect or near perfect values (which 

would indicate multicollinearity). The results of these 

tests when conducted on the selected set of independent 

variables revealed no indication of multicollinearity in 

either the high- or low-altitude samples.

Outliers. No univariate outliers (i.e., cases with values 

greater than three standard deviations from the mean) 

were detected in either the low-altitude or high-altitude 

sector samples. Mahalanobis distances using p < .001 

failed to uncover any multivariate outliers (i.e., cases with 

an unusual pattern of values) in either data set.

Table 6. Multiple Regression Analysis Descriptive Statistics

Low-Altitude Sample (N = 70)

Variable Mean S.D. Skew.1 Kurtosis2

Low Altitude Sample (N = 70) 

Complexity Ratings 2.65 1.05 .35 -.93

Aircraft Mix Index 22.91 17.53 1.43 2.95

Square Root Aircraft Mix Index 4.44 1.79 .35 -.02

Mean Horizontal Distance (nm) 36.23 7.54 .20 -.57

Mean Vertical Distance (ft/100) 66.46 17.99 .37 .25

Number of Altitude Changes 8.83 3.79 .05 -.05

Number of Heading Changes 5.10 3.10 .55 -.29

High-Altitude Sample (N = 110) 

Complexity Ratings 3.84 1.07 .13 -.26

Aircraft Mix Index 8.63 3.88 -.16 -.21

Mean Horizontal Distance (nm) 56.55 10.80 .28 .06

Mean Vertical Distance (ft/100) 47.04 12.88 .59 .37

Number of Altitude Changes 5.60 3.07 .26 -.47

Number of Heading Changes 3.97 2.91 .83 .10

1

Low-Altitude: S.E. Skew. = .287; High-Altitude: S.E. Skew. = .230

2

 Low-Altitude: S.E. Kurt. = .566; High-Altitude: S.E. Kurt. = .457
9

Table 7. Correlation Matrix: Low-Altitude (N = 70)

Complexity 

Ratings 

Aircraft Mix 

Index

(Transformed)

Mean

Lateral 

Distance 

Mean

Vertical

Distance 

Altitude 

Changes 

Aircraft Mix Index (Transformed) .34** 

Mean Lateral Distance -.04 .35* 

Mean Vertical Distance -.06 .30* .1

1

 

Altitude Changes .45** .29* -

.3

2

** .02

Heading Changes .67** .48** -

.0

4

 -.11 .54**

** p = < .01; * p < .05 

Table 8. Correlation Matrix: High-Altitude Sample (N = 110)

Complexity 

Ratings 

Aircraft 

Mix Index 

Mean

Lateral 

Distance 

Mean

Vertical

Distance 

Altitude 

Changes 

Aircraft Mix Index -.01 

Mean Lateral Distance -.28** .09 

Mean Vertical Distance -.23* -.08 -.10 

Altitude Changes .26** -.04 -.13 .29**

Heading Changes .48** -.07 -.22* .04 .39**

** p = < .01; * p < .05 

Table 9. Regression of POWER Complexity Variables on ATC Complexity Ratings: 

Low-Altitude Sample

Model Summary R R2

 Adj. R2 S.E. Est. 

 .68** .47 .42 .79 

Variable b S.E. ß sr2

Transformed Aircraft Mix Index .003 .072 .005 .00 

Mean Lateral Distance (nm) .001 .015 .009 .00 

Mean Vertical Distance (ft/100) .001 .006 .024 .00 

Number of Altitude Changes .032 .033 .113 .01 

Number of Heading Changes .216 .042 .620 .22**

Significance levels derived from t-tests: ** p < .01; * p <.05
10

Linearity and equality of variance. The assumption of 

linearity and the assumption of constant variance of Y for 

all values of X can be easily tested by visually examining a 

plot of residuals against predicted values. If both assumptions are met, there will be no systematic pattern in the 

plots. Studentized residuals against the predicted values 

were randomly distributed in a horizontal band around 

zero, indicating that the assumptions were met.

Normal distribution of errors. One of the simplest ways 

to test whether errors of prediction are normally distributed is by visual examination of a cumulative probability 

plot of the observed distribution of residuals against that 

expected of a normal distribution. If the two distributions are identical, a straight line results. Cumulative 

probability plots demonstrated that the assumption of 

normally distributed errors was met.

Independence of errors. Because the scenario data were 

collected sequentially, they were screened to determine 

whether the time series had produced systematic variance in errors. Unfortunately, statistical procedures for 

testing sequential correlation of adjacent error terms 

(e.g., the Durbin-Watson) were not designed to test discrete groups of sequential data. Therefore, Studentized 

residuals were plotted against the sequence variable and 

visually examined. Non-independence of prediction 

errors in these data would manifest itself in “zigzag” or 

“herringbone” patterns. That is, each sequential group 

of fi ve observations would have a discernable pattern of 

increasing or decreasing error. No patterns were detected, 

thus indicating that the assumption of independence of 

errors had been met.

Regression Model: Low-Altitude Sample

Standard multiple regression analysis of the low-altitude sector sample produced a multiple R=.69 which 

was signifi cantly different from zero, F(5,62) = 10.97, p

<.01. As shown in Table 9, the regression model derived 

from the selected variables accounted for approximately 

47% of the variance in Complexity ratings. Table 9 also 

contains unstandardized regression coeffi cients (b), their 

standard errors (S.E.), standardized regression coeffi cients 

(ß), and squared semipartial correlations (sr2

) for each 

of the predictors. In standard multiple regression, sr2

represents the unique contribution of a predictor to the 

total variance explained. It is clear that the Number of 

Heading Changes was the only variable that accounted 

for a signifi cant amount of unique variance (22%). The 

difference between R2

 and the sum of sr2 for all predictors in the variable set represents shared variance. Thus, 

23% of the variance described by R2

 was unique whereas 

24% was shared.

Regression Model: High-Altitude Sample

Because the value of the Aircraft Mix Index is set to 

“system missing” in the absence of any aircraft with differing performance characteristics within a given sector, 

there were a considerable number of missing values in 

the high-altitude data set. Nevertheless, standard multiple 

regression analysis produced a multiple R=.57, which 

was signifi cantly different from zero, F(5,40) = 3.94, p

<.01. As shown in Table 10, the only variable to account 

for a signifi cant amount of unique variance in this data 

set was Mean Vertical Distance (16%). The Number of 

Altitude Changes (7%), the Number of Heading Changes 

(5%), and Mean Lateral Distance (3%) added to the total 

31% unique variance explained, but these contributions 

were not statistically signifi cant. Again, the Aircraft Mix 

Index failed to contribute any unique information to the 

prediction of controllers’ Complexity ratings.

CONCLUSIONS

It is important to note that the list of predictors included in the regression analyses was not intended to be 

exhaustive. It is certainly possible, even probable, that 

other measures might account for additional variance in 

Table 10. Regression of POWER Complexity Variables on ATC Complexity Ratings: 

High-Altitude Sample

Model Summary R R2

 Adj. R2 S.E. Est. 

 .57** .33 .25 .80 

Variable b S.E. ß sr2

Aircraft Mix Index .000 .031 -.002 .00 

Mean Lateral Distance (nm) -.017 .013 -.175 .03 

Mean Vertical Distance (ft/100) -.034 .011 -.502 .16**

Number of Altitude Changes .123 .063 .321 .07 

Number of Heading Changes .108 .061 .250 .05 

Significance levels derived from t-tests: ** p < .01; * p <.05
11

controllers’ complexity ratings. Therefore, the results 

should not be interpreted as evidence that a single variable is suffi cient to describe complexity in the en route 

environment. The focus of the analyses was simply to assess 

the relative contribution of aircraft mix to sector complexity. The fact that the Aircraft Mix Index failed to explain 

a signifi cant amount of unique variance in controllers’ 

Complexity ratings was disappointing, particularly with 

respect to the low-altitude sample, but not entirely unanticipated. Historically, the evidence supporting aircraft 

mix as a complexity factor has been anecdotal rather than 

statistical, no doubt because aircraft mix was considered 

to be “non-quantifi able” (e.g., FAA, 1984). Certainly 

controllers’ verbal representations/reports constitute a 

valuable heuristic, but with such evidence comes the risk 

of misattribution. For example, a specifi c factor might 

be particularly salient to the controller during periods of 

perceived increases in “workload” or “complexity” and 

yet be a mere correlate of the factor that is actually driving their subjective experience. Thus, it is particularly 

important to make every attempt to quantify and assess 

each proposed complexity factor to determine its relative 

infl uence and relationship with other factors. 

This is not to say that aircraft mix should be automatically discounted based on the results of a single analysis. 

It is entirely possible that aircraft mix does not, in fact, 

share a linear relationship with complexity and therefore 

cannot be captured using a linear regression analysis. 

Hilburn and Flynn (2004) have proposed that linear 

regression is ill suited for the study of air traffi c complexity because “complexity factors combine in a non-linear 

way. Though the constellation of factors might well apply 

across contexts, the relative importance of each differs 

by context” (p. 200). 

Another potential reason that aircraft mix failed to 

describe a signifi cant amount of variance in controllers’ 

complexity ratings is that it might only be a relevant 

factor in a few sectors, but in those sectors it is a major 

contributor to traffi c complexity. Every sector is unique. 

This presents a challenge when attempting to build models 

that will generalize. It is, therefore, vital to investigate the 

potential of aircraft mix and other prospective complexity 

factors using data collected at multiple facilities with a 

number of different statistical strategies before drawing 

any fi rm conclusions.

In that same vein, as gratifying as it may be that all 

the variables selected to represent “Traffi c Complexity/

Proximity” in the principal components analysis formed 

a single dimension, there is no guarantee that this would 

be the case in all data sets. Neither do these variables 

represent a comprehensive list of factors that might relate 

to sector complexity. A considerable amount of work 

has been done in this area (for an excellent review of the 

literature, see EUROCONTROL, 2004) and many of 

the proposed measures will be considered as possible additions to the POWER variables. Each candidate will be 

tested with the same rigor as the Aircraft Mix Index using 

similar methodology (i.e., examining the validity of each 

measure individually, testing its performance within the 

framework of the POWER variable suite, then examining 

its contribution relative to an external criterion). Each 

iteration of this process brings us closer to developing a 

set of measures that might comprehensively describe the 

sector environment to better understand the nature of 

sector complexity and its effects on controller workload 

and performance.

REFERENCES

Christien, R., & Benkouar, A. (2003, June). Air traffi c 

complexity indicators and ATC sectors classifi cation.

Paper presented at the 5th USA/Europe Air Traffi c 

Management R&D Seminar, Budapest, Hungary.

Comrey, A.L. (1973). A fi rst course in factor analysis. New 

York: Academic Press.

Delahaye, D. & Puechmorel, S. (2000). Air traffi c complexity: Towards intrinsic metrics. Paper presented at 

the 3rd USA/Europe Air Traffi c Management R&D 

Seminar, Napoli, Italy.

EUROCONTROL (2004, March). Cognitive complexity 

in air traffi c control: A literature review. EEC Note 

No. 04/04. Brétigny-sur-Orge, France: Author.

Federal Aviation Administration (1984). Establishment 

and validation of en route sectors. (FAA Order No. 

7210.46). Washington, DC: Author.

Federal Aviation Administration (1999). Position classifi cation standard for air traffi c control: Series 

ATC – 2152 terminal and en route. Washington, 

DC: Author.

Fox, J. (1991). Regression diagnostics: An introduction.

Sage University Paper series on Quantitative Applications in the Social Sciences, 07-079). Newbury 

Park, CA: Sage.

Grossberg, M. (1989, April). Relation of sector complexity to operational errors. Quarterly report of 

the Federal Aviation Administration’s Offi ce of Air 

Traffi c Evaluations and Analysis. Washington, DC: 

Federal Aviation Administration.
12

Hilburn, B., & Flynn, G. (2004, March). Toward a nonlinear approach to modeling air traffi c complexity. 

Paper presented at the 2nd Human Performance 

Situation Awareness and Automation Conference, 

Daytona Beach, FL.

Histon, J.M., Aigoin, G., Delahaye, D., Hansman, R.J., 

& Puechmorel, S. (2001, December). Introducing 

structural considerations into complexity metrics.

Paper presented at the 4th USA/Europe Air Traffi c 

Management R&D Seminar, Santa Fe, NM.

Kopardekar, P., & Magyarits, S. (2003, June). Measurement and prediction of dynamic density. Paper presented at the 5th USA/Europe Air Traffi c Management R&D Seminar, Budapest, Hungary.

Laudeman, I.V., Sheldon, S.G., Branstrom, R., & Brasil, C.L. (1998). Dynamic density: An air traffi c 

management metric. (NASA/TM-1998-112226). 

Washington, DC: National Aeronautics and Space 

Administration.

Mann, H.B., & Whitney, D.R. (1947). On a test of 

whether one of two random variables is stochastically larger than the other. Annals of Mathematical 

Statistics, 18, p. 50-60.

Manning, C.A., Mills, S.H., Fox, C., Pfl eiderer, E.M., 

& Mogilka, H. (2001, May). The relationship between air traffi c control communications events and 

measures of controller taskload and workload. Paper 

presented at the 72nd Annual Scientifi c Meeting of 

the Aerospace Medical Association, Reno, NV.

Mills, S.H., Pfl eiderer, E.M., & Manning, C.A. (2002). 

POWER: Objective activity and taskload assessment 

in en route air traffi c control. (Report No. DOT/

FAA/AM-02/02). Washington, DC: Offi ce of 

Aerospace Medicine. 2

Mogford, R.H., Murphy, E.D., Roske-Hofstrand, R.J., 

Yastrop, G. & Guttman, J.A. (1994). Application 

of research techniques for documenting cognitive 

processes in air traffi c control: Sector complexity 

and decision making. (Report No. DOT/FAA/

CD-TN94/3). Atlantic City, NJ: Federal Aviation 

Administration Technical Center.

Pfleiderer, E.M. (2000). Multidimensional scaling 

analysis of controllers’ perceptions of aircraft performance characteristics. (Report No. DOT/FAA/

AM-00/24). Washington, DC: Offi ce of Aviation 

Medicine. 2

Pfl eiderer, E.M. (2003a). Development of an empiricallybased index of aircraft mix. (Report No. DOT/FAA/

AM-03/8). Washington, DC: Offi ce of Aviation 

Medicine. 2

Pfl eiderer, E.M. (2003b, May). Relationship between 

computer-detected altitude, heading, and speed 

changes with controller clearances in en route air 

traffi c control. Paper presented at the 74th Annual 

Scientifi c Meeting of the Aerospace Medical Association, San Antonio, TX.

Robertson, A., Grossberg, M., & Richards, J. (1979). 

Validation of air traffi c controller workload models. 

(Report No. DOT/FAA/RD-79/83). Cambridge, 

MA: U.S. Department of Transportation Research 

and Special Programs Administration – Volpe National Transportation System Center.

Rodgers, M.D., Duke D.D. (1993). SATORI: Situation 

assessment through the re-creation of incidents. (Report 

No. DOT/FAA/AM-92/12). Washington, DC: Offi ce of Aviation Medicine.2

Stein, E.S. (1985). Air traffi c controller workload: An examination of workload probe. (Report No. DOT/

FAA/CT-TN84/24). Atlantic City, NJ: Federal 

Aviation Administration Technical Center.

Tabachnik, B. G., & Fidell, L. S. (1989). Using multivariate statistics (2nd ed.). New York: HarperCollins.

van Gent, R., Hoekstra, J.M., & Ruigrok, R.C.J. (1997). 

Free fl ight with airborne separation assurance. In: Proceedings of the Confederation of European Aerospace Societies (CEAS) 10th European Aerospace 

Conference, 20-21, October 1997, Amsterdam, 

The Netherlands.

2

This publication and all Offi ce of Aerospace Medicine technical 

reports are available in full-text from the Civil Aerospace Medical 

Institute’s publications Web site: http://www.faa.gov/library/reports/

medical/oamtechreports/
1

APPENDIX A 

Calculation of the Aircraft Mix Index 

The first step in calculating the Aircraft Mix Index is assigning aircraft type codes, which have values 

ranging from one to four. Piston-driven aircraft are assigned a code value of 1, turboprops a value of 2, jet 

aircraft a value of 3, and high-performance jet aircraft a value of 4. The next step is creating a half matrix of 

aircraft type differences for all pairs of aircraft within a given sector. Table A1 lists aircraft type differences for 

the sample of aircraft in Figure A1. For instance, DAL589 is a commercial jet and has been assigned an 

aircraft mix code of 3. N149RJ is a turboprop with an aircraft mix code of 2. The aircraft mix difference 

between N149RJ and DAL589 is 1. The final step in the computation of the index involves summing all 

items in the half matrix. For example, the Aircraft Mix Index for the group of aircraft in Figure A1 is 17.00 

(see Table A1). For each minute of data, the Aircraft Mix Index is calculated for all aircraft pairs at 

approximately 12-second intervals and stored in an array. At the end of each minute, the mean of these values 

is calculated and sent to an array. These stored values are used to calculate the average Aircraft Mix Index for 

whatever processing interval (e.g., 15-minutes, 30-minutes, etc.) has been selected. 

Figure A1. Sample Sector 

With Aircraft Mix Codes 

Table A1. Aircraft Mix Index

DAL589 AAL123 N149RJ UAL556 BLADE76

AAL123 0 

N149RJ 1 1 

UAL556 0 0 1 

BLADE76 1 1 2 1 

N456CP 2 2 1 1 3 

Index = 17 4 4 4 2 3 

1

APPENDIX B 

Traffic Sample Descriptions and Sector Maps

Atlanta High-Altitude Airspace Samples

Sector Time (local) 

03 1945-2015 

03 2030-2100 

06 1918-1948 

06 1940-2010 

20 1730-1800 

20 1935-2005 

22 1725-1755 

22 1918-1948 

39 1450-1520 

39 2240-2310 

Atlanta Low-Altitude Airspace Samples

Sector Time (local) 

04 2005-2035 

04 2012-2042 

19 1240-1310 

19 1830-1920 

38 1645-1715 

38 1815-1845 

41 1330-1400 

41 1950-2020 

03

06

20

22

39
2

Fort Worth High-Altitude Airspace Samples 

Sector Time (local) 

28 0035-0105

28 1815-1845

46 1520-1550

46 1505-1535

47 1550-1620

47 1555-1625

48 1223-1253

48 2235-2305

49 1505-1535

49 1825-1855

86 1245-1315

86 1855-1925

Fort Worth Low- and Intermediate-Altitude* Airspace Samples 

Sector Time (local) 

29 1240-1310 

29 1845-1915 

75 1555-1625 

75 2235-2305 

96 1255-1325 

96 1325-1355 

* Low/Intermediate SFC to FL230