DOT/FAA/AM-00/24

Elaine M. Pfleiderer

Civil Aeromedical Institute

Federal Aviation Administration

Oklahoma City, Oklahoma 73125

July 2000

Final Report

This document is available to the public

through the National Technical Information

Service, Springfield, Virginia 22161.

Multidimensional Scaling

Analysis of Controllers’

Perceptions of Aircraft

Performance Characteristics

Office of Aviation Medicine

Washington, D.C. 20591

U.S. Department

of Transportation

Federal Aviation

Administration
NOTICE

This document is disseminated under the sponsorship of

the U.S. Department of Transportation in the interest of

information exchange. The United States Government

assumes no liability for the contents thereof.
i

Technical Report Documentation Page

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.

DOT/FAA/AM-00/24

4. Title and Subtitle 5. Report Date

Multidimensional Scaling Analysis of Controllers’ Perceptions of Aircraft July 2000

Performance Characteristics 6. Performing Organization Code

7. Author(s) 8. Performing Organization Report No.

Pfleiderer, E.M.

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)

FAA Civil Aeromedical Institute

P.O. Box 25082

Oklahoma City, OK 73125 11. Contract or Grant No.

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered

Office of Aviation Medicine

Federal Aviation Administration

800 Independence Ave, S.W.

Washington, DC 20591 14. Sponsoring Agency Code

15. Supplemental Notes

This work was accomplished under the approved task AAM-A-HRR-516.

16. Abstract

Thirty full performance level (FPL) en route air traffic control specialists participated in an

investigation of the salient features of aircraft mix, a proposed sector complexity factor.

Controllers rated the “familiarity” (i.e., frequency of encounter) of 30 selected aircraft. They also

provided weight class, engine number, engine type, cruising speed, climb, and descent rate

estimates for each aircraft. A matrix of squared Euclidean distances derived from summary

estimates (i.e., means of speed, climb, and descent) was used to construct a multidimensional

scaling model of the aircraft. Multiple regression interpretation revealed that Dimension 1 was

related to engine type, whereas Dimension 2 was associated with weight class. The position of

elements in the derived stimulus space indicated that controllers may develop performancerelated prototypes through the use of multiple cues derived from a number of sources. Results are

presented as justification for further investigation into potential advantages of providing

enhanced prediction cues (e.g., engine type and weight class) from a single source, which may

increase the efficiency of controller decision making and decrease perceived workload.

17. Key Words 18. Distribution Statement

Air Traffic Control, Complexity, Workload, Decision- Document is available to the public through the

Making National Technical Information Service,

Springfield, Virginia 22161

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price

Unclassified Unclassified 34

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized

TABLE OF CONTENTS

INTRODUCTION -------------------------------------------------------------------------------------------- 1

METHOD -------------------------------------------------------------------------------------------------------- 5

RESULTS --------------------------------------------------------------------------------------------------------- 8

DISCUSSION --------------------------------------------------------------------------------------------------11

CONCLUSIONS ----------------------------------------------------------------------------------------------14

REFERENCES--------------------------------------------------------------------------------------------------15

APPENDIXES

A. Aircraft Familiarity Questionnaire (AFQ) ------------------------------------------------------ A1

B. Summary Estimates and Number of Observations -------------------------------------------- B1

iii

1

MULTIDIMENSIONAL SCALING ANALYSIS OF CONTROLLERS’ PERCEPTIONS OF

AIRCRAFT PERFORMANCE CHARACTERISTICS

The domain of an en route air traffic control

specialist (ATCS) is the sector. Sectors are defined

both horizontally and vertically “and may overlie or

underlie airspace controlled by another sector or

approach control facility” (National Airspace System, 1995, p. 5:19). Note the irregular shape of the

sector shown in Figure 1. This sector lies adjacent to

others like one piece in a three-dimensional jigsaw

puzzle. No two sectors are identical, and the relative

complexity of each involves a complex interaction of

sector geometry and traffic-related factors (Buckley,

DeBaryche, Hitchner, & Kohn, 1983). Physical aspects (e.g., size, orientation of conflict points, airway

configuration, terrain, limitations of radio or radar

coverage) are considered in conjunction with traffic

characteristics when sector boundaries are defined.

This is due to the relationship between sector complexity and controller workload. “As contributors to

task demand, sector characteristics can be thought of

as workload generators. Workload is the controllers’

subjective response to the ‘objective’ conditions which

create sector complexity” (Mogford, Murphy, RoskeHofstrand, Yastrop, & Guttman, 1994, p. 4).

During the validation stage of an investigation of

controller workload models, five air traffic control

specialists were asked to evaluate the relative difficulty of a set of sectors. They were also asked to

provide explanations as to why they believed the

sectors differed in complexity. Aircraft mix was one

of 12 “difficulty factors” cited (Robertson, Grossberg,

& Richards, 1979). These same factors were later

used as the basis for an investigation of sector complexity relative to operational errors.1

 A group of 97

controllers and supervisors from the Chicago Air

Route Traffic Control Center (ARTCC) was asked to

estimate the degree to which each factor contributed

to the complexity of their particular sector or area of

specialization. Notably, aircraft mix was one of the

highest-ranked factors in ten of the 30 sectors sampled

(Grossberg, 1989).

Figure 1. Air Route Traffic Control Center Sector Example

1

 An operational error occurs when an ATCS allows two or more aircraft to violate separation minima.
2

At the time, it was assumed that aircraft mix

referred to problems associated with the disparate

performance capabilities of propeller and jet aircraft.

This assumption is consistent with sector design

principles included in Appendix 2 of Order No.

7210.46, which imply that complexity is increased

when aircraft with “vastly different performance characteristics” occupy a controller’s airspace (Federal

Aviation Administration, 1984, p. 2). Nevertheless,

this interpretation may be incomplete or inaccurate.

For example, what are the parameters of “vastly

different?” What constitutes aircraft mix from a

controller’s perspective? Participants were never asked

to elaborate on the salient features of aircraft mix in

either of the studies cited. Consequently, it is possible

that each controller’s definition was not uniform.

Results of an investigation conducted at the Jacksonville ARTCC (Mogford, et al., 1994) offer limited insight into the potential components (or

correlates) of aircraft mix. In the preliminary stages of

the study, aircraft mix was defined as the proportion

of commercial, private, and military traffic. The

number of aircraft flying Visual Flight Rules (VFR)

versus Instrument Flight Rules (IFR) was also considered to be a distinct factor. A subject matter expert

who provided detailed factor definitions introduced

engine type as an element of aircraft mix. In the final

list of 19 “candidate” sector complexity factors, aircraft mix was defined as “VFR, IFR, props, turboprops, and jets, etc.” (p. 37). Although this definition

seems extensive, it may be neither comprehensive nor

accurate. That is, verbal representations of aircraft

mix may be only marginally related to the underlying

dimensions of interest (see Payne, Bettman, &

Johnson, 1993).

It is logical to assume that the availability of

information would influence controllers’ underlying

representations as well as the expression of those

representations. For example, engine type may be an

excellent predictor of aircraft performance capabilities, but controllers do not have direct access to this

information. On the other hand, controlled aircraft

are accompanied by a datablock tag on the radarscope

that displays, among other items, the aircraft identifier (AID). From this information, controllers may

determine whether an aircraft is commercial, military, or private. For instance, commercial AIDs are

recognizable because they correspond to flight numbers (e.g., AAL1550 represents American Airlines

flight 1550). Military AIDs consist of a combination

of pronounceable words and numbers, whereas privately-owned aircraft are distinguishable because they

commonly fly under their aircraft registration number. Though all aircraft have registration numbers,

only private (general aviation) aircraft are identified

by them. Whether an aircraft is commercial, military,

or general aviation is a fairly accurate predictor of

engine type (ergo performance). Approximately 79%

of all commercial aircraft are jets (International Civil

Aviation Organization, 1995). Many military aircraft are also jets. In addition, military AIDs often

indicate whether the aircraft is a high-performance

jet or a larger aircraft designed for air transport. In

contrast, general aviation aircraft are often, though

not always, smaller, propeller-driven aircraft.

In addition to the AID, controllers are provided

with partial information regarding an aircraft’s weight

class. There are three weight class categories: Heavy,

Large, and Small. A single character (e.g., H), printed

on the flight progress strip and available in the flight

plan readout display, flags aircraft in the Heavy

weight class. The reason controllers are provided

with this information has to do with procedures

associated with wake turbulence. However, weight

class is also an imprecise indicator of engine type. Jets

are somewhat more likely to be Heavy than are other

engine types. Weight class may also serve as a predictor of climb and descent capabilities.

At best, AIDs and Heavy prefixes provide only a

general indication of aircraft performance capabilities. However, controllers do have access to more

specific information. For instance, aircraft type designators are alphanumeric labels that indicate the

make and model of an aircraft (e.g., a B737 is a

Boeing Model 737, a C150 represents a Cessna

Model 150, and a BE77 is a Beech Skipper). Designators are printed on the flight progress strip and as

part of the flight plan readout display. In addition,

each aircraft’s current speed is displayed on the

datablock. Because fuel is expensive, most aircraft

operate at average speeds that optimize fuel efficiency. As a result, controllers probably develop associations through repeated pairings of specific

designators and average speed ranges. Simultaneously,

broad classes of AIDs would be associated with both

designators and speeds. A model of these proposed

associations is provided in Figure 2.
3

It would be parsimonious for controllers to develop prototypes based on this information, rather

than rely on the recall of explicit ranges (Posner,

Goldsmith, & Welton, 1967; Solso, 1991). The

model in Figure 2 implies that performance-based

prototypes would probably correspond to commercial, military, and private classifications, due to the

salience and accessibility of the AID. Controllers’

expectations of aircraft performance would then be

based on comparisons with the prototypical aircraft

within each category. It is likely that these groups

would be characterized by graded membership and

“fuzzy boundaries” because the performance of some

aircraft would be more representative of the prototype than others (see Schwartz & Reisberg, 1991).

Thus, an aircraft might be characterized as “a slow

commercial jet,” or “an average commercial jet,” or

“just like a commercial jet.”

The model in Figure 2 also illustrates how air

traffic controllers might use AIDs to predict aircraft

performance. If broad classes of AIDs are associated

with prototypes, an aircraft’s AID could be used to

calibrate performance estimates when the designator

is not an effective cue (i.e., when the aircraft is

unfamiliar). In Figure 3, the current speed of the

unknown aircraft designator is 325 knots. A commercial AID would predict speed capabilities ranging

from 325 to 500 knots, suggesting that the aircraft

was cruising well below maximum. On the other

hand, a general aviation AID would indicate the

aircraft was operating at top speed.

The availability of information inherent to both

proposed models brings up an important point. Aircraft mix has been proposed as a traffic-related sector

complexity factor. In other words, controllers have

attributed their perceptions of increased workload to

450

200

225

250

275

300

325

350

375

400

425

175

475

500

MD80

PA46

E120

B737

T38

550

575

600

C141

General Aviation

Commercial

Military

AID

Legend

Designator Airspeed

Figure 2. Model of Prototype Development
4

“aircraft mix,” which they believe to be characteristic of

certain sectors. However, humans tend to confuse

inference with observation (e.g., Brown, Deffenbacher, & Sturgill, 1977). It is doubtful that controllers are aware of all the factors contributing to their

subjective experience. It is possible that one of the

components of aircraft mix involves inadequate or

inefficient cues for the prediction of aircraft performance. Controllers might be using information derived from a number of sources to estimate the true

predictors of performance (e.g., engine number, engine type, weight class) to which they have no direct

access. Workload in complex tasks has been related to

the number of information sources and the number

of cues (Proctor & Dutta, 1995). Inadequacies of

prediction cues, or increased workload due to multiple information sources, might only be detectable

under conditions in which the strain on the system

exceeded a threshold; as would be the case in a low

altitude sector with numerous aircraft of unknown

performance characteristics.

By presenting only aircraft designators as stimuli,

it should be possible to evaluate the effectiveness of

the designator as a retrieval cue for aircraft performance estimates (i.e., average speed, climb, and descent rates). These estimates may then be employed

to examine controllers’ perceptions of each aircraft’s

performance capabilities relative to other aircraft in

the sample by using multidimensional scaling (MDS)

analysis. MDS translates patterns of responding into

patterns of elements in a dimensional space. These

patterns should provide evidence of knowledge-based

prototypes. If prototypes are based on aircraft performance, adherence to the commercial, military, and

private classifications would be subject to a considerable number of exceptions. This would be particularly true for aircraft that are encountered frequently

because knowledge of exceptions to rules is gained

through experience. In summation, while the subtleties of graded membership and fuzzy boundaries

characteristic of prototype-based groups would be

lost by asking controllers directly to classify aircraft,

the nuances of the underlying dimensions governing

controllers’ perceptions might be revealed through

the use of multidimensional scaling.

450

200

225

250

275

300

325

350

375

400

425

175

475

500

Unfamiliar

Designator General Aviation

Commercial

AID

Legend

Designator Airspeed

IF THEN

Unfamiliar IF Designator THEN

Figure 3. Model of Aircraft Performance Prediction When the Designator

is Unfamiliar
5

METHOD

Participants

Thirty en route air traffic control specialists participated in the study. All were full performance level (FPL)

controllers meeting currency requirements. Six were in

Oklahoma City as participants for an Air Traffic Selection and Training (AT-SAT) test validation study conducted by the FAA. Additional data were provided by

specialists from the Boston (N = 7) and Kansas City (N

= 17) Air Route Traffic Control Centers. Volunteers

were treated according to guidelines established by the

American Psychological Association. All were assured

complete anonymity and reminded of their right to

terminate participation at any time.

Six women and 24 men participated in the study.

The mean age of participants was 40 with a standard

deviation of 6 years, 4 months. As shown in Table 1,

specialists had been at their current centers an average

of 11 years, 10 months and had been certified as FPL

controllers for an average of 9 years, 8 months.

Discrepancies between the number of years as an en

route FPL controller versus number of years as an air

traffic control specialist represent time spent in training, in other areas, or at other facilities.

It was important for participants to have experience

with the different altitude levels because the limitations

of some propeller-driven aircraft preclude flight at higher

altitudes. Of the 25 specialists who responded to items

pertaining to sector stratification, 100% had worked

high sector strata, 96% had worked low altitude, and

80% had worked ultra-high sectors. Though most

acquired their experience at a single center, one specialist had worked at two centers, and another at three. The

number of centers represented by participants was crucial for the generalizability of the results. Air Route

Traffic Control Centers and the number of responses

associated with each are listed in Table 2. These centers

constitute 40% of the ARTCCs in the continental

United States.

Table 1

Summary of Participants’ Professional Experience*

Mean SD Minimum Maximum

Amount of Time Yrs. Mos. Yrs. Mos. Yrs. Mos. Yrs. Mos.

In Current Area 9 1 5 4 0 1 25 0

At Current ARTCC 11 10 4 8 1 9 25 0

As En Route FPL 9 8 4 8 0 7 23 0

As ATCS 15 2 6 9 3 4 35 0

* Summary statistics were calculated in months and converted to years/months.

Table 2

Air Route Traffic Control Center Experience

Air Route Traffic Control

Center

Number of Responses

(N = 30)

Albuquerque 3

Atlanta 1

Boston 7

Houston 2

Jacksonville 1

Kansas City 17

Miami 1

New York 1

Total Responses 33
6

Materials

The Aircraft Familiarity Questionnaire (AFQ). The

AFQ is composed of 210 questions (excluding comments) that can be completed in 30-45 minutes. The

questionnaire is divided into two sections: Aircraft

Performance Characteristics and Familiarity.

In the Aircraft Performance Characteristics section of the AFQ, controllers were asked to supply

speed, climb rate, descent rate, weight class, engine

number, and engine type estimates for each of 30

aircraft. Space was also provided for comments regarding turn rate, limitations due to inclement

weather, or any other performance-related information participants might have wished to add.

In the Familiarity section, volunteers were asked

to estimate the frequency with which they had encountered the aircraft within the previous six months.

Aircraft never or very rarely seen were to be classified

Level 1. Aircraft were to be classified as Level 2 if the

controller might not have seen one for several days, or

if aircraft of this type are only found in certain

sectors. Aircraft encountered daily in virtually all

sectors were to be classified Level 3. A sample copy of

the AFQ is provided in Appendix A.

AFQ aircraft list selection. The AFQ comprises 30

aircraft drawn from an initial pool of 403 designators

extracted from the aircraft characteristics record of

New York Center’s (ZNY) Adaptation Control Environment System (ACES) tape. For the 30 aircraft in

the AFQ to be truly representative, it was important

to begin with a list of aircraft that might be encountered in any given airspace. Therefore, the New York

list was compared with aircraft characteristics records

from the Los Angeles (ZLA) and Cleveland (ZOB)

centers. All three records were relatively recent updates,

ranging from June 1996 (ZLA) to January 1998

(ZNY). Only designators found in all three lists were

retained. Further reduction of the remaining 314

aircraft types was accomplished in two phases. Phase

1 elimination addressed the availability of information and the distinctiveness of the stimuli. Phase 2

focused on maintaining a balance of commercial,

military, and private aircraft, while ensuring that

some aircraft in the sample would be less familiar

than others. In other words, Phase 2 was about

diversity and variability.

As shown in Table 3, most Phase 1 deletions were

due to lack of inclusion in Appendix A of the

7110.65L. The 7110.65L is the most recent version

of the air traffic control manual (FAA, 1998). Therefore, all air traffic control specialists would have

access to this information. This would not be true of

aircraft not listed in the 7110.65L. Multiple designators and “cross-designators” (i.e., designators that

have been reassigned) were also eliminated to avoid

stimulus ambiguity. For example, in the recent adoption of new designators aircraft with old designators

C135, KC35, KE35, and KR35 were grouped under

the new designator C135. Conversely, an aircraft

previously designated as T35 were assigned to either

the new designator PILL or PA28.

Phase 2 list reduction began with a search of log

reports generated by the National Airspace System

(NAS) Data Analysis and Reduction Tool (DART).

Log reports contain the aircraft identifier and type of

every aircraft crossing a given airspace. A 20-minute

sample of the July 1996 log report from the New York

ARTCC was examined for the incidence of AIDs

corresponding to the 148 aircraft types remaining

after Phase 1 deletions. There were 14 aircraft types

Table 3

Phase 1 List Reduction: Frequencies and Exclusion Criteria

A/C Deleted Exclusion Criteria

82 Not in the 7110.65L

64 Multiple Old Designators/ One New Designator

8 Multiple New Designators/ One Old Designator

4 “Cross-Designators”

7 Redundancies (Both Old and New Designators)

1 Test Dummy

166 Total Deletions
7

detected more than once within the 20 minutes

sampled (e.g., more than one B757 crossed the airspace). In that same time frame, 41 of the aircraft

types appeared only once or twice. The 93 aircraft

types that did not appear in the New York log report

were excluded from further consideration.

The next step involved examination of the July

1996 log report from the Atlanta ARTCC to determine the frequency of occurrence of the 55 remaining

aircraft types. The sample time frame was extended to

30 minutes to ensure that a sufficient number of

aircraft would be selected. Ten of the 14 designators

with multiple occurrences in the New York log report

were also detected more than once in Atlanta’s airspace. These ten designators, representing frequentlyencountered aircraft, were included in the final list.

Ten of the 17 general aviation aircraft that had been

detected only once or twice in the 20-minute New

York sample were retained for the final list by randomly sorting them and selecting the first ten. Ten

military aircraft, also with minimal frequency of

occurrence, were selected via the same process.

Aircraft Familiarity Data Analysis Tool (AFDAT).

Completed AFQ forms were transferred to an electronic format using the AFDAT, a Visual Basic

program developed to facilitate data entry and guard

against error. For example, two versions of the AFQ,

which differed in the order in which the aircraft were

presented, were given to participants. The AFDAT

“Version” option allows for items to be entered

according to version order, and automatically sorts

the aircraft alphabetically for output. AFDAT code

also prohibits entry of any number outside the possible range of values for categorical variables.

Evaluation of weight class, engine number, and

engine type responses are performed automatically by

AFDAT code, thereby eliminating the possibility of

scoring errors. In addition, the AFDAT generates a

categorical variable describing the exact nature of the

error (e.g., a turboprop mistaken for a jet, a piston

mistaken for a turboprop) when an incorrect engine

type response is detected. The AFDAT returns a

single speed, climb, and descent rate for each aircraft.

When two values are entered (e.g., a range of speed

capabilities), AFDAT code automatically computes a

mean estimate.

Design

Multidimensional scaling refers to a group of

descriptive procedures that transform data into

mapped elements in one or more spatial dimensions

(Kruskal & Wish, 1978). Theoretically, the configuration describes the underlying dimensions upon

which judgments were based. The appropriate data

for MDS analysis are proximities, numbers that indicate the similarity or dissimilarity of a set of objects.

Proximities may be obtained directly, or derived

mathematically from a set of variables. In this application, a single matrix of dissimilarity measures was

computed from summary estimates of performance

characteristics (i.e., mean speed, climb, and descent

rates). The squared Euclidean distance between vectors of estimates was computed for each aircraft. The

distances, therefore, represent composite measures of

controllers’ perceptions of each aircraft’s capabilities

relative to other aircraft in the sample. For example,

if summary estimates for the first and second aircraft

were:

The squared Euclidean distance for these aircraft

would be: (1-4)2

 + (2-5) 2 + (3-6) 2 = 27 (D. Nichols,

personal communication, November 18, 1998).

Aircraft sample size was evaluated prior to data

collection, and the number of aircraft was limited to

30 so that the time required to complete the questionnaire remained under one hour. Early in the

planning stages, it was understood that volunteers

would be donating their time and expertise gratis,

and it was considered to be unreasonable to request

any more than 30 to 45 minutes of time.

In addition to performance estimates, participants

were asked to provide information regarding a number of variables to be used for interpretation of the

MDS model (i.e., engine number, engine type, and

weight class). Tests of accuracy for these variables

might also prove highly informative. Estimates of

frequency of encounter for each aircraft were collected as well. Research has shown that humans are

relatively accurate with such judgments (Vlek, 1970;

as cited in Tversky & Kahneman, 1973). These

Speed Climb Descent

Aircraft 1 1 2 3

Aircraft 2 4 5 6
8

“familiarity ratings” were used in the multiple regression interpretation of the stimulus configuration and

to assess controllers’ certainty regarding all other

estimates.

Procedure

For the six participants drawn from the AT-SAT

pool, testing was conducted at the Assessment Center

located on the grounds of the Mike Monroney

Aeronautical Center in Oklahoma City, Oklahoma.

Testing took place from April 1 to 23, 1998, after

participants had completed the AT-SAT experimental protocol. The 15% who volunteered to remain

and take the AFQ were given consent forms to read

and sign. Once written consent had been obtained,

packets were distributed. Packets consisted of biographical data forms, instructions, and one of two

versions of the AFQ. As soon as the questionnaires

were finished, the specialists were debriefed.

Data collection at the Boston and Kansas City

ARTCCs took place between April 28 and May 24,

1998. Volunteers meeting subject criteria were solicited by a representative from each center. Both representatives had been thoroughly briefed on important

points of procedure, particularly with regard to obtaining written consent prior to dispensing the packets. Though test administration at the centers was

somewhat less formal, the basic experimental protocol remained unaltered.

RESULTS

Data Management

Text files generated by the AFDAT program were

aggregated to create a new data file. Observations in

the aggregated file represent summary APC scores

(weight class, engine number, engine type) and performance estimates (speed, climb, and descent) for

each aircraft. Prior to aggregation, distributions of

performance estimates were examined separately for

each of the 30 aircraft in the AFQ. Departures of

skewness and kurtosis for all 180 distributions (90

scores, 90 estimates) were reduced to a maximum of

3.36 (skewness) and 3.38 (kurtosis) standard deviations from that expected of a normal distribution (see

Tabachnik & Fidell, 1989, p. 72) by the removal of

outliers and extreme values (see SPSS Inc., 1990, p.

174 for criteria). After deletion, tests of normality

indicated the mean was a sufficient measure of central tendency for all observations. Summary observations and the number of cases they were based upon

are listed in Appendix B. Observations with N < 30

resulted from missing values, deletion, or both.

Missing Values

 Controller characteristics. The total number of

missing values was computed for each participant.

This variable was compared with biographical information stored in the BIODAT data file generated by

the AFDAT program. The distributions of chronological age (in years), time spent as an air traffic

controller (in months), and total number of missing

values approximated normality in terms of skewness

and kurtosis. The Pearson product-moment correlation coefficient (N = 29) found no reliable linear

relationship between missing values and chronological age (r = .12), or time spent as an air traffic

controller (r = .10).

Aircraft characteristics. The total number of missing values was computed for all variables and compiled by aircraft designator. This summary variable

was merged with the existing AFDAT data base for

comparison with APC measures. The distribution of

mean Familiarity ratings sufficiently approximated

normality for parametric analysis; the distribution of

total missing values did not. Departures of skewness

(Skewness = 2.23; S.E. Skewness = .43) and kurtosis

(Kurtosis = 4.59; S.E. Kurtosis = .83) were improved

with a square root transformation (Skewness = 1.38;

Kurtosis = 1.33). Nevertheless, a considerable number of outliers remained. Sample size contraindicated

deletion of cases, so Spearman’s correlation coefficient for ranked data was selected as the appropriate

measure of association. Ranking invariably results in

a loss of information, with a subsequent loss of

power. A relatively small sample size (N = 30) further

increased the risk of a Type II error. In spite of that,

ranks of total missing values demonstrated a significant inverse relationship with those of mean Familiarity ratings (rs

 = -.65, p < .01). Missing values

demonstrated a reliable positive association with the

percentage of incorrect engine type responses (rs

 =

.60, p < .01), but not with other APC measures. As

might be expected, mean Familiarity ratings and

incorrect engine type responses were significantly

and negatively related (rs = -.52, p < .01).
9

Multidimensional Scaling

Squared Euclidean distances were calculated from

standardized speed, climb, and descent rate summary

estimates and submitted to classical, nonmetric MDS

analysis. Both two- and three-dimensional models

demonstrated excellent fit. Squared correlations describe the relationship between the original distances

and the derived stimulus coordinates: Both solutions

produced an r2

 = .99. Kruskal’s stress 1 formula is

sometimes referred to as a “badness-of-fit” measure

because larger values indicate poorer fit. The twoand three-dimensional solutions yielded stress values

of .0098 and .0078, respectively. In the best of all

possible models, stress equals zero.

Selection of the appropriate dimensionality concentrated on issues of parsimony and interpretability.

Intercorrelations of stimulus coordinates in Table 4

reveal that Dimension 1 remained stable (r = 1.00)

regardless of dimensionality. Dimension 2 was relatively unchanged as well (r = -.97). Examination of

stimulus coordinates and regression analysis failed to

uncover any distinctive, interpretable features of the

added dimension. Because a higher dimensionality is

of little use if it contributes nothing to the interpretation of the solution (Kruskal & Wish, 1978), two

dimensions were deemed sufficient.

The most objective technique available for dimensional interpretation is the regression method. Variables

believed to correspond with the stimulus configuration

are regressed on vectors of coordinates. According to

Kruskal and Wish (1978), two conditions are necessary

for satisfactory interpretation of a dimension. First, the

multiple correlation must be extremely high (correlations in the .90s are recommended, although correlations in the .70s will suffice). As shown in Table 5, only

engine type (R = .92) and weight class (R = .80) achieved

the recommended degree of association with the dimensions. The percentage of incorrect engine type responses

produced an R = .65. Although the magnitude of this

association is less than the recommended level, significance exceeds the minimum requirement of .01 (Kruskal

& Wish, 1978, p. 39).

Table 4

Intercorrelations of Multidimensional Scaling Model Stimulus Coordinates for Two- and ThreeDimensional Solutions (N = 30)

Dimension 1 2 3 4 5

1 Dimension1 (of Two)  .05 1.00* .16 -.21

2 Dimension 2 (of Two)  .07 -.97* -.75*

3 Dimension 1 (of Three)  .15 -.22

4 Dimension 2 (of Three)  .65*

5 Dimension 3 (of Three) 

* Correlation is significant at p < .01 level (2-tailed).

Table 5

Summary of Multiple Regression Analyses Used to Interpret Dimensional Characteristics of the

Two-Dimensional MDS Model a

Criterion R R2 F p β1 β2

Engine Number .51 .26 4.65 .02 -.11 .50

Percent Incorrect: Engine Number .26 .07 .99 .38 -.24 -.10

Engine Type .92 .85 78.12 .000 -.84 .43

Percent Incorrect: Engine Type .65 .42 9.75 .001 .62 -.22

Weight Class .80 .63 22.59 .000 -.41 .70

Percent Incorrect: Weight Class .24 .06 .81 .45 -.23 -.05

AFQ Familiarity Rating .37 .13 2.08 .15 -.02 .37

Square Root of Total Missing .37 .14 2.17 .13 .28 -.26

a Bold items represent significant associations.
10

The second condition required for a satisfactory

interpretation is that the criterion must have a high

regression weight on the dimension. Standardized

regression weights show that Dimension 1 was related

to engine type (β 1 = -.84) and the percentage of incorrect

engine type responses (β 1 = .62), whereas Dimension 2

was associated with weight class (β 2 = .70). Table 6

contains a list of aircraft designators ordered by Dimension 1 coordinates. The relationship between engine

type and the arrangement of elements along this dimension

is evident. The association between modal engine type and

mean speed estimates (rs

 = .86, p < .01) is also apparent.

Notice that the proportion of errors increases around

“fuzzy boundaries” of engine type estimates.

Table 7 provides information about the exact nature

of engine type errors. Most errors (60%) involved

confusion between turboprops and piston-driven aircraft. Specialists were nearly twice as likely to mistake a

turboprop for a jet than to mistake a piston-driven

aircraft for a jet. Participants were least prone to mistake

a jet for any other kind of aircraft.

Table 6

Aircraft Type Designators, Engine Type Estimates, Percentage of Engine Type Estimates

Incorrect, and Speed Estimates Ordered by Dimension 1 Coordinates

Designator Engine Type Percent Incorrect Speed Estimate

C208 Piston 62% 177

BE36 Piston 7% 163

BE58 Piston 20% 191

AC50 Piston 47% 196

PA46 Piston 41% 203

V10 Turboprop 48% 224

E120 Turboprop 14% 258

SF34 Turboprop 7% 253

C2 Turboprop/Jet * 64% 320

D328 Turboprop 25% 272

JSTB Turboprop 39% 305

C97 Turboprop 75% 376

C560 Jet 3% 393

MD80 Jet 0% 453

C141 Jet 10% 450

B52 Jet 3% 475

A3 Jet 19% 377

L101 Jet 3% 492

MD11 Jet 0% 478

A310 Jet 0% 461

A300 Jet 0% 465

A320 Jet 0% 472

B757 Jet 0% 474

B767 Jet 0% 480

C650 Jet 0% 472

LJ55 Jet 0% 464

HAR Jet 0% 473

F111 Jet 0% 500

F16 Jet 0% 500

T38 Jet 0% 488

* Bimodal
11

The relationship between weight class and Dimension 2 was not as straightforward as that of

engine type and Dimension 1. Estimates of weight

class were related to those of climb rates (rs

 = .40) and

descent rates (rs

 = .41) at the p < .05 level of significance, but weight classes were not evenly distributed

along a continuous dimension. Table 8 contains a list

of aircraft ordered by Dimension 2 coordinates,

separated according to their position in the four

quadrants of the stimulus configuration map (see

Figure 4). Organized in this manner, at least three

distinct groups emerge: (a) heavy and large jet aircraft

with climb and descent capabilities greater than

2100 feet per minute (fpm) but less than 3000

fpm, and average cruising speeds greater than 450

knots,(b) small turboprops and pistons with slower

cruising speeds and somewhat limited climb and

descent capabilities, and (c) high-performance jets.

DISCUSSION

Visual examination of the stimulus configuration

map, shown in Figure 4, reveals three clusters of

aircraft. At first glance, these groups appear to correspond to commercial, military, and private classifications. For instance, aircraft in the upper-left region

are predominantly commercial air carriers. The lowerleft area consists primarily of military aircraft, whereas

general aviation aircraft dominate the lower-right

quadrant. Upon closer examination, however, the

perception of discrete categories must be dismissed.

By definition, categories have boundaries, and all

elements within the boundary belong equally to the

group (Schwartz & Reisberg, 1991). Therefore, a

categorical interpretation cannot account for the

small cluster of three aircraft in the upper-right

quadrant. Neither can it explain the presence of civil

aircraft in the lower-left area, nor the number of

military aircraft scattered throughout the configuration. Clearly then, a strict categorical interpretation

is insufficient.

One solution may be prototype-based categories,

as described by Schwartz and Reisberg (1991), that

are not absolute. Prototypes are characterized by

graded membership and fuzzy boundaries. That is,

some members of the group are more representative

of the prototype than others. The systematic progression of participants’ speed estimates along Dimension 1, shown in Table 6, is typical of the graded

membership often associated with prototypes. Ranges

of climb and descent rates, shown in Table 8, also

suggest graded membership.

The position of the C650 and LJ55 in the stimulus

configuration suggests graded membership as well.

Speed estimates of the C650 and LJ55 were similar to

other high-performance aircraft, though not to the

same degree as military jets situated in the far left

corner of the configuration. Participants’ speed estimates for these aircraft were comparable to manufacturers’ published cruising speeds. In addition, speed

estimates for these aircraft had below-average standard deviations. However, it should be noted that

modal weight class estimates for the C650 and LJ55

were incorrect: Estimated to be Large aircraft, both

are actually in the Small weight class category. Some

participants greatly underestimated the climb and

descent capabilities of these aircraft, resulting in

standard deviations that were high, as compared with

Table 7

Frequencies and Percentages of Specific Engine Type Errors

Description of Error Frequency Percent

Turboprop Mistaken for Piston 35 30

Piston Mistaken for Turboprop 34 30

Turboprop Mistaken for Jet 23 20

Piston Mistaken for Jet 14 12

Jet Mistaken for Turboprop 6 5

Jet Mistaken for Piston 3 3

Total 115 100
12

the rest of the sample. It is possible that the position

of the C650 and LJ55 might have been closer to other

high-performance jets, had participants been aware

of their actual weight class.

Because prototypes are abstracted from stored information, they will generally accommodate exceptions (Posner, Goldsmith, & Welton, 1967). The

proximity of the C650 and LJ55 to military aircraft

such as the F111 and F16 demonstrates this phenomenon to some degree. Even more dramatic is the

appearance of military aircraft (i.e., the B52 and

C141) within the tightly-knit cluster of commercial

air carriers in the upper-left area of the configuration.

Controllers’ speed, climb, and descent rate estimates

for the B52 and C141 were within the same range as

other performance estimates in the group. According

to the manufacturers’ published averages, their estimates were extremely accurate. Thus, these civil and

military aircraft were related by performance characteristics.

Although prototypes allow for recognition of elements that are unusual, research involving pattern

recognition has demonstrated that well-learned forms

do not accommodate as wide a range of distortion as

less familiar ones (e.g., Petersen, Meagher, Chait, and

Gillie, 1973). Most controlled aircraft are commercial jets. If controllers develop prototypes, the commercial jet would be analogous to a well-learned

form, much as a triangle or a circle would be in

pattern recognition. One of the most distinctive

Table 8

Aircraft Type Designators, Estimates of Engine Type, ✹ Weight Class, ✦ Speed, Climb and

Descent Rates, Ordered by Dimension 2 Coordinates

Designator Engine/

Weight

Speed Climb Descent Designator Engine/

Weight

Speed Climb Descent

L101 J H 492 2284 2836

B52 J H 475 2180 2341

MD11 J H 478 2478 2583

MD80 J L 453 2144 2315

C141 J H 450 2320 2204

A320 J H 472 2714 2530

A300 J L 465 2580 2542

A310 J L 461 2496 2665

B767 J H 480 2893 2807

B757 J H 474 2915 2738

C560 J L 393 1985 2160

C2 T L 320 1628 1522

C97 T H 376 1892 2103

C650 J L 472 3148 3166

E120 T L 258 1347 1654

C208 P S 177 981 1010

PA46 T S 203 1180 1265

V10 T S 224 1412 1425

BE36 P S 163 1006 1193

BE58 P S 191 1096 1250

AC50 P S 196 1128 1309

LJ55 J L 464 3502 3633

SF34 T L 253 1673 1615

JSTB T L 305 2052 2150

D328 T L 272 1897 1847

F111 J L 500 4005 4540

A3 J L 377 2989 2627

HAR J L 473 3961 4221

F16 J L 500 4685 5333

T38 J S 488 5007 5280 ✹ J = Jet, T = Turboprop, P = Piston; ✦ H = Heavy, L = Large, S = Small
13

of these prototypes are performance-based. Multiple

regression interpretation revealed a strong relationship between Dimension 1 and engine type. However, this relationship does not appear to describe a

simple “jets, turboprops, pistons” delineation. As

mentioned previously, one of the most striking features of the configuration is the line of demarcation

between jets, to the left, and all other aircraft types.

Two distinct groups of jet aircraft are distinguishable

in the stimulus configuration: Jets with climb and

descent capabilities ranging from approximately 2200

to 3000 fpm, and high-performance jets with climb

and descent capabilities greater than 3000 fpm. The

third group comprises propeller-driven aircraft with

climb and descent capabilities ranging from approximately 1000 to 2150 fpm. Two subgroups consisting

of pistons and turboprops, barely visible in the stimulus configuration, are readily apparent in Table 6.

These subgroups are separated by fuzzy boundaries,

evidenced by patterns of engine type errors. The

features of the stimulus configuration map is the

nearly inviolable line separating jets and other aircraft. The only exception is the C560, Cessna Citation, a notoriously slow jet aircraft. According to

self-report estimates, controllers were extremely familiar with this aircraft. Of a possible 2.00, the C560

had a mean Familiarity rating of 1.90. As might be

expected, performance estimates for the C560 were

fairly accurate. Therefore, it is likely that the C560 is

proximal to turboprops and pistons because it does

not perform within the same parameters as other jet

aircraft. (As one controller stated, “The Citation is

really a turboprop masquerading as a jet.”) Thus, the

Citation’s placement in the configuration may be

attributed to excessive distortion of a well-learned

form (i.e., the commercial jet).

Characteristics of the configuration indicate that

controllers may develop prototype-based categories

of aircraft, while interpretation of the multidimensional scaling dimensions suggests that the exemplars

A320 A300

C560

C97 C2

B757

B767

C650

LJ55

A3 F111

HAR

F16

T38

V10

A310

MD11

C141

MD80

B52

L101

D328

JSTB SF34

E120

PA46

AC50

BE58

BE36

C208

Dimension 1

Dimension 2

JETS

OTHERS

Figure 4. Derived Stimulus Configuration of the Two-Dimensional MDS Model
14

nature of these errors, shown in Table 7, indicates that

this “fuzziness” may be due to a lack of discriminability

between the performance capabilities of turboprops and

pistons. Apparently, propeller-driven aircraft are perceived as being more alike than different.

The model of prototype development in Figure 2

proposes that controllers develop expectations regarding aircraft performance based on repeated pairings of aircraft designators and current speeds. Notably

however, the significant inverse relationship between

Familiarity and missing values indicates that the

designator was an ineffective cue for the retrieval of

performance characteristics when estimated frequency

of encounter was low. Furthermore, aircraft with

extremely low Familiarity ratings were not strongly

associated with any group. The uncertainty represented by the position of these elements in the configuration also demonstrates the inadequacy of the

designator as a sufficient cue. For example, the A3

Skywarrior “floats” to the right of other high-performance military aircraft. The Skywarrior had the

lowest Familiarity rating in the sample (.03) and one

of the highest incidences of missing values. Standard

deviations for all three performance estimates were

notably high for this aircraft (see Appendix B). Contrary to participants’ average perception of 377 knots,

the Skywarrior’s published long-range cruising speed

is approximately 450 knots. Individual speed estimates for this aircraft fluctuated widely, ranging

from as low as 160 knots to as high as 550 knots.

Variability in estimated climb and descent rates were

similarly erratic. Estimates ranged from 750 fpm to a

notably accurate 6000 fpm.

The positions of the C2 and C97, located near the

C560 in the upper-right area of the configuration, also

appear to be due to uncertainty. Like the A3, the C2 and

C97 had very low Familiarity ratings (.14 and .27,

respectively). Furthermore, a considerable number of

missing values was associated with both aircraft (see

Appendix B). The C2 engine type estimate was bimodal. Approximately half of the participants thought the

C2 might be a jet, while others accurately estimated the

aircraft to be a turboprop. (The remainder guessed that

the aircraft was piston-driven.) Participants’ mean speed

estimate for the C2 was 320 knots, although the maximum speed of this aircraft is listed as only 300 knots.

Participants (75%) inaccurately estimated the C97 to be

a turboprop. The C97 is actually a piston-driven aircraft. Participants estimated the average speed of the

C97 to be 376 knots. The published average cruising

speed of this aircraft is 200 knots, and its maximum

speed is only 350 knots. Standard deviations of speed

estimates for both the C2 and C97 were higher than

average as well. The average standard deviation for all

speed estimates was approximately 48 knots. The standard deviations for these two aircraft were 113 knots

(C2) and 103 knots (C97). Climb and descent rate

estimates demonstrated similar attributes.

It seems likely that the position of the C2 and C97

in the configuration reflects an average between inaccurately high estimates of performance and those

characteristic of aircraft in the lower-right quadrant.

Individual performance estimates appeared to be

based on assumptions regarding the engine type of

the aircraft. Participants who correctly estimated the

aircraft’s engine type were also more accurate in their

performance estimates. The strength of the positive

association between missing values and incorrect

engine type responses, in conjunction with the inverse relationship between missing values and Familiarity ratings, suggests that repeated exposure may

increase the precision of engine type estimates. Patterns of responding associated with the C2 and C97

indicate that the accuracy of engine type estimates

may have a direct impact on the prediction of an

aircraft’s performance capabilities.

CONCLUSIONS

In the aggregate, the results of the analysis are

compelling evidence supporting further investigation into the efficacy of display cues for prediction of

aircraft performance. It is interesting to note that

controllers appear to have developed prototypes corresponding to information that is not readily available to them. This implies that prototypes might have

been developed using multiple cues (i.e., AID, designator, current speed) obtained from a number of

sources (i.e., datablock, flight progress strip, flight

plan readout). Although the accuracy with which

participants estimated the capabilities of familiar

aircraft indicates that controllers have managed to

overcome the inadequacies of available cues, patterns

of responding associated with unfamiliar aircraft

suggest that this knowledge might have been purchased at a greater cost than necessary.

It has been demonstrated that workload in complex tasks is related to the number of information

sources, and the number of cues (Proctor & Dutta,

1995). It logically follows that workload might be
15

reduced by providing more effective cues from a

single source. Results of multiple regression interpretation of the dimensions underlying controllers’ perceptions of aircraft groups suggest that the use of

engine type and weight class might be beneficial. For

example, a Heavy jet aircraft might be accompanied

by an “HJ” on the datablock tag, proximal to the

AID. The effects of these indicators could be tested in

a simulated environment using control and experimental groups to compare subjective measures of

workload and objective measures of performance

associated with each condition. It is possible that

enhancement of prediction cues, similar to the one

suggested, might increase the efficiency of controllers’ decision making and decrease perceived workload,

particularly in sectors with a high ratio of unfamiliar

vs. familiar aircraft.

REFERENCES

Brown, E., Deffenbacher, K., & Sturgill, W. (1977).

Memory for faces and the circumstances of encounter. Journal of Applied Psychology, 62, 311-18.

Buckley, E.P., DeBarysche, B.D., Hitchner, N., &

Kohn, P. (1983). Methods and measurements in

real-time air traffic control system simulation. (Report No. DOT/FAA/CT83/26). Atlantic City,

NJ: Federal Aviation Administration.

Federal Aviation Administration. (1998). Air traffic

control. (FAA Order No. 7110.65L). Washington, DC: Federal Aviation Administration Air

Traffic Operations Program.

Federal Aviation Administration. (1984). Establishment

and validation of en route sectors. (FAA Order No.

7210.46). Washington, DC: Author.

Grossberg, M. (1989, April 30). Relation of sector

complexity to operational errors. Quarterly Report

of the Federal Aviation Administration’s Office of

Air Traffic Evaluations and Analysis. Washington,

DC: Federal Aviation Administration.

International Civil Aviation Organization. (1995, September). Civil aviation statistics of the world -

1994. (ICAO Doc. 9180/20). Montreal, Quebec:

Author.

Kruskal, J.B., & Wish, M. (1978). Multidimensional

scaling. Beverly Hills, CA: Sage Publications.

Mogford, R.H., Murphy, E.D., Roske-Hofstrand, R.J.,

Yastrop, G., & Guttman, J.A. (1994). Application

of research techniques for documenting cognitive

processes in air traffic control: Sector complexity and

decision making. (Rep. No. DOT/FAA/CTTN94/3) Atlantic City, NJ: Federal Aviation

Administration.

National Airspace System. (1995). En route configuration management document. (Rep. No. NAS-MD310). Atlantic City, NJ: Author.

Payne, J.W., Bettman, J.R., & Johnson, E.J. (1993).

Studying contingent decisions. Cambridge, MA:

Cambridge University Press.

Peterson, M.T., Meagher, R.B., Jr., Chait, H., & Gillie,

S. (1973). The abstraction and generalization of

dot patterns. Cognitive Psychology, 4, 378-98.

Posner, M.I., Goldsmith, R., & Welton, K.E., Jr.

(1967). Perceived distance and the classification

of distorted patterns. Journal of Experimental Psychology, 73, 28-38.

Proctor, R.W., & Dutta, A. (1995). Skill acquisition

and human performance. Thousand Oaks, CA:

Sage Publications.

Robertson, A., Grossberg, M., & Richards, J. (1979).

Validation of air traffic controller workload models.

(Rep. No. DOT/FAA/RD-79/83). Cambridge,

MA: U.S. Department of Transportation Research and Special Programs Administration –

Volpe National Transportation System Center.

Schwartz, B., & Reisberg, D. (1991). Psychology of

learning. New York: W. W. Norton.

Solso, R.L. (1991). Cognitive psychology (3rd ed.).

Needham Heights, MA: Allyn and Bacon.

SPSS Inc. (1990). SPSS reference guide. Chicago:

Author.

Tabachnik, B.G., & Fidell, L.S. (1989). Using multivariate statistics (2 nd ed.). New York:

HarperCollins.

Tversky, A., & Kahneman, D. (1973). Availability: A

heuristic for judging frequency and probability.

Cognitive Psychology, 5, 207-32.
16
A1

APPENDIX A

Aircraft Familiarity Questionnaire (AFQ)

1) At how many En route Centers have you worked? _____________ (number of centers)

Please list the centers you have worked at, beginning with the most recent:

1) _______________________________

2) _______________________________

3) _______________________________

4) _______________________________

2) How long have you worked in your current area?

__________ years __________ months

3) How long have you worked at your current ARTCC?

__________ years __________ months

4) How many years and months (total) have you worked at an En Route Center as an FPL?

__________ years __________ months

5) Please indicate your total number of years as a controller.

__________ years __________ months

6) Please indicate all operations in which you have been an FPL (check all that apply)

__________ En Route

____ Low

____ High

____ Ultra High

__________ Terminal

__________ Flight Service Station

__________ Other

7) When were you last certified or re-certified?

__________ (year)

OPTIONAL:

1) Age _________

2) Gender _________
A2

Instructions

The suggested strategy for both sections of the questionnaire is to “fly” through the aircraft

types. Do not deliberate too long over your responses. Your initial response will probably be

the most accurate.

Section 1: Familiarity

“Familiarity” is defined as the frequency with which you might encounter a particular aircraft in

your airspace. For example, some controllers encounter a great deal of military or general

aviation aircraft and are relatively familiar with them. These same aircraft might be unfamiliar

to a specialist working a sector that generally handles commercial flights only. Level 1 aircraft

are those you rarely or never see. If you might not come across a particular aircraft type for

several days, or only when working certain sectors, that aircraft should be classified as Level 2.

Aircraft types encountered daily in virtually all sectors are Level 3 aircraft. Simply mark the

box corresponding to the level that best describes the frequency with which you have

encountered each aircraft type over the past six months.

Section 2: Aircraft Performance Characteristics

Weight Class: Please choose from one of three basic weight classes: Heavy (H),

Large (L), or Small (S).

Speed: Please provide an approximate range of performance capabilities. (For example, a

B737 generally cruises somewhere between 400 and 470 knots). It is not necessary to indicate

whether your answer is expressed in Mach speed or knots, but feel free to circle the appropriate

indicator located to the right of the answer blank.

Climb Rate / Descent Rate: It is anticipated that you will possess general knowledge of

aircraft climb and descent capabilities. If your answer is an average climb or descent rate,

simply write a single value in the space. If you think of climb and descent rates in ranges, then

write the range in the space provided. If you think only in terms of maximums, then write the

value in the space and be sure to indicate that it is a maximum rate rather than an average.

Engines:

Type: Please choose from one of three basic engine types. The following abbreviations

are provided only for your convenience. If you prefer, ignore the suggested

abbreviations and write “piston” or “turboprop” or “jet.”

P = reciprocating, or piston-driven

T = turboprop

J = turbojet

Number: If you are unsure, give it your best guess.

Comments: Please write information regarding the aircraft’s turn rate, limitations due to

inclement weather, or other performance-related information in the space provided.
A3

SECTION 1: FAMILIARITY

Please check the box that corresponds to the level that best describes the frequency with which you

have encountered each aircraft type over the past 6 months:

Level 1: Aircraft rarely or never seen.

Level 2: Aircraft you might not come across for several days, or only encounter when working certain

sectors.

Level 3: Aircraft types encountered daily in virtually all sectors.

Old Designator New Designator Level 1 Level 2 Level 3

1 LR55 LJ55 o oo

2 A3 A3 o oo

3 C208 C208 o oo

4 T38 T38 o oo

5 AV8 HAR o oo

6 KC97 C97 o oo

7 BA41 JSTB o oo

8 EA31 A310 o oo

9 SF34 SF34 o oo

10 D328 D328 o oo

11 BE36 BE36 o oo

12 F111 F111 o oo

13 PA46 PA46 o oo

14 C560 C560 o oo

15 EA30 A300 o oo

16 BE58 BE58 o oo

17 MD11 MD11 o oo

18 MDD8 MD80 o oo

19 OV10 V10 o oo

20 C2 C2 o oo

21 B767 B767 o oo

22 AC50 AC50 o oo

23 B52 B52 o oo

24 EA32 A320 o oo

25 B757 B757 o oo

26 C650 C650 o oo

27 E120 E120 o oo

28 L101 L101 o oo

29 C141 C141 o oo

30 F16 F16 o oo
A4

SECTION 2: AIRCRAFT PERFORMANCE CHARACTERISTICS

1 Old Designator New Designator

E120 E120

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

2 Old Designator New Designator

LR55 LJ55

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

3 Old Designator New Designator

T38 T38

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

4 Old Designator New Designator

OV10 V10

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________
A5

5 Old Designator New Designator

L101 L101

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

6 Old Designator New Designator

BE58 BE58

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

7 Old Designator New Designator

T38 T38

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

8 Old Designator New Designator

B52 B52

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________
A6

9 Old Designator New Designator

KC97 C97

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

10 Old Designator New Designator

C650 C650

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

11 Old Designator New Designator

AC50 AC50

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

12 Old Designator New Designator

BA41 JSTB

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________
A7

13 Old Designator New Designator

B757 B757

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

14 Old Designator New Designator

A3 A3

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

15 Old Designator New Designator

D328 D328

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

16 Old Designator New Designator

MD11 MD11

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________
A8

17 Old Designator New Designator

C208 C208

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

18 Old Designator New Designator

C2 C2

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

19 Old Designator New Designator

C560 C560

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

20 Old Designator New Designator

F16 F16

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________
A9

21 Old Designator New Designator

BE36 BE36

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

22 Old Designator New Designator

EA30 EA30

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

23 Old Designator New Designator

C141 C141

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

24 OOld Designato New Designator

EA30 A320

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________
A10

25 Old Designator New Designator

MD80 MD80

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

26 Old Designator New Designator

EA31 A310

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

27 Old Designator New Designator

F111 F111

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

28 Old Designator New Designator

PA46 PA46

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________
A11

29 Old Designator New Designator

AV8 HAR

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________

30 Old Designator New Designator

B767 B767

Weight Class: ________

Climb Rate: _____________ fpm

Speed: ______________kts M

Descent Rate: ___________ fpm

Engines:

Number ________

Type ___________

Comments: __________________________________________________________________
A12
B1

Summary Estimates and Number of Observations

Designator Familiarity

Rating

Speed Climb

Rate

Descent

Rate

Weight

Class

Engine

Number

Engine

Type

A3

N 29 15 14 13 N 17 16 16

Mean .03 377 2989 2627 Mode Large 2 Jet

SD .19 116 1480 1104

A300

N 29 27 25 24 N 28 28 28

Mean 1.83 465 2580 2542 Mode Large 2 Jet

SD .47 42 865 849

A310

N 30 28 26 26 N 29 29 29

Mean 1.73 461 2496 2665 Mode Large 2 Jet

SD .52 40 702 822

A320

N 30 29 28 27 N 30 30 30

Mean 1.87 472 2714 2530 Mode Heavy 2 Jet

SD .35 32 766 716

AC50

N 30 28 23 23 N 30 30 30

Mean 1.20 196 1128 1309 Mode Small 2 Piston

SD .61 56 402 531

B52

N 30 28 25 27 N 30 30 30

Mean .57 475 2180 2341 Mode Heavy 8 Jet

SD .63 38 610 760

B757

N 29 29 27 26 N 30 30 30

Mean 1.97 474 2915 2738 Mode Heavy 2 Jet

SD .19 30 761 860

B767

N 30 28 28 27 N 30 29 29

Mean 1.90 480 2893 2807 Mode Heavy 2 Jet

SD .31 35 762 873

BE36

N 29 25 26 26 N 29 29 29

Mean 1.55 163 1006 1193 Mode Small 1 Piston

SD .51 21 254 502

BE58

N 30 29 26 27 N 30 30 30

Mean 1.53 191 1096 1250 Mode Small 2 Piston

SD .51 32 293 388

C141

N 30 24 27 27 N 29 29 29

Mean .87 450 2320 2204 Mode Heavy 4 Jet

SD .63 38 821 691

APPENDIX B
B2

Designator Familiarity

Rating

Speed Climb

Rate

Descent

Rate

Weight

Class

Engine

Number

Engine

Type

C2

N 29 11 9 9 N 11 11 11

Mean .14 320 1628 1522 Mode Large 2 Jet/TurboSD .35 113 762 616 Prop*

C208

N 29 29 27 24 N 29 29 29

Mean 1.21 177 981 1010 Mode Small 1 Piston

SD .68 39 286 297

C560

N 29 27 23 25 N 29 29 29

Mean 1.90 393 1985 2160 Mode Large 2 Jet

SD .31 63 602 843

C650

N 30 27 28 28 N 30 30 30

Mean 1.83 472 3148 3166 Mode Large 2 Jet

SD .38 40 1127 1167

C97

N 30 19 18 18 N 21 20 20

Mean .27 376 1892 2103 Mode Heavy 4 Jet

SD .58 105 698 793

D328

N 29 18 16 16 N 20 20 20

Mean .62 272 1897 1847 Mode Large 2 TurboSD .82 64 796 744 Prop

E120

N 30 24 17 26 N 29 27 28

Mean 1.20 258 1347 1654 Mode Large 2 TurboSD .66 28 221 534 Prop

F111

N 30 21 22 25 N 27 28 28

Mean .27 500 4005 4540 Mode Large 2 Jet

SD .58 35 1464 2272

F16

N 29 24 20 24 N 30 30 29

Mean 1.17 500 4685 5333 Mode Large 1 Jet

SD .71 41 1414 2448

HAR

N 30 24 23 24 N 27 28 28

Mean .14 473 3961 4221 Mode Large 2 Jet

SD .38 62 1375 1731

JSTB

N 29 27 22 24 N 28 27 28

Mean 1.31 305 2052 2150 Mode Large 2 TurboSD .71 84 686 843 Prop

L101

N 30 27 25 28 N 29 29 29

Mean 1.43 492 2284 2386 Mode Heavy 3 Jet

SD .68 33 550 739

* Bimodal
B3

Designator Familiarity

Rating

Speed Climb

Rate

Descent

Rate

Weight

Class

Engine

Number

Engine

Type

LJ55

N 30 29 26 26 N 30 29 30

Mean 1.77 464 3502 3633 Mode Large 2 Jet

SD .50 39 1108 1320

MD11

N 29 28 27 27 N 29 29 29

Mean 1.59 478 2478 2537 Mode Heavy 2 Jet

SD .68 42 615 730

MD80

N 30 29 24 26 N 30 30 30

Mean 1.93 453 2144 2315 Mode Large 2 Jet

SD .25 27 429 716

PA46

N 30 26 25 24 N 27 27 27

Mean 1.33 203 1180 1265 Mode Small 1 Piston

SD .48 37 358 447

SF34

N 30 25 22 20 N 27 27 27

Mean 1.30 253 1673 1615 Mode Large 2 TurboSD .65 29 388 409 Prop

T38

N 30 26 22 25 N 29 29 30

Mean .93 488 5007 5280 Mode Small 2 Jet

SD .87 44 1505 2156

V10

N 30 20 21 20 N 23 23 23

Mean .13 224 1412 1425 Mode Small 2 TurboSD .43 47 672 535 Prop
B4