DOT/FAA/AM-01/9 Pilot Visual Acquisition of

Traffic: Operational

Communications From OpEval-1

O. Veronika Prinzo

Civil Aeromedical Institute

Federal Aviation Administration

Oklahoma City, Oklahoma 73125

May 2001

Final Report

This document is available to the public

through the National Technical Information

Service, Springfield, Virginia 22161.

Office of Aviation Medicine

Washington, D.C. 20591

U.S. Department

of Transportation

Federal Aviation

Administration
NOTICE

This document is disseminated under the sponsorship of

the U.S. Department of Transportation in the interest of

information exchange. The United States Government

assumes no liability for the contents thereof.
i

Technical Report Documentation Page

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.

DOT/FAA/AM-01/9

4. Title and Subtitle 5. Report Date

Pilot Visual Acquisition of Traffic: Operational Communications From

OpEval-1

May 2001

6. Performing Organization Code

7. Author(s) 8. Performing Organization Report No.

Prinzo, O.V.

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)

FAA Civil Aeromedical Institute

P.O. Box 25082

Oklahoma City, OK 73125 11. Contract or Grant No.

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered

Office of Aviation Medicine

Federal Aviation Administration

800 Independence Ave., S.W.

Washington, D.C. 20591 14. Sponsoring Agency Code

15. Supplemental Notes

This work was performed under Task AM-B-00-HRR-516

16. Abstract:

Avionics devices designed to provide pilots with graphically displayed traffic information will enable pilots to

acquire and verify the identity of any intruder aircraft within the general area, either before or in accordance

with a controller-issued traffic advisory or alert. A preliminary evaluation was performed of an airborne

capability to display traffic information (OpEval-1, July 1999). As part of that evaluation, audiotapes were

analyzed of the communications between pilots flying aircraft equipped with a cockpit display of traffic

information (CDTI) and terminal radar approach controllers, who provided them with air traffic services. The

results revealed that pilots and controllers participated more frequently in collaborative communications that

resulted in a reduction in radio-frequency congestion and improved overall communications. Pilot persistence

in scanning for traffic called by air traffic control (ATC) — especially when that traffic was readily visible on

the CDTI display, but not out the window — may have led to more responsive traffic reports, increases in

positive visual acquisitions, and consequently proportionally fewer pilot reports of "negative contact." Only

4% of the pilots' and controllers' messages revealed communication problems such as inaccuracies, procedural

deviations, and non-routine transactions. Information load, the novelty of pilot-initiated traffic calls, access to

and knowledge of the traffic call sign by pilots, as well as the variability in ATC message structure each

contributed to the occurrence of communication problems. The voice tape analyses suggest that new

procedures and operational communications will be needed to support CDTI and guidance in collaborative

decision-making involving air-ground traffic flow management. Whenever any new system, technology,

capability, or application is evaluated prior to implementation in a well-defined environment such as the

National Airspace System, the importance of pilot and controller communication training to overcome the

interference effects of past experiences with ATC communication will need to be included as part of a

comprehensive plan towards implementation.

17. Key Words 18. Distribution Statement

Pilot Communication, ATC Communication,

Aviation Safety, Air Traffic Control

Document is available to the public through the National

Technical Information Service, Springfield, Virginia 22161

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price

Unclassified Unclassified 18

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized

1

INNOVATIONS IN PILOT VISUAL ACQUISITION OF TRAFFIC: NEW PHRASEOLOGY

FOR AIR TRAFFIC CONTROL OPERATIONAL COMMUNICATION

INTRODUCTION

Pilots and air traffic controllers operate as coordinated teams to ensure the safety of passengers and

cargo. In flight, pilots scan the airspace for the presence of other aircraft to avoid. Meanwhile, air traffic

controllers scan their radar displays to ensure separation between airborne aircraft according to prescribed

minimums. “Unless an aircraft is operating within

Class A airspace or omission is requested by the pilot,

issue traffic advisories to all aircraft (IFR or VFR) on

your frequency when, in your judgment, their proximity may diminish to less than the applicable separation minima. When no separation minima applies,

such as for VFR aircraft outside of Class B/Class C

airspace, or a TRSA, issue traffic advisories to those

aircraft on your frequency when in your judgment

their proximity warrants it”1

. When issuing traffic

advisories, controllers use standard phraseology contained in FAA Order 7110.65, Air Traffic Control.

For example the radio call, “Traffic, eleven o’clock,

one zero miles, south bound converging, Boeing

Seven Twenty Seven, one seven thousand” directs a

pilot’s visual scan to a limited viewing area of the sky.

Upon detecting the aircraft, the pilot typically would

inform the controller that the traffic is in sight.

However, if the intruder goes undetected, the controller will issue instructions for a pilot response to

resolve the threat. Aircraft flying at speeds in excess

of 4 miles per minute2

 make receipt of timely

replies from the pilots to controllers that much

more compelling.

Avionics designed to provide pilots with graphically displayed traffic information are being developed to aid the visual acquisition process. The cockpit

display of traffic information (CDTI) will present

the visual depiction of the geometry of another aircraft

in relation to the pilot’s own aircraft. It would seem

that having a CDTI would facilitate and direct the

pilot’s visual scan to a more precise location to detect

the other aircraft — provided the other aircraft can

transmit its location. Otherwise, the utility of the

CDTI would be limited in use. These devices will

enable pilots to acquire the aircraft and verify the

identity of any intruder within the general area either

before, or in accordance with, a controller-issued

traffic advisory or alert. However, direct access to

information involving the location and identity of

other aircraft in the vicinity by pilots may necessitate

the development of a new phraseology to accommodate operational communication and procedures.

A preliminary evaluation was performed of an

airborne capability to display traffic information

(OpEval-1, July 1999). Before proceeding any further, it must be pointed out that OpEval-1 provided

an opportunity to demonstrate new air- and groundbased capabilities and systems at a FAA-controlled

airport and en route facility. With that in mind, it is

important to note that it was not possible to apply

true experimental and control conditions, comparable to what would be expected when planning and

executing laboratory-based experiments. In addition,

since the amount of instruction and training that the

pilot and controller participants received was not

documented, it was impossible to exercise any statistical control on the data (e.g., treating the number of

hours of training as a co-variant). Finally, since the

demonstration was an operational evaluation of the

CDTI during actual flights, much more communication data were obtained when CDTI was in use than

not, which precluded a quantitative statistical analysis of the data. A final report of that evaluation was

prepared by the Operational Evaluation Coordination Group (2000).

1

 FAA Order 7110.65M, TRAFFIC ADVISORIES Para 2-1-21. Air Traffic Control 7110.65M, is a FAA order that prescribes air traffic

control procedures and phraseology for use by personnel providing air traffic control services. Controllers are required to be familiar

with the provisions of this order that pertain to their operational responsibilities and to exercise their best judgment if they encounter

situations not covered by it.

2

 1 knot is equal to 1.15078030303 miles per hour.
2

As part of that evaluation, audiotaped recordings

of communications were analyzed between pilots

flying aircraft equipped with a CDTI device and

terminal radar approach controllers who provided

them with air traffic services. The objective of the

voice tape analysis was to identify any change in

visual acquisition time (VAT), flight identifier phraseology, or workload that resulted when pilots were

flying with and without the benefit of CDTI. This

report provides a general description of those findings.

METHOD

Participants

Sixteen pilots, serving as a captain or first officer,

flew aircraft equipped with CDTI while three air

traffic controllers provided air traffic control (ATC)

services. The pilot participants were paid volunteers

who received briefings and participated in training

exercises prior to the evaluation. The controllers,

who also were volunteers, were on a temporary detail

during training and on regular schedule during the

evaluation.

Procedure

Training on CDTI Phraseology. Prior to OpEval1, pilots and controllers participated in several preOpEval-1 simulations that were conducted at the

Integration and Interaction Laboratory (I-Lab) of the

MITRE Corporation Center for Advanced System

Development (CAASD). During these simulations,

pilots received instruction on how to respond to

ATC-issued traffic calls and listened to a combination of ATC and pseudo-pilot communications over

a party line. Two weeks prior to OpEval-1, each pilot

had received a set of flight-crew maneuver cards

describing the CDTI and standard phraseology contained in FAA Order 7110.65/Aeronautical Information Manual (AIM) that they would use during

OpEval-1. Furthermore, during the preflight briefing conducted the day of the evaluation, pilots were

reminded to reply to ATC-issued traffic calls according to the standard or the CDTI Phraseology —

depending on CDTI usage. As a further reminder, a

set of the cards was prominently displayed onboard

each participating aircraft.

Experimental Flight. As shown in Figure 1, each

circuit, depicted with directional arrows, was flown

by the pilots who followed a basic racetrack pattern of

performing all right or left turns and missed approaches. A circuit consisted of an aircraft completing a full cycle around the traffic pattern that

culminated in a missed approach or landing. Unfortunately, Instrument Meteorological Conditions

(IMC) prevailed during the morning flights and

precluded the extensive use of visual separation and

Alt 30 -50

210 kts

Alt 30 - 50

210 kts

10 - 15 Mile Final

Figure 1. Depiction of the Circuits Flown by Participating Aircraft.
3

visual approaches. However, during the afternoon

session, the weather conditions were more favorable:

Visual Meteorological Conditions (VMC) allowed

controllers to use visual separation and visual approaches. Consequently, the pilots flew between 6-

15 circuits during the afternoon, receiving instructions

over their headsets to vector their aircraft into a right- or

left-hand pattern at altitudes specified by the controller.

Materials

The data consisted of 3 hours of audiotaped pilot/

ATC communications provided by the participating

Terminal Radar Approach Control (TRACON) facility. Each 1-hour tape was labeled with the name of

the facility, date of recording, radio frequency

sampled, time interval, and runway assignment. Specifically, Channel 1 contained the voice communications, and Channel 2 contained the local time stamped

by the day, hour, minute, and whole second (s) according to the Universal Time Coordinated standards.

Data Extraction Procedure

Two copies of the audiotapes were made from each

original to minimize stretching of the tape medium.

The transcriber received one copy, and the other was

used by the Subject Matter Expert (SME) who listened to the recordings while verifying the accuracy

of the verbatim transcripts. Each message was subsequently encoded with its corresponding transmission

start and end time. The original audiotapes local time

stamps were translated by a Datum Time Code Generator/Translator and then converted into whole

seconds. A continuous record of radio frequency use

and nonuse was included with each transcript. Next,

the SME parsed each message into message elements

and labeled them by speech-act category (e.g., Instructions, Advisories) and aviation topic (e.g., traffic, heading, altitude, speed).

The audiotapes and transcripts aided the SME in

the identification of air traffic (AT) communication

sets. As shown in Figure 2, an AT communication set

involved only those transmissions between an air

traffic controller and the pilot of the aircraft receiving

a traffic-related transmission.

Generally, an AT communication set began with

the air traffic controller issuing a traffic advisory, as

illustrated by transmission 1. Often pilots respond

with “looking,” followed by either “negative contact”

or “traffic in sight,” (or similar words), as was the case

in message 6. A controller also could query the pilot

to “report the traffic in sight.” Alternatively, if no

pilot response was forthcoming, the controller might

restate the traffic advisory.

Dependent Measures

ATC Workload. Although various measures of

ATC workload exist, for the purposes of the voicetape analysis, it was defined as the number of active

aircraft still on frequency when the controller initiated a transmission. An aircraft was counted as being

under positive control when it was radar identified by

the controller and the pilot established initial contact

with the controller. An aircraft was no longer under

positive control when the controller instructed the

Time (in seconds)

Speaker Message Start End Lapse Acquired Total

ATC 1. CAA1 / TRAFFIC ONE O'CLOCK FIVE

MILES ON FINAL THREE THOUSAND

BOEING

783 788 4 39 48

CAA2 2. CAA2 790 792 2 - -

ATC 3. CAA2 / YES SIR 793 795 1 - -

CAA2 4. AIRPORT IN SIGHT / CAA2 796 798 1

ATC 5. CAA2 / CLEARED {TYPE} APPROACH 805 808 7

CAA1 6. WE HAVE THE BOEING CAA2 / CAA1 827 831 19 - -

Figure 2. An Example of an Air Traffic Communication Set.
4

pilot to contact the tower. Other indicators of

workload included the amount of time on frequency

(i.e., frequency occupancy time, FOT) and the amount

of time the frequency was not in use (i.e., ‘dead-air’

or lapse time), as well as the total number of messages

exchanged between a controller and the pilot of an

aircraft during each circuit, as previously displayed in

Figure 1.

Visual Acquisition Time (VAT). VAT was defined as the time lapsed, in whole seconds (s), from

the end of the controller’s first traffic-related transmission until the start of the pilot’s reply indicating

visual contact or negative contact with the aircraft

(denoted as traffic by the controller). For example,

the dashed arrow in Figure 2 illustrates the computation of VAT that began at 783s (message 1) and

ended in visual acquisition at 827s (message 6) when

the pilot of CAA1 informed ATC that the target

traffic was acquired. In that example, VAT was completed in 39s.

Total AT Communication Set-Time. To determine the total AT communication set-time, a simple

difference was computed between the start of a controller-initiated traffic advisory or report and the end

of the pilot’s final reply to that message. For instance,

in Figure 2, the solid arrow indicates a transaction

began at 783s and ended at 831s when the pilot of

CAA1 completed the transaction. This resulted in a

total AT set time of 48s. Notably, it might have been

possible for the pilot to report spotting the traffic

sooner, but the radio frequency was unavailable. To

account for this possibility and to determine if the

total AT set time was inflated, lapse time (or “dead

air”) was computed as the amount of silence between

consecutive transmissions. As seen in Figure 2, the

amount of silence varied from 1 - 19s.

Communication Workload. Communication

workload has many facets and information load is but

one attribute. Information load consists of both the

amount and complexity of information present in a

message. The amount of information in a message

was determined by counting the number of message

elements present in a transmission (Prinzo, Britton,

& Hendrix, 1995). Complexity was determined by

counting the number of digits, letter groups, or both,

that indicated a direction or distance, aircraft call

sign, an aircraft type, etc. (e.g., 12 o’clock, left, right,

north, CAA123, Boeing 727) in a message element.

These digits and letter groups were labeled as bits of

information.

For example, the following transmission,

“CAA321 / Following traffic six miles ahead CAA

123 / Cleared visual approach runway 22 left /

Multiple aircraft off your right landing on the parallel / Contact Wilmington Tower” has an information

load of 12 bits of information. It has 5 message

elements consisting of (1) the receiving aircraft’s call

sign, (2) route/position advisory, (3) runway clearance, (4) traffic advisory and a (5) transfer of communications. Complexity consists of 7 bits, and they are

presented in a bolded-italics font.

Flight ID Phraseology. Because the traffic flight

identifier (TFID) is a deviation from the FAA Order

7110.65/AIM, its inclusion in ATC/pilot communications may have unknowingly changed the normal

exchange of information. To evaluate the usage and

Table 1. Pilot Response to a Traffic Advisory and the Evaluation Rule

Source Phraseology Rule

7110.65/AIM "traffic in sight"

"negative contact"

A pilot response to a controller-issued traffic

advisory with words other than "traffic in

sight" or "negative contact" did not comply

with the stated phraseology.

CDTI Phraseology

CDTI In Use

"(Ownship call sign) (target

TFID) in sight." "(Ownship call

sign) roger, traffic not in sight."

A pilot response to a controller issued traffic

advisory either did or did not comply with

the stated phraseology.

CDTI Not In Use "(Ownship call sign) traffic in

sight." "(Ownship call sign)

roger, traffic not in sight."

A pilot response to a controller issued traffic

advisory either did or did not comply with

the stated phraseology.
5

effect of the TFID on operational communication,

the SME coded pilot responses to traffic calls according to FAA Order 7110.65/AIM and the CDTI Phraseology (Table 1). Initially, the rule to include “roger”

was accepted; however, it was excluded when it became clear from the SME that pilots did not include

it as part of their traffic reports. As a result, “roger”

was excluded from the evaluation rule.

Communication Problems. Given the novelty

of the traffic-flight identifier as a new message

element in a traffic-related message, new communication problems may have emerged. Specifically, a communication problem “refers to any

disturbance of routine communication, where

controllers and pilots do not follow standard procedures, and/or where they must interrupt information transfer in order to clarify the communication”

(Morrow, Lee, & Rodvold [1990] pp. 36). Communication problems include inaccuracies, procedural

deviations, and non-routine transactions involving

misunderstandings or other problems related to successful information transfer. Although communication problems often contribute to frequency

congestion and workload, they do not necessarily

lead to operational errors or incidents. Since the

TFID is a new message element designed for pilot use

during OpEval-1, it was evaluated to determine its

influence, if any, on ATC/pilot communication.

In summary, message counts, contents, duration,

rates, and reply latencies were the objectively derived

measures of communication that were extracted from

the time-stamped voice tapes. They were used to

compute descriptive statistics that summarized CDTI

use versus non-use on traffic awareness, workload,

and the confirmation of visual traffic acquisition

latencies between ATC and the participating flight

crews3

. They also provided some insights and implications for future air traffic operations and communication procedures.

RESULTS

ATC Workload

There were 67 circuits (20 CDTI Not In Use, 47

CDTI In Use) in which pilots and controllers exchanged 1127 messages containing air traffic (n=300)

and other (n=827) information. Each complete circuit lasted between 160-590s (M=352.92s) when

CDTI was in use and 132-562s (M=363.56s) when it

was not. Of particular interest in OpEval-1 was the

communication between the controllers and pilots

during these circuits. Notably, as seen in Table 2,

while controllers sent fewer messages per circuit when

CDTI was in use, they also spent less time on average

(per aircraft), conveying traffic-related information

to the pilot. For the pilot, there also seemed to be a

slight reduction in the number of air traffic-related

messages when CDTI was in use, without an appreciable change in frequency occupancy time.

Visual Acquisition Time (VAT). Before examining VAT, it was important to determine if any differences in pilot reply times to traffic calls resulted from

3

 Since the conditions necessary to perform inferential techniques were not met, any statistically significant effects that may have

resulted from CDTI use or non-use could not be inferred from the data (Kerlinger, 1986).

Table 2. Mean and Standard Deviation (S.D.) of ATC/Pilot Messages and Frequency

Occupancy Time (FOT) per Circuit Presented by Speaker, CDTI Usage, and Type of

Message

Messages per Circuit Frequency Occupancy Time

Mean (S.D.) Mean (S.D.)

Source Air Traffic Other Total Air Traffic Other Total

Speaker by CDTI Usage

ATC - Not In Use 3.1 (1.9) 6.0 (3.0) 8.7 (2.8) 17.7 (08.1) 17.5 (9.8) 33.5 (13.5)

ATC - In Use 2.6 (1.9) 5.6 (2.0) 7.8 (2.7) 14.3 (10.2) 16.8 (6.4) 29.3 (11.7)

Pilot - Not In Use 2.6 (1.6) 6.4 (2.7) 8.7 (2.8) 055.4 (03.5) 13.4 (5.8) 18.3 (07.0)

Pilot - In Use 2.1 (1.3) 6.5 (1.8) 8.4 (2.3) 055.5 (03.5) 13.6 (4.4) 18.5 (05.2)
6

frequency non-availability. To do this, the percentages of silence and mean lapse time between successive transmissions (i.e., the average duration of silence)

was computed for all transmissions. The findings

presented in Table 3 revealed that for all of the

message transmitted by pilots when CDTI was in use,

87% were preceded by 0-3s of silence compared with

93% when CDTI was not in use. The overall increase

in mean silence when CDTI was in use suggested that

the radio frequency was available to pilots for reporting to ATC and it was concluded that it would not

have artificially inflated the VAT.

To be considered as a part of the database from

which visual acquisition times would be examined,

AT communication sets must have originated with

the controller and not the pilot. Furthermore, because controller-updated traffic calls could produce

multiple pilot responses, VAT was measured from

the end of the controller’s first issuance of an ATC

message with traffic-related content to the start of the

pilot’s reply indicating “positive” or “negative contact,” or “looking” as an outcome.

A preliminary examination of the data resulted in

the removal of 3 of the 84 AT communication sets

because they contained delays greater than 180s; all

of the other sets were less than or equal to 68s. The

data were also excluded because they would have

artificially inflated the mean VATs and total time to

complete an AT communication set. Presented below

are the controller messages to each of the aircraft

involved in these long delays. The pilots may have

been busy setting up for the approach, the traffic was

too far away to spot, or the delays may have resulted

for other reasons.

“... TURN RIGHT HEADING TWO ZERO ZERO

JOIN THE TWO TWO RIGHT LOCALIZER THE AIRPORT IS TWO O’CLOCK ONE FIVE MILES THE

TRAFFIC TO FOLLOW IS THREE O’CLOCK AND

FIVE MILES ADDITIONAL TRAFFIC OPPOSITE BASE

LANDING ON THE PARALLEL A DC NINE.”

“... TRAFFIC TEN O’CLOCK EIGHT MILES

NORTHEAST BOUND ON THE DOWNWIND

THIRTY SIX HUNDRED A BOEING CONTACT

WILMINGTON TOWER YOUR SPEED AND SPACING IS FINE.”

“... FOLLOWING TRAFFIC SIX MILES AHEAD

CLEARED VISUAL APPROACH RUNWAY TWO TWO

LEFT MULTIPLE AIRCRAFT OFF YOUR RIGHT

LANDING ON THE PARALLEL CONTACT

WILMINGTON TOWER.”

When the remaining data were examined, 76% of

the 54 AT communication sets that occurred while

CDTI was in use involved pilot reports of “traffic in

sight.” However, as the data in Table 4 show, when

CDTI was not in use, positive reports decreased to

63%. In addition to an increase in CDTI-based

visual acquisitions, another benefit resulting from

CDTI was that pilots reported traffic faster. The

longer mean VATs suggest that, when traffic was not

immediately visible out-the-window, CDTI may have

encouraged pilots to continue scanning for traffic

called by ATC — especially when that traffic was

readily visible on their displays. The presence of

CDTI on the flight deck may have encouraged the

pilots to keep looking and, thus, contributed to the

13% increase in positive sighting reports.

Total AT Communication-Set Time. Total AT

communication-set time was measured in whole seconds from the start of the controller’s first issuance of

a traffic-related message until the end of the pilot’s

final reply conveying “positive” or “negative” contact

or “looking.” The data in Table 5 show that when

CDTI was not in use, proportionally fewer of the

pilots’ positive sightings were completed in 15s or

less. At 60s, 58% of the positive sightings were

completed when CDTI not in use and 66% when it

was not.

Measures of Communication Workload. A second set of analyses, performed on the entire 1,127

messages and presented in Table 6, revealed that

30.5% of the messages not considered to be complex

involved 1.6% of the communication problems (left

panel). As indicated by the column in that table

labeled “Messages with Communication Problems,”

pilots and controllers did exceptionally well communicating with one another — especially since only

3.7% of all their messages contained communication

problems. As shown in the right-hand panel of Table

6, when complexity was combined with the amount

of information in a message to provide an index of

information load, the data revealed that 79% of the

messages with an information load of 6 or less involved 2.9% of the communication problems. In

addition, a majority of the messages had a moderate

information load of 5-6. Of the 42 messages that had

communication problems, the majority had information loads ranging from three to six and rarely

were problems found in messages with a light information load of 1-2.
7

Table 3. Percentage and Mean Lapse Time Between Successive Transmissions (in seconds)

Presented by Message Contents and CDTI Usage (Pilot-Initiated Messages Only)

Amount of Silence Preceding a Pilot Message

Source 0-1s 2-3s 4-5s 6-15s 16-30s 31-45s 46-60s >60s Total Mean (SD) N

Message Type by CDTI

AT - Not In Use 16% 9% 1% 1% 0% 0% 0% 0% 26% 1.7 (2.2) 46

AT - In Use 13% 7% 2% 1% 0% 0% 0% 0% 22% 1.7 (1.4) 90

Other - Not In Use 53% 14% 1% 2% 2% 0% 0% 0% 74% 2.0 (3.6) 128

Other - In Use 56% 11% 2% 4% 3% 1% 1% 1% 78% 3.5 (8.6) 313

All Messages - Not In Use 70% 23% 2% 3% 2% 0% 0% 0% 100% 1.9 (3.3) 174

All Messages - In Use 69% 18% 4% 4% 3% 1% 1% 1% 100% 3.1 (7.6) 403

Table 4. Frequency of AT Communication Sets Presented by VAT, Outcome and CDTI Usage

Pilot Visual Acquisition Time

Source 0-1s 2-3s 4-5s 6-15s 16-30s 31-45s 46-60s 61-68sTotal Mean (SD) N

Outcome by CDTI Usage

Positive - Not In Use 26% 18% 4% 4% 4% 4% 4% - 63% 08.6 (15.9) 17

Positive - In Use 39% 15% 2% 4% 4% 7% 2% 4% 76% 10.6 (18.2) 41

Negative - Not In Use 18% 15% - 4% - - - - 37% 02.3 (03.1) 10

Negative- In Use 02% 11% 6% 2% - 4% - - 24% 09.2 (14.5) 13

Table 5. Total Time to Complete an AT Set (in seconds) Presented by Outcome and

CDTI Usage

Total Time on Frequency Per AT Communication Set

Source 6-10s 11-15s 16-30s 31-45s 46-60s 61-75s 75-274s Total Mean(SD) N

Outcome by CDTI Usage

Positive - Not In Use 22% 22% 7% 7% - 4% - 63% 17.6 (16.4) 17

Positive - In Use 28% 22% 7% 4% 7% 4% 4% 76% 28.1 (46.3) 41

Negative - Not In Use 7% 18% - - 7% 4% - 37% 25.3 (23.6) 10

Negative - In Use - 13% - - 4% 6% 2% 24% 48.0 (54.5) 13
8

A more comprehensive analysis of communication

workload was performed separately for the 300 pilot

and controller messages involving only traffic information. As part of that analysis, communication

problems were again identified and categorized according to speaker, CDTI usage, and information

load. The results (Table 7) show that the pilots’

traffic-related messages generally were more complex

and contained slightly more information when CDTI

was in use, compared with when it was not in use.

Based on the data, pilot messages had a greater overall

information load for controllers to process. Interestingly, the opposite pattern emerged for controller

communication. That is, when pilots were flying

CDTI approaches, the controllers tended to send less

information per message, and their messages also

were less complex. It would seem that the messages

that were transmitted when CDTI was in use had a

smaller information load for pilots to process. This

reversal is not surprising since CDTI provides pilots

with an increased opportunity to actively share relevant traffic information with controllers.

As with the analysis of the entire data set, those

associated with the 300 air traffic messages were not

without problems. In fact, like the overall data,

roughly 4% involved one or more communication

problem. For pilots, communication problems involved a request for ATC to repeat a traffic position

report, a readback error, an incorrect call sign usage,

and call sign confusions. Generally, as shown in

Table 6. Percentages and Number of All Messages and Only Messages with Communication

Problems Presented at Each Complexity and Information Load Index

Complexity Information Load

Index All Messages Messages with

Communication

Problems

All Messages Messages with

Communication

Problems

%n%n %n%n

0 030.5% 0344 1.6% 18 000.1% 0001- -

1-2 054.4% 0613 1.4% 16 08.8% 0099 0.1% 01

3-4 012.2% 0138 0.7% 08 032.9% 0371 1.8% 20

5-6 002.4% 0027 - - 037.4% 0422 1.1% 12

7-8 004.0% 0005- - 014.7% 0166 0.4% 05

9+ - - - - 006.0% 0068 0.4% 04

Total 100.0% 1,127 3.7% 42 100.0% 1127 3.7% 42

Table 7. Communications Workload Presented by Speaker and CDTI Usage

Amount of

Information

Message

Complexity

Information

Load

Number of

Messages

Source Mean (SD) Mean (SD) Mean (SD) n

Speaker by CDTI Usage

ATC - Not in Use 2.95 (1.2) 3.80 (1.8) 6.75 (2.6) 055

ATC - In Use 2.77 (1.2) 3.73 (2.3) 6.50 (3.0) 109

Pilot - Not in Use 2.41 (0.9) 1.33 (0.9) 3.74 (1.4) 046

Pilot - In Use 2.44 (0.9) 1.52 (1.0) 3.97 (1.6) 090
9

Figure 3a, pilot AT messages with communication

problems had information loads of 3, 4, or greater,

and all of them occurred when CDTI was in use.

As shown in Figure 3b, unlike the pilots, 2 of the

controllers’ AT messages with communication problems occurred when CDTI was in use and 3 occurred

when not. For controllers, as with pilot-AT messages,

communication problems occurred in AT messages

with information loads of 3, 4, or greater. Some

specific examples of communication problems that

were encountered during the OpEval-1 involved the

delivery of traffic advisories to the pilot of the wrong

aircraft, calling aircraft by the wrong flight identifiers, misunderstandings, and providing clarification

to pilots.

Flight ID Phraseology. Flight ID phraseology was

evaluated by comparing the content of the pilot’s

reply to a traffic call with standards contained in FAA

Order 7110.65/AIM and the CDTI Phraseology. Of

the 134 pilot-generated replies, 68 resulted in positive contact (51%), 16 in negative contact (12%),

and 28 ended with looking (21%) as a final outcome.

An additional 7 were updates to previous traffic

advisories (5%), 10 closed the transaction (8%) and

5 were not relevant to the traffic situation (4%).

In spite of the pre-OpEval training, briefings, and

memory aides, only 36.6% of their traffic reports

complied with FAA Order 7110.65/AIM standards

and only 14.4% were in agreement with the CDTI

phraseology (e.g., “{OWNSHIP} {TARGET TFID} in sight,”)

(see Figure 4). In fact, the only time that the phraseology for CDTI and FAA Order 7110.65/AIM was in

agreement was when CDTI was not in use, and the

pilot reported the traffic in sight. Thus, more often

than not, pilots’ previous communication practices

took precedence over the phraseology developed for

the operational evaluation. Consequently, 45% of

the pilot responses did not comply with either phraseology. Some examples of non-standard phraseology

include, “This is {Ownship} well, we got him sir” and

“{Ownship} ... will call either the traffic or the field.”

Pilot responses to ATC traffic calls were subsequently categorized according to the types of traffic

identifiers presented in Table 8. Some representative

examples of each type of traffic identifier are presented for clarity.

As shown by the data in Table 9, when positive

outcomes were considered, 58% of pilot replies that

occurred while CDTI was in use included the full or

partial call sign of the traffic aircraft, compared with

12% when CDTI was not in use. As shown under the

column labeled “Generic,” 75% of positive sightings,

regardless of CDTI usage, failed to provide the controller with information ensuring that the designated

traffic was acquired (19% with CDTI and 56% without

CDTI). For non-acquired traffic, again regardless of

CDTI availability, approximately 37% of the pilot

replies only provided a generic referent to the aircraft

previously identified by ATC in the traffic call (20%

CDTI not in use and 17% CDTI in use).

0

5

10

15

20

25

30

35

40

1,2 3,4 5,6 7,8 9+

Information Load

Number of AT Messages

Not In Use - No Problem

Not In Use - Problem

In Use - No Problem

In Use - Problem

0

5

10

15

20

25

30

35

40

1,2 3,4 5,6 7,8 9+

Information Load

Number of AT Messages

Not In Use - No Problem

Not In Use - Problem

In Use - No Problem

In Use - Problem

Figure 4. Pilot Compliance with CDTI and

7110.65/AIM Phraseology during OpEval-1.

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

7110.65/AIM

Only

B oth CDTI Only Neither

Phra se ology Source

Percent Compliance

Figure 3a. Pilot Air Traffic Messages Presented by Information Load.Figure 3b. Controller Air Traffic Messages

Presented by Information Load.
10

As previously mentioned, although a reference list

of pilot phraseology was constructed, taught, briefed,

and displayed on the flight deck, pilot use of that

phraseology and the form of their responses were

highly variable and often reflected previous communication practices. The data presented in Table 10

certainly reflect that variability. Some examples using

Ownship call sign (SID) and Traffic Flight IDentifier

(TFID) included: (1) Okay we have uh [TFID] in

sight [SID], (2) Ah we have the traffic in sight uh

[TFID] confirm that, (3) [numbers] on the heading

have the airport in sight and the traffic is [TFID], for

[SID] has the airport in sight — we also have [TFID]

in sight, and (4) [SID] has [TFID] [SID] has [partial

TFID] in sight.

Communication Problems. Of the 10 detected

AT communication problems, 3 are presented in

Table 11, and their possible explanations are presented in Table 12. Problem #8 is a request from the

controller for the pilot to repeat the transmission. It

would seem that the controller was not prepared to

receive a pilot-initiated traffic report. Problems #9

and #10 each stem from an unsolicited traffic report

by the pilot of CAA124.

In problem #9 (Table 11), the controller misinterpreted “in sight” as the pilot of CAA1 - 11 reported

the airport in sight. Hearing the phrase “in sight”

often prompts controllers to hand-off the aircraft to

the tower. Since the pilot had not previously closed

that transaction with an acknowledgment, upon

Table 8. Examples of the Type of Traffic Identifiers Included in Pilot Relies to ATC Traffic Calls

Traffic Flight Identifier Pilot Responses to ATC Traffic Calls

Full call sign: ... HAS CAA ONE TWENTY THREE

Partial call sign: ... WE GOT THE CAA — or I HAVE THE ONE TWENTY THREE

Aircraft type: ... WE HAVE THAT BOEING TRAFFIC IN SIGHT

VFR: ... THE VFR IS NOT IN SIGHT

Generic: ... NEGATIVE CONTACT ON THE SECOND TRAFFIC

Table 9. Pilot Traffic Replies (n=84) Presented by Traffic Flight Identifier and CDTI Usage

Types of Traffic Flight Identifiers Included in Response to ATC Traffic Calls

Source Full Call Sign Partial Call Sign Aircraft Type VFR Traffic Generic

Outcome by CDTI Usage

Positive - Not In Use 04% 08% 12% - 56%

Positive - In Use 46% 12% 5% - 19%

Negative - Not in Use - - - - 20%

Negative - In Use - - - 2% 17%

Table 10. Location of Pilot's Ownship Call sign and TFID Presented by CDTI Usage

CDTI Usage

Source Not In Use In Use

Identifier First Last Embedded Missing First Last Embedded Missing

Ownship 7 5 1 5 21 9 6 18

Traffic - 1 - - 3 5 18 -
11

hearing “in sight,” the controller may have thought it

was still open. Becoming partially confused, the controller thought he had completed the hand-off and

again instructed the pilot to contact tower. Problem

#10 also results from the same unsolicited traffic

report. When the pilot initiated the traffic call, the

controller apparently processed only the second half

of the message and thought CAA1 - 11 was the

speaker. The communication problem was recognized only after CAA124 explained what transpired

in the previous transmission and then the events that

led to the communication problem were understood

and resolved.

As shown in Table 12, resolution techniques varied with the type of AT communication problem in

the transaction. The number of messages needed to

resolve the problem varied with the complexity of the

AT communication set. Some possible causal or

contributing factors revealed that 70% of the communication problems involved TFIDs, of which 30%

resulted in some type of confusion stemming from

the pilots’ knowledge of the TFID either through the

CDTI, voice radio, or both.

DISCUSSION

As airspace congestion increases, controllers and

pilots will continue to share precise and detailed

traffic information to ensure safety. To accommodate this process, new technologies such as CDTI are

being introduced to aid pilots in the detection, and

visual acquisition of other aircraft. As these technologies are certified and made operational, the roles and

responsibilities of pilots and controllers who use

them will inevitably change. Some of these changes

were demonstrated during the operational evaluation

of CDTI.

For instance, the presence of CDTI onboard the

aircraft seemed to create an apparent trade-off in airground workload. That is, when CDTI was in use,

controllers sent fewer messages and spent less time

conveying traffic-related information to pilots, while

pilots sent fewer traffic-related messages to ATC. The

very slight increase in the time pilots spent on frequency may be attributed, in part, to the traffic flight

identifier being included in their traffic reports.

Table 11. Examples of AT Communication Sets with Communication Problems

PROBLEM SPEAKER Examples of Three Communication Problems and Their Resolutions

CAA111 Departure / CAA uh 1 - 11 has CAA 1 - 23 in sight

8 ATC I'm sorry uh / say that again

CAA111 CAA 1 - 11 / has CAA 1 - 23 in sight

ATC CAA 1 - 11 / that works good for me / cleared {type} approach {rwy} /

follow the CAA 1 - 23 Boeing

CAA111 Okay / here we go

. ATC CAA 1 - 11 / contact {name} Tower

CAA144 CAA 44 / heading 0 - 4 - 0

CAA987 {name} Departure / CAA 9 - 87 / is with you passing 22 hundred ...

ATC CAA 9 - 87 / {name} Departure / radar contact / you can expect ...

9, 10 CAA124 CAA 1 - 24 / has CAA 1 - 11 in sight

ATC CAA 1 - 11 / I'm sorry I thought we did that / contact {name} Tower

CAA124 No / that was CAA 1 - 24 that has CAA 1 - 11 in sight

ATC CAA 1 - 24 / roger / I'm going to extend ...
12

In contrast, the introduction of graphically displayed traffic information in the flight deck allows

pilots to assume a more active role in traffic management. Together, pilots and controllers participated

in collaborative communications. This was demonstrated by pilot-initiated traffic calls during OpEval1. When the CDTI was in use, pilots detected nearby

aircraft and occasionally provided ATC with unsolicited traffic sightings. In response, controllers either

instructed the pilot to follow that aircraft for the

approach (in lieu of issuing a traffic advisory or

request a report the traffic in sight) or requested the

pilot to repeat the transmission. Accordingly, when

the opportunity to access the radio frequency became

greater, as demonstrated by an increase in dead-air

time, their communication became more effective.

Consequently, CDTI can be a double-edged sword

— when the novelty of pilot-initiated traffic calls

wears off, there is a positive affect on the pilotcontroller collaboration process. However, it also

increases communication when ATC is either not

prepared or does not expect a call from the pilot.

Other benefits of CDTI included more responsive

traffic reports from pilots and increases in positive

visual acquisitions. As mentioned previously, pilots

did not always respond faster to controller-issued

traffic advisories — in fact, at times they were slower.

Pilots may have been encouraged to continue scanning for traffic called by ATC — especially when that

traffic was readily visible on their CDTI displays but

not out the window. This increased vigilance may

have contributed to more sighting reports that were

positive and proportionally fewer pilot reports of

“negative contact.”

In addition to pilots using the radio frequency less

often, their messages also became more complex and

had a greater information load when CDTI was in

use. Interestingly, the opposite pattern emerged for

Table 12. Types of AT Communication Problems Presented by Resolution Technique and Outcome

Communication Problem Resolution Technique Outcome

Uncertainty of the identity of traffic TFID used to verify/confirm traffic Pilot visually acquires the

designated traffic

Aircraft Type confused with Flight ID ATC restates traffic location Conflict resolved by a

correction in the flight path

TFID used as Receiver ID None Uncorrected

TFID reported as Ownship None Uncorrected

TFID given as referent for target traffic

position results in all or some of the

message not understood or received

Pilot requests a repetition of the

target traffic position

Traffic was not acquired

Pilot requests clarification of who the

recipient was of the last transmission

Controller restates previous message Recipient understood to be

the pilot making the request

ATC request for confirmation that the

pilot reported a positive visual acquisition

of traffic

Pilot restates negative sighting report Mutual understanding that

the traffic was not acquired

Unsolicited traffic sighting results in all or

some of the message not understood or

received

Pilot retransmits per ATC request Mutual understanding that

traffic was acquired

Unsolicited traffic sighting results in ATC

misidentifying TFID as the speaker of the

previous transmission

None ATC reissues a transfer of

communication

Unsolicited traffic sighting results in ATC

misidentifying TFID as the speaker of the

previous transmission

Speaker identifies self and traffic Mutual understanding of the

identities of speaker and

traffic
13

controller communications. When pilots were flying

CDTI approaches, the controllers’ messages tended

to be less complex and had a smaller information

load. This reversal is not surprising since CDTI

affords pilots with an increased opportunity to participate more actively with ATC in traffic-awareness.

Clearly, pilots and controllers communicated accurately, as only 4% of the messages contained communications problems. Communication problems

did not lead to operational errors or incidents; however, on occasion, they did contribute to frequency

congestion and increased workload as resolution techniques were applied. The communication problems

reported here included inaccuracies, procedural deviations, and non-routine transactions. If the controllers repeated all or part of their initial

transmissions, this was not counted as a communication problem unless they provided clarification,

resolved a misunderstanding, or corrected misinformation. The application of this definition of a communication problem was more conservative than that

used by Cardosi (1993a), who analyzed tapes of pilot/

controller communications from 3 Air Route Traffic

Control Centers. In that report, 12% of the 508

transmissions contained communication problems

that involved maneuvers for traffic avoidance, turns

not for traffic, and traffic advisories. The difference

in the number of communication problems involved

controllers repeating some or all of their initial transmissions — in some cases pilots failed to reply,

whereas in others, they missed or read back incorrectly the controllers’ messages. For the OpEval-1

data, if a pilot failed to reply to the initial message and

the controller did not query the pilot, it was not

included as a communication problem.

While the overall number of communication problems was relatively low, many factors contributed to

their occurrence. Some of the prominent factors included information load, the novelty of pilot-initiated

traffic calls, access to and knowledge of traffic flight

identifiers by pilots, as well as the variability in ATC

message structure. Perhaps most important: All of the

pilots’ traffic-based communication problems occurred

when CDTI was in use. Generally, pilots’ messages with

an information load of 3, 4, or greater were more likely

to involve communication problems, as were controllers’. This finding is consistent with a similar pattern

reported for message length (Morrow, 1996) in the

TRACON environment and message complexity for en

route communications (Cardosi, 1993b).

Given the novelty of pilot-initiated traffic calls,

that controllers asked them to repeat unsolicited

traffic reports is not at all surprising. Pilots typically

do not initiate traffic calls but rather, receive them.

Thus, controllers’ requests for pilots to repeat their

last messages increased controller communication

workload and frequency congestion since 2 additional messages were exchanged — the “say again”

and the unsolicited traffic report. In addition to

using “say again” as a resolution technique, others

included restatements, clarifications, and embellishments. All the techniques seemed to vary with the type

of communication problem. The number of messages

exchanged to resolve the problem depended partially on

the complexity of the AT communication set.

A closer examination of these communication

problems revealed that most of the communication

problems involved traffic flight identifiers. It remains unclear whether the unique or combined effects of message structure (i.e., syntax) or the

knowledge/presence of 2 different aircraft call signs

in the same message (i.e., Ownship and traffic flight

identifier), led to confusion and resulted in a communication problem. Recall that the participating

pilots received practice on the CDTI phraseology,

the syntax to use when reporting traffic, and they had

access to phraseology reference list onboard their

aircraft. Nevertheless, their responses were highly

variable, generally reflecting their previous communication practices.

As the data show, the locations of Ownship call

sign and the traffic call sign in a message were unpredictable and could have caused problems in comprehension and understanding. As Anderson (1990)

pointed out, when the phrase structure of a message

is unpredictable or ambiguous, the listener (reader)

has a difficult time ascribing the intended meaning of

the speaker (author). Consequently, comprehension

suffers and misunderstandings arise. For example,

the sentence, “They are cooking apples.” The word

“cooking” can be assigned either to the verb class

(they are doing something — the something they are

doing is cooking) or as an adjective that modifies the

noun apple (the type of apple — a cooking apple).

Therefore, the meaning of the sentence is derived

from how it is parsed (i.e., its syntax).

Just as the sentence, “They are cooking apples” is

syntactically ambiguous to the person trying to comprehend it, so was “CAA1 - 11 in sight” for the

controller (see problems #9 and #10). The sentence
14

can be interpreted as CAA1-11 reporting to ATC that

the airport is in sight and is waiting for an approach

clearance. Alternatively, the pilot of an aircraft has

informed ATC that CAA1-11 has been visually acquired as traffic. The ambiguity stemming from

“CAA1 - 11 in sight” is whether to assign “CAA1 -

11” as the subject noun or the object of the message.

Unpredictable and ambiguous messages can create

communication problems for the reader (listener). In

each case, the reader (listener) must parse the message

elements correctly before the author’s intended meaning can be inferred. This example underscores the

importance of a highly predictable message structure

for ATC messages. When the presence of a syntactic,

lexical, or semantic ambiguity arises from unpredictable message structures, they pose the threat of communication problems for the receiver of those

messages. It is well known that the opportunities for

miscommunications between pilots and ATC are

ever present, existing mainly from the sheer volume

of communication that occur daily. Fortunately,

most miscommunications are detected, resolved, and

are nothing more than minor nuisances — adding

slightly to the communication workload. On the

other hand, those that go undetected have the potential

to lead to more serious and potentially dangerous events.

This finding actually highlights an issue related to

introducing any new system, technology, capability,

application, procedure, or phraseology into an existing, and well-defined environment such as the National Airspace System. First, it suggests the

importance of pilot and controller communication

training to overcome interfering effects of past experiences with ATC communication. This finding also

suggests that the introduction of any procedural

change to support CDTI may demonstrate similar

initial interference effects for pilots and controllers,

since they are highly skilled at using the existing, repetitive procedures to perform their respective duties.

The inclusion of a traffic flight identifier in a

traffic advisory was a deviation from FAA Order

7110.65/AIM. Its presence in the message added an

unknown factor into the normal exchange of traffic;

its inclusion is not a normal way of issuing traffic.

Therefore, before FAA Order 7110.65 is modified, it

would be beneficial to determine whether the same

types of confusion would have occurred if the current

procedures and phraseology for traffic advisory services had been included. Until such a test is performed, the continued use of the traffic flight identifier

as part of traffic-related messages will continue to be

a confusing factor.

The voice-tape analyses suggest that modifications

to the existing procedures and operational communications may be needed to support CDTI. Clearly,

operational procedures for conveying unique traffic

information (such as the inclusion of the traffic flight

identifier) could prove worthwhile as a means of

minimizing communication problems, workload, and

frequency congestion. As an example, consider the

development of new ATC procedures that will be

associated with ADS-B and CDTI. It would seem

that, as part of that effort, there also should be a

clearly defined phraseology with a standard message

structure to minimize ambiguities — especially if

two different aircraft flight identifiers are included in

the same transmission.

In addition, and in support of the pilots who will

use CDTI, it would seem prudent to develop standard operating procedures and practices that would

include a phraseology that is both easy to learn and

use. Both might prove beneficial for pilots when

CDTI is operational. When pilots and controllers are

provided adequate training and practice in applying

these procedures and supporting phraseology, there

should be a reduction in miscommunications resulting from confusions arising from message-structure

ambiguities such as the ones that occurred during

OpEval-1. Once developed, such a standard, when

used faithfully, could ensure that only the intended

recipient of the message replied to that message. In

conjunction with the new technologies, these procedures will improve safety, enhance efficiency, and

reduce the potential of human error associated with

ATC/pilot communication.
15

REFERENCES

Anderson, J.R. (1990). Cognitive psychology and its implications (Third Ed.). New York: W. H. Freeman

& Company.

Cardosi,, K.M. (1993a).Time required for transmission of time-critical air traffic control messages in

an en route environment. The International Journal of Aviation Psychology, 3(4), 303-13.

Cardosi, K.M. (1993b). An analysis of en route controller-pilot voice communications. US Department of Transportation, Office of Research

and Development Report DOT/FAA/RD-93/

11, Washington, DC.

FAA Order 7110.65M, Air Traffic Control. (2000).

Federal Aviation Administration. Washington,

DC: US Government Printing Office.

Kerlinger, F.N. (1986). Foundations of behavioral

research (Third Ed.). New York: Holt, Rinehart,

& Winston.

Morrow, D.G., Lee, A., & Rodvold, M. (1990). Analysis of routine pilot-controller communication. In:

Managing the modern cockpit: Third Human

Error Avoidance Techniques Conference Proceedings. Warrendale PA: Society of Automotive Engineers, Inc.

Morrow, D.G. (1996). Collaboration in controllerpilot communication. In: B.J. Kanki & O.V.

Prinzo (Eds.) Methods and metrics of voice communications. Federal Aviation Administration, Office of Aviation Medicine Technical Report DOT/

FAA/AM-96/10, Washington, DC. Available

from: National Technical Information Service,

Springfield, VA 22161; ordering no. ADA307148.

Operational Evaluation Coordination Group (2000).

CAA/FAA ADS-B/Safeflight 21 Phase 1–

Operational evaluation final report.

Prinzo, O.V., Britton, T.W., & Hendrix, A.M. (1995).

Development of a coding form for approach control/pilot coice communications. In: B.J. Kanki &

O.V. Prinzo (Eds.) Methods and metrics of voice

communications. Federal Aviation Administration,

Office of Aviation Medicine Technical Report

DOT/FAA/AM-96/10, Washington, DC. Available from: National Technical Information Service, Springfield, VA 22161; ordering no.

ADA307148.
16