35% » EiHeading
30% | Altitude
DAIt. Restrict.
25% 4 Speed
20% B App./Dep.
15% 4 M1 Frequency
EIRt./Position
10% TT
5% - =
0% ia ain EN a no
Se me %, Sey, Ys,
2, %6, %y ; oy “ey
on On Y
%,
Figure 5. Field Tapes: Distribution of Instruction Irregular Communications
20% - Ei Heading
NAltitude
5 DAIt. Restrict.
15% Speed
App./Dep.
10% = 0 Frequency
GRt./Position

5%

0%

Figure 6. Simulation Tapes: Distribution of Instruction Irregular Communications

12



TABLE 2. Aviation Topics/Speech Acts Taxonomy

Speech Act Category

Aviation Topics

Address/Addressee
Courtesy

Instruction/Clearance--
Readback/Acknowledgment

Advisory/Remark--
Readback/Acknowledgment

Speaker, Receiver
Thanks, Greetings, Apology

Heading, Heading Modification, Altitude, Altitude
Restriction, Speed, Approach/Departure, Frequency
Holding, Route/Position, Transponder Code, General
Acknowledgment

Heading, Heading Modification, Altitude, Altitude
Restriction, Speed, Approach/Departure, Route/Position,
NOTAM, ATIS, Weather, Sighting, Traffic, General
Acknowledgment

Request--Readback/Acknowledgment Heading, Altitude, Speed, Approach/Departure,

Non-Codable Remarks

Route/Position, Type, NOTAM, Weather, Traffic, Say
Again, General Acknowledgment

Equipment, Delivery, Other

TABLE 3. Types of Irregular Communications in ATC/Pilot Transcripts

Definition

Types of IC Code
Non-Standard Phraseology
Grouped G
Sequential (Non- N
grouped)

Omission fe)
Substitution Ss
Transposition T
Excessive Verbiage E
Partial Readback’ P

Delivery Technique

Dysfluency D

Misarticulation M

Grouping of numerical information contrary to paragraph 2-85,
FAA Order 7110.65G.

Failure to group numbers in accordance with paragraphs 2-87, 2-
88, 2-90, and non-use of the phonetic alphabet in accordance
with paragraph 2-84, FAA Order 7110.65G.

Leaving out number(s), letter(s), word(s), prescribed in
communication requirements in FAA Order 7110.65G.

Use of word(s) or phrases(s) in lieu of communication outlined in
FAA Order 7110.65G (e.g., “verify altitude” vs. “say altitude”).

Number(s) or word(s) used in the improper order (e.g., “Universal
six forty-five” instead of “Universal five forty-six”).

Adding word(s) or phrase(s) to communication outlined in FAA
Order 7110.65G, and the communication suggested in the
Aeronautical Information Manual (e.g., “Universal the number
one airline six forty-five”).

Pilot report or readback that does not include specific reference
to a topic subject (e.g., altitude topic “out of six for four” would be
recorded as a P.).

Pause(s), stammer(s), utterance(s), that add no meaning to the
message (e.g., “uh,” “ah,” or “OK” when not used as a General
Acknowledgment.

Improperly spoken words (i.e., slurs, stutters, mumbling, etc.).

Note: A verbatim readback of a controller's instruction or advisory would not be recorded as a P;
nor would a readback containing a General Acknowledgment and the aircraft identifier.


20% - Heading |
| NAltitude

5% | iSpeed
@App./Dep. |
Rt/Position |

|OGnl. Ackn.

10%

5%

0%

Figure 11. Field Tapes: Distribution of Request Irregular Communications

40% Heading
35% NAltitude
Speed
30%
@App./Dep.
25% Rt./Position
20% OGni. Ackn.
15%
10%
5% |
0% Bo Ar EN
a ar , %, oy
Xe, ey, Sy ~,
‘, 8, 74
— %

Figure 12. Simulation Tapes: Distribution of Request Irregular Communications

16


ACKNOWLEDGMENTS

This research was sponsored by the Federal Aviation Administration’s Offices of Aviation
Safety Analysis (ASY-200) and the Quality Assurance Division of Air Traffic System
Effectiveness (ATH-200). We also acknowledge Barbara Burke, Monte “Buzz” Taets, Sandy
Dooty, Elaine Pfleiderer, and Jennifer Thompson for the many monotonous and tedious
hours of effort that they devoted to transcribing, coding, encoding, and analyzing the data
presented in this report.

iii


00, £801
00° 0 00° 0 00" 0] oO 0} 00° 0 00° 0} 00° 0} 00 0} 00° 0} 00 0 WV.LON
so 8s 00" v 7" ve 00° 0] to wT 00" I 00° 0 00° L 00° 0 00° 0 AUEIM
a Tel 00" £ tO” 1% 00° oO] £0 SE 00" 4 Ww (<4 £0" 8¢ 00° 9 00" v ouserL
00" t 00° 0 00" 0 00° 0] 00 a 00" 0 00° 0 00° 0 00° 0 00" 0 paads
as sil 00° 0 tO" 61 00° 0] co St 00" € 0" 6€ £0" Te 00" 0 00° I uOHIsog/anoy
10° 6 00° 0 00" 0 00° 0} 00 I 00" 0) 00" v 00" v 00" 0 00" 0 Suypeay
or" oze 00° s 10° vl 00° O} St | 897 00" 0 £0" 87 00" s 00" 0 00" 0 Bapysis wey
ar 961 00° 0 wo 9% 00° O} St] or 00" 0 00° I 00" s 00° 0 00° 0 AOU YV “Was
10° 6 00" 0 00" 4 00" 0} 00 € 00" 0 00° z 00" 0 00" 0 00° 0 SLLV
ainjiedaq
ira 1&7 10° or so” vs 00" 0} 0 ey 00" 0 v0" er LO 08 00" 0 00° I mpeoiddy
10° 6 00" 1) 00° € 00" 0} 00 v 00° 0 00° I 00° I 00" 0 00° 0 apmniy
NOLLVTOWIS
86° Los
00° v 00" 0 00° T 00" 0] 00 € 00° 0 00’ 0 00° 0 00° 0 00° 0 WV.LON
wu raat 00° 0 £0" Lt 00° 0] LO 9F 00° 0 00" 4 as Ls 00" 0 00" 0 ATOYIEIM,
or IL 00° T cw ol 00° 0] LO 9F 00° I £0 €l 00° v 00° 0 10° 9 oygeaL
00" v 00° 0 00° 0 00° 0] 00 I 00" 0 00" 0 00° I 00° 0 00" 4 paads
Lv 98 00° q cw I 00° O|] €r 9 00" 0 00° € 10° 9 00° 0 00° I uonIsog/ainoy
10° 8 00° 0 00" I 00° 0] 00 s 00° 0 00° i 00" 0 00° 0 00" I Surpeay
v0" 61 00° 0 00° € 00° 0} 7 OL 00° 0 00° € 00" £ 00° 0 00" i) Bunysig way
10° L 00" 0 00" S|} 00 0| 00 4 00° 0} 00 0} 00° 0} 00 0} 00 0 MOLDY “UI
80° Or 00° 0 10° 9 00° oO} So SZ 00° 1) 00" v 00° s 00" 0 00" 0 SLLV
aanjiedeq
ve raat 00° 0 w 8 00° Z| so 9% 00° 0 7" II vl eh 00" 0 00" € Ppeoiddy
LO” ve 00° 0 00" v 10° 9] £0 €l 00" 0} 0 Z} 00° if 00° 0] tw 8 apmaniv
CCMARIB CICS
% N % u % ul % ul % uw % ul % ul % ul % ul % u
uoHe[NonEsi, ‘AQuanysAq: yorqpeay: BBRIGIOA uontsodsueiy, == uonnjnsqng UOISSTUQ qenuanbog pednoin,
yermeg sseoxq aOUNOS VLYG
IVLOL anbruysay Araatfaq. Ppiepurig-uoN NV
O1dOL
SNOILVOINAWWOQ AVINDAAA] AO SAdAL NOLLVIAV

LOV HOAAdS SHAVNAW/AUOSIAGV

C3


35%
30% m Field Tapes _
f Simulation Tapes
25% + 7
20% +
15% +
10%
5% +
0% + T ,
%, & % % 4, Ss 2.
i, % 4 K + & YH,
“oe a, e, S Op op,
2 Oy

Figure 7. Distribution of Aviation Topics in the Advisory Speech Act
That Contained Irregular Communications

moderate turbulence was reported by a pilot. As is
often the case, other pilots who are listening in on the
party line will use this information to avoid a “rough
ride.” Upon hearing other pilots’ reports, they too will
get on the radio frequency to either report current
weather conditions and/or request deviations from
their flight plan if weather seems to be a potential
factor in their approved flight plans. These additional
communications increase frequency congestion in an
already complex situation and add to the controllers’
workload. As an example of weather-related commu-
nications, a portion of communications between a
controller and several different pilots is presented
below.

“Company went through the weather [aircraft]
straight ahead he said ten second [sic] of moderate
turbulence the rest was light chop. [aircraft] approach
roger I'm gonna let you find an area where you'd like to
turn to the northwest and get through the weather uh

company said there was ten seconds of moderate turbu-
lence in the weather off to your right and the rest was
light chop then a [aircraft] went through there and said
that they were really getting down drafts in that. Roger
[aircraft] ok thank you let me know how the ride is over
the next five miles or so you should be west of that well
about ten miles you should be west of the weather. Er
[aircrafi] you're eight miles behind [aircraft] increase to
two zero zero if you like. If the weather permits [air-
craft] the [airport] weather is five thousand scattered
higher ceiling broken visibility one five wind three four
zero at six altimeter two niner seven six expect runway
three four right or left.”

Although weather was not a factor in the simula-
tion study, the ghost pilot did request a speed reduc-
tion because of mild chop. In the transmission,
“American Thirteen Eighty- One Heavy, you can
expect visual runway three five right approach,” the
phrase “you can” was coded as excess-verbiage. Since
other ghost pilots were not present to make similar


Morrison, R., & Wright, R.H. (1989). ATC controls
and communications problems: An overview of
recent ASRS data. In R.S. Jensen (ed.,) Proceedings
of the Fifth International Symposium of Aviation
Psychology, Columbus, OH: The Ohio State Uni-
versity, pp. 902-07.

Morrow, D., Lee, A., & Rodvold, M. (1993). Analyzing
problems in routine controller-pilot communica-
tion. International Journal of Aviation Psychology,
3, pp. 285-302.

Morrow, D., Rodvold, M. & Lee, A. (1994). Nonroutine
transactions in controller-pilot communication.
Discourse Processes, 17, pp. 235-58.

Predmore, S. (1991). Microcoding of communications
in accident investigation: Crew coordination in
United 811 and United 232. In R.S. Jensen (ed.),
Proceedings of the Sixth International Symposium on
Aviation Psychology. Columbus OH: The Ohio
State University, pp. 350-55.

Prinzo, O.V. (1996). An analysis of approach controlipilot
voice communications. Washington, DC: Federal Avia-
tion Administration, Office of Aviation Medicine
Technical Report, DOT/FAA/AM-96/26. Available
from: National Technical Information Service,
Springfield, VA 22161; ordering no. ADA317528.

Prinzo, O.V. & Britton, T.W. (1993). ATCipilot voice
communications: A survey of the literature. Wash-
ington, DC: Federal Aviation Administration, Of-
fice of Aviation Medicine Technical Report DOT/
FAA/AM-93/20. Available from: National Tech-
nical Information Service, Springfield, VA 22161;
ordering no. ADA274457.

19

Prinzo, O.V., Britton, T.W., & Hendrix, A.M. (1995).
Development of a coding form for approach controll
pilot voice communications. Washington, DC: Fed-
eral Aviation Administration, Office of Aviation
Medicine Technical Report DOT/FAA/AM-95/
15. Available from: National Technical Informa-
tion Service, Springfield, VA 22161; ordering no.
ADA295009.

Prinzo. O.V. & MacLin, O. (1996). Aviation topics
speech acts taxonomy (ATSAT) pc user's guide version
2.0... Washington, DC: Federal Aviation Admin-
istration, Office of Aviation Medicine Technical
Report, DOT/FAA/AM-96/20. Available from:
National Technical Information Service, Spring-
field, VA 22161; ordering no. ADA314179.

Prinzo, Lieberman, and Pickett (in review). An acoustic
analysis of ATC communication. Washington, DC:
Federal Aviation Administration, Office of Avia-
tion Medicine Technical Report.

Searle, J.R. (1969). Speech Acts. London: Cambridge

University Press.

Zajonc, R.B. (1965). Social facilitation. Science, 149,
269-74.


requests of the controller, there were fewer opportuni-
ties for the controller to issue deviations due to weather.
A Runs Test performed on the field- and simulation
distributions of irregular communication codes (e.g.,
grouped, sequential, and so on) was not significant, r
= 10,z=0, p= 1.0. This result indicated that, overall,
there was no statistical difference in the types of
irregular communications produced by controllers
whether in the field or simulation laboratory.
3.3.4Request. It is important to remember that
only 2% of all the communication elements made by
controllers in the field and in the simulation labora-
tory involved Requests, and only 3-4% contained
ICs. Figure 10 shows that, for field tapes, the majority
of controller Request ICs involved Speed and Altitude
aviation topics. For simulated communication, the
majority of aviation topics involved General Acknowl-
edgment followed by Approach/Departure and Altitude

aviation topics. The Runs Test revealed that the
distributions were not significantly different r = 8,
z= 0, p> .6, and it was concluded that the simulation
results were representative of the communication prob-
lems identified from field tapes.

Given the small number of irregular communica-
tions it was not possible to perform a Runs Test on the
two distributions of types of irregular communica-
tions. As shown in Figure 11, the majority of the field
ICs involved substitutions and/or excess-verbiage
(36% Substitutions and 38% Excess-verbiage summed
across aviation topic). For example, in the transmis-
sion, “American Fourteen Zero Eight Heavy verify head-
ing,” “verify” was coded as a substitution for the word
“say.” The controller should have said, “American Four-
teen Zero Eight Heavy, say heading.” Figure 12 shows
that, for simulated communications, 37% of the excess-
verbiage ICs involved General Acknowledgments.

60% 5
55% |
50% +
45% +
40% -
35% -
30% -
25% ~

Hi Field Tapes
Bi Simulation Tapes

20% +
15% +
5% |
0% 4 1

Figure 10. Distribution of Aviation Topics in the Request Speech Act Category
That Contained Irregular Communications


66° | 261
ZO" rv} 00 0} 10 Z| 00" 0} 00 if 00° 0} 00 0] 00 T] 00° 0| 00° 0 POUEIM
zo" vy} 00 0} 10 Z| 00" 0} 00 0} 00 Oo} 10 Z| 00° 0} 00 0} 00 0 paads
v0" 8| 00 I 10° Z| 00" 0} co €} 00° 0} 00 T] 00 1] 00 0| 00 0 wonIsog/aynoy
60° 81 | 00" 1] 0 S| 00° 0} 9) 00° Oo} £0 Ss} 00° 1] 00 0| 00° 0 Supra,
8s" M1] 10° ZT] LO v1 | 00° 0} ze} wz] oo 0} so} ot} wo ry} 00" 0] 00 0 MOINY "Uae
er st] 20° €] 10 L| 00" 0} S| 00° Oo} cw €| £0 s| 00 0] 00 0 | aanjedagppeoiddy
rr zz | 00° 1] 7 y| 00 0} S| 00 IT} 40] vt} 00 0} 00 0} 00 0 apply
NOLLV'TOWIS

001 78
00" 0} 00 0} 00° 0} 00 0] 00 0} 00 0} 00 0} 00 0} 00° 0} 00 0 Joye,
se 6%} 00" 0} 00° 0} 00 o} ery} i 00" o} sr} za} io 1] 00° 0} 90° $ paads
sv zi] 00° 0} 00 0] 00 0} 90 S| 00" 0} so v| 10 1] 00° 0} 70 z wonsog/amnoy
Ir 6| 00° 0} 10 1] 10 1] so | 00 0| +0 £} 00 0} 00 0| 00 0 Supra
80° L} 00° 0| +0 €] 10 1] 00° 0} 00 0} ww z| 10 1} 00° 0} 00 0 MOP U9,
or 8} 00° 0} 00° 0} 00 0} so | 00 0] wo Zz} wo Z] 00° 0| 00 0. | aamaedaqppeoiddy
Iv Lt] 00" 0} 00 0} 00 0} 60 L| 0 0} 80 L| 00 0} 00 0} 0 € apmpIy
advil Oval
% N % a % ul % ut % a % ul % ul % ul % u[ % a
uonejnonsesijA, Rouanyskq yorqpeay, ‘DBRIQIOA, uontsodsuezy, = uorNyNsqng UOISSTUQ, Tenuonbog podnoin
eneg Ss00% gounos VLVa
IVLOL anbruyoay A@arjoq prepueig-uoN, Nv
SNOLLVOINAWWOZ AV INDIA] AO SAdAL, O1dO.L NOLLVIAV

LOV HOdddS LSandad

‘¥xUS. GOVERNMENT PRINTING OFFICE: 1998 -660-088/60085

C4


40%

435% OFT: Speaker ID
GST: Speaker ID
30% MFT: Receiver ID
MST: Receiver ID
25%
20%
15%
10%
: i I
0% imal
Q Lt
% % % % %, %, %
%, %, %, % %, %
% % % % % 4, %
° % % % z % %
e % % & %.
% %

Figure 3. Distribution of Field and Simulation Address Irregular Communications

For simulated traffic, controllers made proportion-
ally more ICs that involved how numbers were spo-
ken'!. For example, flight numbers in an air carrier’s
call sign are to be spoken in grouped format and for
general aviation aircraft, the numbers of the aircraft
registration are to be spoken sequentially. As stated in
FAA Order 7110.65, “‘Group form’ is the pronuncia-
tion of a series of numbers as the whole number, or
pairs of numbers they represent rather than pronounc-
ing each separate digit [sic].” For example, a descent
instruction to the pilot of SWA943 should have been
spoken as “Southwest Nine Forty-three descend and
maintain ....”

" See FAA Order 7110.65 2-4 Aircraft Identification.

3.3.2Instruction. Figure 4 reveals that, for field
communication, the majority of Instruction ICs in-
volved Radio Frequency, Speed, and Heading avia-
tion topics. For simulated communication, ICs
occurred most frequently in the Approach/Departure,
Radio Frequency, and Heading aviation topics. How-
ever, these differences were not statistically signifi-
cant, r= 11, z= -.218, p > .4. Based on the outcome
of the Runs Test, it was concluded that the simulation
results were representative of the communication ir-
regularities identified from field tapes.

A Runs Test was performed on the field- and
simulation-tape distributions of IC codes (i.e.,
grouped, sequential, and so on). Overall, there was no


Prinzo, Britton, and Hendrix (1995) developed the
aviation topics speech acts taxonomy (ATSAT) to
analyze communication elements? in accordance with
FAA Order 7110.65 and the Aeronautical Information
Manual (AIM)*‘. Both were used to identify irregulari-
ties in spoken communication such as grouping num-
bers together when they should be spoken sequentially
or omitting, substituting, or adding words contrary to
required phraseology. These irregularities constitute
departures from the standard phraseology specified in
FAA Order 7110.65 and will be referred to as irregular
communication (IC) in this report.

Communication elements combine to form mes-
sages that are transmitted over voice radio or data link
communications systems (Prinzo 1996). One com-
monly referenced communication elementis the speech
act (Searle, 1969; Kanki & Foushee, 1989) which
Prinzo defines as an utterance, either spoken or writ-
ten, which describes or suggests one discourse func-
tion. There are five speech act categories in the ATSAT:
Address, Instruction, Advisory, Request, and Cour-
tesy. A sixth category, Non-codable, was included for
communication elements that could not be catego-
rized (the communication element could be an in-
complete phrase or be unintelligible). The aviation
topic is the subject matter of the speech act. It places
a constraint on the communication element by im-
posing a restriction on its identified speech act cat-
egory. For example, there are only two types of aviation
topics in the Address speech act category: one identi-
fies the speaker and the other identifies the receiver of
a transmission.

To illustrate, consider the transmission, “[Name]
Approach, Universal 744 descend and maintain niner
thousand. "It contains three communication elements.
[Name] Approach and Universal 744 identify the par-
ticipants; each is tagged with the speech act category,
Address. The aviation topic distinguishes one Address
from the other by identifying “[Name] Approach” as
the Speaker and “Universal 744” as the Receiver of the
transmission. The third communication element,
“descend and maintain niner thousand,” is an Instruc-
tion speech act, and its aviation topic identifies itas an
altitude.

Recently, Prinzo (1996) performed acontentanaly-
sis of ATC communications from field tapes provided
by several terminal approach control (TRACON)

facilities. The field tapes were used to develop a
baseline database of typical controller and pilot voice
communications. The results of that analysis found
that 2,500 of 6,300 (40%) controller communication
elements contained at least one irregularity. For con-
trollers, 93% of those irregularities occurred in the
Instruction (55%), Advisory (24%), and Address
(14%) speech act categories. Of 5,900 pilot commu-
nication elements, 3,500 (59%) contained at least one
irregular communication. For pilots, 96% of their
irregular communications involved the Instruction
(53%), Address (25%), and Advisory (18%) speech
acts categories. Irregular communication involved
call sign ambiguity, call sign confusion, two aircraft
on frequency talking to each other, report of an
emergency locator transmitter (ELT), open mike,
traffic, weather, and others. The communication prob-
lems resulted in a loss of efficiency but did not result
in hazardous consequences, such as loss of separation.

Based on telephone interviews with pilots and the
content/acoustic analyses of field tapes, several re-
searchers have attempted to identify some of the
correlates and causes of communication problems.
Morrison and Wright (1989) and Morrow, Rodvold,
and Lee (1994) report that miscommunications tend
to occur more often when controllers experience over-
load due to heavy traffic, frequency congestion, mes-
sage length, etc. Three voice qualities appear to vary
systematically with workload: pitch, loudness, and
rate of speech. For example, Griffin and Williams
(1987) reported that people under emotional stress or
increased task complexity havea higher pitch and they
tend to talk louder and faster. Brenner et al., (1994)
indicate that mental workload also appeared to pro-
duce similar effects on language production.

To gain a better understanding of the relationship
between workload and ATC communication, a simu-
lation study was designed to examine the relationship
between workload and the efficiency and accuracy of
controller voice communication. Two components of
controller voice communication, message content and
speech production, were examined. An acoustic/pho-
netic analysis was performed on the controller voice
characteristics thought to be associated with changes
in workload. The results of that analysis are presented
in Prinzo, Lieberman, and Pickett (in review).

} The communication clement is conceptualized as a fundamental unit of meaningful verbal language.

* Previously called the Airman’s Information Manual.


001 SI81
so pelt as LOt or v8l 00" 4 90° | 901 10° £7 Ory Sst St} ple Br | ste St | Str I 294790
so” 16 00" 9 cw oe 00° 0 10 87 00" 0 00° s 10° 6L 00" 0 £0" £ I seyeadg
NOILVTNNIS
001 LOE
9L Tet 70" 9 sv Ly | 00° i LO" &% 00° 0} 90° 81 ve | Pol 80° € £0" ol I F9a7900y
ve SL 00" 0 £0" 8 00° 0 10° v 00" 0 00" I or 6S 10° £ 00" 0 GI s2yvadg
CCMARIBCIES
N % uw % ul % u[ % ul % ul % ul % ul % ul % u
uonelnorest yA, AQuanysAq: yorqpeay ‘DBEIGIIA, uontsodsuelL = uonninsqng UOISSTUQ yenuonbag poednoig,
rene ssooxq aounos vLva
TIVLOL anbruysay, AraaTjaq, puepurig-uoN, NV
O1dOL
SNOLLVIINNWWOZ AVINOANY] AO SAAT, NOLLVIAV

DOV HOdadS SSHUGAV

AroBayeg wy ypeeds yorg unprpy sordoy uonemy Aq suonestunururoy senZergy jo sadé 7, Jo soBeuao10g

O XIGNdddV

3




TABLE OF CONTENTS

SECTION
1.0 INTRODUCTION
2.0 METHOD woneeccccsscsssssssestessessssssscsssssesseceveseeesecsnecsessseesecsuessussuessessussssesvesuesseeauesseeauesuesneseesneearennenerenee 3

3.0

4.0
5.0
6.0

7.0

2.1 Participants

2.2 Equipment

2.3. Simulation Support Staff.

2.4 — Materials

2.5 Procedure
2.6 Data Encoding Tool and Procedure ........sssecssessssesssessseecsusessseesneessesssueessueesnsesnseenneesaneeneeeeanes 5

2.7. Data Encoding Procedure

RESULTS weeeccssesscesseeseesssessesneesscssecsesssecsecssessecssesuessscssesssesueasscsneerscanseseesecsusesecsvssnnaseesesssesaneaseeanessessuess 7

3.1. Analysis of Communication Elements .

3.2. Analysis of Irregular Communication Elements...

3.3 Analysis of Irregular Communications

DISCUSSION ..essessseessssesssssssseessecesseesnecessscensessnsessnecessessusessucesnessssesssessnesssvsssnsessssesssessusesneessesenseeeas 17
CONCLUSION ..

REFERENCES ...0essscsssssssssssssessseencesneesssssesseesssessnessseessusessneessessssscssecesseessesssneessssessecsseesneeenerenseneey 18
APPENDICES

Appendix A

Appendix B





35%

30% 4 @ Field Tapes

Simulation Tapes

25% +

20% +

15% |

10% 4

5% 4
0% +
on % f°)
% % 8
& %
co

g
&

Figure 4. Distribution of Aviation Topics in the Instruction Speech Act Category that
Contained Irregular Communications

significant difference in the two distributions, r =
8, z = -.729, p = .47. As shown in Figures 5 and 6, the
most frequent IC involved the omission of part of a
Radio Frequency. Other omissions involved Head-
ing, Altitude, and Speed. In the transmission “Ameri-
can Fourteen Ninety Heavy contact approach,” the
controller failed to include the radio frequency in the
transfer of communication. And in the transmission,
“... expect localizer back course three five left ap-
proach,” the controller omitted the word “runway” as
part of the advance approach information.

3.3.3. Advisory. As shown in Figure 7, Advisory ICs
involving field tapes occurred most often in Ap-
proach/Departure, Weather, and Route/Position avia-
tion topics. For simulated communication, Advisory

11

ICs occurred most frequently in Sighting, Approach/
Departure, and General Acknowledgment aviation
topics. The Runs Test revealed that the distributions
were not significantly different from one another r =
9, z = -1.092, p > .1, and it was concluded that the
simulation results were representative of the commu-
nication problems identified from field tapes.

As shown in Figures 8 and 9, excess-verbiage was
the most prevalent IC code for field and simulated
communications. Omissions were higher on the field
tapes because weather was a factor at one of the
facilities and traffic was heavy. Under those condi-
tions, controllers often would have additional de-
mands placed on them by pilots who want to be
vectored around weather cells. On one field tape,


AN ANALYSIS OF VOICE COMMUNICATION
IN A SIMULATED APPROACH CONTROL ENVIRONMENT

Words differently arranged have a different meaning, and

meanings differently arranged have different effects.

1.0 INTRODUCTION

Accurate communication between air traffic con-
trol specialists! and pilots is essential to air safety (see
Prinzo & Britton, 1993 for a review of the literature).
To gain a better understanding of the problems that
arise from spoken communication, analyses on the
message contents and acoustic properties of speech are
often performed. The results from those analyses
range from describing current communication prac-
tices among pilots and controllers to determining
causal factors in accident investigations,

Researchers and investigators who perform mes-
sage content analyses often develop their own classifi-
cation schemes to identify patterns of communication
and organize similar communication problems into
categories. Message content analyses were recently
performed on field tapes obtained from various air
traffic control (ATC) facilities (en route, Cardosi
1993; tower, Burki-Cohen 1995; terminal, Morrow,
Lee, & Rodvold, 1993; Cardosi, Brett, & Han, 1996;
Prinzo 1996) and cockpit voice recorder transcripts
obtained from the National Transportation Safety
Board (Helmreich 1994; Predmore 1991). Generally,
the results indicate that clearances and instructions
make up the largest proportion of communication
between controllers and pilots.

Typically, acoustical analyses examine the frequency
spectrum and temporal amplitude of identical words
or parts of words spoken by the same person at
different times to determine whether changes are
present (Mayer, Brenner, & Cash, 1996). These
changes might serve as markers or indicators of stress
(Brenner, Shipp, Doherty, & Morrissey, 1985; Griffin

— Pascal

& Williams, 1987) or physiological change
(Lieberman, Protopapas, & Kanki, 1995). Generally,
a frequency shift to the right (pitch gets higher) is
indicative of emotional stress and a shift to the left
(pitch gets deeper) indicates stress-reduction. Brenner,
Doherty, and Shipp (1994) analyzed crew conversa-
tions recorded during routine and emergency situa-
tions. In all of the tapes, the fundamental frequency
(pitch) increased significantly during emergency air
to ground communications.

Frequently, the research question and paradigm
direct the manner in which data are selected, col-
lected, extracted, and analyzed (see Kanki & Prinzo,
1996). As long as these paradigms and their associated
protocols lead researchers and investigators to view
aspects of the same event differently, conflicting in-
terpretations may occur. The lack of standardized
metrics and representative measures can be problem-
atic for many reasons. First, the comparison of results
obtained from one laboratory to that of another is
difficult or impossible to perform. Second, the results
obtained by one laboratory can be difficult to replicate
because the encoding process may not be consistent.
Finally, the results and conclusions made by research-
ers and investigators often are difficult to translate
into a curriculum or instructional practice. Fortu-
nately, when communicating with pilots, controllers
are required to use the phraseology presented in FAA
Order 7110.65 Air Traffic Control as the standard.
Researchers also can apply it to the analysis of ATC
communication.

' Air traffic control specialists may be referred to as controllers, specialists, ot ATCSs in this document.

? For brevity, the publication will be referred to as FAA Order 7110.65 throughout this document.


4.0 DISCUSSION

Following the installation of radio communica-
tions equipmentin the Cleveland, Ohio control tower
in 1936, radio communication became the primary
means of relaying information between controller

and pilots in the domestic air traffic control syste-
m, Communication is an integral component in an air
traffic controller's ability to maintain separation be-
tween aircraft and obstacles. A standard phraseology
was developed by the FAA to ensure that miscommu-
nication was kept to a minimum. Likewise, to reduce
the possibility of mishearing similarly sounding num-
bers and/or letters, a standard for pronunciation was
developed. Both are included in FAA Order 7110.65
Air Traffic Control. This standard was applied to the
analysis of controller communication derived from field
and simulation tapes that were transcribed verbatim.

There were three sets of analyses performed on the
data. The overall analysis of all communication elements
revealed the percentages of Address, Instruction, Advi-
sory, and Request speech acts within each database to be
very similar. Only Courtesy and Non-codable commu-
nication elements were present on field tapes. Courtesy
speech acts often signaled pilots that the transaction was
completed (much like saying good-bye) and radio com-
munication switched to another controller. Courtesy
speech acts during simulation were misinterpreted by the
Verbex voice recognition system and controllers were
provided the opportunity to experience this first-hand
during training. It is not surprising that Non-codable
communication elements appeared on field tapes be-
cause tapes become unclear and unintelligible as they are
reused and then duplicated onto recycled cassettes, Simu-
lated communication was always recorded onto new
cassette tapes.

The second set of analyses examined only irregular
communication elements. The primary difference
between field and simulation communication was
that there were 21% fewer irregular communications
produced by controllers during simulation than in the
field. There are several reasons why this occurred.
First, although controllers are required to use FAA
standard phraseology in the field, this requirement
was stressed by the SME during Verbex voice training
and practice using the TRACONsim. Also, a note was
posted at the workstation to remind the controller to

use required phraseology. Second, controllers knew
the Verbex voice recognition system would not work
properly unless that standard was applied throughout
the simulation. Third, prior to the onset of the first
simulation, each controller received extensive prac-
tice with the voice recognition system and the oppor-
tunity to correct irregular communication; and lastly,
controllers were aware that their communication would
be audio- and video-recorded. Results of studies on
social facilitation, brought on by the observation of
behavior by spectators (i.e., audience effects), indi-
cated that when participants knew they were being
observed on tasks in which they were skilled, their
vigilance and performance improved (Triplett, 1897;
see the review article by Zajonc, 1965). Knowing that
they were being recorded contributed to the improved
performance of controllers in this study. The second-
ary finding, revealed by the Runs Tests, was that the
distributions of irregular communication elements
(distributions of aviation topics) within each of the
speech act categories were not significantly different
for field and simulation communications. Thus, al-
though there were fewer irregular communication
elements that were produced under simulated condi-
tions, the distribution of those elements were much
the same as those in the field.

The final set of analyses examined the field- and
simulation tape distributions of the 8 types of irregular
communication codes (grouped, sequential, and so on).
Given the type and complexity of the irregular commu-
nications, only a global test was performed on the
Receiver ID aviation topic, and the Instruction, and
Advisory speech act categories. Requests were excluded
because there was insufficient data with which to per-
form an analysis. Only the distribution of irregular
communication codes for Receiver ID differed from the
field communications. This finding is easily explained.
In the field, controllers know that pilots will communi-
cate with them even if a portion of the aircraft call sign
is omitted’. During simulation training, controllers
quickly learned that omitting a part of the aircraft call
sign often required them to re-transmit the entire mes-
sage. The Verbex voice recognition system was less
forgiving than a pilot. Knowing that omitting a portion
of the aircraft call sign would increase their workload was
effective in reinforcing the importance of including the
entire call sign in a message.

Often, while pilots are listening on their assigned radio frequency, they will question the controller about a transmitted message

if the numbers in the call sign are similar to their own.

17


DOTIFAA/AM-98/17 An Analysis of Voice

- 7 Communication in a
Office of Aviation Medicine .
Washington, D.C. 20591 Simulated Approach
Control Environment

O. Veronika Prinzo

Civil Aeromedical Institute
Federal Aviation Administration
Oklahoma City, OK 73125

May 1998

Final Report

This document is available to the public
through the National Technical Information
Service, Springfield, Virginia 22161.

@

U.S. Department
of Transportation

Federal Aviation
Administration


a Sony color video monitor PVM2530, equipped with
2 Sony SS-XGA speakers, and recorded by a Sony
video cassette recorder SVO-1610 on standard VHS
T120 cassettes.

2.2.3Audio Recording Equipment. A Sony Elec-
tret condenser microphone (ECM-77B) was attached
to a Shurlite headset and positioned approximately
1.5cm from the controller’s lips. The output signals of
the microphone were amplified by a Panasonic audio
mixer WR-450 and then sent to a Sony digital audio
recorder PCM-2700, where each utterance was time
stamped and recorded on 120-minute BASF DAT
cassettes.

2.3. Simulation Support Staff

The simulation support staff consisted of one ghost
pilot, a retired controller, and several representatives
from each TRACON facility. The certified ghost pilot
from the FAA Academy was trained on the scenarios
constructed for this experiment. A recently retired
FPL controller served as the subject matter expert
(SME). He constructed the scenarios, trained the
ghost pilot on the TRACONpro system and sce-
narios, developed briefing materials, and provided the
ghost pilot with on-line instruction during each simu-
lation. Several staff members from each TRACON
facility provided subject matter expert information

and guidance during the development of the airspace,
procedures, and scenarios. Also, prior to the onset of
the experiment, several controllers reviewed and of-
fered suggestions, which increased the fidelity and
realism of each scenario.

2.4 Materials

2.4.1Scenario Construction. The number of air-
craft requiring radar service was experimentally ma-
nipulated to simulate high- and low-workload
scenarios. For example, light traffic density at one of
the TRACON facilities averaged approximately 1.5
aircraft communicating with approach control per
minute, and heavy traffic averaged 2 aircraft commu-
nicating with the controller per minute. Three con-
troller positions from each facility were simulated.
Traffic density was crossed with controller position to
produce 6 scenarios for each facility.

2.42 Ghost Pilot Communication Scripts. Based
on field tape analyses, normal and problematic pilot
communication scripts were constructed and fully
counter-balanced for use in each scenario. The scripts
were used by the ghost pilot, who initiated calls to
ATC at pre-determined times and responded to mes-
sages generated by the controller. The problematic
transmissions that were incorporated in the commu-
nications are presented in Table 1.

Is it OK if we reduce to "2" "1" "0" now?

Request ILS approach at (airport name).

Request runway "3" "5" right ILS approach.

Table 1. Target Aircraft Transmissions Made by the Ghost Pilot to Controllers
Target Pilot Transmissions to Controllers at TRACON Facility 1

| wanna confirm we are going to runway "3" "5" right.

Verify the altimeter is "3" "0" "0" "4" and our runway assignment is "
Give us the wind and altimeter again and we would like to land "1" "8"
Can | reduce down to “1" "9" "0" knots on account of the chop?

Request runway "1" "3" right if it’s not too much of a problem for you.
Target Pilot Transmissions to Controllers at TRACON Facility 2

(Name) Approach, any chance landing north today for "
(Name) Approach, I'd like to request the ILS "1" "7" on the right with a full stop at (airport name).

Requesting an ILS runway " " right approach.

Sir, we'd like to make a missed approach this afternoon and we'll just come back for the localizer.

left?



The results that are presented in this report are
limited to a discussion of the analysis performed on
the message content of controller transmissions. Full
performance air traffic control specialists were re-
cruited from two terminal radar approach control
(TRACON)’ facilities to participate in this study.
They provided radar separation for simulated aircraft
during periods of typical low and high traffic that
represented actual traffic counts at their respective
facilities. Recorded, digitized pilot messages were
generated by a TRACON simulator in response to
communications initiated by the controller. If the
simulator failed to generate an appropriate pilot re-
sponse, a certified “ghost pilot” from the FAA Acad-
emy intervened with the correct response. Simulated
communications were analyzed in accor-
dance with the procedures outlined in Prinzo
et al. (1995). Communication irregulari-
ties were identified and statistically com-
pared with irregular communications
identified from field tapes obtained from
two approach control facilities. If the re-
sults could be replicated in a simulation envi-
ronment, then they could generalize to and
have relevance in real-world applications.

2.0 METHOD

2.1 Participants

Twenty-four full performance level (FPL)
controllers from two TRACON facilities
completed this study. One facility pro-
vided 9 male and 3 female controllers. Col-
lectively, they had 13.17 mean years of
terminal experience (SD = 3.49) with 9.88
mean years (SD = 3.19) at the FPL. The
other facility provided 10 male and 2 fe-
male controllers who, on the average, had
12.13 years of experience (SD = 3.16) with 10.04
mean years at the FPL (SD = 3.20) and 11.30 mean
years working as terminal controllers (SD = 2.63).

2.2 Equipment

2.2.1TRACON and Ghost Pilot Workstations.
Wesson International’s TRACONpro® software was
installed on two 486/66 MHz DX2 personal comput-
ers. Each workstation displayed radar traffic on a 21"

multi-scanning capable monitor (1280x1024x256)
with high-resolution video adapters. The TRACON
workstation included an amber 14" monitor for dis-
playing ATIS, a track ball and ARTSIIIA-simulated
keyboard, standard 101-style keyboard, Verbex 6000
Voice Systems continuous voice recognition “slave”
computer board, push-to-talk headset, and Sound-
Blaster 16-bit digitized sound board. Figure 1 shows
several TRACON workstations in use. The ghost
pilot workstation included a standard 101-style key-
board and computer mouse. The TRACON work-
station was housed in a room separate from the
ghost pilot workstation. The workstations com-
municated with each other through a LANtastic
network operating system.

Figure 1. TRACONpro simulator

2.2.2Video Recording Equipment. A Sony
Handycam CCD-TR81 video Hi8 Camcorder,
mounted on a Bogen 3165 tripod, was positioned
approximately 4 meters to the left and 6 meters in
front of the controller’s workstation. Only the radar
display, side view of the controller, and hand move-
ments were recorded. The audio/video output of the
Sony Handycam was routed to a Panasonic audio/
video distribution amplifier (15-1103), displayed on

* Aterminal radar approach control (TRACON) facility is associated with an air traffic control tower that uses radar to provide approach

control services to aircraft.




ssc‘ s 8 761 £€80'T T87'E SI8‘T NOILVIOWIS

LST bl ol 7B LOS LLY LOE adVL ayaa

TVLOL ‘W1ava0O-NON ASaLINOD ISanORy, RUOSTATY NOLLOMMISNT ssauaay GOuNOS VLVG
AYOOALVO LOV HOGAdS

Ar0Baxe9 iy ypsadg ypeg uA suoNeorUNUTUIOD Ie;NBosI] Jo JoquINN [eI0 |,
a XIGNdddv

Bl




30%

25%

20%

15% 4

10%

5%

0% +

Altitude
WApp./Dep.
BATIS
DRt./Position
Weather

O Sighting
Traffic

Gen. Acknow.

30%

25%

20%

15%

10%

5%

WApp./Dep.
BATIS
MRt./Position
Weather

O Sighting
Traffic

_ Gen. Acknow.

0% |

Figure 9. Simulation Tapes: Distribution of Advisory Irregular Communications

14







codification of 290 communication elements within
120 transmissions by the SME and one of the authors
of the ATSAT’. If their encoding of acommunication
element matched, it received a value of 1; otherwise a
0 was assigned. Percentage agreement was defined as:
Percentage agreement =
¥ concordant pairs of communications elements
~ communication elements

There was 98% agreement for segmentation of the
entire message into identical communication elements
and 96% agreement for classification of communica-
tion elements in the same speech act category and
aviation topic.

The identical match criterion for IC codes was
stringent. It required that both SMEs assigned a
communication element the same type and number of
IC codes. For example, there are two communication
elements in the transmission, “American (uh) five five
one / fly heading zero one zero.” Both SMEs might
encode the “zh” in the first communication element
as a Dysfluency; and only one encode ‘five five one” as
Sequential (non-grouped). In such a case, the com-
munication element received a value of 0. There was
81% agreement for selection of the same IC code
associated with a communication element.

3.0 RESULTS

The results of the analysis performed on the com-
munication elements are presented in three sections.
In Section 3.1, all of the communication elements are

presented by speech act category. In Section 3.2,
communication elements that contained irregular
communications are presented by speech act category.
In Section 3.3, the communication elements that
contained irregular communications were analyzed
according to their speech act category using non-
parametric and descriptive statistics.

3.1 Analysis of Communication Elements
Atotal of 13,900 ATCS transmissions (TRACON-
1: 6,100 and TRACON-2: 7,800) consisting of
33,000* communication elements from the simula-
tion tapes were compared with field taped transmis-
sions. There were 1,900 ATCS transmissions
(TRACON-1: 1431 and TRACON-2: 469) consist-
ing of 5,336 communication elements from the field
tapes. All transmissions were analyzed in accordance
with ATSAT., procedures (Prinzo et al., 1995). Table
4 presents the distribution of the total number of
communication elements within each speech act cat-
egory by tape source. The majority of communication
elements appeared in the Address and Instruction
followed by Advisory and Request speech act catego-
ries. Communication elements in the Courtesy and
Non-Codable speech act categories were absent from
the simulation transmissions and virtually absent from
the field communications (5% and 4% respectively).

TABLE 4. Distribution of All Communication Elements Within Each Speech Act Category

Speech Act Category FIELD TAPE (5,336) SIMULATION TAPE (33,045)
Address 36% 47%
Instruction 36% 43%
Advisory 16% 9%
Request 2% 1%
Courtesy 5% 0%
Non-Codable 4% 0%

7 Both coders were retired FPL controllers.

* For reading case, numbers have been rounded. For an exact count, see Appendix A.


For example, 34% of the (104/307) ICs within the
Address speech act category involved the omission of
part of the Receiver ID aviation topic from field tapes
(see Appendix C).

3.3.1Address. The data presented in Figure 2 indi-
cate that the majority of Address ICs involved the
receiver identification (i.e., aircraft call sign). It was
expected that there would be considerably fewer ICs
that involved the Speaker ID, especially since there are
fewer sector names (i.e., Tower, Approach Control,
Center) that are prefaced with location or facility
names and sector functions (e.g., ground, local, de-
parture, approach) when compared with the number
and type of aircraft flying in terminal airspace daily.

A Runs Test performed on the frequency of each
type of irregular communication for Receiver ID was
significant, r = 6, z = -1.702, p = .04. As shown in

Figure 3, there were proportionally more omissions
on field tapes than on tapes from the simulation
laboratory. The omitted information could be a
word(s) such as “heavy,” type, model, name, etc. or
number(s). Following initial radar contact, control-
lers might reply to a second transmission by simply
saying, “four three Charlie” instead of “November
four three Charlie.” In the field, pilots are more likely
to respond to an abbreviated call sign and, controllers
knowing this, will omit portions of the call sign as a
strategy to minimize their time on an already con-
gested radio frequency. Applying this strategy during
simulated conditions was counter-productive. The
computer’s voice recognition system would fail, forc-
ing the controller to repeat the transmission. Repeat-
ing the transmission added to the controller's workload
and frequency congestion.

100%

90% 4 @ Field Tapes

80% 4 Simulation Tapes

70% +

60% +

50% +

40% 4

30% 4

20% 4

10% +

0% +

*

%,
2

2

‘
6

Figure 2. Distribution of Aviation Topics in the Address Speech Act Category
That Contained Irregular Communications

9


Technical Report Documentation Page

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.
DOT/FAA/AM-97/17
4. Title and Subtitle 5, Report Date

An Analysis of Voice Communication in a Simulated Approach Control May 1998
Environment

6. Performing Organization Code

7. Author(s) 8, Performing Organization Report No.
Prinzo, O. V.
9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)

FAA Civil Aeromedical Institute
P.O. Box 25082

Oklahoma City, OK 73125 17, Contractor Grant No

72, Sponsoring Agency name and Address 13. Type of Report and Period Covered
Office of Aviation Medicine
Federal Aviation Administration
800 Independence Ave., S.W.

Washington, D.C. 20591 74, Sponsoring Agency Code

15. Supplemental Notes
This work was performed under Task AM-B-96-HRR-513

16. Abstract

This report consists of an analysis of simulated terminal radar approach control (TRACON) air traffic control
communications. Twenty-four full performance level air traffic controllers (FPLATC) from 2 TRACON facilities
participated in the simulation study. Each controller worked 2 light- and 2 heavy-traffic density scenarios for feeder and
final sectors, All communications were audio recorded and transcribed verbatim by a retired FPLATC. Once
transcribed, transmissions were parsed into communication elements. Each communication element was assigned a
speech act category (e.g., address, instruction, request, or advisory), an aviation topic (e.g., altitude, heading, speed) and
then coded for irregularities (e.g., grouping numbers together when they should be spoken sequentially, or omitting,
substituting, or adding words contrary to required phraseology) (ATSAT, Prinzo et al., 1995). The simulated
communications were compared to an analysis performed on audiotapes from the same TRACON facilities. Percentages
in 3 speech act categories were comparable (Instruction, 55% versus 51%; Address; 14% versus 26%; Advisory, 24%
versus 18%). Detailed analyses revealed that, although there were fewer irregular communications produced during
simulation, the distributions of those communication irregularities were very much the same, with the exception of
aircraft call sign. The differences in those distributions were attributed to the voice recognition system; it could not
recognize a call sign spoken sequentially and then restated in grouped form.

17. Key Words 18. Distribution Statement
ATC Communications, Communication Taxonomy Document is available to the public through the
ATC Phraseology National Technical Information Service,
Springfield, Virginia 22161
19. Security Classi (ofthis report) 20. Security Classi. (of this page) 21. No. of Pages 22. Price
Unclassified Unclassified 30

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized



5.0 CONCLUSION

The overall research findings indicated the control-
lers generally communicate with the simulation pilots
in ways that are consistent with how they communi-
cate with pilots at their TRACON facilities. Although
there were proportionately fewer irregular communi-
cation elements produced during simulation, the dis-
tribution of those irregularities was consistent with
those produced in the field. This conclusion also
holds true of irregular communication codes for In-
struction and Advisory speech act categories. Simula-
tion can be a useful tool to address issues associated
with the effects of changes in procedures and technol-
ogy on the communication process.

The use of voice recognition technology can be
instrumental in teaching and reinforcing basic air
traffic control phraseology as demonstrated by the
fewer irregular communications produced by control-
lers once trained on the VERBEX voice recognition
system. A limitation of the technology is that it is not
advanced enough to accommodate everyday language
usage. Sometimes it is necessary for controllers to
restate numbers spoken sequentially in a call sign,
instruction, advisory, or request in a grouped format
for emphasis or clarification. Although pilots easily
understand what is spoken, current voice recognition
capabilities lag behind.

6.0 REFERENCES

Airman’s Information Manual, Official Guide to Basic
Flight Information and ATC Procedures. (1994).
Washington, DC: US Government Printing Of-
fice,

Brenner, M., Doherty, E.T., 8 Shipp, T. (1994). Speech
measures indicating workload demand. Aviation,
Space, & Environmental Medicine, 65, pp. 21-6.

Brenner, M., Shipp, T.T., Doherty, T., & Morrissey, P.
(1985). In IR. Titze and R.C. Scherer (eds.),
Vocal fold physiology, biomechanics, acoustics, and
phonatory control. Denver: The Denver Center for
the Performing Arts, pp. 239-48.

Burke-Cohen, J. (1995). An analysis of tower (ground)
controller-pilot voice communications Washington,
DC: US Department of Transportation, Office of
Research and Development Report DOT/FAA/
AR-95/19.

18

Cardosi, K.M. (1993). An analysis of en route controller-
pilot voice communications. Washington, DC: US
Department of Transportation, Office of Research
and Development Report DOT/FAA/AR-93/11.

Cardosi, K.M., Brett, B., & Han, S. (1996). An analysis
of TRACON (Terminal Radar Approach Control)
controller-pilot voice communications. Washington,
DC: US Department of Transportation, Office of
Research and Development Report DOT/FAA/
AR-96/7.

FAA Order 7110.65G Air Traffic Control. (March, 1992).
Washington, DC: Air Traffic Rules and Proce-
dures Service.

FAA Order 7110.65H Air Traffic Control, (September,
1993). Washington, DC: Air Traffic Rules and
Procedures Service.

Griffin, G.R., & Williams, C.E. (1987). The effects of
different levels of task complexity on three vocal
measures. Aviation, Space, & Environmental Medi-
cine, 58, pp. 1165-70.

Helmreich, R.L. (1994). Anatomy of a System Acci-
dent: The Crash of Avianca Flight 052. /nterna-
tional Journal of Aviation Psychology, 4, pp. 265-84.

Kanki, B.G., & Foushee, H.C. (1989). Communica-
tion as group process mediator of aircrew perfor-
mance. Aviation, Space, and Environmental Medi-
cine, 60, pp. 402-10.

Kanki, B.G., & Prinzo, O.V. (1996). Methods and
Metrics of Voice Communications, Washington, DC:
Federal Aviation Administration, Office of Avia-
tion Medicine Technical Report DOT/FAA/AM-
96/10, Available from: National Technical Infor-
mation Service, Springfield, VA 22161; ordering
no. ADA307148.

Lieberman, P., Protopapas, A., & Kanki, B.G. (1995).
Speech production and syntax comprehension
deficits on Mt. Everest. Aviation, Space, and Envi-
ronmental Medicine, 9, pp. 857-69.

Mayer, D.L., Brenner, M., & Cash, J.R. (1996) Devel-
opment ofa Speech Analysis Protocol for Accident
Investigation. In B.J. Kanki & O.V. Prinzo (eds.)
Methods and metrics of voice communications. Wash-
ington, DC: Federal Aviation Administration,
Office of Aviation Medicine Technical Report
DOT/FAA/AM-96/10. Available from: National
Technical Information Service, Springfield, VA
22161; ordering no. ADA307148.


NOTICE

This document is disseminated under the sponsorship of
the U.S. Department of Transportation in the interest of
information exchange. The United States Government
assumes no liability for the contents or use thereof.


66° Z8ZE
00° 0 00" 0 00° 0 00" 0] 0 0 00° 0}; 00 0} 00 0} 00 0}; 00° 0 SuIplOH
00° s 00° 0 00° € 00° oO] oo I 00" 0} 00 i 00" 0} 00 0} 00° 0 depuodsuety,
LO TET 00" 6 10° 8e 00° t 10° Tw 00° 0 10° 9% co} Fil 00" 9 10° SI Paads
Iv ese 00" v 90" 861 00° oO} oOo} tL 00" 0 10 Or] 80°) 94%} 00° 0| 00° 0 WOH Isog/anoy

9

v1 89r 10 St 40° T€T 00" s £0} OL 00° 9 10° x4 7 ps 00" 6 00" 9 Surpeay
00" 4 00" 0 00° I 00" 0} 0 I 00" 0} 00 0} 00 0| 00° 0} 0 0 UOHeIUIPOW “H
00" 07 00" 0 00" € 00" 0 00} cI 00° I 00° 0 00° € 00" 0 00" 0 MOUYY UI
rca 6IL 00" II wo os 00" 0 wo} €9 00° 0 00° € el | 6th 10° al 00" v Apuanboay
aanjredaq
St 808 10° 0% v0" Sel 00" 8 10 | 8F 00" s +0 971 Pl | 99r 00" 0 00° i} Mpeoiddy
as Ole 10° 6£ v0" Srl 00" c 10 | €¢e 00" T 10° 61 +0" 1€l 00° 0 00° 1) apmntVy
60° Soe 90" 981 10° 1c 00" 0 00° I 00" I 00" or 70" 08 00" 0 00° 0 uOHIIISeY ITV
NOLLVTOWIS

66° SLIT
00° 4 00 0 00" 0 00" 0} 00 t 00° 0} 00° 0} 00 0} 0 0} 0O i} SuIPIOH
10” SI 00° 0 00" a 00" 0} 00° 9 00° 0} 00 S| 00° 7} 00 0| 00 0 Jepuodsuesy,
we vST 00° 0 70 cw 00" 8] 90] so 00" 0 £0" ee 80° 76 | 00° 0 £0" ve paeds
80° 16 00° 0 00° 9 00° a TO} cz 00" 0 70" 81 £0" ep 00" 0 00" 0 WonIsog/enoy
Lv 961 00° 1) £0" Iv 7 8% 90 | So 00° if 00° 8 £0" or 00" 0 10° €1 Surpeay
00" 8 00" 0 00° 0 00" 0} 00 v 00" 0} 00° Zz} 00° i 00" 0} 00 I ORI IPOW “H
00" t 00° 0 00° if 00" 0} 00 i 00" 0} 00 0} 00° 0} 00° 0} 00 0 mop “Ua
we BLE 00" 4 10° IT 10° ST po | es 00° 0 00" 6 Te | soe 00° 0 rau € Apuanbady
aanjiedaq:
80" 86 00" 0} 00 €] 00 v}| wo] 9 00" 0| oo €} so 19 | 00° 0} 00 I mpeoiddy
so” £9 00" 0 cw ve 00" 8 To | Lt 00° 0 00° 8 00° 9 00° 0 00° 0 ePnanlV
90° OL 00" 0 00° s 00" s 10 | 91 00" 0} 00 6 £0" ££} 00° 0} 00° t OHIHISIY WV
CCMARIBCICE

% N % u % T % vu] w]e % v % Tl % T % T % T
UOnE[NOMUESI Aguonyshq yorqpeay ‘QBEIGIOA, uonlsodsursy, = uoRMNsqng UOISSTUIQ, yenuanbag pednoin
renieg ss0oK7y aOunos V.Lva
IVLOL anbruyoey, Ar2at]oq pyepurjg-uoN anv
O1dOL
SNOLLVOINAWWOD AV INDAAAT JO SAGA T, NOLLVIAV

TOV HOdAdS ADNVAVATO/NOLLONULSNI

C2


Spo'ee 6r 67 oe 6L8‘T Srl'Pl pOs'st NOLLVTNWIS
oes See Ore al SL8 Ove’! 976'1 adVL OTA
"IVLOL ‘H1aVd0I-NON ASHLAND Tsanoay, AYOSIAGY NOILINAISNT ssauday AOUNOS VIVA

AYODALVO LOV HOdddS:

sqUsWo[y UOLESTUNUIUTOD Jo equ] [eI],

V XIGNdddV

Al


Upon initial contact, the ghost pilot reported the
aircraft call sign and current altitude to the controller.
Once radar contact was established, the ghost pilot
made a request for a lower altitude or particular
runway. In addition to requests, a rule was established
that every other ghost pilot response would include a
realistic, yet incorrect readback.

2.4.3Computer-Generated Pilot Responses. Each
non-target aircraft computer-generated pilot response
(CGPR) was created by the TRACONpro software.
Each aircraft call sign, the (International Civil Avia-
tion Organization) ICAO phonetic alphabet, and
phrases used in operational communications were
recorded, edited, and stored as SoundBlaster wave
files. The intelligibility of the CGPRs was evaluated
by the Federal Bureau of Investigation (FBI) Speech
Processing Laboratory at Quantico, VA. ACGPR was
selected at random and compared with the same
message recorded live by the originator of both mes-
sages. The spectrograms were judged to be the same.

2.5 Procedure

Upon arrival at the TRACON simulation labora-
tory, the controller was told the purpose of the study,
instructed on Verbex voice training procedures, com-
pleted voice-training on a limited vocabulary, and
gained familiarity and experience with the voice rec-
ognition system. It took approximately 2-3 hrs. to
complete voice training. Once voice trained, the con-
troller completed a 15-minute practice scenario at
which time the SME determined whether additional
voice training or practice was warranted.

On the second and third day, the controller again
completed the same practice problem to re-establish
baseline performance and then completed a 45-minute
simulation, received a 15-minute break while the next
scenario was loaded, and so on until each of 6 sce-
narios was completed. Using standard FAA phraseol-
ogy, the controller provided ATC services to all arrival
aircraft within the controller’s area of jurisdiction.
The following constraints were imposed on the order
of scenario presentation: (1) The controller did not
receive 3 consecutive heavy traffic volume scenarios,
(2) the controller provided ATC service on each of the

3 positions before working the same position again,
and (3) all controllers worked the FINAL or AR2
position first.

2.6 Data Encoding Tool and Procedure

2.6.1Aviation Topics Speech Acts Taxonomy
(ATSAT). The ATSAT isa tool for categorizing pilot/
controller communications according to their pur-
pose, operation, or action and for classifying Irregular
Communications (IC) (Prinzo, Britton & Hendrix,
1995). Aviation topics are the subject matter of speech
acts. The speech act categories and aviation topics are
presented in Table 2.

There are two categories of ICs presented in Table
3: non-standard phraseology and delivery technique.
Non-standard phraseology pertains to the use of words
and/or phrases other than those prescribed in FAA
Order 7110.65; delivery technique refers to stammers,
stutters, or misspoken words. ICs may or may not lead
to the occurrence of an operational error, pilot devia-
tion, or mishap. The reader is cautioned NOT to make
the improper inference that an IC is in any way related
to an operational error in this report. Operational errors
will not be presented nor discussed.

2.6.2Aviation Topics Speech Act Taxonomy...
ATSAT,. (Prinzo & MacLin, 1996) isa computerized
version of ATSAT.,. It was used to post transcribed
data into a pre-defined electronic spreadsheet accord-
ing to the procedures outlined in Prinzo, etal. (1995).

2.7 Data Encoding Procedure

Transmissions between controllers and pilots were
transcribed verbatim by one SME and then encoded
by another who parsed each transmission into com-
munication elements and classified them into speech
acts and aviation topics. Communication elements
that deviated from standard communication practices
specified in FAA Order 7110.65 were identified using
the IC codes included in the ATSAT®. The context in
which the transmission was spoken was vital to how
the SME encoded the communication elements.

2.7.1Intercoder Reliability. Intercoder reliability
was assessed by computing the percentage agreement
between the segmentation, categorization, and

S Pilots are not required to use the same standard phraseology as controllers when communicating. To achieve a standard for comparison
between pilot and controller communications, the following rule was established: If a pilot attempted a verbatim readback of a
controller's transmission, then the same coding procedures that were used on controllers transmissions were applied to the pilot's

verbatim readback.


3.2 Analysis of Irregular Communication
Elements

Nineteen percent of the simulation and 40% of the
field tape databases contained at least one IC code
(i.e., communication element with non-standard
phraseology or irregular delivery technique). Table 5
shows the distribution of those irregular communica-
tion elements within each speech act category by tape
source. The majority of communication elements that
contained one or more irregularities involved the
Instruction speech act category for both the field and
simulated TRACON environments (55% and 52%).
In comparing Address and Advisory speech act cat-
egories, there were more irregular communication
elements that involved the Address speech act cat-
egory produced in the simulated TRACON environ-
ment and more irregular communication elements
that involved the Advisory speech act category pro-
duced at real TRACON facilities, For both TRACON
environments, less than 5% of the irregular commu-
nication elements involved the Request speech act.

3.3 Analysis of Irregular Communications. The
Wald-Wolfowitz Runs Test’, a nonparametric test of
the null hypothesis that two samples come from the
same population, was used to determine whether the
simulation and field tape data samples differed in any
respect from one another. The Runs Test requires that
data from both samples are combined, ranked from

smallest to largest, and the number of runs" in the
distribution counted. The computed value is com-
pared to an expected value obtained from a statistical
table in which various sample sizes and probabilities
are presented. If the computed value is larger than the
expected value, then the null hypothesis is rejected.
When the null hypothesis is rejected, it indicates that
the two groups differ in some measurable way. Sepa-
rate Runs Tests were performed on the total number
of Instruction, Advisory, and Request ICs with p =
.05. Since the ATCSs’ communications during simu-
lation were expected be the same or similar to those on
the field tape, it was expected that the Runs Tests
would be non-significant.

Since each irregular communication element could
contain a maximum of three IC codes, the total
number of IC codes could exceed the number of
irregular communication elements. Irregular Com-
munication that involved the Courtesy and Non-
Codable speech act categories accounted for less than
1% of the total distribution and were omitted from all
figures. The percentage for each type of IC was calcu-
lated separately for field and simulation tapes using
the formula presented below:

Percentage Irregular Communications =
2 Aviation Topic Speech Act IC
= Address ICs

TABLE 5. Distribution of Irregular Communication Elements
Within Each Speech Act Category

Speech Act Category FIELD TAPE (2,157) SIMULATION TAPE (6,385)
Address 14% 28%
Instruction 55% 52%
Advisory 24% 17%
Request 4% 3%
Courtesy 0% 0%
Non-Codable 3% 0%

° For brevity, the Wald-Walfowitz Runs Test will be referred to as The Runs Test.
” Runs are defined as: any sequence of scores from the same group (cither from the simulation or field tape).

8