" 

" 

{ 

NASA Contractor Report 3300 NASA-CR-3300 

1(( ~ 0 (90 as-Seta 

Synthesized Voice Approach Callouts 

for Air Transport Operations 

Carol A. Sim pson 

CONTRACT A64184B 

OCTOBER 1980 

NI\S/\ 

111111\1\ 11111\\\ 11\\1 \\1111\11\ \\1\111111111 

NF02117 
NASA Contractor Report 3300 

Synthesized Voice Approach Callouts 

for Air Transport Operations 

Carol A. Simpson 

Psycho-Linguistic Research Associates 

Menlo Park, Califomia 

Prepared for 

Ames Research Center 

under Contract A64184B 

NI\S/\ 

National Aeronautics 

and Space Administration 

Scientific and Technical 

Information Branch 

1980 
TABLE OF CONTENTS 

List of Tables .............................. iii 

List of Figures ............................... v 

List of Appendices •.•..••.•••••••.......•..••• v 

For eward ....................•................ v i 

Su!nmar y .•...•.......•.......•.•.....•...... vii i 

Introduction .................................. 1 

Objective ..................................... 2 

Experimental Approach ••••••••••••••••••••••••• 2 

Flight Deck Data ~ollection •.••.•..••.••••.•.. 3 

Approach and Landing Procedures ••••••• 4 

ATC Communications Workload 

by Phase of Flight •••••••••.•. 8 

Overlapping Audio Signals an1 

Voice Co~munications •••••••••• 8 

Synthesizer Callout System (SYNCALL) .•.••..••• 11 

System Design •••..•••••••••••••••••••• 11 

Voice Message Design •••••••••.•••••••• 14 

Experimental Design .••.••••..•.•.•.•••.•.••••• 16 

Development of Performance Measures ••.•••.•.•• 19 

Percent Time Out of Tolerance •.•••.••• 20 

Maximum Deviation •••••••••••••••••••.• 20 

Mean Positive Deviation ••.•.•.••••••.• 20 

Mean Negative Deviation ••••.•.••.•••.• 21 

Flight Performance Parameters •••••••.•••.••••• 21 

Jperational Airspeed Tolerance ....•••• 22 

Operational Sink Rate Tolerance ••••••• 22 

Operational Localizer Deviation 

Toleran:::e ..................... 25 

Operational Tolerance for Glideslope 

Deviations •••••••.•••.•••••.•• 25 

Measures of Pilot Ju1gments of System 

Per for:nance ...................•....... 28 

Procedure ..................................... 30 

Resul ts ....................................... 33 

Flight Perfor~ance Results •..••..••..•.••••••• 33 

Analyses of Approaches Overall .••.•••• 34 Su~mary of 4-Way AOV's .•.•..•.......•• 42 

Analyses of Individual Appro3ches •.•.• 43 

Airspeed Deviations During Localizer 

Back Course Approach ..•••.•••. 44 

Sink Rate Deviations During the VOR 

Approach ...................... 45 

1-Engine Approach and Landing 

Performance •.•..••••••...••... 48 

Summary of Individual Approach 

R·esults ••.••....••••..•...•••• 51 

SYN:ALL and Pilot-not-Flying Callout 

Per for!nanc e ........................... 43 

Reliability of Callouts ••.•••••••••.•• 52 

Relative Timing of Normal Callouts 

made by PNF and by SYNCALL ••.• 57 
P Hot 

Other 

Timing 

Audio Information that Overlapped SYNCALL Callouts •••• 

Accuracy of Ten-Foot Callouts 

.58 

by SYNCALL •••••••••••••••••••• 59 

Inappropriate 

8allouts ~ade 

Callouts by 

by PNF but 

SYNGA.LL. 

not 

Su:nmary 

Juig:nent 

Su;n.nary 

by SYN8ALL •••••••••••••••• 

of C:lllout Performance Data. 

D.3 t a ••••••• " ••••••••••••• 

of Pilot Judg:nent Oa ta ••• 

•• 60 

•• 60 

• .65 

• .65 

• .74 

Discussion ••••• • .76 

Conclusion ••..•••..••••.•••••••••. .78 

Refine:nent of a 

Callout 

Synthesized Voice 

Syste:n ••••••••••••• 

Components •••••••• 

Altituie Callouts •• 

••• 79 

SYN8ALL System • •• 79 

Normal .80 

Normal Position Callouts. ..80 

Jeviation Callouts........ • •• 80 

Critical Altitude Callouts... .80 

Priority Logic.............. ..8l 

Callout Timing and Repetition Logic ••• 32 

Tolerance Values............... ..83 

Sum~ary of Recommended Changes 

to SYNCALL................ .84 

Su:n~ar~ of Proposed Modifications 

to SYNCALL. •• • ••••••••• 88 

References ..••.••..•..•••.••..•••.•.••.•.••.•• 89 

-ii-
ra~le 1. 

Table 2. 

Ta~le 3. 

Table 4 • 

Table 5. 

Table 6. 

Table 7 • 

Table g. 

Table 9. 

Table 10. 

Table 11. 

Table 12. 

Table 13. 

Table 14. 

Table 14. 

Table 15. 

Table 15. 

Table 16. 

Table 17. 

Table 18. 

Table 19. 

Table 20. 

Table 21. 

Table 22. 

Table 23. 

Table 24. 

LIST OF TABLES 

- Types of flight operations observed •.•..••.••. 4 - Mean Numbers of ATC changes During 

Descent and Approach .•..•••.•.••.••.•.•••.••.•• 5 - Air Traffic Control Voice Communications 

Workload by Phase of Flight ...•..•.•.••••••.•.• 8 - SYNCALL Triggering Conditions and 

and Wordings for Normal Callouts •.•.••.•.•.•• 13 - SYNCALL Triggerings and Callout 

Wordings for Deviation Callouts •••....••.•••• 15 - Examples of SYNCALL Callouts .•..••..••...•.•. 15 - Approaches Flown for SYNCALL Evaluation ...•.. 18 - Experimental Design •••.•••.••••••.•••••.•••.. 19 - Semantic Differential Scales ..••.•..••.••.••. 29 - Types of Callouts Measured with 

Semantic Differential ..•...•.•......•••..•••• 30 - Introductory Passage by Speech Synthesizer ..• 31 - Sink Rate Performance for All Approaches 

Dur ing Segmen t 11 ............................ 37 

- Airspeed Performance for All Approaches 

During Segment II ••••••••.• ~ .•••..••••.••.•.• 37 - (a & b) Airspeed Performance During 

Segment II1 .................................. 39 - (c & d) Sink Rate Performance During 

Segment 111 .................................. 40 

- (a & b) Airspeed Performance During 

Segment IV .....•............................. 42 

- (c & d) Sink Rate Performance During 

Segment IV .................................•. 43 

- Airspeed Performance for Localizer 

Back Course Approach During Segment IV ••••••• 45 - (a & b) Number of Approaches Inside and 

Outside Tolerance for Airspeed During 

Localizer Back Course Approach .••.••••••....• 48 - Sink Rate Performance for VOR Raw Data 

Approach during Segment III ••.••...•••••••••• 49 - Airspeed Performance for 1-Engine 

Approach and Landing ••••••••••••.•••..•.••••• 50 - Reliability of SYNC ALL and PILOT-NOT-FLYING 

for Normal Callouts in the Simulator •.••...•• 53 - Reliability of PILOT-NOT-FLYING on the 

Line and in the Simulator for 

Normal Callouts ..........•................... 54 

- Reliability of Normal Callouts by 

PILOT-NOT-FLYING on the Line and in the 

Simulator and by SYNCALL •..•.•.••••.•.•.••••• 54 - Reliability of Deviation Callouts by 

SYN8ALL and PILOT-NOT-FLYING ••••.•••••••••••• 55 - Reliability of Deviation Callouts by 

PILOT-NOT-FLYING with SYNCALL and with I~O S YN CALL ••••••••••••••••••••••••••••••••••• 56 

Table 25. - Reliability of Individual Normal Callouts 

by SYNCALL and by PILOT-NOT-FLYING in 

the simulator and on the Line ••.•••.••••••..• 57 

-iii-
Table 26. - Relative Timing of Normal Callouts by 

PILOT-NOT-FLYING and SYNCALL •••••••. ~ •••••••• 5B 

Table 27. - Percent of SYNC ALL Callouts Overlapped 

with other Audio Information •••••••••.•••.••• 59 

Table 28. - Percent of Pilots who wanted SYNCALL 

to Make Normal and Deviation Callouts 

for Four Types of Approach Conditions •.••..•• 66 Ta~le 29. - Percent of Pilots who wanted PILOTNOT-FLYING to Make Normal and Deviation 

Callouts for Four Types of Approach 

Cond i tions .........................•...•..... 67 

Table 30. - Perceived Intelligibility of Speech 

Heard during SYNCALL Study ••••••••••••••••••• 70 

Table 31. - Line Pilot Preferences for SYNCALL 

Deviation Callout Tolerances ••.•••.•••.•••••• 71 

Table 32. - Line Pilot Preferences for SYNCALL 

Deviation Callout Repetion Rates and 

Ac t i v e Z 0 n e s ••••••••••••••••••••••••••••••••• 71 

Table 33. - Pilot Ratings of Their Own Suggested 

Modifications to SYNCALL ••••••••••••••••••••• 72 

-iv-
LIST OF FIGURES 

1. - American Airlines DG-10 \pproach Callout Procedures, 

Flight Manual, page 63, 2-27-75. Reprinted by permission .•.•.•. 7 

2. - Experimental SYNCALL (Synthesized Approach Callout) 

System ••••••••.•••.•••••••••••••••••.••••••.••••.••••••••••••• 12 

3. - Operational Tolerance: Airspeed .............•.•.•••...•. 23 

4. - Jperatio~al Tolerance: Sink Rate ...•........•.••.•...•.. 24 

5. - Operational Tolerance: Localizer •••...•.•...•...•...•..• 26 

5. - Operational Tolerance: Glideslope .......•••......•...•.. 27 

7. - Analysis of Variance Design for Approach Sat 1. ......... 35 

8. - Analysis of Variance Design for Approach Set 11 ••••••••• 35 

9. - Design for Analysis of Variance for 

Ind i v id ua 1 A;:>;:>roac hes ••••••...•••••.•••••••••••••••.•••••••••• 45 

10. - ~ean Airspeed Deviaiton~ for Localizer Back 

Course Approach .••••.••..••••••.•.•.••..••.........•.•...•.... 47 

11. - Al ti tude Error 'of Ten-Foot Callouts by SY11:ALL 

as a Function of Sink Rate •..........•...•.••••..••.•...•...•. 61 

12. - Minimum Altitude Error by Sink Rate 

for Ten-Foot Callouts by SYNCALL •.•...•.•.•.••......•..••..••• 62 

13. - Mean Altitude Error by Sink Rate 

for Ten-Foot Callouts by SYNCALL •••..•.•.•••...•..•.•.•••..•.• 63 

14. - ~aximum Altitude Error by Sink Rate 

for Ten-Foot Callouts by SYNCALL ..•.••.•.....•...•...••••..••. 64 

15. - Semaritic Differential Ratin~s of Callouts 

by 14 Captains and 14 First Officers .••.•.••••••••...•...•.••• 68 

16. - Semantic Differential Ratings of the 

GPWS "Glideslope" and the GPWS "Whoop Whoop! Pull Up!" by 14 Captains and 1~ first Officers •••.•••••.•.•......•.••••• 69 

LIST OF APPENDICES 

A. - Altitude and Approach Callouts by Speech Synthesizer, 

paper prepared by Carol A. Simpson,National Research Council 

Associate, NASA Ames Research Center, April, 1976. 

B. - Flight Scenario for One of Four Experi~ental Orders 

C. - SYNCALL Debriefing Form 

D. - Foliow-up SYNC ALL Debriefing Form 

-v-
FORE~ARD 

This project owes its very existence to a unique combination 

of talents, resources, and dedication from numerous individuals. 

Neither NASA nor A~erican Airlines could have undertaken the project alone, yet by their collaboration, the entire industry 

stands to benefit from the knowledge gleaned from this study. 

Dr. Robert C. Houston, Director-Technical Training Support for 

American Airlines and Captain J.A. (AI) Brown, then DirectorFlying Training for American, saw the potential gain and supported the project throughout its course with whatever was needed 

that could be accomplished by AAL. For NASA, first Dr. Edward ~. 

Huff as Chief of the Man-Machine Integration Branch and Thomas E. We~pe -as Director of the Flight Management Program there made 

certain that the NASA contribution could be realized. Later, Dr. 

Alan B. Chambers, Chief of the Man-Vehicle Systems Research 9ivision, gave the project his full support. The initial contact 

between NASA and AAL grew from pioneering work done by Drs. 

Charles Billings and John K. Lauber of the Aviation Safety Reporting Program at NASA. Special thanks to Captain Jay Theder 

and :harles Coulter of Allied Pilots Association for their review 

of the project and support throughout. H. J. Bell and Jessel V. ~illiams, FAA Principle Operations Inspector for American Airlines, at the Federal Aviation Administration Air Carrier District Office in Euless, Texas, gave their full support to the 

project, agreeing to the incorporation of the SYNCALL system into 

the normal recurrent training curriculum on an experimental 

basis. And Captain Al Reeser, Assistant Vice President-Training 

and Procedures, as skeptic and our most relentless critic was our 

strongest supporter in disguise. 

The software used to edit and test the coding for the callouts themselves was developed by Lon Radin and Robert R. Krones 

of Informatics, Inc., in record time. The software that turned 

the SYNCALL design into a working system in the flight training 

simulator was the creation of Gene Tomlinson, Specialist-Flight 

Simulator Design, of AAL Simulator Support. Without his dedication to the project and the constant support of Joe DePaola, 

Manager-Simulator Support, SYNCALL would never have left the 

drawing boards. Captain Walt Estridge as Director of Flying In- stru~tion melded together flight training requirements and the 

proposed experimental design into a workable set of maneuvers. 

Colby ~oyas, Instructor Pilot, then turned this into a realistic 

scenario that could ~e used to fulfill recurrent training requirements without compromising the balanced experimental design. 

Instructor Pilots Captain L.W. (Wes) King, Captain George A. 

Mitchell, Lou gritton, and Colby Noyes went above and beyond the 

call of duty to ensure the successful performance of the experimental procedure. The spirit they brought to the dual task of 

instructing and playing the role of air traffic controller added 

much to the realism of the simulation. And their comments, based 

on continued exposure to the system over the course of the experiment were invaluable to the interpretation of the data. 

Throughout the course of the study Captain George Hazelhurst 

Chief Pilot for DC-l0 Flight Instruction gave his full support t~ 

-vi-
coordinating the input from instructor pilots, crew scheduling, 

and final check-out of the experimental SYNCALL system. Captain 

George Hoff and Captain Bob Cornelison, each as Chief Pilot-San 

Francisco facilitated arrangements for the wide rang~ of flight 

deck observation flights. And without the enthusiastic help of 

Captain Frank Nehlig, Chief Pilot-Los Angeles, the study would 

not have included realistic ATC background audio. James Marigliano (AAL) literally worked miracles with crew schedules to ensure 

that each session had a complete three-man crew. Once the data 

had been collected, the massive task of analysis could not have. 

been completed in several years time without the help of Libby 

Stager, Chris Wagenet, William Hardy, and Donna Miller at the 

Ames Research Center. 

Final thanks goes to the pilots and flight engineers who 

flew SYNC ALL and gave the project the benefit of their combined 

thousands upon thousands of hours of experience flying the line 

and to the many many pilots who enthusiastically shared their 

flight deck and their ideas on cockpit warning system deSign, 

present and possible future. Without their help, instruction, 

and ideas this project could not have happened. 

The research for this project was supported over the course of 

four years by the National Research Council, San Jose State 

University under NASA Grant NGL-05-046-002, and The State University of Utah under NASA Grant NGR-45-003-10B. Preparation of 

this special report was supported by NASA Contract A64184B to 

Psycho-Linguistic Research Associates. 

This paper is dedicated to Dr. H.P. (Pat) Ruffle-Smith. 

To each of you who played an essential role, thank you. 

Carol A. Simpson 

October 23, 1979 

-vii-
SUMMARY 

A flight simulation experiment was performed to determine 

the effectiveness of synthesized voice approach callouts for air 

transport operations. Flight deck data was first collected on 

scheduled air carrier operations to describe existing pilot-notflying callout procedures in the flight context and to document 

the types and amounts of other auditory cockpit information during different types of air carrier operations. A flight simulation scenario for a wide-body jet transport airline training 

simulator was developed in collaboration with a major U.S Air 

Carrier and flown by three-man crews of qualified line pilots as 

part of their normally scheduled recurrent training. Each crew 

flew half their approaches using the experimental synthesized 

voice approach callout system (SYNCALL) and the other half using 

the company Pilot-not-Flying approach callout procedures (PNF). 

Airspeed and sink rate performance was better with the SYNCALL 

system than with the PNF system for non-precision approaches. 

For the One-Engine Approach, for which SYNC ALL made inappropriate 

deviation callouts, airspeed performance was worse with SYNCALL 

than with PNF. Reliability of normal altitude approach callouts 

was comparable for PNF on the line and in the simulator and for 

SYNCALL in the simulator. However, SYNCALL was more reliable 

than PNF for making deviation approach callouts in. the simulator. 

pilots generally favored the concept of SYNCALL and judged it 

more reliable than PNF callouts. They suggested modifications 

before it would be appropriate for operational use. It was concluded that SYNCALL improved flight performance for non-precision 

approaches, and that a SYNCALL system should make deviation callouts only. For consistency, it is recommended that the modes of 

the Ground Proximity Warning System (GPWS) be incorporated into a 

SYNCALL system. The detrimental effects on performance associated with inappropriate deviation callouts led to the further conclusion that such callouts should be designed out of the system. 

Finally, the results and conclusions of the experiment are used 

to develop suggestions for improvements to SYNCALL before further 

testing. 

-viii-
SYNTHESIZED VOICE APPROACH CALLOUTS FOR AIR TRANSPORT OPERATIONS 

Carol A. Simpson 

INTRODUCTION 

The final approach and landing segment of jet transport 

operations requires exact and constant awareness by the crew of 

altitude and position. This information is currently provided in 

visual form by the flight instruments and in spoken form by callouts made by whichever pilot is not actively flying the approach. Airline procedures call for the pilot-not-flying to monitor the performance of the pilot-flying by scanning the flight 

instruments and calling out specific altitudes and any observed 

deviations outside company-prescribed tolerances. Prior to the 

point where the pilot-flying expects to be able to see the runway, one pilot (depending on the airline) begins looking outside 

and reports when the runway is in sight. The other pilot focuses 

visually inside the cockpit and monitors the flight ·instruments. 

This pilot and, in some cases, the flight engineer are then 

responsible for making the necessary approach callouts. The 

specific set of caliouts used and assignment of responsibility 

for making the callouts to pilot-not-flying and/or to the flight 

engineer varies somewhat across air carriers. Despite some 

differences in the choice of callouts, the intended functions of 

the callouts are common to all carriers. These are 1) to reduce 

the visual workload for the pilot-flying, 2) to keep all three 

crewmembers aware of aircraft altitude and position, and 3) in 

the case of callouts that require acknowledgment from the pilotflying, to serve as a check for incapacitation of the pilotflying. 

In the discussion that follows, the term "pilot-not-flying 

callouts" will stand for all of the approach callouts regardless 

of which crew member is actually expected to make particular callouts. This will permit discussion of the approach callouts 

themselves without referring to the differences among air carriers. 

A number of air carrier approach and landing accidents during 

low or impaired visibility have been associated with the absence 

of approach callouts. This fact at first suggests that the absence of altitude and deviation callouts may have contributed to 

inadvertent flight into terrain during these approaches. Such a 

conclusion was drawn by the National Transportation Safety Board 

in a special study on flight crew coordination procedures in accidents during air carrier instrument landing system approaches. 

(1) However, due to the absence of data on callout performance 

for successful approaches made during the same period (1970-1975) 
no test of this suggested relationship of cause and effect can be 

performed. 

Another untestable post hoc hypothesis is that increased 

workload during the approaches that terminated in accidents 

prevented the pilot-not-flying from making the callouts. While 

neither hypothesis can be directly tested, it is reasonable to 

ask whether the current procedure can be improved, especially 

when unexpected events cause a high workload in the cockpit. One 

alternative medium for information transfer would be a speech 

synthesizer which could receive raw data from the onboard central 

air data computers and automatically make the callouts at appropriate times during the approach. 

OBJECTIVE 

The purpose of the present study was to compare a pilot-notflying (PNF) approach callout system to a system composed of PNF 

callouts augmented by an automatic synthesized voice callout system. A full task flight simulation experiment was conducted to 

determine if one or the other system transfers altitude and deviation information more reliably than the other and/or results in 

better flight performance by the pilot-flying. 

EXPERIMENTAL APPROACH 

This objective was accomplished by a collaborative effort 

between American Airlines Flight Academy and NASA Ames Man Vehicle Systems Research Division. A DC-10 flight training simulator 

was flown during normal flight training using crews as assigned 

but on a strictly voluntary basis. All pilots offered the opportunity to participate did so. Twenty 3-man crews flew several 

types of approaches half with the standard callout procedures and 

half with a synthesized voice callout system (SYNCALL). Measurements were made of flight performance, callout reliability, callout interference with other audio events, and pilot ratings of 

callouts on several scales. 

The experimental approach actually incorporated two types of 

data collection: 1) Flight Deck data collection and 2) data collection during the formal flight simulator experiment comparing 

pilot-not-flying callouts to SYNCALL callouts. 

2 
FLIGHT DECK DATA COLLECTION 

Design of an automatic approach callout system and measurement of the effectiveness of approach callouts in the flight context in a way that is meaningful for airline flight operations 

requires some degree of familiarity with flight operations and 

the complex environment in which those operations take place. 

The author did not at the onset of the project, have the requisite familiarity with multi-engine jet transport airline 

operations. Thus the first phase of the project was a program of 

flight deck observation by the author. The necessary authorizations from the air carrier and the Federal Aviation Administration (FAA) were issued for "the purpose of observing flight crew 

activity to gain insights into the potential uses and pitfalls 

for synthesized speech." 

A systematic approach was taken for the flight deck observation program. The program goals were: 

1) To gain over-all familiarity with airline flight operations and the tasks performed during each phase of the flight by 

each crew member and to observe interactions among crew members, 

2) To collect data on time spent with voice communications 

with ATC as a function of phase of flight. Tnis was needed in 

order to assess the possibility of simultaneous voice messages 

from ATC and a synthesized voice callout system. 

3) To collect data on the frequency of and type of occurrences of simultaneous or overlapping voice messages from mUltiple sources and to observe the methods used by the crew to cope 

with such situations; 

4) To become familiar with the existing pilot-not-flying approach callout procedures as a function of type of weather, type 

of approach, and pilot flying; 

5) To observe any instances in which pilot-not-flying callouts were not made to determine possible reasons for each such 

instan~e, e.g. distraction from ATC, higher than normal or different from normal workload for the pilot-not-flying, intentional 

omission of certain callouts due to approach circumstances, etc; 

6) To collect the above data for a cross section of type of 

flight operations long, cross-continental, short haul, overwater, night, day, large or small airport, high or low traffic 

density, type of approach; 

7) To have an opportunity to observe instances of the crew 

performing other than normal operations. Since abnormal or emergency operations are extremely rare, a large number of observation flights was needed in order to obtain any such instances. 

8) To collect crew preferences for the possible uses of synthesized speech in the cockpit while they were in the actual 

cockpit environment. 1 

1. Such conversations were conducted only during the cruise 

3 
Approach ~ Landing Procedures 

The following narrative description of crew approach and 

landing procedures is based on the author's experience and data 

collected for a sample of flights or "trip legs" during which 

scheduled air carrier operations were observed from the "jump 

seat" directly behind the captain's seat. Table 1 lists the 

types of aircraft and flight operations that were observed. The 

flight crew was observed to be a close-knit team with each member 

performing certain functions. Throughout the approach the crew 

members must use a certain amount of judgment in planning when to 

carry out their tasks so as to complement rather than interfere 

with each other. The approach callouts are only a part of the 

procedures carried out during the approach t~ landing and as such 

will be described in this context. 

TABLE 1 • TYPES OF 

NUMBER OF FLIGHTS 

NUMBER OF DIFFERENT 

AIRCRAFT TYPES: 

727 30 

707 16 

DC-10 18 

747 8 

TIME OF DAY: 

DAY 51 

NIGHT 21 

FLIGHT OPERATIONS OBSERVED 

AIRPORTS 

72 

24 

TRIP DURATION: 

< 1 HR 5 

1 - 2 HRS 21 

2 - 3 HRS 13 

3 - 4 HRS 19 

> 4 HRS 14 

TYPE OF OPERATION: 

DOMESTIC PASSENGER 

OVERwATER PASSENGER 

DOMESTIC FREIGHT 

66 

2 

4 

----------------------------------------------------------------

Throughout the entire flight the captain and the first officer are responsible for flying the aircraft with the captain 

having final decision-making authority as pilot-in-command. Typically, the. captain and first officer alternate "legs" actually 

flying the aircraft so that each pilot receives time at the 

flight controls. So, for a given approach one of the pilots will 

fly the aircraft, either manually or using one of several autopilot modes. The other pilot communicates by voice over the radio 

with the ground-based air traffic controllers. 

segment of the flight and only at the convenience of the crew and 

with the Captain's permission. Ongoing conversations were halted 

immediately, often in mid-sentence; whenever any flight duty, 

such as communication with ATC, was required. 

4 
The Flight engineer, in addition to his primary responsibilities as the aircraft systems operator, is usually delegated additional monitoring and communications responsibilities. During 

Take-Off and Landing he serves as a back~up for the two pilots to 

double-check that checklist items are performed. 

The flight engineer's back-up function is particularly important during the approach to landing since the pilots are busy 

with frequent changes in ATC communications and navigation frequencies, and aircraft heading, airspeed, and altitude. Table 2 

lists the observed average numbers of such changes that were 

given to the crew by ATC per descent and approach. In addition 

to complying with instructions from ATC, the pilots must make 

other changes in heading, altitude, airspeed, and aircraft confi-· 

guration in order to follow their desired flight route to runway 

touchdown. 

TABLE 2. MEAN NUMBERS OF ATC CHANGES DURING 

DESC8NT & APPROACH FOR 30 SCHEDULED AIR CARRIER 

FLIGHTS ACROSS FOUR AIRCRAFT TYPES AND SIXTEEN AIRPORTS 

Segment Time (Beginning of Descent to Landing) (min) 

Number of Heading Changes 

Number of Altitude Changes 

Number of Frequency Changes 

Number of Speed Changes 

TOTAL Number of Changes 

mean 

24.0 

2.6 

5.3 

4. 1 

1.8 

13.8 

sd 

7.5 

1.8 

1.3 

1.1 

1 .7 

Preparation for the approach begins while still at cruise altitude. The flight engineer tunes his communications radio to 

the frequency on which the Automatic Terminal Information Service 

(ATIS) broadcast is being transmitted. He copies down the information on winds, runways in use, and approaches to be expected. 

He then re-copies the information onto a prepared form and passes 

it forward to the pilots. Once the pilots have this information, 

they can plan the approach. They select the appropriate approach 

charts and review the flight profile in terms of altitudes, navigational fixes, distances, headings, navigational aid frequencies, and missed approach procedures. The captain gives any special instructions warranted by the particular conditions. 

The approach segment, which has been defined here as starting 

when the ATC approach controller clears the aircraft for the approach, usually starts at approximately 3000 feet Above Field 

Level (AFL) and includes interception of the final approach 

course, passage over a final approach fix, and descent down the 

5 
desired glidepath to touchdown on the runway. All of the Approach Callouts are made during this segment, which lasts some 8 

to 10 minutes. Thus this segment was chosen for the comparative 

study of the two callout systems. 

Figure 1 is a reproduction of the page in the airline's 

flight manual describing the approach callouts. During the 

course of the approach the pilot-flying must intercept the final 

approach course and descend to the runway touchdown point at the 

necessary rate so as to maintain the desired glidepath. Tracking 

of the glidepath may be accomplished either by reference to raw 

data such as time, speed, and rate of descent or by reference to 

a ground-transmitted glideslope signal. If the latter is available, the approach is classified as a precision approach, and the 

approach can be flown to a relatively lower altitude and with 

less visibility than is possible without a glideslope signal. If 

a glideslope signal is not available, then tha~ approach is 

called a non-precision approach. Either type o~ approach can be 

flown using what is called "raw data" or ~";ing command data from 

a "Flight Director". The data provided dir\:.ctly by the flight 

instruments is called "raw data". The Fright Director, on the 

other hand, processes the raw data and provides maneuvering commands via steer and pitch bars or needles or similar display, 

depending on the aircraft. With a Flight Director, the pilot 

flies according to the steer and pich commands on the display. 

For either type of approach, there is a minimum altitude to 

which the aircraft is per~itted to descend without having the 

runway in sight. In the United States, this is called Minimum 

Descent Altitude (MDA) for non-precij~on approaches and Decision 

Height (DH) for precision approaches. If the runway is in sight 

by the time the minimum altitude is reached and if the pilotflying judges that the landing is assured, then the approach is 

continued to landing (or go-around, should the situation subsequently become unfavorable for landing). There is a further detail in the case of non-precision approaches, tllose using an MDA. 

For these there is also a Missed Approach Point (MAP) at a specified point on the final approach course. I(~',at MDA the runway is 

still not in sight, the aircraft can be, flown at MDA altitude until the MAP is reached, descending to landing if the runway is 

acquired visually. Upon reaching the MAP with no runway in 

sight, the crew makes a go-around. 

During the approach the pilot-not-flying monitors the approach, operates the slats, flaps, and landing gear as desired by 

the pilot-flying, and prepares mentally to make a go-around if 

necessary. He makes the normal approach callouts applicable to 

the particular type of approach being flown, as indicated in Figure 1. An1, if deviations occur in airspeed, sink rate, localizer, or glideslope, he calls these to the attention of the pilotflying. 

The Flight Engineer finishes the "Before Landing Checklist" 

6 
dA NORMAL PROCEDURES 

Section 3 

DC-1O Operating Manual 

STANDARD CALLOUTS - ALL DESCENTS, APPROACHES AND LANDINGS 

...4--

/" "--Transition Ll'wl • 

'*-

FlO CALLOUTS 

Page 63 

2-27-76 

\

'11.000' MSt. (FL 110 if Transition Level is at or brlow) 

FAF ~------------------------------------, 

~_~ *"!..P:..:IL~O:..!T..!.N~O:..!T...!F...!L:...!Y~IN~G~CA=LL:.:O~U~T~S 

(if making ILS approach) r "'-1/ M 

GIS interception I . ~ 1,000' AFL - Verbally Verify When Flaps at Landing Setting 

FGS Mode Annunciators ~ 

•. ......... 500' AFL, Airspeed ± Approach Speed 

Final Approach Fix and· ......... and Descent Rate 

AFL Crossing Altitude "'-

~ " 400'AFL 

'''-.( 300' AFL 

CAPTAIN CALLOUTS 'Q. 200' AFL 50' Radio Altimeter 

.L 

~ 

I DH To Touchdown At 

I 10' Increments 

""'-

~------------~---~---~ Pilot 

Airspeed 

Not 

-

Flying 

With Landing Flaps anytime speed varies from approach speed by more 

~ 

than ±5 Kts. 

Rate of Descent - When Below: If Descent Rate Exceeds: 

2,000' . 2,000 FPM 

1,000' 1,000 FPM 

300' 700 FPM 

During Cat II Approaches FlO will make all callouts from 500' to landing. including Airspeed or Descent Rate if 

not within limits. 

LOC and GS indication Callout-On final, anytime any crewmember observes LOC displa(;~ll1ent greater than 1/3 

dot andlor GIS displacement greater than I dot, other pilot will acknowledge 

NOTES: this deviation. 

1. Use the Captain's or F/O's altimeter for all altitude callouts. except for the CAT II approach, in which case 

use the radio altimeter for the 300' callout and remaining callouts below 300' to touchdown. 

2. When executing a non-precision approach, the pilot not flying will call out "100' ABOVE MDA," MDA 

and MAP. 

3. Callouts not applicable on non-precision or visual approaches should be disregarded. 

• For Transition Level outside U.S. refer to avigation charts. 

FIE MONITORING 

In addition to his normal monitoring of engine instruments and FIE panel, the FIE will monitor flight 

instruments, especially altitude and airspeed. He will assist in maintaining a watch for traffic and other factors 

that could adversely affect safety. He will monitor HSI and AD! navigation indications and call out any 

discrepancies between the two instruments. On final. call out LOC displacement anytime it is greater than 1/3 

dot andlor GIS displacement anytime it is greater than I dot. 

Figur~ 1. - Anerican Airlines DC-1Q ~~pro3ch Callout Proc~jures, 

Flight ~3nual, page 63, 2-27-76. Reprinted by per~ission. 

7 
to ensure that all systems are set for landing or, where appropriate, armed for go-around. He assists the Pilot-not-Flying 

with the monitoring of the approach. 

Since this entire segment lasts only 'some 8 to 10 minutes, it 

should be apparent that there is a fair amount of intra-cockpit 

voice communication that occurs in a short time, along with the 

ATC communications. 

~ Qommunications Workload ~ fhase Qf Flight 

It was found that the amount of time used by the crew to communicate with ATC varied as a function of phase of flight. 

Correspondingly, so did the number of ATC transactions (consisting of one exchange between pilot and controller of either 1) pilot transmission to controller + controller acknowledgement or 

other response, or 2) controller transmission to pilot + pilot 

acknowledgement or other response). Table 3 lists this data. 

TABLE 3. AIR TRAFFIC CONTROL VOICE COMMUNICATIONS WORKLOAD 

BY PHASE OF FLIGHT. MEANS FOR 39 TRIPS. STANDARD 

DEVIATIONS IN PARENTHESES. 

PHASE OF FLIGHT 

TAXI CLIMB CRUISE DESCENT TAXI TOTAL 

SEGMENT 

DURATION 

(S.D.) 

(min) 19.68 24.77 86.68 25.61 

(12.33) (11.48) (71.55) (11.72) 

TIME TALKING/ 

LISTENING (min) 

(S.D.) 

PERCENT OF TIME 

SPENT W/ ATC 

NO. OF ATC 

TRANSCACTIONS 

(S.D.) 

MEAN TIME PER 

TRANSACTION (s) 

(S.D.) 

1 .45 

(1.63) 

7 % 

9.25 

(4.09) 

8.65 

(6.00) 

2.83 

(1.44) 

1 1 % 

4.30 

(3.47) 

5 % 

17.29 21.81 

(8.34) (17.31) 

8.86 12.55 

(3.21) (6.48) 

3.9J 

(1.88) 

20.89 

(8.99) 

1 1 .84 

(5.09) 

5.58 

(3.66) 

0.47 

(0.50) 

8 % 

2.87 

(2.60) 

9.06 

(6.42) 

163.41 

(76.41) 

13.58 

(5.50) 

8 % 

70.05 

(23.55) 

1 1 .70 

(3.46 ) 

----------------------------------------------------------------

Overlapping Audio Signals ~ Voice Communications 

The types of situations in which overlapping of audio signals 

and/or speech communications occurred can be classified as either 

8 
-9-

routine or unexpected. That is, certain combinations of overlapping audio occur on nearly every flight, necessitating either 

formal or informal crew procedures for handling them. Others occur occasionally by chance and must be handled by the crew on an 

event-specific and unique basis. 

Many of the routine pairs of overlapping audio signals involve concurrent voice communications between 1) pilot-not-flying 

and ATC and 2) Flight Engineer and AIRINC (Aeronautical Radio Incorporated) or Company. Some examples are: 1) ATC Push-back or 

Taxi Clearance and Flight Engineer Time Out Report, 2) Initial 

ATC transaction with Departure Control and Flight Engineer's Time 

Off and Load Report, 3) Initial Descent from Cruise Clearance and 

FIE Monitor the ATIS, 4) Outer Marker Passage with Captain's Callout and Pilot-not-Flying contact Tower. Except for the last 

pair, the need for concurrent audio signals is handled routinely 

by assigning the ATC communications tasks to the Pilot-nct-Flying 

and the AIRINC and Company Communications tasks to the Flight Engineer. However, in the case of the Outer Marker Callout by the 

Captain and the Pilot-not-Flying ATC call to the Tower, it happens that 50% of the flights involve the Captain and the Pilotnot-Flying being one and the same individual. This individual 

must then sequence the two voice messages, with the result that 

one or the other may be late. When the First-Officer is the 

Pilot-not-Flying, he may be calling the Tower at the same time as 

the Captain is making the Outer Marker Callout, with the result 

that he may not verbally process the callout; The Captain can and 

was observed to delay the callout until after contact with the 

tower had been established (duration of approx 8 s during which 

the aircraft descends another 136 feet at, say 1000 FPM). Thus, 

the First Officer as pilot-flying on these occasions will see an 

altimeter reading th~t could be as much as 100fe~t different 

from that which the Captain as pilot-not-flying calls out. Types 

of overlapping auditory messages that occurred by chance included 

SELCAL or CABIN CALL bells or chimes overlapping with ATC communications or with crew cockpit voice communications. The Gear 

Warning Horn, which was observed to trigger several times per 

flight inappropriately, sometimes overlapped ATC communications 

or crew checklist callouts. Crew members handled these situations by sharing the workload, i.e. the Flight Engineer would 

respond to SELCALL or CABIN CALL while the pilot-not-flying would 

respond to the ATC transmission. There were instances when the 

pilot-not-flying requested a repetion of the message from the 

controller. There were also instances when the Flight Engineer 

delayed his response to the CABIN CALL until after completing his 

other verbal transaction, say reporting observed winds aloft to 

the company. 

There were 

louts omitted 

occasional instances of individual approach calwhen they would have overlapped with other voice 

communications. On one flight, for example, the controller advised the crew of men and equipment beside the runway when the 

9 
aircraft was at 300 feet AFL. The 300-foot callout was not made 

in this instance although other hundred foot-foot callouts were 

made. 

Occasionally certain ten-foot callouts were systematically 

omitted by the PNF when the sink rate during flare was relatively 

higher. The PNF was observed to call "50, 30, 10". This seemed 

to convey the added information about the higher than average 

sink rate to the pilot-flying. 

There were no approaches observed for which no callouts were 

made. Rather, individual callouts were systematically omitted 

according to specific circumstances of the approach. But in 

these cases, prior and succeeding callouts were made. 

10 
SYNTHESIZER CALLOUT SYSTEM (SYNCALL) 

Figure 2 shows the experimental SYNCALL system designed to 

test the concept of automatic approach callouts. This system was 

designed to include all of the types of automatic approach callouts that might be useful. As such it was an experimental tool 

rather than a prototype system. 

System Design 

The synthesizer was programmed to make both normal and deviation callouts. The normal callouts were the standard callouts 

then required for all approaches by the participating airline. 

Table 4 lists the normal callouts made by the SYNCALL system with 

the flight condition that triggered each callout listed in the 

left column and the wording of each corresponding callout in the 

right column. A given normal callout was spoken by SYNCALL only 

once for a given approach. Also, if SYNCALL detected that it 

would deliver a normal callout within +1-50 feet of the altitude 

at which a Decision Height callout, a 100 feet above MDA callout, 

or an MDA callout was to be made or had been made, it supressed 

that normal callout. 

The SYNCALL system was programmed to make deviation callouts 

only if glideslope deviation, localizer deviation, airspeed, or 

sink rate deviated beyond the tolerances that the Pilot-notFlying is required to use for PNF deviation callouts. For example, the Pilot-not-Flying and the FIE were required to call deviations of +1-5 KTS or more deviation from the selected approach 

airspeed. The SYNC ALL system was programmed to make an airspeed 

deviation callout if the deviation reached +1-7 KTS. The SYNCALL 

system was designed with larger tolerances so that it would serve 

as a back-up on deviation callouts for both the PNF and the FIE 

rather than functionally replacing these two crew members for the 

deviation monitoring and callout task. This was done because it 

was feared that the PNF and FiE might not monitor the flight instruments often enough or be left out of the information loop if 

the SYNCALL deviation callouts were not designed to give them 

first chance to detect any deviations. Table 5 lists the deviation callouts with the flight conditions that triggered them on 

the right and the callout wordings on the left. 

It will be noted that some of the modes for the Ground Proximity Warning System (GPWS) coincide with conditions that would 

normally trigger deviation callouts by the SYNCALL system, cf. 

"Glideslope" for 1.3 dots or more below the glideslope. In order 

to avoid the aural confusion of concurrent voice messages from 

different sources but stating the same message, the aural warning 

of the GPWS was disabled when the SYNCALL system was in use. The 

SYNCALL system was programmed to perform all of the relevant GPWS 

callouts in addition to the deviation callouts that are not made 

by the GPWS system. In the case of simultaneously occurring 

demands for two or more deviation callouts, the callouts were 

made in sequence. The priority assignment from highest to lowest 

11 
SYNCALL APPROACH CALLOUTS 

/wI.-:=- TRANSITION LEVEL * 

~-11'000' MSL (FL 110 IF TRANSITION 

LEVEL IS AT OR BELOW) 

~_ F~r 1,000' AFL 

/ 

i1"'-. / 500' AFL, AIRSPEED ± APPROACH 

G/S :: '" SPEED AND DESCENT RATE 

INTERCEPTION:; "'''- 400' AFL 

(IF MAKING :. 100' 'J 300' AFL 

I LS APPROACH) ABOVE ~ ~~ 200' AFL 

MDA /~.....-DH MDA~ 

OUTER MARKER AND 

AFL CROSSING ALTITUDE 

50' RADIO ALTIMETER 

TO TOUCHDOWN AT 

10' INCREMENTS 

DEVIATION CALLOUTS 

AIRSPEED - BELOW 600 AFL, APPROACH SPEED ±7 kt 

RATE OF DESCENT - BELOW 600 AFL 1200 fpm 

BELOW 300 AFL 1000 fpm 

LOCALIZER - ± 1/2 DOT 

GLiDESLOPE - ± 1-1/3 DOT 

Fi3ura 2. - Experimental SYNCALL (3ynthcsizad Approach Callout) 

System. 

12 
TABLE 4. SYNCALL CALLOUT TRIGGERING CONDITIONS AND 

CORRESPONDING CALLOUT WORDINGS FOR NORMAL CALLOUTS 

TRIGGERING CONDITION 

Glideslope Interception 

Localizer Interception 

5 s after Outer Marker 

Triggering 

Descending Through 1000 ft AFL 

Descending Through 500 ft AFL 

Descending Through 400 ft AFL 

Descending Through 300 ft AFL • 

Descending Through 200 ft AFL 

Descending Through 100 ft AFL 

Descending Through DH Setting 

on Radio Altimeter and 

Setting < or = 250 ft 

Descending Through 100 feet 

Above Radio Altimeter DH 

Setting & Setting > 250 ft 

Descending Through DH Setting 

on Radio Altimeter and 

Setting > 250 ft 

Descending Through 50 ft Radio 

Altimeter 

" 40 ft Radio Alt. 

" 30 ft " 

" 20 ft " 

" 10 ft " 

CALLOUT WORDING 

GLIDESLOPE INTERCEPT 

LOCALIZER INTERCEPT 

OUTER MARKER (----) ABOVE FIELD 

ONE THOUSAND FEET ABOVE FIELD 

FIVE HUNDRED FEET. AIRSPEED 

(+1- ----), SINK IS (----) 

FOUR HUNDRED 

THREE HUNDRED 

TWO HUNDRED 

ONE HUNDRED 

DECISION HEIGHT 

ONE HUNDRED FEET ABOVE MDA 

MDA 

FIFTY 

FORTY 

THIRTY 

TWENTY 

TEN 

KEY! (----) refers to a specific value spoken by SYNCALL 

e.g. "Five Hundred Feet, Airspeed + 10, Sink is 800." 

DH Decision Height 

MDA Minimum Descent Altitude 

• Note: Theoretically, when the Autoland Mode of the autopilot 

was engaged, indicating a Category II Approach, SYNCALL monitored the Radio Altimeter for altitude information from 300 

feet to touchdown. Otherwise SYNC ALL monitored Barometric 

Altitude from the Captain's Altimeter, which was set to 

Altitude Above Field Level (AFL) until 100 feet. This was in 

13 
accordance with airline procedure to eliminate erroneous AFL 

altitude readouts due to uneven terrain on final approach. 

was GPWS mode 2a or 2b (Excessive Terrain Closure), glideslope, 

indicated airspeed, sink rate, and localizer. A given deviation 

callout was spoken only once. If, however, the out-of-tolerance 

condition had not been corrected after a specific time lag, the 

deviation callout was repeated. This time lag was 5 s for the 

glideslope, airspeed, and sink rate deviation callouts and 10 s 

for the localizer deviation callouts. For airspeed and sink rate 

deviation callouts, a repetition was triggered after the time lag 

if the deviation was still out of tolerance. For localizer and 

glideslope deviation callouts, a repetition was triggered only if 

after the lag the aircraft was not closing on the desired intolerance window, i.e. only if the deviation was of the same magnitude or greater than it had been when the previous deviation 

callout was triggered. This type of repetition logic was 

designed because it takes time for the aircraft to respond to the 

pilot's control input to correct flight path deviations in 

glideslope and localizer. As long as the flightpath was in the 

process of being corrected, the SYNC ALL system made no repetitions of the deviation callout, so as to reduce nuisance callouts. 

Voice Message Design 

The wording of the voice messages was selected so as to 

adhere as much as possible to the phraseology used by the crew 

for the approach call outs while applying previous research on 

voice warning message wording (2) to ensure high intelligibility 

even with competing human speech messages. Table 6 lists the 

wordings for each type of SYNCALL approach callout. These wordings were then tested for intelligibility under conditions that 

were more difficult than any expected to occur in a real cockpit. 

(See Appendix A) 

14 
----------------------------------------------------------------

TABLE 5. SYNCALL CALLOUT TRIGGERINGS AND CORRESPONDING 

CALLOUT WORDINGS FOR DEVIATION CALLOUTS 

TRIGGERING CONDITION 

After GIS Capture, if 

AIC >1= 1 1/3 dots 

above Glideslope 

Same as Above, except 

AIC same amount below 

Glideslope (=GPWS Mode 5) 

After Localizer Capture, 

if AIC >1= 1/2 dot right 

of Localizer 

Same as Above, except 

AIC same amount left 

of Localizer 

AIC Below 600 ft AFL 

and Indicated Airspeed 

> or = 7 kts above 

Approach Speed Setting 

of Autothrottle System 

Same as Above, except 

Indicated Airspeed is 

> or = 7 kts below 

Approach Speed Setting 

If 600-300 AFL & 

Sink> 1200 fpm or 

If < 300 AFL & 

Sink> 1000 fpm 

GPWS Mode 2a or 2b 

Triggered 

CALLOUT WORDING 

YOU'RE ABOVE THE GLIDESLOPE 

YOU'RE BELOW THE GLIDESLOPE 

YOU ARE RIGHT OF THE LOCALIZER 

YOU ARE LEFT OF THE LOCALIZER 

AIRSPEED PLUS (----) 

AIRSPEED MINUS (----) 

SINK IS (----) 

WHOOP WHOOP PULL UP 

KEY: (----) refers to a value spoken by SYNCALL 

e.g. "Sink is 1300." 

Notes: Deviation Callout repetition rate if condition not 

corrected = 5 s for Glideslope, Airspeed, and Sink Rate 

and 10 s for Localizer. Callout Priority from highest to 

lowest was GPWS Mode 2a or 2b (Terrain Closure), Glideslope, 

Airspeed, Sink Rate, Localizer. 

15 
----------------------------------------------------------------

----------------------------------------------------------------

TABLE 6. EXAMPLES OF SYNCALL CALLOUTS 

LOCALIZER INTERCEPT 

GLIDESLOPE INTERCEPT 

OUTER MARKER 1460 ABOVE FIELD 

1000 FEET ABOVE FIELD 

500 FEET, AIRSPEED +4, SINK IS 1300 

500 FEET, ON SPEED, SINK IS 1200 

400 

300 

200 

100 

100 FEET ABOVE MDA 

MDA 

DECISION HEIGHT 

50 

40 

30 

20 

10 

YOU'RE ABOVE THE GLIDESLOPE 

YOU'RE BELOW THE GLIDESLOPE 

YOU ARE LEFT OF THE LOCALIZER 

YOU ARE RIGHT OF THE LOCALIZER 

AIRSPEED MINUS SEVEN 

AIRSPEED PLUS NINER 

SINK IS FIFTEEN HUNDRED 

WHOOP WHOOP PULL UP 

EXPERIMENTAL DESIGN 

In order to study and evaluate the concept of an automatic 

approach callout system using synthesized speech, the experimental SYNCALL system was compared to a current airline system of 

approach callouts by the crew - referred to here as PNF callouts. 

For the data to be representative of the range of approach conditions encountered in the operational enVironment, several types 

of approaches, varying in crew workload and amount of manual 

flight control, were flown both by captains and first officers. 

For each type of approach, data was collected for each crew for 

both the SYNCALL and PNF approach callouts for both captains and 

first officers as the pilot flying. 

16 
So as to obtain representative user-pilot performance and judg~ent data, current line pilots, i.e. regular airline pilots 

who routinely fly the aircraft, served as partiCipating pilotsubjects. The experimental sessions were incorporated into the 

airline's normal recurrent training that is required periodically 

for air transport pilots. The crew members involved in the study 

and the Federal Aviation Administration were given a guarantee 

that participation in this experimental study would in no way be 

allowed to jeopardize a crew member's training in required 

maneuvers. In all cases, participation in the study was voluntary, and refusal to participate in no way affected a pilot's 

rating or flight status. In fact, the pilots typically expressed 

interest in the study and pleasure at being asked to participate. 

None refused to partiCipate. 

A major advantage of incorporating the experiment into ongoing recurrent training in an airline simulator was the added 

realism that could be attained fro~ the presence of completely 

operational aircraft systems at all crew member stations and from 

the use of flight procedures flown by pilots who were familiar 

with them and with the aircraft being simulated. It would have 

been difficult to obtain the same degree of realism in a flight 

research simulator since such simulators lack all but the most 

basic flight syste~s. The major disadvantage of conducting the 

study in the context of airline recurrent training was the set of 

restrictions imposed by the particular types of maneuvers that 

were part of the training syllabus. Since many of these ~aneuvers were different types of approaches, they lent themselves well to the purpose of the study. 

Twenty crews composed of captain, first officer, and flight 

engineer 2 flew approaches of varying difficulty using PNF callouts for half their approaches and SYNCALL callouts for the other half of their approaches. Table 7 lists the different approaches in the order that they were flown. Approaches 1, 2, and 

3 comprised "Set 1" and were always flown first. Approaches 4, 

5, and 5 comprised "Set 2" and were always flown last. For a 

given crew, Approach Set 1 was flown with one )f the callout sys- te~s (SYNCALL or PH?) and Approach Set 2 was flown with the opposite callout system. For Approach Set 1, both captains and first 

officers flew each of the three approaches (1, 2, and 3). For 

Approach Set 2, both captains and first officers flew the first 

two approaches (4 and 5), but only the captains flew the last approach (6), the single-engine approach and landing, in accordance 

with the airline procedures. Other than this discrepancy, the 

two approach sets were roughly balanced in that each began with a 

non-precision, manually flown approach. Next came an autopilot 

2. When scheduling did not permit the participation of a line 

flight engineer, a flight engineer instructor served as flight 

engineer to complete the three-man crew. 

17 
TABLE 7. APPROACHES FLO~N FOR SYNCALL EVALUATION 

PILOT FLYING APPROACH 

CAPTAIN 

LOCALIZER BACK COURSE 

SET 

2 DUAL LAND 

3 2-ENGIN~ ILS 

4 VOR RAW DATA 

SET 

5 SINGLE LAND 

2 

6 1-ENGINE LANDING 

FIRST OFFICER 

LOCALIZER BACK COURSE 

ILS APPROACH ONLY 

2-ENGINE ILS 

VOH RA~ DATA 

ILS APPROACH O~LY 

assisted, precision approach. Finally there was an approach with 

one or more engines out. A given approach as flown by a captain 

or a first officer was flown to similar but not identical 

mlnlmums due to the fact that captains are qualified to fly to 

lower minimums using more of the autopilot capability than are 

first officers. These small compromises in experimental design 

were deemed well worth the added realism that was gained by following airline procedures exactly. Table 3 shows the experimen- tal design in block form. 

For half the crews, the captain flew Approach Set first, 

followed by the first officer flying Approach Set 1. Then the 

captain flew approaches 4 and 5 of Approach Set 2 followed by the 

first officer flying these two approaches. The last approach to 

be flown was always approach 6, by the captain. For the other 

half of the crews, the first officer flew first for each approach 

set followed by the captain. For half the crews assigned to each 

of these orders, the SYNCALL callouts were used for Approach Set 

1 and the PN? callouts for Approach Set 2. For the other half of 

the crews, the PH? callouts were used for Approach Set 1 and the 

SYNCALL callouts for Approach Set 2. Appendix B contains detailed descriptions of the four resulting experimental orders for 

the approaches. 3 

3. Captain Walt Estridge and Instructor Pilot Colby Noyas were 

18 
----------------------------------------------------------------

TABLE 8. EXPERIMENTAL DESIGN 

APPROACH SET 

ORDER SET 1 SET 2 

SYNC ALL PNF 

CAPT FlO CAPT FlO 

2 SYNCALL PNF 

FlO CAPT FlO CAPT 

3 PNF SYNC ALL 

CAPT FlO CAPT FlO 

4 PNF SYNCALL 

FlO CAPT FlO CAPT 

DEVELOPMENT OF PERFORMANCE MEASURES 

The man-vehicle system composed of three flight crew members 

interacting with one another and with the multiple systems of a 

modern jet transport aircraft is extremely complex; the task of 

measuring the effectiveness of different types of approach callout systems within that environment shares many of these complexities. Thus measurements of system effectiveness must take 

these system co~plexities into account. The measurements must 

also have operational relevance. They must relate directly to 

airline operations in a way that allows the system users (pilots, 

airline management, manufacturers, government regulatory agencies) to base decisions on system implementation directly on the 

data. Special flight performance measures were devised for this 

study. These measures were computed from the raw data on aircraft altitude, airspeed, selected approach speed, localizer deviation in degrees, and glideslope deviation in degrees. This 

raw data was sampled by the simulation computer once every 0.2 s 

and recorded once every 0.8 s during the experimental runs. 

It was assumed that a pilot generally attempts to fly an aircraft within certain tolerances or windows and"that as long as 

performance remains within the desired window, no correction will 

instrumental in evolving the formal experimental design into a 

workable flight training syllabus that conformed to the airline 

training requirements. 

19 
be deemed necessary by the pilot. This assumption is supported 

by the observation that both FAA and airline company regulations 

are written with allowable tolerances. Also, pilots if asked 

will say they fly to tolerances rather than try to keep a needle 

exactly centered or positioned. Indirect support is also suggested by the need to incorporate an "indifference threshold" 

into control models of human operator performance in order to obtain an acceptable fit between model-predicted performance and 

observed human operator performance (3). If this assumption is 

correct, measures of mean deviation from an ideal value for, say, 

airspeed would have little operational relevance to what a pilot 

judges to be good performance. For example, if the pilot attempts to fly within +1-5 KTS of a selected approach speed, he 

will be satisfied if he remains within +1-2 KTS but will make a 

correction if airspeed deviates by +9 KTS from approach speed. 

It would seem more relevant to keep track of whether a pilot 

flies inside or outside the tolerance window for a given parameter (airspeed, sink rate, localizer deviation, or glideslope deviation) and, when outside tolerance, to monitor the average and 

maximum deviations from the boundary of the tolerance window -

not from the ideal value. This reasoning led to the choice of 

four computed operationally relevant measures of flight performance for the approach segment. 

Percent ~ ~ Qf Tolerance 

For each of the three approach segments (I, II, and III) the 

percentage of the total duration of the segment that 

outside the tolerance window for a given parameter was 

This was done by taking the number of 0.8-s samples for 

parameter was outside the tolerance window and dividing 

number of 0.8-s samples in the entire segment. 

Maximum Deviation 

was flown 

computed. 

which the 

by the 

The maximum deviation, the deviation with the largest absolute value, and the direction of that deviation (positive, negative, right, left, etc.) for each of the four parameters for each 

of the three approach segments was recorded. This value was then 

corrected to reflect the deviation from the boundary of the 

tolerance window by subtracting the absolute value of the tolerance. 

~ Positive Deviation 

Since deviations both right and left of the localizer, above 

and below glideslope, and faster or slower than approach speed 

could occur, mean deviations from the tolerance window boundary 

were computed for each direction. Due to the arbitrary assignment of signs (+ and -) to the direction of deviation by the 

simulation computer program, the term "positive" deviation is associated with real world directionality in an intuitively opposite manner. So, the arbitrary meanings of "positive" and "negative" are given for each of the four parameters below: 

20 
Airspeed 

Sink Rate 

Loc. Dev. 

GIS Dev. 

-21-

"Positive" Deviation 

lAS greater than V(APPR) 

Sink Rate greater than 

tolerance 

Deviation left of course 

Deviation below GIS 

"Negative" Deviation 

lAS less than V(APPR) 

Not applicable 

Deviation right of course 

Deviation above GIS 

The mean positive deviation was computed by taking only those 

0.8-s samples for which the parameter in question was out of 

tolerance in the "positive" direction, summing these deviations, 

and dividing by the number of out-of-tolerance samples. Thus, 

using an operational tolerance window of +1-8 KTS for lAS; if 

measured lAS was 158 KTS and selected V(APPR) was 141 KTS, then 

deviation from the positive tolerance window boundary of 149 KTS 

(141+8) is +9 KTS. For lAS samples of 158, 156, 150, 148, 149, 

144, and 140 KTS, the mean positive deviation would be (9+7+1)/3· 

or +5.7 KTS. Expressed in terms of deviation from the selected 

approach speed, the mean positive deviation would be +13.7 KTS 

(5.7+8). 

~ Negative Devia~ 

Mean negative deviation from the tolerance window boundary 

for each flight parameter for each approach segment was computed 

for deviations in the "negative" direction as defined above. 

This was done in the same manner as the computation for "positive" deviations. However, this measure was not applicable for 

sink rate since there is no minimum acceptable sink rate for an 

approach. Thus the computed "negative" deviations for sink rate 

were disregarded. 

Flight Performance Parameters 

As stated above, four parameters were measured for deviation 

outside allowable tolerance windows: airspeed, sink rate, localizer displacement, and glideslope displacement. These were 

chosen since they are the parameters which the pilot-not-flying 

and the flight engineer are required to monitor during the approach; and a comparison of performance on these four parameters 

as a function of callout system would give a measure of comparative effectiveness of the two callout systems in keeping the aircraft within allowable tolerance windows. . 

In choosing the operational tolerance window for each parameter, it was necessary to consider the discrepancies between the 

raw data in the Central Air Data Computer (CADC) (simulated by 

the simulation computer) and the visual indicators used by the 

pilots to determine current airspeed, sink rate, localizer devia21 
tion, and glideslope deviation. The pilots could not be expected 

to correct excessive airspeed as ~easured by the CADC, for example, if their airspeed indicators showed them to be within tolerance. Thus an operational tolerance for each flight parameter 

was derived by adding its operating manual ("book") tolerance 

(cf. Figure 3) and its "indicator" tolerance. 

Operational Airspeed Tolerance 

The operating manual for this air carrier calls for lAS to be 

within +/-5 KTS of selected approach speed (V(APPR» once landing 

flaps have been extended. The permissible error between the digital autothrottle command setting, which controls the position 

of the orange bug on the airspeed indicator, and the actual position of the orange bug is +/-3 KTS. Thus the airspeed deviation 

as read by the pilot-not-flying and the flight engineer as they 

monitor airspeed from the airspeed indicator could appear as a 5 

KT deviation when in fact it was an g ~T deviation according to 

data in the CADC. Therefore, so as to count as out of tolerance 

only those airspeed deviations which appeared as deviations to 

the pilot-not-flying and the flight engineer, the book tolerance 

of 5 KTS and the indicator tolerance of 3 KTS were added together 

to produce an operational tolerance of +/-3 KTS. Figure 3 sho~s 

this pictorially. 

Operational Sink Rate Tolerance 

The book tolerance for sink rate varies with altitude above 

field (AFL). From 1000 feet AFL to 300 feet AFL, sink rate is 

not to exceed 1000 feet per minute (FPM). Below 300 feet AFL, it 

should not exceed 700 FP~. There is a considerable time lag, approximately 2 to 3 s, between instantaneous vertical speed and 

the indication of vertical speed shown on a vertical speed indicator (VSI). 4 Thus the pilots visual indication of sink rate is 

2 to 3 s old. The recorded sink rate data was taken directly 

from the simulation computer and was thus a derived instantaneous 

vertical speed. So, an indicator tolerance was needed for this 

data that would compensate for the time lag. The computation of 

this tolerance involved the average standard deviation of a random sample of 2.4 s intervals of sink rate. 

Sink rate was averaged over 40 sets of three consecutive 

4. Although Instantaneous Vertical Speed Indicators (IV3I's) 

are installed in the wide-body jet used in this study, a 2 to 3 s 

time lag is deSigned into the indicators to to smooth the data 

going to the Central Air Data Computers (CADC's), which in turn 

drive the vertical speed instruments and altimeters in the cockpit. This is done to prevent momentary "opposite direction" 

vertical speed and altitude indications which would otherwise occur following changes in aircraft pitch. Pitch changes produce a 

momentary change in local pressure distribution at the static 

ports which results in a driving pressure opposite to that needed 

to give a true indication of direction of vertical speed. 

22 
OPERATIONAL TOLERANCE: AIRSPEED 

-8 kt 

BOOK TOLERANCE ±5 kt 

+ INDICATOR TOLERANCE ±3 kt 

OPERATIONAL TOLERANCE ±8 kt 

Figure 3. - Jperutional Tolcranc2: Airspeed. 

23 
OPERATIONAL TOLERANCE: SINK RATE 

BELOW 300 AFL: 974 fpm 

1000 TO 300 AFL: 1174 fpm 

BOOK TOLERANCE 

+ INDICATOR TOLERANCE 

OPERATIONAL TOLERANCE 

1000 

TO 300 AFL 

1000 fpm 

174 fpm 

1174 fpm 

Figure 4. - Jparational Tolerance: Sin~ Rate. 

24 

BELOW 

300 AFL 

700 fpm 

174 fpm 

874 fpm 
0.8-s samples of sink rate, i.e. over 2.4-s intervals. The standard deviation for each of these sets of three samples was computed. Then the mean and standard deviation of these 40 standard 

deviations was computed and these values added together. The 

original 40 sets of three 0.8-s samples of sink rate were taken 

from 10 of the 20 crews, randomly selected. For each of the ten 

crews selected, a 2.4-s interval of sink rate was taken for each 

of the four non-precision approaches (LOC BK CRS and VOR flown by 

Captain and by First Officer) flown by that crew. The altitude 

at which each 2.4 -s interval started was randomly chosen from 

the range 1 - 1000 feet AFL. 

The mean and standard deviation of the 40 resulting standard 

deviations were 51 feet and 111 feet respec~ively. Their sum of 

114 feet was used as the indicator tolerance for sink rate. Adding 114 feet to the book tolerances of 1000 FPM and 100 FPM 

resulted in operational tolerances of: 

1000 - 300 AFL 

BELOW 300 AFL 

1114 FPM 

814 FPM 

Figure 4 shows this pictorially. 

Operational Localizer Devia~ion Tolerance 

The book tolerance for localizer deviations is +1-1/3 dot on 

the indicator scale, which is equal to +1-0.42 deg displacement 

from the center of the localizer course. The allowable tolerance 

for the localizer indicator needle is expressed in terms of 

"needlewidths". The needle referred to is the indicator needle 

which is driven right or left of center depending on whether the 

aircraft is left or right of the localizer course (valid only for 

the "front course" direction) The tolerance is +1-1 needlewidth. 

To convert needlewidth to dots - the unit used by the pilots 

the needlewidth and the distance between center and full deflection to one side were measured on an actual Course Direction Indicator (CDI) which displays localizer and glideslope information. Accordingly, 1 needlewidth measured 0.22 dots or 0.28 deg. 

This indicator tolerance was added to the book tolerance of 

+1-1/3 dot (:+1-0.42 deg) to give an operational tolerance of 

+1-0.56 dots (:+/-0.10 deg). In round numbers, this operational 

tolerance is just slightly greater than +1-1/2 dot. See Figure 

5 • 

Operational Tolerance for Glideslope Deviations 

The indicator tolerance for glideslope deviation was computed 

1n a manner analogous to that for localizer deviation. This 

resulted in the +1-1 needlewidth tolerance being equal to +1-0.18 

dots (:+/-0.01 deg). The book tolerance of +1-1 dot (:+1-0.25 

deg) plus the indicator tolerance gives an operational tolerance 

of +1-1.28 dots (:+1-0.32 deg), or approximately 1 1/4 dots. See 

Figure 6. 

25 
OPERATIONAL TOLERANCE: LOCALIZER 

LEFT 

0.70 

= 0.56 DOT 

BOOK TOLERANCE 

+ INDICATOR TOLERANCE 

OPERATIONAL TOLERANCE 

RIGHT 

0.70 

= 0.56 DOT 

1/3 DOT 

1 NEEDLEWIDTH 

1/2+ DOT 

Fi~ure J. - Jperationnl Tolerance: Localizer. 

26 
~ 

-:J 

OPERATIONAL TOLERANCE: GLIDESLOPE 

~~o '/,;~ o 

'~o 

O~ 

lJE:lOII/t 0.3<, 0/i 

7'<8 DOrS 

BOOK TOLERANCE 

+ INDICATOR TOLERANCE 

OPERATIONAL TOLERANCE 

~ 

.~c9 

~b /'~ 

1 DOT 

1 NEEDLEWIDTH 

1% DOTS 

Figure 5. - Jperational Toleranc~: Glideslup~. 
MEASURES OF PILOT JUDGMENTS OF SYSTEM PERFORMANCE 

The purpose of administering a formal post-experiment debriefing to the pilots was to obtain in a systematic and objective fashion the pilots' subjective responses to the two callout 

systems. It was expected that experienced line pilots would be 

able to answer questions about potential advantages and disadvantages of the two systems that could not be otherwise measured by 

flight performance parameters. By presenting each pilot with an 

identical set of questions and by designing the questions so that 

possible answers were mutually exclusive and yet covered the entire range of possible answers by the inclusion of an option for 

an "other" response, it was possible to collect data that could 

be submitted to non-parametric statistical tests for significance. 

Design Qf Pilot ~-Briefing Materials 

A copy of the de-briefing form exactly as presented to the 

pilots is in Appendix C. Preference grids, a measure that had 

been successfully used with airline pilots in a study of line pilot preferences for design of cockpit warning systems (4) were 

used to study the effects of type of approach conditions (Night 

VFR, Day VFR, IFR to Weather Minimums, or Abnormal/Emergency) on 

whether or not pilots would want a particular callout to be made 

by the Pilot-not-Flying and/or by SYNCALL. A 7-point scale from 

"unsafe" to "highly desirable" with "no preference" in the middle 

was used to compare the present PNF system, the SYNC ALL system as 

configured for the study, the PNF system as reconfigured according to the pilot's own responses on the preference grid concerning desirable callouts, and the SYNC ALL system as reconfigured 

according to the pilot's own preference grid responses. Pilots 

were asked to rate the intelligibility of four types of speech 

they had heard in the simulator on a scale of 0% to 100%. The 

four types of speech were 1)SYNCALL synthesized speech, 2) Terminal Radar Control (TRACON) ATC human voice messages, 3) Tower 

ATC human voice messages, and 4) Pilot-not Flying human voice 

messages (callouts and other crew voice communications). The 

last item was a list of possible modifications to the SYNCALL 

system as configured for the study. Pilots were asked to rate 

each modification on a scale of 1) Highly Desirable, 2) Desirable, 3) No preference, 4) Undesirable, or 5) Unsafe. At the end 

of the list were blank lines with instructions to fill in additional modifications the pilots thought they would want. They 

were asked to rate their own suggested modifications using the 

sa~e 5-point scale. This concluded the main debriefing form. 

A second, optional form was used to obtain additional, more 

detailed information about the individual callouts from those pilots who were interested enough and willing to spend additional 

time. This form consisted of a set of semantic differential 

scales (5) chosen for their relevance to cockpit flight information systems. Several of these scales had been previously used 

28 
in the study on line pilot preferences for cockpit warning system 

design (4) with consistent results. Others were added specifically for this study. Table 9 lists the semantic differential 

scales and Table 10 lists the set of callouts by each system that 

were measured with these scales. For purposes of comparison, the 

current GPWS "Whoop, Whoop. Pull up!" and "Glideslope" voice messages were included in the set of callouts to be rated on each of 

the semantic differential scales. These items were included because they were then and still are the only examples of electronically generated speech that have been experienced by all airline 

pilots. Both the de-briefing form and the optional semantic differential form were designed to be self-administered. Finally, 

to preserve annonymity, no names or other personal information 

were requested on either form. 

TABLE 9. SEMANTIC DIFFERENTIAL SCALES 

GOOD •••••••••••••••••••••••••••••• BAD 

WORTHLESS •••••••••••••••••••• VALUABLE 

LOUD ••••••••••••••••••••••••••••• SOFT 

ACTIVE •••••••••••••••••••••••• PASS: TE 

NOISy ••••••••••••••••••••••••••• QUIET 

SAFE •••••••••••••••••••••••• DANGEROUS 

ALERTING •••••••••••••••• IMPERCEPTIBLE 

ANNOYING ••••••••••••••••••••• SOOTHING 

HELPFUL ••••••••••••••••••• DETRIMENTAL 

DEPENDABLE ••••••••••••••••• UNRELIABLE 

DISRUPTIVE ••••••••••••••• COORDINATING 

29 
----------------------------------------------------------------

TABLE 10. TYPES OF· CALLOUTS MEASURED WITH SEMANTIC DIFFERENTIAL 

AS SPOKEN BY PILOT-NOT-FLYING AND BY SYNCALL 

1000-FOOT CALLOUT 

TEN-FOOT CALLOUTS 

HUNDRED FOOT CALLOUTS 

GIS & LOCALIZER INTEaCEPT 

FINAL APPROACH FIX & CROSSING ALTITUDE 

AIRSPEED DEVIATIONS 

SINK RATE DEVIATIONS 

GIS & LOCALIZER DEVIATIONS 

DECISION HEIGHT, MDA, & 100 ABOVE MDA 

AS SPOKEN BY GPWS 

"WHOOP WHOOP PULL UP" 

"GLIDESLOPE" 

PROCEDURE 

Each crew received a taped briefing of approximately 

5-minutes len3th explaining the scope and purpose of the experiment. If all crew members decided to participate, they were 

given a taped description of the SYNCALL system compared to the 

PNF system accompanied by diagrams on projected transparencies. 

General questions about systems operation were encouraged. Next, 

the pilots heard a taped introductory passage by the speech synthesizer to famil-iarize them with the electronic voice quality 

and "accent" of the speech. This was followed by a recording of 

each type of approach callout in the SYNCALL system with each 

callout pronounced twice. To speed learning of the synthesizer 

accent, the printed text of the speech was displayed concurrently. Tables 11 (following) and 6 (in section on Voice Message 

Design) are a reproduction of the printed displays. 

After the 15-minute briefing, the instructor pilot conducted 

the normal pre-simulation briefing on the flight maneuvers to be 

practiced. This was followed by four hours of recurrent simulator training in accordance with the appropriate set of experimental conditions for each crew. The first hour was spent on other 

maneuvers and served as a "warm-up" period before flying the experimental approaches. A break of 15 minutes was scheduled 

between Approach Set 1 and Approach Set 2 in accordance with the 

normal training breaks used. To simulate the potential mutual 

30 
TABLE 11. INTRODUCTORY PASSAGE BY SPEECH SYNTHESIZER PRESENTED 

TO PILOTS FOR FAMILARIZATION WITH SYNC ALL VOICE 

YOU ARE ABOUT TO HEAR THE SYNCALL SYSTEM SAY 

EACH OF THE APPROACH CALLOUTS SO YOU CAN GET 

FAMILIAR WITH THE SOUND OF THE VOICE AND WITH 

EACH OF THE CALLOUTS. THE CALLOUTS ARE LISTED 

IN ORDER ON THE SLIDE SO YOU CAN LOOK AT THEM 

AS YOU HEAR THEM. EACH ~ALLOUT WILL BE SAID TWO 

TIMES WITH A SLIGHT PAUSE BETWEEN REPETITIONS. 

masking of concurrent speech messages from Air Traffic Control 

(ATC) and the SYNCALL callouts, a pre-recorded set of Approach 

and Tower transmissions, addressed to other aircraft, were played 

over the cockpit headsets to the pilots. The pilot instructor 

issued clearances in ATC phraseology to the crew, and they read 

back their' clearances with the cockpit microphone as they would 

have done in the actual aircraft. The author controlled other 

aircraft ATC message presentation on different simulated ATC frequencies to assure that the simulated ATC chatter was appropriate 

for the runway in use and the simulator's position in the airspace. The ATC tapes were edited from actual ATC communications 

recorded at the airport that was simulated for the study. 

Flight performance data was collected for each approach by 

the simulation computer. Data collection began at 3000 feet AFL 

and continued until either the initiation of a go-around on a 

missed approach or the touchdown ona landing. The following 

flight parameters were sampled once every 0.8 s: 

1) Indicated Airspeed (lAS) 

2) Selected Approach Speed (from ATS command airspeed) 

3) Altitude Mean Sea Level (MSL) 

4) Glideslope deviation 

5) Localizer deviation 

6) Vertical Speed 

7) Elapsed Time 

The times of occurrence of each callout together with actual 

values that triggered any deviation callouts were also recorded. 

The cockpit audio including crew conversation, ATC, approach ca1-

louts, and cockpit noise was recorded on audio tape. 

After the simulator session, each pilot completed the debriefing form and discussed the experiment with the author. 

Those pilots who were interested also completed the optional semantic differential form. Many took it with them and mailed the 

completed form back. Pilots were offered a copy of the final re31 
port, when published. 

32 
RESULTS 

The results will be presented in sectio~s according to the 

individual measures (flight performance, system reliability, system interference with other systems) that were used to assess 

callout system effectiveness. Because of the large number of 

different measures used, there will be some discussion of each of 

the major types of results in the respective results sections. 

Then the results of the separate measures will be compared for 

consistency. Finally, a general discussion will treat the 

results from different measures as a whole. 

Flight Performance Results 

Analysis of Variance (AOV) was used to test for statistically significant effects of 1) type of callout system, 2) pilot 

flying, 3) order of pilot flying, and 4) type of approach. 

Analysis of variance was chosen as a means of testing the data to 

determine the probability that any observed differences in the 

data associated with each of the four effects above could be due 

to chance rather than be caused by those effects themselves. 

Airspeed, sink rate, localizer deviations, and glideslope deviations were analyzed as appropriate to each approach type for each 

of the four measures: (1) percent time out of tolerance, (2) maximum deviation beyond tolerance, (3) maximum "positive" deviation, and (4) maximum "negative" deviation. 

First, all five approaches (1-engine approach & landing not 

included here) were compared for the flight parameters they had 

in common, namely airspeed and sink rate, for each of the four 

measures listed above. The final approach was divided into four 

segments for analysis, with the assumption that these segments 

might be flown differently and that the operational tolerances 

for the different segments would be different. The four segments 

were: 

I: 

II: 

III: 

IV: 

Final Approach Fix (FAF) to 1000 AFL 

1000 AFL to 500 AFL 

500 AFL to DH or MDA (depending on 'type approach) 

DH or MDA to Landing or Go-around 

(depending on how approach was terminated) 

Due to large differences among the five types of approach in 

Segment I, the AOV's were performed only on the data for Segments 

II, III, and IV. While a few callouts were made by SYNCALL or 

the Pilot-not-Flying (PNF) during Segment I (outer marker with 

AFL crossing altitude, localizer deViations, and glideslope deviations), most of the callouts occur from 1000 AFL on down. Thus 

any differences in performance due to the type of callout system 

would be expected to occur in segments II, III, and IV. 

33 
Figures 7 and 8 show the different 4-way AOV's that were per-

-formed. The comparison of all five approaches was done with a 

2x2x2x3 mixed design for approaches 1, 2i and 3 and with a 

2x2x2x2 mixed design for approaches 4 and 5. The reason for the 

division of the analysis into two parts: Approaches 1, 2, and 3 

versus Approaches 4 and 5 was that those pilots who flew one set 

of approaches with one callout system flew the other set of approaches with the other callout system. Analysis of all five approaches in a 2x2x2x5 mixed design would have resulted in an unbalanced design. There were thus 24 four-way AOV's (3 segments x 

4 ~easures x 2 parameters) for approaches 1, 2, and 3 and another 

24 four-way AOV's for approaches 4 and 5. 

Analyses of All Approaches Overall 

Aoproach Segment ~: 1QQQ AEL ~ 5QQ AlL 

Between 1000 and 500 feet AFL the only factor with significant effects on airspeed and sink rate performance was type of 

approach. Significant differences (p<O.001) were obtained for 

sink rate performance for both sets of approaches (1,2,3) and 

(4,5) for the following measures: 

Percent time out of tolerance 

(Operating tolerance = -1174 FPM) 

Maximum deviation beyond tolerance 

Mean "positive" deviation beyond operating tolerance 

Significant differences (p<0.001) were obtained for airspeed performance only for the first set of approaches (1,2,3). The following measures resulted in significant differences: 

Percent time out of tolerance (Oper Tol = +1-8 KTS) 

Maximum deviation from Approach Speed 

Mean "positive" deviation from Approach Speed 

The term "mean negative deviation" as applied to the data for 

sink rate and airspeed deserves some comment. During approach 

and landing, only sink rates greater than tolerance are of concern. "Negative deviation", i.e. rate of climb, has no upper 

limit and was therefore not analyzed. 

For airspeed, there were no instances for any of the 20 crews 

for either pilot for any of the approaches for which airspeed was 

below the minimum operating tolerance of -8 KTS. Since all data 

points were thus equal to 0, analysis of variance was neither appropriate nor necessary. 

Since the results for mean "positive" deviation were analogous to those for percent time out of tolerance and for maximum 

deviation, Table 12 presents means (N=40 pilots) just for percent 

time out of tolerance and means fOr maximum sink rate deviation 

and Table 13 presents similar data for airspeed for Segment II. 

34 
(!) 

z 

-

> 

...I 

U. 

I0 

...I 

c.. 

ANALYSIS OF VARIANCE DESIGN FOR 

APPROACH SET 1 

APPROACH APPROACH 

Ic.. 1 LOC BK CRS 1 LOC BK CRS 

<C 

(.) 2 AUTO LAND 2 AUTOLAND 

3 2 ENG ILS 32ENGILS 

APPROACH APPROACH 

Q 1 LOC BK CRS 1 LOC BK CRS 

u. 2 AUTOLAND 2 AUTOLAND 

3 2 ENG ILS 3 2 ENG ILS 

SYNCALL PNF 

-.. -----CALLOUT SYSTEM ---~ .. ~ 

Figure 7. - \nalysis of Variance Jesi3n for Approach Sat I. 

35 
e" 

2 

-

> ..J 

LL 

I0 

..J 

Q. 

ANALYSIS OF VARIANCE DESIGN FOR 

APPROACH SET 2 

I- APPROACH APPROACH 

Q. 4 VOR 4 VOR « 

(.) 5 AUTOLAND 5 AUTO LAND 

Q APPROACH APPROACH 

4 VOR 4 VOR LL 

5 AUTOLAND 5 AUTOLAND 

SYNCALL PNF 

-4 ___ ---CAL LOUT SYSTEM ------11 .. _ 

FiJur~ 3. - ~nalysis of Variance Desi~~ for Approach Sat II. 

36 
TABLE 12. SIN' RATE P~RFORMANCB FOR ALL APpaOACHES DUHING 

3SGMENT II: 1003 FT AFL TO 500 FT AFL 

N = 40 PILOTS 

MEAN PERCENT TIME ~EAN MAXIMUM 

TYPE APPHOACH our OF TOLERANCE DEVIATION 

LOCALIZER BACK COURSE 39 ~ 1483 FPl1 

2 DUAL LAND / ILS APPR ONLY 1 % 1 '3' FPH 

3 2-BNGINE ILS 1 .~ 1 1 '78 F Pl'1 

4 VOR HAW DATA 30 ~ 1405 F Pl'l 

5 SINGL LAND / ILS APPH ONLt % 1174 FPI1 

Note: Tne were no statistically significant differences due to 

Pilot Flying, Order, or Type of Callout System During Segment II 

TABLE 13. AIRSPEED PERFORMANCE FOR ALL APPHOACHES 

DUlUNG SEGMENT II:' 1000 ~T AFL TO 500 FT AFL 

N = 40 PILOTS 

MEAN PERCENT TIME MEAN MAXIMUM 

TtPE APPROACH our OF TOLERANCE DEVIATION 

LOCALIZER BACK COURSE 34 % + 8 KTS 

2 DUAL LAND / ILS APPR ONLY 44 % + a KTS 

3 2-ENGINE ILS 16 % + 5 KTS 

4 VOR RAW DATA 41 % + 10 KTS 

5 SINGL LAND / ILS APPR ONLY 39 ~ + 12 KTS 

Note: Tnere were no statistically significant differences for 

Pilot Flying, Order, or Type of Callout System during Segment II 

37 
It is not surprising that sink rate performance on 

autopilot-assisted ILS and Category II Autoland approaches was 

better than for the manually flown non-precisio~ approaches. It 

is however surprising to note that airspeed performance between 

1000 and 500 feet AFL was worse for the Autoland approaches than 

for the manually flown approaches. Possibly the pilot anticipates needs for slight adjustments in pitch and power as the 

landing flaps and gear extend and is in this way aole to track 

desired approach speed better than the autothrottle system which 

can only react after airspeed deviates from programmed tolerances. No factors other than type of approach were significant 

for Segment II. 

A P P rca c h Se gm e n t .Ill: ~ A..E1. 1..Q llii. .ru::. 1iILA 

Between 500 feet AFL and DH or MOA (depending on type of approach) there were significant effects for type of approach 

(p<0.OJ1), callout system (p<O.05), and pilot flying (p<O.OS). 

Table 14 (a-d) shows means (N=40 pilots) for the sink rate and 

airspeed data for Segment III. The only significant effect of 

callout system was for maximum sink rate deviation for approaches 

4 and S with SYNCALL approaches resulting in no instances of sink 

rate exceeding the maximum operating tolerance of -1114 fpm and 

the PNF callout system resulting in a mean maximum deviation of 

21 FPH beyond the operating tolerance of -1114 fpm. No significant differences in sink rate due to callout system occurred for 

Approaches 1, 2, and 3. Pilot flying had a significant effect 

(p<O.OS) on percent time out of tolerance (+1-8 KTS) for airspeed 

for approaches 1, 2, and 3. Captains on the average were out of 

tolerance on airspeed 22% of the time while First Officers were 

out of tolerance only 10% of the time. As with data for the 

previous segment, the type of approach flown had a significant 

effect (p<O.O~) on both airspeed and sink rate. In this segment, between 500 AFL and DH or MOA, performance was better for 

the autopilot-assisted ILS approaches than for the manually flown 

non-precision approaches. 

38 
TABLE 14a. AIRSPEED PERFORMANCE FOR SET 1 APPROACHES DURING 

SEGMENT III: 500 FT AFL TO DH OR MDA 

N = 40 PILOTS 

FACTOR MEAN PERCENT TIME MEAN MAXIMUM 

OUT OF TOLERANCE DEVIATION 

CALLOUT SYNCALL 

SYSTEM PNF 

PILOT WHO CAPTAIN 

FLEW FIRST FIRST OFFICER 

PILOT CAPTAIN 

FLYING FIRST OFFICER 

APPROACH 1 LOCALIZER BACK COURSE 

TYPE 2 DUAL LAND/ILS APPR ONLY 

3 ·2-ENGINE ILS 

13 % 

20 % 

15 % 

18 1-

22 r. 

10 % 

34 % 

3 % 

13 % 

+ 3 KTS 

+ 4 KTS 

+ 3 KTS + 4 KTS 

+ 4 KTS + 3 KTS 

+ 6 KTS + 1 KT 

+ 3 KTS 

----------------------------------------------------------------

TABLE 14b. AIRSPEED PERFORMANCE FOR SET 2 APPROACHES DURING 

SEGMENT III: 500 FEET AFL TO DH OR MDA 

N = 40 PILOTS 

FACTOR HEAN PERCENT TIME MEAN MAXIMUM 

OUT OF TOLERANCE DEVIATION 

CALLOUT SYNCALL 20 % + 3 KTS 

SYSTEM PNF 12 % + 3 KTS 

PILOT WHO CAPTAIN 19 % + 3 KTS 

FLEW FIRST FIRST OFFICER 14 J, + 3 KTS 

PILOT CAPTAIN 13 % + 2 KTS 

FLYING FIRST OFFICER 19 % + 5 KTS 

APPROACH 4 VOR RAW DATA 31 % + 5 KTS 

TYPE 5 SINGL LAND/ILS APPR ONLY 2 % + 2 KTS 

-----------------------------------------------------------------

39 
----------------------------------------------------------------

TABLE 14 c • SINK RATE PERFORMANCE FOR SET 1 APPROACHES DURING 

SEGMENT II I: 500 FEET AFL TO DH OR MDA 

N = 40 PILOTS 

FACTOR MEAN PERCENT TIME HEAN HAXIMUM 

OUT OF TOLERANCE DEVIATION 

CALLOUT SYNCALL 6 % 56 FPM 

SYSTE:M PNF 6 % 51 FPM 

PILOT WHO CAPTAIN 5 % 44 FPM 

FLEW FIRST FIRST OFFICER 7 % 53 FPM 

PILOT CAPTAIN 7 j, 70 FPM 

FLYING FIRST OFFICER 5 'f, 37 FPM 

APPROACH 1 LOCALIZER BACK COURSE 1 1 % 106 FPl'1 

TYPE 2 DUAL LAND/ILS APPR ONLY a J, a FPM 

3 2-ENGINE ILS 5 '/. 58 FPM 

TABLE 14d. SINK RATE PERFORMANCE FOR SET 2 APPROACHES DURING 

SEGMENT III: 500 FEET AFL TO DH OR MDA 

N = 40 PILOTS 

FACTOR MEAN PERCENT TIME MEAN MAXIMUM 

OUT OF TOLERANCE DE:VIATION 

CALLOUT SYNCALL a % a FPM 

SYSTEM PUF 6 % 27 FPM 

PILOT WHO CAPTAIN 6 % 14 FPM 

FLEW FIRST FIRST OFFICER 1 % 1 3 FPl'1 

PILOT CAPTAIN 3 % 16 FPl'1 

FLYING FIRST OFFICER 4 % 1 1 FPM 

APPROACH 4 VOR RAW DATA 6 % 23 FPM 

TYPE 5 SINGL LAND/ILS APPR ONLY 1 % 3 FPM 

----------------------------------------------------------------

Note: For sink rate, Maximum Deviation refers to the deviation 

BEYOND the operation tolerance of 1174 fpm. 

----------------------------------------------------------------

40 
Approach Segment~: QR ~ MQA ~ Landing ~ YQ-Around 

Between DH or MDA and Landing or Go-Around, there were also 

significant effects for type of approach (p(0.05) and callout 

system (p(0.05). Table 15 (a-d) shows the means (N=40 pilots) 

for the sink rate and airspeed performance data for Segment IV. 

In this segment, the only significant difference resulting from 

type of callout system was for percent time out of tolerance for 

airspeed, and the difference was in the opposite direction from 

that which was expected - performance was worse with SYNCALL (35% 

time out of tolerance) than with PNF callouts (21% time out of 

tolerance). This reversal occurred only for the second set of 

approaches (Numbers 4 and 5). One possible explanation is that 

the crews who were using PNF callouts for these approaches had 

already experienced SYNC ALL for the first set of approaches and 

may have experienced some learning transfer to fly tighter tolerances with SYNCALL which carried over into their performance 

without SYNC ALL in the second set of approaches. Further examination of the raw'data would be needed to determine whether such 

transfer effects may have actually occurred. It should be noted 

that airspeed performance was generally worse for the second set 

of approaches than for the first, possibly due to fatigue. All 

experimental runs were flown between 20:00 and 24:00 Central 

Time, causing the second set of approaches to be flown between 

22:30 and 24:00. 

41 
----------------------------------------------------------------

TABLE 15a. AIRSPEED PERFORMANCE FOR SET 1 APPROACHES DURING 

SEGMENT IV: DH OR MDA TO LANDING OR GO-AROUND 

N = 40 PILOTS 

FACTOR MEAN PERCENT TIME MEAN MAXIMU~ 

OUT OF TOLERANCE DEVIATION 

CALLOUT SYNCALL 

SYSTEt1 PNF 

PILOT WHO CAPTAIN 

FLEW FIRST FIRST OFFICER 

APPROACH 1 LOCALIZEWR BACK COURSE 

TYPE 2 DUAL LAND/ILS APPR ONLY 

3 2-ENGINE ILS 

13 

21 

16 

1 1 

23 

4 

25 

, + 4 KTS 

% + 4 KTS 

% + 4 KTS 

% + 3 KTS 

% + 4 KTS 

% + 2 KTS 

% + 6 KTS 

TABLE 15b. AIRSPEED PERFORMANCE FOR SET 2 APPROACHES DURING 

SEGMENT IV: DH OR MDA TO LANDING OR GO-AROUND 

N = 40 PILOTS 

FACTOR MEAN PERCENT TIME MEAN MAXIMUM 

OUT OF TOLERANCE DEVIATION 

CALLOUT SYNCALL 35 % + 14 KTS 

SYSTEM PNF 21 % + 1 1 KTS 

PILOT WHO CAPTAIN 25 ." + 1 1 KTS 

FLEW FIRST FIRST OFFICER 25 % + 1 3 KTS 

APPROACH 4 VOR RAW DATA 23 % + 9 KTS 

TYPE 5 SINGL LANDIILS APPR ONLY 36 'i + 16 KTS ,. 

Summary of ~-~ AQY'~ 

While there were statistical IV significant differences for 

type of callout system, the absolute magnitude of the differences 

was generally small. This is especially true for the sink rate 

data. Also, the differences due to type of approach were of far 

greater magnitude than those due to either the type of callout 

system or the pilot flying. Order of pilot flying had no significant effects on sink rate or airspeed performance. 

42 
TABLE 15c. SINK RATE PERFORMANCE FOR SET 1 APPROACHES DURING 

SEGMENT IV: DH OR MDA TO LANDING OR GO-AROUND 

N = 40 PILOTS 

FACTOR MEAN PERCENT TIME MEAN MAXIMUM 

OUT OF TOLERANCE DEVIATION 

CALLOUT SYNCALL 8 % 50 FPM 

SYSTEM PNF 3 % 59 FPH 

PILOT WHO CAPTAIN 8 % 55 FPM 

FLEW FIRST FIRST OFFICER 8 % 79 FP!1 

APPROACH 1 LOCALIZER BACK COURSE 13 % 1 18 F P:-i 

TYPE 2 DUAL LAND/ILS APPR ONLY 2 ~ 16 FPH 

3 2-ENGINE ILS 10 % 53 FPH 

TABLE 15d. SINK RATE PERFORMANCE FOfi SET 2 APPROACHES DURING 

SEGMENT IV: DH OR MDA TO LANDING OR GO-AROUND 

N = 40 PILOTS 

FACTOR MEAN PERCENT TIME MEAN MAXIMUM 

OUT OF TOLERANCE DEVIATION 

CALLOUT SYNCALL 5 % 

SYSTEM PNF 9 S 

PILOT WHO CAPTAIN 6 % 

FLEW FIRST FIRST OFFICER 9 % 

APPROACH 4 VOR RAW DATA 9 % 

TYPE 5 SINGL LAND/ILS APPR ONLY 2 % 

Analyses of Individual Approaches 

129 FPM 

65 FPM 

112 FPM 

117 FPI-i 

166 FPM 

2 FPM 

Because of the large differences in performance as a function 

of type of approach and because the major purpose of the study 

was to measure manual flight performance, fu~ther AOV's were performed on individual approaches. The approaches chosen for this 

further analysis were the non-precision approaches, the 2-Engine 

ILS, and (for Captains only) the 1-Engine Landing. 

43 
Figure 9 shows the different AOV's that were performed on the individual approaches. The design for analysis of individual approaches was a 2x2x2 across subjects AOV. 

All together this resulted in 3 segments x 4 measures x 4 

parameters for each of the 3 precision approaches plus 3 segments 

x 4 measures x 3 parameters for the Localizer Back Course approach plus 3 segments x 4 measures x 2 parameters for the VOR 

approach for a total of 204 3-way AOV's. 

When individual approaches were analyzed, Type of Callout 

System (SYNCALL or PNF) was the only factor that had statistically significant effects on flight performance. Furthermore, significant differences in flight performance resulted only for the 

non-precision approaches (Localizer Back Course and VOR) only in 

the lower altitude segments (III: 500 AFL to MDA; and IV: MDA to 

Landing/Go-Around) and only for airspeed and sink rate performance - not course tracking performance (Localizer and Glideslope 

where applicable). In each case, flight performance was better 

with SYNC ALL than with PNF callouts. 

Airspeed Deviations During Localizer Back Course Approach 

Table 16 shows airspeed performance for Segment IV (MDA to 

Landing/Go-Around) for the Localizer Back Course Approach for 

each callout system. There were 19 such approaches flown by either a Captain or a First Officer for which SYNC ALL was used and 

11 approaches for which PNF callouts were used. 5 Percent time 

out of tolerance for the SYNCALL approaches was 13% compared to 

34% time out of tolerance for the PNF approaches. This difference was significant (F=4.30, df=1,28, p<0.05) Similarly, fewer 

approaches were flown out of tolerance at all (5) with SYNCALL 

compared to PNF (12). 

To discover in more detail what was occurring with airspeed 

performance during these approaches, a tally was taken to obtain 

the number of approaches that were in tolerance and outside 

tolerance at an earlier pOint in the approach (500 feet) when the 

normal 500 foot callout with airspeed and sink rate is made and 

again at the beginning of the segment for which performance 

differences occurred (MDA). Tables 11 a and b show number of approaches in and out of tolerance for SYNC ALL and PNF callouts at 

50n AFL and the corresponding data taken at MDA. A Chi-Square 

analysis resulted in no significant correlation between callout 

system and airspeed performance at 500 AFL (X(2)=0.07, df=1, 

p)0.10). But the Cni-Square for the data at MOA was significant 

(X(2)=5.55, df=1, p<0.02). Thus, while there was no difference 

5. If the crew on a given approach initiated a go-around before reaching MOA (as sometimes happened) then there was no data 

collected for that approach for that crew. This is the reason 

for the unequal N number of approaches for the two callout systems. 

44 
DESIGN FOR ANALYSIS OF VARIANCE FOR 

INDIVIDUAL APPROACHES 

I0. 

<C 

(.) 

(!) 

2: 

-

> ..J 

U. 

I0 

..J 

-

0. 

g 

U. 

SyNCALL PNF 

~ CALLOUT SYSTEM ~ 

~iJure J. - JesiJn for Analysis of Variance for Individu3l A~-

pro3chcs. 

45 
----------------------------------------------------------------

TABLE 16. AIRSPEED PERFORMANCE FOR LOCALIZER BACK COURSE 

APPROACH DURING SEGMENT IV: MDA TO LANDING/GO-AROUND 

CALLOUT SYSTEM 

SYNCALL 

(19 APPROACHES) 

PILOT-NOT-FLYING 

(17 APPROACHES) 

MEAN PERCENT TIME 

OUT OF TOLERANCE 

13 % 

34 % 

NUMBER OF APPROACHES 

OUT OF TOLERANCE 

5 

12 

in airspeed performance at 500 AFL that was associated with type 

of callout system, by the time MDA was reached, airspeed performance with SYNCALL was better than with PNF callouts only. Figure 10 graphs the mean airspeed deviation from Selected Approach 

Speed at 500 AFL and at MDA for SYNCALL and PNF approaches. 

Average airspeed deviation for PNF approaches tends to increase 

slightly (+6.5 KTS +/-4.9 KTS at 500 AFL increasing to +7.9 KTS 

+/-4.3 KTS by MDA). In contrast, average airspeed deviation for 

SYNCALL approaches tends to decrease (+6.3 KTS +/-3.9 KTS at 500 

AFL decreasing to +4.6 KTS +/-3.2 KTS by MDA) Analysis of Variance yielded no significant effect on average airspeed deviation 

due to either altitude (500 vs MDA) or callout system (SYNCALL vs 

PNF), but the probability of an interaction between the two approached signiricance (Fmax =1.85, df=17, p(0.10) Tnis analysis 

taken together with the significant effect on airspeed performance associated with type callout system in the lower segment 

from MDA to Landing or Go-Around suggests that airspeed performance improved during the descent when SYNC ALL was in use but did 

not improve during the same part of the descent when only the PNF 

callouts were in use. 

link Rate Deviations Durin,g ..t.h.g, YQ.R Approach 

Table 18 shows the sink rate data for Segment III (500 AFL to 

MDA) for the VOR Raw Data Approach. Analysis of Variance for effects of callout system, pilot flying, and order of pilot flying 

on sink rate performance for this approach yielded a significant 

difference (F=5.01; df=1,31; p(0.05) due to callout system on the 

maximum deviation beyond operational tolerance. The effect of 

callout system on percent time out of tolerance approached significance (F=4.06; df=1,31; p=O.053) When SYNCALL was in use, there 

were no instances for the 19 approaches flown with SYNC ALL of 

sink rate exceeding the operational tolerance of 1174 FPH. Of 

the 19 approaches flown with PNF callouts 4 had sink rate deviations in excess of tolerance. The mean maximum deviation of 

46 
~ 

-1 

MEAN AIRSPEED DEVIATIONS FOR LOCALIZER 

BACK COURSE APPROACH 

~ 

~ +10 .. :2: 

0 

l- I PILOT_NOT-FL'{ING cd: 

-

> 

w 

0 

+5~ SYNCALL c 

W 

w 

a.. 

C/) 

0: 

-

cd: 

:2: 

« 

w 0 ~ 500 AFL MDA (289 AFL) 

ALTITUDE 

Fi;urc 10. - ~~an ~irspeej Deviations for LU~31izer Ba~k Cuurse 

''\;JprC'ilC:1. 
TABLE 17a. NUMBER OF APPROACHES INSIDE AND OUTSIDE TOLERANCE 

FOR AIRSPEED DURING LOCALIZER BACK COURSE APPROACH 

AT 500 FEET AFL 

SYNCALL PILOT-NOT-FLYING 

INSIDE TOLERANCE 12 10 

OUTSIDE TOLERANCE 7 7 

Note: Data were available for only 36 of the 40 approaches flown 

since 3 approaches terminated in a go-around before the aircraft 

reached MDA and since data were not available for a 4th approach. 

TABLE 17b. NUMBER OF APPROACHES INSIDE AND OUTSIDE TOLERANCE 

FOR AIRSPEED DURING LOCALIZER BACK COURSE APPROACH 

AT MDA (= 239 FEET AFL) 

SYNCALL PILOT-NOT-FLYING 

INSIDE TOLERANCE 16 8 

OUTSIDE TOLERANCE 3 9 

these four "out-of-tolerance" approaches was 1412 FPM +1-167 FPM 

or 238 FPM beyond the operating tolerance of 1174 FPM. Percent 

time out of tolerance for all approaches together including those 

that were always within tolerance was 0 J for SYNC ALL approaches 

compared to 9~ for PNF approaches. Finally, for those 4 PNF approaches that were out of tolerance the mean percent time out of 

tolerance was 37.3%. 

~-Engine Approach snQ Landing AirsDeed Performance 

As stated above, the 1-Engine Approach and Landing is flown 

by Captains only and thus had to be analyzed separately. It also 

differs from other types approaches in the procedures for 

airspeeds to be flown at various segments of the approach. The 

other approaches are flown fro~ 1000 feet to landing using Approach Speed, abbreviated V(APPR). 6 On aircraft equipped with 

5. The pilots compute this mentally by adding one half the 

48 
TABLE 18. SINK RATE PERFORMANCE FOR VOR RAW DATA APPROACH 

DURING SEGMENT III: 500 FEET AFL TO MDA (435 FEET) 

CALLOUT SYSTEM 

SYNCALL 

(19 APPROACHES) 

PILOT-NOT-FLYING 

('9 APPROACHES) 

NUMBER OF APPROACHES 

OUT OF TOLERANCE 

o 

4 

MEAN MAXIMUM 

DEVIATION 

1412 FPM +/- 167 

(FOR 4 APPROACHES) 

an Auto Throttle System (ATS), like the one simulated in this 

study, the pilots set an orange airspeed command bug on their 

airspeed indicators to their chosen V(APPR). Tney then use this 

bug to visually assist them if flying manually; also, the ATS, if 

engaged, will automatically maintain this commanded airspeed. A 

callout of airspeed referenced to Approach Speed is made at 500 

AFL as a cross check. For the 1-Engine Approach, Approach Speed 

is defined differently. It is the 0 deg flaps/slats extended 

minimum maneuvering speed for the landing gross weight of the 

aircraft, abbreviated V(O deg/EXT). This higher approach speed 

is maintained until a visual landing commit point of 700 feet AFL 

in case the crew should have to make a go-around and thus need 

the added speed for a climb-out with only one engine. If they 

decide a landing can be assured from 700 AFL then the Captain begins to reduce airspeed so as to arrive at 100 AFL with a speed 

of V(Ref) plus the conventional wind additives of one half runway 

wind speed plus gust velocity. The orange command airspeed bug 

is left at the higher Approach Speed of V(O deg/EXT) for the 

duration of the approach. Thus, in contrast to other types of 

approaches, the pilot now attemps to fly an airspeed well below 

that indicated by the orange bug, 10 to 20 KTS in the aircraft 

simulated for this study. 

The pilot-not-flying in this instance knows that the captain 

will be reducing airspeed below the approach speed shown by the 

orange bug and makes allowance for this when calling out routine 

airspeed checks or when making an airspeed deviation callout. 

The SYNCALLsystem, on the other hand, obtains its Approach Speed 

setting from the orange ATC command bug setting. Thus, when the 

headwind plus the added speed factor of gusts to V reference with 

the restriction that Approach Speed will never be less than 

V(Ref) plus 5 KTS or greater than V(Ref) plus 20 KTS. 

49 
captain slows the aircraft to the slower approach speed, the SYNCALL system would be inappropriately triggered to make an 

airspeed deviation callout. 

In order to determine whether such inappropriate deviation 

callouts had an adverse effect on airspeed performance during the 

Single Engine Approach and Landing, airspeed was measured· referenced to the desired approach speed of V(Ref) + wind when the 

aircraft was at 100 feet AFL, the point at which procedures call 

for the speed to be at this slower approach speed. Airspeed performance would be adversely affected if, for the SYNCALL approaches, airspeed was excessively above V(Ref) + wind as compared to the PNF approaches. Table 19 shows mean Indicated 

Airspeed (lAS) minus V(Ref) + wind additives at 100 AFL for the 

10 approaches with SYNCALL and the 9 approaches with PNF callouts. While SYNCALL approaches averaged 9.4 KTS above the 

desired approach speed and PNF approaches averaged only 6.5 KTS 

above approach speed, this difference, when tested for statistical significance by T-Test was not significant (T=1.01, df=17, 

p>O.10). Also, the number of approaches that were flown outside 

tolerance were almost equal with SYNCALL and with PNF callouts: 4 

out of 10 approaches out of tolerance with SYNCALL compared to 5 

out of 9 approaches out of tolerance with PNF callouts. However, 

the standard deviation of the airspeed performance with PNF callouts was only +/-3.6 KTS compared to +/-7.8 KTS for the airspeed 

performance with SYNCALL. This difference in variability of 

airspeed performance was significant by two-tailed test for 

differenoes between independent variances (F=4.69; df=9,8; 

p<O.05) Tnus the captains' airspeed at 100 feet during the 

Single-Engine Approach and Landing was more variable with SYNCALL 

than with PNF callouts. This is certainly an adverse effect on 

airspeed performance from an operational standpoint. 

----------------------------------------------------------------

TABLE 19. AIRSPEED PERFORMANCE FOR 1-ENGINE APPROACH 

AND LANDING - CAPTAIN ONLY 

AIRSPEED MEASURED AT 100 FEET AFL 

MEAN AIRSPEED STANDARD 

DEVIATION DEVIATION 

CALLOUT SYSTEM I 

SYNCALL + 9.4 KTS +/- 7.8 KTS 

(10 APPROACHES) 

PILOT-NOT-FLYING + 6.5 KTS +/- 3.6 KTS 

(9 APPROACHES) 

----------------------------------------------------------------

50 
Summary Qf Individual Approach Results 

In summary, there were specific instances of improved performance on airpseed and sink rate when SYNCALL was in use for the 

non-precision approaches. However, when SYNCALL made inappropriate airspeed deviation callouts for the 1-Engine Approach, 

airspeed performance became more variable and there was a trend 

for the pilot-flying to erroneously heed the inappropriate callout. 

51 
3YNCALL And Pilot-Not-Flyin~ Callout Perform3nca 

The COCkpit audio recordin~s for ten of the twenty crews were 

analyzed to obtain several types of d~ta. The ten crews were 

chosen such that half the crews had flown witn 3YN~ALL first ~nd 

tnerefore with Approach Set 1, and the other half had flown with 

SYNCALL last and therefore with Approacn Set 2. Each callout 

made either by the SYNCALL system or by a crew ~e~ber was transcribed. Wnen both SYNCALL ~nj PNF callouts were made for a 

given item, the relative order of the two callouts was noted. 

For the approacnes flown using PN~ callouts, the audio of tne 

SYNCALL system had been turned off in tne COCkpit. However, t~e 

SYNCALL system continued to function, albeit Silently dS far as 

the cre~ was concerned. These callouts made by 3YNCALL were 

recorded on audio tape mixed with the on~oin3 cockpit audio for 

the PNF approaches. Thus it was possible to determine, for the 

PUP approaches, whether SYNCALL would nave made a given callout, 

had it been operational and to compare the reliability of 3YNCALL 

to PUF callouts in terms of 1) whether a given callout was made 

or not and 2) which system made the callout first: SYNCALL or 

PNF. 7 Finally, a tabulation was made of other concurrent audi~ 

events that occurred in the cockpit W~! I SYNCALL callouts would 

have been made for the approaches that used PNP callouts. 

Reliabi-LLU..Q...( C3.l1outs 

Table 2~ shows the reliability of ?NF and of SYNCALL for making normal approach callouts in the simUlator. For the 53 approaches for which PUP callouts were used, a total of 746 callouts were required. 507 of these or 72~ were roa1e by the 

Pilot-not-Flying. For the 53 approaches witn SYNCALL, ~ total of 

535 normal callouts were required. 8 6J9 of these or 371 were 

made by SYNCALL. It should be noted that there were no approaChes for which no callouts at all were made, fo~ either system. Rather, for some approaches not all the callouts were ~ade, 

and this was true for both systems. The difference in number of 

callouts made by SYHCALL and by PNF was significant (X(2)=91.24, 

df=1, p<O.OJ2). 

Since any conclusions regardin~ tne relative reliability of a 

7. This ingenious idea was the suggestion of G2ne Tomlinson, 

simulation computer programmer for the project. 

8. Tne difference of 745 required callouts the 53 PNF approaches compared to the 535 required callouts for the 53 SYNC ALL 

approaches resulted from the differences in go-around altitudes 

used by different pilots for those approaches that were terminated in go-arounds. For eXample, a go-around at 450 feet would require altitude callouts only to 500 feet while a go-around at '50 

feet would require altitude callouts all the way down to 200 

feet. 

52 
----~------------------------------------------------- ----------

TABL~ 20. R~LIABILITY 0F SYNCALL AND OF PILOT-tIOT-FLYING ?OR 

~AKING NORMAL APPRJACH CALL0UTS Iii TH~ SIMULATOR 

CALLOUT 3YST~M 

3YNCALL 

(53 APPR0ACn£3) 

PILOT-NOT-FLYING 

(53 APPRDACHES) 

dAS CALL8UT ~AD~ ? 

YES ~O 

6J9 76 

239 

rOTAL P~RC~NT 

535 37 i 

746 72 , 

system like SYNGALL compared to Pilot-not-Flying callouts were to 

be made on the basis of data collected in a simulatio~ of actual 

flight, the reliability of ?NF callouts in the si~ulator was compared to the reliability of PNF callouts in tne actual aircraft 

on the. line. This comparison could only be performed for normal 

callouts and could not be done for ~atchad sets of approaches 

since most approaches flown on the line are flown in visual 

weatner conditions even if oy instrument procedures. By contrast, all but the single-engine approach in the simulator are 

flown with simulated instrument weather conditions down to 1ecisio~ height or MD~. If any thin;, however, tnis snould tend to 

mnke PN~ callout reliability on the line lower than in the simulator, if low callout reliability is associated with better 

weather co~jitions on the approach. Table 21 shows the reliability of normal PNF callouts ~eing ~3de on the line and in the 

simulator with 73' of the nor~31 call0uts made on the line compared to 72k of the nor~al callouts made in tne simulator. Tne 

difference in tne nu~oer of callouts made in each case was significant (X(2}=9.91, df=', p<a.J02) with the normal callouts on the 

line actually oain; ~ore reliably made than the normal callouts 

made in the simulator durin~ the experim9nt. Thus, if anything, 

PNF normal callouts would De expected to be closer in reliability 

to SYNCALL normal callouts When made on the line durin~ actual 

flignt operations. 

In an attempt to maKe the line data and the simulator data 

more comparaole, the percenta~e of normal callouts made under the 

three conditions: 1} PNF on the line, 2} PNF in the simulator 

durin~ the experi~ent, and 3} SYUCALL in the simulator during the 

experi~ent were re-calculated, deleting from consider~tion all 

the MDA and DH callouts, since these were rarely required on the 

line due to tne ~ood weather (4 of the 64 line approaches were 

flown in weather close to the publisned minimums). Table 22 

shows the resulting percenta~es with the previo~s percentages 

53 
rABL~ 21. RELIABILITY OF PILOT-NOT-FLYING ON THE LINE AND IN 

TH~ SIMJLATOR FOR MAKING NORMAL APPROACH CALLOUT3 

FROM 10JO FEET AFL TO 10 FEET RADIO ALTIM~TER 

WAS CALLOUT :1ADE ? TOTAL PC:RCEIH 

YES NO 

ON TiE LINE 596 189 335 79 ~ 

(S4 APPiWACH~S) 

Iii THE SHIULATOR 432 172 604 72 ~ 

( 5 3 APPnJACHlS 

th3t did incluje MDA and DH callouts, for comparison. No statistical analysis of this three-way comparative data was performed since the validity of such a comparison is doubtful. 

Rather, it is simply interesting to note that the 831 reliability 

of PNF normal callouts on the line would appear to be very close 

to the 85% reliability of SYNC ALL normal callouts in the simulator. And this in turn brin~s into question the importance of the 

statistically significant difference between PNF and SYNC ALL normal callout reliability in the simulator. Tnis point will be 

discussed further in the Discussion section. 

TABLE 22. RELIABILITY OF NORMAL APPROACH CALLOUTS MADE BY 

PILOT-NOT-FLYING ON THE LIN~ AND IN THE SIMULATOR 

AND BY 3YNCALL IN THE SIMULATOR 

~DA & DH CALL OUTS EXCLUDING 

INCLUDED MDA & DH CALLOUTS 

ON TifE LINE 83 % 

PNF IN THE SIMULATOR 72 ~ 74 ~ 

SYNCALL IN THE SIMULATOR 37 % :3 5 :h 

----------------------------------------------------------------

The reliability of the deviation callouts made by SYNCALL and by 

PNF is shown in Table 23. While SYNCALL was designed to repeat a 

deviation callout if the problem was not corrected, for purposes 

of analYSis, such repetitions were scored as belon3in3 to the 

sa~e instan~e of a required callout. Tnus, if for a given devia54 
tion, the callout system made one or more callouts, that required 

callout was scored as havin~ been made once. It was noted that 

SYNCALL repeated deviation callouts for a given deviation far 

~ore frequently than did the PNF and this was the main reason for 

the chosen method of counting number of callouts made. 

---------------------------------~-------------------- ----------

TABLE 23. RELIABILITY OF DEVIATION CALLOUTS BY SYNCALL AND BY 

PILOT-NOT-FLYING IN THE SIMULATOR 

WAS CALLOUT HADE ? TOTAL PERCENT 

CALLOUT SYSTEM 

SYNCALL 

PILOT-NOT-FLYING 

(SYNCALL OFF) 

YES NO 

15 3 

39 49 % 

Data are totals from 53 approaches for each callout system. 

When PNF deviation callouts were used, 49% of the instances 

requiring deviation callouts had deviation callouts made by the 

pilot-not-flying. This is compared to 90' of the instances requiring deviation callouts for which callouts were made by the 

SYNC ALL system when it was in use. Tne difference in numbers of 

callouts. made by the two systems was significant (X(2)=4 1 .63, 

df=1, p<O.002). 

Table 24 shows the number of deviation callouts made by the 

pilot-not-flying when SYNC ALL was not in use compared to the 

number of deviation callouts made by the pilot-not-flying when 

SYNCALL was operational. The PNF made only 24% of tne required 

deviation callouts when SYNCALL was also making deviation callouts, compared to 49S of the required deviation callouts when he 

had sole responsibility for making these callouts. Tnis difference in numbers of callouts made in the two conditions was also 

significant (X(2)=1.41, df=1, p<O.01) ThUS, while SYNCALL was far 

more reliable than PNP for making deviation callouts, SYNCALL 

also seems to have decreased the reliability of the PNF callouts 

themselves even though the pilots were instructed to make deviation callouts if required when SYNC ALL was in use and even though 

SYNC ALL was designed with larger tolerances than those used by 

the pilot-not-flying. 

Table 25 lists the individual normal callouts 

numbers of each that were or were not made by PNF in 

tor, by SYNCALL in the simulator, and for comparison, 

the line. The asterisks indicate those callouts for 

55 

with the 

the simulaby PNF on 

which there 
TABLE 24. RELIABILITY OF DEVIATION CALLOUTS BY PILOT-NOT-FLYING 

WITH SYNC ALL AND WITH NJ SYNCALL 

WAS CALLOUT MADE ? rOTAL PERCENT 

PILOT-NOT-FLYING 

WITH SYNC ALL 

PILOT-NOr-FLYING 

NO SYNCALL 

YES NO 

20 63 

39 50 

83 

Data are totals from 53 approaches for each condition. 

24 % 

49 % 

was a significant difference (p < 0.01) between the number of 

callouts made by SYNCALL and by PNF in the simulator. The callouts for which SYNCALL was more reliable than PNF appear to fit 

into certain cate~ories: 

Critical Descent Altitudes 

Decision Height 

MDA 

100 above MDA 

Outer Marker crossing altitude 

Flight Path position 

Localizer Intercept 

Glideslope Intercept 

Routine read-out of performance values 

lAS at 5~O feet 

Sink Rate at sao feet 

Ten-foot altitude callouts 

50 

40 

30 

10 • T~e difference for the 20 foot callout 

could not be tested since two cells of 

the Chi-3quare had expected values less 

than 5. 

Finally, it is interesting to note that both in the simulator 

ani on the line, the PNF makes a 100 foot callout with fair reliaoility, 73% in the simulator and 38£ on the line compared to 77% 

by SYNCALL. This is particularly interestin~ since the flight 

56 
TABLE 25. RELIABILITY OF INDIVIDUAL NJRMAL APPRJACH GALLOUTS MAD~ BY SYNC ALL AND BY PILOf-NOT-FLYING l~ THe 

SIMULATOR AND ON fH~ LIN~ 

CALLour 

LOC INT 

GIS lNf 

a [1 

a (1 ALT 

1000 FEC::T 

500 FEc::r 

lAS f! sao 

SNK @ SOD 

4JO 

300 

2;)0 

100 

10Q+t1DA 

:1DA 

or 

DH 

50 

4::> 

30 

20 

10 

IN SHIULATOR 

PNF 

YES NO • /0 sig? 

14 

26 

23 

12 

50 

45 

27 

30 

30 

34 

34 

24 

15 

'0 

25 

24 

23 

25 

23 

29 

7 

10 

21 

2 

6 

20 

, 9 

22 

12 

13 

9 

7 

1 1 

14 

9 

9 

3 

5 

6 

33 

19 

70 

35 

96 

38 

57 

51 

58 

74 

72 

73 

63 

48 

74 

73 

74 

33 

79 

if 

It 

* 

f 

* 

;r 

,. 

S'lNCALL 

Y E3 :~J 

39 

30 

32 

32 

51 

44 

44 

44 

32 

36 

23 

20 

21 

21 

25 

25 

25 

24 

21 

19 

1 

8 

:3 

8 

, 9 

17 

15 

5 

;) 

o 

'J 

o 

o 

o 

a 

o 

100 

9'7 

97 

9'7 

93 

35 

35 

8S 

63 

84 

59 

77 

100 

100 

100 

10,0 

100 

100 

10::> 

10D 

'ON fHE LINe: 

Pi'lf 

'lE3 NO ~ 

55 

57 

33 

27 

49 

55 

55 

56 

o 

o 

9 

, 

u 

26 

37 

1 4 

3 

9 

3 

86 

90 

59 

42 

78 

38 

35 

33 

o 

o 

3 60 5 

59 5 92 

55 3 88 

60 4 94 

52 2 97 

53 93 

manual for toe participating airline does not call for a '00 foot 

callout. None-the-less, the author in designing the SYNCALL system and the line pilots both in the airplane and in the simulator 

appear to have 3enaralized tne concept of 100-foot callouts to 

include a callout at 100 feet. 

R e 1 a t i veT 1:0 L'1.5. Q.[ .!Lo r mal C a 11 0 u t s l!!.i~.cL~ Ju. P N t. s..lJ ... Q. Ju. s UH~ ... ~J,,-"" 

Table 26 shows the relative timing of normal callouts made by 

PNF compared to those that would have been made by SYNCALL for 

the PNF callout approaches. Of all 746 callouts that were requi~ed, 5~ werelwould not have been ~ade by S'lNCALL, compared to 

25% tnat were not made by PNF. 81 of the required callouts were 

57 
-58-

not made by either system. When both the PNf callout was made 

and tne SYNC ALL callout was/would have been made, a total of 62~ 

of the callouts, the distribution was closely oalanced between 

the PNF callout coming first (24%) and the SYNCALL callout coming 

first (29%) with a few instances of the two callouts being started simultaneously (9%). 

TABLE 26. RELATIVE: TIMING OF NORMAL APPROACH CALLOUTS MADE BY 

PILOT-NOT-FLYING COMAPARED TO 3AME CALLOUTS MADE BY 

SYNCALL FOR 53 APPROACHES. THS CREW DID NOT HEAR THE 

SYNCALL CALLOUTS. NORMAL CALLOUr PROCEDURES WERE USED 

NU:1BER P£RCE:NT 

i'1ISSING SYNCALL CALLOUT 41 5 ; 

PNF CALLOUT BEFORE SYNCALL 182 24 ;, 

PNF' CALLOUT 3Ar-1E T 1(12 AS SYNC ALL 55 9 % 

PNF CALLOUT AFTER SYNCALL 216 29 ." jo 

MISSING PNF CALLOUT 184 25 .; 

ND CALLOUT BY EITHER SYNCALL/PNF 57 3 " jO 

TOTAL NUi1BER 02 CALLOUTS 746 100 .. ' 

jO 

lllJ1:'..J:: 1llLd i 0 i n for ;113 t ion ..tjLU 0 vel" 1 a~ .;LULC_JtiJ... ~3. 11 QQ..t..§. 

Table 27 shows the percentages of SYNCALL callouts that occurred at the same time as various other types of audio information. Again, tne data was taken from the approaches for which 

PNF callouts were in use and SYNCALL callouts were recorded on 

the audio tape only, not heard in the cockpit. 50% of the SYNCALL callouts did occur/would have occurred at the sa~e time as 

some other audio information. Tne interference with ATC communications between the controller and the simulated flight was 

minimal (3% of the crew's transmissions to ATC and 3~ of ATC's 

transmissions to the crew. The overlap with other ATC chatter 

was higher (13%) but probably acceptable since pilots only occasionally report learning useful information from listening to 

background ATC chatter (such as routing to expect and, occasionally critical traffic information). However, 23~ of the callouts 

would nave occurred during crew checklists and other callouts. 

This relatively large percentage could ~ave masked eitner the 

58 
SYNCALL callout or the crew voice message, had it actually occurred. 

In contrast, the Pilot-not-Flying when making Approach Callouts was observed to sequence these with other crew checklist 

calls so as to minimize overlap. Like SYNCALL, however, the PNf 

callouts often overlapped with Other ATC. 

TABL~ 21. pg~GBNf OF SYNCALL CALLOUTS WHICH WERE OVERLAPPED 

WITH OTHeR, SIMULTANEOUS COMPETING AUDIO INFORMATION 

CREW TO ATC 3 ~ 

ATC TO CRC;W 3 ,<f 

/0 

OTHE~ ATC 18 % 

CREW CHECKLISTS/CALLS 23 ~ 

/IJ 

OTHER SOUNDS 3 " /IJ 

NO OVERLAP 50 J, 

Tit1LLIl&. Accuracy Q.[ ~-Foot Callouts .Q.y SYNCALL 

Figure '1 shows the minimum, average, and ~aximum errors in 

the timing of ten-foot callouts made by SYNCALL in terms of the 

difference between the called altitude and the ~ctual aircraft 

altitude when the callout started. The altitude errors are shown 

as a function of sink rate. ThUS, tha higher the rate of descent, the less timely is the information given by a ten-foot callout. For example, with a sink rate of 200 FPM, the ten-foot 

callout for 30 feet would, on the average, start when the aircraft had actually descended to 27.5 feet. For a sinK rate of 

1000 FPM, the same callout would start, on the average, when the 

aircraft had descended to 21.5 feet. Thus for high rates of descent ten-foot callout altitude information by SYNCALL was not 

timely. Figures 12, 13, and 14 contain the actual data pOints 

from which the three curves were derived. Tne timliness of the 

corresponding PNF callouts can be inferred by examining the relative timing of each PNF ten-foot callout with that of the 

corresponding SYNCALL ten-foot callout and tabulating these relative time categories (Before, Same Time As, After) for different 

levels of sink rate. When this analysis was done, no systematic 

relationship was found, suggesting factors other than sink rate 

for the timeliness of PNF 10-foot callouts. 

59 
Ln_'ill..2.rooriate Cal..UL~ .Q:i SYNC ALL 

Certain callouts were made by the SYNC ALL system at inappropriate times or contained inappropriate information. Most of 

these were artifacts related to the instructional aspects of flying approaches in a flight simulator. For example, pilots heard 

an immediate "Localizer Intercept", each time the simulator position was reset to a position on the localizer track. Another 

anomaly was the message "100 feet above MDA" if the instructor 

pilot used the altitude reset capability. These problems would 

not be expected in an actual aircraft. 

Otha: 3YNCALL callouts were made occasionally for situations 

which had been unintended in the design of the system. Unlike 

the inappropriate callouts listed above, these had some relevance 

to the current flight situation, and, in fact, some of the. pilots 

upon hearing them remarked that they gave useful information. 

SYNCALL called out the Outer Marker and crossing altitude any 

time the aircraft passed over the outer marker while being vectored by ATC. "Localizer Intercept" was called out any time the 

aircraft was vectored through the localizer at an altitude less 

than 3000 AFL. Localizer deviation callouts were then made following such unintended localizer intercept callouts. Wnile the 

pilots found the unintended localizer intercept callouts useful, 

the ensuing deviation callouts were generally perceived as a nuisance. 

Finally, there was one instance of SYNCALL callouts that were 

not only inappropriate but which, while giving true information, 

presented this information in a way which implied exactly the oppOSite of the intended message to the crew. This was the case of 

airspeed deviation callouts which called airspeed as being excessively slow when in fact the pilot-flying was purposely slowing 

the aircraft. for the single-engine approach, discussed above in 

the section on flight performance results. 

J&l...Lo u t s l.tl.<Lg, .Q:i P N F JiY.t Not ~Y ti CAL L 

During the experiment, it was observed that the Pilot-notFlying made other callouts appropriate and useful to the approach 

that the SYNCALL system had not been designed to m~ke. Some of 

these had been purposely omitted from SYNCALL system design to 

avoid nuisance callouts. One such was deviation callouts right 

or left of voa course. Tne simulator Course Deviation Indicator 

(CDI) did not distinguish Localizer and VOH signals; thus no 

difference in sensitivity for course deviation callouts could be 

programmed for VOH course deviations as compared to Localizer 

course deviations. The PNF did however, take this difference 

into account when ~aking deviation callouts during the VOH approach. SYNCALL was purposely designed to make airspeed and sink 

rate deviation callouts only between 500 and 100 feet AFL to 

avoid nuisance callouts. The PNF, however, occasionally judged 

it necessary to make such callouts above or below this altitude 

range. The result of the inappropriate negative airspeed devia60 
0') 

I-" 

...., 

ALTITUDE ERROR OF TEN-FOOT CALLOUTS 

BY SYNCALL AS A FUNCTION OF SINK RATE 

(AT LEAST 5 SAMPLES PER DATA POINT) 

20 

'1-.. 15 

ex: 

0 

ex: 

ex: 

w 10 

w 

C 

::J 

I-

- 5 I-

-I 

<t 

o 200 400 600 800 1000 1200 1400 

SINK RATE, fpm 

Figure 11. - AltituJc Error of Ten-Foot Callouts by SYNCALL as a 

Function of Sin~ R~te. 
C') 

~ 

MINIMUM ALTITUDE ERROR BY SINK RATE 

FOR TEN-FOOT CALLOUTS BY SYNCALL 

...., -20 ~ 

.. a:: 

o 

a:: -15 a:: 

w 

w 

o 

:::> -10 II-

-J 

« 

:E -5 

:::> 

:E 

z 

-

:E o 200 400 600 800 1000 1200 1400 

SINK RATE, fpm 

FiJu!"'0 12. - '1ini:nu'n Altitu'jc Error ';)y 3ir,:< It'lt~ for T~r,-?oL\t 

C311uuts by 3Y~:~LL. 
0) 

t..:I 

MEAN ALTITUDE ERROR BY SINK RATE FOR 

TEN-FOOT CALLOUTS BY SYNCALL 

+-' -20 

\to-

.. ex: 

o 

ex: -15 ex: 

w 

w 

o 

::J -10 

I-

-I-

...J 

« 

z -5 

« 

w 

:E 

o 200 400 600 800 1000 1200 1400 

SINK RATE, fpm 

Fi3 ur e 13. - ~~an ~ltitud0 Error by 3ink R3te for T~~-?uot C31-

luuts ~y 3Y~:ALL. 
(j) 

"'" 

MAXIMUM ALTITUDE ERROR BY SINK RATE 

FOR TEN-FOOT CALLOUTS BY SYNCALL 

~ -20 

a:." 

o 

a:. 

a:. -15 

w 

w 

o 

::J !:: -10 

I-

-I 

« 

~ -5 

::J 

~ 

-

X 

« 

~ 0 200 400 600 800 1000 1200 1400 

SINK RATE, fpm 

Figure 14. - ~aximum AltituJe Error by Sink Rate for Ten-Foot 

Callouts by SYNCALL. 
tion callouts during the single-engine approach was often positive airspeed deviation callouts by the PNF, reflecting the actual situation if it appeared that the pilot-flying was erroneously 

heeding the SYNCALL messages and not slowing the aircraft. Finally, the PNF called out "Below MDA" or "Above MDA" if either 

occurred after reaching MDA but before the runway was in sight. 

Since SYNCALL had no information about visual acquisition of the 

runway, it would have been unable to make such callouts appropriately. 

Summary of Callout Performance Data 

SYNCALL was more reliable than PNF for making deviation callouts and also for making normal callouts. However, comparison 

with line data for normal callouts suggests that the PNF normal 

approach callouts on the line would be close in reliability to 

those made by SYNCALL. SYNCALL systematically missed certain 

callouts that were made by PNF, and neither system made all the 

callouts. Crew communications with ATC had minimal overlap with 

SYNCALL, however, a relatively large percentage of SYNCALL callouts overlapped crew checklist and callout messages. 

Pilot Judgment Data 

Table 28 shows the percent of the 40 pilots who found each 

category of callout by SYNC ALL (normal and deviation) "useful and 

desirable" for each of the four types of approach conditions. 

Nearly all (85%) of the pilots who flew SYNCALL during this 

evaluation wanted the system to make normal altitude callouts 

during approaches when the weather was down to IFR minimums. 73% 

wanted SYNC ALL to make deviation callouts under these same conditions. For the other conditions, the percent of pilots who wanted SYNCALL callouts of the two types ranged from 49% for deviation callouts during Day VFR conditions to 78% for normal altitude callouts during abnormal or emergency conditions. For N=40 

pilots, Chi-sqaure analysis with 1 degree of freedom yields 

p<O.Ol for any condition in which 71% or more pilots wanted SYNCALL. Thus the data support a strong pilot preference for SYNCALL during approaches with IFR weather, or with abnormal or emergency conditions. For night VFR conditions, only normal altitude callouts received this strong preference. There is no such 

preference for SYNCALL during day VFR operations. 

65 
TABLE 28. PERCENT OF PILOTS WHO FLEW SYNC ALL THAT WANTED IT TO 

MAKE NORMAL CALLOUTS AND OR DEVIATION CALLOUTS 

FOR FOUR TYPES OF APPROACH CONDITIONS. N = 40 PILOTS 

(20 CAPTAINS AND 20 FIRST OFFICERS) 

NORMAL 

ALTITUDE 

CALLOUTS 

(10 TYPES) 

DEVIATION 

CALLOUTS 

(4 TYPES) 

MEAN FOR ALL 

CALLOUTS 

APPROACH CONDITION 

NIGHT VFR ABNORMAL/ 

EMERGENCY 

71 % 78 % 

56 % 69 % 

67 % 75 % 

66 

DAY VFR 

57 % 

49 % 

55 % 

IFR TO MINIMUMS 

85 % 

73 % 

82 % 
TABLE 29. PERCENT OF PILOTS WHO FLEW SYNC ALL THAT WANTED THE 

PILOT-NOT-FLYING TO MAKE NORMAL CALLOUTS AND OR 

DEVIATION CALLOUTS FOR FOUR TYPES OF APPROACH 

CONDITIONS 

N = 40 PILOTS (20 CAPTAINS AND 20 FIRST OFFICERS 

APPROACH CONDITION 

NIGHT VFR ABNORMAL/ DAY VFR IFR TO MINIMUMS 

EMERGENCY 

NORMAL 

ALTITUDE 

CALLOUTS 57 % 68 % 51 % 68 % 

(10 TYPES) 

DEVIATION 

CALLOUTS 52 % 67 % 47 % 65 % 

(4 TYPES) 

MEAN FOR ALL 

CALLOUTS 54 % 68 % 50 % 67 % 

Table 29 shows the corresponding data for Pilot-not-Flying 

callouts. Using the Chi-Square probabilities from above, there 

were no conditions that reached the 0.01 level of significance. 

Thus, the pilots as a group were neither in favor of nor against 

pilot-not-flying callouts. 

Table 30 shows pilots' perceived intelligibility ratings for 

the synthesized voice compared to the human voice messages of the 

ATC -tapes and of the pilot-not-flying callouts. Interestingly, 

after this initial exposure, the intelligibility ratings for the 

synthesized voice were close to those for actual ATC communications. Many pilots noted independently that the electronic voice 

quality of the synthesizer made it very distinctive and left no 

doubt regarding the source of the voice. 

Figure 15 shows the mean ratings for all callouts on four of 

the semantic differential scales for each of the callout systems 

(SYNCALL and PNF). These data come from the 28 of the 40 pilots 

or 70% who volunteered to complete this optional form on their 

own time. Differences between ratings for SYNCALL and PNF callouts were significant for all four scales (InformativeDistracting t=-3.23, P<0.05; Helpful-Detrimental t=-5.12, 

p<0.002; Dependable-Unreliable t=7.98, P<0.002; DisruptiveCoordinating t=-6.42, p<0.002; df=8 for each scale). 

67 
0) 

00 

SEMANTIC DIFFERENTIAL RATINGS OF 

CALLOUTS BY 14 CAPTAINS AND 14 

FIRST OFFICERS 

"'000 2-< 

INFORMATIVE ."T1 2. • • • • DISTRACTING --'--'--'--'--'--'--

HELPFUL _. • • . • • • DETRIMENTAL '--'--'-'-'--

COORDINATING • • • . • DISRUPTIVE -' '--'-'-'--

DEPENDABLE _. • : __ :_: __ I~NRELIABLE* 

SYN CALLOUTS MADE BY SYNCALL 

PNF 11111111 II II CALLOUTS MADE BY PILOT-NOT-FL YING 

*POLARITY OF THIS SCALE IS REVERSED FOR CLARITY OF 

PRESENTATION. SCALE POLARITY ON DEBRIEFING SHEETS 

WAS RANDOM. 

Figure 15. - 3e~antic Differential Ratinss of Callouts by 14 

Captains and 14 ?irst Officers. 
SEMANTIC DIFFERENTIAL RATINGS OF THE 

GPWS IIGLIDESLOPE" AND THE GPWS 

IIWHOOP WHOOP! PULL UP!" BY 14 

CAPTAINS AND 14 FIRST OFFICERS 

'1J 

G) C ......... . 

INFORMATIVE • • • en .'. • DISTRACTING __ '_' __ '.,,-.' 1_'-

- / 

HELPFUL_:_:--*-: 

, 

DETRIMENTAL 

COORDINATIN~:_:_I~"'--_:_I_DISRUPTIVE 

/ 

DEPENDABLE _: __ // :--L-' UNRELIABLE* 

GIS - - - - GPWS IIGLIDESLOPEIl 

PULL 8 8 I 0 0 0 0 GPWS IIWHOOP WHOOP! PULL UP!" 

*POLARITY OF THIS SCALE IS REVERSED FOR CLARITY OF 

PRESENTATION. SCALE POLARITY ON DEBRIEFING SHEETS 

WAS RANDOM. 

Figure 16. - Se~antic Differential Ratin3s of the ~P~S 

":Jlideslope" and the GP't/S "Hhoop Hhoop! Pull Up!" by 14 Captains 

and 14 ?irst Officers. 

69 
TABLE 30. PERCEIVED INTELLIGIBILITY (PILOT JUDGMENTS) OF 

DIFFERENT TYPES OF SPEECH HEARD DURING SYNCALL STUDY 

SYNCALL (SYNTHESIZED VOICE) 75 % 

ATC COMMUNICATIONS (TAPED, LIVE) 76 % 

PILOT-NOT-FLYING CALLOUTS 88 % 

While both systems were judged as informative, helpful, 

dependable, and coordinating, callouts given by PNF were judged 

as more informative, more helpful, and more coordinating than the 

same callouts by SYNCALL. On the other hand, callouts by SYNCALL 

were judged as more dependable than the same callouts by PNF. 

For purposes of comparison to an existing system that produces voice messages on the approach to landing, the pilots ratings of the GPWS "Whoop, Whoop! Pull Up!" and "Glideslope" messages are given in Figure 16. 

The next pilot judgment data was obtained from a follow-up 

debriefing form that was mailed to the 28 pilots who participated 

in the optional semantic differential form. Of these, 13 pilots, 

or 46%, responded in a period from 3 to 9 months after the experiment. The follow-up debriefing form is reproduced in Appendix 

IV. 

Table 31 shows the modal responses for tolerance values for 

each of the deviation callouts. These pilots' recommendations 

for sink rate, localizer, and glideslope tolerances matched those 

of the SYNC ALL system as designed for the study. However, their 

recommendations for airspeed tolerances were different from the 

SYNCALL tolerance of +/- 7 KTS. They suggested +10 KTS and -5 

KTS. 

Table 32 shows these pilots' recommendations for active zones 

for SYNCALL deviation callouts. Their recommendations matched 

the original design of the SYNC ALL system. 

Finally, Table 33 shows these pilots' mean and median ratings 

of the recommended modifications suggested by one or more of the 

40 pilots who participated in the study. In the column on the 

far right is listed, for each recommendation the number of the 13 

pilots who judged that recommendation to be "essential in order 

for you to accept SYNCALL in your cockpit". 

70 
TABLE 31. LINE PILOT PREFERENCES FOR SYNCALL DEVIATION CALLOUT 

TOLERANCES. ALL DATA ARE MODES. 

AIRSPEED 

SINK RATE 

2000 TO 1000 AFL 

1000 TO 300 AFL 

BELOW 300 AFL 

LOCALIZER 

GLIDES LOPE 

RECOMMENDED TOLERANC~ 

V CAPPR) + 10 OR - 5 

2500 FPI1 

1200 FPM 

1000 FHP 

1/2 DOT 

1 1/3 DOTS 

Note: One captain suggested 1 1/2 dots for localizer back course 

and 2 dots for a VOR approach. Another captain suggested 2 dots 

glideslope to 500 feet AFL and then 1 dot when below 580 AFL. 

TABLE 32. LINe PILOT PREFERENCES FOR SYNCALL DEVIATION CALLOUT 

REPETITION RATES AND ACTIVE ZONES. ALL DATA ARE HODES 

REPETITION RATE ACTIVE ZONE 

AIRSPEED 5 SEC 600 TO 100 FEET AFL 

SINK RATE 5 SEC oQO TO 100 FEET AFL 

LOCALIZER 10 SEC LOC CAPTURE TO 

10';) FEET AFL 

GLIDESLOPC: 5 SEC GIS CAPTURE TO 

100 FE2:T AFL 

Note: Five pilots suggested an active zone for glideslope and 

localizer from approximately 1000 to 100 feet AFL. 

N = 13 pilots who responded to second debriefing by ~ail up to 

9 ~onths after having flown SYNCALL in the experiment. 

71 
TABLE 33. PILOT RATINGS OF THEIR O~N SUGGSST8D MODIFICATIONS 

TO SYN~ALL USING THE SCALE: 

UNSAF~ UNDESIRABLE NO PREFERENCE DESIRABLE HIGHLY 

OR CANtT DECIDE DESIRABLS 

5-------------4-------~-----3-------------2----------- __ 1 

SUGGSSTIO~S W~aE ~ADE BY ALL 4a PILOTS. RATINGS WBRE OBTAINED 

FROM '3 PILOTS WrlO RESPONDED TO SECOND DEBRIEFING BY MAIL UP TO 

9 MONTHS AFTER HAVING FLOW~ SYNCALL IN rHE ~XPERIMENT. 

SUGGESTION 

OBTAIN FAA & COMPANY AGREEMENT 

NOT TO USE COCKPIT VOICE RECORDINGS OF SYNCALL FOR 

DISCIPLINARY ACTIONS 

O}i IOF F SW ITCH 

ELIMINATE NUISANCE CALLOUTS 

PROVIDE TEST SWITCH 

MANUAL VOLUME CONTROL 

CALLOUT FOR "BELOW MOA" 

INCREASE TOLERANCE ON 

AIRSPEED DEVIATION CALLOUTS 

INHIBTT AIRSPEED AND SINK 

RATE DEVIATION CALLOUTS 

DUrlING ABNORMAL OR EMERGENCY 

APPROACHES 

MINIMIZE FREQUENCY OF 

t1~AN 

1.17 

, .23 

1.40 

1 .54 

1 .69 

2.03 

2.09 

DEVIATION CALLOUTS 2.15 

ELIMINATE GPWS AND REPLACE 

IT WITH SYNCALL 2.15 

REPEAT 10-FOOT CALLOUTS IF 

AIRCRAFT BALLOONS DURIN~ FLARE 2.15 

72 

SD HEDIAN 

0.58 

0.52 

0.70 

).66 

0.95 

1 • , 5 2 

1 .04 2 

1 .51 

0.80 2 

0.90 2 

0.90 2 

NU:1BER 

JUDG8D 

ESSENTIAL 

eN = 13) 

10 

10 

5 

6 

5 

3 

4 

4 

2 

3 

3 
WARN IF AIRSPEED IS 

BELOH ALPHA SPE:ED 2.17 1 • 1 1 2 

CLARIFY 3YNCALL AND PILOTN8T ["LYING RESPONSIBILITIES 2.23 1 .03 3 2 

CALL LaC CAPTURE AND THEi~ 

LOCALIZEH INTERCEPT 2.23 ·).33 2 

SHORTEN ME3SAGE ~.~? 

? ?- 0.75 2 2 

SPEED UP 3YNCALL SPEECH RATE 2.37 L ,.'3 2 4 

ADD OTH~R lVARNINGS, ~.G. ENGIN~ 

["IHE, LOSS Or' CABIN PRESSURE 2.45 1 .20 2 

AU r 0 [1 A TIC ALL Y CHANGE 3YNCALL 

PArlA!-IETi:':RS ACORDING TO 

TYPE Or APPROACH 2.45 , .05 2 2 

FOR :-JON-PRECISION APPROACHES 

HAKE SINK RATE DSVIATION 

CALLOUT3 HAVE TOP PRIORITY 2.46 1 .05 3 2 

;.1OHE ALERTING VOICE QuALITY 

FOR DEVIATIONS THAN FOR NJRMAL 

ALTITUDE CALLOUT S 2.52 0.87 '< J 2 

INCHi::A3S TOLERANCE FOR 

GLIDE3LOPE DEVIATIONS 2.85 0.9J 3 2 

HAVE SYNC ALL SAY "CHECK FLAP3" 

AT 1000 FEET AFL 3.00 1 .0 J 3 

HAVE S'fNCALL READ OllT FLAPS 

SETTING AT 1000 FEi::T ~.FL 3.00 1 .29 2 

INCRiASE TOLERANC3 FOR 

LOCALIZER DEVIATIOlJS 3.03 'J. B 3 

ADD HL'13DIATt:: ASTIaN 

CfECKLIST IT Ef'1S 3.03 1 • 32 4 

HAVS 3YNCALL !'IAKi DEVIATION 

CALLOUTS ONLY 3.09 1 • 14 4 0 

HAVE 3YNCALL READ OUT 

FLIGHT GUIDANCE SYSTEl'1 t10DES 

AFTEtl LAND 110DE IS A Rt·ED 3. 15 0.99 3 

73 
D2:LAY CALLOUT U' CREil IS 

i'1AXING RADIO TRANSMISSION 

UNTIL T RAN 3:-1 IS S ION COHPLETED 3.23 1 .09 3 4 

INCREASE: VOLUME- 3.36 0.5::> 3 

HAVE SYN8ALL MAKE NOR!-1AL 

ALTITUDE CALLOUTS ONLY 3.46 0.97 4 

tilTH THE: 50-FOOT CALLOUT, 

CALL AIRSPt:ED A:W 3INK RATE 

AND ELHlIIIAT1!: 40,30,20,10-FOOr 

CALLour.s 3.54 1 .27 4 

Several of the pilots noted independently that the airspeed 

deviation callouts in particular were repeated too often and before they could slow the airspeed to within tolerance. They 

found this to be a nuisance and distraction. This judgment was 

supported in the rating data on the list of modifications that 

the pilots suggested for SYNC ALL on the de-briefing forms. Of 

the 31 suggested modifications, "Eliminate Nuisance Callouts" 

ranked 3rd with a~ean of 1.4 6n a scale of 1 to 5 for highly 

desirable to unsafe and had a ~edian res90ns~ of 1 (highly desirable). Tne suggestion "Minimize Frequency of Deviation Callouts" 

ranked 9th out of 31 with a mean of 2.'5 and a ~edian response of 

2 (desirable). It will be remembered that more pilots in selecting those callouts by SYNCALL that they though useful and desirable, ch~se normal callouts than did deviation callouts. Tne 

perceived nuisance of excessive repetition rates for deviation 

callouts could possibly explain the smaller percentage of pilots 

who wanted deviation callouts by SYNCALL compared to those who 

wanted normal callouts by SYNCALL. There was, inCidentally, no 

correlation between individual pilots' flight performance in each 

of the four parameters and the desirability of deviation callouts 

in each of the parameters. If anything, there was a trend for 

those pilots with greater percent time out of tolerance in a 

given p~rameter to be more likely to find deviation callouts for 

that parameter useful and desirable. The item "Increase Tolerance on Airspeed Deviation Callouts" ranked 7th with a mean of 

2.03 and a median response of 2 (desirible). Also, the airspeed 

tolerance was the only tolerance that the pilots who completed 

the follow-up debriefing form wanted chan3ed.- And the change 

they recommended was to increase the tolerance on the positive 

side frow +7 to +10 KTS while decreasing the tolerance on the 

negative side from -7 to -5 KTS. 

Summary ..Q..[ .e ..tl .. ~t Judgment Data 

Clearly the pilots who evaluated SYNC ALL generally found the 

74 
system desirable for some but not all types of approach conditions. They seem to indicate that they would find it a useful 

aid but would not want to have to use it for every approach. An 

onloff switch was judged "highly desirable" and "es~ential" by 10 

of the 13 pilots who responded to the suggested modifications. 

It should be noted here that there were no differences between 

captains and first officers in the way they evaluated SYNCALL; 

nor did tha particular set of approaches the ~ilots flew with 

SYNCALL have any statistically significant effect on their 

evaluation of the system. This suggests that the pilot judgment 

data is reliable and that there is considerable agreement among 

pilots regardless of the position they fly or the specific conditions of the approaches they flew using SYNCALL. 

75 
DISCUSSIO~ 

Im.2.!.ic~£.!2.~ ~ Improved F1i~ Performance 

The results of this study suggest that the concept of synthesized voice approach callouts should be developed further. 

Flight performance with SY~CALL making the callouts was better 

during non-precision approaches than when only th~ current PNF 

call outs were used. The parameters of flight performance that 

were better when associated with SYNCALL were airspeed and sink 

rate. Airspeed (as an input to estimated ground speed) and sink 

rate constitute the raw data used by a pilot to fly a desired 

glide-path during a non-precision approach which, by definition, 

has no direct glide-path information displayed to the pilot. A 

non-precision approach is considered more difficult to fly just 

because of the lack of direct glide-path information as compared 

to the easier precision approach which provides glide-path information displayeJ directly to the pilot. Thus the use of SYNCALL 

was associated with improved flight perfor~ance for 3 type of approach that is more difficult to fly. 

By comparison data for the precision approaches, showed no 

statistically significant differences in flight performance as a 

function of callout system. If SY~CALL were a benefit only for 

non-precision approaches, what might be the relative gain to airline operations as a whole by the implementation of a properly 

designed synthesized voice approach callout system? 

Non-precision approaches result in a large percentage of the 

air carrier approach and landing accidents and incidents, 53% according to the ~rSB study (1). If a system which improves flight 

performance for this type of approach in particular made a sufficiently large contribution to lowering the over-all approach and 

landing accident rate it might pay for its installation. Of 

course, one would have to consider other solutions as well, including elimination of the non-precision approach from airline 

operations, different types of cockpit displays, and different 

types of flight information disptayed to the pilots. A comparative cost anglysis of such alternatives is beyond the scope of 

this stu;!y. 

There is no single reasonable conclusion that can be reached 

from the experimental observation that SYNCALL was associated 

with better flight performance only for non-precision approaches 

approaches which demgnd high levels of attention by the PF and 

by the PNF who is monitorin3. It co~ld be that the higher attention workload that is usually but not exclusively associated with 

non-precision approaches is the underlying factor whose effect 

could be mitigated by SY~CALL. If this is correct, then SYNCALL 

would be expected to be helpful during any approach with high attention workload. 

76 
Implications of Differential Callout Reliability 

The data on PNF and SYNCALL callout reliability suggest that 

the function of SYNCALL should be to alert the crew to deviations 

from desired flight parameters but not to make routine callouts. 

Altitude callout reliability by PNF was comparable to that of 

SYNCALL. Also 23% of the SYNCALL callouts overlapped crew checklist callouts. . In contrast, the PNF minimized this overlap by 

sequencing altitude callouts between checklist callouts whenever 

possible. Thus one cannot attribute improved flight performance 

to any greater reliability of SYNCALL over PNF for routine altitude callouts. And it is possible that SYNCALL altitude callouts 

actually interferred with pilots' comprehension of checklist callouts. Pilots did, in fact, rate SYNCALL less "coordinating" 

than PNF. All this argues against automatic altitude callouts by 

SYNCALL. 

The case for deviation callouts by SYNCALL is very different. 

They were substantially more reliable when made by SYNCALL than 

by PNF. Also, because they occurred infrequently compared to altitude callouts, their interference with crew checklist callouts 

would be minimal and thus could be an acceptable price in increased audio workload to pay to the benefit of timely and reliable alerts to flight performance deviations. 

Occasionally, however, an altitude callout can function as a 

warning of an unnoticed deviation. One can speculate that certain airline accidents might not have occurred, had automatic altitude deviation callouts been made (cf. 6 and 7). If a reliable 

algorithm to detect unnoticed altitude deviations during the approach could be developed, this could be a valuable component of 

a SYNCALL system. One algorithm for this is proposed for consideration in the section entitled "Refinement of a synthesized 

voice approach callout system" • 

. Consistent with a SYNCALL system that calls out deviations 

only would be the incorporation of the Ground Proximity Warning 

System (GPWS) modes into SYNCALL. SYNCALL as tested included 

warnings for excessive sink rate and for deviation below the 

glideslope. The "excessive rate of closure with terrain" mode of 

the GPWS logically could be added to the set of conditions SYNCALL monitors and calls out. 

If this reasoning is correct, a SYNCALL system that automatically called out deviations in airspeed, sink rate, glideslope, 

localizer, altitude, and terrain closure, only when deviations 

actually existed, could be a valuable addition to airline cockpit 

systems. 

77 
CONCLUSION 

The concept of an automatic approach callout system using 

synthesized speech definitely deserves further study. Improvements in flight performance by airline pilots were obtained for 

approaches that have high manual and visual workload for the pilots. By extrapolation, similar improvements might be obtained 

for any high workload approach. For the One-Engine Approach, for 

which SYNCALL made inappropriate deviation callouts, airspeed 

performance was worse with SYNCALL than with PNF. Reliability of 

normal altitude approach callouts was comparable for PNF on the 

line and in the simulator and for SYNCALL in the simulator. However, SYNCALL was more reliable than PNF for making deviation approach callouts in the simulator. pilots generally favored the 

concept of SYNC ALL and judged it more reliable than PNF callouts. 

They suggested modifications before it would be appropriate for 

operational use. It was concluded that SYNCALL improved flight 

performance for non-precision approaches, that a SYNCALL system 

should make deviation callouts only. For consistency, it is 

recommended that the modes of the Ground Proximity Warning System 

(GPWS) be incorporated into a SYNCALL system. The detrimental 

effects on performance associated with inappropriate deviation 

callouts led to the further conclusion that such callouts should 

be designed out of the system. Finally, in the section entitled 

"Refinement of a synthesized voice approach callout system", the 

results and conclusions of the experiment are used to develop 

suggestions for improvements to SYNC ALL before further testing. 

78 
REFINEMENT OF A SYNTHESIZED VOICE APPROACH CALLOUT SYSTEM 

Clearly the SYNCALL system as configured for this experiment 

is not appropriate for air transport use. This is evidenced by 

the fact that some of the data for some of the measures of system 

effectiveness found SYNC ALL to be no better than or, in some 

cases, not as good as PNF callouts. For example, SYNCALL was 

judged as more reliable than PNF callouts; but it was also judged 

less informative, less helpful, and less coordinating than PNF 

callouts. Tnese pilot rating scale data agreed with the callout 

system performance data on reliability of the callouts being 

made. And, the rating data on the scales informative/ distracting, helpful/ detrimental, and coordinating/ disruptive were at 

least indirectly supported by the data on percent overlap of SYNCALL callouts with other voice communications in the cockpit. 

Finally, the long list of pilot recommended modifications and the 

"highly desirable" and "desirable" ratings they assigned to these 

modifications argue against implementation of the system exactly 

as configured for the experiment. 

Wnat, then, might an appropriate SYNCALL system be like? It 

is difficult to derive a coherent recommendation for system 

design froffi only one type of data. However, when the flight performance data is considered together with the pilot rating data, 

the callout reliability data, and the percent overlap data, a 

consistent argument can be made in support of a general design 

for a synthesized voice callout system with certain variables 

still undefined or requiring experimental comparison among alternatives. Such a system, once configured, would then have to undergo testing using a methodology similar to that used in this 

study. 

S YNC/i.k1. llS t em CompoJl..en Ul 

While SYNCALL was designed as a single experimental system so 

as to test several different types of synthesized approach callouts, it was composed of differ~nt functional parts. These are 

listed below: 

1) normal altitude callouts e.g. 1000 AFL, 500, 400 ••• 

2) normal position callouts e.g. outer marker, G/S intercept, 

3) deviation callouts e.g. airspeed +15, left of localizer, 

4) critical altitude callouts, e.g. decision height, MDA 

5) priority logic 

6) callout timing and repetition logic 

7) tolerance values 

The experimental design tested the entire SYNC ALL systeffi compared to the entire PNF system of callouts. Thus no conclusions 

can be drawn from the results regarding the relative contribution 

of different functional components of each callout system to the 

observed flight performance, system performance, and pilot rating 

79 
differences. However, analysis of the data was done in ~ w~y 

that permits rational .l?-Q..eculatior), on the contribution of individu~l SYNGALL components to the observed differences. And the 

resulting speculations can oe used to derive further hypotheses 

for testing. 

When looking for the underlying reasons for the observed 

differences associated with the two systems, one naturally looks 

for diff~rences in performance between corresponding components 

of the two callout systems. Eacn of the seven system components 

listed above will be treated in turn. 

1) ~ql altitude callouts Both the PNF and SYNGALL made the 

altitude callouts reliably. Although reliability of PNF altitude 

callouts was slightly less (74%) than that of SYNC ALL altitude 

callouts (85%), there were no approaches for either system for 

which no altitude callouts were made. Neither system was 100% 

reliable in making altitude callouts since both the PNFand SYNCALL gave priority to calling deviations when faced with ~ demand 

for simultaneous altitude and deviation callouts. Furthermore, 

the reliability of normal altitude callouts made on the line 

(33" was nearly equal to that of SYNC ALL (35%) when the line 

callout reliability was adjusted to account for the high percentage of visual approaches on the line. Since both systems 

resulted in high reliability for altitude callouts, it is difficult to attribute any flight performance differences to any 

differences in the altitude callout component of SYNCALL versus 

PNF. Rather than introduce automatic normal altitude callouts 

when no particular benefit to flight performance would be expected, altitude callouts by SYNCALL if implemented at all ought to 

be designed as a back-up - i.e. SYNC ALL should make them only if 

the PNF is unable to make them or forgets to make them for some 

reason. It must be noted, however, that the pilots actually 

favored altitude callouts by SYNCALL more than they did SYNCALL 

deviation callouts. The discussion of deviation callouts, callout timing and repetition logic, and tolerance values below may 

shed some light on this apparent discrepancy. 

2) ~~ ~Ltion callouL~ SYNCALL was more reliable than 

PNF for these callouts. 

3) deviatior), callouts SYNCALL was more reliable than PNF in 

calling out deviations in airspeed, sink rate, localizer position, and glideslope position: SYNC ALL 90%, PNF 49%, where instances of deviations requiring callouts were counted rather than 

number of callouts since a gi~en deviation could conceivably 

result in multiple callouts. This difference between the performance of the two callout systems, then, suggests that the deviation callout component of SYNCALL was important to the observed 

i~provement in flight performance with SYNCALL. 

4) critical altitude callou~ A direct comparison of the two 

callout systems for this component is not possible because these 

callouts are required only if the PF has not announced that the 

runway is in sight by the time the critical altitude (DH or MDA) 

80 
is reached. SYNCALL could not detect whether this was the case, 

so it always called the critical altitudes. Thus a,comparison 

between PNF and SYNC ALL for critical callout reliability is not 

meaningful. 

5) priority logi~ SYNC ALL was designed with different priorities assigned to different types of callouts to cover situations 

when more than one callout was triggered. The criteria used for 

this priority assignment were 1) Make the most time-critical callout first, and 2) Make only less time-critical callouts afterwards only if they still apply. 

Decision height and MDA were judged prior to the experiment 

to be the most time-critical of all the callouts since the 

pilot-flying must act immediately on the information conveyed by 

-these callouts so as not to compromise the safety of the approach 

by descending below minimums when this is not warranted. 

The next highest priority was assigned to deviation callouts; 

and the deviation parameters were ordered within this from higher 

to lower priority as glideslope, indicated airspeed, sink rate, 

and localizer. 

Lowest priority was given to ~ormal altitude callouts and 

normal position callouts. Since these by definition were not expected to occur simultaneously, no priorities within these were 

assigned. In fact, on rare occasions localizer intercept, 

glideslope intercept, and/or outer marker passage occur simultaneously. 

This priority logic had an effect on the reliabilty of different types of callouts made by SYNCALL. The most reliable were 

the DR and MDA callouts (100~). Next in reliability were deviation callouts (90%). Lowest in reliability for SYNCALL were altitude callouts (83~). The callouts that SYNCALL made most reliably were the ones that were given the highest priority when the 

system was designed. It is noteworthy that even SYNCALL, which 

presumably ~~ect~ 100% of all the callouts that should have 

been made, did not have time to speak all the callouts in cases 

where critical altitude and deviation callouts were required. 

We have only anecdotal evidence regarding the priority system 

used by the PNF to decide which of several simultaneously required callouts to make. Very likely it is much more complex 

than the unidimensional one employed by the SYNC ALL system. It 

probably takes into account a wide range of real world contextual 

factors such as the type of approach being made, the type of terrain under the approach path, the weather, the wind conditio~s 

(above and beyond considerations made in computing the approach 

speed), whether or not the runway is in sight, and the amount of 

excursion beyond tolerance when deciding between multiple deviation callouts. Also, the PNF knows that deviations in certain 

parameters are associated with deviations in other parameters and 

that correcting one may effectively correct the other. Excessive 

airspeed and excessive sink rate exemplify this type of association. If one is present, the other may also be present, espe81 
cially on non-precision approaches. Whether the PNF calls 

airspeed or sink rate first will depend, then, on many factors. 

We cannot hope to duplicate the human's priority system so SYNCALL will function as the PNF does because we do not have all the 

informatio~, properly weighted, that the PNF brings to the situation. 

However, it was possible to observe, both on the flight deck 

and in the simulator, some general patterns of PNF behavior when 

confronted with the requirement simultaneous callouts. By noting 

what callouts the PNF made in these situations and when he made 

them, we can infer something about his priority system. 

Tne PNF employed a different timing strategy than SYNCALL for 

integrating altitude callouts and deviation callouts. Often a 

PNF deviation callout was made immediately after a 100-foot callout, presuillably to ensure that the 100-foot callout was made on 

time." SYNCALL, in contrast, called out a deviation as soon as it 

was detected and postponed any 100-foot callouts that were requir~d during ennunciation of the deviation callout. Further, 

such an altitude callout was skipped entirely if the postponement" 

would nave caused it to be spoken more than 50 feet late. Had 

SYNCALL employed the strategy used by the PNF, it might have been 

more reliable for normal altitude callouts with no detriment to 

deviation callout reliability. Tnis would have been at the ex~ 

pense of SYNCALL's deviation callout timeliness, however. 

6) callout timing ~~q repeLition JL~~ Callout timeliness is 

closely linked to callout timing and repetition logic. The PNF 

strategy seems to be to wait for 100-foot altitude intervals and 

to callout any deviations at these times, just after making a 

100-foot callout. SYNCALL, on the other hand, called a deviation 

as soon as it detected one and checked again in 5 s. Assuming a 

700 FPM rate of descent, the 5-s interval would be the same as 

checkin~ every 58 feet or about twice the rate of the PNF in 

terms of altitude loss. For a 1200 FPM rate of descent, which 

would trigger a SYNCALL sink rate deviation callout, the altitude 

loss between SYNCALL repetitions would be 100 feet, the same as 

for PNF callouts. Thus, only when sink rate itself was excessively high was the repetition rate by SYNCALL equivalent to that 

of PNF. Otherwise, SYNCALL made deviation callout repetitions 

for glideslope and airspeed twice as often as the PNF would have 

done. The approximately double repetition rate of SYNCALL deviation callouts compared to PNF callouts is a second difference 

between the two systems ihat could have caused the observed 

flig~t performance difference in airspeed and sink rate for the 

non-precision approaches. Whether increased deviation callout 

reliability or increased callout repetiton rate or some combination of these produced the improvement in flight performance with 

SYNCALL cannot be determined from the data since the two factors 

are confounded. The pilots did judge the system to have an unacceptably high rate of nuisance deviation callouts. If a synthesized voice approach callout system were to be designed for 

82 
further testing, serious consideration should be given to ways of 

reducing deviation callout repetition rates to a level that would 

not be judged a nuisance by the pilots. Nuisance callouts could 

also be reduced by changing the decision logic for triggering repetitions of a callout for the same deviation. Pilots commented 

that if they saw that the pilot-flying was in the process of 

correcting ,a deviation, they did not repeat it even if the actual 

value was still outside tolerance. Only if they observed no 

change in deviation or an increased deviation did they say they 

would call it again. The SYNCALL localizer and glideslope deviation callout repetition logic was in fact designed in exa6tly 

this way. The logic for the airspeed and sink rate deviation 

callouts was not. That is, SYNCALL made a repetition of the deviation after 5 s if the value was still outside tolerance regardless of whether the pilot was in the process of correcting or 

not. In addition to changing the rate of deviation callout repetitions, or perhaps instead of changing this rate, a greater 

reduction in nuisance callouts might be achieved by making the 

logic for airspeed and sink rate deviation callouts be the same 

as that used for the localizer and glideslope deviation callouts 

and that which the pilots report they use: repeat the deviation 

callout only if no correction is being made. This approach to 

reducing nuisance callouts would also be consistent with the 

finding that pilots were satisfied with the repetition rates for 

each type of deviation callout and yet wanted a reduction in nuisance callouts. 

The timeliness of SYNCALL's initial deviation callouts, however, could have been one of the major contributing factors to 

the improved flight performance with SYNCALL in that deviations 

un-noticed by the Pilot-Flying were brought to his attention immediately rather than at the next 100-foot interval of descent. 

Again, the data do not permit direct inference of this conclusion, but serious consideration should be given to preserving 

this aspect of SYNCALL if future systems are designed for testing. 

7) tolerance values SYNCALL was designed with tolerance 

values that were 25% to 33% greater than the "book" values used 

by the PNF. This was done in hopes of ensuring that the PNF 

would continue to watch for and callout observed deviations even 

when SYNC ALL was in use. Instead, the effect of SYNCALL on PNF 

deviation callout reliability was to reduce it from 49% to 24%. 

This suggests a potential problem, that of the PNF relying too 

heavily on SYNCALL to monitor for and make deviation callouts. 

Perhaps still larger differences between SYNC ALL and PNF "book" 

tolerances would have relegated SYNCALL to more of a back-up 

role. One would want to see PNF deviation callout reliability at 

least as good with SYNCALL as without SYNCALL in order to be certain that SYNCALL was not creating a new problem by its introduction. According to the pilot judgment data, the SYNCALL tolerances, particularly for airspeed, may indeed have been too tight. 

83 
A future system might be designed with the airspeed tolerances 

suggested by the pilots: +10 KTS and -5 KTS. 

Summary Qf Recommended Changes ~ SYNCALk 

On the basis of the analysis of the possible contributions of the 

different components of SYNCALL to the observed improvements in 

flight performance and in callout reliability, and to the ratings 

and suggested modifications by the pilots, a tentative SYNCALL 

system for further testing can be proposed. The system would 

make deviation callouts in airspeed, glideslope, localizer, and 

sink rate using the same tolerances for all but airspeed. The 

airspeed tolerance would be changed to +10 KTS and -5 KTS from V 

(APPR). This would give the following tolerances: 

GLIDESLOPE 

AIRSPEED 

SINK RATE 

2000 -1000 AFL 

1000 - 300 AFL 

BELOW 300 AFL 

LOCALIZER 

+1- 1 113 DOTS 

V(APPR) +10,-5 KTS 

2500 FPH 

1200 FPH 

1000 FPH 

+1-.1/2 DOT 

In addition, consideration should be given to including a 

callout and associate~tolerance for VOR course tracking and for 

Back Course Localizer course tracking. One could use the values 

suggested by one of the participating pilots as starting points. 

These were 1 1/2 dots for a Localizer Back Course and 2 dots for 

a VOR radial as final approach course. 

It will be remembered that for the experiment the functions 

of the GPWS were incorporated into the SYNCALL system on a nonduplicating basis. Any future SYNC ALL system ought to include 

the excessive terrain closure warning of the GPWS for consistency. While not used in this study, the "Sink on Take-off" functional warning of the GPWS would presumably also be incorporated 

into a modified SYNCALL system. The other GPWS modes, which all 

are designed to pertain to sink rate and glideslope in various 

aircraft configurations would be unnecessary because the information they are designed to convey and some additional information 

as well, e.g. actual sink rate values, aircraft above glideslope, 

is given by SYNCALL. In support of this merging of GPWS with 

SYNCALL there is the semantic differential data which indicates 

that the pilotL ~ound SYNCALL to be more informative, more helpful, more dependable, and more coordinating that either the GPWS 

Whoop Whoop Pull Up or the GPWS Glideslope message. Also, one of 

the 31 modifications suggested by the pilots was "Eliminate GPWS 

and Replace with SYNCALL". This item r~ceived a mean rating of 

2.15 +1- 0.90 and a median response of 2 (desirable). 

Merging SYNCALL and GPWS warning functions would provide the pilot with more information, and in particular would state the 

problem and the degree of the problem. It can be suggested but 

84 
not tested after the fact that such a system in place of the GPWS 

system could have alerted the crew of a 727 that accidentally 

landed sho~t due to unintentional excessive sink rate on an overwater approach despite the t~iggering of a Whoop Whoop Pull Up 

message from the GPWS. ·(6) 9 

The repetition logic for the proposed SYNCALL system should 

have the feature used by the PNF and by the experimental SYNC ALL 

system for glideslope and localizer deviation callouts; once a 

deviation has been called, it should not be called again unless 

the deviation is not being corrected. The time interval of 5 s 

before a recheck of airspeed, glideslope, or sink and 10 s before 

a recheck of localizer deviation would be expected to work well 

provided this change of logic is implemented. 

The priority assignment of the different deviations was, 

from highest to lowest, GPWS Terrain Closure, Glideslope, 

Airspeed, Sink Rate, and Localizer. Some of the pilots and two 

of the instructor pilots suggested that Sink Rate deviations 

should be given higher priority than Airspeed deviations. They 

thought this would be especially useful for the non-precision approaches, for which glide path information is not available. An 

experimental comparison of these two priority assign~ents will 

have to be made using precision and non-precision approaches and 

measuring flight performance, callout reliability, callout timeliness, and pilot ratings before one can be chosen over the other. 

Such an experiment should include more extreme conditions of 

wind, gusts, and wind direction and velocity changes (wind sheer) 

than was possible in this study. 

Once a particular type of deviation had been called, the next 

highest priority parameter should be checked before the first is 

checked again. Without this provision, a SYNC ALL system could 

conceivably call one type of deviation while ignoring others until that deviation was corrected. 

The proposed SYNC ALL system would not make normal altitude 

callout~. PNF callout reliability both in the experiment and on 

the line was, for all practical purposes, equal to that of SYNCALL for normal altitude callouts. Looked at the other way, SYNCALL was unable to improve on the performance of the pilot-notflying for normal altitude callout reliability; in part this was 

9. The GPWS in this accident was of the type that was designed 

to callout "Whoop Whoop, Pull Up" for any of four different 

problems: excessive terrain closure, excessive sink rate, sinking 

on take-off, and flaps or landing gear not in landing configuration. Newer models of this system have been designed to state 

the actual problem that triggered the warning, partially conforming to the suggestion here to merge GPWS and SYNCALL design 

features. One can also hypothesize that for the accident referenced above, the use of this other type of GPWS might have 

prevented the accident. 

85 
because SYNCALL placed higher priority than did the PNF on callin3 out deviations as soon as detected, often to the detriment 

of SYNCALL normal callout reliability. There exist, however, 

rare instances when normal altitude callouts by SYNCALL could actually perform the warnin3 function served by the deviation callouts. If for any reason, the crew is not aware of descent to 

low altitude and the PNF for that reason does not make the normal 

altitude callouts, SYNCALL could alert them to this by calling 

out altitudes only on these rare occasions. This could be 

designed into SYNCALL by having it automatically armed at the 

start of each ap~roach to make warning altitude callouts as follows: 

Callout Wording 

BELOW 1000 FEET 

BELOW 500 ?EET 

BELOW 100 FEET 

AFL Altitude 

goo feet 

400 feet 

80 feet 

These warning altitude callouts would be made by SYNC ALL only if 

an abnormal descent in terms of sink rate was occurring. Abnormal sin~ rate would be defined as either excessively high ~ excessively low sink rate. Appropriate values cannot be given 

here. As a starting point, one could use sink rate in excess of 

SYNCALL sinx rate deviation tolerances or sink rate less than 100 

FPM to cover unintentional gradual descents. It must be emphatically stressed that this part of the proposed SYNCALL system is 

extremely tentative and is derived from general human factors 

design principles to avoid false alarms and not from the data 

collected and analyzed in this study. A far better way of obtaining sink rate values to use as activators for warning altitude callouts by a SYNC ALL system would be to analyze sink rate 

histories from a large sample of different types of approaches 

that were terminated successfully and compare the resulting means 

and standard deviations to sink rate histories for approaches 

that resulted in controlled flight into terrain (CFIT). Unless a 

combination of values could be found that would virtually eliminate false alarms while virtually guaranteeing that all instances of unintentional descent would be called out, such a 

feature should not be incorporated into an automatic approach 

callout system such as SYNCALL. 

The Ten-Foot normal altitude callouts deserve special mention, however. Unlike the rest of the normal altitude callouts, 

SYNCALL was definitely more reliable than PNF for these callouts 

(100~ compared to 77%). Tnis was noted independently during and 

after the simulator session by some of the participating pilots. 

In some case, pilots said they felt they were late making tenfoot callouts and that the other pilot made the ten-foot callouts 

late. Also, captains noted that despite a procedural requirement 

that they make ten-foot callouts as pilot-not-flying, that they 

86 
as pilot in command felt obliged to look outside the cockpit during the flare and touchdown. 10 Interestingly, there were no 

differences in ten-foot callout reliability as a function of position flown. Rather, both captains and first officers simply 

were not as reliable as SYNCALL in making these callouts. Assuming that the timing of SYNCALL ten-foot callouts could be made 

acceptable, such callouts could be an effective means of normal 

altitude information transfer at a point in the approach where 

SYNCALL would not be making deviation callouts. The timing of 

SYNCALL ten-foot callouts might be improved an algorithm utilizing sink rate to remove the inherent delay in the altitude callout. 

For purposes of design consistency and simplicity from the 

viewpoint of the user, SYNCALL should not make normal position 

callouts. Even though SYNCALL was more reliable than PNF for 

this type of callout, inclusion of such callouts in a SYNCALL 

system would possibly dilute the warning function of the ~ystem 

in that pilots would then not associate SYNCALL callouts strictly 

with warnings. Also, SYNCALL was not able to make normal position callouts with 100% reliability. This might tend to lessen 

pilot confidence in the system. 

The question of critical altitude callouts (Decision Height, 

MDA, 100 above MDA) is difficult. On the one hand, these are 

warnings only if the pilot-flying does not have the runway in 

sight. If the pilot-flying has announced the runway in sight, 

then the decision height or MDA callout and Missed Approach Point 

(MAP) callout will not be needed. The latter would also be extremely difficult to design into a SYNCALL system unless there 

were access to Inertial Navigation system or RNAV system data. 

On the other hand, SYNCALL was much more reliable than PNF for 

these callouts (100% compared to 58%, 48% and 52% respectively 

for 100 above MDA, MDA, and Decision Height callouts by PNF). In 

those instances when the decision height callout serves its function and the pilot-not-flying (for monitored approaches) or 

pilot-flying (for standard approaches) initiates a go-around, it 

is critical that the callout be made in a timely and reliable 

fashion. Were a SYNCALL system to make decision height and MDA 

callouts in a more timely way than the PNF is able to do, it 

might be useful to have SYNCALL make such callouts for all approaches with the provision that approach procedures be modified 

to have the pilot-not-flying callout "Going Around" if SYNCALL 

says Decision Height and the Pilot-Flying has not announced the 

runway in sight. In the case of MDA, possibly giving the task of 

10. Sometime after completion of the experimental approaches 

for SYNCALL and following informal presentation of these comments 

to the flight training department of the participating airline, 

the airline's procedures were modified to permit the captain as 

pilot-not-flying at his discretion to omit the ten-foot callouts. 

87 
calling out MDA to SYNCALL could make the task of monitoring for 

the MAP easier for the PNF. This entire discussion of Critical 

altitude callouts by SYNC ALL and possible changes to approach 

procedures is presented here to stimulate further thought on the 

question and not to advise for or against implementation of critical altitude callouts in any future SYNCALL systems for testing. 

SummarV·QL Proposed Modifications ~ SYNCAk~ 

To summarize, it is suggested on the basis of the findings 

from this study that SYNCALL be re-configured essentially as an 

approach warning callout system and be merged with the GPWS as 

one entire system. For the system to be effective, the recommendations to eliminate nuisance alarms will have to be followed. 

Also the repetition logic will have to be changed. Finally, on 

the basis of the detrimental effects observed due to inappropriate airspeed deviation callouts during the single-engine approach, no inappropriate deviation or other warning callouts can 

be tolerated. If these cannot be designed out of the system, or 

at best designed with a pilot-selectable deactivation switch, 

then they should not be included as part of it at all. The 

resulting system will then have to be tested in a flight simulation study similar in comprehensiveness of systems and approach 

environment to that used for this study. 

88 
REFERENCES 

1. National Transportation Safety Board: Special Study: 

Flightcre~ Coordination Procedures in Air Carrier Instru~ent 

Landing System Approach Accidents. NTSB-AAS-76-5, 1976. 

2. Simpson, C.A.: Effects of Linguistic Redundancy on Pilots' 

Comprehension of Synthesized Speech. Procedings of the 

Twelfth Annual Conference on ~anual Control. NASA TM 

X-73,170, 1976, pp. 294-308. 

3. Curry, 

cedure 

no. 5, 

R.E.; Kleinman, D.L.; and Hoffman, W.C.: A Design Profor Control/Display Systems. Human Factors, vol. 19, 

1977, pp. 421-436. 

4. Williams, D.H.; and Simpson, G.A.: A Systematic Approach to 

Advanced Cockpit Warning Systems for Air Transport Operations: Line Pilot Preferences. Procedings of the Aircraft 

Safety and Operating Problems Conference, NASA Langley 

Research Center, Ha~pton, VA, October 18-20, 1976. NASA 

SP-416, 1976, 617-644. 

5. Osgood, C.E.; Suci, G.J.; and Tannenbaum, P.H.: The Measurement of Meaning. University of Illinois Press, 1957. 

6. National Transportation Safety Board: Aircraft Accident Report: National Airlines, Inc., Boeing 727-235, N4744NA, Escambia Say, Pensacola, Florida, May S, 1978. Report Number 

NTSB-AAR-78-13. 

7. National Transportation Safety Board: Aircraft Accident Report: Eastern Airlines, Inc., L-I011, N310EA, Miami, Florida, 

29 December, 1972. Report Number NTSB-AAR-73-14. 

89 
ALTITUDE AND APPROACH CALLOUTS BY SPEECH SYNTHESIZER 

Carol.A. Simpson, NRC Associate 

NASA Ames Research Center 

In preparation for a flight simulation study of synthesized speech 

Approach Callouts in an American Airlines DC-10 flight training Simulator, 

phoneme codes for the r'IL-l Speech Synthes ize·rl .lere programmed to develop a Vocabulary of words and short phrases that .could be appropriate-I 

ly concantenated to produce all of the required callouts including 

readout of actual altitude, airspeed, and rate of sink. Callout 

messages composed from this Vocabulary were then tested for intelligibility. 

In order to maximize the chances for misunderstanding of any messages 

that might be of low intelligibility, the testing conditions were 

chosen so as to be more difficult than any of the conditions that were 

expected to actually occur in the flight simulator. Backe;round 

noise shaped to simulate DC-10 cockpit noise was presented at a 

level 10 db above the level of the loudest speech see;ment in the 

Approach Callout messages. The airline pilots who attempted to 

understand the callouts were given no prior exposure to the callouts 

and were told only that the messages they would hear were similar 

to the callouts made durine; the approach by the pilot-not-flying. 

The pilots repres~nted several different airlines and thus were 

not necessarily familiar with the particular set of callouts or 

the phraseology used in this study. The results are presented in 

outline form on page 3. The mean articulation score of 90.4% correct 

under these extremely difficult testing conditions indicates that 

this set of cullont rneGsuGcG is very. intelliGible and sUSGests that 

these messaGcs would be acceptable for use in the flight simulator. 

lManufactured by Vocal Interface Division, Federal Screw Works, Troy, 

Michie;an 

-A 1-
As a further test of the individual words used to compose altitude callouts, 

a separate study was conducted with messuGes consisting entirely 

of altitudes ranging from Flight Level 440 to 10 feet radio altimeter. 

Two pronunciations of the digits for the altitude callouts were tested, 

a "phonetic" pronunciation and a "nonnal" pronunciation. In addition, 

pilots were asked to give their preferences for type of pronunciation 

and also for altitude phraseology, e.g. one three thousand versus 

thirteen thousand for 13000. These results are presented on pages 

4 and 5. The high articulation scores of 94.7 to 100% correct 

at a siN of -10 db also suggest that these individual words are highly • 

intelligible and adequate for use in the simulator. No significant 

difference in intelligibility was obtained for the two types of pronunciation. 

A majority of the pilots, however, preferred the "normal" pronunciation 

over the "phonetic" pronunciation, e~{Cert for the d:ic,it (), which a 

IIJ.::l.jority of pJ.lots JJreferred. to jJavc ·pronounced as "niner". Therefore, 

it is recommended that "normal" pronunciation be used for all digits 

except "niner". For altitude phraseology, nearly all pilots tested 

preferred the standard ATC phraseology, e.g. one three thousand for 

13000 and one three hundred for 1300. It is therefore recommended that 

the synthesizer follow ATC phraseology rules in making altitude 

callouts. 

Finally, pilot preferences for speech rate, voice pitch, and signal-tonoise ratio for this synthesizer were obtained. These results are presented 

on page 6. In general, pilots preferred a voice pitch in the 

middle of the male voice pitch range with no significant differences 

in voice pitch or speech rate as a function of presence or absence 

of cockpit noise. The mean preferred signal-to-noise ratio was -1-").J1 db. 

It should be noted that the standard deviation of these preferences 

was quite large, 7.6 db. In order to accomorlate those pilots requiring 

a hf.gh s:ienul-to-nojse :r:ntjo, :it is recommended thut the callouts 

be presented at u s.ignnl-to-noise ratio of 8 db, slightly less than 

the mean + 1 standard deviation. 

-A 2-
Approach Callouts at siN = -lOdb: Intelligibility in DC-IO cockpit noise 

stimuli: Standard American Airlines DC-IO Approach Callouts presented over headphones with simulated DC-IO cockpit noise 

as background; SiN = -10 db. Each Callout presented 

once only. No prior familiarization with the messages. 

Subjects: 7 Airline pilots 

Results: 

Percent Word Articulation for 27 callouts containing a .. total of 97 words 

Pilot Number Age Word Articulation = Percent Correct 

1 31 90.7 

2 34 95.9 

3 43 88.7 

4 33 100.0 

5 39 89.7 

6 37 88.7 

7 42 79.4 

Mean 37 90.4 

-A 3-
Phonetic versus Normal Pronunciation: Intelligibility Comparison 

stimuli: Altitude readouts ranging from 10 feet to Flight Level 1+40 

presented over headphones with simulated DC-IO cockpit noise 

as background; siN = -lOdb. 

Subjects: 5 airline pilots 

Test ~terial: The "Phonetic" and "Normal" sets contained identical 

altitudes and differed only in the pronunciation of the digits 

3, 4, and 5. The digit 9 was pronounced "niner" in both sets. 

R~sults: 

Digit 

3 

4 

5 

o 

-' 

Phonetic Pronunciation 

tree LtriJ 

fower [fowL~] 

fife [fajfJ 

niner [najnc;] 

Normal Pronunciation 

three [Gri] 

four (f1':::-] 

five [fajv] 

nine [najnJ] 

Percent Word Articulation for 2 sets of 24 altitudes containing 

a total of 171 words 

Pilot Number "Phonet:;c Pron!mciation" "Normal Prol1'mciation" 

Set 1 Set 2 Set 1 Set 2 

1 96 .6 100.0 100.0 97.7 

2 94.0 100.0 100.0 100.0 

3 100.0 100.0 100.0 100.0 

4 83.1 100.0 100.0 100.0 

5 100.0 100.0 98.8 100.0 

Mean 94.7 100.0 99.8 99.5 

Analysis: There was no significant difference in aritculation scores 

for the two types of pronunciation. 

* All data were obtained using a VOTRAX ~rrJ-l voice synthesizer, manufactured 

by Vocal Interface DiVision, Federal Screw Works, Troy, Michigan 

..;.A 4-
"Airline P1lot Preferences for Pronunciation of Altitude Readouts 

Subjects: 12 airline pilots 

Results: 

Digit Phonetic Pronunciation Normal Pronunciation Either One OK 

3 1 10 1 

4 3 8 1 

5 3 9 0 

9 8 2 2 

Altitude A'ro Phraseology+ Common Phraseology + Either One 

13000 11 1 0 

1300 12 0 0 

Numbers are number of pilots responding in each category 

Conclusion: A majority of airline pilots prefer normal pronunciation for 

the digits 3, 4, and 5 and phonetiC pronunciation of the 

digit 9 as niner. They prefer the standard A'ro phraseology 

for altitudes-over the Common phraseology, e.g. one three 

thousand over thirteen thousand and one thousand three hundred over thirteen hundred. 

+ ATC Phraseology = one three thousand and one thousand three hundred 

Normal. Phraseology ~ thirteen thousand and thirteen hundred 

-A 5-

OK 
Airline Pilot Preferences for speech rate, voice pitch, and signalto-noise ratio for the VotraxML-l Speech Gynthesizer 

stimuli: Synthesized Altitude and J\ppr08ch Callout r.Iessagcs presented 

over headphones ,\-lith and without simulated DC-IO cockpit noise as 

background. 

GUIJ.iects: CornmercJal airline pilots 

Procedure: For speech rate and voice pitch preferences, each pilot 

adjusted the appropriate knobs (continuous analog adjustment) first 

with no noise, then with background noise while listening to the 

Callout messau,es repeated 10 times each. For sir;ml-to-noise 

preferences, pilots told experimentor to adjust the. message level 

up or down until satisfied "lith the level of the messages relative to 

the noise level. Experimentor made adjustments in 10, 5, and 1 db 

units using a calibrated attenuator. 

Results: 

Mean Stand. nev. 

Iillob Setting Measured Freq. Kb st. F .". N 

Rate 5.7 ~ 1.1 13 

No Noise 

Pitch 5.3 91 Hz :t 1.0 +9 Hz 13 

-8 Hz 

With DC-IO Rate 5.6 ± 1.3 13 

Cockpit +7 Hz Noise Pitch 5.6 92 Hz ± 0.5 13 

-3 Hz 

siN +3 .l~ db ± 7.6 db 9 

* Note: The relatlonship between units marked on the pitch control 

knob of the VOTRJ\X and the resulting fundamental frequency of the 

synthesized voice is nonlinear. 

-A 6-
VOCABULARY OF HORDG AND PHRf\SES US:rn TO GENERI\TE APPROACH CALLOUTS 

Item Number 

5. 

6. 

7. 

8. 

9. 

10. 

II. 

12. 

13. 

11+. 

15. 

16. 

17. 

18. 

19. 

20. 

2l. 

22. 

23. 

2h. 

25. 

26. 

27. 

28. 

29. 

30. 

Vocabuary Item 

zero. 

one 

tvlO 

three. 

four 

five 

sty. 

seven 

eight 

niner 

ten 

eleven 

twelve 

thirteen 

fourteen 

fifteen 

sixteen 

seventeen 

eie;hteen 

nineteen 

hundred 

thousand 

twenty 

thirty 

forty 

fifty 

~ixty 

seventy 

eir;hty 

ninety 

-A 7-
Item Number 

31. 

32. 

33. 

31~ • 

35. 

36 • 

37. 

38. 

39. 

~o. 

41. 

1~2 • 

Ii3. 

41~ • 

45. 

~6. 

h7. 

48. 

h9. 

50. 

51. 

52. 

·53. 

54. 

55. 

VOC!lBUTAHY ITEr;![) (cont.) 

Vocabulary Itr-.:m 

Flight Lc"rcl 

1<13L 

Above Field 

feet 

minus 

plus 

/messagc-initial pause/ 

/phrase or message-final pause/ 

Rate of climb is 

You're above thc glidcslope. 

You're beloH the glideslope. 

You are right of the localizer. 

You are left of the localizer. 

and 

Whoop Hhoop! Pull Up! 

Passing through 18000. 

Leaving 11000. 

Glideslope intercept. 

Localizer lntcrcept. 

Outer Marker 

Decision Height 

100 feet above r-IDA. 

Sink is 

J.1DA 

Airspeed 

-A 8-
APPENDIX B 

Flight Scenario for One of Four Experimental Orders 

Captain: A. Takeoff 

B. ATC Departure 

C. Holding and/or Steep Turns and/or Stalls 

(Instructor Pilot's discretion) 

D. Vectors for Approach 

1. (SYNCALL) Backcourse 5R (10 KTS cross-wind) 

with complete missed approach. 

Vectors or reset to: 

2. (SYNCALL) ILS DUAL LAND approach and landing 

(10 KTS cross-wind) 

E. Takeoff Engine Failure between V1 and V2 (no wind) 

Vectors or reset to: 

3. (SYNCALL) Two-engine ILS to 100 feet (10 KTS . quartering wind) and ~issed approach. 

Vectors or reset to: 

F. Two-engine landing. 

F/O: G. Takeoff 

H. ATC Departure 

I. Holding and/or Steep Turns and/or Stalls 

(Instructor Pilot's discretion) 

J. Vectors for Approach 

1. (SYNCALL) Backcourse 5R (10 KTS crosswind) 

with complete missed approach. 

Vectors or reset to: 

2. (SYNCALL) Coupled ILS APPCH ONLY and land 

(no wind) 

K. Takeoff engine Failure between V1 and V2 (no wind) 

Vectors or reset to: 

3. (SYNCALL) Two-Engine ILS to 200 feet and land 

(10 KTS quatering wind) 

-B 1-
L. Takeoff and vectoring for approaches 

Captain: 4. (PLTNFLY) VOR to 7L/R using Raw Data with 

rejected landing at 10 feet.(10 KTS 

crosswind) Vectors or reset to: 

5. (PLTNFLY) ILS DUAL LAND to SINGL LAND to goaround (10 :<TS ',-lind) Vectors or reset 

to: 

FlO: 4. (PLTNFLY) vaR to L/R using Raw Data with 

rejected landing at 10 feet. (10 KTS 

crosswind) Vectors or reset to: 

5. (PLTNFLY) ILS DUAL LAND to SINGL LAND to 

APPCH ONLY and go-around(10 KTS wind) 

Captain: M. Engine Failure or shutdown, then: 

N. Engine Fire, the Vectors or reset to: 

6. (PLTNFLY) One-Engine Landing (VFR and no wind) 

Notes: SYNCALL = Synthesized Voice Approach C~llout System 

PLTNFLY = Pilot Not Flying Callout System 

The above scenario shows the assigment of pilot-flying 

and callout system used to Presentation Order 1. 

Presentation Orders 2, 3, and 4 were derived by altering 

the pilot to fly first (Captain or First Officer) and 

the callout system to be used. first (SYNCALL or PLTNFLY) 

The order of the approaches flown: (1, 2, 3, 4, 5, 6) 

was constant for all Presentation Orders in accordance 

with American Airlines' recurrent training syllabus. 

-9 2-
PILOT EVALUATION OF APPROACH CALLOUT SYSTEMS 

The speech synthesizer was programmed to make all of the altitude and 

deviation approach callouts. This was done to give you a chance to 

experience the range bf capability of such a system. In your own professional judgment, certain types of callouts, either by the speech 

synthesizer or by the pilot-nat-flying, may be more or less desireable. 

The next two pages contain "preference grids". The different types of 

callouts are listed down the side - first for SYNCALL, the speech 

synthesizer, then (on the next page) for the PILOT-NaT-FLYING. Also 

4 different types of approach conditions are listed across the top 

of each preference grid: 

1) Night VFR 2) Abnormal or 

Emergency 

3) Day VFR 4) IFR to r~inimums 

Starting with the grid for SYNCALL, put a check in the appropriate box 

for each type of callout that you think would be useful and desireab1e 

for the type of approach listed. Do this for each type of approach 

condition. Then repeat the process for the PILOT-NaT-FLYING grid. 

Check as many or as few ca110uts for each type of approach condition 

and system as you want. If you want no ca110uts for a particular, type 

of approach and system, check the lowest box, "none". 

-C 1-
SYNCALL 

---. I I~R_ , Night VFR Abnormal/! Day VP'R to I Emergency! m~n~mums ! 

I 

Glideslope Intercept ! j 

1 

I . . Localizer Intercept ! I 

: ~ . ; 

OM and Altitude AFL I _ .... _._ .... - I ~ . 1 -- 1000 Feet AFL i I I , - 100 Feet Above HDA I I , .......... i I 

MDA i • I 

-. , ----' .. ~.-- I 

Decision i Height ! 

I I I I 

! i 

~ 500 Airspeed I Feet, & : ! Sink Rate • I j 

FrunorecrFOotCalToutsi. -_. 

I 

I 

I 

Ten foot Callouts ! 1 i l 

G/S Deviations ! I I 

I 

LOC Deviations ..,. I 

Sink Rate Deviations f 

- Airspeed Deviations 

---_._- - >--._-----

J.'{one (no ~llouts) .L~ 

-C 2-
PILOT-NOT-FLYING 

Night VFR IFR to 

• 

VFR!Abnormal/! Day I Emergencyl minimu.rns i 

Glideslope Intercept I I 

i 1 ~-----------'-.----------.-+-------------;--------~------------~------~ Locatizer Intercept i 

I 

OM and Altitude AFL 

-- t 

1000 Feet AFL ! 

100 j 

Feet Above HDA ! 

I 

MDA ! 

I 

Decision Height I 

, I 

500 Feet, Airspeed & I 

Sink Rate I 

Hundrea-FOot Ca [louts! 

Ten Foot Callouts 

~-I 

! -

I I 

I 

____________________ .4 ____________________ ~--------~--------~ 

G/S Deviatioris 

LOC Deviations 

Sink Rate Deviations 

Airspeed Deviations 

f------------.--oof--.----+--------l------I--------i i.'ione (no ca llouts )_ I 

-c 3-
Assume for the moment that the PILOT-NOT-FLYING callouts were modified 

to be exactly the waf you s~ecified in the preference grid, Rate your system on the 10110w n9 sea e by plAcing an "X" in the ~ppropr1ate space, 

unsafe no 

preference 

, 

, --- h .... i-.gh ..... ,-ydes1reable 

Now assume for the moment 'that SYNCALL were implemented exactly the ~ 

you specified in the preference grid, Rate your system on the scalelOelow. 

. , 

-un-s-a""fe-- ---------no---· ... · -------'highly 

preference' desireable 

Now assume t~~ PILOT-NOT-FLYING callouts remain the same as they are now. 

Rate the current system of pilot-not-flying callouts on the scale below. 

unsafe no 

preference 

, 

---'highlY 

desireable 

Now assume. that the SYNCALL system were implemented exactly as you experienced it today in the simulator. Rate this system on the scale below, 

, 

--un~s--a"'ll'fe-- ----' ----- no --------'highly 

preference desireable 

On the next four scales below, rate the intelligibili~ of the four types of speech which you just heard in the simulator. 

SYNCALL 

• 0% ' -..,--------' 50% 100% 

LA TRACON 

0% 50% 100% 

LAX TOWER 

" , 

0% 50% 100s 

PI LOT-NOT-FLYING 

as 50i 1001 

C 1\ 
If a SYNCAll system were installed in the cockpit, it might be desireab1e 

to build in some limits on the conditions for the different callouts. 

listed below are several technologically feasible constraints for such 

a system. Using the following scale, rate each suggestion by placing a 

1, 2, 3, 4, or 5 in front of it. 

1 Highly desireab1e 

2 Desireab1e 

3 No preference 

4 Undesireab1e 

5 Unsafe 

Deviation ca1louts not activated until 500 feet above field and 

-lower. 

_ Hundred Foot callouts activated only for coupled approaches. 

Hundred Foot call outs· can be deactivated by Captain at Captain's ----- discretion for VFR approaches. 

_ Glideslope deviations can be deactivated at Captain's discretion 

for VFR approaches. 

(In the blank spaces, add any limits that you would want to impose 

on such a system and rate your suggestions with the same scale. 

-C 5- -
PILOT RECOMMENDATIONS FOR MODIFICATION OF SYNCALL 

Introduction 

The tolerance windows for the SYNCALL deviation callouts were picked as "ball 

park" values so as to test the general concept of an automatic approach callout 

system. I figured the values might need changing once a group of line pilots 

had actually flown SYNCALL in the simulator. When you were flying SYNCALL. you 

may have thought the deviation callouts were made too frequently or maybe not 

often enough and that the allowances on airspeed. sink rate, glideslope. and 

localizer were either too loose or too tight to work well in the real world. 

I would like to get your recommendations for setting the tolerances for each 

type of deviation callout. And I would like to also get your recommendations 

for the priority assignments to each type of callout. Please answer each of 

the following questions based on your short experience with SYNCALL and on your 

extensive experience flying the line. And feel free to add comments ! 

If you have any questions about what I'm asking, please call me (collect) at 

NASA. When you call. just say that you were in the SYNCALL study. When you 

have finished filling this out you can use the stamped. self-addressed envelope 

to mail it back to me. 

Return to: 

Carol A. Simpson, PhD 

Hail Stop 239-2 

NASA Ames Research Center 

Moffett Field, CA 94025 

Call collect: 415/965-6128 

-0 1-
1. SYNCALL tolerance "windows" 

The chart below shows the tolerance "windows" that were set for SYNCALL when you flew 

it in the simulator. In each case, the "window" is wider than the tolerances you have 

in your operating manual for the pilot-not-flying and the flight engineer to use when 

making deviation callouts. 

AIRSPEED Approach ("Bug") Speed + 7 KTS 

SINK RATE Bebteen 2000 and 1000 AFL 2500 fpm 

Between 1000 and 300 AFL 1200 fpm 

Below 300 AFL 1000 fpm 

LOCALIZER 1/2 dot 

GLIDESLOPE 1 and 1/3 dot 

In the chart below please fill in the values that ~ think SYNCALL should use. Note 

that for the airspeed you don't have to choose the same amount on the negative side 

as on the positive side. For example, you could choose -20 and +3. Keep in mind that 

you want to avoid nuisance callouts but that you want the system to tell you about any- thing really dangerous. 

If you would not want SYNCALL to make a particular type of deviation callout at all, 

regardless of the size of the "window", please put a check in the box at the far right of the chart on the line corresponding to that type of callout. Note, for example, that you might want SYNCALL on sink rate for only some but not all the altitudes listed • 

TYPE OF CALLOUT 

AIRSPEED Approach ("Bug") Speed 

SINK RATE Between 2000 and 1000 AFL 

Between 1000 and 300 AFL 

Below 300 AFL 

LOCALIZER 

GLIDESLOPE 

-0 2-

"WINDOW" 

./ IF DON'T WANT 

THIS TYPE CALLOUT 
2. Deviation callout repetition rates and "active" segments 

The chartbe10w shows the rates at which the different SYNCALL deviation ca1louts were 

repeated. For example, airspeed deviations were repeated once every 5 seconds as long 

as the aircraft remained outside the "window". The chart also shows the point in the 

approach where each callout became "active". At that point SYNCALL would start checking for that type of deviation. Before that SYNCALL would not make any callouts even if 

the aircraft was outside the "window". For example, SYNCAlL started checking for sink 

rate deviations at 600 feet AFL. Finally, the chart shows the point where SYNCALL was 

automatically "turned off" for each type of deviation so as to avoid distractions at 

low altitude. For example, SYNCALL stops checking for glides10pe deviations at 100 feet 

AFl. 

TYPE OF CALLOUT REPETITION RATE START CHECKING STOP CHECKING 

AIRSPEED 5 sec. 600 AFL 100 AFL 

SINK RATE 5 sec. 600 AFL 100 AFL 

LOCALIZER 10 sec. at LOC CAP 100 AFL 

GLIDESLOPE 5 sec. at GIS CAP 100 AFl 

In the chart below, please fill in the values that ~ think SYNCALL should use for 

repetition rate, start checking, and stop checking.~ on page 1, if you would not 

want SYNCALL to make a particular type of deviation callout at all, regardless of the 

repetition rate or the "active" segment, please put a check in ~he box at the far 

right side of the chart on the line corresponding to that type of callout. 

tI IF DON'T WANT 

TYPE OF CALLOUT REPETITION RATE START CHECKING STOP CHECKING THIS TYPE CALLOUT 

AIRSPEED 

SINK RATE 

LOCALIZER 

GLIDESLOPE 

-D 3-
3. SYNCALL Priority System 

SYNCALL systematically missed certain callouts for certain approaches due to its priority 

system and the timing of the callouts. For example, on a VOR 7R Approach, MDA was 436 AFL. 

So SYNCALL called "100 feet above MDA" at 536 AFL and did not make the 500 AFL callout. 

If a deviation callout coincided with an altitude callout, SYNCALL made the deviation 

callout and skipped the altitude callout. The chart on the left lists the types of SYNCALL 

callouts. including the GPWS Whoop Whoop for Terrain (Mode 2), in their order of priority from highest to lowest. If you would want a different order of priorities, please list 

the types of call0uts in the order you would want in the chart in the middle. If there 

are any of these call0uts that you would not want SYNCALL to make at all, please list them 

in the chart to the right. If, on the other hand, you like the priorities the way they 

are. please check the box below the three charts. 

3: 

o 

.... 

.If 

Current Priorities 

DH or MDA 

GPWS Mode 2 (Terrain) 

G1ideslope Dev. 

Airspeed Dev. 

Sink Rate Dev. 

Localizer Dev. 

Altitude Ca110uts 

Your Recommended Priorities 

L:::J Leave Priorities as they are 

4. Suggested Modifications to SYNCALL 

SYNCALL should not 

make these ca110uts 

Listed below are all the suggestions given by one or more pilots who flew SYNCALL. Please 

consider each one and give it a rating using the scale at the top of the list. Put your rating for each suggestion on the line just to the left of that suggestion. For example, if you find a particular suggestion "highly desirable", put a "1" on the line next to the 

suggestion. If you find it "undesirable", then put a "4" beside it. Do this for each 

suggestion. After you finish, go back and pick out any and all of the suggestions that 

for you would be essential in order for you to be willing to accept SYNCALL - i.e. if 

SYNCALL were not modified according to these particular suggestions, then you would not 

want it in your airplane. _ 

Unsafe Undesirable 

5 4 

---------- ---

.. here if 

Essential 

Put your Rating here 

(1.2.3.4.5) 

Note that the list continues on the next page. 

RATING SCALE 

No Preference 

or Can't Oed de 

3 

Desirable Highly Desirable 

2 

-'-----------------------------------

SUGGESTED MODIFICATIONS TO SYNCALL 

o __ Have ON/OFF switch in case of erroneous info or distracti on from SYNCALL 

-D 4-
Suggested Modifications to SYNCALL continued. 

RATING SCALE 

unsafe undesirable no preference 

or can't dec1 de 

desirable highly desirable 

_. __ ~ __ . _____ ._~ _____________ ~. ______ . ________ 2 _ .. _________ ._ 

-----Put your 

V here if rating here 

essential (1,2,3.4,5) SUGGESTED MODIFICATIONS TO SYNCALL 

CJ 

o 

CJ 

o 

o 

D 

CJ 

o 

D 

D 

0' 

CJ 

o 

C1 

o 

o 

o 

o 

o 

D 

o 

Provide Test switch for accuracy check 

Minimize frequency of deviation callouts 

Delay callouts during radio transmissions (if possible) 

Eliminate GPWS & replace with SYNCALL 

Add a warning in case airspeed gets below Alpha Speed 

Shorten message wording 

Speed up SYNCALL's speech rate 

Eliminate nuisance callouts 

Add other warnings, e.g. engine fire, loss of cabin pressure, etc. 

Add immediate action checklist jtems 

Provide manual volume control 

Increase volume 

Increase tolerance on localizer deviations 

Increase tolerance on glides lope deviations 

Increase tolerance on airspeed deviations 

Spell out SYNCAlL and pilot-not-flying responsibilities more clearly 

At the 50 ft callout, callout airspeed and sink. Then eliminate 

the 40, 30, 20, 10 callouts by SYNCALL 

Inhibit airspeed and sink rate deviation callouts during abnormal or 

emergency approaches 

Automatically change SYNCALL parameters depending on type of approach 

Call Localizer capture, then Localizer Intercept 

Read out Flight Guidance Mode Annunciators after LAND mode has 

been armed 

(continued on next page) 

-0 5-
Suggested Modifications to SYNCAll continued -

RATING SCALE 

unsafe undesirable no preference 

or can't dec1 de 

desirable highly desirable 

5 4 3 2 

vlhere if Put your essential rating here 

(1,2,3,4,5) 

o 

o 

o 

o 

D 

o 

o 

CJ 

o 

Call out NBe10w MDA" if aircraft descends below MDA (unless pilot 

has pressed a "disab len switch) 

On non-precision approaches, give sink rate deviations top priority 

Obtain FAA & Company agreement not to use CVR recordings of SYNCAll 

against the crew 

Have SYNCAll call the flaps setting at 1000 AFl 

Have SYNCAll say "Check Flaps" at 1000 AFl 

Make voice quality for deviations more alerting compared to altitude calls 

Have SYNCALl make deviation callouts only (i.e." only if something's wrong) 

Have SYNCAll make altitude ca1louts only (leave deviations to pilot-notflying and flight engineer) 

If aircraft balloons on landing, have SYNCALL repeat 10-foot callouts 

as appropriate, e.g. "50, 40, 30, 20, 30, 20, 10" 

5. Background Data 

Please fill in the chart below 2Dlx for the type aircraft you were current in at the 

time that you flew SYNCALl 

TYPE AIRCRAFT POSITION FLOWN APPROX. NO. HOURS IN TYPE 

-0 6-
I 

o 

........ 

I 

6. Preference Grids for desired callouts for different types of Approach Conditions 

The two preference grids below are like the ones you filled out after flying SYNCALL in the simulator. Please fill 

them out using the same instructions as before. That is, put a check in the appropriate box for each type of callout 

that you think would be useful and desirable for the type of approach listed. Do this for each of the four types of 

approach conditions. By~ng a box blank, you are saying you don't want that type of callout for that type of 

approach or that you are not sure_~hatyou would want it. Check the box only if you really would want SYNCALL to make 

that callout for that type of approach. ..... .. _ There are two preference grids. Fill outth~.one .. on_theJeft. assuming SYNCALL stayed the way it was when you flew it 

in the simulator. Then f111 out the one on the right assuming SYNCALL was changed according to your own recommendations 

on the preceeding pages of this evaluation. 

ust-liiis GRID FOR S'/IICAll THE VAY IT liAS IIIEII YOU FUll IT 1M TM£ SIIIJlATOR 

TYPO or ApI roltJI Q)ftGl 11 ans 

IIl&ht YrR Abnor.all Day VPI IB to Typo of tan ... t -laftCJI .lftlooln. 

OUd •• lop. Int ... cept 

LacaUz ... lnt ... .,.pt 

Q( and Altitud. An. 

1000 ' •• t AlL 

100 ' •• t AbOY. KIM 

HIlA. 

Dachlon H.lsht 

SOO ' •• t. A1repeed • SlnII lIat. 1 .. _ .... _ 'uu, ..... vu,. 

ra. root c:allouto 

0/. O""latlon. 

I.OC D..,iatlon. 

SlIlk Ita te Dav1a tlon. 

Air.peed O""latlDn. 

~. (110 callouto) 

I 

IJS£llI1S ·GRIO FOR SYlICAll THE VAY 1T.IIOOlO 8E IF 11' VElIE CIWIG£b TIl[ VAY YOO HAVE RECOtODED 1M THIS EYAlUATlOII Typo of All_ell CDndltlOlls 

"l&ht YrR Abn_ll Day v,. IB to 

Typo ., Ca n ... t DI«S.,.". ~I.ouu 

Clllde.lop. lzotercept 

!.Dealt,.« lftt ... eept 

Q( and Altitud. ArL 

1000 h.t ArL 

100 'Ht Abov. KIM i 

HIlA. 

Dachlon H.lsbt 

500 'Ht, Air.p.ed • Slnk Rat. IIIUDC1Cea roo< <Auoun 

ten root c:a lloun 

Cl/s O""lation. 

I.OC Il..,latl!'n. 

Sink it&t. Il."lat~no 

Alz.p •• d Devl.clDn. 

)Ion. (no .,alloun) 

That's the end. Thank you very much. and I'll let you know how this all turns out. 
1. Report No. I 2. Government Accession No. 3. Recipient's Catalog No. 

NASA CR-3300 

4. Title and Subtitle 5. Report Date 

Synthesized Voice Approach Callouts for October 1980 

Air Transport Operations 6. Performing Organization Code 

7. Author(s) B. Performing Organization Report No. 

Carol A. Simpson 

10. Work Unit No. 

9. Performing Organization Name and Address 505-35-21 

Psycho-Linguistic Research Associates 11. Contract or Grant No. 2055 Sterling Avenue 

Menlo Par~, CA 94025 A64l84B 

13. Type of Report and Period Covered 

12. Sponsoring Agency Name and Address Contractor Report 

National Aeroanutics and Space Administration 14. Sponsoring Agency Code Washington, D. C. 20546 

15. Supplementary Notes 

Ames Technical Monitor: Thomas Wempe Work was performed in collaboration with: 

Fort Worth, Texas 

American Airlines Flight Academy, 

Final Report 

16. Abstract 

A flight simulation experiment was performed to determine the effectiveness of synthesized voice approach callouts for air transport operations. Flight deck data was first collected on scheduled air carrier operations to describe existing pilot-not-flying callout procedures 

in the flight context and to document the types and amounts of other auditory cockpit information during different types of air carrier operations. A flight simulation scenario for a wide-body jet transport airline training simulator was developed in collaboration with a major U.S. air carrier and flown by three-man crews of qualified line pilots as part of 

their normally scheduled recurrent training. Each crew flew half their approaches using the 

experimental synthesized voice approach callout system (SYNCALL) and the other half using 

the company pilot-not-flying approach callout procedures (PNF). Airspeed and sink rate 

performance was better wlth the SYNCALL system than with the PNF system for non-precision approaches. For the one-engine approach, for which SYNCALL made inappropriate deviation 

callouts, airspeed performance was worse with SYNCALL than with PNF. Reliability of normal 

altitude approach callouts was comparable for PNF on the line and in the simulator and for 

SYNCALL in the simulator. However, SYNCALL was more reliable than PNF for making deviation 

approach callouts in the simulator. Pilots generally favored the concept of SYNCALL and 

judged it more reliable than PNF callouts. They suggested modifications before it would be 

appropriate for operational use. It was concluded that SYNCALL improved flight performance 

for non-precision approaches, and that a SYNCALL system should make deviation callouts onlY, 

For consistency, it is recommended that the modes of the Ground Proximity Warning System 

(Q1VS) be incorporated into a SYNCALL system. The detrimental effects on performance assoc- iated with inappropriate deviation callouts led to the further conclusion that such callouts 

should be designed out of the system. Finally, the results and conclusions of the experiment are used to develop suggestions for improvements to SYNCALL before further testing. 

17. Key Words (Suggested by Author(s)) 1 B. Distribution Statement 

Cockpit Displays, Synthesized Speech, Unlimited - Unclassified 

Approach Callouts, Voice, Callouts, Warnings, 

Airline, Human Factors, and Flight Performance 

Measures 

Star Category - 53 

19. Security Classif. (of this report) 120

. 

Security Classif. (of this page) 21. No. of Pages 

1

22

. 

Price· 

Unclassified Unclassified 120 A06 

"For sale by the National Technical Information Service, Springfield, Virginia 22161 

NASA-Langl ey. 1980 
End of Document