NASA/TMm2000-209607

Flight Crew Factors for CTAS/FMS Integration

in the Terminal Area

Barry W. Crane, Thomas Prevot, and Everett A. Palmer

Ames Research Center, Moffett Field, California

August 2000

https://ntrs.nasa.gov/search.jsp?R=20010039516 2018-05-15T06:54:48+00:00Z
The NASA STI Program Office ... in Profile

Since its founding, NASA has been dedicated to

the advancement of aeronautics and space

science. The NASA Scientific and Technical

Information (STI) Program Office plays a key

part in helping NASA maintain this important

role.

The NASA STI Program Office is operated by

Langley Research Center, the lead center for

NASA's scientific and technical information.

The NASA STI Program Office provides access

to the NASA STI Database, the largest

collection of aeronautical and space science STI

in the world. The Program Office is also

NASA's institutional mechanism for

disseminating the results of its research and

development activities. These results are

published by NASA in the NASA STI Report

Series, which includes the following report

types:

TECHNICAL PUBLICATION. Reports of

completed research or a major significant

phase of research that present the results

of NASA programs and include extensive

data or theoretical analysis. Includes

compilations of significant scientific and

technical data and information deemed to

be of continuing reference value. NASA

counterpart of peer-reviewed formal

professional papers, but having less

stringent limitations on manuscript length

and extent of graphic presentations.

TECHNICAL MEMORANDUM.

Scientific and technical findings that are

preliminary or of specialized interest, e.g.,

quick release reports, working papers, and

bibliographies that contain minimal

annotation. Does not contain extensive

analysis.

CONTRACTOR REPORT. Scientific and

technical findings by NASA-sponsored

contractors and grantees.

CONFERENCE PUBLICATION.

Collected papers from scientific and

technical conferences, symposia,

seminars, or other meetings sponsored

or co-sponsored by NASA.

SPECIAL PUBLICATION. Scientific,

technical, or historical information from

NASA programs, projects, and missions,

often concerned with subjects having

substantial public interest.

TECHNICAL TRANSLATION.

English-language translations of foreign

scientific and technical material

pertinent to NASA's mission.

Specialized services that complement the STI

Program Office's diverse offerings include

creating custom thesauri, building customized

databases, organizing and publishing research

results ... even providing videos.

For more information about the NASA STI

Program Office, see the following:

• Access the NASA STI Program Home

Page at http://www.sti.nasa.gov

• E-mail your question via the Internet to

help@sti.nasa.gov

• Fax your question to the NASA STI

Help Desk at (301) 621-0134

• Telephone the NASA STI Help Desk at

(301) 621-0390

Write to:

NASA STI Help Desk

NASA Center for AeroSpace

Information

712 ! Standard Drive

Hanover, MD 21076-1320
NASA/TM--2000-209607

Flight Crew Factors for CTAS/FMS Integration

in the Terminal Area

Barry W. Crane, Thomas Prevot, and Everett A. Palmer

Ames Research Center, Moffett Field, California

National Aeronautics and

Space Administration

Ames Research Center

Moffett Field, California 94035

August 2000
NASA Center for AeroSpace Information

7121 Standard Drive

Hanover, MD 21076-1320

301-621-0390

Available from:

National Technical Information Service

5285 Port Royal Road

Springfield, VA 22161

703-605-6000
TABLE OF CONTENTS

Summary ........................................................................................................................................... 1

Background ................................................................................................................................ 1

National Airspace System Problems ............................................................................................... 1

Increasing congestion at airports ................................................................................................ 1

Current operations in and near the terminal airspace ................................................................... 1

Current and forecasted problems ................................................................................................ 1

Problems from the user perspective ............................................................................................ 2

Proposed Solutions ........................................................................................................................ 2

Controller aiding ........................................................................................................................ 2

CTAS/FMS integration ............................................................................................................... 2

FMS Descent Procedures ........................................................................................................ 4

Flying FMS and CTAS/FMS Descents ................................................................................................. 5

Focus of Study ................................................................................................................................ 5

Study Approach ............................................................................................................................. 6

Development of FMS descent procedures ................................................................................... 7

Development of CTAS/FMS descent procedures .................................................... 7

Interface enhancements ............................................................................................. 7

Descent scenarios .................................................................................................................. 8

Theoretical Perspective of Crew Activity Analysis ........................................................................... 8

Control and management automation descriptions and definitions .............................................. 8

Expected crew activity changes ................................................................................................... 9

Relationship to previous research .............................................................................................. 1 1

Method ............................................................................................................................................ 1 2

Participants .................................................................................................................................. 1 2

Equipment ................................................................................................................................... i 2

Design and Procedure .................................................................................................................. 1 2

Crew training ............................................................................................................................ 12

Dependent Measures .................................................................................................................... 12

Crew activity analysis ................................................................................................................ 12

12 Crew workload measure ............................................................................................................

Workload definition and measure description ........................................................................... 13

Crew performance criteria ........................................................................................................ 13

Video and digital data .............................................................................................................. 13

Crew questionnaire ................................................................................................................... 1 3

Results and Discussion ..................................................................................................................... 1 3

Crew Activities: Current Day vs. FMS for Baseline and Complex Scenarios .................................. 1 4

Planned comparisons ................................................................................................................ 1 4

Number of activities ............................................................................................................ 1 4

Activity types ........................................................................................................................... 1 5

Subjective Workload: Current Day vs. FMS for Baseline and Complex Scenarios ......................... 18

Planned comparisons ................................................................................................................ 18

Baseline current day procedure scenario vs. baseline FMS procedure scenario .......................... 18

Complex current day procedure scenario vs. complex FMS procedure scenario ....................... 18

Crew Performance: Compliance with Procedural Elements ........................................................... 20

Speed and altitude compliance ................................................................................................. 20

VNAV and LNAV use .............................................................................................................. 21

Crew Performance: CTAS/FMS Procedures Used During Complex Scenarios ............................... 21

V
TRACONroutemodifications.................................................................................................. 21

Performanceof activityelements.............................................................................................. 22

Totalactivitytime..................................................................................................................... 22

Totalheadsdowntime.............................................................................................................. 23

Dwelltime................................................................................................................................ 23

PostExperimentQuestionnaire.................................................................................................... 25

Acceptabilityof FMSprocedure............................................................................................... 25

Acceptabilityof CTAS/FMSprocedures................................................................................... 29

ConclusionsandFutureWork............................................................................................. -.............30

References....................................................................................................................................... 33

AppendixA..................................................................................................................................... 35

STAR,FMS,andCTAS/FMSCharts................................................................................................. 35

AppendixB..................................................................................................................................... 41

ParticipantConsentForm................................................................................................................. 41

AppendixC..................................................................................................................................... 45

DescentScenarios............................................................................................................................ 47

AppendixD..................................................................................................................................... 53

WorkloadDimensionDescriptionsandWorkloadRatingSheet....................................................... 53

AppendixE..................................................................................................................................... 57

ProcedurePerformanceCriteria....................................................................................................... 59

AppendixF..................................................................................................................................... 61

CrewQuestionnaire.......................................................................................................................... 63

AppendixG.................................................................................................................................... 79

CrewQuestionnaireResponses........................................................................................................ 81

vi
Summary

Center TRACON Automation System

(CTAS)/Flight Management System (FMS)

integration on the flightdeck implies flight

crews flying coupled in highly automated

FMS modes [i.e. Vertical Navigation (VNAV)

and Lateral Navigation (LNAV)] from top of

descent to the final approach phase of flight.

Pilots may also have to make FMS route edits

and respond to datalink clearances in the

Terminal Radar Approach Control (TRACON)

airspace. This full mission simulator study

addresses how the introduction of these FMS

descent procedures affect crew activities,

workload, and performance. It also assesses

crew acceptance of these procedures. Results

indicate that the number of crew activities and

workload ratings are significantly reduced

below current day levels when FMS

procedures can be flown uninterrupted, but

that activity numbers increase significantly

above current day levels and workload ratings

return to current day levels when FMS

procedures are interrupted by common ATC

interventions and CTAS routing advisories.

Crew performance showed some problems

with speed control during FMS procedures.

Crew acceptance of the FMS procedures and

route modification requirements was generally

high; a minority of crews expressed concerns

about use of VNAV in the TRACON airspace.

Suggestions for future study are discussed.

Background

National Airspace System Problems

Increasing congestion at airports

"By the year 2011, aircraft operations at the

top 100 airports are projected to increase 32

percent beyond current day (1998)

operations. This increase and projected further

increases indicate that the busiest U.S. airports

are getting busier, which will compound

problems of congestion at these key airports

unless airport and airspace capacity

enhancements are made. Assuring that the

capacity of the National Airspace System

(NAS) can accommodate the growing demand

for aviation services is critical to the Nation's

economic future" (FAA Office of System

Capacity, 1998).

Current operations in and near the terminal

airspace

The current air traffic control system uses

analog voice communications, and tactical

vectoring for aircraft separation and traffic

flow management in the arrival and approach

phases of flight. Aircraft in general begin their

descent in an Air Route Traffic Control Center

(ARTCC or Center) on charted Standard

Terminal Arrival Routes (STARs) that include

both lateral and vertical constraints for the

path of flight. The lateral constraints consist of

a path defined by straight line segments

between navigational database waypoints that

terminate at or near a "corner post" or entry

gate into Terminal Radar Approach Control

(TRACON) airspace. This lateral path defined

on the STAR is generally cleared by Center

controllers and flown by pilots as specified on

the STAR chart. The vertical constraints of a

STAR are specified as speed and altitude

restrictions associated with the navigational

waypoints on the STAR. When an aircraft

nears the termination point of the STAR, it is

handed off to TRACON air traffic controllers.

TRACON controllers guide the aircraft to an

assigned runway by issuing tactical altitude,

speed, and heading instructions. Unlike the

STAR used in the center airspace, these

TRACON routes are not charted. Although

nominal paths are usually followed in the

TRACON, the aircraft crew is generally only

aware of their current instructions (altitude,

speed, and heading). As traffic levels increase,

both Center and TRACON controllers are

forced to direct aircraft on 'radar vectors' that

deviate from STARs and the nominal

preferred TRACON paths to runways.

Current and forecasted problems

This approach to air traffic management is

becoming inefficient in meeting growing air
traffic demands.Thecontinuedgrowthof air

traffic at majorairportshascausedincreasesin

air trafficdelaysandhasputconsiderable

stressonbothexistingair traffic control

(ATC)systemsandontheairtraffic

controllersthemselves(Davis,etal., 1997).

Theprojectedincreasein aircraftoperations

wouldproduceunduestrainonthecurrentair

traffic management system.A moreefficient

systemis neededin orderto accommodatehe

anticipatedgrowthin operations.These

concernshavemotivateddevelopmentof

computerizedcontrolleraidingtoolsaimedat

helpingcontrollerscopewith increasedtraffic

demands.

Problems from the user perspective

The airborne Flight Management System

(FMS) is a highly capable present day

technology. An estimated 10,000 FMS

equipped aircraft exist in the current

worldwide fleet. Aircraft FMSs enable

airplanes to compute and fly fuel efficient

trajectories accurately. However, the FMS is

seldom utilized within the TRACON airspace

due to the current controlling method of

tactical vectoring. This requires aircraft to

remain in the air longer and fly less efficient

descent profiles than is possible when the FMS

is used. The result is significant airspace

congestion and major airline costs due to fuel

burn and added delays.

Proposed Solutions

Controller aiding

To address these problems, NASA and the

FAA have been developing the Center

TRACON Automation System (CTAS). CTAS

is set of ground-based decision support and

aiding tools intended to assist controllers with

aircraft sequencing, separation, flow control,

scheduling, and trajectory prediction in and

around the terminal area (see Erzberger et al.,

1993). CTAS is comprised of three primary

tools. The first tool is the Traffic Management

Advisor (TMA). The TMA computes and

plans a landing time and sequence number for

every aircraft approaching an airport while

2

that aircraft is still in cruise flight. Next the

Enroute Descent Advisor (E/DA), advises a

fuel efficient descent which can include a

particular routing, profile, or speed, in order to

meet the arrival time at the TRACON meter fix

computed by the TMA. As the aircraft crosses

the meter fix and enters the TRACON airspace,

TMA updates the sequencing and arrival time

plan and assigns a runway for the aircraft.

Next, while the aircraft is in the terminal

airspace, the third CTAS tool, the Final

Approach Spacing Tool (FAST), advises the

controller in order to accurately space and

sequence the aircraft in the TRACON airspace.

FAST advisories may include speeds or

particular routings for the aircraft to fly in

order to meet the TMA computed threshold

arrival time.

•The CTAS tools manage arrival traffic

strategically to prevent airspace overload and

reduce delays. By assigning each aircraft

approaching the airport a "slot" defined by a

scheduled arrival time at the runway threshold

and various intermediate arrival times along

the descent path, CTAS can optimize traffic

flow to an airport. CTAS can then aid

controllers in implementing this optimal traffic

flow by providing strategic routing and speed

advisories for aircraft to fly in order to meet

their scheduled arrival times (see figure 1).

CTAS/FMS integration

The Enroute Descent Advisor tool has

undergone field testing at Denver International

Airport in recent years (see Green et al., 1996;

Cashion et al., 1995; Palmer et al., 1997). The

E/DA implementation utilized in these field

tests involved issuance of top of descent point

and cruise and descent speed advisories to

aircraft. These advisories served to adjust

arrival time at the TRACON metering fix while

keeping aircraft on conflict free trajectories. A

more advanced implementation of the E/DA to

aid controllers with arrival time adjustments

supports rerouting of the aircraft. This is

accomplished with a Trial-
efficient,_

TMA uvJatesplan

and assigtls runway

Descent planning region !

Final approach

planning region

Metering fix

_al

Figure 1. CTAS Tools: Traffic Management Advisor (TMA), Enroute Descent Advisor (E/DA), Final

Approach Spacing Tool (FAST).

Planning-Waypoint functionality that allows

controllers to propose routings to the E/DA

with a movable waypoint to adjust arrival times

and avoid conflicts. This new routing, once

verified by the E/DA and air traffic controller,

can then be communicated to the aircraft by

voice or by data link. The data link

implementation would allow uplink of this

route to Flight Management System (FMS)

equipped aircraft. This E/DA functionality is

proposed as a possible approach to integrating

CTAS with the airborne FMS in the Center

airspace. It is hypothesized that the uplink of

these routes, to be loaded and then flown by

the aircraft FMS, could improve aircraft

flightpath accuracy and therefore increase

arrival-time accuracy.

The FAST tool can also be utilized to various

degrees to aid controllers with scheduling and

sequencing aircraft. The Passive FAST

implementation, which has been under field

study for the past few years at the Dallas Fort

Worth Metroplex (see Davis et al., 1997),

provides controllers with landing sequence and

landing runway advisories for all aircraft

entering the TRACON airspace. It is left to

controllers' discretion whether to heed these

advisories.

A more advanced functionality of FAST

currently under research is known as TAP

FAST. This FAST implementation not only

provides the controller with a computed route

3
for each aircraft to fly in the TRACON

airspace but provides the capability to uplink

this route to Flight Management System

(FMS) equipped aircraft. TAP FAST is

proposed as a possible approach to integrating

CTAS with the airborne FMS in the TRACON

airspace. As with the E/DA functionality, it is

hypothesized that the uplink of these routes to

be flown by FMS aircraft could improve

aircraft flightpath accuracy in the TRACON

airspace to increase arrival time accuracy. This

could allow for closer entrail spacing of

aircraft and increase runway throughput at

airports. This can also be down with the Active

FAST system.

In order to implement these E/DA and FAST

route uplink functionalities as described, an

air-ground data link capability is necessary.

This would entail the implementation of many

technologies not currently available to ATC

ground stations, aircraft flight decks, and the

National Airspace System (NAS)

communications infrastructure. Hence, this use

of these E/DA and FAST concepts is

necessarily intended for more far term

implementation.

To permit CTAS/FMS integration in the near

term, a concept has been developed that is

intended to work with currently available

technologies. This concept utilizes the E/DA

Trial-Planning-Waypoint and TAP FAST

functionalities minus their datalink capability.

Thus, the controller is still provided with a

route for each aircraft to fly but must

communicate this route via the voice channel.

Due to the complexity of information

contained in even the shortest route segment,

voice communication of all this information is

not feasible. To make this concept workable,

this study proposes configuring the E/DA and

TAP FAST to compute nominal default paths

from Standard Arrival Routes (STARS) to

runway approaches utilizing only speed and

path extension degrees of freedom. This

permits the loading of these nominal routes in

airborne FMS databases as well as charting

these routes on cockpit Jeppesen charts. These

4

charted routes are called FMS arrivals and

transitions. The controller can clear an aircraft

for the FMS arrival and transition and verbally

specify any modifications to this route whether

they be route modifications, speed change

points, or path extension lengths (see

Appendix A for a sample FMS arrival and

transition chart).

Coordination of automation tools such as

E/DA, FAST, and the FMS may enable aircraft

to use their FMS computed trajectories from

top of descent to final approach. If an aircraft

can remain on an FMS trajectory until the

final approach, a number of sources of arrival

time error are eliminated. This is because

aircraft on FMS routes can fly more

predictable and precise trajectories, and thus

they are more likely to match trajectories and

arrival times anticipated by controllers and

predicted by the CTAS tools. Placing aircraft

on FMS paths can also help reduce excess

spacing buffers, and increase the efficiency of

the CTAS delay distribution function

(Couluris et al., 1997). This could allow for

increased capacity, efficiency, and safety in the

terminal airspace. Additionally, if controllers

are able to clear aircraft for FMS routes in the

Center and TRACON airspace, this would

reduce the number of clearances required to

guide aircraft through this airspace. This could

result in a reduction in both pilot and

controller communication workload.

FMS Descent Procedures

FMS descent procedures involve descents from

cruise altitude to the final approach course

intercept without radar vectors or altitude

assignments with the aircraft coupled to the

FMS in both the lateral and vertical navigation

and guidance modes. The lateral and vertical

route is specified on paper charts and is coded

and selectable in the aircraft FMS electronic

database. The route is loaded via the Control

and Display Unit (CDU) and can be selected in

the approach page database adjacent STAR

and runway listings. As of 1999, STARs in the

National Airspace System (NAS) generally
terminateat theTRACONentry"corner

post"afterwhichaircraftare typically issued

vectors from ATC to the final approach

course. FMS descent procedures vary in the

extent of their routing but usually describe a

path that includes the current day STAR

routing and then extend into TRACON

airspace to the point of localizer capture (final

approach) at a specified runway or series of

runways. FMS descent procedures are

distinguished from STARs primarily by this

charted transition routing in the TRACON

airspace. Another distinguishing feature of

FMS descent procedures is that when they are

loaded into the flight computer via the CDU,

all charted altitude and speed restrictions

appear adjacent their corresponding waypoints

on the CDU legs page. With STARs, charted

crossing restrictions, whether specified as

"expect" or "cross at" values, require

manual input by the flight crew once the

STAR has been loaded. When a STAR isin

use, ATC is required to explicitly clear aircraft

for all charted restrictions stated as "expect"

on the STAR chart. FMS descent procedures

do not include "expect" restrictions because

all charted values, both speed and altitudes, are

"hard" restrictions that must be flown once

an aircraft is cleared for the FMS descent

procedure.

FMS descent procedures take advantage of the

advanced Flight Management System in

today's aircraft. They are intended to optimize

the lateral and vertical path flown by aircraft in

order to reduce fuel consumption and provide

a predictable descent path for aircrews and

controllers thereby reducing the need for

radar vectoring (Transport Canada, 1997).

FMS descent procedures are designed to allow

maximal utilization of the vertical and lateral

navigation and guidance capabilities of

automated aircraft. They provide the crew with

a well defined predictable path for the aircraft

to fly in the descent phase of flight. FMS

descent procedures require flightcrews to use

their FMS at tow altitudes, in the high

workload descent phase of flight. In today's

operations, crews are accustomed to a lesser

degree of automation during this phase of

flight that involves active speed, heading, and

altitude control via direct autopilot inputs.

FMS descent procedures are proposed as a

primary integration step with the F_./DA and

FAST CTAS controller aiding tools. These

have been termed CTAS/FMS procedures. This

entails the FMS descent procedures described

above with route modification and speed

update capabilities added. This involves the

flight crew receiving an initial clearance to fly

the charted nominal FMS routing. While still

in Center airspace, the crew may receive an

updated routing from ATC, as advised by

E/DA, in the form of a descent route

modification. When entering the TRACON

airspace, the crew is cleared for the remaining

TRACON portion of the charted FMS routing.

While flying this routing, the crew may receive

an amended speed change point and/or a route

modification (path extension), to adjust arrival

time, spacing, or sequencing of aircraft as

advised by FAST. The crew inputs this route

modification into the FMS and proceeds to fly

the updated routing. This procedure requires

crews to fly coupled to the FMS in the Center

and TRACON airspace and in addition to

make edits to their FMS routes during this

phase of flight. In today's vectoring

environment, speed amendments and path

extension clearances are transparent to flight

crews as they are essentially indistinguishable

from nominal vectoring procedures and

routes.

Flying FMS and CTAS/FMS

Descents

Focus of Study

The objective of this research is to determine

how a shift from current day (as of 1999)

descent procedures (STARs and tactical

vectoring in the TRACON) to the proposed

FMS and CTAS/FMS descent procedures will

impact crew activities, workload, and

performance. In addition, the study will elicit

5
pilot perspectives regarding the acceptability

and usability of the proposed procedures.

The introduction of FMS descent procedures

and CTAS/FMS descent procedures may create

substantial changes in the number and nature

of the crew's flightdeck activities. Current day

descent procedures in the NAS only include

routings for the initial descent segment within

the Center airspace and require crews to fly the

approach segment using discrete tactical

clearances from ATC. This allows crews to fly

the initial route segment of the descent in a

highly automated mode with the autopilot

coupled to the FMS, and requires less

automated, tactical "Mode Control Panel

(MCP) flying", in the non-charted TRACON

segment of the descent. FMS descent

procedures require crews to fly coupled to the

FMS, in a highly automated mode, for the

entire descent until turning onto final

approach. CTAS/FMS procedures impose the

same requirements but also require crews to

perform FMS route modifications during

descent. These changes in flightdeck activities

will be investigated through a full mission

simulation of nominal (baseline) and complex

descents for the current and proposed descent

procedures.

A change in the crew's flight deck activities

during the descent phase of flight may impact

their workload. Descents are typically very

busy periods for the flight crew, and concerns

regarding high workload demands during this

period are discussed in the literature (Wiener,

1985). This study aims to determine if and

how workload is affected by the proposed

procedures. If significant shifts in workload

levels are found, the goal will be to ascertain

the nature of these shifts, whether the shifts

maintain workload at acceptable levels, and to

identify any potential problems or benefits

that may emerge as a result of these shifts.

Crew performance is another primary factor

to consider with the introduction of new

procedures. It is important to determine if

crews can maintain acceptable performance

when flying the proposed descent procedures.

6

While workload is a metric indicating task

difficulty, it is often dissociated from

performance measures (Yeh & Wickens,

1988). For example, in a situation where task

demands (and thus workload) are already

high with a given procedure, and the new

procedure imposes even higher demands,

performance may degrade but subjective

workload measures may not increase. This is

because nearly full resources are invested in

the first procedure and thus no further

resources can be invested into the more

demanding procedure producing no

subjective increase in task demand.

Conversely, if task demands are already very

low and increase yet remain low with

introduction of the new task, performance will

not change but subjective workload will

increase (Yeh & Wickens, 1988). This

possible dissociation makes it necessary to

measure and compare both performance and

workload for the new procedures.

Study Approach

The approach for this study was to develop

candidate FMS and CTAS/FMS descent

procedures for a single arrival, approach, and

runway at the Dallas Fort-Worth Metroplex

(DFW) that overlay the current day STAR and

nominal TRACON vector routing. Then a set

of descent scenarios was developed for current

day, FMS, and CTAS/FMS operations. A

nominal baseline descent scenario was

generated for the current day (as of 1999) and

FMS operations. A more complex descent

scenario was generated for the current day,

FMS, and CTAS/FMS operations.

To support the newly developed CTAS/FMS

procedures, various interface enhancements

were designed and implemented. These are

described below. A full-mission simulation was

then conducted to track crews' actual descent

activities, and to gather workload,

performance, and attitudinal measures from

crews.
Development of FMS descent procedures

The FMS descent procedures were developed

by drafting an experimental Jeppesen chart

(see Appendix A) and coding the charted

routing in the FMS database of the test

simulator. The arrival portion of the procedure

overlays the routing of the current day Glen

Rose 3 STAR at DFW (see Appendix A).

Unlike the STAR, all the crossing restrictions

are "hard" restrictions as depicted on the

chart and are loaded into the FMS from the

database. This means that the restrictions are

an integrated element of the procedure and

crews must comply with them when cleared for

the procedure. The transition portion of the

FMS procedure overlays the current nominal

vector routing from the Glen Rose 3 STAR to

runway 18R at DFW. Like the FMS arrival

portion, this routing includes "hard" speed

and altitude crossing restrictions up until the

final approach course. Crews are required to

fly the procedure in Vertical Navigation

(VNAV) and Lateral Navigation (LNAV), the

FMS guidance modes.

Development of CTAS/FMS descent

procedures

The CTAS/FMS descent procedures (see

Appendix A) are identical to the FMS descent

procedures except for the addition of speed

and route modification degrees of freedom as

integrated elements of the procedure. Notes on

the charted CTAS/FMS procedure advise the

crew that they may receive a speed to fly

and/or a route modification in the arrival

segment of the descent. They are also advised

that they may receive a speed amendment

and/or a route modification (downwind

extension) during the transition segment of the

procedure.

Interface enhancements

To support the speed and route modification

degrees of freedom required for the

CTAS/FMS procedures, three cockpit interface

enhancements were prototyped for the current

study. The three enhancements represent a

continuum from near term to far term

technologies.

The near term enhancement, termed Manual

Path Extension (MPE), enables a route

modification to the TRACON portion of the

FMS procedure routing via CDU inputs made

by the crew. The route modification involves

an extension of the downwind and final

approach distances of the FMS procedure.

This is made possible through an experimental

leg type termed a Fix-to-a-Variable-Distance

leg. The length of this leg can be altered by

the crew as many times as desired by down

selecting its reference waypoint (ROSEL for

current study) in the CDU scratch pad and

entering the new desired distance. This is

intended to support the TAP FAST tool which

advises controllers with strategic base turn

points for aircraft in the TRACON airspace.

The second, medium term interface

enhancement, termed CDU DL (Control and

Display Unit Data Link), utilized data link.

route modifications and communications via

the CDU. This interface was inspired by the

current FANS (Future Air Navigation System)

data link implementation in use in the B747-

400 aircraft. The CDU DL interface was used

to support E/DA and FAST route

modifications. Route modifications were

uplinked to the aircraft and loadable directly

into the FMS by crew CDU actions. This

loadable route could be inserted into the

current FMS route and thus implemented the

specified route modification.

The far term interface implementation, termed

DDL (Distributed Data Link), also features an

air-ground data link for route modification

communications. Unlike the CDU DL

interface, data link route messages were

displayed directly in the upper center EICAS

(Engine Indicating and Crew Alerting System)

screen on the flightdeck, and responses to data

link messages were made using buttons placed

adjacent to the MCP on the flightdeck

glareshield. Route modification messages

automatically loaded into the FMS 'mod

7
route' upon arrival. Crews reviewed, and if

they were acceptable, executed and accepted

these route modifications.

Each of these interfaces allows the actual

computed CTAS routing to be input into the

FMS. Because some of these route

modifications occur in the busy TRACON

airspace, key questions regarding each of these

interfaces are: Will crews successfully perform

all the necessary steps required to complete the

route modification task for each interface?

How much time will be required to complete

the route modification tasks? How much total

"'heads-down" time will be required by each

crew member during performance of the

tasks? For each interface, what is the average

length of time each crew member fixes his or

her gaze, or dwells, in the "heads-down"

position?

Descent scenarios

Two descent scenarios were developed for the

study. The first scenario, termed baseline,

consisted of ATC clearances to fly charted

routings from top-of-descent to final approach

with no deviations for vectors, temporary

altitudes, or speed amendments. The second

scenario, termed complex, contained deviations

from the charted routings including vectors,

intermediate altitude level-offs, and speed

amendments. The complex scenario

represented likely CTAS advisory clearances

from ATC. These two descent scenarios were

used in both current day and FMS procedure

conditions to facilitate direct comparison

between the current day (STAR) and FMS

procedure contexts. The complex scenario was

also used for the three CTAS/FMS procedure

contexts (Manual Path Extension, CDU DL,

and DDL.) to test use of these interfaces as a

method of dynamically updating FMS routes

(see figure 2).

Theoretical Perspective of Crew Activity

Analysis

Prior to analysis of crew activities, a short

background and theoretical discussion of pilot

interaction with aircraft automation is

necessary. This will provide a framework for

discussion and analysis of crew activities.

Control and management automation

descriptions and definitions

Automatic piloting devices have been in use in

aircraft since the 1930's. These early

autopilots relieved pilots of the manual labor

of hand-flying on long flights. They provided

inner-loop control to the aircraft in response

to direct pilot instructions but left the pilot to

perform all navigation and other essential

piloting tasks (Billings, 1996). By the 1980's

autopiloting advances built on microprocessor

technology provided the capability to program

complete routes into a flight computer, to

couple this computer to an autopilot, and to

hand over the navigation, guidance, and

control tasks to the computer.

Procedure

Current Day (as of 1999)

FMS

Baseline Scenario

tno route deviations)

X

X

Complex Scenario

tCTAS advisories)

X

X

Manual Path Extension X

CDU DL X

DDL X

Figure 2. Test matrix.

8
The crew acts to monitor the computer's

"actions", to update the programmed flight

route, or make any necessary interventions.

These two categories of automation have been

termed control automation (e.g. Fadden,

1990) and management automation (e.g.

Billings, 1996) respectively (see Billings

(1996) for a more complete discussion).

In the aviation domain, control automation

entails the use of an autopilot as a tactical

controller of the aircraft in response to explicit

speed, heading, and altitude command inputs

from the pilot. Management automation

involves strategic programming of a complete

lateral and/or vertical flight plan into a flight

management computer which can be coupled

to the autopilot to autonomously carry out the

aircraft navigation, guidance, and control

necessary to follow the programmed route

with limited pilot input.

With an aircraft under control automation, the

pilot makes discrete tactical inputs

commanding immediate output (behavior)

from the autopilot. The crew ensures that the

autopilot is moving toward and then

maintaining this goal (monitoring) until the

time for further tactical input arises. For

example, the pilot may select a new heading

for the aircraft to fly by dialing the value into

the autopilot (Mode Control Panel). The

autopilot in turn commands the aircraft to turn

immediately to this heading and maintain it

until the pilot makes further tactical inputs.

Thus, control automation is characterized by

the pilot specifying a discrete goal state to the

autopilot and the autopilot proceeding

immediately to satisfy and maintain that goal.

With an aircraft under management

automation, the pilot makes primarily strategic

inputs to a flight management computer

(FMC). The pilot can then couple the FMC to

the autopilot so that it carries out the

programmed inputs. The pilot then must

verify that the aircraft is behaving as

programmed (monitoring). For example, the

pilot may input a series of waypoints that

define a route from one airport to another,

couple the autopilot to the FMC via the lateral

navigation mode (LNAV), and then monitor

the aircraft behavior as it flies the programmed

routing. Thus, management automation is

characterized by the pilot specifying a series

of goal states to the FMC in one programming

cycle, and the FMC commanding the autopilot

to carry out these goals without further pilot

input. Unlike control automation, in

management automation the aircraft does not

necessarily proceed immediately with

programmed goals. The aircraft can only

pursue one goal (in each control axis) at a

time and thus a latency often exists between

goal specification and goal execution (aircraft

behavior).

For ease of discussion the phrase "tactical

input" or "tactical activity" will be used

interchangeably with the phrase "use of

control automation", and the phrase

"strategic input" or "strategic activity" will

be used interchangeably with the phrase "use

of management automation".

Expected crew activity changes

Based on activity models developed with

subject matter experts prior to the simulation

study, it is anticipated that compared to current

day descent procedures, crews flying FMS

descent procedures during the baseline

scenario will show [hypothesis (1)] a marked

reduction in the number of observable crew

activities (physical inputs into aircraft

automation) that results from [hypothesis (2)]

performance of significantly fewer tactical

activities (use of control automation), and

therefore, performance of primarily strategic

activities (use of management automation).

However, when flying FMS descent procedures

during the complex scenario, it is anticipated

that [hypothesis (3)] the number of crew

activities will be roughly equal to that of

current day descent procedures.

The anticipated changes in crew activities with

the FMS procedure baseline scenario will alter

crew flight deck monitoring behavior by

9
requiring intermittent and continuous

monitoring of multiple latent aircraft

behaviors typical of management automation

use rather than immediate monitoring of

immediate aircraft behaviors as is seen with use

of control automation. Also, with the FMS

procedures, crews will be entrusting

automation guidance to comply with

programmed altitude restrictions; rather than

meeting charted restrictions tactically, and

crews are expected to utilize the LNAV and

VNAV automated flight modes in the

TRACON airspace. These modes are not

available with current day (as of 1999) STAR

procedures. This study investigates the

workload and performance impact of these

procedural changes in the number and

character of flight deck activities, and the

acceptance of these changes by crews.

The CTAS/FMS descent procedures are

adjuncts to the FMS descent procedure

designed to better accommodate the dynamic

advisories generated by the E/DA and FAST

controller aiding tools. Aside from the

assigned descent speed in the CTAS/FMS

procedures, the complex scenario events are

identical between the FMS and CTAS/FMS

procedures. The primary difference is that the

CTAS/FMS procedures permit crews to modify

an existing FMS routing in a strategic fashion

thereby retaining accurate FMS guidance of

the flight. This is made possible in this study

by the three interface enhancements

previously discussed.

The primary differences among the three

CTAS/FMS procedures are the manner in

which FMS route modifications are made. The

interface enhancements provided in the

CTAS/FMS procedures explore alternative

methods for responding to the ATC

prompting events. In this study, the

acceptability and performance by flight crews

of these route modifications and their

interfaces is explored. A description of each

CTAS/FMS procedure interface is provided

below.

10

The first CTAS/FMS interface, Manual Path

Extension (MPE), involves manipulation of the

existing FMS transition procedure base turn

point via the CDU LEGS page. There are six

tasks necessary to complete the activity. The

crew member must (1) acknowledge and

readback the ATC cleared base turn distance,

(2) line select the reference waypoint ROSEL

into the CDU scratchpad, (3) enter a backslash

(/) and the new base turn point value, (4)

reselect the waypoint into the flightplan, (5)

review the modification on the CDU and

Navigation display, and (6) execute the

modification by pressing the execute button

on the CDU. Observe that this activity requires

the crew to make entries of numerical data by

hand in a specified entry format (i.e./X.X)

making it potentially error prone and subject

to forgetting (Billings, 1996). In addition, this

is a CDU task that requires the non-flying crew

member to go "heads-down" in the busy

TRACON airspace. The MPE interface does

not support E/DA route modifications issued

in the Center airspace.

The second CTAS/FMS interface, CDU DL,

involves loading, executing, and accepting, via

the CDU, an ATC data link message that

contains an E/DA rerouting of the FMS Arrival

or a FAST base turn point extension routing.

This activity contains seven tasks. Upon receipt

of the data link message (indicated by a chime,

and an EICAS and CDU message), the nonflying crew member must (1) press the ATC

key on the CDU, (2) read the message text on

the CDU. page aloud to the other crew

member, (3) receive confirmation from the

other crew member, (4) load the new route by

pressing a LOAD prompt, (5) the flying crew

member reviews and verifies the loaded

message on the LEGS page (this is necessary

because the non-flying crew member is on the

ATC page) and Navigation display, (6) the

non-flying crew member accepts the message

by pressing the ACCEPT prompt, and (7) the

flying crew member executes the new route.

The CDU DL activity requires both crew

members to go "heads-down" in the

TRACON airspace.
ThethirdCTAS/FMSinterface,DDL,involves

executingandacceptinganATC data link

message through use of a "heads-forward"

interface. This activity contains five tasks.

Upon receipt of the data link message

(indicated by a chime, and an EICAS and

CDU message) the non-flying crew member

must (1) read the message text on the EICAS

display aloud to the other crew member, (2)

receive confirmation from the other crew

member, (3) review and verify the

automatically loaded message on the LEGS

page and Navigation display, (4) execute the

new route by pressing the execute button on

the CDU, and (5) accept the message by

pressing the accept button on the glareshield.

Unlike all the tasks in the previous two

activities, only task (4) requires the non-flying

crew member to go "heads-down" in this

activity. Furthermore, when the message

arrives, the text is immediately displayed on

the EICAS screen, and the route modification

is automatically loaded. This eliminates the

access and loading steps required with the

CDU DL activity. Both the CDU DL and DDL

activities eliminate the requirement to

manually enter new route data as in the MPE

condition.

It is important to note a distinction between the

MPE and data link conditions. With the MPE

path extension, crews manually modify only

the base turn point. In the data link conditions,

the FAST uplink may contain other changes to

the route in addition to the base turn point

extension such as a modified speed schedule

or new crossing restriction parameters.

Activity models generated with subject matter

experts reveal a modest reduction in the

number of required crew activities with the

introduction of the data link interfaces only.

The reduced number of activities for the CDU

DL and the DDL procedures result from the

uplink of CTAS route segments that provide

modifications to the FMS descent procedure

routing. This permits crews to remain coupled

to their FMS routing, and thus to continue

strategic flying, through most of the descent.

The number of crew activities for the MPE

procedure is nearly identical to the complex

FMS procedure scenario for the initial descent

segment. Differences in activities arise in the

TRACON airspace. The MPE interface

supports strategic modification of the base

turn point distance. This prevents the need to

revert to control automation as in the FMS

procedure when the base turn point is

extended by ATC.

Relationship to previous research

Since the advent of automation technology in

commercial aircraft, a steady stream of

research has emerged that addresses the impact

of automation on flight crews (Wiener &

Curry, 1980; Wiener, 1985; Curry, 1985;

Wiener, 1989; Tsang & Vidulich, 1989;

Billings, 1996). Two main assertions appear

repeatedly in this body of literature. First is the

assertion that the human is poorly suited as a

system monitor. And second is the claim that

increased automation on the flight deck may

not reduce crew workload but will merely shift

its nature from physical operations (e.g.

button presses, etc.) to more mental operations

(e.g. monitoring, etc.). Billings (1996)

discusses issues of monitoring and workload

of management automation specifically. He

remarks that use of management automation

means management by exception. This means

the automation is given the authority to carry

out all programmed tasks and performs them

unless the crew takes exception. He comments

that this may lighten physical workload, but

may increase the cognitive burden on the crew

as they are required to anticipate and confirm

correct automation behavior. Also, Billings

warns that use of this type of automation may

decrease the crew's involvement with the

flying task and the mission as a whole. He

remarks that even the most motivated and

capable pilot can fatigue and lose vigilance

when activity demands are so dramatically

reduced. While the claims in this literature

hold theoretical and intuitive merit, empirical

study is needed to support or oppose these

assertions. This study is intended to provide

11
such empirical data. The workload and

performance measures to be gathered are

intended as an initial examination into the

effects on pilots of increased automation in the

descent phase of flight. The assertions that

pilots are poorly suited as system monitors and

that greater use of management automation

may decrease pilot involvement in the flying

task are not directly addressed in this study.

However, performance results may be able to

provide hints as to possible problems brought

on by increased use of management

automation during the descent phase of flight.

Method

Participants

Twelve flight crews who fly for commercial air

carriers were recruited to participate in the

study. Participants were recruited by the

Raytheon Corporation and were paid for their

time and compensated for travel and

accommodation expenses. Crew participation

requirements included: a type rating on the

Boeing 757, 767, 737-500, or 777 aircraft and

no prior data link experience.

Equipment

Crews flew in the Advanced Concepts Flight

Simulator (ACFS) at NASA Ames Research

Center (see Blake, 1996). This simulator is a

full-mission type with a "generic" glass

cockpit layout. It resembles the B757 aircraft

in its avionics layout and functionality.

Additions to the cockpit avionics are four

interface enhancements: a vertical situation

display (see Prevot, et. al. 1998); a variablefix-to-a-distance leg type in the FMS database;

a FANS data link functionality; and an

advanced "heads-forward" data link

interface.

Design and Procedure

After reading and signing a consent form (see

Appendix B), each crew flew seven descents

(due to time constraints, four current day and

six FMS descents were not flown) from the

descent procedures and scenarios previously

12

discussed (see figure 2 and Appendices A and

C). Twelve unique random orders were

arranged for the seven descents with the

condition that no descent appear first more

than twice. Crew members alternated as pilotflying (PF) and pilot-not-flying (PNF) for

each descent. Two pseudo controllers acted as

confederates and provided ATC clearances to

the crews from an ATC laboratory at the

simulator facility. A pseudo pilot acted as a

confederate and interacted with the pseudo

controllers to simulate background chatter

during the descent runs.

Crew training

Crews underwent a training session prior to

beginning each descent procedure. The

training session included presentation and

explanation of procedure charts and activities

on overhead slides and crew performance of

activities in the ACFS with the simulator in

"flight freeze". The training criterion was that

the crews successfully performed procedure

interface activities (e.g. MPE, CDU DL, and

DDL) once before the data collection runs

were flown. In addition to the procedurespecific training sessions, crews received a

background briefing on the CTAS/FMS

integration concept. Also, six of the twelve

crews received training that allowed them to

use the VSD during the study.

Dependent Measures

Crew activity analysis

The number and type of crew activities

performed during descents within each

procedure condition and scenario type were

gathered using the Crew Activity Tracking

System (CATS) (Callantine, et. al., 1999).

Comparisons were made between the baseline

and complex scenarios of the current day and

FMS procedures and complex scenario of the

FMS and CTAS/FMS procedure scenarios.

Crew workload measure

Following each descent crews were asked to fill

out the NASA-TLX subjective workload rating

sheet (see Appendix D). Crews were briefed on
useof thisinstrumentprior to flying the

descents.Subjectivecrewworkloadis a

dependentmeasurein the study.

Workload definition and measure description

Crew workload is a measure used in the

aviation domain to assess the level of difficulty

of required cockpit tasks and procedures. The

difficulty of a task can be inferred from the

interaction between an operator and the

assigned task. Workload refers to the "cost"

imposed on the operator as the task is

performed (Gopher & Donchin, 1986). This

implies that the operator has a limited

information processing capacity with a given

level of available resources (Kahneman, 1973)

and that a given task has variable demands on

those resources. Hence, workload can be

thought of as the relationship between

resource supply and task demand (Wickens,

1992). The more operator resources are

imposed on by a task, the less the resource

supply will be, resulting in higher workload.

Subjective workload measurement is aimed at

capturing task demand and amount of

operator resources expended (operator output)

through operator self-report. Hart and

Staveland (1988) developed the NASA Task

Load Index (NASA-TLX), a multidimensional subjective workload measure that

assesses perceived task demands across three

dimensions and perceived operator outputs

across three dimensions. The task demand

dimensions are Mental Demand, Physical

Demand, and Temporal Demand. The

operator output dimensions are Performance,

Effort, and Frustration. Descriptions of these

workload dimensions are found in Appendix

D.

Crew performance criteria

Crew performance was evaluated on each

descent based on crew compliance with

procedural and ATC requirements. All

requirements were enumerated during crew

training sessions. A list of the specific

performance criteria for each procedure can

be found in Appendix E. This is a dependent

measure in the study.

Video and digital data

To aid with crew performance data collection,

crews were videotaped during performance of

the descent procedures using an overall

cockpit view and a "face" view tracking crew

gazing behavior. The "face" view was used to

track heads-down and dwell times during route

modification activities. Additional cameras

were placed on the MCP, CDUs, Primary Flight

Display (PFD), and Navigation Display to

collect crew activity data. Digital flight data

was also recorded.

Crew questionnaire

Crew acceptance of the proposed procedures

was evaluated with a post-experiment

questionnaire and debriefing session. After

flying the seven descent scenarios, crews were

presented with the post-experiment

questionnaire. Following completion of the

questionnaire crews were interviewed and

debriefed and allowed to ask any questions

regarding the study. Results of the

questionnaire are another dependent measure

in the study. The crew questionnaire is shown

in Appendix F.

Results and Discussion

In order to address the hypotheses described

earlier, this section begins with analyses that

make direct comparisons between the current

day and FMS procedures in both the baseline

and complex scenario conditions. The first

analysis focuses on the changes in the number

and type of crew activities. Activity changes

between current day and FMS procedures in

both the baseline and complex scenarios will

be analyzed. This initial analysis will

investigate the hypothesized activity changes

previously mentioned. These are: (1) The

expected reduction in the number of

observable crew activities (physical inputs into

aircraft automation) when crews fly FMS

descent procedures during the baseline

13
scenario as compared to current day descent

procedures. (2) This reduction is expected to

result from significantly fewer tactical

activities, and therefore, use of primarily

strategic activities. (3) During the complex

scenario, the number of crew activities in the

FMS procedure will be roughly equal to that

of current day descent procedures. Following

the activity analysis, subjective workload

ratings will be analyzed with the same

comparisons used in the activity analysis. Next,

two categories of crew performance will be

assessed. The first category will address crew

compliance with procedural elements such as

automation use requirements and speed and

altitude crossing restrictions. The second

category will address crew performance during

the CTAS/FMS procedure (complex) scenarios

using the interface enhancements to perform

TRACON route modifications during descent.

Finally, selected elements from the post

experiment questionnaire will be discussed.

Crew Activities: Current Day vs. FMS

for Baseline and Complex Scenarios

Planned comparisons

In order to determine the shift in the number

and type of crew activities (automation inputs)

with the introduction of FMS procedures,

analyses were conducted using the Crew

Activity Tracking System (Callantine, 1997).

Of particular interest was the shift in activities

with the introduction of FMS procedures as

compared to current day procedures in both

the baseline and complex descent scenarios

used in the study.

Number of activities

To determine the differences in the total

number of crew activities between the FMS

and current day procedures and the baseline

and current day scenarios, a 2 (Procedures) X

2 (scenarios) analysis of variance (ANOVA)

was conducted and revealed no significant

main effect for procedureF (1,11) = 1.7, 1_>

.05, but a significant main effect for scenario _F

(1,11) = 217.3, p__<.05, such that crews

performed more activities overall during the

complex scenario, M = 37 (S.E. = 1.1), than

during the baseline scenario, M = 19 _ =

1.3). The Procedure x Scenario interaction was

statistically significant,_F (1,11) = 39.7, _ <

.05, such that the pattern of activity count

differences between the FMS and current day

procedures depended on which scenario crews

were flying. To further investigate this

interaction and test the stated hypotheses,

simple comparisons were performed between

the procedures within each scenario. A oneway ANOVA between the FMS and current

day procedures in the baseline scenario

revealed, as expected [hypothesis (1)], that

based on activity model predictions, crews

performed significantly fewer activities on

average when flying the baseline FMS

procedure scenario, M = 14 (S.E. = 1.2) as

compared to the baseline current day

procedure scenario, M = 24 (S.E. = 1.4), F

(1,11) = 42.6, 1_< .05 (see figure 3). However

contrary to activity model predictions

[hypothesis (3)] stating that activity counts

would be roughly equal between current day

and FMS procedures in the complex scenario,

a one-way ANOVA in the complex scenario

showed that crews performed significantly

more activities during the FMS procedure, M

= 41 (S.E. = 1.8), than during the current day

procedure, M = 34 _ = 1.6), F (1,11) = 14,

t2 < .05.

14
This interaction reveals that when flying FMS

descent procedures crews tend to make fewer

automation inputs than in current day flying

[hypothesis (1)] when the procedure can be

flown uninterrupted. If interruptions to the

FMS procedure are made, such as off-route

vectors, intermediate altitude levei-offs, and

speed amendments, then crews tend to make

more automation inputs when flying FMS

procedures than with current day procedures.

This is due to the necessity to shift back and

forth between strategic and tactical automation

modes when flying the FMS procedure. This

finding is consistent with debriefing comments

made by pilots who stated that FMS

procedures are most desirable and require less

automation interaction when ATC does not

intervene with the charted routing. The

changes in the number of specific activity

types and the effects on workload of FMS

procedures in both the baseline and complex

scenarios are discussed below.

Activity types

Activity types used for this analysis were the

strategic (management automation) and

tactical (control automation) categories

described above. Strategic inputs imply use of

higher levels of automation that enable the

aircraft to carryout future behaviors

autonomously. For example, using the LNAV

mode for lateral navigation. Tactical activities

imply use of lower levels of automation with

no autonomy granted to the automation other

than to maintain the current tactical target. For

example, using heading select mode for lateral

navigation.

To investigate differences in tactical activity

counts between the FMS and current day

ti)

.r-I

or.t

r3

0

r_

z

45

40

35

30

25 24

20

15

10

5

0

m Current Day Procedure

,-,FMS Procedure

Baseline Scenario

41

34

Complex Scenario

Figure 3. Mean number of crew activities (+/- SE) in Current Day and FMS procedures

for baseline and complex scenarios.

15
procedures during the baseline and complex

scenarios and to test hypothesis (2), a 2

(Procedures) X 2 (scenarios) analysis of

variance (ANOVA) was conducted and

revealed a significant main effect for

procedure, F (1,11) = 22.4, 12< .05, with crews

performing significantly more tactical

activities during the current day procedure, __M

= 16 (S.E. = 0.9), over the FMS procedure, __M

= 13 (S.E. = 1.5), a significant main effect for

scenario, F (1,11) = 143.6, 12< .05, such that

crews performed significantly more tactical

activities during the complex scenario, __M= 20

(S.E. = 0.5) over the baseline scenario, M = 10

(S.E. = 0.9), and a significant interaction, _F

(1,1 I) = 18.0, l_ < .05, such that the pattern in

the number of tactical activities performed

between the two procedures depended on

which scenario was being flown (see figure 4).

To investigate this interaction further one-way

ANOVAs were conducted between the two

procedures within each scenario condition. As

expected in the baseline scenario [hypothesis

(2)], a one-way ANOVA showed that crews

made significantly fewer tactical inputs in the

FMS procedure, M = 6 (S.E. = 0.8), as

compared to the current day procedure, M

=13 (S.E. = 0.8), _F(1,11) = 31.8, _ < .05, but

in the complex scenario, there were no

significant differences, 12> .05, in the number

of tactical activities between the current day, M

= 20 (S. E. = 0.7), and FMS, M = 20 (S.E. =

0.7), procedures. The drop in tactical activity

count for the FMS procedure in the baseline

scenario is congruent with the overall activity

count pattern for the baseline scenario, but the

equal procedure activity count means in the

complex scenario does not reflect the rise in

the number of FMS activities in this scenario

in the overall activity count. A look at the

strategic activity counts should clear up this

discrepancy and complete the picture.

To investigate the differences in the strategic

activity counts between the FMS and current

day procedures during the baseline and

complex scenarios, a 2 (Procedures) X 2

(scenarios) analysis of variance (ANOVA) was

conducted and revealed significant main

effects for procedure, F (1,11) = 8.5, 12< .05,

with crews performing significantly more

strategic activities during the FMS procedure,

M = 14 (S.E. = 1.5), over the current day

procedures, M = 13 (S.E. = 0.5), a significant

main effect for scenario, _F (1,11) = 142.3, 12<

.05, with crews performing significantly more

strategic activities during the complex

scenario, M = 17 _ = 1.0), than during the

baseline scenario, M = 10 (S.E. = 0.6), and a

significant interaction, _F(1,11) = 46.0, 12< .05,

such that the number of strategic activities

performed between the two procedures

depended on the type of scenario being flown

(see figure 5). To investigate this interaction, a

one-way ANOVA of the number of strategic

inputs made during the baseline scenario was

conducted which revealed that crews made

significantly fewer strategic inputs with the

FMS procedure, M = 8 (S.E. = 0.8), than in

the current day procedure, M =11 CS.E. =

0.7), _F(1,11) = 24.7, ,_ < .05. However, the

trend reversed in the complex scenario, where

a one-way ANOVA revealed that crews made

significantly more strategic inputs during the

FMS procedure, M = 21 (S.E. = 1.2), than in

the current day procedure, M =14 (S.E. =

0.5), F (1,11) =, 12< .05. This rise in the

number of strategic FMS activities in the

complex scenario, not found in the tactical

activity counts, accounts for the overall activity

interaction previously discussed. (see figure

3).

These results highlight the expected large

decline in tactical inputs during descent in the

baseline scenario with the introduction of FMS

procedures. There was also a drop in the

number of strategic inputs in the baseline

16
25

20

>

._ 15 -

0 :

P_ _oE

z

r-,

.N

5 -

[] Current Day Tactical

a FMS Tactical

13

20 20

Baseline Scenario Complex Scenario

Figure 4. Mean number of strategic crew activities in Current Day and FMS procedures for

baseline and complex scenarios.

25

-_ 2O

* 15

0

_ 10

..Q

Z

= 5

0

[] Current Day

Strategic

a FMS Strategic

11

Baseline Scenario

14

Complex Scenario

Figure 5. Mean number of tactical crew activities in Current Day and FMS procedures for

baseline and complex scenarios.

17
scenario with the introduction of FMS

procedures. With this drop in the number of

inputs comes an increase in the number of

latent behaviors that are relegated to the

automation. Both these decreases led to a

reduction in perceived workload in the

baseline FMS scenario, but they raise questions

as to the implications of increased automation.

These issues are discussed below.

The results also indicate that when crews are

forced to deviate from and rejoin FMS

procedures to comply with ATC constraints as

in the complex scenario, the number of tactical

activities required rises dramatically (from a

mean of six to a mean of twenty) but matches

current day procedure levels (twenty) for the

same complex scenario. This dramatic rise for

the FMS procedure is also found in the

strategic activity counts. During the FMS

procedure crews performed an average of

eight strategic activities in the baseline scenario

and an average of twenty-one in the complex

scenario. However, unlike the pattern found in

the tactical activity analysis, strategic activity

levels in current day scenario do not rise so

dramatically between the baseline and

complex scenarios (from 11 to 14). This

finding confirms the notion that during the

complex scenario, where there are multiple

interruptions to the FMS procedure, crews are

required to make frequent strategic inputs,

such as re-engaging LNAV and VNAV modes,

in order to resume the procedure. This

illustrates that because of the strategic nature

of FMS procedures, any deviations from the

planned and programmed trajectory can lead

to a precipitous rise in the required crew

activities, both tactical and strategic. This

brings into question of the utility of

continuing FMS procedures when deviations

are required. This issue and the effects on

workload of these activity changes is discussed

below.

Subjective Workload: Current Day vs.

FMS for Baseline and Complex Scenarios

Overall workload ratings were in the low range

with a grand mean of 4.8 (S.E. = .4) over a

18

one to twenty point range. Crews reported that

lack of out-the-window traffic, no TCAS

display, no adverse weather, minimal radio

chatter, no company communications

requirements, and no interruptions from the

passenger cabin during the simulation to be

the major factors influencing the lower

workload ratings. Crews also commented that

the lack of these factors detracted from the

realism of the simulation.

To determine workload differences between

the two procedures within the two scenarios, a

2 (procedure) X 2 (scenario) ANOVA was

conducted which revealed no main effect for

procedure, F (1,23) = 1.1, I_ > .05, a main

effect for scenario, _F (1,23) = 5.0, p < .05,

such that crews reported higher workload

during the complex scenario, M = 5.1 (S.E. =

0.3), than during the baseline scenario, M =

4.4 (S.E. = 0.3), and a significant interaction,

_F(1,23) = 6.0, _ < .05, such that the pattern of

workload differences between procedures

depended on the which scenario was being

flown (see figure 6).

Planned comparisons

To investigate the nature of the interaction, the

following workload comparisons were

conducted based on comparisons of particular

procedure scenarios from the activity analyses

above.

Baseline current day procedure scenario vs.

baseline FMS procedure scenario

A one-way ANOVA revealed that pilots

reported significantly lower mean workload

scores in the baseline FMS procedure scenario,

M = 3.9 (S.E. = 0.5), than in the baseline

current day procedure scenario, M = 5.0 (S.E.

= 0.5), _F(1,23) = 6.0, I_ < .05 (see figure 6).

Complex current day procedure scenario vs.

complex FMS procedure scenario

A one-way ANOVA showed that pilots

reported a non-significant difference in mean

workload scores in the complex FMS

condition, M = 5.2 _ = 0.5), than in the
complexcurrentdaycondition,M = 4.9

= 0.5). F_(1,23) = .18, 12> .05 (see figure 6).

For the baseline scenario, the lower mean

activity count corresponds to the lower

reported workload in the FMS procedure as

compared to the current day procedure. But

the higher mean activity count for the FMS

procedure complex scenario does not show a

corresponding significant increase in workload

over the complex current day procedure

scenario.

These results indicate that use of FMS

procedures during the descent phase of flight

leads to reduced flight crew subjective

workload ratings when these procedures can

be flown uninterrupted. These procedures

include significantly fewer crew automation

inputs than current day procedures, and more

use of management automation. The use of

management automation includes use of the

VNAV and LNAV modes throughout the

descent up until glideslope capture. This is

evidence that higher levels of automation use

can lead to lower crew subjective workload.

When FMS procedures are interrupted by

common ATC clearance amendments, forcing

crews to switch between high and low levels of

automation multiple times, the average number

of crew activities nearly triples (from a mean

of 14 to a mean of 41) as compared to the

current day rise from 24 to 34 activities.

However, workload does not rise significantly

above current day levels. This indicates that

the requirement to shift between automation

levels during descent does not necessarily

increase the workload burden on flightcrews.

During the debriefing sessions, many crews

remarked that they are accustomed to making

transitions between higher and lower levels of

automation in their current day environment.

The conclusion that workload is not increased

when crews are required to frequently switch

between management and control automation

is made with caution due to the artificially low

workload levels experienced by crews overall

during the study. These findings need further

validation from studies with more realistic task

loading on flightcrews during the descent

phase of flight. It should also be noted that a

6 -

¢3

O

----' 4

o

Q

1 -

o

m Current Day Procedure

= FMS Procedure

5 4.9

i

.....

Baseline Scenario Complex Scenario

Figure 6. Mean subjective workload ratings (+/- SE) for Current Day and FMS procedures in the

baseline and complex scenarios.

19
reduction in workload is not necessarily the

goal of these procedures. The objective was to

ensure that with the introduction of FMS

procedures, workload remains at manageable

levels. These results suggest that this is the

case.

The current results indicate that if FMS

procedures are implemented, then greater use

of management automation in the descent

phase of flight will result. The drop in

workload levels during the baseline FMS

procedures as compared to baseline current

day procedures raises questions about the

possibility of introducing excessively low

workload levels during FMS procedures. The

use of management automation assigns most

flying tasks to the autopilot after the flight

crew loads the FMS procedure and sets up the

automation to carry-out the procedure

parameters. The flight crew then acts primarily

as a system monitor. This calls on concerns

raised by Billings (1996) that greater use of

management automation may decrease crew

involvement in the flying task. This study does

not directly address this issue. Performance

results discussed below address how well crews

comply with FMS procedure requirements, but

give no indication of changes in crew vigilance

due to increased monitoring demands imposed

by FMS procedures. This issue requires the

use of specific techniques to test crew vigilance

such as the inclusion of automation errors or

failures during use of these procedures.

Crew Performance: Compliance with

Procedural Elements

The first set of performance results refers to

how well crews complied with the requirements

and constraints of the procedures flown. This

includes compliance with speed and altitude

constraints prescribed by each procedure. Also

included are the operational requirements in

the FMS (and CTAS/FMS) procedures to use

VNAV and LNAV during the descent.

Speed and altitude compliance

In two of the seventy-four descents (2.7%),

crews were more than ten knots fast at the

2O

TRACON boundary crossing restriction

waypoint, FEVER. In two of the seventy-four

descents (2.7%), one crew was more than 300

feet high at the FEVER restriction and another

crew was more than 300 feet low. In four of

the seventy-four descents (5.4%), crews were at

least ten knots fast at the DELMO waypoint

along the TRACON routing. In 30 of the 54

FMS procedure descents (56%), crews were at

least ten knots fast as they flew the downwind

approach routing during the FMS procedure.

In 9 of the 74 descents flown (12.2%) crews

were at least ten knots slow at the assigned

final approach fix crossing speed.

The high number of crew overspeeds (56%)

observed during the downwind approach

routing of the FMS procedure may be due to

multiple factors. First, the charted vertical

• routing for the downwind leg of the FMS

procedure requires a relatively steep angle of

descent in order to comply with altitude

crossing restrictions along the route. Second,

when the aircraft is in the VNAV vertical

mode, the automation prioritizes tracking of

the computed path and does not track the

programmed speed. These two factors

combine so that, without pilot actions, the

aircraft tends to overspeed during this

segment. Crews had the option to extend flaps

and/or speed brakes to keep the aircraft at

required speeds. They were slow to use these

controls in the overspeed cases. This may

indicate that pilots are willing to sacrifice

precision in flying charted speeds in order to

meet altitude restrictions, save fuel, or maintain

passenger comfort. Also, this portion of the

routing may not be considered as speed

constrained by pilots, because speed

constraints are only specified at waypoint

crossings. It is important for the accuracy of

the CTAS system and the ATC advisories it

computes that crews comply as closely as

possible with charted and cleared speeds at and

between waypoints. Therefore, the high

number of crew overspeeds while on the FMS

procedure during this portion of the flight is a

concern. It brings into question the utility of

requiring use of VNAV to descend on the
FMSprocedure.TheFLCHandV/Smodesare

alternativewaysof descendingthe aircraft that

puts more priority on speed tracking and thus

may serve the needs of the CTAS system better

than the VNAV descent mode.

Nine of seventy-four crews (12.2%) crossed

the final approach fix more than ten knots

slower than the charted and cleared speed of

170 knots. This speed is set for the CTAS

system to precisely control an aircraft's arrival

time at the runway threshold. During the post

experiment debriefings crews that failed to

comply with this speed restriction mentioned

two reasons for doing so. First these crews

mentioned that they habitually set their

approach reference speed into the autopilot

upon receipt of the approach clearance by

ATC. They also mentioned the requirement to

be "stabilized on the approach" by the final

approach fix when flying in instrument

conditions. This means that they have the

aircraft fully configured and that airspeed is

equal to the approach reference speed. Given

this requirement, crews said that they felt

uncomfortable crossing this point at the higher

assigned speed and preferred to be slowed

down to their reference speed by this point.

VNA V and LNA V use

When flying the FMS procedures, crews were

required to use the VNAV and LNAV

autopilot modes. During particular parts of the

complex scenario, use of these modes was

prevented by ATC clearances that diverted the

aircraft from the charted FMS procedure.

Crews were eventually cleared to resume the

FMS procedure following the diversions which

reinstated the VNAV and LNAV requirements.

In 11 of the 54 (20.4%) FMS descents flown,

crews used autopilot modes other than VNAV

(i.e. FLCH or V/S) to descend after being

diverted from and then re-cleared for the FMS

procedure. All crews used the LNAV autopilot

mode during all 54 of the FMS procedures.

These results indicate some crew resistance to

use of VNAV, or preference for the FLCH

and/or V/S modes, during the descent phase of

flight. Crew responses on the post-experiment

questionnaire indicate some crew discomfort

with the VNAV mode in the TRACON

airspace. This issue if discussed at length

below.

All crews used the VNAV top of descent point

when required during the FMS procedures.

Crew Performance: CTAS/FMS

Procedures Used During Complex

Scenarios

In this section, the analysis shifts and addresses

crew performance using the interface

enhancements during the complex scenarios.

Recall that these procedures are termed

CTAS/FMS procedures.

TRA CON route modifications

Crew performance of the CTAS route

modifications was evaluated for the TRACON

route modifications only. This approach was

chosen to facilitate comparison among the

three interface concepts (MPE, CDU DL, and

DDL) in which crews flew the complex

scenarios and were able to respond to ATC

route modifications using these technologies.

Recall that MPE was not used for the Center

route modification, and that it was the most

progressive procedure proposal in the study.

To evaluate crew performance of the CTAS

TRACON route modification interface

activities, a number of measures were

collected. First, an evaluation of crew

performance of the particular elements of each

activity was performed. This entailed viewing

the videotape data to record the elements of

each activity that crews performed and

comparing this with the expected performance

based on how crews were briefed and trained

for each interface activity. This analysis also

included the recording of any entry errors by

crews. Second, the total time to complete the

route modification activity was recorded.

Third, the total heads down time for each crew

member was determined by viewing the

"face" cameras from the videotape data.

Heads down time is defined here as the total

amount of time that a crew member spends

21
with his/her gaze on the CDU during the route

modification activity. Fourth, the total heads

down time for each observation was broken

down into a collection of dwell times. Dwell

time is defined here as the duration of a single

fixation on the CDU during performance of

the route modification activity. The sum of

these dwell times equals the total heads down

time. Note that dwell time was so large

compared to travel time that travel time was

ignored in this analysis.

Performance of activity elements

All crews in the MPE interface condition

performed the activity without any errors or

element omissions. In the CDU DL interface

condition, four of twelve pilots not flying

(PNFs) (33%) omitted the step of reading the

clearance text aloud upon receiving the data

link message. Five of twelve crews (42%)

executed the route modification before

reviewing the newly loaded route. No entry

errors were made. In the DDL interface

condition, three of twelve PNFs (25%) failed to

read the message text aloud upon receipt of

the data link message, and three of twelve

crews (25%) failed to review the route

modification before executing and accepting

it. One of the twelve DDL crews (8%) failed to

both read the message text aloud and to review

the route modification before executing and

accepting it. No entry errors were made in the

DDL interface condition.

The cases in which the PNFs failed to read the

data link message text aloud and where crews

failed to review the message may indicate task

shedding due to crew occupation with other

tasks. This possibility cannot be evaluated in

this study because continuous crew task

tracking data was not collected. However the

low workload study environment that crews

reported is not congruent with the notion that

crews were saturated with other tasks. This

issue of task omissions may be addressed in

future studies by introducing errors or

unforeseen elements in route clearance

messages. It should be noted that these omitted

tasks were checking and monitoring tasks and

22

were not essential for successful completion of

the route modifications activities. This means

that the system can work properly when pilots

do not perform these tasks. This is true with

most procedural aspects of flying, such as use

of checklists. It emphasizes the necessity to

make checking and monitoring during data

link operations a required standard operating

procedure for crews.

Total activity time

The mean total activity times in seconds for

the three route modification interfaces were, M

= 24.1 (S.E. = 2.3) for MPE, M = 26.7 (S.E.

= 2.7) for CDU DL, and _.M= 20.3 (S.E. =

2.1) for DDL. Although DDL took the

shortest amount of time on average, a one-way

ANOVA revealed that mean total activity time

differences among the three interfaces were

not statistically significant, _F (2,22) = 2.04, I_ >

.05.

These route modification activities resulted in

activity times in the range of twenty to twentyseven seconds. This raises the question as to

what amount of time to complete this activity

is operationally acceptable. To the author's

knowledge, no objective standard has been set.

Crews frequently comment that they prefer to

minimize time programming their FMS in the

terminal area because it distracts from the vital

tasks of instrument and out-the-window

monitoring required in this airspace. All

participants in the current study found these

interfaces to be operationally acceptable for

route modifications in the terminal area. The

only crew concerns focused on the MPE

interface which crews felt produced excessive

heads-down time. This will be discussed below

with the heads-down time and dwell time

results. As mentioned before, crew operational

acceptance of these interfaces and activities

based on this study should be taken with

caution. It is necessary to further validate the

use of these interfaces and activities in flight

conditions that more closely match those

found at busy airports. This includes

simulations with out-the-window traffic, TCAS
traffic,andlikely distractionsencountered

duringdescentintotheterminalarea.

Total heads down time

Mean total heads down times for the PNFs

(crew member who performed that majority of

elements for each route modification) were, M

= 15.3 (S.E. = 1.8) for MPE, M = 18.2 (S.E.

= 1.8) for CDU DL, and_M = 6.3 (S.E. = 1.7)

for DDL (see figure 7). The DDL interface

required less than half the heads down time

than did the MPE interface and roughly onethird the heads down time than did the CDU

DL interface. A one-way ANOVA of headsdown time revealed significant differences

among the three interfaces, _F(2,22) = 13.3, 12

< .05. Simple comparison ANOVAs revealed

that the DDL interface heads-down time was

significantly shorter than both the MPE (F

(1,11) = 16.9, I_< .05) and CDU DL (_F (1,11)

= 21.6, 12< .05) interfaces. There was no

significant difference between the MPE and

CDU DL heads down times,_F (1,11) = 1.4, p >

.05

Mean total heads down time for the Pilots

flying (PFs) were,._M = 5.2 (S.E. = 1.2) for

MPE, __M= 9.7 (S.E. = 1.4) for CDU DL, and

M = 4.1 (S.E. = 1.2) for DDL (see figure 8).

The DDL required the least amount of heads

down time for the PF, a little more than a

second less than the MPE interface, but more

than five-and-a-half seconds less than the CDU

DL interface. A one-way ANOVA of headsdown time revealed significant differences

among the three interfaces, _F (2,22) = 9.27, 12

< .05. Simple comparison ANOVAs revealed

that the CDU DL interface heads-down time

was significantly longer than both the MPE (F

(1,11) = 17.9, t_< .05) and the DDL (_F (1,11)

= 17.7, 12< .05) interfaces. There was no

significant difference in heads-down time for

the PF between the MPE and DDL interfaces, _F

(1,11) = .42,12 > .05.

Dwell time

PNFs had the shortest mean dwell times while

using the DDL interface M = 2.2 (S.E. = .4),

followed by the CDU DL interface __M= 8.3

(S.E. = 2.4). The longest dwell times for PNFs

occurred with the MPE interface, __M= 10.7

(S.E. = 2.1) (see figure 7). A one-way

ANOVA of dwell time revealed significant

differences among the three interfaces, _F

(2,22) = 7.30, p < .05. Simple comparison

ANOVAs revealed that the DDL interface had

a significantly shorter mean dwell time than

both the MPE (F (1,11) = 15.1, 12< .05) and

CDU DL _ (1,11) = 5.9, 12< .05) interfaces.

There was no significant difference in mean

dwell time for the PNF between the MPE and

CDU DL interfaces,_F (1,11) = 1.2, 12> .05

The mean dwell times for the PFs were less

varied than for the PNFs: ___M= 2.8 (S.E. = 0.9)

for MPE, M = 2.5 (S.E. = 0.3) for CDU DL,

and 2.0 (S.E. = 0.5) for DDL (see figure 8). A

one-way ANOVA of dwell time revealed nonsignificant differences among the three

interfaces, _F (2,22) = .39, p > .05.

The results indicate that the DDL route

modification interface required the least

amount of heads-down time for both PNFs

and PFs among the three interfaces. As with

task time, the question remains as to what

amount of heads-down time is operationally

acceptable. While it is difficult to establish an

objective standard for what is acceptable, pilot

comments point to the common sense answer

of "the less heads-down time in the TRACON,

the better". Perhaps a more operationally

significant measure of time taken away from

instrument and out-the-window scan is dwell

time, because it represents the duration of a

fixation on the CDU. If dwell times are short

despite long cumulative heads-down times, this

may indicate that pilots are time sharing, or

have the opportunity to time share the route

modification task with instrument and out-thewindow scanning tasks. To validate this

assumption, a higher resolution analysis of

crew gaze and fixation behavior is required. In

the current study, the PNFs average dwell time

was considerably shorter with the DDL

interface in addition to the considerably

shorter total heads-down time found with this

interface. These results indicate that the DDL

23
O

u

O'J

20-

15.

i

I0_

0

15.3

''" \\ Heads-down PNF

10_'--- _... 8.3"--. xxx _ 6.3

Dwell PNF " -.

"_'2.2

MPE CDU DL DDL

Interface

Figure 7. Mean total heads-down time and dwell time (+/- SE) for pilots not flying.

20-

15

0

U

O2

10-

_

0

t._9x 7 Heads-down PF

5.2 1- --'_'-t .a_ -_, _

.. t Dwell PF _ x -._T

.... _J --_ _4.1 2.8 --- -275- ...... 2

MPE CDU DL DDL

Interface

Figure 8. Mean total heads-down time and dwell time (+/- SE) for pilots flying.

interface accomplished the objective of

keeping the PNF more heads-forward during

the route modification activity. For the PFs

average dwell times for all interfaces were

around two seconds indicating a tendency for

the PF to include observation of the route

modification activity into the flight deck scan.

However, average total heads-down time for

the PFs was significantly longer with the CDU

DL interface than with the others indicating a

24
tendency for the PF to dwell more frequently

on the CDU during the route modification

activity.

Post Experiment Questionnaire

The following results cover only selected items

from the post-experiment questionnaire. A full

listing of all pilot responses to the yes or no

and Likert scale questions is listed in

Appendix G

Acceptability of FMS procedure

Crews were asked to respond to a series of

questions regarding the acceptability of the

Glenn Rose FMS Arrival and Delmo FMS

Transition Procedure. The following are the

results from selected items.

In response to the question: "Was any portion

of the Glenn Rose FMS Arrival and Delmo

FMS Transition Procedure unclear or

confusing?", four of the twenty-four pilots

answered yes. The stated reasons for this

confusion were, (1) "speed assignments and

alterations were confusing at first, then after

re-reading the notes it became clear." (This

comment refers to note #2 on the chart that

states speed and downpath routing

amendments do not cancel the FMS

procedure, which is contrary to current

procedures), (2) "The ... procedure notes

were too long. Chart should be read quickly

and not have to be studied.", and (3) "(it was

unclear) at what point to abandon the LNAV

transition and switch to localizer for

approach."

In response to the question: "How acceptable

or unacceptable was the requirement to Use

LNAV during The Delmo FMS Transition?",

seventeen crew members responded that it was

very acceptable, six responded that it was

somewhat acceptable, and one responded that

it was borderline. In response to the question:

"How comfortable or uncomfortable were

you using LNAV during the Delmo FMS

Transition?", nineteen pilots responded that

they were very comfortable and five

responded that they were somewhat

comfortable (see figures 9 and 10). In

response to the question: "How acceptable or

unacceptable was the

requirement to use VNAV during the Delmo

FMS Transition?", eleven pilots said this was

very acceptable, six said it was somewhat

acceptable, three pilots said it was borderline,

three said it was somewhat unacceptable, and

one pilot said it was very. In response to the

airspace, approximately one-quarter of the

twenty-four pilots felt that use of VNAV in the

question: "How comfortable or

uncomfortable were you using VNAV during

the Delmo FMS Transition?", eleven pilots

responded that they were very comfortable, six

said they were somewhat comfortable, two said

they were borderline comfortable, three

responded that they were somewhat

uncomfortable, and two said that they very

•uncomfortable (see figures 11 and 12).

These results reveal that while pilots had no

objections to use of LNAV in the TRACON

airspace, approximately one-quarter of the

twenty-four pilots felt that use of VNAV in the

TRACON was unacceptable and that they were

uncomfortable with it in some way. Pilots

confirmed these views during post experiment

debriefings. All pilots mentioned that use of

LNAV was fine within the TRACON. Those

pilots who found the use of VNAV in the

TRACON unacceptable or problematic stated a

number of reasons listed here: (1) Use of

VNAV below 10,000 feet lowers workload too

much which may produce complacency and

make the crew feel out of the loop; (2) Use of

VNAV requires increased monitoring and

constant asking of the question: "Is it going to

do it?"; (3) Use of VNAV is made difficult

when changes are made to the charted routing,

forcing interaction with the CDU resulting in

heads-down in the TRACON; (4) There are

too many speed and altitude crossing

restrictions requiring constant cross checking

with charts, the CDU, and aircraft performance

increasing workload and heads-down time;

(5) Using FLCH or V/S makes it easier to get

what you want out of the airplane; (6) (I) feel

more assured MCP flying in the TRACON, (7)

25
2O

0

m

to

_._

0

E

Z

1

6

:,i'_ii_iI_ _

17

:_i! i_i !i/ ̧:

i_!I _

!:

!!

T

Very Somewhat Borderline Somewhat Very

Unacceptable Unacceptable Acceptable Acceptable

Figure 9. Pilot questionnaire responses to the question: How acceptable or unacceptable was the

requirement to use LNAV in the TRACON airspace?

20 19

tO

c'_

112

0

_._

Q

c_

E

Z

5

Very Somewhat

Uncomfortable Uncomfortable

Borderline Somewhat Very

Comfortable Comfortable

Figure 10. Pilot questionnaire responses to the question: How comfortable or uncomfortable were

you using LNAV in the TRACON airspace?

VNAV "jerks" the a/c around and does not

provide smooth transitions between crossing

restrictions; (8) the wind model in the FMS

must match the actual winds to get good

performance from VNAV, they usually do not

26
match; and (9) anti-ice changes the descent

profile and FMS performance.

These complaints about use of VNAV in the

TRACON airspace are varied. Comments (1)

and (2) seem to address the baseline scenario

condition in which the crew is cleared for the

entire vertical FMS routing without deviation.

This minority of crews reported feeling out of

the loop and complacent when they delegated

full vertical guidance to the automation which

required vigilant monitoring to ensure the

VNAV function meets all the charted

restrictions. These comments echo Billings

(1996) statements regarding use of

management automation as increasing

cognitive (monitoring) load and potentially

decreasing crews involvement in the flying

task.

Comments (3) and (4) refer to increases in

workload and heads-down time with VNAV

usage when the route is complex or altered by

ATC. This suggests the need to keep charted

routes simple and quickly and easily

decipherable, and to create effective route

modification interfaces that minimize headsdown time and clearly and efficiently

highlight changes to the FMS routing.

Comments (5) and (6) refer to pilot

preferences for the less automated vertical

modes FLCH and V/S. Pilots report that these

modes provide more precise speed control

than VNAV and also provide the current speed

target in the MCP speed window for quick

heads-up reference and manipulation.

Comments (7) through (9) refer to side-effects

of VNAV use that degrade its performance

and precision.

In response to the question: "In General, how

was your workload affected when flying the

Glenn Rose FMS Arrival and Delmo FMS

Transition as compared to your current day

descent procedures", four pilots responded

that their workload was greatly decreased, nine

pilots responded that their workload was

somewhat decreased, five said that their

workload was unaffected, and four responded

that their workload was somewhat increased. In

response to the question: "In general, how was

you monitoring behavior affected when flying

the Glenn Rose FMS Arrival and Delmo FMS

Transition procedure as compared to your

current day descent procedures?", three pilots

said that their monitoring was greatly

increased, nine said that it was somewhat

increased, eight said that their monitoring

behavior was unaffected, and three said that

their monitoring behavior was somewhat

decreased (see figures 13 and 14).

Pilot responses to these two questions indicate

that the FMS procedures generally reduce

workload while increasing monitoring

demands. This dissociation between workload

and monitoring reports indicates that pilots

perceive an increase in monitoring demands to

create a decrease in workload. This explains

the complacency and out-of-the-loop

experienced by crews during the FMS

procedures.

In response to the question: "How acceptable

or unacceptable was the Glenn Rose FMS

Arrival and Delmo FMS transition procedure

as a whole?", ten pilots said it was very

acceptable, eleven said it was somewhat

acceptable, two said it was borderline, and one

said it was somewhat unacceptable (see figure

15).

The following are typical favorable comments

made about the FMS procedure: "Procedure

was easy and straight forward. Any confusion

or difficulties had to do with the airspeed and

altitude requirements which required a

different cockpit scan"; "The FMS procedure

made situational awareness much better.

Planning ahead was much simpler. It was very

easy to plan speed changes and flap/gear

extension". The following are typical

cautionary or negative comments made about

the FMS procedure: "...was not comfortable

with the VNAV descent in the transition phase

of flight"; "Using VNAV to fly the FMS

transition was a bad idea"; "This procedure

27
20-

_D

g_

t-,

O

I:X

O

4D

e_

Z

6

3 3

11

*i!_i!"i?i_;i_i _ii;_i !il_

: iiii_ii_ •

T

Very Somewhat Borderline Somewhat Very

Unacceptable Unacceptable Acceptable Acceptable

Figure 11. Pilot questionnaire responses to the question: How acceptable or unacceptable was the

requirement to use VNAV in the TRACON airspace?

20 -

t/l

o

¢a,..i

O

2;

0

3

2

6

11

Very Somewhat

Uncomfortable Uncomfortable

Borderline Somewhat Very

Comfortable Comfortable

Figure 12. Pilot questionnaire responses to the question: How comfortable or uncomfortable

were you using VNAV in the TRACON airspace

was not difficult or task overloaded. It

required a little more concentration in terms of

monitoring the automation to ensure it is

doing what it is commanded. Setting the

altitude down to the final cleared altitude

28
2O

i1)

O

0

_0

E

Z

9

5

4 ..... 4

!

Greatly Somewhat Unaffected Somewhat Greatly

Decreased Decreased Increased Increased

Figure 13. Pilot questionnaire responses to the question: In General, how was your workload

affected when flying the Glenn Rose FMS Arrival and Delmo FMS Transition as compared to

your current day descent procedures?

20

t"

O

c_

0

_,

E

9

8

_ i i _

3

Greatly Somewhat Unaffected Somewhat Greatly

Decreased Decreased Increased Increased

Figure 14. Pilot questionnaire responses to the question: In general, how was you monitoring

behavior affected when flying the Glenn Rose FMS Arrival and Delmo FMS Transition

procedure as compared to your current day descent procedures?

could pose some problems with crews

descending before "authorized" if they

change to descent modes other than VNAV, or

if descent mode changes automatically";

"Everything was okay. I still don't completely

trust VNAV in tight situations. LNAV is okay

though"; "Too many speed restrictions and

waypoints."

Acceptability of CTAS/FMS procedures

The pilots were asked to respond to a series of

questions to assess their acceptance and

opinion of the CTAS/FMS procedures that

they performed during the study. The

following are the results from selected

questionnaire items.

29
In response to the question: "How acceptable

or unacceptable was the requirement to use

data link for route modifications while in

cruise near the top of descent?", eighteen

pilots said it was very acceptable and six said

that it was somewhat acceptable. In response to

the question: "How acceptable or

unacceptable was the requirement to use data

link for route modifications while in the

TRACON?", Twelve responded that it was

very acceptable, nine said it was somewhat

acceptable, two that it was borderline, and one

pilot responded that it was somewhat

unacceptable (see figure 16). This indicates

that while all pilots felt comfortable with use of

datalink before top of descent, three of twelve

(25%) of pilots felt that use of datalink in the

TRACON airspace was either borderline or

somewhat unacceptable. Debriefing comments

regarding this issue focused on the increased

heads-down time required when processing

datalink messages that pulled the PNF out of

the instrument and traffic scanning tasks

during a critical phase of flight. Pilots were

asked to state the FMS route modification

interface they most preferred. Fifteen said the

DDL, seven the CDU DL interface, and one the

MPE interface (see figure 16). The strong

preference for the DDL is congruent with the

shorter heads-down and dwell times required

with this interface and pilot concerns about

excessive heads-down time in the TRACON

airspace.

Conclusions and Future Work

This study investigated the impact of FMS and

CTAS/FMS descent procedures on crew

activities, workload, and performance. It also

assessed crew acceptance of these procedures.

The findings can be summarized as follows:

(1) The study demonstrated that use of FMS

descent procedures can significantly reduce

both crew activity load and workload below

current day flying levels in situations where

the charted procedure can be flown

uninterrupted. However, activity loads

significantly increase beyond current day

levels and workload returns to current day

levels when FMS descent procedures are

interrupted by common ATC interventions

and CTAS routing advisories. Further study is

required to validate these conclusions and

investigate issues that were not addressed in the

current study. (2) Crew performance during

use of FMS procedures was generally

acceptable but suffered in a few key areas.

More than half of the crews demonstrated

poor speed control between waypoints in the

TRACON airspace, and about one in ten crews

did not comply with charted and cleared

speeds at the final approach fix. (3) Pilots were

very comfortable with the use of LNAV in the

TRACON airspace, but at least one in four

pilots expressed discomfort with or nonacceptance of the use of VNAV in the

TRACON airspace for safety, performance,

and/or situation awareness reasons. (4) One in

five crews demonstrated that once taken off of

the FMS procedure and then re-cleared for it,

that they were reluctant to re-engage the

VNAV mode. (5) Crews in general liked the

FMS and CTAS/FMS procedures and found

them acceptable. Crews mentioned the ability

to look ahead and plan energy management as

a primary positive contribution of the

procedures. (6) Crews were comfortable with

the use of datalink for route modifications in

both the Center and TRACON airspace and

expressed concern mainly for interfaces that

produce excessive heads-down time or require

manual entry of alpha-numeric information.

(7) Crews demonstrated the best overall

performance in terms of minimizing headsdown and dwell times using the DDL interface

as compared to the CDU DL and MPE

interfaces.

3O
20-

O

e-.

O

c_

q_

0

Z

0

1

11

H

10

i '

Very Somewhat Borderline

Unacceptable Acceptable

Somewhat Very Acceptable

Unacceptable

Figure 15. Pilot questionnaire responses to the question: How acceptable or unacceptable was the

Glenn Rose FMS Arrival and Delmo FMS transition procedure as a whole?

20

O1

o

ttl

o

r-i

H

Z

0

1

2

9

12

Very Somewhat

Unacceptable Unacceptable

Borderline Somewhat Very

Acceptable Acceptable

Figure 16. Pilot questionnaire responses to the question: How acceptable or unacceptable was the

requirement to use data link for route modifications while in the TRACON?

31
To validate the workload results gathered in

this study, future simulation studies should

include contexts that more closely match realworld operations. While creating this

environment in a simulation setting is difficult,

pilot recommendations gathered in the current

study helped produce a list of practical steps

for accomplishing this goal. Pilots reported

that out-the-window (visual) traffic, a TCAS

display, realistic radio chatter, adverse or

unpredictable winds and weather, requirements

for company communications, and cabin

interruptions were the most conspicuous realworld elements missing from the current

study.

The drop in workload found in the baseline

FMS procedure scenario runs raised the issue

of possible under stimulation of pilots creating

the potential for automation-induced

complacency on the flight deck. As the air

traffic system evolves, and FMS descent

procedures become more commonplace,

studies addressing this issue become necessary.

While performance problems were more

common in this study when crews were flying

FMS procedures (e.g. poor speed control), this

cannot be attributed directly to crew

automation complacency as other factors

previously mentioned likely played a role. An

experimental design, well controlled and

geared specifically toward the question of

complacency is required.

Because of the resistance by a few of the pilots

to the use of VNAV in the TRACON, and the

inherent speed control problems this mode

introduces, the requirement to use VNAV in

FMS procedures should be carefully

considered. This is especially true for

CTAS/FMS integration where speed control is

vital to operational success. The question of

what benefits are derived with the use of

VNAV in the TRACON seems appropriate.

Based on the results in this study, the VNAV

requirement has been removed from the

planned follow-up study on CTAS/FMS

integration.

Crew performance and acceptance of the

TRACON route modifications required for

CTAS/FMS integration is an encouraging

finding. As with the FMS procedure workload

•findings, these results also need further study

and validation in the more realistic simulation

context outlined above. Also, the introduction

of message errors and equipment malfunctions

in future studies would provide a measure of

crew error trapping capabilities with these

interfaces. The fact that some crews failed to

read the initial datalink message text and/or

failed to review the loaded route raises

concerns of crew complacency with this

communication format.

32
References

Billings, C. E. (1996). Human-centered

aviation automation: principles and

guidelines. NASA Technical Memorandum

#110381. Moffett Field, CA: NASA Ames

Research Center.

Blake, Mathew W., (1996) The NASA ACFS:

A Unique Transport Research Environment.

AIAA-96-3518-CP.

Callantine, T. J., Mitchell, C. M., & Palmer,

E. A., (1999). GT-CATS: Tracking operator

activities in complex systems. NASA Technical

Memorandum #208788. Moffett Field, CA:

NASA Ames Research Center.

Cashion, P., Feary, M., Goka, T., Graham, H.,

Palmer, E., & Smith, N. (1995). Development

and initial field evaluation of flight deck

procedures for flying CTAS descent

clearances. In Proceedings of the Eighth

International Symposium on Aviation

Psychology. Columbus, Ohio.

Couluris, G. J., Weidner, T., Sorensen, J. A.

(1997). Final approach enhancement and

descent trajectory negotiation potential

benefits analysis. Los Gatos, CA: Seagull

Technology, Inc.

Crane, B., Palmer, E., & Smith, N. (1997).

Evaluation of a new descent procedure for

airline pilots. In Proceedings of the Eighth

International Symposium on Aviation

Psychology. Columbus, Ohio.

Curry, R. E., (1985). The introduction of new

cockpit technology: a human factors study.

NASA Technical Memorandum #86659.

Moffett Field, CA: NASA Ames Research

Center.

Davis, T., Isaacson, D., Robinson, J., den

Braven, W., Lee, K., Sanford, B. (1997).

Operational test results of the passive final

approach spacing tool. In Proceedings of the

IFAC Eighth Symposium on Transportation

Systems 97. Chania, Greece.

Erzberger, H., Davis, T.J. and Green, S. M.

(1993). Design of the center-TRACON

automation system, In Proceedings of the

AGARD Guidance and Control Panel 56 'h

Symposium on Machine Intelligence in Air

Traffic Management. Berlin, Germany, pp. 11-

12.

FAA Office of System Capacity, 1998,

http://www.asc.faa.gov/

Fadden, D. M., (1990). Aircraft automation

challenges. In Abstracts of AIAA-NASA-FAAHFS Symposium, Challenges in Aviation

Human Factors: The National Plan.

Washington, D. C.: American Institute of

Aeronautics and Astronautics.

Gopher, D., and Donchin, E., (1986).

Workload: an examination of the concept. In

K. R. Boff, L. Kaufman, J. P. Thomas (Eds.),

Handbook of Perception and Human

Performance. John Wiley and Sons, New York.

Green, S.M., and R. Vivona, (1996). Field

Evaluation of Descent Advisor Trajectory

Prediction Accuracy, AIAA 96-3764, AIAA

Guidance, Navigation, and Control

Conference, July, 1996

Hart, S. G., and Staveland, L. E., (1988).

Development of NASA-TLX (Task Load

Index): Results of empirical and theoretical

research. In P. A. Hancock, N. Meshkati

(Eds.), Human Mental Workload. Elsevier

Science Publishers B. V. Amsterdam, The

Netherlands.

Kahneman, D. (1973). Attention and effort.

Englewood Cliffs, N J: Prentice Hall.

Montalvo, (1999). In preparation

33
Palmer, E., Crane, B., Johnson, N., Smith, N.,

Feary, M., Cashion, P., Goka, T., Green, S., and

Sanford, B. (1997). Field evaluation of flight

deck procedures for flying CTAS descents. In

Proceedings of the Eighth International

Symposium on Aviation Psychology.

Columbus, Ohio.

Prevot, T., Crane, B. W. (1998). A vertical

situation display for automated aircraft-design

and evaluation in full mission simulation. In

Proceedings of the 8 'h International

Conference on Human-Computer Interaction.

Munich, Germany

Sheridan, T. B., (1992). Telerobotics,

automation, and human supervisory control.

London: The MIT Press.

Tsang, P. S., and Vidulich, M. A., (1989).

Cognitive demands of automation in aviation.

In R. S. Jensen (Ed.), Aviation Psychology.

Brookfield: Gower Technical.

Transport Canada (1997). Report of the RNA V

Task Force FMS SID/STAR Working Group.

Canada.

Wickens, C. D. (1992). Engineering

psychology and human performance. New

York: Harper Collins.

Wiener, E., and Curry, R., (1980). Flight-deck

automation: promises and problems. NASA

Technical Memorandum #81206. Moffett

Field, CA: NASA Ames Research Center.

Wiener, E. L., (1985). Human factors of

cockpit automation: afield study of flight crew

transition. NASA Contractor Report #177333.

Moffett Field, CA: NASA Ames Research

Center.

Wiener, E. L., (1989). Human factors of

advanced technology ("glass cockpit")

transport aircraft. NASA Contractor Report

#177528. Moffett Field, CA: NASA Ames

Research Center.

Yeh, Y., and Wickens, C., (1988). Dissociation

of performance and subjective measures of

workload. Human Factors, 30(1), 111-120.

34
Appendix A

STAR, FMS, and CTASfFMS Charts

35

f

Ill.t-

-¢'0

tn f.J

<

_l

...3

<

¢_

_0

q'J,

to

0

Q.

t-l.u

_.=_

7

37
38
39

Appendix B

Participant Consent Form

41

NASA Ames Research Center

Human Research Minimal Risk Consent

To the Test Subject: Please read this consent form and the attached protocol and/or subject

instructions carefully. Make sure all questions have been answered to your satisfaction before

signing.

I agree to participate as a subject in CTAS/FMS Integration Crew Factors research experiment as

described in the attached protocol or subject instructions. I understand that I am employed by

Raytheon Corporation who can be contacted at (650) 604-2118.

I understand that my participation could cause me minimal risk*, inconvenience, or discomfort.

The purpose and procedures have been explained to me and I understand the risks and discomforts

as described in the attached research protocol.

To my knowledge, I have no medical conditions, including pregnancy, that will prevent my

participation in this study. I understand that if my medical status should change while a participant in

the research experiment that there may be unforeseeable risks to me (or the embryo or fetus if

applicable). I agree to notify the Principal Investigator (P.I.) or medical monitor of any known

changes in my condition for safety purposes

My consent to participate as a subject has been freely given. I may withdraw my consent, and

thereby withdraw from the study at any time without pearly or loss of benefits to which I am entitled.

I understand that the P.I. may request my withdrawal or the study may be terminated for any reason.

I agree to follow procedures for orderly and safe termination.

I am not releasing NASA from liability for any injury arising as a result from my participation in

this study.

I hereby agree that all records collected by NASA in the course of this study are available to the

research study investigators, support staff, and any duly authorized research review committee. I grant

NASA permission to reproduce and publish all recorded, notes, or data collected from my

participation, provided there is no association of my name with the collected data and that

confidentiality is maintained, unless specifically waived by me.

I have had the opportunity to ask questions and have received satisfactory answers to all my

questions. I understand the P.I. for the study is the person responsible for this activity and that any

pertinent questions regarding the research will be addressed to him/her during the course of the

study. I have read the above agreement, the attached protocol and/or subject instructions prior to

signature, and understand the contents.

*Minimal risk means that the probability and magnitude of harm or discomfort anticipated in the

research are not greater, in and of themselves, than those ordinarily encountered in daily life or

during performance of routine physical or psychological examinations or tests.

43
Signature of Test Subject Date Signature of Principal Investigator

Printed name of test subject Printed name of Principal Investigator

Address Telephone Number of Principal Investigator

City, State, Zip Code Subject Signature: Authorization for videotape

Telephone Number of Test Subject

For Questions regarding this study contact:

Serena Stanford at SJSU (408) 924 2480

Everett Palmer at NASA (650) 604 0673

Barry Crane at SJSU (650) 604 2011

Subject Signature: Authorization for release of

information to Non-NASA Source

44
Appendix C

Descent Scenarios

45

AIRCRAFT

POSITION

CENTER: 127.15

Appendix C

Descent Scenarios

Baseline Current Day Procedure Scenario

CLEARANCE PHRASEOLOGY

•IP: 10 miles ("NASA 21, cleared to DFW via the GLENN ROSE 3 Arrival,

southeast of Abilene Abilene transition.")

on JEN3 Arrival

• Following IP "NASA 21, descend via the GLENN ROSE 3 Arrival, cross

FEVER at and maintain 11,000 feet."

• A/C at Glenn Rose "NASA 21, contact Regional Approach 133.62."

FEEDER: 133.62

• A/C at Glenn Rose

• A/C before

DELMO

"NASA 21, After FEVER, Maintain 11,000 feet, comply with

speeds on STAR, expect vectors to ILS 18R."

"NASA 21, Contact Regional Approach 118.42."

FINAL: 118.42

o A/C at "NASA 21, fly heading 350 vectors to ILS18R, descend and

DELMO maintain 5000'."

• A/C between two "NASA 21, Reduce speed to 190, descend and maintain 3000."

miles before ROSEL

• A/C-six miles "NASA 21, turn right heading 090."

beyond ROSEL

• A/C on base

• A/C on final

"NASA 21, turn right heading 140, intercept ILS18R localizer.

"NASA 21, cleared for ILS18R approach, cross Final Approach

Fix at 170 knots."

• A/C on final "NASA 21, contact DFW tower 124.15"

TOWER: 124.15

• A/C on final "NASA 21, cleared to land runway 18R

Baseline FMS Procedure Scenario

AIRCRAFT PHRASEOLOGY

POSITION

CENTER: 127.15

• IP: 10 miles ("NASA 21, cleared to DFW via GLENN ROSE FMS Arrival,

southeast of Abilene Abilene transition.")

on JEN FMS Arrival

• Following IP "NASA 21, descend via GLENN ROSE FMS

Arrival"

• A/C at Glenn Rose NASA 21_ Contact Regional Approach 133.62

47
I

FEEDER: 133.62

• A/C at Glenn Rose "NASA 21, Roger, cleared for DELMO FMS Transition to

ILS18R"

• A/C before "NASA 21, Contact Regional Approach 118.42"

DELMO

FINAL: 118.42

• AJC before "NASA 21 Roger"

DELMO

• A/C on base "NASA 21, cleared for ILS18R approach."

• A/C on final "NASA 21, Contact DFW tower 124.15"

TOWER: 124.15

• MC on final [ "NASA 21, cleared to land runway 18R"

AIRCRAFT

POSITION

Complex Current Day Procedure Scenario

CLEARANCE PHRASEOLOGY

CENTER: 127.15

•IP: 10 miles ("NASA 21, cleared to DFW via the GLENN ROSE 3 Arrival,

southeast of Abilene Abilene transition.")

on JEN3 Arrival

• Following IP "NASA 21, fly heading 080, descend pilots discretion, maintain

FL240, expect to cross FEVER at 11,000 feet."

• A/C level at FL240

for 25 secs.

"NASA 21 traffic 2 o' clock, five miles, northbound, 2000

below you."

"NASA 21, traffic no factor, cleared direct FEVER, cross

FEVER at and maintain 11,000 feet."

• A/C abeam Glenn "NASA 21, contact Regional Approach 133.62."

Rose

FEEDER: 133.62

• AJC abeam Glenn

Rose

• A/C before

DELMO

"NASA 2t, After FEVER, Maintain 11,000 feet, comply with

speeds on the STAR, expect vectors to runway 18R."

"NASA 21, Contact Regional Approach 118.42"

FINAL: 118.42

• A/C at "NASA 21, fly heading 350 vectors to ILS18R, descend and

DELMO maintain 5000'."

• A/C between six "NASA 21, Reduce speed to 190, descend and maintain 3000."

miles before ROSEL

• A/C-six miles "NASA 21, turn right heading 090."

beyond ROSEL

• A/C on base

• A/C on final

"NASA 21, turn right heading 140, intercept ILS18R localizer.

"NASA 21, cleared for ILS18R approach, cross Final Approach

48
Fix at 170 knots."

• A/C on final "NASA 21, Contact DFW tower 124.15"

TOWER: 124.15

• MC on final "NASA 21, Cleared to land runway 18R"

Complex FMS Procedure Scenario

AIRCRAFT PHRASEOLOGY

POSITION

CENTER: 127.15

• IP: 10 miles ("NASA 21, cleared to DFW via GLENN ROSE FMS Arrival,

southeast of Abilene Abilene transition.")

on JEN FMS Arrival

• Following IP "NASA 21, fly heading 080, descend at pilots discretion,

maintain FL240, expect direct FEVER to rejoin GLENN ROSE

FMS Arrival"

• A/C at FL240 for

25 secs.

"NASA 21 traffic 4 o' clock, five miles, northwestbound, 2000

below you."

"NASA 21, clear of traffic, cleared direct FEVER, resume

GLENN ROSE FMS Arrival."

• A/C abeam Glenn NASA 21, Contact Regional Approach 133.62

Rose

FEEDER: 133.62

• A/C abeam Glenn "NASA 21, Roger, after crossing FEVER, cleared for DELMO

Rose FMS transition to ILS 18R"

• A/C before "NASA 21, Contact Regional Approach 118.42"

DELMO

FINAL: 118.42

• A/C before "NASA 21 Roger"

DELMO

• A/C six miles prior "NASA 21, Slow to 190 knots."

to ROSEL

• A/C just past "NASA 21, Maintain current heading, standby for base turn

ROSEL clearance."

• A/C seven miles "NASA 21, turn right heading 090."

beyond ROSEL

• A/C on base "NASA 21, turn right heading 140, intercept ILS18R localizer

• A/C on final cleared for ILS 18R approach, cross Final Approach Fix at 170

knots"

• A/C on final "NASA 21, Contact DFW tower 124.15"

TOWER: 124.15

• A/C on final [ "NASA 21, cleared to land runway 18R"

Manual Path Extension Procedure Scenario

49
AIRCRAFT

POSITION

CENTER: 127.15

PHRASEOLOGY

• IP: 10 miles

southeast of Abilene

on JEN FMS Arrival

• Following IP

• A/C at FL240 for

25 secs.

("NASA 21, cleared to DFW via the GLENN ROSE FMS

Arrival, Abilene transition.")

"NASA 21, fly heading 080, descend pilots discretion at 300

knots, maintain FL240, expect direct FEVER to rejoin GLENN

ROSE FMS Arrival"

"NASA 21 traffic 5 o' clock, five miles, northeastbound, 2000

below you."

"NASA 21, clear of traffic, resume descent at 300 knots, cleared

direct FEVER, resume GLENN ROSE FMS Arrival."

• A/C abeam Glenn NASA 21, Contact Regional Approach 133.62

Rose

FEEDER: 133.62

• A/C abeam Glenn "NASA 21, Roger, cleared for DELMO FMS transition to

Rose ILS18R"

• A/C before "NASA 21, Contact Regional Approach 118.42"

DELMO

FINAL: 118.42

• A/C before "NASA 21, Roger"

DELMO

•A/C just past "NASA 21, make base turn at ROSEL plus x.x nautical miles."

DELMO

• A/C six miles prior "NASA 21, Slow to 190 knots."

to ROSEL

• A/C on base "NASA 21, cleared for ILS18R approach."

• A/C on final "NASA 21, Contact DFW tower 124.15"

TOWER: 124.15

• A/C on final

i

[ "NASA 21, cleared to land runway 18R"

5O
CDU DL and DDL Procedure Scenarios

AIRCRAFT PHRASEOLOGY

POSITION

CENTER: 127.15

• IP: 10 miles ("NASA 21, cleared to DFW via the GLENN ROSE FMS

southeast of Abilene Arrival, Abilene transition.")

on JEN FMS Arrival

• Following IP

• Following IP

"NASA 21, Expect route uplink."

Data link message: NASA 21

...... ROUTE MODIFICATION ......

REVISED FMS RTE TO FEVER

• Following data link "NASA 21, descend via revised FMS route to FEVER at 300

clearance knots."

•A/C at ~FL260 "NASA 21, maintain FL240 for traffic"

• A/C at FL240 for "NASA 21 traffic 3 o' clock, five miles, northbound, 2000

25 secs. below you."

"NASA 21, clear of traffic, resume FMS descent at 300 knots."

• A/C abeam Glenn NASA 21, Contact Regional Approach 133.62

Rose

FEEDER: 133.62

• A/C abeam Glenn "NASA 21, Roger, cleared for DELMO FMS Transition to

Rose ILS 1 8R"

• MC before "NASA 21, Contact Regional Approach 118.42"

DELMO

FINAL: 118.42

• A/C before "NASA 21, Roger, expect route modification uplink."

DELMO

•A/C just past Data link message: NASA 21

DELMO --ROUTE MODIFICATION--

REVISED FMS RTE TO LEGRE

• A/C six miles prior "NASA 21, Slow to 190 knots."

to ROSEL

• A/C on base "NASA 21, cleared for ILS18R approach."

• A/C on final "NASA 21, Contact DFW tower 124.15"

TOWER: 124.15

• MC on final "NASA 21, cleared to land runway 18R"

51

Appendix D

Workload Dimension Descriptions and Workload Rating Sheet

53

Appendix D

Workload Dimension Descriptions and Workload Rating Sheet

Workload Dimension

MENTAL DEMAND

PHYSICAL DEMAND

TEMPORAL DEMAND

PERFORMANCE

Endpoints

Low/High

Low/High

Low/High

Good/Poor

Description

How much mental

and perceptual

activity was required

(e.g. thinking,

deciding, calculating,

remembering,

monitoring, looking,

searching, etc)? Was

the task easy or

demanding, simple or

complex, exacting or

forgiving?

How much physical

activity was required

(e.g. pushing, pulling,

turning, controlling,

activating, etc. )? Was

the task easy

demanding, slow or

brisk, slack or

strenuous, restful or

laborious?

How much time

pressure did you feel

due to the rate or

pace at which the

tasks or task elements

occurred? Was the

pace slow and

leisurely or rapid and

frantic ?

How successful do

you think you were

in accomplishing the

goals of the task set

by the experimenter

(or yourself)? How

satisfied were you

with your

performance in

accomplishing these

55
EFFORT

FRUSTRATION LEVEL

Low/High

Low/High

/_oals

How hard did you

have to work

(mentally and

physically) to

accomplish your level

of performance?

How insecure,

discouraged, irritated,

stressed, and annoyed

versus secure,

gratified, content,

relaxed and

complacent did you

feel during the task?

56
Appendix E

Procedure Performance Criteria

57

Appendix E

Procedure Performance Criteria

General criteria that apply to all procedures:

Altitudes

Comply with all charted and ATC specified altitude restrictions within plus or minus 300 feet

as measured when passing over location (e.g. waypoint, navaid, etc.) of specified restriction. If

compliance is not possible due to limitations in aircraft performance or other reason, inform ATC.

Speeds

Comply with all charted and ATC specified speed restriction within plus or minus 10 KIAS as

measured when passing over location (e.g. waypoint, navaid, etc.) of specified restriction or when in

phase of flight where a speed restriction has been specified (e.g. descent speed). If compliance is not

possible due to limitations in aircraft performance or other reason, inform ATC.

Criteria specific to FMS and CTAS/FMS procedures:

Use of

LNAV

LNAV

Unless given a heading vector by ATC, fly the entire FMS or CTAS/FMS procedure in the

guidance mode until the point of Localizer capture.

Use of VNAV

When flying the DELMO FMS Transition, remain in the VNAV guidance mode until the

point of Glideslope capture.

Use of VNAV top of descent

Unless given a heading vector by ATC, when flying the Glen Rose FMS Arrival, begin descent

within plus or minus 5 miles of the VNAV calculated top of descent point. If compliance is not

possible, inform ATC.

Responding to, loading, and executing DELMO FMS Transition

When cleared for the DELMO FMS Transition to ILS 18R, correctly perform this activity.

Correct performance of this activity is as follows: (1) Readback clearance to ATC, (2) access the

DEP/ARR page, (3) select ILS 18R on the runway menu, (4) select DELMO on the transition menu,

(5) press Execute on the CDU, (6) select 2400 in the MCP altitude window.

Responding to, loading and executing a Manual Path Extension clearance

Upon receipt of a Manual Path Extension clearance, correctly perform this activity. Correct

performance of this activity is as follows: the PNF must (1) acknowledge and readback the ATC

cleared base turn distance, (2) line select the reference waypoint ROSEL into the CDU scratchpad, (3)

enter a backslash (/) and the new base turn point value, (4) reselect the waypoint into the flightplan,

(5) review the modification on the CDU and Navigation display, and (6) execute the modification by

pressing the execute button on the CDU.

59
Responding to, loading and executing a CDU DL clearance

Upon receipt of a CDU DL clearance, correctly perform this activity. Correct performance of

this activity is as follows: the PNF must (1) press the ATC key on the CDU, (2) read the message text

on the CDU page aloud to the other crew member, (3) receive confirmation from the other crew

member, (4) load the new route by pressing a LOAD prompt, (5) press the LEGS key on the CDU,

(6) review and verify the loaded message on the LEGS page and Navigation display, (7) execute the

new route by pressing the execute button on the CDU, (8) press the ATC key on the CDU, and (9)

accept the message by pressing the ACCEPT prompt.

Responding to, loading and executing a DDL clearance

Upon receipt of a DDL clearance, correctly perform this activity. Correct performance of this

activity is as follows: the PNF must (1) read the message text on the EICAS display aloud to the other

crew member, (2) receive confirmation from the other crew member, (3) review and verify the loaded

message on the LEGS page and Navigation display, (4) execute the new route by pressing the execute

button on the CDU, and (5) accept the message by pressing the accept button on the glareshield.

6O
Appendix F

Crew Questionnaire

61

CTASfFMS Integration Evaluation: ACFS Crew Factors Study -- Fall 1998

Pilot Questionnaire

BACKGROUND INFORMATION

Date: Crew Position:

Approximate total hours:

Approximate total hours flying "glass" aircraft:

Please specify approximate total hours in each type aircraft 727

737- 200 DC -10 L10-11 747-200/300

MD-80 737-300/500 747-400 757

767 777 A300 A310 A320

A340

DC-9

Other type aircraft (please specify)

Estimated number of times you have flown into DFW in the past 3 years

Estimated number of times you have flown the Glen Rose 3 Arrival into DFW in the past 3

years

Instructions: Please circle the tick mark on each scale that best fits your response to

the following questions or statements. Please also provide answers to the yes/no

questions and comments or explanations where indicated. Please consider your

responses carefully. Your responses will play an important role in the evaluation

being conducted

FMS ARRIVAL AND TRANSITION CHARTS

How clear or unclear was the information presented on the Glenn Rose FMS Arrival and

Delmo FMS Transition chart___s used in this study?

I I I I I

Very Somewhat Borderline Somewhat Very

Unclear Unclear Clear Clear

How organized or unorganized was the information presented on the Glenn Rose FMS

Arrival and Delmo FMS Transition charts used in this study?

I t I I I

Very Somewhat Borderline Somewhat Very

63
Unorganized Unorganized Organized Organized

How adequate or inadequate were the Glenn Rose FMS Arrival and Delmo FMS Transition

charts in helping you perform the procedures?

I I I I I

Very Somewhat Borderline Somewhat Very

Inadequate Inadequate Adequate Adequate

Was there any information missing from the FMS Arrival and Transition charts that you

would have liked to see?

Yes No

If yes, please explain

Do you think the FMS Arrival and Transition charts should have contained les__.__s

information, or less detailed information?

Yes No

If yes, please explain

How acceptable or unacceptable were the Glenn Rose FMS Arrival and Delmo FMS

Transition charts as a whole?

I I i I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

64
FMS ARRIVAL AND TRANSITION CLEARANCE PHRASEOLOGY

Initial FMS Arrival Descent Clearance

"NASA 21, Descend via the Glenn Rose FMS Arrival, maintain XXX Knots in the

descent"

Was any portion of this clearance phraseology unclear or confusing the first time you

heard it?

Yes No

Please explain

When you first received this clearance, was it clear to you what to do?

Yes No

Please explain

Delmo FMS Transition Clearance

"NASA 21, Cleared for the Delmo FMS Transition to ILS18R"

Was any portion of this clearance phraseology unclear or confusing the first time you

heard it?

Yes No

Please explain

When you first received this clearance, was it clear to you what to do?

Yes No

Please explain

How acceptable or unacceptable was the Glenn Rose FMS Arrival and Delmo FMS

Transition phraseology as a whole?

I I I I

Very Somewhat

UnacceptableUnacceptable

Borderline Somewhat Very

Acceptable Acceptable

65
PROCEDURE ACCEPTABILITY

FMS Procedure: Questions in this section refer specifically to the Glenn Rose FMS

Arrival and Delmo FMS Transition procedure without regard to the

adjunct route modification procedures (i.e. Manual Path Extension,

CDU DL, and DDR).

Was any portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure

unclear or confusing?

Yes No

Please explain

How acceptable or unacceptable was the requirement to set your MCP altitude window-to

the las__tcrossing restriction on the routing of the Glenn Rose FMS Arrival and Delmo FMS

Transition procedure?

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How acceptable or unacceptable was the requirement to use LNAV during the Arrival

portion (at and above 11,000) of the Glenn Rose FMS Arrival and Delmo FMS Transition

procedure?

'Ve+ SomJw.a, ' X I Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How comfortable or uncomfortable were you using LNAV during the Arrival portion of the

Glenn Rose FMS Arrival and Delmo FMS Transition procedure?

I I I I I

Very Somewhat

UncomfortableUncomfortable

Borderline Somewhat Very

ComfortableComfortable

How acceptable or unacceptable was the requirement to use VNAV during the Arrival

portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure?

I I I I

66
Very Somewhat

UnacceptableUnacceptable

Borderline Somewhat Very

Acceptable Acceptable

How comfortable or uncomfortable were you using VNAV during the Arrival portion of the

Glenn Rose FMS Arrival and Delmo FMS Transition procedure?

I I I I I

Very Somewhat Borderline Somewhat Very

UncomfortableUncomfortable ComfortableComfortable

How acceptable or unacceptable was the requirement to use LNAV during the Delmo

Transition portion (below 11,000) of the Glenn Rose FMS Arrival and Delmo FMS

Transition procedure?

I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How comfortable or uncomfortable were you using LNAV during the Delmo Transition

portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure?

I I I I I

Very Somewhat Borderline Somewhat Very

UncomfortableUncomfortable ComfortableComfortable

How acceptable or unacceptable was the requirement to use VNAV during the Delmo

Transition portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure?

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How comfortable or uncomfortable were you using VNAV during the Delmo Transition

portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure?

I I I I I

Very Somewhat Borderline Somewhat Very

UncomfortableUncomfortable ComfortableComfortable

"Nasa 21, fly heading 080, descend at pilot's discretion, maintain FL240, expect direct

67
FEVER to rejoin Glenn Rose FMS Arrival"....

"Nasa 21, clear of traffic, cleared direct FEVER, resume Glenn Rose FMS Arrival."

How acceptable or unacceptable were the above set of clearances that vectored you off the

Glenn Rose FMS Arrival and then had you resume it?

Veryl Somelwhat Borde!hne" SomeJhat Very I

UnacceptableUnacceptable Acceptable Acceptable

"Nasa 21, Maintain current heading, standby for base turn clearance."

How acceptable or unacceptable was the above clearance given during the Delmo FMS

Transition procedure?

I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

Did you feel rushed at any time during the Glenn Rose FMS Arrival and Delmo FMS

Transition procedure?

Yes No

Please explain

In general, how was your workload affected when flying the Glenn Rose FMS Arrival and

Delmo FMS Transition procedure as compared to your current day descent procedures:

I I I I I

Greatly Somewhat Unaffected Somewhat Greatly

Decreased Decreased Increased Increased

In general, how was your monitoring behavior affected when flying the Glenn Rose FMS

Arrival and Delmo FMS Transition procedure as compared to your current day descent

procedures:

I I I I I

Greatly Somewhat Unaffected Somewhat Greatly

Decreased Decreased Increased Increased

The Glenn Rose FMS Arrival and Delmo FMS Transition procedure made flying the

descent:

I I I I

Very Somewhat Borderline Somewhat Very

Easy Easy Difficult Difficult

68
Compared to current day descent procedures, the Glenn Rose FMS Arrival and Delmo

FMS Transition procedure was:

I I I I

Very Somewhat Borderline Somewhat Very

Undesirable Undesirable Desirable Desirable

How acceptable or unacceptable was the Glenn Rose FMS Arrival and Delmo FMS

Transition procedure as a whole?

I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

Please provide any comments you have regarding the Glenn Rose FMS Arrival and Delmo

FMS Transition Procedure:

CTAS/FMS PROCEDURES

General

How acceptable or unacceptable was the requirement to use data link for route modifications

while in cruise near the top-of-descent (i.e. "Revised FMS route to FEVER")?

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

69
.... while in the TRACON?

[Very Some[what

UnacceptableUnacceptable

Borderline [ Somew[hat Very

Acceptable Acceptable

How acceptable or unacceptable was the issuance of a descent speed in the Glenn Rose

FMS Arrival?

I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

In general, how acceptable or unacceptable was the requirement to make FMS edits to the

Delmo FMS transition routing while on downwind in the TRACON airspace?

[Very Some[what Borde!line Somew]hat Very [

UnacceptableUnacceptable Acceptable Acceptable

General Data link

How acceptable or unacceptable was the data link phraseology used in the Center airspace

(i.e. "Revised FMS route to FEVER" )?

[Very Some]what Borde!line SomeJhat Very [

UnacceptableUnacceptable Acceptable Acceptable

How acceptable or unacceptable was the data link phraseology used in the TRACON

airspace (i.e. "Revised FMS route to LEGRE'" )?

I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

Manual Path Extension Procedure

How acceptable or unacceptable was the Manual Path Extension procedure requirement to

manually enter the modified base turn point into the CDU during the Delmo FMS

Transition procedure?

I I I I I

Very Somewhat

UnacceptableUnacceptable

Borderline Somewhat Very

Acceptable Acceptable

How acceptable or unacceptable were the tasks required to complete the Manual Path

70
Extension procedure?

I I I I

Very Somewhat Borderline Somewhat

UnacceptableUnacceptable Acceptable

Very

Acceptable

Did you feel rushed at any time during the Manual Path Extension procedure?

Yes No

Please explain

How acceptable or unacceptable was the Manual Path Extension procedure phraseology

("Make base turn at Rosel +X.X miles ")?

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How acceptable or unacceptable was the Manual Path Extension procedure as a whole?

I I I I I

Very Somewhat

UnacceptableUnacceptable

Borderline Somewhat Very

Acceptable Acceptable

Please provide any comments you have regarding the Manual Path Extension procedure:

CDU DL Procedures

How acceptable or unacceptable were the tasks required to complete the CDU DL

procedure?

I I I I

Very Somewhat Borderline Somewhat Very

71
UnacceptableUnacceptable Acceptable Acceptable

How acceptable or unacceptable was the CDU DL interface for route modifications received

in the Center airspace ("'Revised route to EFVER")?

Iv++ somewhat I I I

Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

Did you have any trouble understanding the CDU DL data link message you received in the

_enter airspace once it was loaded

(i.e. "Revised FMS route to FEVER" )?

Yes No

Please explain

How acceptable or unacceptable was the CDU DL interface for route modifications received

in the TRACON airspace ("Revised route to LEGRE")?

I I i i I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

Did you have any trouble Understanding the CDU DL data link message you received in

the TRACON airspace once it was loaded

(i.e. "Revised FMS route to LEGRE" )?

Yes No

If yes, please explain

Did you feel rushed at any time when completing the CDU DL procedure in the Center

airspace?

Yes No

72
If yes, please explain

Did you feel rushed at any time when completing the CDU DL procedure in the TRACON

airspace?

Yes No

If yes, please explain

How acceptable or unacceptable was the CDU DL Interface as a whole?

I I I I

Very Somewhat Borderline Somewhat

UnacceptableUnacceptable Acceptable

Very

Acceptable

How acceptable or unacceptable was the CDU DL Procedure as a whole?

I I I I

Very Somewhat Borderline Somewhat

UnacceptahleUnacceptable Acceptable

Very

Acceptable

Please provide any comments you have regarding the CDU DL procedure:

DDR Procedures

How acceptable or unacceptable were the tasks required to complete the DDR procedure?

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How acceptable or unacceptable was the DDR interface for route modifications received in

the Center airspace ("Revised route to FEVER")?

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

73
Did you have any trouble understanding the DDR data link message you received in the

Center airspace (i.e. "Revised FMS route to FEVER" )?

Yes No

Please explain

How acceptable or unacceptable was the DDR interface for route modifications received in

the TRACON airspace ("Revised route to LEGRE")?

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

Did you have any trouble deciphering the DDR data link message you received in the

TRACON airspace (i.e. "'Revised FMS route to LEGRE" )?

Yes No

Please explain

Did you feel rushed at any time when completing the DDR procedure in the Center

airspace?

Yes No

Please explain

Did you feel rushed at any time when completing the DDR procedure in the TRACON

airspace?

Yes No

Please explain

How acceptable or unacceptable was the DDR Interface as a whole?

I I I I

Very Somewhat Borderline Somewhat

UnacceptableUnacceptable Acceptable

Very

Acceptable

74
How acceptable or unacceptable was the DDR Procedure as a whole?

[Very Some[what BorderlineI Somewhat

UnacceptableUnacceptable Acceptable

Very

Acceptable

Please provide any comments you have regarding the DDR procedure:

Please state the FMS route modification interface (e.g. MPE, DDR, CDU DL) that you

most preferred.

Why?

Which interface(s) do you foresee as most likely to be used in day to day operations?

AUTOMATION

Indicate the degree to which you feel you were able to anticipate automation behavior while

flying the descents

I I I I I

Never Not Often Sometimes Usually Always

Were you ever confused by the automation behavior during any of the descents flown?

Yes No

If yes, please explain

Were you ever surprised by the automation behavior during any of the descents flown?

Yes No

If yes, please explain

75
Did you feel "ahead" of the aircraft when flying the FMS and CTAS/FMS procedures?

I I I I I

Never Not Often Sometimes Usually Always

Did the increased use of high level automation with the FMS and CTAS/FMS procedures

impact your crew coordination activities in any way?

Yes No

If yes, please explain the impact they had?

Did the increased use of high level automation with the FMS and CTAS/FMS procedures

allow you more or less time to look out the window?

Borderline Somewhat Much

Less Time Less Time More Time More Time

With the FMS and CTAS/FMS procedures, how adequately or inadequately do you feel

you were able to picture the vertical situation of the descent given the current displays?

I I I I I

Very Somewhat Borderline Somewhat Very

Inadequatelylnadequately Adequately Adequately

ADEQUACY OF BRIEFING AND TRAINING

Did the Glenn Rose FMS Arrival and Delmo FMS Transition briefing you received prepare

you sufficiently for this procedure?

Yes No

(please elaborate)

Did the Manual Path Extension briefing and training you received prepare you sufficiently

for this procedure?

Yes No

76
(please elaborate)

Did the CDU DL briefing and training you received prepare you sufficiently for this

procedure?

Yes No

(please elaborate)

Did the DDR briefing and training you received prepare you sufficiently for this procedure?

Yes No

(please elaborate)

Do you think simulator training is needed for introduction of the Glenn Rose FMS Arrival

and Delmo FMS Transition Procedure?

Yes No

(if so, why?)

OTHER

How many descents did it take for the Glenn Rose FMS Arrival and Delmo FMS

Transition to become comfortable, or routine?

Describe any techniques you may have developed for flying the Glenn Rose FMS Arrival

and Delmo FMS Transition.

77

Appendix G

Crew Questionnaire Responses

79
80
Pilot Questionnaire Responses to Likert scale and yes or no questions

FMS ARRIVAL AND TRANSITION CHARTS

How clear or unclear was the information presented on the Glenn Rose FMS Arrival and

Delmo FMS Transition charts used in this study?

0 0 0 7 16

I I I I I

Very Somewhat Borderline Somewhat Very

Unclear Unclear Clear Clear

How organized or unorganized was the information presented on the Glenn Rose FMS

Arrival and Delmo FMS Transition charts used in this study?

0 0 0 9 14

! I I- I I

V ery Somewhat Borderline Somewhat Very

Unorganized Unorganized Organized Organized

How adequate or inadequate were the Glenn Rose FMS Arrival and Delmo FMS

Transition charts in helping you perform the procedures?

0 0 1 6 17

I I I I I

Very Somewhat Borderline Somewhat Very

Inadequate Inadequate Adequate Adequate

Was there any information missing from the FMS Arrival and Transition charts that you

would have liked to see?

Yes_2 No 21

Do you think the FMS Arrival and Transition charts should have contained less

information, or less detailed information?

Yes6 No 18

81
How acceptable or unacceptable were the Glenn Rose FMS Arrival and Delmo FMS

Transition charts as a whole?

0 0 1 8 15

I Some]what I I I

Very Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

FMS ARRIVAL AND TRANSITION CLEARANCE PHRASEOLOGY

Initial FMS Arrival Descent Clearance

"NASA 21, Descend via the Glenn Rose FMS Arrival, maintain XXX Knots in the

descent"

Was any portion of this clearance phraseology unclear or confusing the first time you

heard it?

Yes5 No 19

When you fu'st received this clearance, was it clear to you what to do?

Yes 2___0No 4

Delmo FMS Transition Clearance

"NASA 21, Cleared for the Delmo FMS Transition to ILS18R"

Was any portion of this clearance phraseology unclear or confusing the fu'st time you

heard it?

Yes_4 No 20

When you first received this clearance, was it clear to you what to do?

Yes 21 No 4

How acceptable or unacceptable was the Glenn Rose FMS Arrival and Delmo FMS

Transition phraseology as a whole?

0 2 0 7

I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

15

I

82
PROCEDURE ACCEPTABILITY

FMS Procedure: Questions in this section refer specifically to the Glenn Rose FMS

Arrival and Delmo FMS Transition procedure without regard to the

adjunct route modification procedures (i.e. Manual Path Extension,

CDU DL, and DDR).

Was any portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure

unclear or confusing?

Yes_4 No 20

How acceptable or unacceptable was the requirement to set your MCP altitude window to

the las.___t crossing restriction on the routing of the Glenn Rose FMS Arrival and Delmo FMS

Transition procedure?

1 1 4 10 8

Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How acceptable or unacceptable was the requirement to use LNAV during the Arrival

portion (at and above I 1,000) of the Glenn Rose FMS Arrival and Delmo FMS Transition

procedure?

0 0 0 5 19

I ] I J I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How comfortable or uncomfortable were you using LNAV during the Arrival portion of the

Glenn Rose FMS Arrival and Delmo FMS Transition procedure?

0 0 0 6 18

I I I I I

Very Somewhat Borderline Somewhat Very

UncomfortableUncomfortable ComfortableComfortable

How acceptable or unacceptable was the requirement to use V'NAV during the Arrival

portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure?

0 1 3 6 13

I I I I I

Very Somewhat Borderline Somewhat Very

83
UnacceptableUnacceptable Acceptable Acceptable

How comfortable or uncomfortable were you using VNAV during the Arrival portion of the

Glenn Rose FMS Arrival and Delmo FMS Transition procedure?

2 1 1 8 12

Very Som)whatBorderline I Som.lat I

UncomfortableUncomfortable ComfortableComfortable

How acceptable or unacceptable was the requirement to use LNAV during the Delmo

Transition portion (below 11,000) of the Glenn Rose FMS Arrival and Delmo FMS

Transition procedure?

0 0 1 6 17

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How comfortable or uncomfortable were you using LNAV during the Delmo Transition

portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure?

0 0 0 4 19

I I I I I

Very Somewhat Borderline Somewhat Very

UncomfortableUncomfortable ComfortableComfortable

How acceptable or unacceptable was the requirement to use VNAV during the Delmo

Transition portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure?

1 3 3 6 11

[Very Some]what Borde!line SomeJhat Very [

UnacceptableUnacceptable Acceptable Acceptable

How comfortable or uncomfortable were you using VNAV during the Delmo Transition

portion of the Glenn Rose FMS Arrival and Delmo FMS Transition procedure?

2 3 2 6 11

I I I I I

Very Somewhat Borderline Somewhat Very

84
UncomfortableUncomfortable ComfortableComfortable

"Nasa 21, fly heading 080, descend at pilot's discretion, maintain FL240, expect direct

FEVER to rejoin Glenn Rose FMS Arrival"....

"Nasa 21, clear of traffic, cleared direct FEVER, resume Glenn Rose FMS Arrival."

How acceptable or unacceptable were the above set of clearances that vectored you off the

Glenn Rose FMS Arrival and then had you resume it?

0 1 1 11 10

[Very Some[what Borde!!ine SomeJhat Very [

UnacceptableUnacceptable Acceptable Acceptable

"Nasa 21, Maintain current heading, standby for base turn clearance."

How acceptable or unacceptable was the above clearance given during the Delmo FMS

Transition procedure?

0 0 1 6 16

I I I I f

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

Did you feel rushed at any time during the Glenn Rose FMS Arrival and Delmo FMS

Transition procedure?

Yes2 No 21

In general, how was your workload affected when flying the Glenn Rose FMS Arrival and

Delmo FMS Transition procedure as compared to your current day descent procedures:

4 9 5 5 0

I I I ! I

Greatly Somewhat Unaffected Somewhat Greatly

Decreased Decreased Increased Increased

In general, how was your monitoring behavior affected when flying the Glenn Rose FMS

Arrival and Delmo FMS Transition procedure as compared to your current day descent

procedures:

0 3 8 9 3

I I I 1 I

Greatly Somewhat Unaffected Somewhat Greatly

Decreased Decreased Increased Increased

85
The Glenn Rose FMS Arrival and Delmo FMS Transition procedure made flying the

descent:

8 11 3 2 0

[Very SomeWhat Borde!line SomeJhat Very I

Easy Easy Difficult Difficult

Compared to current day descent procedures, the Glenn Rose FMS Arrival and Delmo

FMS Transition procedure was:

0 2 1 10 11

I I I I I

Very Somewhat Borderline Somewhat Very

Undesirable Undesirable Desirable Desirable

How acceptable or unacceptable was the Glenn Rose FMS Arrival and Delmo FMS

Transition procedure as a whole?

0 1 2 11 10

[Very Somewhat I Borderline[ Some..!hat,, Very I

UnacceptableUnacceptable Acceptable Acceptable

CTAS/FMS PROCEDURES

General

How acceptable or unacceptable was the requirement to use data link for route modifications

while in cruise near the top-of-descent (i.e. "Revised FMS route to FEVER")?

2 0 0 6 16

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

.... while in the TRACON?

0 3 2 7 12

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

86
How acceptable or unacceptable was the issuance of a descent speed in the Glenn Rose

FMS Arrival?

0 0 1 10 13

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

In general, how acceptable or unacceptable was the requirement to make FMS edits to the

Delmo FMS transition routing while on downwind in the TRACON airspace?

0 4 3 I0 7

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

General Data link

How acceptable or unacceptable was the data link phraseology used in the Center airspace

(i.e. "'Revised FMS route to FEVER" )?

0 0 1 4 19

I I J I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How acceptable or unacceptable was the data link phraseology used in the TRACON

airspace (i.e. "Revised FMS route to LEGRE'" )?

0 0 0 3 21

1 I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

87
Manual Path Extension Procedure

How acceptable or unacceptable was the Manual Path Extension procedure requirement to

manually enter the modified base turn point into the CDU during the Delmo FMS

Transition procedure?

0 1 3 12 8

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How acceptable or unacceptable were the tasks required to complete the Manual Path

Extension procedure?

0 1 2 10 11

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

Did you feel rushed at any time during the Manual Path Extension procedure?

Yesl No 22

How acceptable or unacceptable was the Manual Path Extension procedure phraseology

( "Make base turn at Rosel +X.X miles ")?

0 0 1 7

I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

16

I

How acceptable or unacceptable was the Manual Path Extension procedure as a whole?

0 1 2 8

[Very Some]what Borde!line Somewlhat Very

UnacceptableUnacceptable Acceptable Acceptable

13

I

CDU DL Procedures

How acceptable or unacceptable were the tasks required to complete the CDU DL

88
procedure?

0 0 0 8 15

I I I I I

Very Somewhat

UnacceptableUnacceptable

Borderline Somewhat Very

Acceptable Acceptable

How acceptable or unacceptable was the CDU DL interface for route modifications received

in the Center airspace ("Revised route to EFVER")?

0 0 0 8 15

I 1 I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

Did you have any trouble understanding the CDU DL data link message you received in the

Center airspace once it was loaded

(i.e. "Revised FMS route to FEVER" )?

Yes_.0 No24

How acceptable or unacceptable was the CDU DL interface for route modifications received

in the TRACON airspace ("Revised route to LEGRE")?

0 1 4 8 11

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

Did you have any trouble Understanding the CDU DL data link message you received in

the TRACON airspace once it was loaded

(i.e. "Revised FMS route to LEGRE" )?

Yes_0 No 24

Did you feel rushed at any time when completing the CDU DL procedure in the Center

airspace?

Yes_0 No 24

Did you feel rushed at any time when completing the CDU DL procedure in the TRACON

airspace?

Yes2 No 22

89
How acceptable or unacceptable was the CDU DL Interface as a whole?

0 0 0 12 12

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How acceptable or unacceptable was the CDU DL Procedure as a whole?

0 0 1 12 11

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

DDR Procedures

How acceptable or unacceptable were the tasks required to complete the DDR procedure?

0 0 1 6 17

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How acceptable or unacceptable was the DDR interface for route modifications received in

the Center airspace ("Revised route to FEVER")?

0 0 2 6 16

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

Did you have any trouble understanding the DDR data link message you received in the

Center airspace (i.e. "Revised FMS route to FEVER" )?

Yes._0 No 24

9O
How acceptable or unacceptable was the DDR interface for route modifications received in

the TRACON airspace ("Revised route to LEGRE")?

0 0 3 6 15

Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

Did you have any trouble deciphering the DDR data link message you received in the

TRACON airspace (i.e. "Revised FMS route to LEGRE" )?

Yes _0 No 24

Did you feel rushed at any time when completing the DDR procedure in the Center

airspace?

Yes_O No 24

Did you feel rushed at any time when completing the DDR procedure in the TRACON

airspace?

Yes_2 No 22

How acceptable or unacceptable was the DDR Interface as a whole?

0 0 3 7 14

I I I I I

Very Somewhat Borderline Somewhat Very

UnacceptableUnacceptable Acceptable Acceptable

How acceptable or unacceptable was the DDR Procedure as a whole?

0 0

I . I

Very Somewhat

UnacceptableUnacceptable

1 8 15

I I I

Borderline Somewhat Very

Acceptable Acceptable

91
AUTOMATION

Indicate the degree to which you feel you were able to anticipate automation behavior while

flying the descents

0 0 3 18 I

I I I I I

Never Not Often Sometimes Usua ly Always

Were you ever confused by the automation behavior during any of the descents flown?

Yes 15 No 9

Were you ever surprised by the automation behavior during any of the descents flown?

Yes 1__2 No 12

Did you feel "ahead" of the aircraft when flying the FMS and CTAS/FMS procedures?

0 0 3 12 9

I I I I I

Never Not Often Sometimes Usually Always

Did the increased use of high level automation with the FMS and CTAS/FMS procedures

impact your crew coordination activities in any way?

Yes_8 No16

Did the increased use of high level automation with the FMS and CTAS/FMS procedures

allow you more or less tim_____e to look out the window?

2 6 4 9 3

I I I I I

Much Somewhat Borderline Somewhat Much

Less Time Less Time More Time More Time

92
With the FMS and CTAS/FMS procedures, how adequately or inadequately do you feel

you were able to picture the vertical situation of the descent given the current displays?

0 2 0 6 16

[Very Somelwhat Borde!line SomeJhat Very [

Inadequatelylnadequately Adequately Adequately

ADEQUACY OF BRIEFING AND TRAINING

Did the Glenn Rose FMS Arrival and Delmo FMS Transition briefing you received prepare

you sufficiently for this procedure?

Yes 2___4No __0

Did the Manual Path Extension briefing and training you received prepare you sufficiently

for this procedure?

Yes 24 No 0

Did the CDU DL briefing and training you received prepare you sufficiently for this

procedure?

Yes 2___4No _0

Did the DDR briefing and training you received prepare you sufficiently for this procedure?

Yes 24 No 0

Do you think simulator training is needed for introduction of the Glenn Rose FMS Arrival

and Delmo FMS Transition Procedure?

Yes_9 No15

OTHER

Did you ever feel bored when flying any of the FMS or CTAS/FMS descents?

Yes 14 No 10

If you were to fly these procedures regularly, do you think boredom would become an

issue?

Yes_5 No14

93
Form Approved

Report Documentation Page OMB No 0704-0188

noedKI,anti c_rclltbg md rq_nlrmngN col_ of i_ Send¢ommlm= rlgardingthis burn e,_l_e or_ _r _ d _ _ _ _, _ s_ _ _eg _ b_. to

Wa._r_ton He_ers Sewmu, Did:to.to _or _omltion (_era_n= and R_. 1215 Jeflerso__s Hig_ww. Suite 1204,Adi_ll_, VA 2220_-4302. an¢t1ono O_ce o( Managementand Budget,Paperwo_

R_uct_t Pm_ (070¢0188), Wa¢_o_. OC2O5O3.

1. AGENCYUSEONLY(Leaveblank) 2. REPORTDATE

August 2000

¢.TITLEANDSUBTITLE

Flight Crew Factors for CTAS/FMS Integration in the

Terminal Area

6. AUTHOR(S)

Barry W. Crane, Thomas Prevot, and Everett A. Palmer

7.PERFORMINGORGANIZATION NAME(S) ANDADDRESS(kS)

NASA Ames Research Center

Moffett Field, California 94035-1000

9.SPONSORINC._ONITORINGAGENCYNAME(S)ANDADDRESS(kS)

National Aeronautics and Space Administration

3. REPORTTYPEAND DATESCOVERED

Technical Memorandum

5. FUNDINGNUMBERS

538-04-14

8. PERFORMINGORGANIATION

REPORTNUMBER

IH-012

10. SPONSORING/MONITORING

AGENCYREPORTNUMBER

NASAJTM--2000-209607

11.SUPPLEMENTARY NOTES

Point of Contact: Everett A. Palmer, Ames Research Center, M/S 262-4, Moffett Field, CA 94035

(650) 604-6073

12A.DISTRIBUTION/AVAILABILITY STATEMENT

Subject Category: 03-01, 54-04 Distribution: Public

Availability: NASA CASI (301) 621-0390

12B. DISTRIBUTIONCODE

13. ABSTRACT(Maximum200words)

Center TRACON Automation System (CTAS)/Flight Management System (FMS) integration on

the flightdeck implies flight crews flying coupled in highly automated FMS modes [i.e. Vertical

Navigation (VNAV) and Lateral Navigation (LNAV)] from top of descent to the final approach

phase of flight. Pilots may also have to make FMS route edits and respond to datalink clearances

in the Terminal Radar Approach Control (TRACON) airspace. This full mission simulator study

addresses how the introduction of these FMS descent procedures affect crew activities,

workload, and performance. It also assesses crew acceptance of these procedures. Results

indicate that the number of crew activities and workload ratings are significantly reduced below

current day levels when FMS procedures can be flown uninterrupted, but that activity numbers

increase significantly above current day levels and workload ratings return to current day levels

when FMS procedures are interrupted by common ATC interventions and CTAS routing

advisories. Crew performance showed some problems with speed control during FMS

procedures. Crew acceptance of the FMS procedures and route modification requirements was

generally high; a minority of crews expressed concerns about use of VNAV in the TRACON

airspace. Suggestions for future study are discussed.

t4,SUBJECTERMS

Automation human factors, Glass cockpit transport aircraft, Air traffic control

procedures, Datalink, Flight management system arrival procedures

17. SECURITY CLASSIFICATION

OF REPORT

Unclassified

NSN 7540-01-280-5500

18. SECURITY CLASSIFICATION

OF THIS PAGE

Unclassified

19. SECURITY CLASSIFICATION

OF ABSTRACT

Unclassified

15. NUMBER OF PAGES

93

16. PRICECODE A05

20. LIMITATION OF

ABSTRACT

Unlimited

Standard Form 298 (Rev. 2-89)

Prescribed by ANSI Std. Z-39-1e

298-102