October 2012

NASA/CR–2012-217777

Airborne Precision Spacing for Dependent 

Parallel Operations Interface Study 

Paul M. Volk, M.A. Takallu, and Keith D. Hoffler 

Adaptive Aerospace Group, Inc., Hampton, Virginia 

Jarold Weiser and Dexter Turner 

Optimation Technologies, Inc., Portland, Oregon 
National Aeronautics and 

Space Administration 

Langley Research Center Prepared for Langley Research Center 

Hampton, Virginia 23681-2199 under Contract NNL10AA14B

 

October 2012 

NASA/CR-2012-217777 

Airborne Precision Spacing for Dependent 

Parallel Operations Interface Study 

Paul M. Volk, M.A. Takallu, and Keith D. Hoffler 

Adaptive Aerospace Group, Inc., Hampton, Virginia 

Jarold Weiser and Dexter Turner 

Optimation Technologies, Inc., Portland, Oregon 
Available from: 

NASA Center for AeroSpace Information 

7115 Standard Drive 

Hanover, MD 21076-1320 

443-757-5802 

The use of trademarks or names of manufacturers in this report is for accurate reporting and does 

not constitute an official endorsement, either expressed or implied, of such products or 

manufacturers by the National Aeronautics and Space Administration. 
3

1 Table of Contents 

1 Table of Contents 3

2 Abbreviations 7

3 Introduction 8

4 Fully Integrated APS Interface Concepts 9

4.1 APS Interface Concepts for Primary Flight Display 10

4.1.1 PDS Target Speed Indication 10

4.1.2 PDS Mode Annunciation 12

4.2 APS Interface Concepts for the Navigation Display 14

4.2.1 Target Aircraft Depiction 14

4.2.2 Trend Indication 15

5 Methodology 17

5.1 Experiment Objectives 17

5.2 Dependent Measures 17

5.3 Experiment Scenario Design 17

5.4 Experimental Protocol 20

5.5 Subject Pilots 21

6 Results and Discussion 22

6.1 Subject Pilot Ratings for Usability of Proposed Interfaces 22

6.2 Subject Pilot Ratings of Interface Importance to APS-DPA Operations 26

6.3 Subject Pilot Interface Preferences 27

6.3.1 PDS Target Speed 27

6.3.2 PDS Mode Annunciation 28

6.3.3 Target Aircraft Depiction 28

6.3.4 Trend Indication 28
Page | 3 

1 Table of Contents

1 Table of Contents 3

2 Abbreviations 7

3 Introduction 8

4 Fully Integrated APS Interface Concepts 9

4.1 APS Interface Concepts for Primary Flight Display 10

4.1.1 PDS Target Speed Indication 10

4.1.2 PDS Mode Annunciation 12

4.2 APS Interface Concepts for the Navigation Display 14

4.2.1 Target Aircraft Depiction 14

4.2.2 Trend Indication 15

5 Methodology 17

5.1 Experiment Objectives 17

5.2 Dependent Measures 17

5.3 Experiment Scenario Design 17

5.4 Experimental Protocol 20

5.5 Subject Pilots 21

6 Results and Discussion 22

6.1 Subject Pilot Ratings for Usability of Proposed Interfaces 22

6.2 Subject Pilot Ratings of Interface Importance to APS-DPA Operations 26

6.3 Subject Pilot Interface Preferences 27

6.3.1 PDS Target Speed 27

6.3.2 PDS Mode Annunciation 28

6.3.3 Target Aircraft Depiction 28

6.3.4 Trend Indication 28
Page | 4 

6.3.5 Other Preference Considerations 29

6.4 Subject Pilot Preferences for Combinations of Interfaces 29

6.5 Suggested Improvements by Subject Pilots 30

6.5.1 PDS Target Speed Interface 30

6.5.2 PDS Mode Annunciation 31

6.5.3 Target Aircraft Depiction 31

6.5.4 Trend Indication 32

7 Conclusions 33

8 Acknowledgements 34

9 References 35

10 Appendix A: Tabulated Experimental Results 36

10.1 Results of Post-Run Questionnaires 36

10.1.1 Usability of Display Elements 36

10.1.2 Preference for Combinations of Display Elements 48

10.1.3 Comments and Suggestions for Improvement 51

10.2 Results of Final Questionnaire 51

10.2.1 Importance of Display Elements 51

10.2.2 Display Element Individual Preferences 53

10.2.3 Combined Display Element Preferences 54

10.2.4 Comments and Suggestions for Improvement 55

11 Appendix B: Supplemental Design Concepts for Fully Integrated Implementation 56

11.1 Engine Instrument & Crew Alerting System/Multifunction Display 56

11.2 Mode Control Panel 57

11.3 Multifunction Control and Display Unit 59

12 Appendix C: Electronic Flight Bag Interface Concepts for APS-DPA 65
Page | 5 

12.1 Description of Design Concept 65

12.1.1 PDS Home Page 65

12.1.2 Spacing Clearance Sub-Page 67

13 Appendix D: Additional Interface Concepts for APS-DPA 69

13.1 Stagger Interval Depiction 69

13.2 PDS Speed Reversion Mode 70

13.3 PDS Profile Mode 71

14 Appendix E: ASTOR Implementation Issues 74

14.1 PDS Mode Annunciation 74

14.2 Target Aircraft Depiction 74

14.3 Trend Indicators 75
Page | 6 

Abstract

This paper describes a usability study of proposed cockpit interfaces 

to support Airborne Precision Spacing (APS) operations for aircraft 

performing dependent parallel approaches (DPA). NASA has proposed 

an airborne system called Pair Dependent Speed (PDS) which uses their

Airborne Spacing for Terminal Arrival Routes (ASTAR) algorithm to 

manage spacing intervals. Interface elements were designed to facilitate 

the input of APS-DPA spacing parameters to ASTAR, and to convey PDS 

system information to the crew deemed necessary and/or helpful to 

conduct the operation, including: target speed, guidance mode, target

aircraft depiction, and spacing trend indication. In the study, subject 

pilots observed recorded simulations using the proposed interface 

elements in which the ownship managed assigned spacing intervals from 

two other arriving aircraft. Simulations were recorded using the Aircraft 

Simulation for Traffic Operations Research (ASTOR) platform, a 

medium-fidelity simulator based on a modern Boeing commercial glass 

cockpit. Various combinations of the interface elements were presented 

to subject pilots, and feedback was collected via structured 

questionnaires. The results of subject pilot evaluations show that the

proposed design elements were acceptable, and that preferable 

combinations exist within this set of elements. The results also point to

potential improvements to be considered for implementation in future 

experiments.
Page | 7 

2 Abbreviations

ADS-B Automatic Dependent Surveillance – Broadcast

AFDS Autopilot Flight Director System

ALT Altitude

APS Airborne Precision Spacing

AOC Airline Operations Center

APPR Approach 

ASTAR Airborne Spacing for Terminal Arrival Routes 

ASTOR Aircraft Simulation for Traffic Operations Research

ARINC429 Aeronautical Radio, Incorporated 429 data bus 

ATC Air Traffic Control

ATOL Air Traffic Operations Laboratory

ATSP Air Traffic Service Provider

CDA Continuous Descent Arrival 

CDTI Cockpit Display of Traffic Information 

CPDLC Controller Pilot Datalink Clearance

ConOps Concept of Operation 

DEV Deviation

DPA Dependent Parallel Approaches

EFB Electronic Flight Bag

EICAS Engine Instrument and Crew Alerting System 

EXEC Execute

FAA Federal Aviation Administration

FMA Flight Mode Annunciator 

FMC Flight Management Computer

FNL PDS Final Mode 

HITL Human-in-the-Loop 

ID Aircraft Identification

MCDU Multifunction Control and Display Unit 

MCP Mode Control Panel 

MFD Multifunction Display

MIN Minimum

MOD Modified 

N/A Not Applicable 

NASA National Aeronautics and Space Administration

NAVAID Navigation Aid 

NCT No Closer Than 

ND Navigation Display 
Page | 8 

NG Next Generation

NM Nautical Miles

PAIR PDS Paired Mode 

PFD Primary Flight Display

PDS Pair Dependent Speed

PRI Primary Spacing Aircraft, also known as Active Spacing Aircraft

PTH Path

REFAC Reference Aircraft, also known as Target Aircraft

RTA Required Time of Arrival

RWY Runway 

SEC Seconds 

SPD Speed

STA Scheduled Time of Arrival

TOD Top of Descent 

VNAV Vertical Navigation

3 Introduction

Airborne Precision Spacing (APS) is an operational concept where the control of the aircraft’s 

speed is delegated by the air traffic service provider (ATSP) to the flight crew in order to 

precisely achieve an assigned inter-aircraft spacing. The concept allows the flight crew to make 

minor speed adjustments based on cues from an on-board system while flying a continous 

descent arrival (CDA). The concept is intended to address both capacity and efficiency issues 

facing the air transportation system by enabling the merging of efficient trajectories by a variety 

of aircraft into a precisely-spaced arrival stream in the terminal area. 

Airborne systems that could be used for generating the speed commands have been under 

development for more than 10 years[1][2][3]. Fast-time simulations and human-in-the-loop 

(HITL) studies have been conducted during that period which have verified the effectiveness of 

the algorithms in achieving precise inter-aircraft spacing in the terminal environment[4][5]. The 

current NASA instantiation of this algorithm is called Airborne Spacing for Terminal Arrival 

Routes (ASTAR), and the airborne system is called Pair Dependent Speed (PDS).

As the development of airborne spacing systems progresses, so also does research into the 

procedures and human interfaces required to conduct spacing operations. EuroControl has 

already proposed flight deck user requirements for airborne spacing[6]. The HITL studies 

already mentioned also sought to determine if the procedures and cockpit interfaces proposed for 

conducting APS operations were acceptable. A recent human-in-the-loop (HITL) experiment

conducted at NASA Langley’s Air Traffic Operations Laboratory (ATOL) applied a prototype 

PDS system and associated procedures to the conduct of paired CDA profiles to a single runway. 

The results of this study showed that the procedures and interfaces used for airborne spacing can 

achieve the desired results with little negative performance impact over conventional CDAs[5]. 
Page | 9 

NASA has been working to extend the APS concept to support dependent parallel approaches 

(DPA), where the runways are close enough together that spacing must be managed between 

aircraft arriving on both runways[7]. This requires a redesign of the way the crew will interface 

with the aircraft, including the traffic display, speed guidance display, data entry and crew 

notifications. The focus of the study described in this report was to design, test, and refine crew 

interface concepts for APS-DPA operations. Two crew interface systems were investigated: a 

system with the interfaces fully integrated with the standard cockpit displays, and a system with

the application and interfaces hosted on an Electronic Flight Bag (EFB)1

. 

For this study, two interface concepts were designed, one suitable for a typical modern glass 

cockpit and the other for an EFB, and included means for inputting spacing control parameters, 

the display of PDS-commanded speed and PDS mode, the display of target aircraft and spacing 

trend indication. NASA’s Aircraft Simulation for Traffic Operations Research (ASTOR) was 

utilized as a platform on which to implement and display these concepts for the study. The study 

sought to maximize the insight gained from the pilot evaluations by developing alternative 

versions of the interfaces such that pilots could make comparisons and express preferences for 

either individual design elements or combinations of the elements. 

This report describes the conceptual design of the proposed APS interfaces for the fully 

integrated system, the methodology used for the conduct of the experiment, and the results of the 

pilot evaluations. The EFB interface concepts are also described herein, but were not intended to 

be part of the usability study (see Appendix C: Electronic Flight Bag Interface Concepts for 

APS-DPA). Final conclusions and recommendations are made based on analysis of the pilot 

evaluation data.

4 Fully Integrated APS Interface Concepts

The goal for the fully integrated version of the APS-DPA interface was to develop interfaces to 

allow pilots to perform the following functions: 

1. manage voice or datalink reception of spacing clearances;

2. initialize PDS with spacing clearance parameters;

3. verify validity of spacing clearance parameters before executing;

4. engage PDS target speed to the autopilot while also flying CDA profile;

 

1 An EFB (FAA's Advisory Circular - AC120-76A[10]), is an electronic display system in addition to the standard 

cockpit displays that displays a variety of aviation data, automates basic calculations (e.g., performance data, fuel 

calculations, etc.), and/or presents various graphical information to help flight crews perform flight management 

tasks more easily and efficiently. 
Page | 10 

5. monitor progress of APS-DPA, including: 

a. PDS target speed;

b. PDS operation mode; 

c. Target aircraft position and information; 

d. Spacing deviation and trend information; and

e. Off-nominal indications. 

6. disengage PDS from autopilot and/or cancel spacing clearance.

Concepts were developed which would modify the operation of several typical glass cockpit 

displays, interfaces, and their underlying control software. These concepts were implemented in 

a special ASTOR software build created for the study to support simulations, and included 

modifications to the following:

• Engine Instrument & Crew Alerting System/Multifunction Display (EICAS/MFD);

• Mode Control Panel (MCP); 

• Multifunction Control and Display Unit (MCDU);

• Primary Flight Display (PFD); and

• Navigation Display (ND).

The focus of the study was to obtain pilot evaluations for the design elements developed for the 

PFD and the ND. The modifications made to the EICAS/MFD, MCP and MCDU were necessary 

to integrate the APS-DPA functionality and enhance the realism of the spacing scenarios 

presented on the ASTOR, but were not the focus of pilot evaluations. Implementation details of 

the EICAS/MFD, MCP and MCDU concepts are discussed in Appendix B: Supplemental Design 

Concepts for Fully Integrated Implementation. 

4.1 APS Interface Concepts for Primary Flight Display

The PFD typically displays tactical information to the crew. Alternative design concepts were 

proposed to use the PFD to present the PDS-commanded target speed and annunciation of the 

active PDS guidance mode.

4.1.1 PDS Target Speed Indication 

During spacing operations, ASTAR calculates a target speed (also referred to as PDS target 

speed) to be met by the aircraft in order to achieve the spacing interval. Two different concepts 

for displaying the PDS target speed were implemented for evaluation in this study, shown in 

Figure 4-1. 
Page | 11 

Figure 4-1: PDS Target Speed Concepts: Text (Left); Speed Bug (Right). 

In the first concept, the PDS target speed appeared as a textual indication in a dedicated area of 

the upper right corner of the PFD. The PDS target speed text appeared in white until PDS was 

engaged to the autopilot, when the text changed to green. Changes to PDS target speed were 

highlighted for 10 seconds by a flashing box surrounding the text. This concept was identical to 

the implementation of PDS target speed in the preceding APS HITL study. 

This concept was considered acceptable by subject pilots participating in the previous 

experiment who were tasked with manually entering the PDS target speed into the MCP speed 

window. However, the concept had not yet been applied to the conduct of APS-DPA operations, 

nor had it been used in a fully integrated, autopilot-flown profile. 

In the second concept, the PDS target speed was indicated by a filled green “speed bug” on the 

speed tape of the PFD. The green bug appeared when PDS was providing valid speed guidance, 

regardless of the state of autopilot engagement. When the PDS target speed was engaged to the 

autopilot, the implementation caused the green bug and the normal commanded-speed bug 

(magenta) to coincide. The green bug was displayed all the way to touchdown unless PDS was 

terminated. 

It was thought that a graphical PDS target speed indicator provided a more visual, contrasting 

alternative to the textual display. By juxtaposing the PDS target speed bug and the normal 

autopilot commanded speed bug such that neither could be obscured, the pilot could quickly 

determine whether the two coincided or diverged. In autothrottle mode, this allowed the pilot to 

anticipate speed changes from PDS; if manual speed intervention were used (which was not part 

of this study) the pilot could adjust the commanded speed until the indicators coincided. 
Page | 12 

4.1.2 PDS Mode Annunciation

During an APS operation, PDS speed guidance transitions through several different modes of 

operation. Beyond ADS-B range of the target aircraft, the Required Time of Arrival (RTA, 

formerly referred to in the APS concept as STA for Scheduled Time of Arrival) mode drives the 

target speed. The Paired mode is active once all of the data is available to begin spacing relative 

to the target aircraft. The Final mode indicates PDS speed guidance has transitioned to a 

stabilized final approach speed. Two other modes were defined in the ConOps; PDS Profile 

mode (speed guidance without interval management) and PDS Speed Reversion mode (offnominal conditions suspending ASTAR guidance). Neither were implemented for this study, but 

concepts were proposed for their display. Two different concepts were implemented for the 

display of PDS mode annunciation, shown in Figure 4-2. 

Figure 4-2: PDS Mode Annunciation Concepts: Text (Left); FMA (Right). 

In the first concept, the PDS mode appeared as a textual indication in a dedicated area of the 

upper right corner of the PFD. The PDS mode text appeared in white until PDS was engaged to 

the autopilot, when the text changed to green. Changes to PDS mode were highlighted for 10 

seconds by a flashing (white or green) box surrounding the text. This concept was identical to 

the implementation of PDS target speed in the preceding APS HITL study. 

This concept was considered acceptable by subject pilots participating in the previous 

experiment, however, it had not yet been applied to the conduct of APS-DPA operations, nor had 

it been used in a fully integrated, autopilot-flown profile. 

In the second concept, PDS mode annunciation was integrated into the Flight Mode Annunciator 

(FMA) of the PFD. The FMA is used to annunciate autoflight settings and modes of the 

autopilot. To indicate PDS autoflight modes, the pitch mode segment of the FMA was divided 

into four (4) quadrants, as shown in Figure 4-3. The leftmost upper and lower quadrants of the 

pitch mode segment displayed the vertical autopilot mode currently selected and armed, as 
Page | 13 

normal. The modified right upper and lower quadrants of the pitch mode segment displayed the 

active and armed PDS modes, respectively. 

 Figure 4-3: FMA Pitch Segment Modified to Display PDS Modes 

It was thought that annunciating PDS mode with other autopilot modes in the FMA was more 

consistent with an integrated approach than the ad hoc presentation of the first concept. (The 

integration of PDS guidance to the autopilot is also discussed in Appendix B: Supplemental 

Design Concepts for Fully Integrated Implementation.) Table 4-1 shows several examples of the 

concept implementation during typical phases of an APS-DPA operation. The following 

conventions established for FMA annunciation of existing modes permitted enhancements to the 

PDS mode display not implemented in the first concept. For example, display of the next PDS 

mode in sequence was implemented on the line reserved for armed modes on the FMA. Further, 

a convention exists to display degraded autopilot modes on the FMA, and it could be applied to 

off-nominal PDS conditions, such as PDS Speed Reversion mode. (An example of this concept is 

also shown, but was not implemented in ASTOR.) 

Table 4-1: PDS Mode Annunciation Examples; FMA. 

PDS 

FMA 

Example

Description FMA Pitch Mode 

Depiction

PDS RTA 

Mode 

Engaged

When PDS mode 

changes, green box 

appears for 10 

seconds; Pair mode 

armed

 

PDS Pair 

Mode 

Engaged

PDS Final mode 

armed; G/S armed

PDS Final 

Mode 

Engaged

VNAV disengaged; 

G/S engaged

PDS 

Mode 

Degraded

PDS off-nominal 

condition (Speed 

Reversion mode) 
Page | 14 

4.2 APS Interface Concepts for the Navigation Display

The ND presents strategic information to the crew, so this design concept used the ND to present 

the “big picture” of the APS-DPA operation. Concepts were developed for depicting one or two 

target aircraft involved in the spacing operation. Concepts were also developed for depicting the 

spacing deviation output from PDS in the form of a trend indicator, which might be used as a 

strategic tool for monitoring the operation’s progress. 

4.2.1 Target Aircraft Depiction

The concepts for displaying target aircraft relied on existing or proposed standards for Cockpit 

Display of Traffic Information (CDTI) symbols. Both DO-317 (Minimum Operational 

Performance Standards for Aircraft Surveillance Applications System) [8] and a draft symbol set 

appendix for DO-289 (Aircraft Surveillance Applications Minimum Aviations System 

Performance Standards) [9] were used as references. The standards impart specific meanings to 

CDTI symbols and colors, which were applied to the paradigm of an on-board application 

supporting the APS-DPA operation. 

Two similar concepts were proposed, shown in Figure 4-4. For both concepts, the standard CDTI 

symbol for depicting ADS-B targets was a chevron, and in particular, a green chevron with a 

shape-hugging border indicated that the target was coupled to an on-board application (in this 

case, PDS). For both concepts, ND conventions for displaying CDTI symbols were implemented, 

such that the aircraft ID of the target aircraft was displayed, subject to pilot control. Further, if 

the ND range selector put the CDTI symbol(s) out of range, convention dictated that a halfsymbol for the symbol appeared on the outer range ring of the ND at the appropriate bearing. 

Figure 4-4: Target A/C Concepts: Coupled Only (Left); Coupled with Data Block (Right). 
Page | 15 

The first concept displayed the target aircraft as the green “double” chevrons. No distinction was 

made between the two target aircraft other than their associated IDs, i.e., which aircraft was the 

precision or “no closer than” (NCT) target, or which aircraft was the active spacing target. This 

was a basic concept for displaying the target aircraft with a minimum of clutter on the ND. It was 

thought that pilots wishing to distinguish between precision and NCT targets could derive this by 

correlating the aircraft ID and the original spacing clearance parameters on the PDS main menu 

page of the MCDU. Similarly, the active spacing aircraft ID was depicted (in magenta) on the 

same MCDU page, see the implementation detail in Appendix B: Supplemental Design Concepts 

for Fully Integrated Implementation”. 

The second concept interpreted the standards further to make a distinction between the two 

spacing targets by adding a green “selected” circle around the active spacing target. This

enabled another enhancement in the form of a data block associated with the selected target. The 

data block always appeared in the lower left portion of the ND and contained information 

associated with the selected target aircraft, including the aircraft ID, aircraft type, spacing 

interval type, spacing deviation and landing runway. This concept was thought to provide 

significantly more information about the target aircraft, at the expense of additional ND clutter. 

The justification for displaying the additional information was to conveniently present strategic 

information on the progress of the spacing operation that would otherwise only be available 

through the PDS menu pages in the MCDU. (The MCDU has several other uses, and use of the 

data block could avoid interrupting them to look for spacing information.) 

Note that while the first concept could be considered to be a selectable subset of the second, the 

ability to select ND views was not available in the study’s scenario presentation, so the concepts 

were presented separately.

4.2.2 Trend Indication

Trend indication is potentially useful strategic information during phases of APS when PDS is 

managing the spacing interval. The trend indicator concept proposed synthesizing PDS outputs 

to depict the aircraft’s position relative to spacing limits. A spacing operation can be conducted 

within recoverable limits of spacing deviation from the specified spacing interval. Beyond these 

limits, the deviation is termed excessive and the spacing interval can no longer be achieved 

without also exceeding programmed PDS limits. The spacing deviation (in seconds) from 

nominal was displayed on the MCDU’s PDS main menu page (and also on certain target aircraft 

depictions) but the maximum limits of deviation were not displayed. Two different concepts 

were developed for integrating both pieces of information into a trend indicator, shown in Figure 

4-5. 
Page | 16 

Figure 4-5: Trend Indicator Concepts: Conformance Box (Left); Deviation Scale (Right). 

In the first concept, the excessive spacing deviation limits were used to draw pictorial 

“conformance box” brackets around the ownship symbol on the ND. The box was positioned 

around the nose of the ownship symbol relative to the spacing deviation from nominal. Positive 

spacing deviations shifted the box so that the forward brackets moved away from the nose of the 

ownship; negative spacing deviations shifted the forward brackets towards the nose of the 

ownship. There was no center position depicted; as long as the nose of ownship remained within 

the conformance box, the spacing deviation was within recoverable limits. 

The conformance box boundaries were range-based (time-based deviation limits from PDS

converted to distance), thus the size of the depicted conformance box varied in accordance with 

the ND range selector. It was thought that this concept offered an intuitive, scan-friendly option 

for viewing the trend information because it was aligned with the ownship and the course line. 

The second concept proposed a deviation indicator presentation in the lower left corner of the 

ND. The deviation outputs from PDS were synthesized to display the aircraft’s spacing deviation 

relative to the excessive spacing limits with a green pointer on a fixed scale. A positive spacing 

deviation moved the pointer up the scale, and a negative spacing deviation moved the pointer 

down. (A modification to this concept is discussed in Section 6.5.) Unlike the first concept, the 

size of the scale remained constant, therefore the sensitivity increased as the PDS limits 

decreased and vice-versa. Like the first concept, there was no zero deviation center indicator. It 

was thought that this design concept would seem familiar to current commercial pilots, and 

therefore be met with some acceptance, because a similar Vertical Navigation (VNAV) deviation 

scale is depicted in the lower right corner of the ND to indicate altitude deviations relative to the 

profile being flown. 
Page | 17 

5 Methodology

5.1 Experiment Objectives

The primary objective of the experiment was to evaluate the usability and acceptability of the 

interfaces designed to support APS-DPA operations. It was hypothesized that pilots would find 

all versions of the interfaces presented to be both usable and acceptable, and that insight would 

be gained from analyzing the questionnaire responses by subject pilots in the following areas of 

interest:

• importance of the proposed interface to the conduct of the operation; 

• usability of proposed interfaces; 

• preferences in the proposed display interfaces; 

• preferences in combinations of the proposed interfaces; 

• design anomalies; and

• suggestions for improvement.

Since this was a study in which no interaction with the recorded scenario was possible, there was 

no attempt to evaluate subjective workload, procedures, or performance.

5.2 Dependent Measures

The dependent variables for the study consisted of subject pilot evaluations of the interface 

design elements, collected via structured questionnaires administered after individual test runs 

and also after the complete session. The questionnaires were designed to gather the following 

types of information. 

• Usability: Were the design elements understandable? Were they intuitive, user-friendly? 

• Usefulness: Were the design elements needed or not? After seeing them, if they were 

subsequently removed, would the subject pilot miss any of them? Did they impact the 

situation awareness of the APS-DPA operation? 

• Acceptability: Did the design elements make the subject pilot feel more comfortable about 

the APS-DPA operation? Was the subject pilot confident about the status of the operation? 

• Improvements: Assuming the proposed concepts were useful, what could be done to enhance 

the effectiveness, appearance or other aspects of the design elements?

5.3 Experiment Scenario Design

The concept for the experiment was to present the proposed interface concepts in a manner best 

suited to obtaining subject pilot evaluations on their acceptability and usability. The ASTOR 

platform was chosen for its ability to realistically represent the proposed APS interfaces as they 

would appear on modern Boeing commercial glass cockpit displays. Custom ASTOR software 

was developed to implement the APS interface concepts, and modified spacing scenarios from 

the previous ATOL APS experiment were used to run simulations with the ASTOR software and 

generate recordings of a simulated spacing operation from the cockpit perspective of the 

simulated aircraft.
Page | 18 

Although this approach required more effort than traditional storyboarding techniques for 

presenting new display concepts, it was deemed to have several advantages. First, the recorded 

spacing scenario offered a continuous, full-motion representation of the interface concepts as 

they appeared during the spacing operation, enhancing the realism and affording the subject 

pilots an opportunity to evaluate the interfaces appearance at any point in the scenario. Second, 

the ASTOR programming modifications were reusable for future ATOL demonstrations and/or 

experiments at the completion of this study. A key feature of the ASTOR modifications was the 

ability to control the APS interface design elements via a configuration file, such that any 

combination of the design elements could be selectively enabled for a particular simulation. This 

feature allows these elements to be quickly modified to evaluate their usability in future spacing 

scenarios.

The traffic scenario for the recorded simulation utilized models which match CDA operations at 

the Louisville Standiford Airport (SDF). An existing traffic scenario which merged two eastbound streams into a single CDA to Runway 17R was modified to separate the streams and 

simulate dependent parallel approach operations to Runway 17R and Runway 17L. A list of 

simulated test conditions is included below. Simulated winds were set to zero (no wind) to 

enhance the repeatability of the scenarios from run to run. 

• Winds: No wind, all altitudes 

• Aircraft type: Heavy, two-engine, narrow-body transport aircraft 

• ASTOR Build: OpTech_ASTAR10_2010SEP16 

• ASTAR version: 10

The scenario consisted of six aircraft (ID = NASA xxx), each assigned to fly one of the two 

published CDAs: 1) the SEA BISCUIT ONE (CBSKT1) ARRIVAL to Runway 17R, and 2) the 

CALKS ONE (CALKS1) ARRIVAL to Runway 17L. The lead aircraft in the CALKS ONE 

arrival stream was NASA 907, which autonomously flew the CDA to Runway 17L with a 

Required Time of Arrival (RTA) but without an assigned spacing interval. The lead aircraft in 

the SEA BISCUIT ONE arrival stream was NASA 893, which autonomously flew the CDA to 

Runway 17R with a RTA and without an assigned spacing interval. The remaining aircraft in the 

scenario autonomously flew the CDAs and spacing interval assignments specified in Table 5-1. 

See Figure 5-1 for a pictorial diagram of the spacing scenario which shows the spacing intervals 

for the ASTOR station (NASA 995) used as the ownship in the experiment presentation. 

Table 5-1: Spacing Scenario Clearance Parameters. 

Aircraft ID CDA Runway RTA Precision 

Interval 

Target

Aircraft

NCT 

Interval 

Target

Aircraft

NASA 907 CALKS1 17L 06:24:13Z

NASA 859 CALKS1 17L 06:26:52Z 180 sec NASA 907

NASA 917 CALKS1 17L 06:32:53Z 140 sec NASA 893

NASA 893 CBSKT1 17R 06:25:32Z

NASA 995 CBSKT1 17R 06:28:16Z 165 sec NASA 893 90 sec NASA 859

 
Page | 19 

Figure 5-1: Spacing Scenario Diagram. 

The intrinsic recording facilities of ASTOR were utilized to capture data necessary for playback 

of the multi-aircraft spacing scenarios for the usability study presentations. This facility enabled 

a portable demonstration displaying multiple aircraft while using only one ASTOR station. The 

use of a recorded simulation file ensured that the simulated scenario would appear identically for 

each presentation. The simulations were recorded in a laboratory similar to the ATOL, equipped 

with several ASTORs and a scenario traffic generator. The experimental software build used for 

the simulations allowed the ASTOR to: 1) simulate reception of a spacing clearance via datalink; 

2) parse the clearance and load the spacing parameters to the MCDU PDS menu pages 

(Appendix B: Supplemental Design Concepts for Fully Integrated Implementation); and 3) 

execute the spacing instructions by autonomously flying the PDS target speed. After the scenario 

was run to completion, the recorded file from the ASTOR representing NASA 995 was saved for 

playback during the experiment sessions. 

The procedure for recording the experimental scenario was to start the simulation with all 

ASTOR stations (aircraft) at a point prior to top of descent (TOD). Aircraft then received spacing 

clearances via datalink, also before TOD. The test operator visited each ASTOR station to 

manually load the received clearance parameters to ASTAR via the MCDU PDS menu pages. 

The verified clearance was then executed and the autopilot was manually engaged by selecting 

the “PDS” engage function on the MCP. The manual nature of serially starting all six of the 
Page | 20 

ASTOR stations led to minor timing variations in the eight (8) recorded scenarios, but these 

variations were largely damped out as the scenario progressed. 

In order to program the ASTOR representing NASA 995 with the NCT spacing interval 

necessary to simulate APS-DPA, the test operator manually entered the NCT interval parameters

via the MCDU PDS menu interface. Again, this was done prior to the TOD for NASA 995. Once 

the initial clearance was modified with the NCT interval, the test operator manually executed the 

clearance via the MCDU.

5.4 Experimental Protocol

An experiment session consisted of several presentations displayed on a workstation consisting 

of a computer monitor and a mouse. The associated computer was programmed with the ASTOR 

software, which simulated the operation of a modern commercial Boeing glass cockpit. Before 

the ASTOR software was started, the subject pilot was given a pre-experiment briefing on APSDPA concepts and the proposed design concepts that were to be evaluated. 

The bulk of the session consisted of nine runs, each of which presented the subject pilot with a 

recorded spacing scenario replayed on the ASTOR platform. The experimental runs each 

simulated an APS-DPA spacing operation in which the ownship (NASA 995) was assigned both 

a precision interval and an NCT interval. Played back at normal speed, the recorded simulations 

would have lasted approximately twenty-four (24) minutes from TOD to touchdown. However, 

the ASTOR playback facility allowed the speed of playback to be varied by the test operator. 

This facility was used to fast-forward the playback through periods of relative inactivity, and 

each run was shortened to approximately five to seven minutes. 

During the first familiarization run (R0), the subject pilot was asked to interact with the ASTOR 

by using the mouse to enter a typical spacing clearance into the MCDU pages specifically 

designed for this exercise. In the remaining runs (R1-R8), the subject pilots were asked to only 

observe the presentation as the spacing scenario progressed. 

A post-run questionnaire was administered to capture the subject pilot’s evaluation of the design 

elements presented. After all of the test runs were completed, a final questionnaire was 

administered in which overall impressions of the test were captured.

In an ideal experiment, only one independent variable would change per run in order to best 

associate its interrelation to the dependent variables. This approach was considered both 

impractical and unnecessary for this usability study, as it would have taxed the desired 2-3 hour 

time limit for the experiment, and it was believed the grouping of certain design elements would 

not significantly compromise the subjective data obtained. See Table 5-2 for a description of test 

runs, detailing the design elements which were grouped for each run. The runs were given a run 

number so they could be correlated to the post-run questionnaire data. Although the test runs 

were numbered, the run order was randomized for each test session to prevent unintended bias 

introduced by a set order. 
Page | 21 

Table 5-2: Test Run Matrix Showing Display Options Enabled 

Run PDS Target 

Speed Option

PDS Mode 

Option

Target Aircraft 

Option

Trend Indicator 

Option

R1 Text Text Coupled Only None

R2 Text FMA Coupled Only None

R3 Speed Bug Text Coupled Only None

R4 Speed Bug FMA Coupled Only None

R5 Text Text Coupled Only Conformance Box

R6 Text Text Coupled Only Deviation Scale

R7 Text Text Coupled w/Data Conformance Box

R8 Text Text Coupled w/Data Deviation Scale

Run R0: This run was part of every experiment in order to introduce the subject pilot to the 

MCDU PDS menu pages added to monitor and control the spacing operation, and also 

familiarize the subject pilot with the specific spacing clearance(s) which were part of the 

recorded scenario. The subject pilot was given specific tasks to accomplish in these menu pages, 

which included: 

• Identifying the spacing aircraft and intervals;

• Identifying the landing runway(s) for the spacing aircraft;

• Determining whether the clearance can be accepted (i.e., ASTAR algorithm is providing 

speed guidance);

• Modifying a spacing parameter as requested by the investigator;

• Executing the spacing clearance.

Runs R1-R4: These runs were made holding design elements for target aircraft and trend 

indication constant, while varying the combinations for displaying PDS target speed and PDS 

mode, as shown in Table 5-2. 

Runs R5-R8: These runs were made holding design elements for PDS target speed and PDS 

mode constant, while varying the combinations for displaying target aircraft and trend indication, 

as shown in Table 5-2. 

5.5 Subject Pilots

The subject pilots for this study were either current or former commercial airline pilots with 

experience in one or more of the following aircraft: 1) Boeing 777; 2) Boeing 737 NG (Next 

Gen); 3) Boeing 767-400; and 4) Boeing 747-400. The ASTOR platform’s representation of a 

modern Boeing glass cockpit was sufficiently familiar to all subject pilots to require no 

additional training other than a briefing on the new APS interfaces, and the experimental 

scenarios and procedures. 

Volunteers were solicited by contacting the Airline Operations Centers (AOCs) for United 

Airlines at the Chicago (ORD) and Dulles (IAD) airports. These pilots were tested in-domicile. 
Page | 22 

Additional local pilots were solicited for dry-run experiments held at or near the Contractor’s 

office in Hampton, VA. These pilots had the same experience as listed above, and flew (or had 

recently flown) for Airtran Airlines, American Airlines, and Southwest Airlines. All subject 

pilots received a small stipend for participating. 

6 Results and Discussion

The following is a discussion of the experiment objectives and dependent measures obtained 

from analysis of the subject pilot responses to the test questionnaires. The full text of the 

questionnaires, as well as a complete set of results for the questionnaires, is shown in Appendix 

A: Tabulated Experimental Results. 

6.1 Subject Pilot Ratings for Usability of Proposed Interfaces

The post-run questionnaires asked subject pilots to rate their ability to identify specific 

information presented in the spacing scenario, using a scale of 6 “very easy to identify” to 1 “not 

able to identify at all”. If it was believed the information was missing from a particular run, the 

subject pilot was asked to answer “Not Applicable” (“N/A”), which was interpreted as missing 

data. These data were correlated with the elements presented in that run (see Figure 6-1) to 

determine the rating for a particular design element option (for example, text vs. green bug 

option for PDS target speed.) Since the test run matrix was not balanced (some design options 

were shown more frequently than others), the sample size (N) for each element option is noted in 

the following results analysis.

The post-run questionnaire also asked subject pilots to rate the helpfulness of each design 

element in understanding the spacing operation, using a scale of 5 “very helpful” to 1 “not 

helpful”. These data were correlated with the elements presented in that run as above.

For PDS target speed, the text option was presented to subject pilots in runs R1, R2, R5, R6, R7 

and R8; the speed bug option was presented in runs R3 and R4. After each run, subject pilots 

were asked to rate their ability to identify 1) current PDS target speed and 2) changes to PDS 

target speed. The pertinent data collected from this question is summarized in Table 10-2 and 

Table 10-3. As shown in Figure 6-1, the average ratings for the text option were 5.40 (SD= 0.65, 

N=150) and 5.03 (SD=0.87, N=150), respectively. The average rating for the green speed bug 

were 5.22 (SD=0.92, N=50) and 5.00 (SD=1.06, N=50), respectively. In response to the 

helpfulness question (see Table 10-20), the average rating was 4.22 (SD=0.66, N=1472

) for the 

 

2 The missing response(s) were due to an internet connection drop while the questionnaire was being administered.
Page | 23 

text option, and 4.08 (SD=0.99, N=492

) for the green speed bug. This indicates subject pilots had 

a high degree of confidence in the usability of either design option, with the text option rating 

slightly higher for all categories.

Figure 6-1: PDS Target Speed Option Average Ratings. 

For PDS mode, the text option was presented to subject pilots in runs R1, R3, R5, R6, R7 and 

R8; the FMA option was presented in runs R2 and R4. After each run, subject pilots were asked 

to rate their ability to identify 1) current PDS mode and 2) changes to PDS mode. The pertinent 

data collected from this question is summarized in Table 10-4 and Table 10-5. As shown in 

Figure 6-2, the average ratings for the text option were 5.17 (SD= 0.77, N=150) and 4.83 

(SD=0.88, N=1492

), respectively. The average ratings for the FMA option were 5.16 (SD=1.07, 

N=50) and 4.96 (SD=1.08, N=50), respectively. Subject pilots were also asked to rate their 

ability to identify the next PDS mode in sequence. In this case, only the FMA option provided 

this information, so subject pilots should have responded with “N/A” for the runs with the text 

option. A significant number of subject pilots did not respond with “N/A” (many responded 

instead with “not able to identify”), indicating some confusion regarding the instructions. The 

pertinent data collected from this question is summarized in Table 10-6. The average rating to

this question was 3.23 (SD=2.03, N=120) for the text option, and 4.85 (SD=1.43, N=46) for the 

FMA option. If the N/A responses are converted to “not able to identify”, the average rating 

becomes 2.78 (SD=2.02, N=150) for the text option. Both of these results are significantly lower 

than for the FMA option. These data indicate subject pilots had a high degree of confidence in 

their ability to identify PDS mode and mode changes for either design option, and a higher

degree of confidence in their ability to identify the next mode in sequence for the FMA option. In 

response to the helpfulness question (see Table 10-21), the design alternatives were rated equally 

high, with average ratings of 4.00 for each alternative. 

1 2 3 4 5 6

Current Target Speed

Changes to Target Speed

Helpfulness

Average Rating

speed bug

text

(Scale 1-5)

(Scale 1-6)

(Scale 1-6)
Page | 24 

Figure 6-2: PDS Mode Annunciation Option Average Ratings. 

For target aircraft depiction, the “coupled only” option was presented to subject pilots in runs 

R1, R2, R3, R4, R5, and R6; the “coupled with data block” option was presented in runs R7 and 

R8. Subject pilots were asked to rate their ability to identify several characteristics of the target 

aircraft. Some of these characteristics were only identifiable in the data block, so subject pilots 

should have responded with “N/A” for the runs where the information was not present. The lack 

of “N/A” responses led to the hypothesis that subject pilots may have identified some of this 

information from the MCDU, which was “parked” on the PDS main menu page during the 

recorded scenarios. (In support of this hypothesis, several subject pilots responded correctly for 

target aircraft characteristics available ONLY from the data block and not the MCDU page. For 

instance, assigned runway and aircraft type together received fifty-six of the fifty-seven “N/A” 

responses in the “coupled only” runs, indicating that most subject pilots correctly responded that 

the information was missing.) This menu page would have allowed the subject pilot to determine 

the active spacing aircraft, spacing interval type and spacing deviation without the need for the 

data block. This may have skewed the intended results of the questions about these 

characteristics. The pertinent data collected from this question is summarized in Table 10-7

through Table 10-18, and Table 10-22. The average ratings for rating the usability of target 

aircraft characteristics and the overall helpfulness of the design alternatives are shown pictorially 

in Figure 6-3. The average ratings consistently favor the “coupled with data block” option over 

the “coupled only” option, indicating that subject pilot’s found characteristic(s) of the former 

more useful and helpful. The perceived importance of these characteristics is discussed in the 

next section.

1 2 3 4 5 6

Current Mode

 Changes to Mode

 Next Mode in Sequence

 Helpfulness

Average Rating

FMA

text

(Scale 1-5)

(Scale 1-6)

(Scale 1-6)

(Scale 1-6)
Page | 25 

Figure 6-3: Target Aircraft Depiction Option Average Ratings. 

For the trend indicator, the “conformance box” option was presented to subject pilots in runs R5 

and R7, the “deviation scale” option was presented in runs R6 and R8, and there was no trend 

indicator displayed on runs R1, R2, R3, and R4. After each run, subject pilots were asked to rate 

their ability to determine and understand the aircraft’s progress towards achieving the assigned 

spacing goal. The pertinent data collected from this question is summarized in Table 10-19. As 

shown in Figure 6-4, the average rating for the conformance box option was 4.84 (SD=0.61, 

N=50). The average rating for the deviation scale option was 4.84 (SD=0.76, N=50). The 

average rating for no trend indication was 4.18 (SD=0.75, N=100). The results indicate a slight 

enhancement in the subject pilot’s ability to judge the aircraft’s spacing progress with either 

trend indicator. The benign nature of the recorded spacing scenario did not cause significant 

deviations to be displayed by either trend indicator, possibly diminishing the perception of its 

potential usefulness (see comments in Section 6.3.4). In response to the helpfulness question (see 

Table 10-23), the average ratings indicate a modest preference for the conformance box option 

(3.98, SD=1.19, N=492

) vs. the deviation scale option (3.48, SD=1.22, N=482).

1 2 3 4 5 6

How Many

 Active Spacing Aircraft

 Ownship Position

 Aircraft IDs

 Spacing Interval 893

 Spacing Interval 859

 Spacing Deviation 893

 Spacing Deviation 859

 Assigned Runway 893

 Assigned Runway 859

 Aircraft Type 893

 Aircraft Type 859

 Helpfulness

Average Rating

coupled w/data block

coupled only

Scale (1-5)

Scale 

(1-6)
Page | 26 

Figure 6-4: Trend Indicator Option Average Ratings. 

6.2 Subject Pilot Ratings of Interface Importance to APS-DPA Operations

An evaluation of the relative importance of the information conveyed by the design elements was 

obtained via the final questionnaire. The data was analyzed to verify that each element category 

was considered useful, and to further determine which characteristics of each element was 

considered the most useful. This data indicates that subject pilots considered all of the element 

categories to be important, but not equally important. 

Subject pilots were asked to rate the importance of each of the four categories of design elements 

(i.e., PDS target speed indicator, PDS mode, target aircraft depiction and trend indicator). The 

pertinent data collected from this question is summarized in Table 10-27. Using a scale of 5

“very important” to 1 “not important”, the responses in rank order were: 

1. PDS target speed indicator (average rating = 4.92, SD = 0.27, N=25); 

2. Target aircraft depiction (average rating = 4.72, SD = 0.60, N=25),; 

3. PDS mode annunciation (average rating = 4.36, SD = 0.89, N=25); 

4. Trend indicator (average rating = 4.12, SD = 1.18, N=25). 

For each design element category, subject pilots were asked to rate the importance of several 

characteristics of that element. For example, relative to PDS target speed, they were asked to rate 

the importance of being able to determine both the current target speed, and changes to the target 

speed, using a scale of 5 “very important” to 1 “not important”. The results shed light on which 

element characteristics were most valued, and provide a context for the element preferences 

discussed later.

For the PDS target speed indicator (see Table 10-28), it is clear that both of the element 

characteristics were considered very important; subject pilots responded that detecting speed 

1 2 3 4 5 6

Ability to Judge Progress

 Helpfulness

Average Rating

deviation scale

confirmance box

none

Scale (1-5)

Scale (1-6)
Page | 27 

changes (average rating = 4.80) was almost as important as displaying the PDS-commanded 

speed (average rating = 4.92). 

For the target aircraft depiction (see Table 10-29), it is interesting to note that out of eight listed 

characteristics, the three most important (ownship relative position, active spacing aircraft and 

number of target aircraft) were shown without the use of the data block. While both of the target

aircraft design alternatives were able to depict the ownship relative position and number of 

spacing targets, only one of the alternatives (“coupled with data block”) was implemented to also 

indicate the active spacing aircraft with a selected circle. This one highly valued characteristic 

may be a large contributor to the significant preference for this design alternative discussed later. 

The rank ordering of the five remaining characteristics (spacing deviation, aircraft ID, assigned 

runway, aircraft type, spacing interval type) indicate the information within the data block 

implementation that subject pilots found most useful. 

For the PDS mode annunciation (see Table 10-30), it is interesting to note that two 

characteristics, current mode (average rating = 4.24) and changes to mode (average rating = 

4.20) were rated significantly higher than the ability to determine the next mode in sequence 

(average rating = 3.44). While both of the PDS mode design alternatives were able to show 

current mode and changes to mode, only the FMA design alternative was able to show the next 

PDS mode in sequence. This characteristic was still considered important, and may have been a 

contributor to the preference for the FMA design option (see Section 6.3.2.)

6.3 Subject Pilot Interface Preferences

The final questionnaire asked subject pilots to indicate a preference for individual design 

elements in each of four categories; PDS target speed; PDS mode annunciation, target aircraft 

depiction and trend indicator. Subject pilots were given a choice of both design options in each 

category, or “None” to indicate no option was preferred. They were also asked to consider 

whether their preference changed when the elements were considered in combination rather than 

individually. (only one subject pilot answered in the affirmative). The results are summarized in 

Table 10-31 through Table 10-35. 

6.3.1 PDS Target Speed

Subject pilot responses were almost evenly split on which of the two PDS target speed display 

options was preferred, with thirteen (52%) preferring the green speed bug option and twelve 

(48%) preferring the textual speed indication. This result confirms the hypothesis that both 

options were usable and acceptable, and echoes the usability ratings discussed previously. A 

pilot who favored the green speed bug commented “...best presented as a speed bug for instant 

comparison with actual speed.” A pilot who preferred the text option said, “I prefer the text 

version of PS (sic) target speed because I can always find it. Rapid changes to the bug display 

can cause it to disappear.”
Page | 28 

6.3.2 PDS Mode Annunciation

Subject pilot responses slightly favored the FMA integration option of PDS mode display, with 

fifteen (60%) preferring the FMA option and ten (40%) preferring the textual mode indication. 

This result is again similar to the usability and helpfulness ratings discussed previously, where 

both options were rated almost equally useful. A pilot who preferred the FMA option 

commented, “Most aircrews are trained to be very diligent on FMA observation, mode changes, 

and correct presentation for desired flight mode. So basically this would be in normal aircrew 

scan/procedures...” A pilot who preferred the text option said, “...I think it will enhance 

understanding and avoid confusion by keeping the PDS modes in the text box and separate from 

the VNAV mode annunciators. I know at this airline that pilot understanding of the available 

VNAV modes and their annunciations is a bit of a weak point. By adding more to that portion of 

the FMA annunciations the confusion may increase.” 

6.3.3 Target Aircraft Depiction

Subject pilot responses were overwhelmingly in favor of the “coupled with data block” option, 

with twenty-four (96%) preferring this option and one (4%) preferring the “coupled only” option. 

In examining the ratings on target aircraft characteristics discussed previously, this result can 

largely be attributed to one characteristic of the preferred option, which was the selected circle 

drawn around the active spacing aircraft. This is echoed in comments such as, “I want to know 

which aircraft is controlling my speed...the circle is a BIG HELP”, and, “I prefer the circle with a 

minimal data block adjacent to the aircraft depiction to include A/C number, type, and runway 

assignment until Final mode where the info on the ND should no longer be depicted.”. The one 

subject pilot who preferred the “coupled only” option offered, “I like the circle around it, but this 

is not needed info in most cases.” 

6.3.4 Trend Indication

Subject pilot responses were in favor of the conformance box trend indicator display option, with 

seventeen (68%) preferring this option, five (20%) preferring the deviation scale option, and 

three (12%) preferring no trend indicator. Usability ratings for these options were equal, and the 

helpfulness rating slightly favored the conformance box. Written comments were helpful in 

understanding the more dramatic preferences expressed in this result. Several comments were 

made calling the conformance box “intuitive”, while an equal number of comments labeled the 

deviation scale “confusing”, largely for its similarity to the VNAV path indicator on the opposite 

side of the ND. One commenter said, “My preference is Conformance Box over Deviation Scale. 

Deviation Scale looks too much like VDEV (sic) scale and could be confused. Conformance Box 

is also along track and easy to interpret.” Of the pilots preferring the deviation scale, one 

commented, “...prefer the side deviation indicator, as changes more evident. Probably due to 

constant scale presentation.” Of the three pilots who preferred no trend indication, comments 

included “...provides information I have no control over...”, “...slightly useful but not 

necessary...”, and “...tough to judge because the deviations were so minor”. 
Page | 29 

6.3.5 Other Preference Considerations

The one subject pilot who changed preferences when considering the design elements in 

combination rather than individually responded with an individual preference for the FMA 

option of PDS mode annunciation, but preferred the text option when considered in combination. 

The motivation appeared to be based on ambivalence for the PDS mode annunciation options, 

coupled with a preference to keep both PDS target speed and mode displays together on the PFD. 

6.4 Subject Pilot Preferences for Combinations of Interfaces

In the post-run questionnaire, subject pilots evaluated the presentation of each spacing scenario’s 

design elements for their overall understandability, the level of confidence they provided, and for 

their intuitiveness and user-friendliness. This data was analyzed to determine if the results 

coincided with the individual usability ratings and preferences for the design element options, 

and to see if further insights could be gained about the effectiveness of element combinations. 

Understandability, confidence and intuitiveness were rated using a scale of 5 “very 

understandable, confident, intuitive” to 1 “not understandable, not confident, not intuitive”. The 

data collected for these questions is shown in Table 10-24 through Table 10-26. A summary of 

these results is also shown in Table 6-1, which facilitates a direct comparison of the average 

rating per run for each question, along with an overall rank order of each run’s average rating. 

The “Combined Rating” column of this table combines the weighted responses for each 

question’s responses from Table 10-24 through Table 10-26 into a single average rating. The 

runs are rank-ordered from top to bottom according to the combined rating. The design elements 

presented during each run are displayed as icons for reference.

The results indicate a consistent rank ordering of each run’s average rating from top to bottom. It 

should be noted that the runs were presented in a different order for each subject pilot, so this is 

likely not the result of a bias introduced by the order of presentation. Also, since these were postrun evaluations, it was not a post-experiment determination of the subject pilot’s preferred 

combination of elements. Both the average ratings and the combined ratings show that runs 

which displayed either of the trend indicator options (R5, R6, R7, and R8) scored consistently 

higher than those without a trend indicator (R1, R2, R3, and R4). Further, of the runs which 

displayed a trend indicator, those displaying the “coupled with data block” option of target

aircraft display scored highest for both the mean responses and the combined rating (R7 and R8). 

The results seem to indicate that the design elements presented on the ND drove the rating 

criteria of understandability, confidence and intuitiveness. Subject pilots may have interpreted 

these criteria as primarily strategic in nature, and rated based on the element combination’s 

ability to provide the “big picture”. A contributing factor may have been that the design element 

categories which garnered the highest individual preference scores (target aircraft and trend 

display) preempted consideration of the other categories (PDS target speed and PDS mode), for 

which preferences were not as clearly delineated. Another possibility is that subject pilots may 

have adopted a preferred set of design elements during the pre-experiment briefing, and 

answered the post-run evaluations accordingly. 
Page | 30 

Table 6-1: Combined Design Element Preferences. 

 

6.5 Suggested Improvements by Subject Pilots

6.5.1 PDS Target Speed Interface

Several written comments suggested that a combination of the green speed bug and textual 

modes would be better, and that both options of PDS target speed display could be useful, 

depending on the instrument scan pattern demanded by a particular phase of flight. A mockup of 

one subject pilot’s suggestion for the combination of the two options is shown in Figure 6-5, in 

which the textual presentation of PDS target speed ha been relocated to the area above the speed 

tape, juxtaposing it with the commanded speed text. This concept facilitates easy comparison of 

the two speeds. 
Page | 31 

It was also suggested by some that, while generally important during APS-DPA operations, PDS 

target speed (as well as other PDS design elements) may be an unnecessary distraction during 

final approach, when spacing deviations are no longer being countered by PDS target speed. 

Figure 6-5: Suggested Concept for PDS Target Speed Display. 

6.5.2 PDS Mode Annunciation

Several subject pilots commented that the addition of the PDS information to the FMA was 

“confusing”. Some implementation anomalies in the ASTOR depiction of this option may have 

contributed to the confusion (see Appendix E: ASTOR Implementation Issues). However, 

several subject pilots commented that the FMA information is vital to understanding the 

operation of modern Boeing automation, and therefore this was the rightful place for PDS mode 

annunciation. In retrospect, it may have been less confusing if, as some suggested, the PDS 

annunciation were added by introducing a new segment to the FMA. This would have had the 

advantage of not changing the pilot’s understanding of any existing segment, and would have 

isolated any additional training to that required for the new PDS segment. 

6.5.3 Target Aircraft Depiction

In examining the comments made, it is clear that the identification of the active spacing aircraft 

with a select circle was perceived as enhancing the situation awareness during the spacing 

operation. There were several suggested improvements to enhancing the awareness of the active 

spacing aircraft in the event of a change of active aircraft. These suggestions included: 
Page | 32 

1) “blinking” of the circle, 2) changing the color of the circle, and 3) changing the text color of 

the ID of the active spacing aircraft to magenta.

Several comments were made regarding the efficacy of the data block. Comments supporting the 

data block information were offset by comments recommending: 1) less information in the data 

block, 2) allowing the data block to be selectively turned on or off, and, 3) automatically turning 

the data block off on final approach to eliminate it as a distraction. It was also noted that the data 

block as implemented could conflict with the display of Navigation Aid (NAVAID) information, 

which is selected via the EFIS control panel. 

6.5.4 Trend Indication

Several comments were made about the deviation scale option of the trend indicator being 

confusing. Most cited the scales similarity to the VNAV deviation scale on the opposite side of 

the ND, which would also be displayed while flying a CDA profile. No suggestions for 

improvement were made by the commenters, but the essence of the comments were used to 

propose a mockup of an alternative deviation scale shown in Figure 6-6. In this concept, the scale 

pointer indicating the relative spacing deviation from nominal has been replaced with a small, 

green ownship-like triangle indicator. The original sense of the scale has been reversed, such that 

negative deviations are up or forward on the scale, and positive deviations are down or aft, 

improving the scale’s appearance as a longitudinal deviation indicator. 

Figure 6-6: Alternate Trend Indicator Concept. 
Page | 33 

7 Conclusions

This study built upon previous experiments focused on the Airborne Precision Spacing concept. 

Several candidate interfaces were designed to support spacing operations for aircraft performing 

CDAs while also managing assigned spacing intervals from two other arriving aircraft on 

dependent parallel approaches. An experiment was designed to verify the usability of these 

interfaces using recorded spacing scenarios, which were implemented using an existing 

procedure simulator (ASTOR). 

The results of the study indicate that several options exist for usable interfaces to support these 

spacing operations. All of the interface options evaluated by subject pilots were considered 

acceptable, and valuable insight was collected on their relative effectiveness and importance to 

the conduct of spacing operations. In particular, the key results indicate the following:

1. both textual and graphical presentations of pds target speed were considered useful; 

2. target aircraft indication is considered especially useful if it also indicates which of the 

two targets is the active spacing aircraft;

3. subject pilots favored the flight mode annunciator option over the text option for pds 

mode annunciation, which should be considered carefully since it would likely present a 

more significant pilot training issue than the text option; and

4. spacing trend indication was considered the least important design element, yet it was 

also considered useful by most subject pilots. 

The results also highlighted the need for additional investigation. The evaluations of options for 

presenting PDS target speed indicated equal preference, and suggestions to display both on the 

PFD (similar to the commanded speed) have merit. A more thorough consideration of the 

integration of PDS annunciation into the FMA is recommended, as most subject pilots preferred 

this for monitoring autopilot controlled modes of operation, and there were issues noted with the 

study’s implementation. The actual benefit of the data block containing target aircraft 

information may have been masked by its association with the active spacing aircraft select

circle, but sufficient support existed for the refinement of data block information to be 

investigated. Future studies should characterize the behavior of the trend indicators using a 

variety of spacing scenarios, including scenarios with less benign conditions than used in this 

experiment. The responses for the trend indicator showed it to be lowest in relative importance, 

yet a trend indicator was always included in preferred combinations of elements. This result 

warrants further investigation, especially with off-nominal scenarios, of whether a trend 

indicator is imparting useful information during a spacing operation, or if off-nominal alerts of 

another form (EICAS messages, audio alerts, etc.) would be sufficient. 

Finally, the ASTOR implementation used to create the spacing scenarios leaves a legacy of 

usable software which can easily be used to support future studies or demonstrations. Many of 

the discussed suggestions for improvement of the interfaces involve simple software 

development and/or “bug” fixes. 
Page | 34 

8 Acknowledgements

The authors and Adaptive Aerospace Group, LLC would like to thank the following individuals 

for their guidance, helpful discussions, and support of our work: Mr. Terry Abbott (Booz Allen 

Hamilton), Dr. Bryan Barmore (NASA LaRC) and Capt. Richard F. Shay (United Airlines) for 

lending their technical expertise on the development of airborne spacing concepts; James P. 

Chamberlain (NASA LaRC), Dr. Jennifer L. Murdoch (NASA LaRC) and Dr. Sherri Ann Rehfeld 

(NASA LaRC), who served on the experiment review board, for providing helpful guidance on

the design and implementation of the usability study; and Joseph R. King (Sensis Corporation) 

and Michael T. Palmer (NASA LaRC) for their assistance in both modifying the ASTOR platform 

and creating the spacing scenarios used for the study. 
Page | 35 

9 References

[1] Abbott, T. 2002 : Speed Control Law for Precision Terminal Area In-Trail Self Spacing, 

NASA/TM-2002-211742. 

[2] Barmore, B. 2006 : Airborne Precision Spacing: A Trajectory-Based Approach to Improve 

Terminal Area Operations, 25th Digital Avionics Systems Conference. 

[3] Barmore, B.; Abbott, T.; and Krishnamurthy, K. 2004: Airborne-Managed Spacing in Multiple 

Arrival Streams, 24th International Congress of the Aeronautical Sciences. 

[4] Oseguera-Lohr, R.; Lohr, G.; Abbott, T .; and Eischeid, T. 20 02: Evaluation of Operational 

Procedures for Using a Time-Based Airborne Interarrival Spacing Tool, AIAA-2002-5824. 

[5] Murdoch, J.; Barmore, B.; Baxley, B.; Abbott, T .; and C apron, W. 2009 : Evaluation of an 

Airborne Spacing Concept to Support Continuous Descent Arrival Operations, Eighth 

USA/Europe Air Traffic Management Research and Development Seminar (ATM2009). 

[6] EuroControl Experimental Centre 2006: Flight Deck User Requirements for Airborne Spacing 

(Sequencing and Merging) Volume 1, Project EVP/WP3. 

[7] Barmore, B.; Baxley, B.; Abbott, T.; Capron, W.; Shay, R.; and Hubbs,C. 2012: A Concept for 

Airborne Precision Spacing for Dependent Parallel Approaches, NASA/TM-2012-217346. 

[8] RTCA 2009: Minimum Operational Performance Standards for Aircraft Surveillance 

Applications System, RTCA/DO-317 

[9] RTCA 2007: Aircraft Surveillance Applications Minimum Aviations System Performance 

Standards, RTCA/DO-289. 

[10] FAA 2003: Guidelines for the Certification, Airworthiness, and Operational Approval of 

Electronic Flight Bag Computing Devices, AC120-76A. 
Page | 36 

10 Appendix A: Tabulated Experimental Results

10.1 Results of Post-Run Questionnaires

This is the response data from the post-run questionnaire for the eight (8) test runs R1-

R8. This questionnaire was structured to gather responses regarding design element 

usability and acceptability (i.e., were they understandable, helpful, intuitive, confidenceinspiring). Questions were asked which explored the usability of individual design 

elements and their characteristics, and also for combinations of elements. Except for 

solicited additional comments, responses were collected using Likert-type scales.

The text of each question precedes a set of tables showing the results. The data collected 

for each question is tabulated to make it easier to compare the results from individual test 

runs. Refer to Table 10-1 for a matrix which shows which display elements were 

presented to the subject during each run. Table data is organized by grouping runs which 

displayed a particular option. A summary entry identifies the design option, the overall 

ratings and sample size. Note that the “Rating Average” is the weighted average of the 

responses based on the values shown in the “Weight” row in the table, the “SD” is the 

population standard deviation, and “N” is the sample size, which is the number of valid 

responses collected for the particular run. 

Table 10-1: Test Run Matrix Showing Display Options Enabled 

Run PDS Target Speed 

Option

PDS Mode 

Option

Target Aircraft 

Option

Trend Indicator 

Option

R1 Text Text Coupled Only None

R2 Text FMA Coupled Only None

R3 Speed Bug Text Coupled Only None

R4 Speed Bug FMA Coupled Only None

R5 Text Text Coupled Only Conformance Box

R6 Text Text Coupled Only Deviation Scale

R7 Text Text Coupled w/Data Conformance Box

R8 Text Text Coupled w/Data Deviation Scale

10.1.1 Usability of Display Elements

The following post-run questionnaire responses rated the subject’s ability to understand 

the information being presented by the design elements, which was considered a key 

component of usability. Questions about PDS target speed, PDS mode annunciation and 

target aircraft depiction were sub-divided according to characteristics of the information 

presented, i.e., for PDS target speed, both current target speed and changes to target 

speed were rated. The runs are grouped by the design element option presented during the 

run. An overall rating for the design element option is shown in a summary line of the 

table. The results are discussed in more detail in 6.1. 
Page | 37 

“Based on what you experienced during the scenario you just completed, please rate your 

ability to identify the following information regarding the spacing operation.” 

10.1.1.1 PDS Target Speed

Table 10-2: Current Target Speed

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A Rating Avg. N SD

Wt. (6) (5) (4) (3) (2) (1)

Text Option Ratings

R1 11 12 2 0 0 0 0 5.36 25 0.62

R2 13 9 3 0 0 0 0 5.40 25 0.69

R5 10 13 2 0 0 0 0 5.32 25 0.61

R6 13 10 2 0 0 0 0 5.44 25 0.64

R7 14 8 3 0 0 0 0 5.44 25 0.70

R8 13 10 2 0 0 0 0 5.44 25 0.64

Total 58 60 31 1 0 0 0 5.40 150 0.65

Speed Bug Option Ratings

R3 12 9 3 0 1 0 0 5.24 25 0.95

R4 12 7 5 1 0 0 0 5.20 25 0.89

Total 24 15 9 0 1 1 0 5.22 50 0.92

Table 10-3: Changes to Target Speed

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating 

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1) 

Text Option Ratings

R1 7 8 10 0 0 0 0 4.88 25 0.82

R2 10 6 8 1 0 0 0 5.00 25 0.94

R5 7 11 6 1 0 0 0 4.96 25 0.82

R6 8 9 7 1 0 0 0 4.96 25 0.87

R7 10 9 6 0 0 0 0 5.16 25 0.78

R8 12 8 3 2 0 0 0 5.20 25 0.94

Total 74 62 14 0 0 0 0 5.40 150 0.65

Speed Bug Option Ratings

R3 10 9 5 0 1 0 0 5.08 25 0.98

R4 10 7 5 2 1 0 0 4.92 25 1.13

Total 20 16 10 2 2 0 0 5.00 50 1.06
Page | 38 

10.1.1.2 PDS Mode

Table 10-4: Current Mode 

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating 

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1) 

Text Option Ratings

R1 8 12 5 0 0 0 0 5.12 25 0.71

R3 8 8 9 0 0 0 0 4.96 25 0.82

R5 10 13 2 0 0 0 0 5.32 25 0.61

R6 11 10 3 1 0 0 0 5.24 25 0.81

R7 9 9 7 0 0 0 0 5.08 25 0.80

R8 12 8 5 0 0 0 0 5.28 25 0.78

Total 58 60 31 1 0 0 0 5.17 150 0.77

FMA Option Ratings

R2 12 9 4 0 0 0 0 5.32 25 0.73

R4 12 6 5 0 1 1 0 5.00 25 1.30

Total 24 15 9 0 1 1 0 5.16 50 1.07

Table 10-5: Changes to Mode

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating 

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1) (0) 

Text Option Ratings

R1 5 10 6 3 0 0 1 4.52 24 0.93

R3 5 8 10 2 0 0 0 4.64 25 0.89

R5 7 9 9 0 0 0 0 4.92 25 0.80

R6 7 10 7 1 0 0 0 4.92 25 0.84

R7 7 9 9 0 0 0 0 4.92 25 0.80

R8 8 8 7 2 0 0 0 4.88 25 0.95

Total 39 54 48 8 0 0 1 4.83 149 0.88

FMA Option Ratings

R2 9 10 6 0 0 0 0 5.12 25 0.77

R4 9 8 5 1 1 1 0 4.80 25 1.30

Total 18 18 11 1 1 1 0 4.96 50 1.08
Page | 39 

Table 10-6: Next Mode in Sequence 

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating 

Avg. N SD

 Wt. (6) (5) (4) (3) (2) (1)

Text Option Ratings

R1 3 3 3 1 0 10 5 2.90 20 2.02

R3 3 1 5 3 1 8 4 2.95 21 1.81

R5 5 2 3 1 0 8 6 3.32 19 2.13

R6 4 6 2 2 0 5 6 3.84 19 1.90

R7 4 4 2 0 0 11 4 3.00 21 2.16

R8 4 3 4 2 0 7 5 3.40 20 1.96

Total 23 19 19 9 1 49 30 3.23 120 2.03

FMA Option Ratings

R2 10 9 1 1 0 3 1 4.79 24 1.61

R4 7 11 1 2 0 1 3 4.91 22 1.20

Total 17 20 2 3 0 4 4 4.85 46 1.43

10.1.1.3 Target Aircraft: 

Table 10-7: How Many (one or two)

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating 

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1) 

Coupled Only Option Ratings

R0 2 4 2 0 0 0 17 5.00 8 0.71

R1 7 15 3 0 0 0 0 5.16 25 0.61

R2 9 12 4 0 0 0 0 5.20 25 0.69

R3 9 13 3 0 0 0 0 5.24 25 0.65

R4 9 14 2 0 0 0 0 5.28 25 0.60

R5 12 11 2 0 0 0 0 5.40 25 0.63

Total 58 73 19 0 0 0 0 5.26 150 0.67

Coupled w/ Data Block Option Ratings

R7 13 11 1 0 0 0 0 5.48 25 0.57

R8 16 9 0 0 0 0 0 5.64 25 0.48

Total 29 20 1 0 0 0 0 5.56 50 0.54
Page | 40 

Table 10-8: Active Spacing Aircraft

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating 

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1)

Coupled Only Option Ratings

R1 2 5 6 8 2 1 1 3.75 24 1.23

R2 3 6 10 3 2 1 0 4.08 25 1.23

R3 4 7 8 6 0 0 0 4.36 25 1.02

R4 3 7 11 4 0 0 0 4.36 25 0.89

R5 4 8 6 6 0 1 0 4.28 25 1.22

R6 2 6 10 6 0 1 0 4.04 25 1.08

Total 18 39 51 33 4 4 1 4.15 149 1.14

Coupled w/ Data Block Option Ratings

R7 20 4 1 0 0 0 0 5.76 25 0.51

R8 20 4 0 1 0 0 0 5.72 25 0.66

Total 40 8 1 1 0 0 0 5.74 50 0.59

Table 10-9: Ownship Position Relative to Spacing Aircraft 

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1) 

Coupled Only Option Ratings

R1 5 8 8 1 2 1 0 4.40 25 1.30

R2 4 8 9 3 1 0 0 4.44 25 1.02

R3 8 4 8 4 1 0 0 4.56 25 1.20

R4 5 9 6 4 1 0 0 4.52 25 1.10

R5 9 7 5 2 2 0 0 4.76 25 1.24

R6 4 10 10 1 0 0 0 4.68 25 0.79

Total 35 46 46 15 7 1 0 4.56 150 1.13

Coupled w/ Data Block Option Ratings

R7 17 7 1 0 0 0 0 5.64 25 0.56

R8 17 6 2 0 0 0 0 5.60 25 0.63

Total 34 13 3 0 0 0 0 5.62 50 0.60
Page | 41 

Table 10-10: Aircraft ID(s)

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1)

Coupled Only Option Ratings

R1 6 5 9 2 1 2 0 4.28 25 1.43

R2 4 11 7 1 1 1 0 4.52 25 1.17

R3 7 8 6 2 1 1 0 4.60 25 1.30

R4 2 9 11 0 1 2 0 4.20 25 1.23

R5 6 6 10 2 1 0 0 4.56 25 1.06

R6 3 8 11 1 1 1 0 4.32 25 1.12

Total 28 47 54 8 6 7 0 4.41 150 1.23

Coupled w/ Data Block Option Ratings

R7 15 7 3 0 0 0 0 5.48 25 0.70

R8 18 6 1 0 0 0 0 5.68 25 0.55

Total 33 13 4 0 0 0 0 5.58 50 0.64

Table 10-11: Spacing Interval Type: NASA 893

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1) 

Coupled Only Option Ratings

R1 2 5 13 3 1 1 0 4.04 25 1.08

R2 4 2 12 4 3 0 0 4.00 25 1.17

R3 2 7 13 1 1 1 0 4.20 25 1.06

R4 0 6 15 1 1 2 0 3.88 25 1.07

R5 2 7 8 6 2 0 0 4.04 25 1.08

R6 1 8 11 3 1 1 0 4.08 25 1.06

Total 11 35 72 18 9 5 0 4.04 150 1.09

Coupled w/ Data Block Option Ratings

R7 9 11 4 1 0 0 0 5.12 25 0.82

R8 7 13 4 1 0 0 0 5.04 25 0.77

Total 16 24 8 2 0 0 0 5.08 50 0.80
Page | 42 

Table 10-12: Spacing Interval Type: NASA 859 

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1) 

Coupled Only Option Ratings

R1 1 4 14 3 1 2 0 3.80 25 1.13

R2 4 2 12 3 4 0 0 3.96 25 1.22

R3 2 7 13 1 1 1 0 4.20 25 1.06

R4 0 6 15 1 1 2 0 3.88 25 1.07

R5 2 7 8 5 2 1 0 3.96 25 1.22

R6 1 6 13 3 1 1 0 4.00 25 1.02

Total 10 32 75 16 10 7 0 3.97 150 1.13

Coupled w/ Data Block Option Ratings

R7 8 10 6 1 0 0 0 5.00 25 0.85

R8 7 11 6 1 0 0 0 4.96 25 0.82

Total 15 21 12 2 0 0 0 4.98 50 0.84

Table 10-13: Spacing Deviation: NASA 893

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1) 

Coupled Only Option Ratings

R1 0 3 14 4 3 1 0 3.60 25 0.98

R2 1 3 11 3 6 1 0 3.48 25 1.20

R3 1 5 10 5 2 2 0 3.68 25 1.22

R4 0 5 14 3 3 0 0 3.84 25 0.88

R5 3 7 11 1 3 0 0 4.24 25 1.11

R6 1 9 8 4 2 1 0 4.00 25 1.17

Total 6 32 68 20 19 5 0 3.81 150 1.13

Coupled w/ Data Block Option Ratings

R7 11 9 4 1 0 0 0 5.20 25 0.85

R8 5 12 6 1 1 0 0 4.76 25 0.95

Total 16 21 10 2 1 0 0 4.98 50 0.93
Page | 43 

Table 10-14: Spacing Deviation: NASA 859

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1)

Coupled Only Option Ratings

R1 0 3 14 4 3 1 0 3.60 25 0.98

R2 1 3 12 3 5 1 0 3.56 25 1.17

R3 1 5 10 5 2 2 0 3.68 25 1.22

R4 0 5 13 3 3 1 0 3.72 25 1.04

R5 3 6 10 2 3 1 0 4.04 25 1.28

R6 1 8 10 4 2 0 0 4.08 25 0.98

Total 6 30 69 21 18 6 0 3.78 150 1.14

Coupled w/ Data Block Option Ratings

R7 9 9 5 1 1 0 0 4.96 25 1.04

R8 5 10 7 2 1 0 0 4.64 25 1.02

Total 14 19 12 3 2 0 0 4.80 50 1.04

Table 10-15: Assigned Runway: NASA 893

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1) 

Coupled Only Option Ratings

R1 1 1 8 5 4 5 1 2.96 24 1.37

R2 1 4 3 8 3 5 1 3.04 24 1.46

R3 2 1 4 7 4 6 1 2.83 24 1.49

R4 1 1 8 5 3 6 1 2.92 24 1.41

R5 2 1 7 7 2 5 1 3.13 24 1.45

R6 1 5 2 8 3 4 2 3.17 23 1.46

Total 8 13 32 40 19 31 7 3.01 143 1.45

Coupled w/ Data Block Option Ratings

R7 12 9 3 0 0 1 0 5.20 25 1.10

R8 12 8 2 3 0 0 0 5.16 25 1.01

Total 24 17 5 3 0 1 0 5.18 50 1.05
Page | 44 

Table 10-16: Assigned Runway: NASA 859

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1)

Coupled Only Option Ratings

R1 0 1 6 5 5 7 1 2.54 24 1.26

R2 1 3 2 9 3 6 1 2.83 24 1.43

R3 1 1 4 7 4 7 1 2.63 24 1.38

R4 0 1 5 7 4 7 1 2.54 24 1.22

R5 0 1 4 10 2 7 1 2.58 24 1.19

R6 1 1 2 10 3 6 2 2.65 23 1.31

Total 3 8 23 48 21 40 7 2.63 143 1.30

Coupled w/ Data Block Option Ratings

R7 10 10 1 3 0 1 0 4.96 25 1.25

R8 11 7 3 2 1 1 0 4.88 25 1.37

Total 21 17 4 5 1 2 0 4.92 50 1.31

Table 10-17: Aircraft Type (i.e., 757, hvy, etc.): NASA 893 

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1) 

Coupled Only Option Ratings

R1 0 1 2 7 1 10 4 2.19 21 1.26

R2 0 0 2 7 4 9 3 2.09 22 1.04

R3 0 0 2 6 0 14 3 1.82 22 1.11

R4 0 0 2 4 2 14 3 1.73 22 1.05

R5 0 1 2 7 1 11 3 2.14 22 1.25

R6 0 1 1 8 0 10 5 2.15 20 1.24

Total 0 3 11 39 8 68 21 2.02 129 1.17

Coupled w/ Data Block Option Ratings

R7 13 8 2 1 0 1 0 5.20 25 1.17

R8 13 7 3 1 0 1 0 5.16 25 1.19

Total 26 15 5 2 0 2 0 5.18 50 1.18
Page | 45 

Table 10-18 Aircraft Type (i.e., 757, hvy, etc.): NASA 859 

Run Very

Easy 

Easy IdentifiableNot

Easy Difficult NotAble N/A

Rating

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1)

Coupled Only Option Ratings

R1 0 0 2 6 3 10 4 2.00 21 1.07

R2 0 0 2 7 4 9 3 2.09 22 1.04

R3 0 0 2 6 0 14 3 1.82 22 1.11

R4 0 0 2 4 2 14 3 1.73 22 1.05

R5 0 1 1 8 1 11 3 2.09 22 1.20

R6 0 0 1 9 0 10 5 2.05 20 1.07

Total 0 1 10 40 10 68 21 1.96 129 1.10

Coupled w/ Data Block Option Ratings

R7 11 8 2 3 0 1 0 4.96 25 1.28

R8 11 6 4 1 0 3 0 4.72 25 1.61

Total 22 14 6 4 0 4 0 4.84 50 1.46
Page | 46 

10.1.1.4 Trend Indication

“Based on what you experienced during the scenario you just completed, please rate your 

ability to determine and understand the aircraft’s progress toward achieving the 

assigned spacing goal.” 

Table 10-19: Trend Indication

Run

Very

Easy

to

Understand

Easy

To

Understand

Understandable

Not

Easy

to

Understand

Difficult

to

Understand

Not

Able

to

Understand

Rating

Avg N SD

Wt. (6) (5) (4) (3) (2) (1)

No trend indicator ratings

R1 1 5 14 5 0 0 4.08 25 0.74

R2 1 5 17 1 1 0 4.16 25 0.73

R3 1 7 14 2 1 0 4.20 25 0.80

R4 1 7 16 0 1 0 4.28 25 0.72

Total 4 24 61 8 3 0 4.18 100 0.75

Conformance Box option ratings

R5 0 15 10 0 0 0 4.60 25 0.49

R7 6 15 4 0 0 0 5.08 25 0.63

Total 6 30 14 0 0 0 4.84 50 0.61

Deviation Scale option ratings

R6 0 14 9 2 0 0 4.48 25 0.64

R8 9 12 4 0 0 0 5.20 25 0.69

Total 9 26 13 2 0 0 4.84 50 0.76

The following post-run questionnaire responses rated the subject’s opinion on the 

helpfulness of the information being presented by the design element, which was 

considered a key component of usability. The runs are grouped by the design element 

option presented during the run. An overall rating for the design element option is shown 

in a summary line of the table. Note that a trend indicator was not displayed in runs R1, 

R2, R3, and R4, so the results are not shown. The results are discussed in more detail in

6.1. 

“Using the rating scales, please rate the helpfulness of each element in understanding 

the spacing operation:” 
Page | 47 

Table 10-20: PDS Target Speed Indicator Helpfulness

Run 

Very

Helpful 

HelpfulMod. 

Helpful

Little

Help

Not

Helpful 

Avg.

Rating N SD

Wt. (5) (4) (3) (2) (1)

Text option ratings

R1 9 12 4 0 0 4.20 25 0.69

R2 8 13 4 0 0 4.16 25 0.67

R5 6 18 1 0 0 4.20 25 0.49

R6 8 12 3 0 0 4.22 23 0.66

R7 8 15 1 0 0 4.29 24 0.54

R8 12 9 3 1 0 4.28 25 0.83

Total 51 79 16 1 0 4.22 147 0.66

Speed Bug Option Ratings

R3 10 6 7 0 1 4.00 24 1.04

R4 11 9 3 2 0 4.16 25 0.92

Total 21 15 10 2 1 4.08 49 0.99

Table 10-21: PDS Mode Annunciation Helpfulness 

Run 

Very

Helpful 

HelpfulMod. 

Helpful

Little

Help

Not

Helpful 

Avg.

Rating N SD

Wt. (5) (4) (3) (2) (1) 

Text option ratings

R1 5 13 7 0 0 3.92 25 0.69

R3 4 11 7 2 0 3.71 24 0.84

R5 7 14 4 0 0 4.12 25 0.65

R6 6 13 4 0 0 4.09 23 0.65

R7 5 16 2 1 0 4.04 24 0.68

R8 7 15 2 1 0 4.12 25 0.71

Total 34 82 26 4 0 4.00 146 0.72

FMA Option Ratings

R2 8 10 5 2 0 3.96 25 0.92

R4 8 12 4 0 1 4.04 25 0.92

Total 16 22 9 2 1 4.00 50 0.92
Page | 48 

Table 10-22: Target Aircraft Indicator Helpfulness

Run 

Very

Helpful 

HelpfulMod. 

Helpful

Little

Help

Not

Helpful 

Avg.

Rating N SD

Wt. (5) (4) (3) (2) (1)

Coupled Only Option Ratings

R1 5 15 2 3 0 3.88 25 0.86

R2 4 15 3 3 0 3.80 25 0.85

R3 2 15 4 3 0 3.67 24 0.80

R4 8 12 3 2 0 4.04 25 0.87

R5 4 18 3 0 0 4.04 25 0.53

R6 6 15 1 1 0 4.13 23 0.68

Total 29 90 16 12 0 3.93 147 0.79

Coupled w/Data Block Option Ratings

R7 18 6 0 0 0 4.75 24 0.43

R8 23 1 1 0 0 4.88 25 0.43

Total 41 7 1 0 0 4.82 49 0.44

Table 10-23: Trend Indicator Helpfulness

Run 

Very

Helpful 

HelpfulMod. 

Helpful

Little

Help

Not

Helpful 

Avg.

Rating N SD

Wt. (5) (4) (3) (2) (1) 

Conformance Box Option Ratings

R5 9 7 5 1 3 3.72 25 1.31

R7 12 8 3 0 1 4.25 24 0.97

Total 21 15 8 1 4 3.98 49 1.19

Deviation Scale Option Ratings

R6 4 8 5 3 3 3.30 23 1.27

R8 7 8 5 4 1 3.64 25 1.16

Total 11 16 10 7 4 3.48 48 1.22

10.1.2 Preference for Combinations of Display Elements

The following post-run questionnaire responses rate the subject’s preferences for 

combinations of the design elements presented in the categories of overall understanding, 

confidence and intuitiveness. The tabulated results, shown in rank order of rating average 

each run, illustrate which combinations of elements the subjects preferred in each 

category. The significance of the shaded areas pertains to significant differentiations in 

the elements enabled during the run (see Table 10-1). The gray shaded runs significant 

differentiation from the other runs is that they displayed both the “Coupled w/ Data 

Block” target aircraft depiction and one of the trend indicators (conformance box or 
Page | 49 

deviation scale). The tan shaded runs significant differentiation from the other runs is that 

they displayed both the “Coupled Only” target aircraft depiction and one of the trend 

indicators (conformance box or deviation scale).The unshaded runs significant 

differentiation is that they displayed no trend indicator. This differentiation is also 

illustrated in Table 6-1, and preferred combinations of results are also discussed in 6.4

(Subject Pilot Preferences for Combinations of Interfaces).

“Please rate your overall understanding of what was happening during the APS-DPA 

operation.” 

Table 10-24: Overall Understanding 

Run

Very

Easy to 

Understand

Easy to 

Understand

Able to 

Understand

Not 

Easy to 

Understand

Difficult 

to 

Understand

Not 

Able to 

Understand

Rating 

Avg. N SD

Wt. (6) (5) (4) (3) (2) (1)

Coupled w/ Data Block & Trend Indicator

R8 10 12 3 0 0 0 5.28 25 0.66

R7 7 15 3 0 0 0 5.16 25 0.61

Coupled Only & Trend Indicator

R5 0 20 5 0 0 0 4.80 25 0.40

R6 1 16 6 2 0 0 4.64 25 0.69

No trend indicator

R4 2 11 11 0 1 0 4.52 25 0.81

R3 2 10 11 2 0 0 4.48 25 0.75

R1 1 11 8 5 0 0 4.32 25 0.84

R2 1 10 11 2 1 0 4.32 25 0.84
Page | 50 

“Please rate your level of confidence in the elements presented in the scenario if this 

were an actual APS-DPA operation.” 

Table 10-25: Overall Confidence Level

Run

Very

Confident

Confident

Somewhat 

Confident

Not

Very 

Confident

Not 

Confident 

at All

Rating 

Avg. N SD

Wt. (5) (4) (3) (2) (1)

Coupled w/ Data Block & Trend Indicator

R7 9 14 2 0 0 4.28 25 0.60

R8 9 12 4 0 0 4.20 25 0.69

Coupled Only & Trend Indicator

R5 0 20 5 0 0 3.80 25 0.40

R6 0 19 6 0 0 3.76 25 0.43

No trend indicator

R4 2 14 7 2 0 3.64 25 0.74

R2 1 15 7 2 0 3.60 25 0.69

R3 2 13 8 2 0 3.60 25 0.75

R1 0 12 11 2 0 3.40 25 0.63

“Please give an overall rating of the intuitiveness and user-friendliness of the 

combination of the display elements presented during the scenario you just completed:” 

Table 10-26: Overall Intuitiveness

Run

Very 

Intuitive 

Intuitive 

Moderately

Intuitive

Not

Very Intuitive 

Not at 

All Intuitive 

Rating 

Avg. N SD

Wt. (5) (4) (3) (2) (1)

Coupled w/ Data Block & Trend Indicator

R7 10 13 1 0 0 4.38 24 0.56

R8 9 12 4 0 0 4.20 25 0.69

Coupled Only & Trend Indicator

R5 2 19 4 0 0 3.92 25 0.48

R6 0 16 5 2 0 3.61 23 0.64

No trend indicator

R4 2 12 10 1 0 3.60 25 0.69

R3 2 12 7 3 0 3.54 24 0.82

R2 1 11 10 3 0 3.40 25 0.75

R1 0 13 8 4 0 3.36 25 0.74
Page | 51 

10.1.3 Comments and Suggestions for Improvement 

The written responses made by subject pilots in the post-run questionnaires are discussed 

in Section 6.5. 

10.2 Results of Final Questionnaire

This is the response data from the final questionnaires completed by each of the subjects 

at the completion of the eight (8) experimental runs R1-R8. The final questionnaire 

differed from the post-run in its focus, with questions designed to solicit overall 

impressions of design element importance, individual element preferences, and 

suggestions for improvement of the elements. A variety of responses types were collected 

using Likert-type scales, multiple choice responses and written comments.

10.2.1 Importance of Display Elements 

The following final questionnaire responses explored the subject’s impressions of the 

importance of each category of design element (i.e., PDS target speed, PDS mode 

annunciation, etc.). The tabulated results, shown in rank order by rating average for each 

category or characteristic, illustrate the relative importance of the information which was 

intended to be provided by the design elements during the spacing simulations. The 

results of the importance of display elements responses are discussed in 6.2. 

“Using the rating scales, please indicate which of the following should be required in 

performing the spacing operation:” 

Table 10-27: Required Elements for Spacing Operation 

Element Very

Imp.

Imp. Mod.

Imp.

Little 

Imp.

Not 

Imp.

Rating

Avg.

N SD

Weight (5) (4) (3) (2) (1)

PDS target speed indicator 23 2 0 0 0 4.92 25 0.27

Target aircraft depiction 20 3 2 0 0 4.72 25 0.60

PDS mode annunciation 15 5 4 1 0 4.36 25 0.89

Trend indicator 13 6 4 0 2 4.12 25 1.18
Page | 52 

“Using the rating scales, please rate how important it is to be able to determine the 

following during a spacing operation.” 

Table 10-28: PDS Target Speed Characteristic Importance 

Element

Characteristic

Very

Important

Important

Mod.

Important

Little

Importance

Not

Important

Rating

Avg. N SD

Weight (5) (4) (3) (2) (1)

Current 

target speed 23 2 0 0 0 4.92 25 0.27

Changes to 

target speed 21 3 1 0 0 4.80 25 0.49

Table 10-29: Target Aircraft Display Characteristics Importance

Element

Characteristic

Very

Important

Important

Mod.

Important

Little

Importance

Not

Important

Rating

Avg. N SD

Weight (5) (4) (3) (2) (1) 

Ownship position

relative to

spacing aircraft 22 3 0 0 0 4.88 25 0.32

Active spacing

aircraft 21 4 0 0 0 4.84 25 0.37

How many

(one or two) 18 5 2 0 0 4.64 25 0.62

Spacing

deviation(s) 10 10 3 2 0 4.12 25 0.91

Aircraft ID(s) 7 10 7 1 0 3.92 25 0.84

Assigned

runway(s) 5 11 7 1 1 3.72 25 0.96

Aircraft type

(i.e., 757, hvy, etc.) 5 10 7 3 0 3.68 25 0.93

Spacing Interval type 4 11 6 4 0 3.60 25 0.94
Page | 53 

Table 10-30: PDS Mode Characteristic Importance 

Element

Characteristic

Very

Important

Important

Mod.

Important

Little

Importance

Not

Important

Rating

Avg. N SD

Weight (5) (4) (3) (2) (1)

Current mode 13 6 5 1 0 4.24 25 0.91

Changes to mode 11 10 2 2 0 4.20 25 0.89

Next Mode in 

Sequence 6 7 5 6 1 3.44 25 1.20

10.2.2 Display Element Individual Preferences

In the following final questionnaire responses, the subjects were asked to express a 

preference for design element options in each of the four categories listed (PDS target 

speed, PDS mode annunciation, target aircraft depiction and trend indication.) If subjects 

preferred not to see the design element at all, they could answer “none”. Supporting 

comments about these preferences were captured later in the final questionnaire. The 

results are tabulated in preference order shown below. At the end of this question, 

subjects were asked if their answer would change if considering the design element 

options in combination, rather than individually. 

“Please indicate if you have a preference in each of the four categories of display 

elements listed below:”

Table 10-31: PDS Target Speed Preference

Element 

Option

Response 

Percentage 

Response 

Count

Speed bug 52.0% 13

Text 48.0% 12

None 0.0% 0

Total 100.0% 25

Table 10-32: PDS Mode Annunciation Preference 

Element 

Option

Response 

Percentage 

Response 

Count

FMA 60.0% 15

Text 40.0% 10

None 0.0% 0

Total 100.0% 25
Page | 54 

Table 10-33: Target Aircraft Display Preference

Element 

Option

Response 

Percentage 

Response 

Count

Coupled w/data block 96.0% 24

Coupled only 4.0% 1

None 0.0% 0

Total 100.0% 25

Table 10-34: Trend Indicator Preference

10.2.3 Combined Display Element Preferences

“Do your preferred display elements differ when considered in combination than 

individually?”

Eight (8) of the twenty-five subjects responded “yes” to this question. However, only one 

of the eight actually changed any of the preferred element selections. These responses 

indicate that the question may have been confusing and misinterpreted by at least seven 

of the subjects. The one subject who changed the response for the combined elements did 

so as follows: 

Element

Option

Response

Percentage 

Response

Count

Conformance box 68.0% 17

Deviation scale 20.0% 5

None 12.0% 3

Total 100.0% 25
Page | 55 

Table 10-35 Individual Subject Combined vs. Individual Preferences 

Display 

Element

Individual

Option

Pref.

Combined

Element 

Pref.

PDS Target 

Speed Text Text

PDS Mode 

Annunciation FMA Text

Target

Aircraft 

Depiction

Coupled 

w/Data

Coupled 

w/Data

Trend 

Indicator

Deviation

Scale

Deviation

Scale

10.2.4 Comments and Suggestions for Improvement

The written responses made by subject pilots in the final questionnaire are discussed in 

Section 6.5.
Page | 56 

11 Appendix B: Supplemental Design Concepts for Fully Integrated 

Implementation

11.1 Engine Instrument & Crew Alerting System/Multifunction Display

The Engine Instrument & Crew Alerting System (EICAS/MFD) implements several 

functions that can be adapted to manage a datalinked APS-DPA clearance in the fully 

integrated concept. The interface displays datalinked messages from an Air Traffic 

Service Provider (ATSP). If the message is a clearance, an interface is provided to 

transfer datalinked clearance parameters to an on-board application in the Flight 

Management Computer (FMC), such as ASTAR. A facility is provided for the crew to 

accept or reject the clearance via datalink. The EICAS/MFD is also generally used to 

display information from on-board applications, especially off-nominal conditions. These 

annunciations are in the form of alphanumeric message(s) which contains specific 

information about the condition and its associated priority, often associated with an aural 

indication. 

The ASTOR used for the previous HITL experiment [5] was modified to support the fully 

integrated APS-DPA spacing scenarios used for this study. Simulations were created to 

generate the datalinked clearances for the spacing scenario (see Table 5-1: Spacing 

Scenario Clearance Parameters). The ASTOR was modified to parse the uplinked 

clearance and provide the spacing parameters to ASTAR. The spacing parameters could 

then be viewed and verified via the PDS main menu page in the MCDU (see Section

11.3). Due to time constraints, the off-nominal condition outputs from ASTAR were not 

implemented, and no off-nominal scenarios were generated for the usability study. 

Examples of the ASTOR implementation of the EICAS/MFD for APS-DPA clearance 

management are shown in the following illustrations. Figure 11-1 shows a spacing 

clearance received via CPDLC (Controller Pilot Data Link Clearance) displayed on the 

EICAS/MFD. Note that the length of the clearance (greater than 128 characters) 

necessitated that it be viewed on the MFD, also indicated by the “LARGE ATC 

UPLINK” message on the EICAS.
Page | 57 

Figure 11-1: Example of Spacing Clearance Displays on EICAS/MFD 

11.2 Mode Control Panel

In the proposed concept, the Mode Control Panel (MCP) allowed the crew to control the 

engagement and disengagement of PDS speed guidance to the autopilot. The MCP 

provides control of the autopilot, flight director, altitude alert, and auto-throttle systems, 

and is used to select and activate Autopilot Flight Director System (AFDS) modes, 

establish altitudes, speeds and climb/descent profiles. There were two concepts 

considered for the MCP and its associated control logic: 1) addition of new “PDS” mode 

control select to the MCP, and; 2) modification of MCP control logic to make PDS mode 

a sub-mode of the existing VNAV mode. 

In considering the limited time for training subject pilots prior to the experimental 

session, it was decided that the first concept would be potentially less confusing to 

subject pilots who were already familiar with the operation of the existing VNAV mode 

of the AFDS. As the focus of the experiment was primarily on design elements for the 

Primary Flight and Navigation displays, only the first concept was implemented in the 

modified ASTOR. 

The implementation of the first concept required the straightforward addition of a new 

“PDS” mode to the MCP to implement the engagement of PDS speed guidance for 

spacing operations. Its implementational advantage was that the logic that controls its 

operation was constructed to include little or no interaction with existing AFDS modes. 

The diagram in Figure 11-2 shows the proposed logic to control the engagement of PDS

guidance via the MCP, and describes the progression of nominal PDS modes for a typical 

spacing operation. The diagram does not include the logic for degraded mode 

annunciations due to PDS off-nominal conditions. 
Page | 58 

Figure 11-2: PDS MCP FMA Logic Diagram

Figure 11-3 shows an example of how the modified MCP appeared with the new PDS 

mode select (highlighted by arrow) when PDS guidance was engaged (indicated by the 

green light.) 

Figure 11-3: MCP with PDS Mode Select Added

The implementation of PDS mode as a sub-mode of the VNAV mode is conceptually 

consistent with the operation of APS-DPA, which provides profile-driven speed 

corrections to a CDA. It is also consistent with the paradigm of dividing VNAV 

operation into other sub-modes (i.e., VNAV SPD, VNAV PTH, VNAV ALT). This 

concept would require the modification of the logic that controls the VNAV mode and 

(possibly) its interaction with other existing AFDS modes. In any practical 

implementation of APS-DPA in the cockpit, some modifications to the AFDS are bound 
Page | 59 

to occur anyway, and it is believed that this type of software-only modification may have 

an economic advantage over the implemented concept described above. 

11.3 Multifunction Control and Display Unit

In the fully integrated concept, the Multifunction Control and Display Unit (MCDU) 

provided an interface to the on-board ASTAR application for selecting modes of 

operation, entering spacing parameters, validating spacing clearances, monitoring the 

status of an in-progress APS-DPA operation, and terminating ASTAR if necessary. The 

ASTOR implementation for this study developed several new MCDU menu pages to 

facilitate the required additional functions.

The ASTOR implementation added three (3) new PDS-specific pages to the MCDU. A 

calling page was modified to allow the main PDS menu to be invoked (see Figure 11-4). 

From the main PDS menu, two sub-menu pages for display and control of approach 

parameters (“PDS APPR DATA”) and PDS profile mode (“PDS Profile”) were defined. 

PDS profile mode is discussed in the APS ConOps, but was not implemented or 

evaluated in this study. 

Figure 11-4: MCDU Main Calling Menu

Within the new pages, PDS-specific status and control functions were assigned to the 

twelve (12) “soft” keys aligned vertically on either side of the display bezel. The soft 

keys are arranged in two vertical rows of six (6) each on the left and right, referred to as 

1L through 6L for the left side, and 1R through 6R on the right side. Per MCDU 
Page | 60 

convention, a left or right pointer to any of the twelve bezel keys indicated that a 

programmable action was performed by pressing that key. The action was either 

indicated by the associated label (i.e., “TERMINATE”) or was implied as an editing 

action in the case of an alphanumeric parameter. The absence of the pointer indicated the 

parameter was non-editable. Refer to Table 11-1 and Table 11-2 for a description of the 

soft key functions defined for the PDS main menu menu and approach data pages. 

The main PDS menu page provided most of the necessary control and monitoring 

functions for the APS-DPA operation; the “PDS APPR DATA” page provided 

supplemental information and a means for the crew to enter, modify or monitor ASTAR 

parameters including landing runway assignment(s) for target aircraft, target aircraft 

approach speed(s), minimum in-trail distance behind target aircraft, and ownship 

approach speed.

Table 11-1: PDS Main Menu Page 

MCDU 

Soft Key

Label Functional Description

1L < PRECISION 

REFAC 

Precision REFerence AirCraft (target aircraft); auto-loaded or pilot-entered 

ID; when magenta, 

indicates active spacing aircraft.

2L < SPACING 

INTERVAL 

Auto-loaded or pilot-entered interval in seconds or nm; 

when magenta, indicates PDS speed guidance engaged 

using this parameter. 

3L SPACING DEV ASTAR-provided deviation in seconds. 

4L < STA Auto-loaded or pilot-entered STA in Zulu format; when magenta, indicates 

PDS speed guidance engaged using this parameter; when REFAC ADS-B 

data is acquired, STA will no longer be displayed.

5L < PROFILE* In some implementations, invokes PDS Profile menu page.

6L < APPR DATA Invoke PDS Approach Data menu page.

1R NCT REFAC > No Closer Than target aircraft; auto-loaded or pilot-entered ID; when 

magenta, indicates active spacing aircraft.

2R SPACING 

INTERVAL >

Auto-loaded or pilot-entered interval in seconds or nm; when magenta, 

indicates PDS speed guidance engaged using this parameter. 

3R SPACING DEV ASTAR-provided deviation in seconds or nm.

4R STA DEV ASTAR-provided deviation in seconds; when target aircraft ADS-B data is 

acquired, STA DEV will no longer be displayed.

5R ERASE > Only appears if parameters have been changed; erase all page 

modifications (revert to previous unmodified parameters). 

6R TERMINATE > Only appears if ASTAR is active; terminate ASTAR until new parameters 

entered; disengage PDS from AFDS (see MCP description in 11.2). 

*PDS Profile mode was not implemented for this study, but is discussed in Appendix D: Additional 

Interface Concepts for APS-DPA
Page | 61 

Table 11-2: PDS Approach Data Menu 

MCDU Soft 

Key

Label Function

1L PRECISION 

REFAC 

Precision REFerence AirCraft (target aircraft); AC ID from PDS main 

page. 

2L < APPROACH 

SPEED 

Auto-loaded from ASTAR from ADS-B on-condition report or pilotentered. 

3L < MIN DISTANCE *Auto-loaded from ASTAR or pilot-entered

4L < LANDING RWY Landing runway for precision REFAC from one of following:

auto-loaded from clearance; pilot-entered, or; auto-loaded from ABSB on-condition report. 

5L Unused

6L < PDS Invoke PDS main menu page.

1R NCT REFAC No Closer Than target aircraft; AC ID from PDS main page.

2R APPROACH 

SPEED >

Auto-loaded from ASTAR from ADS-B on-condition report or pilotentered. 

3R MIN DISTANCE > *Auto-loaded from ASTAR or pilot-entered.

4R LANDING RWY > Landing runway for precision target aircraft from one of following: 

auto-loaded from clearance; pilot-entered, or; auto-loaded from ABSB on-condition report.

5R OWNSHIP APPR 

SPEED >

Auto-loaded from profile or pilot-entered.

6R ERASE > Only appears if parameters have been changed; erase all page 

modifications (revert to previous unmodified parameters). 

*Minimum Distance represents the minimum wake turbulence avoidance in-trail distance for the category 

of aircraft involved. In an ADS-B environment, this parameter could be automatically entered via an onboard “look-up” table. 

The following are pictorial examples of MCDU PDS menu pages that were implemented 

for the study, shown in chronological order as a typical spacing scenario progressed. 

Figure 11-5 shows the PDS Main menu as it appeared after loading the received CPDLC 

spacing clearance to the FMC via the EICAS/MFD interface. Unpopulated fields for 

Spacing Deviation and STA Deviation indicate that ASTAR had not yet provided spacing 

guidance from the input spacing parameters, but was in the process of computing them. 

The “PDS-MOD” heading indicate parameters were changed but not executed, also 

indicated by the lit “EXECute” button on the MCDU. 
Page | 62 

Figure 11-5: PDS Main Menu Page; Clearance Loaded

Figure 11-6 shows the PDS Approach Data page as it appeared after loading the received 

CPDLC spacing clearance to the FMC. The approach speed, minimum distance and 

landing runway for the precision target (NASA 893) have been provided by ASTAR, 

indicating ADS-B on-condition reports have been received from NASA 893. The same 

parameters for the no closer than (NCT) target (NASA 859) are not populated, but would 

be automatically filled by ASTAR when ADS-B on-condition reports from the target

aircraft were received. The ownship approach speed has been loaded from the current 

profile being flown by the FMC. 

Figure 11-6: PDS Approach Data Page
Page | 63 

Figure 11-7 shows the PDS main menu page after ASTAR has determined that the 

precision target (NASA 893) is the active spacing aircraft (responsible for the current 

PDS target speed), indicated by the “PRECISION REFAC” and “SPACING INTVL” in 

magenta. ASTAR also maintained the STA Deviation parameter throughout the 

operation, though it did not provide spacing guidance after ADS-B reports were received 

for the target aircraft.

Figure 11-7: PDS Main Menu; ASTAR Guidance Active 

Figure 11-8 is similar to Figure 11-7, except that the test operator had entered the spacing 

parameters for the NCT target aircraft in the spacing scenario (NASA 859, NCT 90 sec), 

and had executed the entire combined spacing clearance. Note that the “EXECute” button 

light is extinguished, and the “PDS-MOD” heading has reverted to “PDS”. Spacing 

deviations from ASTAR now appear for the precision and NCT target aircraft and the 

STA, and the precision target is still the active target, indicated in magenta.
Page | 64 

Figure 11-8: PDS Main Menu; NCT Interval Added

Figure 11-9 is similar to Figure 11-8 except that ASTAR determined that the NCT target 

(NASA 859) was the active spacing aircraft, indicated in magenta.

Figure 11-9: PDS Main Menu; NCT REFAC Active
Page | 65 

12 Appendix C: Electronic Flight Bag Interface Concepts for APS-DPA

12.1 Description of Design Concept

The goal for the EFB version of the APS-DPA interface was to propose concepts 

allowing pilots to perform the following required functions: 

1. initialize ASTAR with spacing clearance parameters;

2. verify validity of spacing clearance parameters before accepting;

3. display PDS target speed necessary to achieve spacing interval(s); and

4. monitor progress of APS-DPA, including; 

a. PDS operation mode; 

b. target aircraft position and information; 

c. spacing deviation and trend information; and

d. off-nominal indications. 

It was proposed to develop an EFB-based application to provide the required functions 

(the application is referred to as Pair Dependent Speed or “PDS” for this discussion.) 

Pilots would initialize PDS with spacing parameters received via voice clearance. Once 

verified and executed, PDS would provide the target speed necessary to achieve the 

spacing interval, which the pilot would use to manually set the speed of the aircraft. PDS 

would provide strategic information about APS-DPA by implementing displays similar to 

those developed for those on the ND in the fully integrated concept. 

In order to function, the embedded ASTAR algorithm in PDS would require access to 

data available from the aircraft (ARINC429) data bus, such as the profile being flown by 

the aircraft and the ADS-B on-condition reports from the target aircraft. This would 

necessitate a minimum class 2 EFB for read-only access to this data; it was not 

envisioned that PDS would require write access to ARINC429 for the implementation of 

this concept. 

Since EFBs differ in user control interfaces, a representative implementation is used for 

illustration purposes that employs a combination of soft keys along the bezels, predefined keys above and below the screen and a touch screen interface. It is assumed that 

the concepts shown here could be easily transferred to other EFB implementations 

12.1.1 PDS Home Page 

Figure 12-1 shows a concept of the PDS application home page on the EFB. This page 

would be accessible by navigating through the EFB main menu to the PDS application. In 

this concept, the PDS home page is used to monitor the APS-DPA operation. 
Page | 66 

Figure 12-1: EFB PDS Home Page Concept

The top of the PDS home page displays PDS mode and target speed. The crew would be 

alerted to target speed changes by highlighting the “Speed Cmd” area (with a box or 

shading) for ten seconds. Similarly, changes to “Mode” would be highlighted for ten 

seconds. 

Below this, an area of the EFB screen is dedicated to displaying clearance parameters and 

ASTAR-supplied information such as active target aircraft (shown in magenta), spacing 

deviation (in seconds) from the assigned interval for each target aircraft, and spacing 

deviation from the original STA. Also, a soft key is implemented to access a sub-menu to 
Page | 67 

enter the spacing clearance parameters. Another soft key is implemented to terminate 

PDS speed guidance and clear the current spacing parameters.

The area below the clearance spacing data values implements a moving map. The concept 

for this map draws directly from the concepts for the Navigation display (ND) in the fully 

integrated implementation. The ownship aircraft symbol is a white triangle at the bottom 

center. The current magnetic heading is displayed at the top of the map for reference. A 

soft key is implemented to control the range of the map (set to 40 NM in the illustration). 

The target aircraft are shown as green double chevrons. An example of the trend indicator 

is also shown. If the stagger interval display concept were implemented, it would also be 

shown here. 

12.1.2 Spacing Clearance Sub-Page 

Figure 12-2 is a concept drawing of the PDS Spacing Clearance sub-menu. This menu 

facilitates entry or amendment of the spacing clearance parameters. A touch screen 

keyboard interface is shown which would simplify entry of alphanumeric clearance 

parameters.
Page | 68 

Figure 12-2: EFB Spacing Sub-Page Concept 
Page | 69 

13 Appendix D: Additional Interface Concepts for APS-DPA

Additional interface concepts were considered for the situations which might occur 

during APS-DPA operations. These concepts were not evaluated in this study, because 

schedule and resource limitations did not permit the development of alternative concepts 

and/or implementation in the ASTOR. The concepts are captured in this appendix to be 

considered in the future. 

13.1 Stagger Interval Depiction

It was considered possible for a step change in the PDS target speed to occur if ASTAR 

transitioned from using the assigned spacing interval to using the stagger interval for a 

target aircraft on approach to a parallel runway. The APS ConOps [7] describes this 

situation in more detail. While not an abnormal situation, abrupt changes in PDS target 

speed might cause the crew to question the new speed unless the context for the change 

was also displayed. For this reason, a concept was developed to inform the crew that the 

stagger interval either was or would soon be the governing spacing interval for ASTAR. 

ASTAR’s transition to stagger interval was considered strategic information, and a 

concept was developed to display this situation on the Navigation display (ND). The 

concept consists of a white dashed line, which pictorially represents the stagger geometry 

relative to the target aircraft on the parallel runway approach. This depiction is shown in 

Figure 13-1. The concept was intended to show the crew where the intercept of the 

stagger interval would occur, and in doing so, provide the necessary context for the PDS 

target speed change.

Figure 13-1: Stagger Interval Depiction
Page | 70 

Figure 13-2 shows the main PDS menu page as would appear after ASTAR had 

transitioned to using the stagger interval. In this depiction, the precision target is the 

active spacing aircraft, and ASTAR has changed the spacing interval from the assigned 

interval (in seconds) to the stagger interval. The stagger interval would have been preprogrammed for this particular dependent parallel approach. 

Figure 13-2: ASTAR Transition to Stagger Interval

13.2 PDS Speed Reversion Mode

PDS Speed Reversion mode is an off-nominal condition indicating PDS speed guidance 

is no longer valid due to invalid target aircraft data. This condition could occur because 

ADS-B on-condition reports from the target aircraft are no longer being received, and the 

condition could be temporary or permanent. The annunciation of PDS Speed Reversion

mode would inform pilots that PDS’s ability to provide target speed guidance was on 

hold. The APS ConOps describes this condition in more detail. 

A concept was developed to display PDS Speed Reversion mode for the PDS mode 

annunciation option in which the FMA was used to display PDS modes. In this concept, a 

yellow line would be drawn through the PDS Paired mode annunciation in the FMA for 

as long as the condition persists, or until ASTAR was terminated by the crew. (PDS 

Paired mode is the only PDS mode that requires ADS-B reports from the target aircraft.) 

See Table 13-1 for an illustration of this concept. 
Page | 71 

Table 13-1: PDS Speed Reversion Mode Concept 

PDS FMA Example Description FMA Depiction

PDS Mode Degraded PDS off-nominal condition (i.e., Speed 

Reversion mode); EICAS message provides 

additional detail; pilot action may be required to 

disengage PDS

13.3 PDS Profile Mode

PDS Profile mode provides a capability that leverages ASTAR’s ability to provide target 

speeds while flying any programmed profile. While these target speeds would be 

identical to the speeds contained in the profile in most cases, ASTAR is programmed to 

use stabilized approach speeds on final approach as the target speed, and may therefore 

be a preferable type of speed guidance for this phase of flight. The APS ConOps 

describes Profile mode in more detail.

A concept was developed to allow the pilot to control the use of Profile mode via the 

MCDU. Figure 13-3 shows a typical PDS main menu page on the MCDU with the profile 

mode select implemented for soft key 5L. Figure 13-4 shows the The PDS-PROFILE 

sub-page that would be used by the crew to control, execute and monitor the PDS Profile 

mode of ASTAR operation. The only choices available are to “Enable” the PDS Profile 

mode, or to return to the main PDS menu. Figure 13-5 shows the PDS-PROFILE menu 

after PDS Profile mode has been enabled. In this example, the profile name appears in 

magenta, indicating Profile mode has been engaged by selecting PDS mode on the MCP. 

Refer to Table 13-2 for a description of this sub-page’s status and control functions. 

Figure 13-3: PDS Main Menu with Profile Mode Selection
Page | 72 

Figure 13-4: PDS Profile Menu Page

Figure 13-5: PDS Profile Menu: Profile Mode Engaged
Page | 73 

Table 13-2: PDS Profile Menu Page 

MCDU Soft Key Label Function

1L Unused

2L Unused

3L Unused

4L Unused

5L Unused

6L < PDS Invoke PDS main menu page. 

1R PROFILE SPEEDS:

ENABLE >

Used to enable PDS Profile mode; when enabled and 

engaged, 

<Profile Name> of profile in use appears in magenta. 

2R Unused

3R Unused

4R Unused

5R Unused

6R TERMINATE > Terminate ASTAR until new parameters entered; 

disengage PDS from AFDS (see MCP description in 

11.2). 
Page | 74 

14 Appendix E: ASTOR Implementation Issues

This section documents ASTOR implementation issues or problems noted during the 

conduct of the experiment. These anomalies were not corrected in the interest of making 

identical presentions to the subject pilots. Any anomalies which may affected the 

outcome of the experiment have been noted in the main body of the report. The 

anomalies are documented here to be of use in any subsequent efforts to leverage the 

ASTOR implementation used for this experiment in future studies or simulations.

14.1 PDS Mode Annunciation

For the text mode option of this design element, it was noted by one subject pilot that 

when PDS mode appeared by itself in text (i.e., without PDS target speed text), that the 

display simply indicated the mode without also indicating “PDS”. For instance, in PDS 

Paired mode, the text on the PFD indicated “PAIR” instead of “PDS PAIR”. This is a 

result of the ASTOR implementation, which removed “PDS” from the text display when 

PDS target speed was configured to display the speed bug option. 

For the FMA option of this design element, several subject pilots noted that the specific 

VNAV mode currently in use (i.e., VNAV PTH, VNAV SPD, VNAV ALT) was never 

shown. This is a result of the ASTOR implementation ignoring the VNAV mode, and 

simply displaying ‘VNAV’ in the pitch mode segment. This VNAV mode is obviously 

important information and should also be shown in the pitch segmant. 

For the FMA mode of this design element, one subject pilot noted that in the FMA 

segment shared by VNAV and PDS modes, a change to VNAV or PDS mode would 

cause a box to be drawn around the entire segment. This obscured the meaning about 

whether the VNAV or PDS mode had caused the change. This is a result of the ASTOR 

implementation not creating a separate box for each portion of the shared pitch segment. 

14.2 Target Aircraft Depiction

For the “coupled with data block” option of this design element, several subject pilots 

noted that the active target aircraft data block location was in conflict with the existing 

implementation in the Boeing glass cockpit for the display of NAVAID data, normally 

selected using the EFIS control panel. This is a result of the ASTOR implementation 

ignoring this potential conflict.

Also, the data block implementation always contained the text “PRI”, which was meant 

to indicate that the the associated information in the data block was for the active spacing 

aircraft. This information was unnecessary, since the data block only appeared in 

association with the active spacing aircraft. This is a result of the ASTOR implementation 

using an older concept which would have been able to show a data block for either target

aircraft. (The “PRI” indicated “Primary”, which was a term formerly used for the active 

spacing aircraft).
Page | 75 

14.3 Trend Indicators

For the conformance box option of this design element, it was noted that the rangesensitive braces used to depict the box could potentially be driven off-scale by the ND 

range select. The convention for depicting off-scale CDTI symbols is to depict a halfsymbol at the edge of the range arc at the appropriate bearing. Adapting this convention 

to the conformance box braces would have prevented the depiction from disappearing at 

lower ND range settings.
REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

2. REPORT TYPE 

Contractor Report

 4. TITLE AND SUBTITLE

Airborne Precision Spacing for Dependent Parallel Operations Interface 

Study 

5a. CONTRACT NUMBER

NNL10AA14B

 6. AUTHOR(S)

Volk, Paul; Takallu, M. A.; Hoffler, Keith D.; Weiser, Jarold; Turner, 

Dexter

 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

NASA Langley Research Center 

Hampton, Virginia 23681-2199 

 

 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration

Washington, DC 20546-0001

 8. PERFORMING ORGANIZATION

 REPORT NUMBER

10. SPONSOR/MONITOR'S ACRONYM(S)

NASA

13. SUPPLEMENTARY NOTES

This report was prepared by Adaptive Aerospace Group, Inc, under NASA contract NNL10AA14B with SGT, Inc.

Langley Technical Monitor: Bryan E. Barmore

12. DISTRIBUTION/AVAILABILITY STATEMENT

Unclassified - Unlimited

Subject Category 03

Availability: NASA CASI (443) 757-5802

19a. NAME OF RESPONSIBLE PERSON

STI Help Desk (email: help@sti.nasa.gov)

14. ABSTRACT

This paper describes a usability study of proposed cockpit interfaces to support Airborne Precision Spacing (APS) operations for aircraft performing dependent 

parallel approaches (DPA). NASA has proposed an airborne system called Pair Dependent Speed (PDS) which uses their Airborne Spacing for Terminal 

Arrival Routes (ASTAR) algorithm to manage spacing intervals. Interface elements were designed to facilitate the input of APS-DPA spacing parameters to 

ASTAR, and to convey PDS system information to the crew deemed necessary and/or helpful to conduct the operation, including: target speed, guidance mode, 

target aircraft depiction, and spacing trend indication. In the study, subject pilots observed recorded simulations using the proposed interface elements in which 

the ownship managed assigned spacing intervals from two other arriving aircraft. Simulations were recorded using the Aircraft Simulation for Traffic 

Operations Research (ASTOR) platform, a medium-fidelity simulator based on a modern Boeing commercial glass cockpit. Various combinations of the 

interface elements were presented to subject pilots, and feedback was collected via structured questionnaires. The results of subject pilot evaluations show that 

the proposed design elements were acceptable, and that preferable combinations exist within this set of elements. The results also point to potential 

improvements to be considered for implementation in future experiments. 

15. SUBJECT TERMS

Airborne precision Spacing; Dependent runway operations; Interval management

18. NUMBER

 OF 

 PAGES

78

19b. TELEPHONE NUMBER (Include area code)

(443) 757-5802

a. REPORT

U

c. THIS PAGE

U

b. ABSTRACT

U

17. LIMITATION OF 

 ABSTRACT

UU

Prescribed by ANSI Std. Z39.18

Standard Form 298 (Rev. 8-98)

3. DATES COVERED (From - To)

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

411931.02.61.07.01 

11. SPONSOR/MONITOR'S REPORT

 NUMBER(S)

NASA/CR-2012-217777

16. SECURITY CLASSIFICATION OF:

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, 

gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 

collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and 

Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person 

shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY)

01- 10 - 2012