May 2015 

NASA/TP–2015-218767 

Air Traffic Management Technology Demostration 

Phase 1 (ATD) Interval Management for Near-Term 

Operations Validation of Acceptability (IM-NOVA) 

Experiment 

Jennifer L. Kibler and Sara R. Wilson 

NASA Langley Research Center, Hampton, Virginia 

Clay E. Hubbs 

National Institute of Aerospace, Hampton, Virginia 

James W. Smail 

Stinger Ghaffarian Technologies, Greenbelt, Maryland
NASA STI Program . . . in Profile

Since its founding, NASA has been dedicated to the 

advancement of aeronautics and space science. The 

NASA scientific and technical information (STI) 

program plays a key part in helping NASA maintain 

this important role. 

The NASA STI program operates under the auspices 

of the Agency Chief Information Officer. It collects, 

organizes, provides for archiving, and disseminates 

NASA’s STI. The NASA STI program provides access 

to the NTRS Registered and its public interface, the 

NASA Technical Reports Server, thus providing one 

of the largest collections of aeronautical and space 

science STI in the world. Results are published in both 

non-NASA channels and by NASA in the NASA STI 

Report Series, which includes the following report 

types: 

• TECHNICAL PUBLICATION. Reports of 

completed research or a major significant phase of 

research that present the results of NASA 

Programs and include extensive data or theoretical 

analysis. Includes compilations of significant 

scientific and technical data and information 

deemed to be of continuing reference value. 

NASA counter-part of peer-reviewed formal 

professional papers but has less stringent 

limitations on manuscript length and extent of 

graphic presentations. 

• TECHNICAL MEMORANDUM. 

Scientific and technical findings that are 

preliminary or of specialized interest, 

e.g., quick release reports, working 

papers, and bibliographies that contain minimal 

annotation. Does not contain extensive analysis. 

• CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees. 

• CONFERENCE PUBLICATION. 

Collected papers from scientific and technical 

conferences, symposia, seminars, or other 

meetings sponsored or 

co-sponsored by NASA. 

• SPECIAL PUBLICATION. Scientific, 

technical, or historical information from NASA 

programs, projects, and missions, often 

concerned with subjects having substantial 

public interest. 

• TECHNICAL TRANSLATION. 

English-language translations of foreign 

scientific and technical material pertinent to 

NASA’s mission. 

Specialized services also include organizing 

and publishing research results, distributing 

specialized research announcements and feeds, 

providing information desk and personal search 

support, and enabling data exchange services. 

For more information about the NASA STI program, 

see the following: 

• Access the NASA STI program home page at 

http://www.sti.nasa.gov 

• E-mail your question to help@sti.nasa.gov 

• Phone the NASA STI Information Desk at 

757-864-9658 

• Write to: 

NASA STI Information Desk 

Mail Stop 148 

NASA Langley Research Center 

Hampton, VA 23681-2199 
National Aeronautics and 

Space Administration 

Langley Research Center 

Hampton, Virginia 23681-2199 

 

May 2015 

NASA/TP–2015-218767 

Air Traffic Management Technology Demostration 

Phase 1 (ATD) Interval Management for Near-Term 

Operations Validation of Acceptability (IM-NOVA) 

Experiment 

Jennifer L. Kibler and Sara R. Wilson 

NASA Langley Research Center, Hampton, Virginia 

Clay E. Hubbs 

National Institute of Aerospace, Hampton, Virginia 

James W. Smail 

Stinger Ghaffarian Technologies, Greenbelt, Maryland
Available from: 

NASA STI Program / Mail Stop 148 

NASA Langley Research Center 

Hampton, VA 23681-2199 

Fax: 757-864-6500 

Acknowledgments 

The authors appreciate the support of the NASA Airspace Systems Program, Systems 

Analysis, Integration and Evaluation Project and gratefully acknowledge the hard work 

and invaluable contributions of civil servants and contractors associated with the NASA 

Langley Research Center Crew Systems and Aviation Operations Branch, the NASA 

Langley Research Center Simulation Development and Analysis Branch, the NASA 

Ames Research Center Human Systems Integration Division, and the NASA Ames 

Research Center Aviation Systems Division. 
iii 

Abstract 

The Interval Management for Near-term Operations Validation of Acceptability (IM-NOVA) 

experiment was conducted at the National Aeronautics and Space Administration (NASA) 

Langley Research Center (LaRC) in support of the NASA Airspace Systems Program’s Air 

Traffic Management Technology Demonstration-1 (ATD-1). ATD-1 is intended to showcase an 

integrated set of technologies that provide an efficient arrival solution for managing aircraft 

using Next Generation Air Transportation System (NextGen) surveillance, navigation, 

procedures, and automation for both airborne and ground-based systems. The goal of the IMNOVA experiment was to assess if procedures outlined by the ATD-1 Concept of Operations 

were acceptable to and feasible for use by flight crews in a voice communications environment 

when used with a minimum set of Flight Deck-based Interval Management (FIM) equipment and 

a prototype crew interface. To investigate an integrated arrival solution using ground-based air 

traffic control tools and aircraft Automatic Dependent Surveillance-Broadcast (ADS-B) tools, 

the LaRC FIM system and the Traffic Management Advisor with Terminal Metering and 

Controller Managed Spacing tools developed at the NASA Ames Research Center (ARC) were 

integrated into LaRC’s Air Traffic Operations Laboratory (ATOL). Data were collected from 10 

crews of current 757/767 pilots asked to fly a high-fidelity, fixed-based simulator during 

scenarios conducted within an airspace environment modeled on the Dallas-Fort Worth (DFW) 

Terminal Radar Approach Control area. The aircraft simulator was equipped with the Airborne 

Spacing for Terminal Area Routes (ASTAR) algorithm and a FIM crew interface consisting of 

electronic flight bags and ADS-B guidance displays. Researchers used “pseudo-pilot” stations to 

control 24 simulated aircraft that provided multiple air traffic flows into the DFW International 

Airport, and recently retired DFW air traffic controllers served as confederate Center, Feeder, 

Final, and Tower controllers. Analyses of qualitative data revealed that the procedures used by 

flight crews to receive and execute interval management (IM) clearances in a voice 

communications environment were logical, easy to follow, did not contain any missing or 

extraneous steps, and required the use of an acceptable workload level. The majority of the pilot 

participants found the IM concept, in addition to the proposed FIM crew procedures, to be 

acceptable and indicated that the ATD-1 procedures could be successfully executed in a nearterm NextGen environment. Analyses of quantitative data revealed that the proposed procedures 

were feasible for use by flight crews in a voice communications environment. The delivery 

accuracy at the achieve-by point was within ±5 sec, and the delivery precision was less than 5 

sec. Furthermore, FIM speed commands occurred at a rate of less than one per minute, and pilots 

found the frequency of the speed commands to be acceptable at all times throughout the 

experiment scenarios. 

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

iv 

Table of Contents 

1 Introduction ..................................................................................................................... 1

Current Study ............................................................................................................. 2

2 Method ........................................................................................................................... 3

Experiment and Scenario Design ............................................................................... 3

Hypotheses ......................................................................................................... 6

Pilot Participants ........................................................................................................ 6

Experiment Procedure ............................................................................................... 7

Scheduling and Spacing Technologies ....................................................................... 7

Traffic Management Advisor with Terminal Metering (TMA-TM) .......................... 7

Controller-Managed Spacing (CMS) ................................................................... 8

Flight Deck-based Interval Management (FIM) ..................................................10

Facilities and Equipment ...........................................................................................11

Air Traffic Operations Laboratory (ATOL) ...........................................................11

Integration Flight Deck (IFD) ..............................................................................15

Dependent Measures ................................................................................................16

3 Results and Discussion .................................................................................................17

Acceptability of the Procedures .................................................................................17

Completeness of the Procedures ..............................................................................18

Pilot Workload ...........................................................................................................19

Usability of the Crew Interface...................................................................................20

Acceptability of Data Entry Procedures .....................................................................21

Rate of Speed Commands ........................................................................................22

Spacing Error at the Final Approach Fix ....................................................................26

Spacing Error at the Runway Threshold ....................................................................28

4 Conclusions ....................................................................................................................30

5 References ......................................................................................................................31

Appendix A. Simulated Airspace and Scenario Design .......................................................35

Appendix B. Experiment Run Order ......................................................................................45

Appendix C. Post-Run Questionnaire ...................................................................................47

Appendix D. Post-Experiment Questionnaire .......................................................................52

Appendix E. Example Interval Management (IM) Clearance and Related Use of and 

Display of Information on the Prototype Flight Deck-based Interval 

Management (FIM) Crew Interface ....................................................................57
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

v 

List of Figures 

Figure 1. Side-mounted electronic flight bag (EFB) and Flight Deck-based 

Interval Management (FIM) application. ................................................. 4

Figure 2. ADS-B guidance display (AGD) presenting target and error speed 

values. ....................................................................................................... 4

Figure 3. Screenshot of the Traffic Management Advisor with Terminal 

Metering (TMA-TM) display used in the IM-NOVA experiment. ............ 8

Figure 4. Elements in the aircraft full data block (FDB) shown on the IM-NOVA 

Center controller’s display. ..................................................................... 9

Figure 5. Full data block (FDB) with speed advisory and slot marker circle as 

shown on the IM-NOVA Feeder controller’s display. .......................... 10

Figure 6. Remote ASTOR Pseudo-Pilot (RAPP) station graphical user interface.

 ................................................................................................................. 12

Figure 7. IM-NOVA Center controller’s Display System Replacement (DSR) 

display. ................................................................................................... 14

Figure 8. IM-NOVA Feeder controller’s Standard Terminal Automation 

Replacement System (STARS) display. ............................................... 15

Figure 9. Integration Flight Deck (IFD) equipped with IM-NOVA’s prototype FIM 

crew interface. ........................................................................................ 16

Figure 10. Spacing error at the FAF (sec) for the Nominal scenario. .................. 27

Figure 11. Spacing error at the FAF (sec) for the Amend ASG scenario. ........... 27

Figure 12. Spacing error at the FAF (sec) for the ATC Termination of FIM 

scenario. ................................................................................................. 27

Figure 13. Spacing error at the FAF (sec) for the Suspend/Resume scenario. .. 27

Figure 14. Spacing error at the FAF (sec) for the ADS-B Loss scenario. ........... 28

Figure 15. Spacing error at the runway threshold (sec) for the Nominal scenario.

 ................................................................................................................. 29

Figure 16. Spacing error at the runway threshold (sec) for the Amend ASG 

scenario. ................................................................................................. 29

Figure 17. Spacing error at the runway threshold (sec) for the ATC Termination 

of FIM scenario. ..................................................................................... 29

Figure 18. Spacing error at the runway threshold (sec) for the Suspend/Resume 

scenario. ................................................................................................. 29

Figure 19. Spacing error at the runway threshold (sec) for the ADS-B Loss 

scenario. ................................................................................................. 29 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

vi 

Figure A-1. NASA Research STAR for the Bonham Five Arrival into DFW. ......... 36

Figure A-2. NASA Research STAR for the Bowie One Arrival into DFW. ............. 37

Figure A-3. NASA Research STAR for the Cedar Creek Six Arrival into DFW. .... 38

Figure A-4. NASA Research STAR for the Glen Rose Nine Arrival into DFW. ..... 39

Figure A-5. Nominal Scenario. ................................................................................. 40

Figure A-6. Amend ASG Scenario. ........................................................................... 41

Figure A-7. ATC Termination of FIM Scenario. ....................................................... 42

Figure A-8. Suspend / Resume FIM Scenario.......................................................... 43

Figure A-9. ADS-B Loss Scenario. ........................................................................... 44

Figure E-1. Initial page of the IM application residing on the prototype FIM crew 

interface’s EFB. ...................................................................................... 57

Figure E-2. Three-step procedure for inputting IM achieve-by point information 

into the prototype FIM crew interface’s EFB ....................................... 58

Figure E-3. Two-step procedure for inputting STA at the IM achieve-by point 

information into the prototype FIM crew interface’s EFB. ................. 59

Figure E-4. Two-step procedure for inputting assigned spacing goal information 

into the prototype FIM crew interface’s EFB. ...................................... 60

Figure E-5. Three-step procedure for inputting a target aircraft’s callsign into the 

prototype FIM crew interface’s EFB. .................................................... 61

Figure E-6. Three-step procedure for inputting a target aircraft’s arrival and 

transition information into the prototype FIM crew interface’s EFB. 62

Figure E-7. IM clearance activation. ......................................................................... 63

Figure E-8. Information elements associated with STA mode displayed on the 

EFB and AGD components of the prototype FIM crew interface. ...... 64

Figure E-9. Information elements associated with FIM operations displayed on 

the EFB and AGD components of the prototype FIM crew interface 65

Figure E-10. Three-step procedure for suspending and then resuming IM spacing 

using the prototype FIM crew interface’s EFB. ................................... 66

Figure E-11. Two-step procedure, and resulting data (re-)entry screen, associated 

with terminating IM spacing using the prototype FIM crew interface’s 

EFB.......................................................................................................... 67

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

vii 

List of Tables 

Table 1. Experiment design matrix. ...................................................................... 5

Table 2. Descriptive statistics for acceptability ratings (from post-run 

questionnaire item 5f). ........................................................................... 18

Table 3. Descriptive statistics for acceptability ratings (from post-run 

questionnaire item 5j). ........................................................................... 18

Table 4. Percentage (%) of pilots who reported procedures were complete 

(from post-run questionnaire items 6a, 6b, and 6c). ........................... 19

Table 5. Descriptive statistics for pilot workload ratings (from post-run 

questionnaire item 3). ............................................................................ 20

Table 6. Descriptive statistics for ratings of the amount of heads-down time 

(from post-run questionnaire item 5g). ................................................ 21

Table 7. Descriptive statistics for ratings of the usability of the displays (from 

post-run questionnaire item 5i). ........................................................... 21

Table 8. Descriptive statistics for ratings of the ease and intuitiveness of 

entering information (from post-run questionnaire item 5h). ............ 22

Table 9. Mean rate of speed commands (number of speed commands per 

minute) for each segment of flight. ...................................................... 25

Table 10. Descriptive statistics for ratings of operational acceptability and 

appropriateness of IM commanded speeds (from post-run 

questionnaire item 5c). .......................................................................... 26

Table 11. Descriptive statistics for ratings of acceptability of the frequency of 

the IM speed commands (from questionnaire item 5d). ..................... 26

Table 12. Descriptive statistics for spacing error at the FAF (sec). ................... 27

Table 13. Descriptive statistics for spacing error at the runway threshold (sec).

 ................................................................................................................. 28 

Table A-1. Distribution of aircraft over the STARs/Transitions. .......................... 35

Table B-1. Latin square designs. ............................................................................ 45

Table B-2. Experiment run order. ........................................................................... 46

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

viii 

Symbols & Abbreviations 

ADS-B Automatic Dependent Surveillance-Broadcast 

AGD ADS-B Guidance Display 

ARC Ames Research Center 

ASG assigned spacing goal 

ASTAR Airborne Spacing for Terminal Area Routes 

ASTOR Aircraft Simulation for Traffic Operations Research 

ATC air traffic control 

ATD-1 Air Traffic Management Technology Demonstration–1 

ATOL Air Traffic Operations Laboratory 

ATOS Airspace and Traffic Operations Simulation

CAS calibrated airspeed

CMS controller-managed spacing 

ConOps Concept of Operations

CPDLC Controller-Pilot Data Link Communications

DFW Dallas-Fort Worth International Airport 

DSR Display System Replacement 

EFB electronic flight bag 

ETA estimated time of arrival 

FAA Federal Aviation Administration 

FAF final approach fix 

FDB full data block 

FIM Flight Deck-based Interval Management 

FMS Flight Management System 

GUI graphical user interface 

HITL human-in-the-loop 

IFD Integration Flight Deck 

ILS Instrument Landing System 

IM interval management 

IM-NOVA Interval Management for Near-term Operations Validation of 

 Acceptability 

LaRC Langley Research Center 

MACS Multi-Aircraft Control System 

MCH Modified Cooper-Harper 

N number of observations 

NAS National Airspace System 

NASA National Aeronautics and Space Administration

NextGen Next Generation Air Transportation System 

OPD optimized profile descent 

p p-value (note: a value < 0.05 indicates a statistically significant difference 

 between sample means) 

PF pilot flying 

PM pilot monitoring 

RAPP Remote ASTOR Pseudo-Pilot
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

ix 

RNAV area navigation 

SD standard deviation 

STA scheduled time of arrival 

STAR standard terminal arrival route 

STARS Standard Terminal Automation Replacement System

TMA Traffic Management Advisor 

TMA-TM Traffic Management Advisor with Terminal Metering 

TOD top-of-descent 

TRACON terminal radar approach control 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

1 

1 Introduction 

Air traffic demand is predicted to increase by 2 to 3 percent per year over the next 20 years, with 

the number of revenue passenger miles nearly doubling by 2032 (ref. 1). If the current air 

transportation system is left unmodified, this projected growth will lead to increased delays, fuel 

costs, noise pollution, and greenhouse gas emissions. The Federal Aviation Administration’s 

(FAA) Next Generation Air Transportation System (NextGen) concept envisions a 

comprehensive transformation of the National Airspace System (NAS) to support this continued 

growth in a safe, reliable, and efficient manner (ref. 2). The National Aeronautics and Space 

Administration (NASA) is collaborating with the FAA and other industry partners to develop 

advanced technologies and automation tools necessary for NextGen. 

Improving the efficiency of terminal area arrival operations is an especially complex task. 

Conditions in busy terminal areas today often result in inefficient arrival paths involving frequent 

changes in speed, heading, and altitude to maintain safe separation between aircraft and absorb 

large amounts of delay. These inefficiencies lead to increased fuel burn and noise pollution, as 

well as higher controller workload and traffic congestion. Furthermore, greater uncertainty in the 

current system causes controllers to add separation buffers between aircraft, thus reducing 

throughput and increasing delays. Although more efficient arrivals are available, current 

technology limits their use to periods of light to moderate traffic conditions. New concepts and 

technologies are needed to make efficient arrival procedures feasible during heavy traffic. 

NASA’s Air Traffic Management Technology Demonstration–1 (ATD-1) will operationally 

demonstrate the feasibility of efficient arrival operations by combining ground-based and 

airborne NASA technologies (refs. 3, 4, and 5). The ATD-1 integrated system consists of the 

following three core components: 

• Traffic Management Advisor with Terminal Metering (TMA-TM), which generates 

precise time-based schedules to the runway and merge points within the terminal area 

• Controller-managed spacing (CMS) decision support tools, which provide controllers with 

speed advisories and other information needed to meet the schedule 

• Flight Deck-based Interval Management (FIM) avionics and procedures, which allow 

flight crews to adjust their speed to achieve precise relative spacing 

The Traffic Management Advisor (TMA) was originally developed at NASA Ames Research 

Center (ARC) and is currently used at Air Route Traffic Control Centers nationwide to determine 

an appropriate arrival schedule (ref. 6). TMA-TM is an enhanced form of TMA that includes 

terminal area metering and enables the use of more efficient arrival procedures. CMS decision 

support tools were also developed at NASA ARC. They provide controllers with the information 

necessary to achieve arrival schedule conformance using speed commands, thus reducing the use 

of tactical vectoring (refs. 7 and 8). The use of TMA-TM in conjunction with CMS tools has 

been assessed, and results indicate an increase in airport throughput (refs. 9–12). 

FIM is an airborne spacing concept in which the flight crew is responsible for flying their aircraft 

at a speed that achieves their assigned time-based spacing interval behind a target aircraft, while 

Air Traffic Control (ATC) remains responsible for ensuring that all aircraft maintain safe 

separation. Typically, ATC designates a spacing buffer in addition to the separation requirement 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

2 

to ensure that separation is always maintained. The goal of airborne spacing is to decrease this 

spacing buffer by decreasing the variability of the time error associated with an aircraft’s arrival 

at a specific point along its arrival route. The precise merging and spacing enabled by FIM 

avionics and flight crew procedures reduces excess spacing buffers and results in higher terminal 

throughput. Studies conducted by MITRE (refs. 13–15), EUROCONTROL (refs. 16–19), and 

NASA Langley Research Center (LaRC) (refs. 20–22) have demonstrated an increase in 

efficiency through the use of FIM operations. 

In addition to utilizing these advanced technologies, aircraft will fly new, more direct area 

navigation (RNAV) routes that extend from en route airspace to the runway. Optimized profile 

descent (OPD) procedures will also be implemented to provide a fuel-efficient continuous 

descent approach rather than the step-down descents used today. The Automatic Dependent 

Surveillance-Broadcast (ADS-B) infrastructure currently being implemented by the FAA will 

also be leveraged. The FIM tools will calculate speed commands using information provided by 

ADS-B, which is more accurate than traditional radar. The ability of flight crews to make more 

precise speed adjustments will enable a reduction in spacing buffers resulting in higher terminal 

throughput. 

These technology components and procedures have been evaluated independently, and each has 

demonstrated benefits. As an integrated system, these technologies are intended to increase 

throughput, reduce delay, and minimize environmental impacts. Initial studies at NASA ARC to 

demonstrate the ATD-1 concept and validate operational feasibility indicate that the concept is 

viable and operations are acceptable (refs. 23–25).

 Current Study 

As part of the preparations for an ATD-1 flight demonstration, the Interval Management for 

Near-Term Operations Validation of Acceptability (IM-NOVA) human-in-the-loop (HITL) 

experiment was conducted at NASA LaRC. The objective of this experiment was to assess if the 

procedures outlined in the ATD-1 Concept of Operations (ConOps) document (ref. 5), when 

used with the integrated ATD-1 technologies, were acceptable to and feasible for use by flight 

crews in a voice communications environment when precision to an achieve-by point (i.e., the 

final approach fix) was expected. This paper describes the experiment’s methodology, results 

associated with the pilot participants’ acceptability and workload ratings, and results of the 

evaluation of the flight crew procedures’ feasibility. 

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

3 

2 Method 

 Experiment and Scenario Design 

Data were collected from current 757/767 pilot participants asked to fly a high-fidelity, fixedbase simulator during scenarios conducted within an airspace environment modeled after the 

Dallas-Fort Worth (DFW) terminal radar approach control (TRACON) area. Each experiment 

scenario consisted of multiple air traffic flows involving 25 arrival aircraft flying into DFW 

airport and landing on runways 17C and 18R. All aircraft flew RNAV arrivals to the Instrument 

Landing System (ILS) approaches. Some aircraft initialized in level cruise and flew the full 

arrival and approach to the runway, whereas others initialized in descent and flew only a portion 

of the arrival before flying the approach. One of the arrival aircraft in the arrival stream was a 

high-fidelity, fixed-base simulator with pilot participants operating as a two-person crew. This 

simulator was equipped with NASA LaRC’s airborne spacing algorithm, Airborne Spacing for 

Terminal Area Routes (ASTAR) (version 11.06.22) (ref. 26), and a prototype FIM crew interface 

(shown in Figs. Figure 1 and Figure 2). The remaining 24 arrival aircraft were flown by two 

researcher pseudo-pilots, each of whom used a graphical user interface (GUI) to control multiple 

medium-fidelity aircraft simulators. To provide a realistic traffic environment, each scenario also 

included 25 departure aircraft. Recently retired DFW air traffic controllers served as confederate 

Center, Feeder, Final, and Tower controllers issuing speed commands, vectors, and IM 

clearances. 

When performing FIM operations, pilot participants were expected to use ASTAR-provided 

speed guidance whenever possible. This speed guidance is designed such that the FIM, or 

spacing, aircraft will achieve the assigned spacing goal (ASG) behind its target, or lead, aircraft 

at a predefined achieve-by point while remaining within 10 percent of the published RNAV 

arrival airspeed. In this experiment, the final approach fix (FAF) served as the FIM aircraft’s 

achieve-by point. The prototype FIM crew interface shown in Figure 1 and Figure 2 consisted of 

two side-mounted electronic flight bags (EFBs) and two ADS-B guidance displays (AGDs) 

mounted under the glare shield within the flight crew’s forward field-of-view. Figure 1 shows 

the position of a side-mounted EFB as well as a screen shot of the FIM application used for data 

entry and speed conformance monitoring. Figure 2 shows the position of an AGD as well as an 

image of the AGD displaying the IM-commanded speed and the aircraft’s deviation from the 

commanded speed. 

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

4 

Figure 1. Side-mounted electronic flight bag (EFB) and Flight Deck-based Interval 

Management (FIM) application. 

Figure 2. ADS-B guidance display (AGD) presenting target and error speed 

values. 

Previous FIM research conducted at NASA LaRC has utilized data link to transfer information 

from ATC to the flight crew. However, since the ground infrastructure necessary to support data 

link will not be available during the execution of near-term FIM operations, voice 

communications will be relied upon to transfer necessary information. In the IM-NOVA 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

5 

experiment, confederate controllers issued IM clearances to the flight crews, who then entered 

information into the EFBs, and activated the FIM avionics. The IM procedure required the flight 

crew to enter the following pieces of information included in the IM clearance into the EFBs: 

• IM achieve-by point (i.e., FAF) 

• Scheduled time of arrival at the IM achieve-by point 

• Target aircraft callsign 

• ASG (spacing interval required at the IM achieve-by point) 

• Target aircraft flight path (arrival and transition) 

Five flight scenarios were defined using the 1×5 experiment matrix shown in Table 1 to allow an 

examination of the five flight crew procedures outlined in the ATD-1 ConOps document. 

Additional details regarding the design of the experiment’s scenarios and graphical 

representations of the research standard terminal arrival routes (STARs) used within the 

experiment are included in Appendix A. 

Table 1. Experiment design matrix. 

Flight Crew 

Procedure Nominal Amend 

ASG 

ATC 

Termination 

of FIM 

Suspend/Resume 

FIM 

ADS-B 

Loss 

1. The Nominal scenario consisted of an IM clearance issued by ATC prior to top of descent 

(TOD). After the FIM aircraft reported that spacing had commenced, the pilot 

participants’ aircraft (i.e., the FIM aircraft) maintained nominal FIM operations until 

reaching the achieve-by point. 

2. During the Amend ASG scenario, the initial IM clearance was issued shortly after TOD. 

Approximately two minutes after the FIM aircraft reported that spacing had commenced, 

ATC issued an amended clearance to increase spacing by 20 sec. 

3. In the ATC Termination of FIM scenario, the initial IM clearance was issued shortly after 

TOD. Once the FIM aircraft and target aircraft were inside the DFW TRACON, ATC 

cancelled the IM clearance and vectored the target aircraft for landing on runway 13R. 

ATC then issued a new IM clearance with a new target for the FIM aircraft. 

4. The Suspend/Resume FIM scenario consisted of an IM clearance issued by ATC prior to 

TOD. After the FIM aircraft reported that spacing had commenced, ATC suspended the 

IM clearance and issued a speed change of 20 kt for the FIM aircraft. Approximately two 

minutes later, ATC cleared the FIM aircraft to resume IM spacing. 

5. During the ADS-B Loss scenario, ATC issued the initial IM clearance prior to TOD. 

After the FIM aircraft reported that spacing had commenced and both the pilot 

participants’ aircraft and target aircraft were inside the TRACON, the target aircraft 

experienced a loss of ADS-B capability. The pilot participants notified ATC that they 

were “IM Unable” due to a loss of the target aircraft’s ADS-B signal, and ATC then 

cancelled the initial IM clearance and issued a new clearance with a new target aircraft. 

During all five scenarios, the closest aircraft in the arrival stream for the same runway as the FIM 

aircraft was designated as the initial target aircraft. If a new target was designated later in the 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

6 

scenario, it was always the next closest aircraft in the arrival stream. For both the ATC 

Termination of FIM and ADS-B Loss scenarios, the second clearance was issued in the 

TRACON at an altitude below 10,000 ft. 

 Hypotheses 

The IM-NOVA experiment was designed to meet the experiment objective and investigate the 

following a priori hypotheses: 

Hypothesis 1: The use of the procedures for receiving and executing IM clearances in a voice 

communications environment will be acceptable to the flight crew, with mean ratings of the 

procedures’ completeness and acceptability higher than 4.5 on a 7-point scale. 

Hypothesis 2: At least 90 percent of the pilots will report that the procedures are complete. 

Hypothesis 3: Pilots will report the workload level required to execute the procedures to be 

acceptable, with a mean rating of less than 3 on the Modified Cooper-Harper (MCH) rating 

scale1 (ref. 27). 

Hypothesis 4: Pilots will report no increase in workload with the Amend ASG, ATC Termination 

of FIM, Suspend/Resume FIM, or ADS-B Loss scenarios as compared to the Nominal scenario 

(i.e., a difference of less than 1 unit on the MCH rating scale). 

Hypothesis 5: Pilots will report that the crew interface was usable, with mean ratings of the 

acceptability of heads-down time required and usability of the displays higher than 4.5 on a 7-

point scale. 

Hypothesis 6: The data entry procedures will be acceptable to the flight crew, with mean ratings 

of ease and intuitiveness of entering information higher than 4.5 on a 7-point scale. 

Hypothesis 7: The rate of speed commands will be acceptable to the flight crew, with speed 

commands occurring at a rate of less than two per minute, and mean ratings of operational 

acceptability and acceptability of the frequency of speed commands higher than 4.5 on a 7-point 

scale. 

Hypothesis 8: The spacing error at the FAF will be within ±5 sec with a standard deviation of 

less than 5 sec. 

 Pilot Participants 

The study involved 20 current 757/767 pilots from major U.S. air carriers who participated in the 

experiment in 10 groups of two-person crews. All pilots were male and ranged in age from 40 to 

62 years. On average, each of the pilots had 23 years of airline experience and over 13,000 hours 

of commercial airline flight time. To minimize potential effects associated with different airline 

operating procedures, all two-person crews were paired from the same airline, and the pilots flew 

in their current operational position (captain or first officer) using their company’s standard 

operating procedures modified to include FIM operations. 

 

1

 A rating of 3 on the MCH rating scale indicates that the instructed task is fair and/or has mild difficulty, and 

acceptable operator mental effort is required to attain adequate system performance. The full MCH rating scale is 

presented in Appendix C. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

7 

 Experiment Procedure

The pilot participants received training materials and had access to computer-based training prior 

to arriving at LaRC. After arriving at LaRC, pilot participants received four hours of classroom 

and hands-on training involving the completion of three simulated flight training scenarios prior 

to completing the first data collection run. Each two-person crew participated in a two-day 

experiment session and completed a total of 10 data collection flights. The first day began with 

training which was followed by data collection flights. Each data collection flight lasted 

approximately 25 min, and pilots were asked to complete a post-run questionnaire immediately 

following each flight. The second day consisted of the remaining data collection flights and postrun questionnaires, the completion of one exploratory scenario involving the presentation of 

aural cues in conjunction with commanded speed changes, and the completion of a postexperiment questionnaire and debriefing session. 

Every crew flew each experiment scenario twice: once with the captain as the pilot flying (PF) 

and the first officer as the pilot monitoring (PM), and once with the first officer as the PF and the 

captain as the PM. The run order of the scenarios was partially counterbalanced, and within each 

crew the pilots switched PF and PM responsibilities between runs. Additional details regarding 

the experiment run order are included in Appendix B. 

 Scheduling and Spacing Technologies 

This experiment utilized an integrated set of ground-based and airborne technologies consisting 

of TMA-TM, CMS decision support tools, and FIM avionics and procedures. These scheduling 

and spacing technologies are described below. 

 Traffic Management Advisor with Terminal Metering (TMA-TM) 

TMA-TM is an extension of the operational TMA that determines an arrival schedule based on 

airport conditions, airport capacity, required spacing, and weather conditions. This scheduling 

tool calculates the estimated time of arrival (ETA) and the corresponding scheduled time of 

arrival (STA) at various meter and merge points along the aircraft flight path. The TMA-TM data 

are broadcast to the en route and TRACON controller positions for use by the CMS tools to 

assist the controllers in maintaining optimum flow rates to the runways. 

The TMA-TM display used during the IM-NOVA experiment (shown in Figure 3) included a 

total of six timelines. From left to right, the first four timelines show the TMA schedule for each 

of the four metering fixes around the DFW TRACON. The fifth timeline shows the arrival 

schedule for DFW runway 18R, and the sixth timeline shows the arrival schedule for runway 

17C. The current clock time (1541:01z) appears in the upper left corner. The white numbers 

located in the middle of each timeline indicate minutes after the hour. As time progresses, the 

timelines scroll toward the bottom of the screen with the current time at the bottom and one hour 

in the future at the top. On the left of each timeline, each aircraft’s ETA at the applicable 

metering fix or runway threshold is shown in green. On the right side of the timeline, each 

aircraft’s STA is shown in amber or blue. An amber callsign indicates that an aircraft is outside 

of the TMA freeze horizon for the corresponding fix/runway (set at 19 minutes in the future). 

Until the freeze horizon is passed, TMA continuously recalculates the STA. When the freeze 

horizon is reached, the STA is frozen, and the aircraft callsign turns blue. If the green indicator 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

8 

on the left side of the timeline is below the corresponding indicator on the right, the aircraft is 

early; if it is above the corresponding indicator, the aircraft is late. 

Figure 3. Screenshot of the Traffic Management Advisor with Terminal Metering 

(TMA-TM) display used in the IM-NOVA experiment. 

 Controller-Managed Spacing (CMS) 

During the IM-NOVA experiment, three CMS tools were used to provide the confederate 

controllers with information needed to meet the TMA-TM generated schedule: (1) early/late 

indicators, (2) slot marker circles, and (3) speed advisories. Early/late indicators located in the 

aircraft full data blocks (FDB) enabled controllers to quickly assess the schedule-conformance 

information for that aircraft. Elements included in the FDB shown on the Center controller’s 

display, including an early/late indicator, are depicted and described in Figure 4. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

9 

Figure 4. Elements in the aircraft full data block (FDB) shown on the IM-NOVA 

Center controller’s display. 

Figure 5 depicts and describes the elements of an FDB with a slot marker circle and a speed 

advisory as shown on the Feeder controller’s display. The IM-NOVA experiment’s confederate 

TRACON controllers’ displays presented slot marker circles that indicated where an aircraft 

should be located at a given time if it were to fly the RNAV arrival, meeting all published speed 

and altitude restrictions and meeting the STA at the next scheduling fix. The relative position of 

the aircraft symbol and the slot marker circle provided a quick visual indication of how an 

aircraft was positioned relative to its STA. The diameter of the slot marker circle represented 15 

sec of flying time at the aircraft’s current ground speed. If the aircraft was on schedule, the 

aircraft symbol was centered within the slot marker circle. 

Speed advisories in an aircraft’s FDB helped confederate controllers formulate speed clearances 

for aircraft not performing FIM operations. The speed advisory provided a recommended 

calibrated airspeed (CAS) intended to place an aircraft back on schedule before reaching a 

scheduling fix. 

 

Line 1: Aircraft callsign 

Line 2: Aircraft altitude (in hundreds of feet) 

Line 3: 

Item 1: Computer assigned aircraft identification 

 number 

Item 2: Ground speed 

Item 3: Magenta symbols, manually toggled by ATC 

® = Aircraft’s STA has been assigned, but no IM 

 clearance has been issued 

® S = IM clearance has been issued, but IM has not 

 yet been engaged (i.e., the aircraft is still flying 

 to its STA) 

S = Aircraft is operating in IM mode 

Line 4: Aircraft type (B752 = B757-200) 

Line 5: Early/Late indicator (shows that aircraft is 

currently 24 sec early)
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

10 

Figure 5. Full data block (FDB) with speed advisory and slot marker circle as 

shown on the IM-NOVA Feeder controller’s display. 

 Flight Deck-based Interval Management (FIM) 

The FIM tools provide onboard speed guidance to the flight crew, enabling them to achieve a 

precise spacing interval behind a target aircraft and thereby meet a schedule set by TMA-TM. To 

allow the performance of FIM operations during the IM-NOVA experiment, the simulator flown 

by the pilot participants (described in a subsequent section of this document) was equipped with 

NASA’s ASTAR airborne spacing algorithm and the prototype FIM crew interface. 

The ASTAR algorithm produces speed guidance by determining time-to-go until an aircraft and 

its target reach an achieve-by point along a 4D trajectory. During the IM-NOVA experiment, 

ASTAR (version 11.06.22) utilized design features intended to reduce the number of required 

speed changes while maintaining arrival precision. One such feature consisted of a gain schedule 

that made commanded speed changes more sensitive to a given time error as the aircraft 

approached the runway. Another feature consisted of a look-ahead function that looked 10 

seconds ahead for a profile speed decrease and inhibited speed increases during that 10-second 

period. Additionally, ASTAR also included mechanisms intended to keep it from generating 

unacceptable commanded speeds and features intended to help ensure adherence to various 

regulations. For example, to keep the commanded speeds within an acceptable range, ASTAR 

limited commanded speed deviations to ±10 percent of the nominal profile speed and adhered to 

the 250 kt speed restriction below an altitude of 10,000 ft. 

As described previously, the prototype FIM crew interface (shown above in Figure 1 and Figure 

2) consisted of two side-mounted EFBs and two AGDs positioned within the flight crew’s 

forward field of view. The IM application residing on the EFB enabled the flight crew to use the 

EFB’s bezel buttons and touch screen to enter the following information, communicated via an 

ATC radio-issued IM clearance, into the EFB: achieve-by point, STA, ASG, target aircraft 

callsign, and target aircraft route. After entering this information into the EFB, the flight crew 

pressed an “activate” button and then flew speeds to meet their STA until ADS-B information 

from their target aircraft was acquired and FIM operations could begin. 

During FIM operations, the EFB displayed the target aircraft’s callsign; a commanded speed 

(“CMD SPD”), in Mach or knots, to indicate the FIM aircraft speed needed to achieve precise 

Line 1: 

Item 1: Aircraft callsign 

Item 2: Assigned runway 

Line 2: 

Item 1: Aircraft altitude (in hundreds of feet) 

Item 2: Ground speed 

Item 3: Aircraft class (F = 757) 

Item 4: S (in magenta) = Aircraft is operating 

 in IM mode 

Line 3: Speed advisory (i.e., recommended 

 CAS) 

Slot marker circle with aircraft symbol “H” 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

11 

interval spacing behind the target upon reaching the FAF; and a “FAST/SLOW” indication, in 

knots, to provide trend information and guidance regarding required FIM aircraft decelerations 

and accelerations to conform with the ASTAR algorithm. In order to provide the pilot 

participants with key information in an easily viewable location, the AGD supplied the following 

three information elements during an IM operation: 

• A white “IM” light located in the upper left corner of the device that, when illuminated, 

indicated that the FIM aircraft was actively spacing relative to a target aircraft 

• The commanded speed (“CMD SPD”) required to achieve precise interval spacing behind 

a target aircraft 

• A “FAST/SLOW” indication that showed the difference between the FIM aircraft’s 

current speed and the speed that ASTAR expected 

Upon reaching the FAF, the ASTAR spacing algorithm switched to VREF mode, and the flight 

crew began decelerating to their landing speed. 

Appendix E provides an example IM clearance and illustrates the use of the EFB for data entry, 

activation, suspension, resumption, cancellation, and termination of FIM operations. 

Additionally, illustrations of the information elements presented on the AGD during FIM 

operations are provided in Appendix E. 

 Facilities and Equipment

The IM-NOVA experiment made use of two NASA LaRC facilities: the Air Traffic Operations 

Laboratory (ATOL) and the Integration Flight Deck (IFD). Descriptions of each facility and the 

equipment utilized during the IM-NOVA experiment are provided below. 

 Air Traffic Operations Laboratory (ATOL) 

The ATOL operates a network of hundreds of real-time, medium-fidelity aircraft simulators and 

utilizes a simulation platform, known as the Airspace and Traffic Operations Simulation 

(ATOS), which can be used for both batch and real-time HITL experiments. During the IMNOVA experiment, the ATOL provided 24 Aircraft Simulation for Traffic Operations Research 

(ASTOR) aircraft, four remote ASTOR pseudo-pilot (RAPP) stations, 25 simulated MultiAircraft Control System (MACS) aircraft, and four ATC controller stations. 

Aircraft Simulation for Traffic Operations Research (ASTOR) 

ASTOR components include: a six degrees-of-freedom aircraft model, primary flight display, 

multi-function display, autopilot and auto-throttle systems, flight management computer, multifunction control display unit, mode control panel, and ADS-B (ref. 28). During the IM-NOVA 

experiment, two researchers used a total of four Remote ASTOR Pseudo-Pilot (RAPP) stations 

(described below) to control the 24 ASTOR aircraft that provided multiple air traffic flows into 

DFW. 

Remote ASTOR Pseudo-Pilot (RAPP) stations 

The use of RAPP stations allowed a single operator, or pseudo-pilot, to control the basic 

functions of multiple ASTORs. Each RAPP station displays a GUI (shown in Figure 6) that 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

12 

allows the operator to view a selection list showing each simulated aircraft controlled by that 

RAPP station, a list of ATC frequencies that can be assigned, and the current airspeed, altitude, 

and heading of each aircraft as it is selected. To command a change in airspeed, heading, 

altitude, or frequency, the RAPP operator enters the desired change in the appropriate command 

window and clicks on the corresponding “Send” button. A message containing the desired 

command is then sent to the selected ASTOR. The sent message and a response from the 

ASTOR are displayed in the scratch pad at the bottom of the RAPP station GUI. 

Since the majority of the ASTORs in each scenario independently flew their assigned flight plan 

without ATC intervention, the RAPP station operators primarily handled the radio transmissions 

for their assigned aircraft. However, there were cases in each scenario where the RAPP operators 

executed ATC-directed speed changes to keep aircraft on the TMA arrival schedule. In the ATC 

Termination of FIM scenario, the RAPP station operator executed ATC heading, airspeed, and 

altitude changes required to vector the ASTOR for an approach to runway 13R, although the 

approach and landing were not actually executed. 

Figure 6. Remote ASTOR Pseudo-Pilot (RAPP) station graphical user interface. 

 Multi-Aircraft Control System (MACS) aircraft 

In addition to the arriving aircraft being simulated with ASTORs, each experiment scenario 

included 25 departing aircraft to provide a more realistic traffic picture for the air traffic 

controllers. The departing aircraft were simulated using the NASA ARC MACS software (ref. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

13 

29). These aircraft took off from DFW runways 17R and 18L at intervals ranging from 30 

seconds to 120 seconds and were flown solely by a computer-based pilot model. 

ATC controller stations

ATC controller stations equipped with MACS were utilized within the ATOL to enable 

confederate air traffic controllers to provide a realistic ATC environment. The confederate 

controllers used either standard Display System Replacement (DSR) or Standard Terminal 

Automation Replacement System (STARS) displays augmented with CMS decision support 

tools. 

A screenshot of the Center controller’s DSR display for an IM-NOVA scenario is shown in 

figure 7. FDBs associated with arrival aircraft are displayed in yellow and include early/late 

indicators. FDBs associated with departure aircraft are displayed in white. 

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

14 

Figure 7. IM-NOVA Center controller’s Display System Replacement (DSR) 

display. 

Figure 8 shows a screenshot of the STARS display provided on the IM-NOVA Feeder ATC 

station. White FDBs indicate aircraft owned by the Feeder controller. Green FDBs indicate 

aircraft that are owned by the Final controller, and yellow FDBs are associated with departure 

aircraft. Slot marker circles and speed advisories are displayed for all arrivals, and TMA 

timelines for both runways are displayed on the left and right sides of the STARS display. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

15 

Figure 8. IM-NOVA Feeder controller’s Standard Terminal Automation 

Replacement System (STARS) display. 

 Integration Flight Deck (IFD) 

The IFD is a full-scale simulator representative of a large commercial transport category aircraft 

and is driven by an appropriate aircraft dynamics mathematical model (ref. 30). The cockpit 

includes standard instruments representative of a line operations aircraft, and the cockpit’s visual 

system is a panorama system that provides 200° horizontal by 40° vertical field-of-view. During 

the IM-NOVA experiment, all pilot participants flew the IFD, and the visual scene used was the 

DFW terminal environment in a daytime setting. As noted previously, the IFD simulator was 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

16 

equipped with the ASTAR algorithm and the prototype FIM crew interface to enable flight crews 

to perform FIM operations. 

Figure 9 shows the positions of the EFBs and AGDs within the IFD. 

Figure 9. Integration Flight Deck (IFD) equipped with IM-NOVA’s prototype FIM 

crew interface.

Dependent Measures

To assess the acceptability of the procedures and test Hypotheses 1–7, qualitative data were 

collected from pilots via electronic questionnaires. Following each run, the pilots were asked to 

rate the level of workload during the scenario they had just completed using the MCH Subjective 

Workload Rating Scale. They were also asked to characterize the following: the acceptability of 

the use of the procedures in a voice communications environment, the completeness of the 

procedures, the usability of the crew interface, the acceptability of the data entry procedures, and 

the operational acceptability of the rate of speed commands. At the end of the second day of the 

two-day experiment session, each flight crew participated in an interactive group debrief session 

with the research team. 

To assess the feasibility of the procedures and test Hypotheses 7 and 8, quantitative data, 

including a spacing error at the FAF and the rate of ASTAR commanded speed changes, were 

collected during each run. Although the achieve-by point was the FAF, the spacing error at the 

runway threshold was an additional metric of interest. Since the ATD-1 procedures used in this 

experiment required that a complex voice clearance be issued to the flight crew by ATC using 

radio communications, data were collected during each run to allow an assessment of the time 

required for clearance issuance and read back, as well as the time required for the flight crew to 

enter information included in the clearance into the EFB.

 	
 
.

Captain’s electronic flight bag (EFB) and ADS-B guidance display (AGD)

First officer’s ADS-B guidance display (AGD) and electronic flight bag (EFB) 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

17 

3 Results and Discussion 

To assess the acceptability of the proposed ATD-1 flight crew procedures, subjective response 

data were collected in the form of acceptability, workload, and usability ratings via electronic 

post-run and post-experiment questionnaires (questionnaires are included in Appendix C and 

Appendix D). To assess the feasibility of the procedures, quantitative data were collected, 

including the rate of speed commands and the spacing error at the FAF. An a priori hypothesis 

associated with the spacing error at the runway threshold was not generated; however, a spacing 

error at the runway threshold was computed for each run. Similarly, a priori hypotheses 

associated with data collected during the exploratory scenario were not generated; however, 

descriptive statistics calculated using the exploratory scenario data are included in Table 2–Table 

13. 

Results from pilot participant crew #2, Scenario 5 (ADS-B Loss), with the first officer as PF and 

the captain as PM, were excluded from this analysis due to a simulation error.2

 Results 

associated with the time required for IM clearance issuance and read back as well as the time 

required for the flight crew to enter information included in the clearance into the EFB are 

reported elsewhere (ref. 31). 

 Acceptability of the Procedures 

Hypothesis 1 is as follows: the use of the procedures for receiving and executing IM 

clearances in a voice communications environment will be acceptable to the flight crew, 

with mean ratings of the procedures’ completeness and acceptability higher than 4.5 on a 

7-point scale. 

To assess Hypothesis 1, data from the following two items of the post-run questionnaires were 

used: 

5f. The use of voice communications to provide the IM clearance(s) was acceptable 

in this scenario. 

5j. The flight crew procedures for the events in this scenario were complete and 

acceptable. 

Descriptive statistics associated with the pilot participants’ acceptability ratings are shown in 

Table 2 and Table 3. Statistical analysis was performed using the Wilcoxon signed rank test, a 

nonparametric test appropriate for analyzing ordinal data (ref. 31). There were no statistically 

significant differences between the mean responses from the PF and PM for either questionnaire 

item 5f or 5j in any scenario (p ≥ 0.236),3

 except for questionnaire item 5j in the Nominal 

scenario (p = 0.036). In this case, the mean acceptability rating was slightly higher for the PM 

(mean = 6.9, SD = 0.3) than for the PF (mean = 6.5, SD = 0.9). For all five scenarios, both the PF 

and PM generally found the proposed ATD-1 procedures to be acceptable in a voice 

communications environment (p ≤ 0.002). However, concerns regarding the acceptability of the 

 

2

 Researchers responsible for communicating information regarding the target aircraft’s loss of ADS-B capability to 

pilot participant crew #2 failed to provide this information at the appropriate time during the experiment scenario. 

3

 A p-value < 0.05 indicates a statistically significant difference between sample means. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

18 

proposed procedures were raised by several pilot participants as evidenced by a total of 10 

ratings of 3 (Slightly Disagrees) or less recorded in response to questionnaire item 5f and a total 

of two ratings of 3 or less recorded in response to questionnaire item 5j. These low ratings of 

acceptability were consistently associated with pilots’ reported uneasiness with the issuance of 

IM clearances below 10,000 ft. Specific pilot participant comments referenced concerns with 

“task saturation below 10,000 ft” and a hesitancy to enter data associated with an IM clearance 

into an EFB while completing other tasks required within a potentially “high demand terminal 

environment.” 

Table 2. Descriptive statistics for acceptability ratings (from post-run 

questionnaire item 5f). 

PF PM 

Scenario N Mean SD Min Med Max Mean SD Min Med Max

Nominal 20 6.7 0.7 4 7 7 6.6 0.9 3 7 7 

Amend ASG 20 6.4 1.1 3 7 7 6.4 1.2 2 7 7 

ATC Terminates 20 6.8 0.9 3 7 7 6.6 1.0 3 7 7 

Suspend/Resume 20 6.9 0.4 6 7 7 6.7 0.7 5 7 7 

ADS-B Loss 19 6.2 1.5 2 7 7 6.1 1.6 2 7 7 

Exploratory 10 6.9 0.3 6 7 7 6.6 1.0 4 7 7 

Table 3. Descriptive statistics for acceptability ratings (from post-run 

questionnaire item 5j). 

PF PM 

Scenario N Mean SD Min Med Max Mean SD Min Med Max

Nominal 20 6.5 0.9 4 7 7 6.9 0.3 6 7 7 

Amend ASG 20 6.6 0.7 5 7 7 6.6 0.8 4 7 7 

ATC Terminates 20 6.7 0.6 5 7 7 6.6 0.9 3 7 7 

Suspend/Resume 20 6.5 0.8 5 7 7 6.8 0.5 5 7 7 

ADS-B Loss 19 6.5 1.0 4 7 7 6.3 1.4 2 7 7 

Exploratory 10 6.9 0.3 6 7 7 6.8 0.4 6 7 7 

 Completeness of the Procedures 

Hypothesis 2 is as follows: pilots will report that the procedures are complete (i.e., at 

least 90 percent of the pilots will report that the procedures are complete). 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

19 

To assess Hypothesis 2, data from the following three items of the post-run questionnaires were 

used: 

6a. The procedures did not contain missing steps. 

6b. The procedures did not contain extra steps that were unnecessary. 

6c. The procedural steps were logical and easy to follow. 

Table 4 shows the proportion of pilots who agreed with questionnaire items 6a, 6b, and 6c. Using 

Fisher’s exact test, there were no statistically significant differences between the proportion of 

PF and PM who reported the procedures were complete in any scenario (p ≥ 0.487) (ref. 31). 

Statistical analysis was performed using the binomial test of one proportion to test whether the 

proportion of pilots who reported the procedures were complete was at least 90 percent vs. less 

than 90 percent (ref. 31). For all five scenarios, the proportion of pilot participants who reported 

that the procedures contained no missing steps, no unnecessary steps, and that the steps were 

logical and easy to follow was not significantly less than 90 percent (p ≥ 0.580). This implies 

that the pilots felt the procedures were complete. 

Table 4. Percentage (%) of pilots who reported procedures were complete (from 

post-run questionnaire items 6a, 6b, and 6c). 

 

Questionnaire 

Item 6a 

Questionnaire 

Item 6b 

Questionnaire 

Item 6c 

Scenario N PF PM PF PM PF PM 

Nominal 20 95.0 100.0 100.0 90.0 95.0 100.0 

Amend ASG 20 95.0 95.0 95.0 90.0 100.0 100.0 

ATC Terminates 20 100.0 95.0 100.0 100.0 100.0 90.0 

Suspend/Resume 20 95.0 100.0 100.0 100.0 100.0 100.0 

ADS-B Loss 19 89.5 94.7 100.0 94.7 100.0 94.7 

Exploratory 10 100.0 100.0 100.0 90.0 100.0 100.0 

 Pilot Workload 

Hypothesis 3 is as follows: pilots will report the workload level required to execute the 

procedures to be acceptable, with a mean rating of less than 3 on the MCH rating scale. 

Hypothesis 4 is as follows: pilots will report no increase in workload with the Amend 

ASG, ATC Termination of FIM, Suspend/Resume FIM, or ADS-B Loss scenarios as 

compared to the Nominal scenario (i.e., a difference of less than 1 unit on the MCH 

rating scale). 

To assess Hypotheses 3 and 4, workload rating data collected using the MCH flow chart 

presented in the post-run questionnaires were used. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

20 

Descriptive statistics associated with the pilot participants’ workload ratings are shown in Table 

5. Statistical analysis was performed using the Wilcoxon signed rank test. There were no 

statistically significant differences between the mean responses from the PF and PM in any 

scenario (p ≥ 0.221). For all five scenarios, both the PF and PM found the workload level 

required to execute the procedures to be acceptable (p < 0.0005). Pilots also reported no 

significant increase in the workload for the Amend ASG, ATC Termination of FIM, 

Suspend/Resume FIM, or ADS-B Loss scenarios as compared to the Nominal scenario for either 

PF or PM (p ≤ 0.003). 

Table 5. Descriptive statistics for pilot workload ratings (from post-run 

questionnaire item 3). 

PF PM 

Scenario N Mean SD Min Med Max Mean SD Min Med Max

Nominal 20 1.6 0.7 1 1 3 1.5 0.6 1 1 3 

Amend ASG 20 1.6 0.6 1 2 3 1.3 0.5 1 1 2 

ATC Terminates 20 1.5 0.5 1 1 2 1.6 0.6 1 2 3 

Suspend/Resume 20 1.5 0.7 1 1 3 1.5 0.6 1 1 3 

ADS-B Loss 19 1.7 0.7 1 2 3 1.8 0.8 1 2 3 

Exploratory 10 1.3 0.5 1 1 2 1.3 0.7 1 1 3 

 Usability of the Crew Interface 

Hypothesis 5 is as follows: pilots will report that the crew interface was usable, with 

mean ratings of the acceptability of heads-down time required and usability of the 

displays will be higher than 4.5 on a 7-point scale. 

To assess Hypothesis 5, data from the following two items of the post-run questionnaires were 

used: 

5g. The amount of heads-down time required to input information from the IM clearance(s) 

into the EFB was acceptable. 

5i. During this scenario, it was easy to obtain needed information from the IM displays. 

Descriptive statistics associated with the pilot participants’ ratings of the amount of heads-down 

time and display usability are shown in Table 6 and Table 7. Statistical analysis was performed 

using the Wilcoxon signed rank test. There were no statistically significant differences between 

the mean responses from the PF and PM for either questionnaire item 5g or 5i in any scenario (p

≥ 0.141). For all five scenarios, pilots reported the amount of heads-down time required to input 

information from the IM clearance(s) into the EFB was acceptable (p ≤ 0.001), and found that it 

was easy to obtain needed information from the IM displays (p < 0.0005). This indicates that the 

pilots generally felt that the prototype FIM crew interface was usable. However, several pilots 

expressed concerns regarding the usability of the crew interface as evidenced by a total of six 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

21 

ratings of 3 or less recorded in response to questionnaire item 5g and a total of four ratings of 3 

in response to questionnaire item 5i. Specific concerns focused on the amount of heads-down 

time required to input information from an IM clearance into the EFB during operations 

occurring below 10,000 ft and the limited amount of pertinent information presented within the 

forward field of view by the AGD. 

Table 6. Descriptive statistics for ratings of the amount of heads-down time 

(from post-run questionnaire item 5g). 

PF PM 

Scenario N Mean SD Min Med Max Mean SD Min Med Max

Nominal 20 6.5 0.7 5 7 7 6.8 0.4 6 7 7 

Amend ASG 20 6.6 1.0 3 7 7 6.6 0.7 5 7 7 

ATC Terminates 20 6.6 0.8 4 7 7 6.3 1.4 1 7 7 

Suspend/Resume 20 6.6 0.8 4 7 7 6.8 0.6 5 7 7 

ADS-B Loss 19 6.0 1.4 3 7 7 6.1 1.6 2 7 7 

Exploratory 10 6.4 1.0 4 7 7 6.6 0.7 5 7 7 

Table 7. Descriptive statistics for ratings of the usability of the displays (from 

post-run questionnaire item 5i). 

PF PM 

Scenario N Mean SD Min Med Max Mean SD Min Med Max

Nominal 20 6.5 1.0 3 7 7 6.6 0.7 5 7 7 

Amend ASG 20 6.7 0.5 6 7 7 6.4 1.1 3 7 7 

ATC Terminates 20 6.6 0.7 5 7 7 6.7 0.7 5 7 7 

Suspend/Resume 20 6.7 0.6 5 7 7 6.7 0.6 5 7 7 

ADS-B Loss 19 6.3 1.1 3 7 7 6.5 1.1 3 7 7 

Exploratory 10 6.9 0.3 6 7 7 6.8 0.4 6 7 7 

 Acceptability of Data Entry Procedures 

Hypothesis 6 is as follows: the data entry procedures will be acceptable to the flight crew, 

with mean ratings of ease and intuitiveness of entering information being higher than 4.5 

on a 7-point scale. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

22 

To assess Hypothesis 6, data from the following post-run questionnaire item were used: 

5h. During this scenario, entering IM clearance information into the EFB was easy 

and intuitive. 

Descriptive statistics associated with the pilot participants’ ratings of the ease and intuitiveness 

of entering information are shown in Table 8. Statistical analysis performed using the Wilcoxon 

signed rank test found no statistically significant differences between the mean responses from 

the PF and PM in any scenario (p ≥ 0.100). For all five scenarios, pilots found the data entry 

procedures to be acceptable (p < 0.0005). However, three ratings of 3 recorded in response to 

questionnaire item 5h were associated with concerns expressed regarding the input of data into 

the EFB at low altitudes within the terminal environment. 

Table 8. Descriptive statistics for ratings of the ease and intuitiveness of entering 

information (from post-run questionnaire item 5h). 

PF PM 

Scenario N Mean SD Min Med Max Mean SD Min Med Max

Nominal 20 6.6 0.8 4 7 7 6.9 0.3 6 7 7

Amend ASG 20 6.4 1.1 4 7 7 6.5 1.0 4 7 7

ATC Terminates 20 6.6 0.9 4 7 7 6.6 1.0 3 7 7

Suspend/Resume 20 6.7 0.7 4 7 7 6.8 0.4 6 7 7

ADS-B Loss 19 6.3 1.2 4 7 7 6.3 1.3 3 7 7

Exploratory 10 6.6 1.0 4 7 7 6.8 0.6 5 7 7

 Rate of Speed Commands 

Hypothesis 7 is as follows: the rate of speed commands will be acceptable to the flight 

crew, with speed commands occurring at a rate of less than two per minute, and mean 

ratings of operational acceptability and acceptability of the frequency of speed commands 

being higher than 4.5 on a 7-point scale, 

To assess Hypothesis 7, the rate of speed commands during each segment of flight as well as 

data from the following two post-run questionnaire items were used: 

5c. The IM commanded speeds were operationally acceptable and appropriate. 

5d. The frequency of the IM speed commands was acceptable at all times throughout the 

scenario. 

For each run, the rate of speed commands was computed. Table 9 presents the mean rate of 

speed commands over the entire flight, as well as for each segment of flight: from Flight Level 

(FL) 240 to FL180, FL180 to 11,000 ft, 11,000 ft to 6,000 ft, and 6,000 ft to the FAF. Statistical 

analysis was performed using the one-sample Poisson rate test, a hypothesis test appropriate for 

analyzing the number of occurrences of an event in a given length of time (ref. 33). For all five 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

23 

scenarios, the mean rate of speed commands was less than two per minute for each flight 

segment (p < 0.0005). 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

24 

Table 10 and 

Table 11 show descriptive statistics associated with the pilot participants’ ratings for 

questionnaire items 5c and 5d. Statistical analysis was performed using the Wilcoxon signed 

rank test. There were no statistically significant differences between the mean responses from the 

PF and PM for either questionnaire item 5c or 5d in any scenario (p ≥ 0.205). For all five 

scenarios, pilots generally reported that the IM commanded speeds were operationally acceptable 

and appropriate (p ≤ 0.001), and pilots found the frequency of the IM speed commands to be 

acceptable (p ≤ 0.001). When referencing Table 9 and 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

25 

Table 10, note that only two of the 20 pilots provided ratings of 3 or less during the experiment 

scenarios. One pilot commented that the issuance of a new clearance below 10,000 ft during the 

ADS-B Loss scenario resulted in too much heads-down time by the PM, and comments from the 

other pilot indicated that two speed commands occurred in less than five seconds during two of 

the scenarios. 

Table 9. Mean rate of speed commands (number of speed commands per 

minute) for each segment of flight. 

Scenario N FL240 to 180 

FL180 to 

11,000 ft 

11,000 ft to 

6,000 ft 

6,000 ft to 

FAF Total 

Nominal 20 0.22 0.44 0.35 0.76 0.49 

Amend ASG 20 0.09 0.65 0.53 0.67 0.54 

ATC Terminates 20 0.12 0.50 n/a 0.67 0.53 

Suspend/Resume 20 0.13 0.40 0.35 0.61 0.43 

ADS-B Loss 19 n/a* 0.43 n/a 0.72 0.61 

Exploratory 10 0.29 0.60 0.63 0.70 0.59 

*Note that mean rate = “n/a” indicates aircraft was conducting FIM operations less than 75% of the time. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

26 

Table 10. Descriptive statistics for ratings of operational acceptability and 

appropriateness of IM commanded speeds (from post-run 

questionnaire item 5c). 

PF PM 

Scenario N Mean SD Min Med Max Mean SD Min Med Max

Nominal 20 6.3 1.3 2 7 7 6.8 0.4 6 7 7 

Amend ASG 20 6.9 0.4 6 7 7 6.7 0.6 5 7 7 

ATC Terminates 20 6.6 0.8 4 7 7 6.5 1.4 1 7 7 

Suspend/Resume 20 6.5 0.9 4 7 7 6.8 0.4 6 7 7 

ADS-B Loss 19 6.3 1.5 1 7 7 6.4 1.3 2 7 7 

Exploratory 10 6.7 0.7 5 7 7 6.5 0.7 5 7 7 

Table 11. Descriptive statistics for ratings of acceptability of the frequency of 

the IM speed commands (from questionnaire item 5d).

PF PM 

Scenario N Mean SD Min Med Max Mean SD Min Med Max

Nominal 20 6.6 0.6 5 7 7 6.7 0.5 6 7 7 

Amend ASG 20 6.6 0.8 5 7 7 6.5 0.9 4 7 7 

ATC Terminates 20 6.5 0.8 4 7 7 6.2 1.4 1 7 7 

Suspend/Resume 20 6.6 0.8 4 7 7 6.6 0.8 4 7 7 

ADS-B Loss 19 6.4 1.5 1 7 7 6.5 1.1 3 7 7 

Exploratory 10 6.6 0.7 5 7 7 6.3 1.3 3 7 7 

 Spacing Error at the Final Approach Fix 

For each run, the spacing error at the FAF was recorded to assess the following a priori 

hypothesis: 

Hypothesis 8: The spacing error at the FAF will be within ±5 sec with a standard 

deviation of 5 sec. 

Descriptive statistics associated with the spacing error at the FAF are provided in Table 12, and 

histograms for each scenario are shown in Figure 150–Figure 19. The one-sample t-test and onesample variance test were used to test the mean and standard deviation, respectively (ref. 33). For 

all five scenarios, the spacing error at the FAF had a mean within ±5 sec (p < 0.0005) and a 

standard deviation less than 5 sec (p < 0.0005). 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

27 

Table 12. Descriptive statistics for spacing error at the FAF (sec). 

Scenario N Mean SD Min Median Max 

Nominal 20 –1.7 1.0 –4.1 –1.5 –0.1 

Amend ASG 20 –1.8 1.3 –5.3 –1.6 0.5 

ATC Terminates 20 –0.8 1.1 –3.4 –0.9 0.9 

Suspend/Resume 20 1.2 1.5 –1.7 1.5 3.4 

ADS-B Loss 19 –0.5 0.9 –1.6 –0.7 1.7 

Exploratory 10 –1.3 1.4 –3.0 –1.4 1.5 

-6 -4 -2 0 2 4 6

60

50

40

30

20

10

0

Spacing Error at the FAF (sec)

Percent of Flights (%)

Nominal

Figure 10. Spacing error at the FAF 

(sec) for the Nominal 

scenario. 

-6 -4 -2 0 2 4 6

60

50

40

30

20

10

0

Spacing Error at the FAF (sec)

Percent of Flights (%)

Amend ASG

Figure 11. Spacing error at the FAF 

(sec) for the Amend ASG 

scenario.

-6 -4 -2 0 2 4 6

60

50

40

30

20

10

0

Spacing Error at the FAF (sec)

Percent of Flights (%)

ATC Terminates

Figure 12. Spacing error at the FAF 

(sec) for the ATC 

Termination of FIM 

scenario. 

-6 -4 -2 0 2 4 6

60

50

40

30

20

10

0

Spacing Error at the FAF (sec)

Percent of Flights (%)

Suspend/Resume

Figure 13. Spacing error at the FAF 

(sec) for the 

Suspend/Resume 

scenario. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

28 

-6 -4 -2 0 2 4 6

60

50

40

30

20

10

0

Spacing Error at the FAF (sec)

Percent of Flights (%)

ADS-B Loss

Figure 14. Spacing error at the FAF 

(sec) for the ADS-B Loss 

scenario. 

 Spacing Error at the Runway Threshold 

The spacing error at the runway threshold was computed for each run. Table 13 gives descriptive 

statistics associated with this metric, and Figure 15–Figure 19 show histograms for each 

scenario. For all five scenarios, the observed mean spacing error at the runway threshold was 

within ±2 sec, and the observed standard deviation was less than 3 sec. 

Table 13. Descriptive statistics for spacing error at the runway threshold (sec). 

Scenario N Mean SD Min Med Max 

Nominal 20 –0.7 2.2 –4.1 –0.6 3.8 

Amend ASG 20 –0.5 2.9 –8.5 –0.2 5.0 

ATC Terminates 20 –1.6 2.3 –6.3 –1.7 1.9 

Suspend/Resume 20 0.4 2.9 –5.8 0.8 5.3 

ADS-B Loss 19 –1.3 2.5 –5.1 –1.8 5.1 

Exploratory 10 –0.7 2.0 –4.8 0.2 0.9 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

29 

-8 -6 -4 -2 0 2 4

30

25

20

15

10

5

0

Spacing Error at the Runway Threshold (sec)

Percent of Flights (%)

Nominal

Figure 15. Spacing error at the 

runway threshold (sec) for 

the Nominal scenario. 

-8 -6 -4 -2 0 2 4

30

25

20

15

10

5

0

Spacing Error at the Runway Threshold (sec)

Percent of Flights (%)

Amend ASG

Figure 16. Spacing error at the 

runway threshold (sec) for 

the Amend ASG scenario. 

-8 -6 -4 -2 0 2 4

30

25

20

15

10

5

0

Spacing Error at the Runway Threshold (sec)

Percent of Flights (%)

ATC Terminates

Figure 17. Spacing error at the 

runway threshold (sec) for 

the ATC Termination of 

FIM scenario. 

-8 -6 -4 -2 0 2 4

30

25

20

15

10

5

0

Spacing Error at the Runway Threshold (sec)

Percent of Flights (%)

Suspend/Resume

Figure 18. Spacing error at the 

runway threshold (sec) for 

the Suspend/Resume 

scenario. 

-8 -6 -4 -2 0 2 4

30

25

20

15

10

5

0

Spacing Error at the Runway Threshold (sec)

Percent of Flights (%)

ADS-B Loss

Figure 19. Spacing error at the 

runway threshold (sec) for 

the ADS-B Loss scenario. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

30 

4 Conclusions 

NASA has developed a set of ground-based and airborne arrival management technologies, 

including TMA-TM, CMS decision support tools, and FIM avionics and procedures. The 

integration of these technologies will increase throughput, reduce delay, and minimize 

environmental impacts. ATD-1 will operationally demonstrate the efficient arrival operations 

provided by this integrated system of NextGen technologies. 

The HITL experiment described in Section 1.1 was conducted as part of initial preparations for 

an ATD-1 flight demonstration. It was designed to assess the acceptability and feasibility of the 

proposed air/ground procedures in a voice communications environment. Five flight scenarios 

were defined to allow flight crews to fully exercise the procedures during different flight phases 

and operational events. These scenarios consisted of a nominal IM clearance flown to landing, an 

amended IM clearance on arrival, a terminated IM clearance with a reissue of a new clearance, a 

suspension and resumption of the IM clearance, and a system error causing a flight crew 

termination of IM (ADS-B loss) with a subsequent new clearance issued at low altitude (below 

10,000 ft). 

Overall, the FIM crew procedures were deemed acceptable and feasible for use by the flight crew 

in all scenarios and phases of flight flown in the experiment. Analyses of qualitative data 

revealed that the procedures used by flight crews to receive and execute IM clearances in a voice 

communications environment were logical, easy to follow, did not contain any missing or 

extraneous steps, and required the use of an acceptable level of workload. The majority of the 

pilot participants found the IM concept and the proposed FIM crew procedures to be acceptable 

and indicated that the ATD-1 procedures could be successfully executed in a near-term NextGen 

environment. Analyses of quantitative data revealed that FIM speed commands occurred at a rate 

of less than two per minute, and pilot participants found the frequency of the speed commands to 

be acceptable at all times throughout the experiment scenarios. Pilots also reported that the IM 

commanded speeds were operationally acceptable and appropriate during all scenarios. In 

addition, the delivery accuracy at both the FAF and the runway threshold was within ±5 sec, and 

the delivery precision was less than 5 sec. The results of this experiment demonstrate the 

effectiveness of the airborne spacing algorithm and the air/ground procedures investigated. 

Future research is planned to investigate the effects of winds, weather, and turbulence on the 

acceptability and feasibility of the ATD-1 air/ground procedures. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

31 

5 References 

1. Federal Aviation Administration, “FAA Aerospace Forecast, Fiscal Years 2012–2032,” 

URL: 

http://www.faa.gov/about/office_org/headquarters_offices/apl/aviation_forecasts/aerospace_f

orecasts/2012-2032/ [cited 12 June 2013]. 

2. Federal Aviation Administration, “NextGen Implementation Plan March 2012,” URL: 

http://www.faa.gov/nextgen/implementation/media/NextGen_Implementation_Plan_2012.pd

f [cited 12 June 2013]. 

3. Prevot, T., Baxley, B. T., Callantine, T. J., Johnson, W. C., Quon, L. K., Robinson, J. E., 

Swenson, H. N., “NASA’s ATM Technology Demonstration-1: Transitioning Fuel Efficient, 

High Throughput Arrival Operations from Simulation to Reality,” International Conference 

on Human-Computer Interaction in Aerospace (HCI-Aero), Brussels, Belgium, September 

2012. 

4. Baxley, B. T., Swenson, H. N., Prevot, T., Callantine, T. J., “NASA’s ATM Technology 

Demonstration-1: Integrated Concept of Arrival Operations,” 31st Digital Avionics Systems 

Conference (DASC), Williamsburg, VA, October 2012. 

5. Baxley, B. T., Johnson, W. C., Swenson, H. N., Robinson, J. E., Prevot, T., Callantine, T. J., 

Scardina, J., and Greene, M., “Air Traffic Management Technology Demonstration-1 

Concept of Operations (ATD-1 ConOps),” NASA/TM-2012-217585, July 2012. 

6. Swenson, H. N., Hoang, T., Engelland, S., Vincent, D., Sanders, T., Sanford, B., and Heere, 

K., “Design and Operational Evaluation of the Traffic Management Advisor at the Fort 

Worth Air Route Traffic Control Center,” First USA/Europe Air Traffic Management 

Research and Development Seminar (ATM1997), Saclay, France, June 1997. 

7. Kupfer, M., Callantine, T., Martin, L., Mercer, J., and Palmer, E., “Controller Support Tools 

for Schedule-Based Terminal-Area Operations,” Ninth USA/Europe Air Traffic Management 

Research and Development Seminar (ATM2011), Berlin, Germany, June 2011. 

8. Martin, L., Kupfer, M., Palmer, E., Mercer, J., Callantine, T., and Prevot, T., “Acceptability 

and Effects of Tools to Assist with Controller Managed Spacing in the Terminal Area,” 

Proceedings of the 9th International Conference on Engineering Psychology and Cognitive 

Ergonomics, edited by D. Harris, Springer-Verlag, Berlin, Heidelberg, 2011, pp. 432–441. 

9. Swenson, H. N., Thipphavong, J., Sadovsky, A., Chen, L., Sullivan, C., and Martin, L., 

“Design and Evaluation of the Terminal Area Precision Scheduling and Spacing System,” 

Ninth USA/Europe Air Traffic Management Research and Development Seminar 

(ATM2011), Berlin, Germany, June 2011. 

10. Thipphavong, J., Swenson, H., Lin, P., Seo, A. Y., and Bagasol, L. N., “Efficiency Benefits 

Using the Terminal Area Precision Scheduling and Spacing System,” 11th AIAA Aviation 

Technology, Integration, and Operations (ATIO) Conference, AIAA Paper 2011-6971, 

Virginia Beach, VA, September 2011. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

32 

11. Martin, L., Swenson, H., Sadovsky, A., Thipphavong, J., Chen, L., and Seo, A. Y., “Effects 

of Scheduling and Spacing Tools on Controllers’ Performance and Perceptions of Their 

Workload,” 30th Digital Avionics Systems Conference (DASC), Seattle, WA. October 2011. 

12. Thipphavong, J., Martin, L., Swenson, H., Lin, P., and Nguyen, J., “Evaluation of the 

Terminal Area Precision Scheduling and Spacing System for Near-Term NAS Application,” 

International Conference on Applied Human Factors and Ergonomics (AHFE), San 

Francisco, CA, July 2012. 

13. Bone, R. S., and Penhallegon, W. J., “En-route Flight Deck-Based Merging and Spacing 

Impact on Flight Crew Operations,” 26th Digital Avionics Systems Conference (DASC), 

Dallas, TX, October 2007. 

14. Bone, R. S., Penhallegon, W. J., and Stassen, H. P., “Flight Deck-Based Merging and 

Spacing during Continuous Descent Arrivals and Approach: Impact on Pilots,” MITRE, 

MTR080208, July 2008. 

15. Penhallegon, W. J., and Bone, R. S., “Flight Deck-Based Merging and Spacing Impact on 

Flight Crew Operations During Continuous Descent Arrivals and Approaches,” 27th Digital 

Avionics Systems Conference (DASC), St. Paul, MN, October 2008. 

16. Hebraud, C., Hoffman, E., Papin, A., Pene, N., Rognin, L., Sheehan, C., and Zeghal, K., 

“CoSpace 2002 Flight Deck Experiments Assessing the Impact of Spacing Instructions from 

Cruise to Initial Approach,” Eurocontrol Experimental Centre, EEC Report No. 388, Volume 

I, February 2004. 

17. Hebraud, C., Hoffman, E., Pene, N., Rognin, L., and Zeghal, K., “Assessing the Impact of a 

New Air Traffic Control Instruction on Flight Crew Activity,” AIAA Guidance, Navigation, 

and Control Conference and Exhibit, AIAA Paper 2004-5104, Providence, RI, August 2004. 

18. Hoffman, E., Pene, N., Rognin, L., and Zeghal, K., “Introducing a New Spacing Instruction. 

Impact of Spacing Tolerance on Flight Crew Activity,” Human Factors and Ergonomics 

Society Annual Meeting Proceedings, Vol.47, 2003, pp. 174–178. 

19. Hoffman, E., Martin, P., Putz, T., Trzmiel, A., and Zeghal, K., “Airborne Spacing: Flight 

Deck View of Compatibility with Continuous Descent Approach (CDA),” 7th AIAA 

Aviation Technology, Integration and Operations (ATIO) Conference, AIAA Paper 2007-

7742, Belfast, Northern Ireland, September 2007. 

20. Baxley, B. T., Barmore, B. E., Abbott, T. S., and Capron, W. R., “Operational Concept for 

Flight Crews to Participate in Merging and Spacing of Aircraft,” 6th AIAA Aviation 

Technology, Integration and Operations (ATIO) Conference, AIAA Paper 2006-7722, 

Wichita, KS, September 2006. 

21. Murdoch, J. L., Barmore, B. E., Baxley, B. T., Abbott, T. S., and Capron, W. R., “Evaluation 

of an Airborne Spacing Concept to Support Continuous Descent Arrival Operations,” Eighth 

USA/Europe Air Traffic Management Research and Development Seminar (ATM2009), 

Napa, CA, June–July, 2009. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

33 

22. Swieringa, K., Murdoch, J. L., Baxley, B., and Hubbs, C., “Evaluation of an Airborne 

Spacing Concept, On-board Spacing Tool, and Pilot Interface,” 11th AIAA Aviation 

Technology, Integration, and Operations (ATIO) Conference, AIAA Paper 2011-6902, 

Virginia Beach, VA, September 2011. 

23. Cabrall, C., Callantine, T., Kupfer, M., Martin, L., and Mercer, J., “Controller-Managed 

Spacing within Mixed-Equipage Arrival Operations Involving Flight-Deck Interval 

Management,” International Conference on Applied Human Factors and Ergonomics 

(AHFE), San Francisco, CA, July 2012. 

24. Callantine, T. J., Cabrall, C. D., Kupfer, M., Omar, F. G., and Prevot, T., “Initial 

Investigations of Controller Tools and Procedures for Schedule-Based Arrival Operations 

with Mixed Flight-Deck Interval Management Equipage,” 12th AIAA Aviation Technology, 

Integration, and Operations (ATIO) Conference, AIAA Paper 2012-5673, Indianapolis, IN, 

September 2012. 

25. Thipphavong, J., Jung, J., Swenson, H. V., Lin, M. I., Nguyen, J., Martin, L., Downs, M. B., 

and Smith, T. A., “Evaluation of the Controller-Managed Spacing Tools, Flight-deck Interval 

Management and Terminal Area Metering Capabilities for the ATM Technology 

Demonstration #1,” Tenth USA/Europe Air Traffic Management Research and Development 

Seminar (ATM2013), Chicago, IL, June 2013. 

26. Abbott, T. S., “An Overview of a Trajectory-Based Solution for En Route and Terminal Area 

Self-Spacing: Third Revision,” NASA/CR-2012-217786, November 2012. 

27. Wierwille, W. and Casali, J., “A Valid Rating Scale for Global Mental Workload 

Measurement,” Proceedings of the Human Factors Society 27th Annual Meeting, pp. 129-

133, Chicago, IL, October 1983. 

28. Peters, M. E., Ballin, M. G., and Sakosky, J. S., “A Multi-Operator Simulation for 

Investigation of Distributed Air Traffic Management Concepts,” AIAA Modeling and 

Simulation Technologies Conference and Exhibit, AIAA Paper 2002-4596, Monterey, CA, 

August 2002. 

29. Prevot, T., Smith, N., Palmer, E., Mercer, J., Lee, P., Homola, J., and Callantine, T., “The 

Airspace Operations Laboratory (AOL) at NASA Ames Research Center,” AIAA Modeling 

and Simulation Technologies Conference and Exhibit, AIAA Paper 2006-6112, Keystone, 

CO, August 2006. 

30. National Aeronautics and Space Administration, “NASA Langley Research Center: The 

Flight Simulation Facilities,” URL: 

http://oim.hq.nasa.gov/oia/scap/docs/SCAP_FLIGHTSIM_112508_508.pdf [cited 12 June 

2013]. 

31. Wilson, S. R., Leonard, R. D., Edwards, D. J., Swieringa, K. A., and Murdoch, J. L., “Model 

Specification and Confidence Intervals for Voice Communication,” Quality Engineering

(accepted), 2015. 

32. Hollander, M. and Wolfe, D. A., Nonparametric Statistical Methods, 2nd ed., John Wiley & 

Sons, New York, NY, 1999. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

34 

33. Casella, G. and Berger, R. L., Statistical Inference, 2nd ed., Duxbury Press, Pacific Grove, 

CA, 2002. 

34. Vining, G. G., Statistical Methods for Engineers, 1st ed. Duxbury Press, Pacific Grove, CA, 

1998. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

35 

Appendix A. Simulated Airspace and Scenario Design 

The Interval Management for Near-Term Operations Validation of Acceptability (IM-NOVA) 

experiment’s common scenario consisted of 25 arrival aircraft created at various points on the 

arrival routes into Dallas-Fort Worth (DFW) airport and landing on runways 17C and 18R. To 

provide a realistic traffic environment, each scenario also included 25 departure aircraft 

generated by the Multi-Aircraft Control System (MACS) developed at NASA Ames Research 

Center (ARC). 

All IM-NOVA inbound aircraft approached the DFW airport via one of four standard terminal 

arrival routes (STARs) with a total of 10 different transitions. The distribution of aircraft over the

STARs/Transitions is shown in Table A-1, and the research STAR charts are included in Figure 

A-1–Figure A-4. 

Table A-1. Distribution of aircraft over the STARs/Transitions. 

STAR Transition Number of Aircraft 

Bonham Five Fort Smith 3 

 Little Rock 2 

 Paris 1 

Bowie One Borger 5 

 Texico 2 

Cedar Creek Six Alexandria 2 

 Gregg County 2 

 Humble 2 

Glenn Rose Nine San Antonio 5 

 Wink 1 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

36 

Figure A-1. NASA Research STAR for the Bonham Five Arrival into DFW.
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

37 

Figure A-2. NASA Research STAR for the Bowie One Arrival into DFW.
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

38 

Figure A-3. NASA Research STAR for the Cedar Creek Six Arrival into DFW.
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

39 

Figure A-4. NASA Research STAR for the Glen Rose Nine Arrival into DFW. 

Each inbound aircraft was represented by an instantiation of the NASA Langley Research Center 

(LaRC) Aircraft Simulation for Traffic Operations Research (ASTOR). Each ASTOR 

instantiation represents a commercial transport aircraft, its flight deck systems, and the airborne 

components of a realistic future communications, navigation, and surveillance infrastructure and 

runs on a single Windows XP workstation. Some aircraft initialized in level cruise and flew the 

full arrival and approach to the runway, while others initialized in descent and flew only a 

portion of the arrival before flying the approach. All aircraft started outside the DFW Terminal 

Radar Approach Control (TRACON) boundary so that the Traffic Management Advisor with 

Terminal Metering (TMA-TM) could develop a workable arrival schedule. 

 Five experiment scenarios were developed from a common scenario in order to assess the 

viability of the IM spacing procedures during each of the operational scenarios defined in the 

ATD-1 Concept of Operations (ConOps). Each experiment scenario included the same 25 aircraft 

defined in the common scenario, with one ASTOR in each scenario replaced by the NASA LaRC 

Integration Flight Deck (IFD): a full-scale simulator representative of a large commercial 

transport category aircraft flown by the pilot participants. The pilot participants’ (i.e., Flight 

Deck-based Interval Management (FIM)) aircraft were changed for each scenario to provide 

variety, exercise all use cases defined in the ATD-1 ConOps, and keep the scenario length to less 

than 30 minutes. The closest aircraft in the arrival stream for the same runway as the FIM aircraft 

was designated as the initial IM spacing target. If a new target was designated later in the 

scenario, it was always the next closest aircraft in the arrival stream. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

40 

The experiment scenarios were as follows: 

1. Nominal Scenario: ATC issued an IM clearance before top of descent (TOD). After the 

FIM aircraft reported that spacing had commenced, the FIM aircraft maintained nominal 

IM spacing operations until passing the final approach fix (FAF). 

CQY6

UKW1

JEN9

BYP5

FSM

AEX

BGD

IAH

INK GGG

LIT

PRX

TXO

SAT

Scenario 1

BTA2722

Target

NASA90 FIM

IM clearance prior to TOD; 

(Center controller)

Figure A-5. Nominal Scenario. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

41 

2. Amend Assigned Spacing Goal (ASG): The initial IM clearance was issued shortly after 

TOD. Approximately two minutes after the FIM aircraft reported that spacing had 

commenced, ATC issued an amended clearance to increase spacing by 20 seconds. 

CQY6

UKW1

JEN9

BYP5

FSM

AEX

BGD

IAH

INK GGG

LIT

PRX

TXO

SAT

Scenario 2

NASA65 FIM

IM clearance ATC 

amend ASG (Center)

AA544

Target

Figure A-6. Amend ASG Scenario. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

42 

3. ATC Termination of FIM: The initial IM clearance was issued shortly after TOD. After 

both the target and FIM aircraft were well inside the DFW TRACON, ATC cancelled the 

IM clearance and vectored the target aircraft for landing on runway 13R. ATC then 

issued a new IM clearance with a new target for the FIM aircraft. 

CQY6

UKW1

JEN9

BYP5

FSM

AEX

BGD

IAH

INK GGG

LIT

PRX

TXO

SAT

Scenario 3

NASA46 FIM

IM clearance; Cancel IM & issue 

new clearance (diff route; 

TRACON)

MXA848

2

nd Target

AA1274

1

st Target

Figure A-7. ATC Termination of FIM Scenario. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

43 

4. Suspend / Resume FIM: The initial IM clearance was issued shortly after TOD. After the 

FIM aircraft reported that spacing had commenced, ATC suspended the IM clearance and 

issued a speed change of 20 kt for the FIM aircraft. Approximately two minutes later, 

ATC cleared the FIM aircraft to resume IM spacing. 

CQY6

UKW1

JEN9

BYP5

FSM

AEX

BGD

IAH

INK GGG

LIT

PRX

TXO

SAT

Scenario 4

NASA81 FIM

IM clearance; Suspend IM & 

issue speed change, Resume 

IM

AA1739

2

nd Target

AA1394

1

st Target

Figure A-8. Suspend / Resume FIM Scenario. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

44 

5. ADS-B Loss: ATC issued the initial IM clearance before TOD. After the FIM aircraft 

reported that spacing had commenced and both aircraft were well within the DFW 

TRACON, the target aircraft experienced a loss of ADS-B capability. After the pilot 

participants notified ATC that they were “IM Unable” due to the loss of the target 

aircraft’s ADS-B signal, ATC cancelled the initial IM clearance and issued a new 

clearance with a new target for the FIM aircraft. 

CQY6

UKW1

JEN9

BYP5

FSM

AEX

BGD

IAH

INK GGG

LIT

PRX

TXO

SAT

Scenario 5

UPS2977

1

st Target

EGF2850

2

nd Target

NASA81 FIM IM 

clearance; Crew 

UNABLE (Target 

loses ADS-B 

(TRACON))

Figure A-9. ADS-B Loss Scenario. 

In addition to the five experiment scenarios, three training scenarios were developed to allow the 

pilot participants to practice the IM spacing procedures prior to beginning the experiment’s data 

collection runs. These training scenarios contained all of the elements of the experiment 

scenarios but with a different sequence of events and different initial conditions. The exploratory 

scenario that pilot participants were asked to fly after completing all experiment scenarios 

consisted of the Amend ASG scenario with the addition of aural alerts associated with 

commanded speed changes from the Airborne Spacing for Terminal Area Routes (ASTAR) 

algorithm. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

45 

Appendix B. Experiment Run Order 

Each two-person crew (with both members employed by the same airline) participated in a twoday experiment session. Every crew flew each scenario twice: once with the captain as the pilot 

flying (PF) and the first officer as the pilot monitoring (PM), and once with the first officer as the

PF and the captain as the PM. One additional exploratory scenario was created to examine the 

effect of aural alerts associated with Airborne Spacing for Terminal Area Routes (ASTAR) 

algorithm commanded speed changes. Therefore, each crew flew a total of 11 runs. 

This experiment utilized two 2-replicate Latin square designs4 with “crew” and “run order” as 

the two blocking factors (see Table B-1). The first 2-replicate Latin square design was used to 

partially counterbalance the run order of the scenarios with the captain as PF and the first officer 

as PM. The second 2-replicate Latin square design was used for the scenarios with the first 

officer as PF and the captain as PM. These two designs are shown below, where “crew” 

represents the row blocks and “run number” represents the column blocks. Within each crew, the 

pilots switched responsibilities between runs. For example, crew 1 flew scenario 5 with the 

captain as PF, then scenario 4 with the first officer as PF, followed by scenario 2 with the captain 

as PF, etc. This resulted in the run order shown in Table B-2. The exploratory scenario was 

always flown after the pilot participants had completed all 10 data collection runs. 

Table B-1. Latin square designs. 

Captain as PF and First Officer as PM First Officer as PF and Captain as PM 

 

Run 

1 Run3 Run5 Run7 Run9 Run2 Run4 Run6 Run8

Run 

10 

Crew 1 5 2 1 4 3 Crew 1 4 3 2 1 5 

Crew 2 3 4 5 2 1 Crew 2 3 2 5 4 1 

Crew 3 1 3 2 5 4 Crew 3 2 4 1 5 3 

Crew 4 2 1 4 3 5 Crew 4 1 5 4 3 2 

Crew 5 4 5 3 1 2 Crew 5 5 1 3 2 4 

Crew 6 2 1 3 5 4 Crew 6 1 4 5 3 2 

Crew 7 3 4 2 1 5 Crew 7 2 5 1 4 3 

Crew 8 4 3 5 2 1 Crew 8 3 1 2 5 4 

Crew 9 5 2 1 4 3 Crew 9 5 3 4 2 1 

Crew 10 1 5 4 3 2 Crew 10 4 2 3 1 5 

 

 

4

 Dean, A. and Voss, D.. Design and Analysis of Experiments. Springer-Verlag New York, Inc., New York, NY, 

1999.
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

46 

Table B-2. Experiment run order. 

 Run1 Run2 Run3 Run4 Run5 Run6 Run7 Run8 Run9 Run10

PF: Capt F/O Capt F/O Capt F/O Capt F/O Capt F/O 

Crew 1 5 4 2 3 1 2 4 1 3 5 

Crew 2 3 3 4 2 5 5 2 4 1 1 

Crew 3 1 2 3 4 2 1 5 5 4 3 

Crew 4 2 1 1 5 4 4 3 3 5 2 

Crew 5 4 5 5 1 3 3 1 2 2 4 

Crew 6 2 1 1 4 3 5 5 3 4 2 

Crew 7 3 2 4 5 2 1 1 4 5 3 

Crew 8 4 3 3 1 5 2 2 5 1 4 

Crew 9 5 5 2 3 1 4 4 2 3 1 

Crew 10 1 4 5 2 4 3 3 1 2 5 

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

47 

Appendix C. Post-Run Questionnaire 

Pilot participants were asked to complete the following post-run questionnaire immediately 

following each of the experiment’s scenarios: 

This questionnaire is intended to capture measures of workload and ratings for the events that 

occurred in the scenario that was just completed. You will be asked to complete a more extensive 

questionnaire at the end of the experiment, so please try and keep written comments as concise 

as possible. 

1. Please circle the scenario you just completed from the list below: 

• Scenario 1 

• Scenario 2 

• Scenario 3 

• Scenario 4 

• Scenario 5 

• Scenario 6 

• Scenario 7 

• Scenario 8 

• Scenario 9 

• Scenario 10 

2. Please circle your role during the scenario you just completed from the list below: 

• Pilot Flying 

• Pilot Not Flying / Pilot Monitoring 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

48 

Average Workload Ratings (Modified Cooper-Harper) 

3. Follow the flow chart above to select the average workload you experienced during the 

scenario you just completed. 

 Rating of your average workload level: _________ 

(Optional) Use the space provided below to record any clarifying comments or interesting 

observations related to the workload level you experienced during the last run: 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

49 

Peak Workload Ratings (Modified Cooper-Harper) 

4. Follow the flow chart above to select the peak workload you experienced during the 

scenario you just completed. 

 Rating of your peak workload level: _________ 

 Please select the segment of flight during which your peak workload level 

occurred (you may select more than one segment if appropriate): 

o >18,000ft (cruise, initial descent) 

o 18,000ft–11,000ft (descent, approach check) 

o 11,000ft–5,000ft (TRACON, low altitude merge) 

o <5,000ft (final approach, configure aircraft) 

(Optional) Use the space provided below to record any clarifying comments or interesting 

observations related to the workload level you experienced during the last run: 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

50 

5. Respond to each of the statements shown below using a scale ranging from “1” 

(Completely Disagree) to “7” (Completely Agree). Circle one number in conjunction 

with each statement. 

Rating Scale 

1 2 3 4 5 6 7 

Completely 

Disagree 

Completely 

Agree

Relevant information, including operational 

plans, decisions, and changes in aircraft state, 

were effectively communicated between 

yourself and your crewmember. 

1 2 3 4 5 6 7 

The time available for tasks was well 

managed. 

1 2 3 4 5 6 7 

The IM commanded speeds were 

operationally acceptable and appropriate. 1 2 3 4 5 6 7 

The frequency of the IM speed commands 

was acceptable at all times throughout the 

scenario.

1 2 3 4 5 6 7 

I understood why the IM commanded speeds 

were provided (i.e. the IM commanded 

speeds made sense). 

1 2 3 4 5 6 7 

The use of voice communications to provide 

the IM clearance(s) was acceptable in this 

scenario. 

1 2 3 4 5 6 7 

The amount of head down time required to 

input information from the IM clearance(s) 

into the EFB was acceptable. 

1 2 3 4 5 6 7 

During this scenario, entering IM clearance 

information into the EFB was easy and 

intuitive. 

1 2 3 4 5 6 7 

During this scenario, it was easy to obtain 

needed information from the IM displays 

1 2 3 4 5 6 7 

The flight crew procedures for the events in 

this scenario were complete and acceptable. 

1 2 3 4 5 6 7 

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

51 

6. Please answer the following questions regarding the IM procedures you were asked to 

exercise in the scenario that you just completed. 

a. Did the procedures contain missing steps? 

YES ___ 

NO ___ 

b. Did the procedures contain extra steps that were unnecessary? 

YES ___ 

NO ___ 

c. Were the procedural steps logical and easy to follow? 

YES ___ 

NO ___ 

7. Please briefly explain any undesirable ratings from the statements above: 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

8. Describe any unusual or unexpected event(s) and your reaction(s), if applicable: 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

9. (Optional) This space is reserved for any additional comments related to awareness and 

acceptability issues. If you have any clarifying comments or interesting observations 

related to awareness and acceptability issues, please provide them below: 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

52 

Appendix D. Post-Experiment Questionnaire 

Pilot participants were asked to complete the following post-experiment questionnaire prior to 

participating in a group debrief session with the research team: 

This questionnaire is intended to gather your comments and suggestions regarding the 

experiment itself as well as the Interval Management concept 

This questionnaire contains items associated with each of the following categories: 

• Simulator and Flight Scenarios 

• Training 

• Interval Management Procedures 

• Interval Management Displays 

• Spacing Tool 

• Additional Comments 

Simulator and Flight Scenarios 

1. Was the workload required to operate the simulator much less than, the same as, or 

greater than the workload required to fly an actual aircraft? 

Much 

More 

Moderately 

More 

Slightly 

More 

The 

Same 

Slightly 

Less 

Moderately 

Less 

Much 

Less 

1 2 3 4 5 6 7 

Please provide any additional comments regarding the simulator: 

2. Please share your impressions of the flight scenarios (e.g., comment on their level of 

realism, appropriateness, and/or diversity) and comment on how the design of the 

scenarios impacted your ability to perform the spacing task:

Training 

3. Did you receive adequate training with respect to flying the simulator? 

YES ___ 

NO ___ 

If not, briefly describe how simulator training can be improved: 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

53 

4. Did you receive adequate training with respect to the IM spacing procedure and the 

spacing tool? 

YES ___ 

NO ___ 

If not, briefly describe how IM procedure or spacing tool training can be improved: 

5. Did you receive adequate training with respect to the entry and interpretation of 

information presented on the EFB? 

YES ___ 

NO ___ 

If not, briefly describe how EFB training can be improved: 

6. Did you receive adequate training with respect to the interpretation of information 

presented on the AGD? 

YES ___ 

NO ___ 

If not, briefly describe how AGD training can be improved: 

Interval Management Procedures 

7. Within this experiment four different aspects of the IM procedures were tested: 

terminating the IM operation, suspending and then resuming the IM operation, 

amending the IM spacing goal, and the loss of the lead aircraft’s ADS-B, as well as 

nominal operations. The following questions are intended to gather your feedback 

about the procedures used for each aspect of the IM operation that was tested (note that 

the last question asks about the general IM procedures). 

a) Were the procedures for terminating IM operation complete, accurate, and logical? 

YES ___ 

NO ___ 

Please provide any suggestions regarding the way(s) in which the procedures for 

terminating the IM operation may be improved: 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

54 

b) Were the procedures for suspending and then resuming an IM operation complete, 

accurate, and logical? 

YES ___ 

NO ___ 

Please provide any suggestions regarding the way(s) in which the procedures for 

suspending the IM operation may be improved: 

c) Were the procedures for amending the IM spacing goal complete, accurate, and 

logical? 

YES ___ 

NO ___ 

Please provide any suggestions regarding the way(s) in which the procedures for 

amending the IM spacing goal may be improved: 

d) Were the procedures for reacting to the loss of your lead aircraft’s ADS-B signal 

complete, accurate, and logical? 

YES ___ 

NO ___ 

Please provide any suggestions regarding the way(s) in which the procedures for 

reacting to the loss of your lead aircraft’s ADS-B signal may be improved: 

e) Were the general (nominal) IM procedures complete, accurate, and logical? 

YES ___ 

NO ___ 

Please provide any suggestions regarding the way(s) in which the general IM 

procedures may be improved: 

8. Was the IM phraseology used in this experiment correct and intuitive? 

YES ___ 

NO ___ 

If “no,” why not, and what could be done to improve the phraseology? 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

55 

9. How difficult do you think it would be for a typical flight crew to learn and integrate 

the IM spacing procedures into their current daily operational flight procedures? 

Very 

Difficult 

Moderately 

Difficult 

Slightly 

Difficult Neutral 

Slightly 

Easy 

Moderately 

Easy 

Very 

Easy 

1 2 3 4 5 6 7 

Briefly describe any challenges involved with integrating the IM procedures with 

existing procedures: 

10. Do you think the division of tasks between the Pilot Flying (PF) and Pilot Monitoring 

(PM) was both desirable and fit within the current distribution of tasks between PF and 

PM? 

YES ___ 

NO ___ 

If “no,” what was wrong with the division, and how would you reallocate the tasks? 

11. Given the experience with IM that you gained during this simulation, what is your 

overall assessment of the safety of the spacing procedure compared with current day 

operations? (“Safety” in this question refers to your holistic opinion to include 

workload, awareness, position relative to other aircraft, etc.) 

Not Safe 

At All 

Moderately 

Less Safe 

Slightly 

Less Safe As Safe 

Slightly 

More 

Safe 

Moderately 

More Safe 

Much 

More 

Safe 

1 2 3 4 5 6 7 

Briefly describe any characteristic or event that determined your rating (if 

appropriate): 

Interval Management Displays 

12. In general, did you find the process of entering IM clearance information into the EFB 

easy and intuitive? 

YES ___ 

NO ___ 

If “no,” what can be done to improve the process of loading information into the EFB? 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

56 

13. Did the AGD and EFB provide you with the information you needed/desired to safely 

and correctly conduct IM, and was this information easy to obtain when needed? 

YES ___ 

NO ___ 

If “no,” what information was missing, or how can the information be presented better? 

Function Allocation and the Spacing Tool 

14. Did following the IM commanded speed and procedure ever cause unexpected or 

undesirable behavior? 

YES ___ 

NO ___ 

If “yes,” please explain what the unexpected or undesirable behavior was: 

15. Did you find the responsibility of using onboard automation to achieve a spacing 

interval behind a lead aircraft acceptable (when ATC is responsible for separation)? 

YES ___ 

NO ___ 

If “no,” why not, and what could be done to make the responsibility or workload 

acceptable? 

16. Did you find your level of engagement with the IM automation acceptable (i.e. the level 

of decision making ability you had, and your understanding of the reasoning behind IM 

speeds that were commanded)? 

YES ___ 

NO ___ 

Please explain your answer. 

Additional Comments 

17. Do you have any additional comments about the experiment? 

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

57 

Appendix E. Example Interval Management (IM) Clearance and 

Related Use of and Display of Information on the 

Prototype Flight Deck-based Interval Management 

(FIM) Crew Interface 

The initial (empty) page of the Interval Management (IM) application residing on the electronic 

flight bag (EFB) portion of the prototype Flight Deck-based Interval Management (FIM) crew 

interface used during the Interval Management for Near-term Operations Validation of 

Acceptability (IM-NOVA) experiment is shown in Figure E-1. 

Figure E-1. Initial page of the IM application residing on the prototype FIM crew 

interface’s EFB.

Figure E-2–Figure E-11 illustrate the use of the prototype FIM crew interface during IM 

clearance data entry activities as well as the activation, suspension, resumption, and 

cancellation/termination of FIM operations. Below is an example of an IM clearance issued via 

voice communications to a flight crew by air traffic control (ATC): 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

58 

NASA 3, for Interval Spacing, cross JIFFY at 1432 plus 30 sec, when able space 90 sec 

behind Delta Alpha Lima 877 on Bonham 5, Fort Smith Transition. Report commencing 

interval spacing.5

As shown in Figure E-2, a three-step procedure is used to enter information associated with the 

IM achieve-by point (i.e., JIFFY) into the IM application that resides on the EFB. First, the “IM 

WPT” button is pressed. This action brings up a page that displays all the published waypoints 

on the arrival, allowing the flight crew to press a button associated with “JIFFY.” Lastly, the 

“ENTER” button is pressed. 

Step 1 Step 2 Step 3 

Figure E-2. Three-step procedure for inputting IM achieve-by point information 

into the prototype FIM crew interface’s EFB 

 

 

5 Note that the IM clearance was designed to provide the flight crew with all required information in an expected 

format. The target aircraft is identified using its three letter phonetic callsign, which corresponds with the callsign 

included in the target aircraft callsign list presented on the EFB (as shown in Figure E-5). The three letter identifier 

was used to assist with the identification on lesser known callsigns.
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

59 

Figure E-3 shows the two-step procedure associated with inputting the scheduled time of arrival 

(STA) at the IM achieve-by point (i.e., 1432 plus 30 sec) into the EFB. First, the “IM STA” 

button is pressed. Then, the touch screen’s keypad is used to enter required data into a 

“scratchpad” area, and the “ENTER” button is pressed. 

Step 1 Step 2 

Figure E-3. Two-step procedure for inputting STA at the IM achieve-by point 

information into the prototype FIM crew interface’s EFB. 

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

60 

Figure E-4 shows the two-step procedure associated with inputting the assigned spacing goal 

(ASG) (i.e., 90 sec) information into the EFB. First, the “IM GOAL” button is pressed. Then, the 

touch screen’s keypad is used to enter required data into a “scratchpad” area, and the “ENTER” 

button is pressed. 

Step 1 Step 2 

Figure E-4. Two-step procedure for inputting assigned spacing goal information 

into the prototype FIM crew interface’s EFB. 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

61 

Figure E-5 shows the three-step procedure associated with inputting a target aircraft’s callsign 

(i.e., Delta Alpha Lima 877) into the EFB. First, the “TGT ACFT” button is pressed. When the 

target aircraft is within Automatic Display Surveillance–Broadcast (ADS-B) range, its callsign 

can be selected, with a button press, from a selectable list. Lastly, the “ENTER” button is 

pressed. 

Step 1 Step 2 Step 3 

Figure E-5. Three-step procedure for inputting a target aircraft’s callsign into the 

prototype FIM crew interface’s EFB.

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

62 

Figure E-6 shows the three-step procedure associated with inputting a target aircraft’s arrival and 

transition (i.e., Bonham 5, Fort Smith Transition) information into the EFB. First, the “TGT 

RTE” button is pressed. Next, the target aircraft’s arrival is selected, with a button press, from a 

selectable list. Then, the target aircraft’s transition is selected, with a button press, from a 

selectable list, and the “ENTER” button is pressed.

Step 1 Step 2 Step 3 

Figure E-6. Three-step procedure for inputting a target aircraft’s arrival and 

transition information into the prototype FIM crew interface’s EFB. 

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

63 

After entering the achieve-by point, STA, ASG, target aircraft callsign, and target aircraft route 

information into the EFB, the flight crew can activate the IM clearance by pressing the 

“ACTIVATE” button located in the lower right portion of the EFB’s touch screen (shown in 

Figure E-7). Once pressed, the “ACTIVATE” button changes to a “SUSPEND” button, and the 

IM system indicates that it is performing various calculations by displaying the word 

“CALCULATING” in the EFB’s status box (shown in Figure E-7). 

→

Figure E-7. IM clearance activation.

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

64 

The flight crew flies in STA mode until ADS-B information from their target aircraft is acquired 

and FIM operations begin. As shown in Figure E-8, an indication of STA mode is displayed in 

the EFB’s status box, and a “CMD SPD,” in Mach or knots, and a “FAST/SLOW” indication, in 

knots, are displayed on both the EFB and ADS-B guidance display (AGD) components of the 

prototype FIM crew interface. 

EFB AGD 

Figure E-8. Information elements associated with STA mode displayed on the 

EFB and AGD components of the prototype FIM crew interface. 

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

65 

When ADS-B information from the target aircraft is acquired and FIM operations are underway, 

the EFB displays the target aircraft’s callsign, a commanded speed, and a “FAST/SLOW” 

indication (as shown in Figure E-9). Additionally, as shown in Figure E-9, the AGD supplies a 

white “IM” light located in the upper left corner of the device that indicates that the FIM aircraft 

is actively spacing relative to a target aircraft, and the AGD repeats the commanded speed and 

“FAST/SLOW” indications presented on the EFB. 

EFB AGD 

Figure E-9. Information elements associated with FIM operations displayed on 

the EFB and AGD components of the prototype FIM crew interface 

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

66 

Figure E-10 shows the three-step procedure associated with using the EFB to suspend and 

resume FIM operations. First, the “SUSPEND” button is pressed. This causes several things to 

happen: the “SUSPEND” button changes to a “RESUME” button, and a “TERMINATE” button 

appears; the commanded speed (“CMD SPD”) and “FAST/SLOW” indication are removed; and 

the word “SUSPENDED” is displayed in the EFB’s status box. At this point, the flight crew 

follows ATC’s speed commands. When instructed by ATC to resume FIM operations, the flight 

crew presses the “RESUME” button, causing the following things to happen: the “RESUME” 

button changes to a “SUSPEND” button; the “CMD SPD” and “FAST/SLOW” indication are 

presented; and the target aircraft’s callsign is displayed in the EFB’s status box. At this point, the

flight crew resumes FIM operations and follows IM commanded speeds. 

Step 1 Step 2 Step 3 

Figure E-10. Three-step procedure for suspending and then resuming IM 

spacing using the prototype FIM crew interface’s EFB. 

 
Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for NearTerm Operations Validation of Acceptability (IM-NOVA) Experiment 

67 

Figure E-11 shows the two-step procedure and resulting data re-entry screen associated with 

using the EFB to terminate FIM operations. First, the “SUSPEND” button is pressed, causing the 

following things to happen: the “SUSPEND” button changes to a “RESUME” button, and a 

“TERMINATE” button appears; the “CMD SPD” and “FAST/SLOW” indication are removed; 

and the word “SUSPENDED” is displayed in the EFB’s status box. Next, the “TERMINATE” 

button is pressed; all IM clearance data are removed from the EFB; and ATC instructions are 

followed. If FIM operations are to take place at this point, information from a new IM clearance 

must be entered into the EFB using the initial (empty) page of the IM application. 

→

Step 1 Step 2 Initial page of 

IM application 

Figure E-11. Two-step procedure, and resulting data (re-)entry screen, associated 

with terminating IM spacing using the prototype FIM crew interface’s 

EFB.
REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

2. REPORT TYPE 

Technical Publication

 4. TITLE AND SUBTITLE

Air Traffic Management Technology Demonstration Phase 1 (ATD-1) 

Interval Management for Near-Term Operations Validation of 

Acceptability (IM-NOVA) Experiment 

5a. CONTRACT NUMBER

 6. AUTHOR(S)

Kibler, Jennifer L.; Wilson, Sara R.; Hubbs, Clay E.; Smail, James W.

 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

NASA Langley Research Center

Hampton, VA 23681-2199

 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration

Washington, DC 20546-0001

 8. PERFORMING ORGANIZATION

 REPORT NUMBER

L-20551

10. SPONSOR/MONITOR'S ACRONYM(S)

NASA

13. SUPPLEMENTARY NOTES

12. DISTRIBUTION/AVAILABILITY STATEMENT

Unclassified - Unlimited

Subject Category 03

Availability: STI Program (757) 864-9658

19a. NAME OF RESPONSIBLE PERSON

STI Help Desk (email: help@sti.nasa.gov)

14. ABSTRACT

The Interval Management for Near-term Operations Validation of Acceptability (IM-NOVA) experiment was conducted at the National Aeronautics and Space 

Administration (NASA) Langley Research Center (LaRC) in support of the NASA Airspace Systems Program’s Air Traffic Management Technology 

Demonstration-1 (ATD-1). ATD-1 is intended to showcase an integrated set of technologies that provide an efficient arrival solution for managing aircraft 

using Next Generation Air Transportation System (NextGen) surveillance, navigation, procedures, and automation for both airborne and ground-based 

systems. The goal of the IMNOVA experiment was to assess if procedures outlined by the ATD-1 Concept of Operations were acceptable to and feasible for 

use by flight crews in a voice communications environment when used with a minimum set of Flight Deck-based Interval Management (FIM) equipment and a 

prototype crew interface. To investigate an integrated arrival solution using ground-based air traffic control tools and aircraft Automatic Dependent 

Surveillance-Broadcast (ADS-B) tools, the LaRC FIM system and the Traffic Management Advisor with Terminal Metering and Controller Managed Spacing 

tools developed at the NASA Ames Research Center (ARC) were integrated into LaRC’s Air Traffic Operations Laboratory (ATOL). Data were collected from 

10 crews of current 757/767 pilots asked to fly a high-fidelity, fixed-based simulator during scenarios conducted within an airspace environment modeled on 

the Dallas-Fort Worth (DFW) Terminal Radar Approach Control area. 

15. SUBJECT TERMS

Acceptability; Air traffic; Air transportation; Control system; Flight management; Flight crews; Ground based controls; 

Simulators; Voice communications

18. NUMBER

 OF 

 PAGES

79

19b. TELEPHONE NUMBER (Include area code)

(757) 864-9658

a. REPORT

U

c. THIS PAGE

U

b. ABSTRACT

U

17. LIMITATION OF 

 ABSTRACT

UU

Prescribed by ANSI Std. Z39.18

Standard Form 298 (Rev. 8-98)

3. DATES COVERED (From - To)

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

330693.04.10.07.07

11. SPONSOR/MONITOR'S REPORT

 NUMBER(S)

NASA/TP-2015-218767

16. SECURITY CLASSIFICATION OF:

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, 

gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 

collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and 

Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person 

shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY)

01- 05 - 2015