December 2016 

NASA/TP–2016-219362 

Human-in-the-Loop Assessment of Alternative 

Clearances in Interval Management Arrival 

Operations 

Brian T. Baxley, Sara R. Wilson, Kurt A. Swieringa, William C. Johnson, and Roy D. Roper 

Langley Research Center, Hampton, Virginia 

Clay E. Hubbs, Paul A. Goess, and Richard F. Shay 

National Institute of Aerospace, Hampton, Virginia
NASA STI Program . . . in Profile

Since its founding, NASA has been dedicated to the 

advancement of aeronautics and space science. The 

NASA scientific and technical information (STI) 

program plays a key part in helping NASA maintain 

this important role. 

The NASA STI program operates under the auspices 

of the Agency Chief Information Officer. It collects, 

organizes, provides for archiving, and disseminates 

NASA’s STI. The NASA STI program provides access 

to the NTRS Registered and its public interface, the 

NASA Technical Reports Server, thus providing one 

of the largest collections of aeronautical and space 

science STI in the world. Results are published in both 

non-NASA channels and by NASA in the NASA STI 

Report Series, which includes the following report 

types: 

• TECHNICAL PUBLICATION. Reports of 

completed research or a major significant phase of 

research that present the results of NASA 

Programs and include extensive data or theoretical 

analysis. Includes compilations of significant 

scientific and technical data and information 

deemed to be of continuing reference value. 

NASA counter-part of peer-reviewed formal 

professional papers but has less stringent 

limitations on manuscript length and extent of 

graphic presentations. 

• TECHNICAL MEMORANDUM. 

Scientific and technical findings that are 

preliminary or of specialized interest, 

e.g., quick release reports, working 

papers, and bibliographies that contain minimal 

annotation. Does not contain extensive analysis. 

• CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees. 

• CONFERENCE PUBLICATION. 

Collected papers from scientific and technical 

conferences, symposia, seminars, or other 

meetings sponsored or co-sponsored by NASA. 

• SPECIAL PUBLICATION. Scientific, 

technical, or historical information from NASA 

programs, projects, and missions, often 

concerned with subjects having substantial 

public interest. 

• TECHNICAL TRANSLATION. 

English-language translations of foreign 

scientific and technical material pertinent to 

NASA’s mission. 

Specialized services also include organizing 

and publishing research results, distributing 

specialized research announcements and feeds, 

providing information desk and personal search 

support, and enabling data exchange services. 

For more information about the NASA STI program, 

see the following: 

• Access the NASA STI program home page at 

http://www.sti.nasa.gov 

• E-mail your question to help@sti.nasa.gov 

• Phone the NASA STI Information Desk at 

757-864-9658 

• Write to: 

NASA STI Information Desk 

Mail Stop 148 

NASA Langley Research Center 

Hampton, VA 23681-2199 
National Aeronautics and 

Space Administration 

Langley Research Center 

Hampton, Virginia 23681-2199 

 

December 2016 

NASA/TP–2016-219362 

Human-in-the-Loop Assessment of Alternative 

Clearances in Interval Management Arrival 

Operations 

Brian T. Baxley, Sara R. Wilson, Kurt A. Swieringa, William C. Johnson, and Roy D. Roper 

Langley Research Center, Hampton, Virginia 

Clay E. Hubbs, Paul A. Goess, and Richard F. Shay 

National Institute of Aerospace, Hampton, Virginia
Available from: 

NASA STI Program / Mail Stop 148 

NASA Langley Research Center 

Hampton, VA 23681-2199 

Fax: 757-864-6500 

Foreword 

This NASA Technical Publication is a comprehensive report about the Interval Management Alternative Clearances 

(IMAC) human-in-the-loop experiment conducted at the NASA Langley Research Center in July and August of 

2015. The authors gratefully acknowledge the many individuals who made important contributions to the research, 

and without whom this very challenging and complex experiment would not have occurred. They include: Terry 

Abbott, Brenda Andrews, Janice Bayer, Tom Britton, Joel Brockman, John Bunnell, Tony Busquets, Mike Clark, 

Jim Davis, Mike Day, Dylan Drake, Peter Franklin, Dennis Frasca, Michael Harper, Stella Harrison, Fred Hibbard, 

Missy Hill, Brian Hutchinson, Richard Jessop, Regina Johns, Lon Kelly, Joe King, Troy Landers, Kara Latorella, 

Ron Maddox, Carolyn Malloy, Doug Mielke, Brendan Moeller, Sharon Otero, Mike Palmer, Benjamin Remy, 

Darrell Sacra, Ed Scearce, Omar Scott, Jim Smail, Jim Sturdy, Erin Thomas, Jon Welters, Dave West, and Chris 

Wyatt. 

Many key contributions were also provided by NASA Ames Research Center, to include the TBFM and CMS 

software. The authors are particularly indebted to Todd Callantine who provided his expertise in air traffic control to 

help design and create the IMAC experiment. The authors are also very appreciative of the tremendous support from 

Matt Gregory, Joey Mercer, and Thomas Prevot. 

The authors benefited greatly from the Project and Program leadership team, and thank the following individuals for 

their support: Sherilyn Brown, John Cavolowsky, Melinda Gratteau, Mike Madson, Leighton Quon, John Robinson, 

and Jane Thipphavong. 

MITRE and the FAA Surveillance and Broadcast Services Program Office also provided input to the experiment, 

and the authors thank Randy Bone, Mike Castle, Ian Levitt, and Wes Stoops for their contributions. 

And finally, organizing and running the experiment would not have been possible without the assistance of our 

interns, and the authors gratefully thank Maryam Amer, Andrew Arkebauer, Stuart Cooke, Rebecca Lidvall, Sarah 

Linthicum, Joshua Milot, and Ryan Petit for their great work. 

The use of trademarks or names of manufacturers in this report is for accurate reporting and does not constitute an 

official endorsement, either expressed or implied, of such products or manufacturers by the National Aeronautics and 

Space Administration.
i 

Table of Contents 

Executive Summary ..........................................................................................................................1

1 Introduction ................................................................................................................................3

1.1 Background ..........................................................................................................................3

1.2 Current Operational Need ....................................................................................................4

1.3 ATD-1 and IM Goals ...........................................................................................................4

1.4 Previous ATD-1 Research ...................................................................................................4

1.5 Terminology .........................................................................................................................5

2 ConOps, Technologies, Clearances, and Procedures .................................................................7

2.1 ATD-1 Concept of Operation ..............................................................................................7

2.2 The ATD-1 Technologies ....................................................................................................8

2.2.1 Traffic Management Advisor with Terminal Metering (TMA-TM) ...................... 8

2.2.2 Controller Managed Spacing (CMS) .................................................................... 10

2.2.3 Interval Management (IM) .................................................................................... 13

2.3 IM Operation Types ...........................................................................................................16

2.3.1 CROSS .................................................................................................................. 17

2.3.2 CAPTURE ............................................................................................................ 17

2.3.3 MAINTAIN .......................................................................................................... 18

2.4 Controller and Pilot Procedures .........................................................................................18

2.4.1 Controller Procedures ........................................................................................... 19

2.4.2 Flight Crew Procedures......................................................................................... 20

3 Experiment Design...................................................................................................................22

3.1 Objectives and Scope .........................................................................................................22

3.2 Assumptions .......................................................................................................................22

3.3 Performance Goals and Hypotheses ..................................................................................22

3.4 Test Matrix .........................................................................................................................24

3.5 Independent Variable .........................................................................................................24

3.6 Dependent Measures ..........................................................................................................25

3.6.1 IM Algorithm Performance................................................................................... 25

3.6.2 Air Traffic System Performance ........................................................................... 25

3.6.3 Controller Objective Performance ........................................................................ 26

3.6.4 Flight Crew Objective Performance ..................................................................... 27
ii 

3.6.5 Controller Subjective Assessments ....................................................................... 27

3.6.6 Flight Crew Subjective Assessments .................................................................... 28

3.7 Case Studies .......................................................................................................................28

3.8 Environment and scenario description ...............................................................................29

3.8.1 KDEN Instrument Arrival and Approach Procedures .......................................... 29

3.8.2 KDEN Airport and Landing Runways .................................................................. 32

3.8.3 Truth and Forecast Wind ...................................................................................... 33

3.8.4 Training Scenarios ................................................................................................ 34

3.8.5 Data Collection Scenarios ..................................................................................... 35

3.8.6 Example of Aircraft Location at Scenario Start .................................................... 36

3.9 Controller and Pilot Participants ........................................................................................37

3.9.1 Qualification and Currency Requirements ............................................................ 37

3.9.2 Protocol for Training and Data Collection ............................................................ 38

4 Facilities and Software .............................................................................................................40

4.1 Air Traffic Operations Laboratory (ATOL) ......................................................................40

4.1.1 MACS Controller Stations .................................................................................... 41

4.1.2 Time Based Flow Management (TBFM) .............................................................. 43

4.1.3 Aircraft Simulation for Traffic Operations Research (ASTOR) ........................... 44

4.1.4 MACS Pseudo-Pilot Station ................................................................................. 45

4.2 Development and Test Simulator (DTS) ...........................................................................46

4.3 Integration Flight Deck (IFD) ............................................................................................47

5 IMAC Experiment Results .......................................................................................................49

5.1 Caveat to Experiment Results ............................................................................................49

5.2 Preconditioning of Data .....................................................................................................50

5.3 IM Algorithm Performance Results ...................................................................................50

5.3.1 IM Spacing Error .................................................................................................. 50

5.3.2 Frequency of IM Speed Changes during IM Operations ...................................... 51

5.3.3 Number of IM Speed Reversals per IM Operation ............................................... 54

5.4 Air Traffic System Performance Results ...........................................................................55

5.4.1 Inter-Arrival Spacing Error at the Final Approach Fix ......................................... 55

5.4.2 PBN Success Rate ................................................................................................. 57

5.4.3 Slot Marker Deviation in the TRACON ............................................................... 58

5.5 Controller Objective Performance Results ........................................................................59

5.5.1 Percentage of Controller-Interrupted IM Operations ............................................ 59
iii 

5.5.2 Schedule Deviation ............................................................................................... 59

5.6 Controller Subjective Assessment Responses ...................................................................62

5.6.1 Controller Acceptability of ATD-1 Operations .................................................... 62

5.6.2 Controller Assessment of Workload of ATD-1 Operations ................................. 63

5.6.3 Controller Comments about ATD-1 ConOps and IM Operations ........................ 66

5.7 Flight Crew Objective Performance Results ......................................................................67

5.7.1 Percentage of Flight Crew Interrupted IM Operations ......................................... 67

5.7.2 Flight Crew Reaction Time to IM Speed Command Change ............................... 67

5.7.3 Flight Crew Conformance to the IM Speed .......................................................... 68

5.7.4 Missed Altitude Constraints .................................................................................. 69

5.7.5 Unstable Approaches ............................................................................................ 72

5.8 Flight Crew Subjective Assessment Responses .................................................................73

5.8.1 Flight Crew Acceptability ..................................................................................... 73

5.8.2 Flight Crew Workload of IM Operations .............................................................. 80

5.8.3 Flight Crew Ratings and Comments about Situation Awareness ......................... 81

5.8.4 Flight Crew Comments about ATD-1 ConOps and IM Operations ..................... 82

5.8.5 Flight Crew Comments about IM Interface and Displays .................................... 87

5.9 Case Study Results .............................................................................................................89

5.9.1 Loss of Separation between Aircraft ..................................................................... 89

5.9.2 Undesirable Propagation of Speed Changes ......................................................... 92

5.9.3 Desirable IM Spacing Algorithm Performance .................................................... 95

5.9.4 Variations to Normal IM Clearances and Operations ........................................... 97

5.9.5 Impact to ASTAR When Target Aircraft not at Published Airspeed ................... 98

6 Conclusions and Lessons Learned .........................................................................................105

6.1 General .............................................................................................................................105

6.2 Meeting Performance Goals and Hypotheses ..................................................................105

6.3 ATD-1 ConOps, Procedures, and Phraseology................................................................106

6.4 IM Cockpit Displays ........................................................................................................107

6.5 IM Spacing Software .......................................................................................................109

6.6 Lessons Learned ...............................................................................................................110

7 Recommendations for Future Research .................................................................................113

References . ..............................................................................................................................115

Appendix A: IM Software ...........................................................................................................117

A.1 IM Software Components ................................................................................................117
iv 

A.2 ASTAR Algorithm ...........................................................................................................117

A.3 Speed Control Laws .........................................................................................................120

A.4 ASTAR Logic Wrapper ...................................................................................................123

A.4.1 IM States ............................................................................................................. 123

A.4.2 Transition between IM States ............................................................................. 124

A.4.3 Alert Levels ......................................................................................................... 129

A.4.4 IM Alerting Messages ......................................................................................... 130

Appendix B: IM Displays and Data Entry ...................................................................................132

B.1 EFB Overview .................................................................................................................132

B.2 CGD Overview ................................................................................................................135

B.3 IM Displays by State ........................................................................................................136

B.4 IM Display Sub-Elements ................................................................................................144

B.5 Flight Crew Data Entry into IM Interface ........................................................................148

Appendix C: Procedures and Phraseology for IM Operations .....................................................160

C.1 Pilot Procedures for IM Operations .................................................................................160

C.2 Controller – Pilot Phraseology for IM Operations ...........................................................161

Appendix D: Airlines Operating into Denver International Airport ............................................164

Appendix E: Questionnaires ........................................................................................................165

E.1 Controller Background Questionnaire .............................................................................165

E.2 Pilot Background Questionnaire ......................................................................................166

E.3 Controller Post-Run Questionnaire ..................................................................................167

E.4 Pilot Post-Run Questionnaire ...........................................................................................172

E.5 Controller Post-Experiment Questionnaire ......................................................................179

E.6 Pilot Post-Experiment Questionnaire ...............................................................................189

Appendix F: Questionnaire Results .............................................................................................199

F.1 Controller Background Questionnaire Results ................................................................199

F.2 Pilot Background Questionnaire Results .........................................................................200

F.3 Controller Post-Run Questionnaire ..................................................................................201

F.4 Pilot Post-Run Questionnaire Results ..............................................................................213

F.5 Controller Post-Experiment Questionnaire Results .........................................................238

F.6 Pilot Post-Experiment Questionnaire Results ..................................................................252

Appendix G: Data Removed from Analysis ................................................................................279

Appendix H: Effectiveness of Training .......................................................................................281

Appendix I: Oculometers .............................................................................................................283
v 

List of Figures 

Figure 1. Integrated NASA technologies used in the ATD-1 ConOps. .......................................... 8

Figure 2. IM information displayed on the controller’s meter list. ................................................. 9

Figure 3. CMS tools and displays. ................................................................................................ 11

Figure 4. IM status indications on the ARTCC controller workstations. ..................................... 12

Figure 5. IM status indicators on the TRACON controller workstations. .................................... 13

Figure 6. IM Cockpit Interfaces: EFB (left) and CGD (right). ..................................................... 15

Figure 7. Arrivals into KDEN for north-flow landing. ................................................................. 30

Figure 8. Arrivals into KDEN for south-flow landing. ................................................................. 31

Figure 9. KDEN runways used for landing during IMAC. ........................................................... 32

Figure 10. Example of aircraft location at scenario start. ............................................................. 36

Figure 11. Controller and pilot room configuration in ATOL for the IMAC experiment. ........... 40

Figure 12. MACS workstations for controllers. ............................................................................ 41

Figure 13. MACS display for ARTCC controller workstation. .................................................... 42

Figure 14. MACS display for TRACON controller workstation. ................................................. 42

Figure 15. TBFM screen and display. ........................................................................................... 43

Figure 16. Three-monitor ASTOR station for a two-person flight crew. ..................................... 44

Figure 17. MACS workstation for a single pseudo-pilot. ............................................................. 45

Figure 18. The Development and Test Simulator (DTS). ............................................................. 46

Figure 19. The Integration Flight Deck (IFD). ............................................................................. 47

Figure 20. IFD on the cockpit motion base................................................................................... 48

Figure 21. Graphs of IM spacing error at the ABP and PTP. ....................................................... 50

Figure 22. Graph of IM of speed change frequency by scenario type. ......................................... 52

Figure 23. Scatterplot of time between all IM speed changes by distance-to-go. ........................ 53

Figure 24. Histogram of IM speed changes by IM operation type by distance-to-go. ................. 53

Figure 25. Number of IM speed reversals per flight by scenario type. ........................................ 54

Figure 26. Graph of inter-arrival spacing error at Final Approach Fix. ....................................... 55

Figure 27. The latitude and longitude of all aircraft for all four landing runways. ...................... 57

Figure 28. The RMS of the slot marker deviation in the TRACON. ............................................ 58

Figure 29. The schedule deviation at Final Approach Fix. ........................................................... 60
vi 

Figure 30. The schedule deviation at the TRACON Meter Fix. ................................................... 61

Figure 31. Controller acceptability of ATD-1 operations. ............................................................ 62

Figure 32. Scatter plot of pilot reaction time in seconds by altitude by simulator type. .............. 68

Figure 33. The magnitude of missed altitude constraints by simulator type. ............................... 70

Figure 34. Flight crew acceptability of operations by IM operation type. ................................... 74

Figure 35. Flight crew acceptability of IM procedure by IM operation type. .............................. 76

Figure 36. Flight crew completeness of IM procedure by IM operation type. ............................. 77

Figure 37. Flight crew operational acceptability of IM speeds by IM operation type. ................. 78

Figure 38. Flight crew acceptability of IM speed change frequency by IM operation type. ........ 79

Figure 39. Flight crew acceptability of head down time by IM operation type. ........................... 80

Figure 40. Flight crew perception of workload by IM operation type. ......................................... 81

Figure 41. Flight crew acceptability of use of voice communications by IM operation type. ..... 84

Figure 42. Loss of separation case #2 between IM and Target aircraft. ....................................... 90

Figure 43. Loss of separation case #3 between IM and Target aircraft. ....................................... 91

Figure 44. Non-ideal IM speed reversals of CTD control law during CAPTURE operation. ...... 94

Figure 45. Desired spacing algorithm performance by IM operation type. .................................. 96

Figure 46: Case study of spacing error when Target has a large ground speed deviation. ......... 103

Figure 47: Target aircraft ground speed compensation term. ..................................................... 103

Figure 48. Relationship of IM software, ASTAR algorithm, and speed control laws. ............... 117

Figure 49. Diagram of the TBO speed control law. .................................................................... 121

Figure 50. Diagram of the CTD speed control law. .................................................................... 123

Figure 51. IM state logic flow diagram. ..................................................................................... 124

Figure 52. Legend for the IM Home page on an EFB. ............................................................... 133

Figure 53. Legend for the IM displays on a CGD. ..................................................................... 135

Figure 54. EFB and CGD in the OFF state with no Ownship information. ............................... 136

Figure 55. EFB and CGD in the OFF state with Ownship information. .................................... 137

Figure 56. EFB and CGD in the ARMED state. ......................................................................... 138

Figure 57. EFB and CGD in the AVAILABLE state. ................................................................ 139

Figure 58. EFB and CGD in the PAIRED state. ......................................................................... 140

Figure 59. EFB and CGD in the SUSPENDED-AVAILABLE state. ........................................ 141

Figure 60. EFB and CGD in the SUSPENDED-ARMED state. ................................................ 142

Figure 61. EFB and CGD in the UNABLE state. ....................................................................... 143

Figure 62. EFB and CGD with IM speed in reverse video. ........................................................ 144
vii 

Figure 63. FAST/SLOW indicator sequence during IM speed change. ..................................... 145

Figure 64. Progress Indicator with 2 minute (left) and 45 second (right) scales. ....................... 146

Figure 65. EFB display to confirm cancellation of IM operation. .............................................. 147

Figure 66. Main Menu and Applications Menu pages. ............................................................... 148

Figure 67. IM Home and Ownship Entry pages. ........................................................................ 149

Figure 68. Airport data field selected and airport entered. ......................................................... 150

Figure 69. Descent Forecast page prior to and after data entry. ................................................. 151

Figure 70. IM Home page with Ownship data entered. .............................................................. 152

Figure 71. Initial IM clearance entry page and IM clearance type sub-page. ............................. 153

Figure 72. IM CROSS clearance page with and without data entered. ...................................... 154

Figure 73. IM Home page with all data entered. ........................................................................ 155

Figure 74. IM CROSS clearance page in DTS and IFD. ............................................................ 156

Figure 75. IM CAPTURE and MAINTAIN clearance pages in DTS and IFD. ......................... 157

Figure 76. ARMED and AVAILABLE states in the DTS and IFD. .......................................... 158

Figure 77. PAIRED and SUSPENDED-AVAILABLE states in the DTS and IFD. .................. 158

Figure 78. The confirm cancel message in the DTS and IFD. .................................................... 159

Figure 79. The CGD showing AVAILABLE and PAIRED in the DTS and IFD. ..................... 159

Figure 80. Controller perception of mental demand by station by IM operation type. .............. 202

Figure 81. Controller perception of physical demand by station by IM operation type. ............ 203

Figure 82. Controller perception of time pressure by station by IM operation type. ................. 204

Figure 83. Controller perception of effort by station by IM operation type. .............................. 205

Figure 84. Controller perception of frustration by station by IM operation type. ...................... 206

Figure 85. Controller perception of success by station by IM operation type. ........................... 207

Figure 86. Controller post-run acceptability (CARS) ratings by operation type. ....................... 208

Figure 87. Modification of controller technique to use CMS tools. ........................................... 209

Figure 88. Modification of controller technique to use IM tools. ............................................... 210

Figure 89. Pilot accept ability ratings of speeds flown by IM aircraft. ...................................... 211

Figure 90. Flight crew perception of workload during BASELINE operation. .......................... 213

Figure 91. Flight crew perception of workload during CAPTURE operation. ........................... 213

Figure 92. Flight crew perception of workload during CROSS operation. ................................ 214

Figure 93. Flight crew perception of workload during MAINTAIN operation. ......................... 214

Figure 94. Flight crew perception of workload during MIXED operation. ................................ 214

Figure 95. Flight crew acceptability of BASELINE operation. .................................................. 215
viii 

Figure 96. Flight crew acceptability of CAPTURE operation. ................................................... 216

Figure 97. Flight crew acceptability of CROSS operation. ........................................................ 216

Figure 98. Flight crew acceptability of MAINTAIN operation. ................................................. 216

Figure 99. Flight crew acceptability of MIXED operation. ........................................................ 217

Figure 100. Flight crew acceptability of CAPTURE operation by phase of flight. .................... 218

Figure 101. Flight crew acceptability of CROSS operation by phase of flight. ......................... 218

Figure 102. Flight crew acceptability of MAINTAIN operation by phase of flight. .................. 219

Figure 103. Flight crew acceptability of MIXED operations by phase of flight. ....................... 219

Figure 104. Flight crew acceptability of voice communication for BASELINE operation. ...... 220

Figure 105. Flight crew acceptability of voice communication for CAPTURE operation. ....... 220

Figure 106. Flight crew acceptability of voice communication for CROSS operation. ............. 221

Figure 107. Flight crew acceptability of voice communication for MAINTAIN operation. ..... 221

Figure 108. Flight crew acceptability of voice communication for MIXED scenario. .............. 221

Figure 109. Perceived flight crewmember communication effectiveness. ................................. 222

Figure 110. Flight crew acceptability of IM commanded speeds for CAPTURE operation. ..... 223

Figure 111. Flight crew acceptability of IM commanded speeds for CROSS operation. .......... 223

Figure 112. Flight crew acceptability of IM commanded speeds for MAINTAIN operation. ... 224

Figure 113. Flight crew acceptability of IM commanded speeds for MIXED operation. .......... 224

Figure 114. Flight crew acceptability of IM speed change frequency for CAPTURE ops. ....... 225

Figure 115. Flight crew acceptability of IM speed change frequency for CROSS operation. ... 225

Figure 116. Flight crew acceptability of IM speed change frequency for MAINTAIN ops. ..... 226

Figure 117. Flight crew acceptability of IM speed change frequency for MIXED operation. ... 226

Figure 118. Flight crew acceptability of head down time to enter CAPTURE clearance. ......... 227

Figure 119. Flight crew acceptability of head down time to enter CROSS clearance. ............... 227

Figure 120. Flight crew acceptability of head down time to enter MAINTAIN clearance. ....... 228

Figure 121. Flight crew acceptability of head down time to enter MIXED clearance. .............. 228

Figure 122. Flight crew acceptability of CAPTURE procedure. ................................................ 229

Figure 123. Flight crew acceptability of CROSS procedure. ..................................................... 229

Figure 124. Flight crew acceptability of MAINTAIN procedure. .............................................. 230

Figure 125. Flight crew acceptability of MIXED procedure. ..................................................... 230

Figure 126. Flight crew perception of procedure completeness for CAPTURE operation. ....... 231

Figure 127. Flight crew perception of procedure completeness for CROSS operation. ............. 231

Figure 128. Flight crew perception of procedure completeness for MAINTAIN ops. ............... 232
ix 

Figure 129. Flight crew perception of procedure completeness for MIXED operation. ............ 232

Figure 130. Controller post-experiment ratings of training by type of training. ........................ 239

Figure 131. Controller operational acceptability of CMS tool procedures. ................................ 240

Figure 132. Controller acceptability of IM procedures for each clearance type. ....................... 241

Figure 133. Impact of IM operations on expediting traffic flow. ............................................... 242

Figure 134. Pilot perceived impact of IM operations on overall situational awareness. ............ 259

Figure 135. Pilot acceptability of IM responsibilities. ................................................................ 261

Figure 136. Pilot rated intuitiveness of entering IM clearance information into the EFB. ......... 265

Figure 137. Pilot ratings of CGD effectiveness. ......................................................................... 269

Figure 138. Pilot operational acceptability of Target aircraft’s call sign in IM clearance. ........ 272

Figure 139. Responses to difficulty of integration into daily flight procedures. ........................ 274

Figure 140. Mean controller acceptability of IM operations by scenario run order. .................. 281

Figure 141. Mean pilot acceptability of IM operations by scenario run order. .......................... 282

Figure 142. Mean pilot workload of IM operations by scenario run order. ................................ 282

Figure 143. A Smart Eye oculometer camera. ............................................................................ 283

List of Tables 

Table 1. Data Required for IM CROSS Clearance ....................................................................... 17

Table 2. Data Required for IM CAPTURE Clearance ................................................................. 18

Table 3. Data Required for IM MAINTAIN Clearance ............................................................... 18

Table 4. Correlation of Performance Goals and Hypotheses to Section of Results ..................... 23

Table 5. IMAC Experiment Test Matrix ....................................................................................... 24

Table 6. STARs and Approaches by Sector for North-Flow Landing .......................................... 30

Table 7. STARs and Approaches by Sector for South-Flow Landing .......................................... 31

Table 8. Description of Training Scenarios .................................................................................. 34

Table 9. Description of Data Collection Scenarios ....................................................................... 35

Table 10. IM Spacing Error at ABP and PTP by Scenario Type in Seconds ............................... 51

Table 11. IM Speed Change Frequency per Minute by Scenario Type ........................................ 52

Table 12. Inter-Arrival Spacing Error Data at Final Approach Fix .............................................. 56

Table 13. RMS Data for Slot Marker Deviation in TRACON ..................................................... 58

Table 14. Schedule Deviation Data at Final Approach Fix .......................................................... 60

Table 15. Schedule Deviation Data at TRACON Meter Fix ........................................................ 61
x 

Table 16. Descriptive Statistics for Controller Workload Ratings (1 of 3) .................................. 63

Table 17. Flight Crew Reaction Time to IM Commanded Speed Change in Seconds ................. 67

Table 18. RMS of Aircraft Deviation from IM Commanded Speed in Knots .............................. 68

Table 19. Arrival Operations with Missed Altitude Constraints .................................................. 69

Table 20. Waypoints with Missed Altitude Constraints ............................................................... 69

Table 21. Number and Percent of Missed Altitude Constraints by Scenario Type ...................... 71

Table 22. Operations not Meeting Stabilized Approach Criteria ................................................. 72

Table 23. Descriptive Statistics for Flight Crew Acceptability Ratings ....................................... 73

Table 24. Descriptive Statistics for Pilot Operational Acceptability of IM Procedures ............... 75

Table 25. Descriptive Statistics for Flight Crew Acceptability of IM Procedures Ratings .......... 76

Table 26. Descriptive Statistics for Completeness of IM Procedures Ratings ............................. 77

Table 27. Pilot Percentage Reporting Missing, Unnecessary and Logical Steps by Type ........... 77

Table 28. Descriptive Statistics of Operational Acceptability of IM Speeds ............................... 78

Table 29. Descriptive Statistics of Acceptability of Frequency of IM Speed Changes ................ 79

Table 30. Descriptive Statistics for Acceptability of Amount of Head Down Time .................... 79

Table 31. Descriptive Statistics for Flight Crew Workload Ratings ............................................ 81

Table 32. Pilot Percentage Reporting Issues with IM Phraseology by IM Operation Type ......... 83

Table 33. Type of Third Party ID Used to Issue the IM Clearance .............................................. 83

Table 34. Descriptive Statistics for Acceptability of the Use of Voice Communications ............ 84

Table 35. Pilot acceptability of Voice Comm during ATD-1 operations ..................................... 85

Table 36. Descriptive Statistics for CGD Effectiveness and Display Element Usefulness .......... 88

Table 37. IM Clearances with ABP Dissimilar to the PTP ........................................................... 97

Table 38. Summary of Performance Goals and Hypotheses ...................................................... 105

Table 39. Alert Levels and Characteristics ................................................................................. 129

Table 40. IM Message Criteria and the Pilot Response .............................................................. 130

Table 41. Airline IDs and Call Signs Used in IMAC ................................................................. 164

Table 42. Descriptive Statistics for Controller Perception of Mental Demand. ......................... 202

Table 43. Descriptive Statistics for Controller Perception of Physical Demand. ....................... 203

Table 44. Descriptive Statistics for Controller Perception of Time Pressure. ............................ 204

Table 45. Descriptive Statistics for Controller Perception of Effort. ......................................... 205

Table 46. Descriptive Statistics for Controller Perception of Frustration. ................................. 206

Table 47. Descriptive Statistics for Controller Perception of Success. ...................................... 207

Table 48. Descriptive Statistics for Controller Acceptability Ratings. ....................................... 208
xi 

Table 49. Responses to How Controllers Changed Working with IM Aircraft. ......................... 210

Table 50. Clearance Types Issued. ............................................................................................. 212

Table 51. Type of Third Party ID Used to Issue the IM Clearance. ........................................... 212

Table 52. Descriptive Statistics for Flight Crew Acceptability Ratings. .................................... 218

Table 53. Percent of Pilots Who Reported an Issue with the IM Phraseology ........................... 235

Table 54. Type of Third Party ID Used to Issue the IM Clearance ............................................ 236

Table 55. Descriptive Statistics for Post-Experiment Controller Training Ratings .................... 238

Table 56. Descriptive Statistics for Controller Acceptability Ratings of IM Procedures ........... 241

Table 57. Descriptive Statistics for Acceptability of Controller Responsibility for CAPTURE 

Clearance ..................................................................................................................................... 243

Table 58. Descriptive Statistics for Acceptability of Controller Responsibility for CROSS 

Clearance ..................................................................................................................................... 244

Table 59. Descriptive Statistics for Acceptability of Controller Responsibility for MAINTAIN 

Clearance ..................................................................................................................................... 245

Table 60. Descriptive Statistics for Usefulness of Controller Tools for Managing Aircraft ...... 246

Table 61. Descriptive Statistics for Usefulness of CMS Tools for Managing Aircraft .............. 247

Table 62. Descriptive Statistics for Controller Operational Acceptability of IM Phraseology .. 248

Table 63. Descriptive Statistics for Perceived Workload of Simulator vs. Actual Aircraft ....... 253

Table 64. Descriptive Statistics for Perceived Preparedness by Training .................................. 254

Table 65. Descriptive Statistics for Perceived Preparedness from EFB Training ...................... 255

Table 66. Descriptive Statistics for Perceived Preparedness from CGD Training ..................... 255

Table 67. Descriptive Statistics for Perceived Preparedness from IM Training ......................... 256

Table 68. Descriptive Statistics for Pilot Operational Acceptability of IM Procedures ............. 257

Table 69. Descriptive Statistics for Pilot Overall Acceptability of IM Procedures .................... 258

Table 70. Descriptive Statistics for Pilot Perceived Impact of IM on Situational Awareness ... 259

Table 71. Descriptive Statistics for Pilot Acceptability of IM Responsibilities ......................... 261

Table 72. Descriptive Statistics for Pilot Intuitiveness of Entering IM Information .................. 265

Table 73. Descriptive Statistics for Usefulness Ratings of EFB Display Elements ................... 266

Table 74. Descriptive Statistics for Usefulness Ratings of EFB Display Elements ................... 267

Table 75. Descriptive Statistics for Usefulness Ratings of CGD Display Elements .................. 268

Table 76. Descriptive Statistics for Pilot Operational Acceptability of IM Phraseology ........... 270

Table 77. Descriptive Statistics for Pilot Acceptability of Using Target Call Sign in IM Clearance

..................................................................................................................................................... 272

Table 78. Descriptive Statistics for Difficulty Integrating IM into Current Flight Procedures .. 274
xii 

Table 79. Aircraft Removed From Quantitative Analysis .......................................................... 280

Abbreviations and Acronyms 

ABP Achieve-By Point 

ACARS Aircraft Communications Addressing and Reporting System 

ADS-B Automatic Dependent Surveillance–Broadcast 

ARTCC Air Route Traffic Control Center 

ASG Assigned Spacing Goal 

ASTAR Airborne Spacing for Terminal Arrival Routes 

ASTOR Aircraft Simulation for Traffic Operations Research 

ATA Actual Time of Arrival 

ATC Air Traffic Control 

ATD-1 ATM Technology Demonstration–1 

ATIS Automatic Terminal Information Service 

ATM Air Traffic Management 

ATOL Air Traffic Operations Laboratory 

CARS Controller Acceptance Rating Scale 

CGD Configurable Graphics Display 

CMS Controller Managed Spacing 

ConOps Concept of Operations 

CTD Constant Time Delay 

DCT Delay Countdown Timer 

DTS Development and Test Simulator 

EFB Electronic Flight Bag 

ETA Estimated Time of Arrival 

ERAM En Route Automation Modernization 

FAA Federal Aviation Administration 

FAF Final Approach Fix 

FIM Flight deck Interval Management 

FL Flight Level 

FMS Flight Management System 

ID Identification (aircraft call sign) 

IFD Integration Flight Deck simulator 

ILS Instrument Landing System 

IM Interval Management 

IMAC Interval Management Alternative Clearances 

KDEN Denver International Airport 

MACS Multi-Aircraft Control System 
xiii 

MCP Mode Control Panel 

MOPS Minimum Operational Performance Standards 

MSI Measured Spacing Interval 

NASA National Aeronautics and Space Administration 

nmi Nautical Mile 

PBN Performance Based Navigation 

PF Pilot Flying 

PM Pilot Monitoring 

PTP Planned Termination Point 

RMS Root mean square 

RNAV Area navigation 

RNP Required Navigation Performance 

SD Standard Deviation 

STA Scheduled Time of Arrival 

STAR Standard Terminal Arrival Route 

STARS Standards Terminal Automation Replacement System 

TBFM Time Based Flow Management 

TBO Trajectory-Based Operation 

TGT Target aircraft 

TLX Task Load Index 

TMA-TM Traffic Management Advisor with Terminal Metering 

TOD Top Of Descent 

TRACON Terminal Radar Approach Control Facility 

TSAS Terminal Sequencing And Spacing 

VMC Visual Meteorological Conditions 

VNAV Vertical Navigation 

VoIP Voice-Over Internet Protocol 

WPT Waypoint 
xiv 

Abstract 

Interval Management Alternative Clearances (IMAC) was a human-inthe-loop simulation experiment conducted to explore the Air Traffic 

Management (ATM) Technology Demonstration (ATD-1) Concept of 

Operations (ConOps), which combines advanced arrival scheduling, 

controller decision support tools, and aircraft avionics to enable multiple 

time deconflicted, efficient arrival streams into a high-density terminal 

airspace. Interval Management (IM) is designed to support the ATD-1 

concept by having an “Ownship” (IM-capable) aircraft achieve or 

maintain a specific time or distance behind a “Target” (preceding) 

aircraft. The IM software uses IM clearance information and the Ownship 

data (route of flight, current location, and wind) entered by the flight crew, 

and the Target aircraft’s Automatic Dependent Surveillance–Broadcast 

state data, to calculate the airspeed necessary for the IM-equipped aircraft 

to achieve or maintain the assigned spacing goal. 

IMAC investigated three different types of IM operations: CAPTURE, 

CROSS, and MAINTAIN. Two weeks of data collection were conducted. 

Each week a new group of 12 subject pilots and four subject controllers 

flew 10 high-density arrival scenarios into Denver International Airport. 

The experiment objective was to explore the acceptability and system 

performance of the ATD-1 ConOps and IM procedures. Overall, both the 

ConOps and the IM procedures were rated very favorably in terms of 

acceptability, workload, and pilot head down time. The mean IM spacing 

error at the planned termination point was 1.0 second, with a standard 

deviation of 6.3 seconds, and the interaction of IM with non-IM operations 

in the same arrival stream was generally compatible and acceptable. 

Several critical issues were identified that must be resolved prior to 

real-world implementation, including: high frequency of IM speed 

changes and speed reversals (especially if on final approach), the use of 

vertical navigation speed mode to conduct the IM operation, and 

ambiguous IM cockpit displays not triggering the intended pilot action. 

The results from this experiment will be used to prepare for flight test 

operations and in developing the advanced IM concept. 

 
 1 

Executive Summary 

Experiment Description 

Interval Management Alternative Clearances (IMAC) was a human-in-the-loop simulation 

experiment conducted by the Air Traffic Management Technology Demonstration-1 (ATD-1) 

Project in 2015 at NASA Langley Research Center. The ATD-1 Concept of Operations (ConOps) 

utilizes the Time Based Flow Management (TBFM) advanced arrival scheduling, the Terminal 

Sequencing and Spacing (TSAS) controller decision support tools, and the Interval Management 

(IM) aircraft avionics to enable multiple time-deconflicted, efficient arrival streams into a highdensity terminal airspace. IM is designed to support the ATD-1 concept by enabling an en route 

air traffic controller to issue a single strategic clearance to the flight crew of an IM-equipped 

(“Ownship”) aircraft to achieve or maintain a specific time or distance behind the preceding 

(“Target”) aircraft. This onboard IM software uses 1) Ownship data (route of flight, current 

location, forecast and actual wind information), 2) information contained in the controller-issued 

IM clearance, and 3) the Automatic Dependent Surveillance–Broadcast (ADS-B) state data 

received from the Target aircraft to calculate the airspeed necessary for the Ownship aircraft to fly 

in order to achieve the assigned spacing goal behind the Target aircraft. 

The experiment objective was to assess the acceptability and system performance of three IM 

operations in realistic, high-density arrival operations. They were the CROSS operation (used in 

previous ATD-1 research and capable of supporting in-trail and merging route geometries), and 

the CAPTURE and MAINTAIN operations (new procedures for when the Target and Ownship 

aircraft are in-trail). Additional objectives were to identify implementation issues for real-world 

operations, and to investigate the integration of these new IM operations with the TBFM and TSAS 

tools. While the CROSS operation is able to support complex merging geometries by the IM and 

Target aircraft, when the aircraft are in trail, much of the information in the clearance does not 

need to be included in the controller’s instruction. Thus the CAPTURE and MAINTAIN 

operations are alternatives that offer a lower workload for the controller and flight crew to 

accomplish essentially the same procedure. 

Two subject groups, each consisting of twelve pilots and four controllers, participated in this 

experiment. Eight of the twelve pilot participants flew two-crew desktop simulators, and the 

remaining four pilots each flew in two-crew high-fidelity full-scale aircraft simulators. The four 

air traffic controllers in each group consisted of two en route controllers, one feeder controller, and 

one final controller. Confederate pilots and controllers were also used to provide background 

traffic and to control the airspace outside the simulated airspace. Scenarios included arrivals from 

all directions ending in either north or south arrival flows into the Denver International Airport, 

with a mixture of Required Navigation Performance (RNP) and Instrument Landing System (ILS) 

approach procedures. A total of 79 IM operations and 150 non-IM operations were flown by the 

subject and confederate pilots in this experiment. 
 2 

Experiment Results 

The ATD-1 ConOps, and in particular the IM procedure as tested in IMAC, was very successful 

and met most of the key metrics it was designed to achieve. Using the ATD-1 procedures, 

controllers were able to conduct high-density arrival operations into Denver without requiring the 

use of vectors within the terminal airspace. The mean IM spacing error at the Planned Termination 

Point (PTP) during IM operations was 1.0 second (STANDARD DEVIATION = 6.3 seconds). 

Eighteen percent of the IM operations were canceled by the controller (overwhelmingly the 

MAINTAIN operation which is not designed to be used as tested in this research), and none of the 

IM operations were canceled by the flight crew. 

Overall, both controllers and flight crew rated the ATD-1 ConOps, IM procedures, and IM displays 

as operationally acceptable, workload as acceptable, and pilots rated the amount of head down 

time required to conduct IM operations as acceptable. 

Although the ATD-1 concept and IM procedure demonstrated promise, many significant 

challenges were identified that must be resolved prior to the IM procedure being implemented in 

real-world operations. A partial list of issues identified during this experiment include: 

• the high frequency of IM commanded speed changes relative to non-IM operations, 

• the IM software commanding a speed increase (reversal), particularly on final approach, 

• confusion when a long time delay occurred between entering the IM clearance and the IM 

operation could commence, 

• issuing MAINTAIN clearances in the Air Route Traffic Control Center that continued into 

the Terminal Radar Approach Control facility, 

• the use of the Vertical Navigation (VNAV) Speed mode (which does not offer the same 

altitude protection as VNAV Path) to conduct IM operations, and 

• ambiguous cockpit displays indicating the IM spacing operation was no longer feasible. 

To the extent possible, each of these positive results and remaining challenges will be evaluated 

during the IM flight test scheduled for the spring of 2017. The results and recommendations of the 

IMAC experiment and the IM flight test will be used to inform follow-on development of the 

advanced IM concept (currently underway and led by the Federal Aviation Administration). 

 
 3 

1 Introduction 

1.1 Background 

To prepare the National Airspace System (NAS) for a predicted increase in traffic volume, and to 

improve the efficiency of the air transportation system, the Airspace Operations and Safety 

Program in NASA’s Aeronautics Research Mission Directorate created several Air Traffic 

Management (ATM) Technology Demonstrations (ATD), numbered one to three. These ATD 

Sub-Projects are designed to support commercial aviation stakeholders including the Federal 

Aviation Administration (FAA), manufactures, and airspace users, with relevant and timely 

research. 

The Langley Research Center Interval Management (IM) research team has been part of NASA’s 

ATD-1 Project for the past four years. This team was an integral part of the joint NASA Ames 

Research Center (ARC) and NASA Langley Research Center (LaRC) effort to develop the ATD1 Concept of Operations (ConOps), which integrates three NASA technologies to achieve high 

throughput, fuel-efficient arrival operations into a busy terminal airspace (ref. 1). These three 

technologies are: the Traffic Management Advisor with Terminal Metering (TMA-TM), which 

generates a precise arrival schedule to the runway threshold and other points within the airport 

terminal area; the Controller-Managed Spacing (CMS) decision support tools, which provide 

information to help terminal area air traffic controllers manage aircraft delay using speed control; 

and IM, which provides the speed guidance necessary to allow flight crews to manage their spacing 

behind an assigned lead aircraft. 

The LaRC IM research team has also worked closely with the FAA and industry partners for the 

past 15 years to support the development of performance standards for an array of aircraft 

surveillance applications (ref. 2) and the requirements for the airborne IM application (ref. 3). 

These two documents are the foundation for the IM Minimum Operational Performance Standards 

(MOPS, ref. 4), written by the FAA, NASA, and industry partners. Since this document was not 

published prior to this experiment, the ATD-1 Project used draft version 7.0 to create the ATD-1 

IM System Requirement Document (SRD), the system requirements for NASA’s IM avionics 

prototype (ref. 5). 

To evaluate the ATD-1 ConOps using the new IM application and performance standards, the 

ATD-1 Project chartered the Interval Management Alternative Clearances (IMAC) human-in-theloop simulation experiment to assess the acceptability and system performance of three proposed 

IM operations in realistic, high-density arrival operations. To accomplish this, the spacing software 

in the simulated aircraft avionics was expanded to enable two new IM operations, and new cockpit 

interfaces, displays, and alerting messages were developed and integrated into the simulation 

software to conduct this testing and validation (ref. 6). 

This document presents the results of this human-in-the-loop simulation experiment, a 

collaborative effort in the summer of 2015 by NASA Langley and NASA Ames, with input from 

the FAA. These results have contributed to the definition of requirements for the planned ATD-1 

flight demonstration in 2017 and to the FAA’s most recent concept of operations for Interval 

Management (ref. 7). 
 4 

1.2 Current Operational Need 

The 2015-2035 FAA Aerospace Forecast predicts U.S. commercial aviation revenue passenger 

miles will grow on average 1.8% annually throughout these twenty years (ref. 8). By 2035, U.S. 

commercial air carriers are projected to fly 1.71 trillion available seat-miles – approximately 167% 

of the seat-miles flown in 2014. Arrivals into high-density airports, especially during peak traffic 

periods and inclement weather, experience inefficiencies due to the use of miles-in-trail procedures 

and step-down descents. These procedures not only fail to achieve the airport’s maximum capacity, 

but also increase controller workload, arrival delay, aircraft fuel burn, emissions, and noise. While 

advanced Performance-Based Navigation (PBN) procedures exist at a limited number of sites (e.g., 

area navigation (RNAV) arrivals and optimized profile descents), they are not well utilized due to 

the lack of supporting scheduling and spacing tools. 

There is a need in airspace operations for deconflicted traffic operations that provide coordinated 

and achievable scheduling. Trajectories currently proceed to the airport terminal area, but do not 

necessarily connect to specific runways. While step-down arrivals and vectors by controllers help 

provide greater controllability of the aircraft’s trajectory, they are less fuel efficient than a PBN 

derived schedule. Decision support tools for controllers and pilots are needed to better manage 

arrival scheduling and throughput, thereby allowing the controller foreknowledge of known 

delays, which can be managed sooner using smaller speed variance. The use of speed control in 

place of step-downs and vectoring allows pilots to maintain their aircraft closer to optimum 

trajectories, thereby improving fuel efficiency. 

1.3 ATD-1 and IM Goals 

The high-level goals of the integrated ATD-1 ConOps include increasing the throughput of highdensity airports, increasing efficiency of arrival operations, and promoting aircraft Automatic 

Dependent Surveillance–Broadcast (ADS-B) equipage. The ATD-1 concept provides deconflicted 

and efficient operations of multiple arrival streams of aircraft from a point prior to Top-of-Descent 

(TOD) to the Final Approach Fix (FAF). Aircraft on these arrival streams primarily use speed 

control along their optimized profile descents to maintain adequate separation from other aircraft 

and to achieve precise schedule conformance, thereby decreasing the number of instances that 

aircraft are vectored off path or required to fly level-flight segments after the aircraft has passed 

the TOD (ref. 9). 

The goal of IM is to improve the precision of spacing between aircraft, thereby improving traffic 

flow and airport throughput and potentially reducing the overall voice communication requirement 

for controllers. 

1.4 Previous ATD-1 Research 

The IMAC experiment leveraged the knowledge gained from previous ATD-1 simulation research. 

From 2011 through 2015, the ATD-1 Sub-Project directed sixteen different simulation 

experiments that explored many aspects of the ATD-1 ConOps, in various traffic scenarios and at 

different airports (see Appendix C of ref. 1 for a summary of the experiments, objectives, and 

findings). These experiments used a range of benign medium-density operations to very complex 

and dynamic high-density operations based on the focus of the research. The ATD-1 research itself 

typically focused on enhancements to arrival scheduling automation and the acceptability of new 
 5 

decision support tools designed to help the controllers achieve the arrival schedule, on cockpit 

displays and the workload and acceptability to pilots, or on analyzing the overall system 

performance of the three NASA technologies in an integrated operation. 

As the final simulation experiment in the ATD-1 research effort, IMAC used all three ATD-1 

technologies, collected data from participating controllers and pilots, and incorporated all the 

lessons learned and best practices of previous ATD-1 research. This experiment also incorporated, 

for the first time in ATD-1 research experiments, two new IM operations (CAPTURE and 

MAINTAIN), and utilized continuous trajectories for the en route airway structure to the runway 

threshold (the arrival procedures and approach procedures into Denver International Airport 

(KDEN) connect to each other). 

Grouped by the three ATD-1 technologies, below are some of the operational procedures and 

techniques used in this experiment that were derived from previous NASA research. 

• TMA-TM (TBFM schedule): 

o A 0.3 nm spacing buffer is added to the minimum wake vortex separation criteria 

(ref. 10, 16, and 20) to account for variability in controller technique, pilot response 

time, and variation in the final approach speed between final approach fix and the 

runway (ref. 28, 29, and 30) 

o Trajectory used by TMA-TM should be almost identical to trajectory used by 

aircraft flight management system to reduce schedule error (led to decision to use 

Denver since arrivals and approaches connect to each other) (ref. 31) 

• CMS: 

o Feeder controllers use CMS displays to minimize the required delay; Final 

controllers use displays to manage the spacing between aircraft (ref. 31) 

o Controllers can use the CMS displays to more precisely control aircraft, and the 

flight crew find the procedures acceptable (ref. 20 and 32) 

• IM: 

o Controllers to issue IM clearance after IM aircraft has less than 60 seconds of delay 

(reduce the probability that the IM aircraft will require vectors off path) (ref. 33) 

o Controllers may use the Target aircraft’s call sign when issuing the IM clearance, 

however using the phonetic or alpha-numeric identifier is also acceptable (ref. 33) 

o Flight crew to use VNAV speed mode (enter IM speed into mode control panel 

instead of the flight management system) to reduce pilot head down time (ref. 11) 

o Flight crew should respond to a change to the IM speed within 10 seconds; the IM 

displays show the speed in reverse video for up to ten seconds until that speed is 

set in the mode control panel, then flashing reverse video is used (ref. 11) 

1.5 Terminology 

Over the course of the ATD-1 Project, the naming convention for some technologies and 

procedures has evolved, particularly when transferred to the FAA. The bulleted items below 

describe the terms used primarily in this document and the corresponding terms found in the 

reference material and in operational FAA use. 

• Time Based Flow Management (TBFM) is the FAA technology that incorporates NASA’s 

TMA-TM software. The terms are interchangeable in this document, with TMA-TM used 

when describing the original NASA technology in Section 2 and Appendix A, and TBFM 
 6 

used when describing the scenarios, operations, and results in the remainder of the 

document. 

• Terminal Sequencing And Spacing (TSAS) is the FAA term for enhancements to the 

Standard Terminal Automation Replacement System (STARS) for controllers in the 

Terminal Radar Approach Control Facility (TRACON), with Initial Operational Capability 

planned for 2019. These enhancements incorporate NASA’s TMA-TM and CMS 

technologies. The term TSAS is used predominately throughout the document, and TMATM and CMS when describing the original technology in Section 2. 

o TSAS operations as used in this document are essentially identical to current day 

operations in Air Route Traffic Control Center (ARTCC) airspace, but within 

TRACON airspace, the controller has enhanced TSAS decision support tools 

available to create a more precise arrival flow. 

• The term Flight deck Interval Management (FIM) originally described cockpit-specific 

equipment or displays, whereas IM described the overall integrated system of both ground 

tools for controllers and airborne tools for flight crews. The terms are interchangeable for 

the purposes of this document, with the term IM used almost exclusively in this document, 

while the references use the term FIM. 

o The term FIM is also used in the aircraft data tag of the controller displays to 

indicate an IM clearance has been issued to the flight crew, but the operation has 

not yet commenced (Section 2.2.2) 

• The IM “operation” occurs when the flight crew of the IM equipped aircraft fly the IM 

commanded speeds. The IM “clearance” is the voice instruction given by the controller to 

the flight crew. 

• The names of the three IM operations explored in this experiment were coordinated with 

an FAA data communication working group to ensure clear and concise terminology. The 

terms and the corresponding reference material definition are: 

o CROSS: Achieve-By and Then Maintain 

o CAPTURE: Capture and Then Maintain 

o MAINTAIN: Maintain Current Spacing 

 
 7 

2 ConOps, Technologies, Clearances, and Procedures 

2.1 ATD-1 Concept of Operation 

The ATD-1 ConOps combines advanced arrival scheduling, controller decision support tools, and 

aircraft avionics to enable deconflicted and multiple efficient arrival streams in high-density 

terminal airspace. To achieve increased fuel efficiency during periods of high traffic demand, 

aircraft use optimized profile descent procedures that include a transition from the arrival 

procedure to the instrument approach procedure of the assigned runway. 

When an arriving aircraft crosses a Freeze Horizon (tailored to each airport, nominally 150 to 250 

nautical miles from the airport), the TMA-TM tool assigns the most suitable runway and freezes 

the Scheduled Times of Arrival (STA) for the Meter Fix, terminal Meter Points, and runway 

threshold. The relevant schedule information is then shown to both en route and terminal 

controllers. 

En route controllers issue a clearance specifying the arrival procedure and expected runway to all 

aircraft, and they use the Delay Countdown Timer (DCT) shown on their displays to issue speed 

instructions for aircraft to achieve the STA calculated at the Meter Fix by TMA-TM. When the 

required delay is predicted to exceed the capability of speed-only operations, the en route controller 

will use path stretching (vectors) or step down the aircraft to lower altitudes to absorb the delay, 

then reverting to speed-only control when feasible. At that point, the controller will issue the flight 

crew the clearance to descend via the arrival procedure, and for equipped aircraft, issue the IM 

clearance. If the delay is not absorbed as expected, the controller interrupts the descend-via arrival 

procedure (and suspends the IM operation if applicable), then uses speed control instructions, 

vectoring, and/or altitude step-downs until the delay has been reduced. 

Terminal controllers are shown aircraft data, STA information, graphical slot markers, and speed 

advisories on their displays to correct the remaining schedule error or delay. The Feeder controller 

uses the slot marker as a spatial indicator to adjust the aircraft in order to meet the STA, and may 

also use the speed advisories as a guide. The Final controller may use the TMA-TM schedule 

information to assist in merging arrival flows, and use the TSAS speeds if appropriate; however, 

the Final controller’s primary responsibility remains ensuring proper separation on final approach. 

All flight crews fly the ATC assigned speed, the IM commanded speed, or the published speed 

during the arrival and approach. Flight crews of IM equipped aircraft are issued the IM clearance 

after the Freeze Horizon and after delay vectors are no longer expected. Depending on the type of 

operations, the IM information contained in the clearance consists of: the Target aircraft’s ID 

(identification, i.e., the airline code and flight number), the Target’s route of flight, the Assigned 

Spacing Goal (ASG), and the Planned Termination Point (PTP). The ASG is calculated by the 

TBFM software and is the desired interval (in time or distance) between the Target and IM aircraft 

at a particular waypoint, defined as the Achieve-By Point (ABP). (Note: during the MAINTAIN 

operation, the ASG is calculated by the IM avionics onboard the aircraft.) During IMAC, the ABP 

was set as the PTP, which was the FAF on the IM aircraft’s route. 

In the ATD-1 ConOps, controllers retain responsibility for the safe separation of all aircraft (TSAS 

or IM operations). For IM operations, the Ownship may be on either the same or different arrival 

procedure as the Target aircraft. Controllers will “suspend” IM operations if the need exists to 
 8 

momentarily vector either the IM aircraft or Target aircraft, and controllers or pilots may “cancel” 

the IM operation if conditions are no longer appropriate for that operation. 

2.2 The ATD-1 Technologies 

The three integrated NASA technologies described in the ATD-1 ConOps are (Figure 1): 

• Traffic Management Advisor with Terminal Metering (TMA-TM), providing precise 

arrival scheduling in the terminal airspace; 

• Controller Managed Spacing (CMS), providing TRACON controllers with decision 

support tools that enable precise schedule conformance; and 

• Interval Management (IM), providing flight deck automation that enables flight crew to 

achieve or maintain precise in-trail spacing behind the preceding aircraft. 

Figure 1. Integrated NASA technologies used in the ATD-1 ConOps. 

2.2.1 Traffic Management Advisor with Terminal Metering (TMA-TM) 

A key element of the ATD-1 ConOps is an advanced ground tool for ATM that generates a time 

deconflicted arrival schedule. This schedule sets the landing time interval between aircraft based 

on wake vortex separation criteria specified in ref. 10 (this interval is the ASG in the IM clearance) 

and calculates the airspeed required for an aircraft to achieve that interval. TMA-TM extends the 

basic TBFM scheduling capability by including terminal meter points, improved trajectory 
 9 

modeling inside TRACON airspace, and optimized flow of multiple arrival stream merges into an 

airport. The terminal delay model is enhanced to be more compatible with PBN procedures, and 

to enforce separation constraints at merge points within the terminal area. 

When TMA-TM has determined the STA for an aircraft, its position in the arrival stream, the 

amount of delay (if any), and the data necessary for an IM clearance, all of that information is 

displayed on the ARTCC controller’s meter list. All arriving aircraft are included in the meter list, 

with the appropriate meter fix STA (hhmm format) and delay (mm:ss format, with “+” indicating 

additional delay required) shown for each. The appropriate IM clearance information (i.e., Target 

aircraft call sign and route, the assigned spacing interval, and landing runway) is displayed for 

only the IM capable aircraft (Figure 2). 

The following items are contained within the list: 

• Aircraft call sign 

• Meter fix STA 

• Amount of delay to the meter fix 

• Achieve-By Point (*) 

• Spacing Goal (+) 

• Target aircraft 

• Target aircraft route (*) 

• Runway 

 (*) Note – These two items are only applicable to CROSS clearance operations. 

 (+) Note – This item is only applicable to the CROSS and CAPTURE operations. 

Figure 2. IM information displayed on the controller’s meter list. 

An example of the CROSS voice clearance that the ARTCC controller would issue based on the 

IM information displayed on the meter list in Figure 2, would be: 

“LOF3368, for Interval Spacing, CROSS LEETS 95 seconds behind ACA4255 on the CREDE 3 

Arrival. Report PAIRED.” 
 10 

2.2.2 Controller Managed Spacing (CMS) 

CMS are tools for TRACON controllers that assist in achieving their goal of maximizing 

throughput on capacity-constrained runways by ensuring they have knowledge of the same arrival 

schedule that en route controllers are using to manage arrival flows into the terminal airspace. The 

CMS tools provide the information necessary to more accurately achieve arrival schedule 

conformance using speed commands. This information is expected to allow TRACON controllers 

to reduce the use of tactical vectoring, thereby minimizing interruptions to fuel-efficient PBN 

arrival procedures. These CMS tools (Figure 3) function as follows: 

• Schedule Timeline (left panel) 

The timeline displays the TMA-TM-computed schedule at the scheduling point 

relevant for a particular controller position. Entries for each aircraft show the aircraft 

identification code and a symbol that identifies the aircraft’s weight class. Estimated 

time-of-arrival (ETA) entries appear on the left side of the timeline (always shown in 

white); STA entries appear on the right side. The STA is colored green for aircraft that 

have not initiated hand-off to the sector, bright white when the upstream controller 

initiates hand-off, and the same white as the ETA when the receiving controller accepts 

the hand-off. 

• Slot markers (top right and bottom right panels) 

Slot markers translate the schedule into a spatial target on a controller’s display. The 

slot marker circle indicates where an aircraft should be at a given time if it were to fly 

the arrival and approach procedures, meet the TMA-TM calculated STA at the Meter 

Fix and Meter Points, and arrive at the runway threshold on schedule. The 

instantaneous indicated airspeed of the slot marker is also displayed adjacent to the slot 

marker circle. The aircraft shown are travelling left to right (eastbound). In the top 

right panel, an aircraft that is close to on-time appears inside the circle, while in the 

bottom right panel an aircraft that is slightly early appears ahead of the circle. Note that 

the slot markers are always positioned along the arrival procedure used to schedule the 

aircraft, even if the associated aircraft has been temporarily vectored off the procedure. 

• Early/Late Indicators (top right panel) 

The Early/Late indicator is located in an aircraft’s data block and shown only when a 

speed advisory (described below) is not shown; i.e., when speed control is insufficient 

to absorb the required delay. It enables controllers to quickly assess the schedule 

conformance of that aircraft in a manner similar to the delay countdown timer presently 

available to ARTCC controllers. An early/late indicator is displayed using three 

characters in the third line of the data block, displaying the required delay with onesecond precision when the absolute delay is less than 100 seconds (e.g., -15 indicates 

an aircraft is fifteen seconds late); larger delay values are shown with one-minute 

precision (e.g., +2M indicates an aircraft is approximately two minutes early). Thus, 

the SWA353 example in Figure 3 is indicating that the TMA-TM schedule is estimating 

it to be three seconds early. 
 11 

• Speed advisories (bottom right panel) 

Speed advisories display airspeeds computed to put the aircraft back on schedule and 

are shown only when an early/late indicator (above) is not shown. The advised airspeed 

is computed using information about the TBFM reference speed profile along the 

assigned arrival and is displayed in ten-knot increments. If an aircraft is late, a speed 

increase may be advised. The speed advisories appear in the same three-character field 

on the third line of the data block that is used to display the early/late indicator. Thus, 

the speed advisory for SWA1184 is 190 knots, which is slower than the nominal speed 

of 210 knots for that segment, causing the aircraft to move towards the slot marker. If 

TMA-TM cannot compute a speed advisory different than the nominal speed, or the 

required speed is outside the available speed control margin, the early/late indicator is 

displayed instead. 

Figure 3. CMS tools and displays. 

Additional displays beyond the basic CMS tools have also been developed to help controllers 

monitor the status of IM operations in ARTCC (Figure 4) and TRACON (Figure 5) airspace. 

On ARTCC controller workstations, a special yellow “@” symbol above the aircraft’s call sign in 

its data tag indicates to the controller which aircraft are IM capable along with the status of the IM 
 12 

operation (Figure 4, left panel). When controllers issue an IM clearance to the flight crew, they 

enter a two character keyboard command that changes the “@” symbol from yellow to magenta in 

the aircraft’s data tag (Figure 4, center panel). The IM symbol in the aircraft data tag is a visual 

reinforcement that the IM clearance has been issued but the aircraft is not actively PAIRED with 

its Target. Upon receiving notification from the flight crew that the aircraft is PAIRED with its 

Target, the controller updates the data tag with a different keyboard command that changes the 

magenta “@” symbol to a magenta colored capital “S” (Figure 4, right panel). During the time 

period between when the controller issues the IM clearance until the flight crew reports PAIRED, 

TSAS operations remain in effect and the controller issues vectors, speed instructions, or altitude 

step-down instructions as required. 

If the IM operation is suspended with the intention of resuming the operation at a later time, the 

controller reverts the “S” in the data tag to the magenta “@”. This indication shows that the flight 

crew still has an IM clearance but is not actively PAIRED. When the IM operation is resumed, the 

controller changes the magenta “@” back to a magenta “S”. If the controller has no intention of 

resuming the IM operation and cancels it, the controller reverts the “S” to a yellow “@”. 

Figure 4. IM status indications on the ARTCC controller workstations. 

On TRACON controller workstations, from left to right in Figure 5, the TRACON data tag 

symbology indicates which aircraft do not have an IM clearance, which aircraft have an IM 

clearance but are not PAIRED (“FIM” located in the lower line), and which aircraft are PAIRED 

(magenta “SPC” in the lower line). If the aircraft enters the TRACON’s airspace indicating “FIM” 

(i.e., aircraft has been issued an IM clearance but has not initiated the IM operation) and 

subsequently reports PAIRED, the TRACON controller makes a two-character keyboard 

command that changes the “FIM” indication to “SPC”. 

If the IM operation is suspended with the intention of resuming the operation at a later time, the 

controller reverts the “SPC” in the data tag to “FIM”. This indication shows that the crew still has 

an IM clearance but is not actively PAIRED. When the IM operation is resumed, the controller 
 13 

changes the “FIM” back to a magenta “SPC”. If the controller has no intention of resuming the IM 

operation and cancels it, the controller removes the “SPC”. 

The CMS tools also provide the TRACON controller with the arrival sequence number of each 

aircraft to that runway. In Figure 5, the number ‘26’ in the bottom line of the data block indicates 

that UAL781 is the twenty-sixth aircraft in sequence to land on runway 35R. 

Figure 5. IM status indicators on the TRACON controller workstations. 

2.2.3 Interval Management (IM) 

2.2.3.1 IM Procedure 

The IM procedure enables the flight crew to achieve and/or maintain an ATC assigned spacing 

interval behind a specific aircraft landing on the same runway immediately in front of the IM 

aircraft, i.e., Target aircraft. Depending on the operational need of the controller and the 

geometrical relationship between the IM and Target aircraft, the controller may issue the IM 

clearance in either time or distance, and may dictate whether the ASG is to be achieved 

immediately or as late as the FAF. 

After the flight crew enters the clearance information into the IM user interface (procedurally 

designed to occur after the flight crew have already entered the Ownship route information and 

forecast wind) and once the software receives the position of the Target (from ADS-B transmitted 

messages), the algorithm calculates an airspeed for the flight crew to fly to meet the clearance. 

Once an airspeed is displayed to the flight crew, they assess the speed and determine as a crew if 

it is operationally acceptable or not. If it is, they manually enter this airspeed into the autoflight 

system via the Mode Control Panel (MCP). This procedure was recommended by an airline partner 

in previous simulation experiments to limit the head down time of the pilots, especially when 

operating below 10,000 feet (United Parcel Service in ref. 36). 

When the flight crew use this procedure to fly the IM commanded airspeed, shown both on the 

side-mounted Electronic Flight Bag (EFB) and the forward-mounted Configurable Graphics 
 14 

Display (CGD), previous research has indicated that the aircraft can cross the specified waypoint 

after the Target aircraft within five seconds of the ASG (ref. 11). 

2.2.3.2 IM Spacing Algorithm 

The basic goal of an airborne spacing algorithm is to provide an airspeed to the flight crew, which 

if flown, nulls the spacing error. The NASA-developed Airborne Spacing for Terminal Arrival 

Routes (ASTAR) algorithm uses detailed route information for both aircraft to allow spacing 

procedures to begin any time after the Target aircraft’s route is communicated to the IM aircraft. 

A more detailed description of the ASTAR algorithm is provided in Appendix A. 

In 2015, the algorithm was updated to version ASTAR13 to support new IM operations described 

in the IM industry standards (ref. 3 and ref. 4). These standards define five different IM clearance 

types: Achieve-by Then Maintain (CROSS), Capture Then Maintain (CAPTURE), Maintain 

Current Spacing (MAINTAIN), Final Approach Spacing (SPACE), and IM Turn (TURN). 

Prior versions of ASTAR consisted of only a trajectory-based speed control law, and therefore 

could only support the Achieve-by portion of the CROSS clearance. To enable the full 

implementation of the IM operations (except for the IM TURN), the previous ASTAR TrajectoryBased Operation (TBO) speed control law (ref. 12) was augmented with a state-based Constant 

Time Delay (CTD) speed control law (ref. 13 and ref. 14). In these supported clearance types, 

ASTAR ceases to provide speed commands for spacing at the PTP, which is a point on the 

Ownship’s route that is either designated by ATC, part of the planned procedure, or a default 

location close to the FAF. 

The TBO speed control law in ASTAR13 is designed to support IM operations both when the IM 

and Target aircraft are in-trail and when they are on merging routes. The spacing error is calculated 

using the time-to-go of the IM and Target aircraft along their predicted 4D trajectories. A 

proportional control algorithm with an additional ground speed compensation term is then used to 

determine the amount of speed compensation that is required to achieve a precise spacing interval 

at a controller designated achieve-by point. The IM commanded speed is calculated by adding the 

speed compensation to predicted 4D trajectory airspeed. A discretized IM command speed is 

shown to the pilots, who are expected to close the control loop by entering the speed into their 

aircraft’s mode control panel speed window. An instantaneous commanded speed is used to drive 

a FAST/SLOW indicator to help pilots maintain better speed conformance when decelerating or 

accelerating toward the discrete commanded end speed. Additional details of the TBO speed 

control law are described in Appendix A.3.1. 

The state-based CTD speed control law measures spacing error differently than the TBO speed 

control law, and can only be used when the IM and Target aircraft are in-trail. The measured 

spacing interval is measured as the difference in time between when the Target aircraft arrived at 

a particular along-path position and when the IM aircraft arrived at the same along path position. 

The spacing error is the difference between the measured spacing interval and the assigned spacing 

goal. A proportional control law is used to determine the amount of speed compensation that is 

needed to capture or maintain the assigned spacing goal. The IM commanded speed is calculated 

by adding the speed compensation to the Target aircraft’s time-history speed, i.e., the speed that 

the Target aircraft flew when it was at the IM aircraft’s current along path position. The IM 

commanded speed is discretized and filtered before being displayed to the flight crew, who are 

expected to close the control loop by entering the speed into their aircraft’s mode control panel 
 15 

speed window. Similarly to the TBO speed control law, an instantaneous commanded speed is 

used to drive a fast/slow indicator to help pilots maintain better speed conformance when 

decelerating or accelerating toward the discrete commanded end speed. Additional details of the 

TBO speed control law are described in Appendix A.3.2. 

2.2.3.3 IM Cockpit Human-Machine Interfaces 

The cockpit avionic devices used during this experiment to conduct IM operations included an 

EFB and a CGD for each pilot (Figure 6). A detailed description and illustrations of the EFB and 

CGD displays are given in Appendix B. 

 

Figure 6. IM Cockpit Interfaces: EFB (left) and CGD (right). 

 
 16 

A high-level description of the functionality of the IM displays to assist the flight crew in 

accomplishing the IM procedure and conduct the IM operation is given below. 

• A design goal of the IM cockpit display was to allow for intuitive data entry of Ownship 

data, forecast wind data, and IM clearance data. When entered in the typical sequence, the 

flight crew enters data in the EFB from left to right, top to bottom. Active data fields are 

highlighted in green, and arrows within the data field indicate another page will be 

presented when selected (similar to aircraft flight management systems). This functionality 

exists only on the EFB. 

• To minimize the time required for the flight crew to monitor the IM equipment, changes to 

the IM commanded speed were made salient (noticeable difference) by changing the 

background from black to green to highlight the new IM speed. For additional saliency, if 

the flight crew did not set the new IM speed within 10 seconds in the mode control panel 

speed window, the IM speed display cycles from black to green. Once the flight crew sets 

the new IM speed in the mode control panel, the background returns to black with the speed 

shown in green. This functionality exists on both the EFB and CGD. 

• The state of IM spacing algorithm (ARMED, PAIRED, etc.) and alert messages (Target 

off path, Ownship off path, etc.) are displayed to the flight crew. Although this 

functionality exists on both the EFB and CGD, only a subset of the most critical messages 

needed to conduct the IM operation are shown on the CGD. 

• A rate cue for decelerating the aircraft, called the FAST/SLOW indicator, allows pilots to 

quickly compare the relationship between the IM instantaneous speed to the aircraft’s 

current speed. This functionality exists on both the EFB and CGD. 

• A cue for the along-path position of the aircraft, called the EARLY/LATE indicator, 

provides the flight crew an awareness of their ability to meet the assigned spacing goal 

within the expected tolerance. This functionality only exists on the EFB due to insufficient 

space on the CGD, and only at certain times (as specified in ref. 4). 

The EFB is mounted on the outboard panel and the CGD just outboard of the aircraft’s navigation 

display (see Figures in Section 4). Since pilots frequently use the EFB for non-IM tasks (e.g., 

reviewing approach charts) and it is mounted outside of the pilot’s optimal primary field of view 

(FAA defined as within ± 15 degrees horizontally of a level line of sight, and vertically level to a 

30 degree downward line of sight), the smaller CGD is designed to show, within the pilot’s optimal 

forward field of view, only the critical subset of information needed to conduct the IM operation. 

2.3 IM Operation Types 

While the industry standard (ref. 3) defines five IM operations, the IMAC experiment assessed 

only three of the five IM operations: the CROSS, CAPTURE, and MAINTAIN. 

Of the three IM operation types used in this experiment, the CROSS operation is the one used in 

all previous IM research and ATD-1 experiments and requires the most information to be issued 

by the controller and entered into the IM software by the flight crew. While the CROSS operation 

is able to support complex merging geometries by the IM and Target aircraft, when the aircraft are 

in trail, much of the information in the clearance does not need to be included in the controller’s 
 17 

instruction. Thus the CAPTURE and MAINTAIN operations are alternatives that offer a lower 

workload for the controller and flight crew to accomplish essentially the same procedure. 

2.3.1 CROSS 

The goal of the CROSS operation is to achieve the ATC specified ASG at a designated ABP and 

then maintain that spacing interval until the PTP. The CROSS operation can be used either when 

the Ownship and Target aircraft are on different routes or when they are on the same route. CROSS 

is the appropriate IM operation type when the controller has a specific ASG to be met at an ABP. 

If the ABP and PTP coincide, there is no maintain phase of the IM operation. 

The CROSS operation type requires both the Ownship and Target aircrafts’ route information to 

be entered into the IM avionics, even if the routes are the same. Although the PTP is required by 

the IM industry standard and by the ASTAR algorithm, the ATD-1 ConOps makes it optional for 

the controller to issue it as part of the voice instruction since it is procedurally defined as the FAF. 

If the controller issues a PTP that is not the FAF, the flight crew modifies the PTP data field on 

the IM interface to comply. 

Data elements, their source, and entry type for the CROSS clearance are shown in Table 1. 

Table 1. Data Required for IM CROSS Clearance 

Data element Source of data Type of entry 

IM clearance type ATC instruction Manually by pilot 

Achieve-By Point (ABP) ATC instruction Auto by software, modifiable by pilot 

Assigned Spacing Goal (ASG) ATC instruction Manually by pilot 

Target aircraft identification ATC instruction Manually by pilot 

Target aircraft routing ATC instruction Manually by pilot 

Planned Termination Point (opt) ATC instruction Auto by software, modifiable by pilot 

Wind Data (optional) Weather service Manually by pilot 

2.3.2 CAPTURE 

The goal of the CAPTURE operation is to capture an ASG and then maintain that goal until the 

PTP (that is, there is no defined point at which the ASG must be achieved, rather a rate of closure 

specified in the IM industry standard is used). The CAPTURE operation is used when the Ownship 

and Target aircraft are on the same route and the controller has a specific ASG (in time or distance) 

to be kept between the two aircraft. There is no ABP since the spacing algorithm provides for a 

moderate rate of closure to the assigned spacing goal interval. 

Although the Target aircraft’s route is required by the IM industry standard and by the ASTAR 

algorithm, it is not issued verbally by the controller since it is procedurally defined as to be the 

same as the Ownship’s route. Although the PTP is required by the industry standard and by the 

ASTAR algorithm, the ATD-1 ConOps makes it optional for the controller to issue it as part of the 

voice instruction since it is procedurally defined as the FAF. If the controller issues a PTP that is 

not the FAF, the flight crew modifies the PTP data field on the IM interface to comply. The forecast 

wind entry is not required since the CTD speed control law does not use that data in the calculations 

for the IM commanded speed. 

Data elements, their source, and entry type for the CAPTURE clearance are shown in Table 2. 
 18 

Table 2. Data Required for IM CAPTURE Clearance 

Data element Source of data Type of entry 

IM clearance type ATC instruction Manually by pilot 

Assigned Spacing Goal (ASG) ATC instruction Manually by pilot 

Target aircraft identification ATC instruction Manually by pilot 

Target aircraft routing (optional) ATC instruction Auto by software, not modifiable 

Planned Termination Point (opt) ATC instruction Auto by software, modifiable by pilot 

2.3.3 MAINTAIN 

The goal of the MAINTAIN operation is to maintain the current spacing interval between the 

Ownship and Target aircraft until the PTP. Similarly to the CAPTURE operation, the MAINTAIN 

operation can only be used when the Ownship and Target aircraft are on the same route. Unlike 

the CROSS and CAPTURE operations that are intended for use during arrival operations when 

metering is in effect, the MAINTAIN operation is intended for use during en route metering 

operations or when metering is not in effect during arrivals. 

The ASG is calculated by the aircraft’s IM spacing software, and therefore is not included in the 

voice instruction by the controller. The Target’s route is also not included in the voice instruction 

since procedurally it must be the same as the Ownship aircraft’s route. Similar to the CROSS and 

CAPTURE clearance, if the PTP was not included in the controller’s voice instruction, the 

software auto-populates the FAF as the PTP and no action is required by the pilot. If the controller 

does issue a PTP waypoint, the pilot manually enters that waypoint into the IM interface. 

Since the TBFM scheduling function calculates the ASG to meet wake turbulence separation 

criteria at the FAF and that value is not passed to the flight crew, there is no expectation that a 

MAINTAIN operation initiated in the ARTCC would be suitable at the FAF. 

Data elements, their source, and entry type for the MAINTAIN clearance are shown in Table 3. 

Table 3. Data Required for IM MAINTAIN Clearance 

Data element Source of data Type of entry 

IM clearance type ATC instruction Manually by pilot 

Target aircraft identification ATC instruction Manually by pilot 

Target aircraft routing (optional) ATC instruction Auto by software, not modifiable 

Planned Termination Point (opt) ATC instruction Auto by software, modifiable by pilot 

2.4 Controller and Pilot Procedures 

This section provides a cursory overview of the controller and pilot procedures used during the 

experiment to conduct non-IM (TSAS only) and IM (TSAS plus IM for those aircraft equipped) 

operations. A more complete description of the pilot procedures and the controller-pilot 

phraseology required for these procedures are listed in Appendix C. 
 19 

2.4.1 Controller Procedures 

2.4.1.1 General 

Subject controllers had the responsibility for maintaining separation at all times during both TSAS 

and IM operations. They were also expected to avoid compromising the TBFM schedule by 

vectoring other aircraft to accommodate IM operations. All aircraft were considered RNAV 

equipped, and capable of Instrument Landing System (ILS) and Required Navigation Performance 

(RNP) approaches. 

The two confederate controllers assisted the research team in conducting the experiment and 

creating as realistic a simulation environment as feasible. The “Ghost” controller had the 

responsibility of communicating to all subject pilots and pseudo-pilots flying aircraft that did not 

initiate at the start of the scenario within one of the en route subject controller’s airspace. At the 

appropriate time, the “Ghost” controller used normal ARTCC hand-off procedures to ensure the 

pilots checked in to the correct en route controller at the sector boundary. The “Tower” controller 

had the responsibility to clear the pilots to land on the appropriate runway and to monitor the 

separation between aircraft while on final approach. 

2.4.1.2 En Route Subject Controllers 

For both non-IM and IM operations, the procedures began when the en route subject controller 

issued the descent clearance and advised the flight crew of which runway to expect. 

For non-IM operations, the en route subject controllers were expected to meter aircraft using 

conventional methods (i.e., speed changes, altitude step-downs, and vectoring) to achieve precise 

schedule conformance. If the en route controller had assigned the flight crew a speed other than 

that shown on the published Standard Terminal Arrival Route (STAR), that controller must either 

coordinate the speed assignment with the Feeder controller, or have the flight crew “resume normal 

speed” prior to entering the terminal airspace. 

For IM operations, the en route subject controllers were expected to absorb any metering delay in 

excess of one minute prior to advising the flight crew that an IM clearance was available. Once 

the crew responded they were ready to copy the IM clearance, the controller issued one of the three 

IM clearance types described in the previous section. 

The controller registered the IM clearance into TBFM using their keyboard, which caused the 

associated aircraft data tag to be displayed as a magenta colored @ symbol above the aircraft call 

sign (Figure 4). Upon notification that the aircraft was actively PAIRED and conducting the IM 

operation, the controller registered that information which caused the data tag to be displayed as a 

magenta colored S symbol above the aircraft call sign (Figure 4). The en route controllers’ 

procedures were completed when the aircraft was handed off to the TRACON Feeder controller. 

2.4.1.3 Terminal Controllers 

After receiving the handoff, the Feeder controller would issue the type of approach (ILS or RNP) 

and runway to expect. All “short side” (i.e., aligned with the landing traffic) aircraft were assigned 

the ILS approach, and all aircraft on the “long side” (i.e., on downwind and requiring a 180 degree 

turn to land) were assigned the RNP approach to the runway closest to their direction of arrival. 
 20 

This procedure was used to ensure the flight crew could be issued a continuous, published 

trajectory, from the en route airway structure to the landing runway. 

For non-IM operations, the controller used the TSAS decision support tools (i.e., slot markers, 

speed advisors, and delay count-down timer) to assist in issuing speed instructions or vectors to 

safely maximize throughput and achieve the schedule. 

For IM operations, TRACON controllers did not have information available to them to issue an 

IM clearance within the terminal airspace, nor was there indication of which aircraft were IM 

capable (based on discussions with the FAA on what functionality and capability would exist by 

2017 in their operational systems). Hence, all IM operations were initiated by en route controllers. 

The aircraft data-tag used by the terminal controller provided IM status information. If an aircraft 

had received an IM clearance but was not actively PAIRED with its Target, the data-tag would 

have FIM displayed in the bottom line. If the aircraft was actively PAIRED with its Target, a 

magenta colored SPC would appear in the bottom line. 

In the event that the IM aircraft entered the terminal airspace with FIM status shown and then 

became PAIRED with its Target within the terminal airspace, the controller would enter an FS 

keyboard command (same entry as the en route controller). This entry would then update the datatag from FIM to SPC to indicate active pairing. 

If the controller needed to suspend the IM operation momentarily with the expectation of allowing 

the operation to resume at a later time, the keyboard command FS would toggle the data-tag from 

SPC to FIM. This would indicate that the aircraft had an IM clearance but was following ATC 

commands versus those that were actively conducting IM operations. 

2.4.2 Flight Crew Procedures 

2.4.2.1 Non-Subject Pilots 

The six non-subject pilots flying the Multi-Aircraft Control System (MACS) pseudo-pilot stations 

(described in 4.1.4) used command line entry commands to control from one to eight different 

aircraft, which varied based on the scenario and time frame within that scenario. These aircraft 

directly correlated to one of the six controller stations, that is, the same MACS pseudo-pilot 

remained on the same controller frequency throughout the entire run, and any aircraft within that 

controller’s sector would be controlled by that MACS pilot. The MACS pilots used their displays 

to hand-off the aircraft to the next MACS pilot when directed by the current controller to change 

to the next controller. 

2.4.2.2 Subject Pilots Flying Non-IM Operations 

During BASELINE scenarios, and during any other scenario when an IM operation was not being 

conducted, the flight crews were expected to comply with published procedures and ATC 

instructions as they do during current real-world operations. A partial list of responsibilities 

included: (1) ensuring FMS was programed correctly; (2) responding to ATC instructions and 

queries; (3) initiating descent when instructed; (4) meeting lateral, vertical, and speed constraints 

as depicted on the published procedure or as directed by ATC; and (5) configuring the aircraft for 

final approach and landing. 
 21 

2.4.2.3 Subject Pilots Flying IM Operations 

When flying IM operations, the subject pilots were expected to meet the requirements and 

accomplish the tasks for current-day non-IM operations. In addition to these tasks, the pilots were 

also expected to conduct the tasks and procedures required for the IM operation. 

A design goal of the IM flight crew procedures was to make them consistent with the normal 

workflow of the pilots and mirror current-day arrival procedures. The procedures are divided into 

three distinct phases: programming the EFB, flying the arrival in vertical navigation (VNAV) 

speed mode, and configuring and landing in VNAV Path mode. 

The flight crew are expected to enter two categories of information into their EFB display: 

Ownship and wind information, and the IM clearance (see Appendix B for descriptions of the IM 

software, displays, messages, indications, and data entry). This programming may occur at 

different times throughout the flight, depending on the type of data entry and the flight crews’ 

workload. Ownship and wind information were entered by the flight crew well prior to arriving at 

the destination airport, and preferably during a low workload phase of flight, such as prior to topof-descent. Using the EFB interface, the destination airport, arrival routing with transition, and the 

approach were selected. Forecast descent and surface winds were then loaded by Aircraft 

Communications Addressing and Reporting System (ACARS) uplink or manually entered. While 

the forecast winds were not required to be entered, they provide improved system performance by 

enabling the IM spacing software to more accurately calculate the Ownship and Target time-togo. Therefore, an alert message was displayed on the EFB when the flight crew did not enter the 

forecast winds. 

The IM clearance information was provided to the flight crew in a voice instruction from an 

ARTCC controller either prior to top of descent or shortly thereafter. This information included 

the type of IM clearance as well as the data elements required by that clearance type. Once the IM 

clearance was received, the flight crew was required to enter the clearance into the EFB. 

When all required data had been entered for the clearance type, the flight crew activated the IM 

system. If all requirements were satisfied, the IM system switched to ‘PAIRED’ mode and the IM 

commanded airspeed was displayed to the flight crew. At that time, the pilot was required to open 

the MCP speed window and set the commanded speed. During the remainder of the arrival, the 

flight crew flew the IM commanded speed displayed on the IM avionics while using thrust and 

drag to stay on their VNAV path. For large speed changes, the pilot followed a FAST/SLOW 

indicator that graphically showed the deviation between the airspeed of the Ownship aircraft and 

the speed expected by the spacing algorithm. The EFB may have been used for other functions at 

this time since all the information that the pilot needed to conduct IM operations was displayed on 

the CGD in the pilot’s forward field of view. 

Once the aircraft was on a published portion of the approach and in VNAV path mode, the pilot 

used thrust and drag to maintain the IM commanded airspeed. As the aircraft approached the 

destination, the EARLY/LATE indicator activated and graphically showed the spacing error. At 

the PTP (set as the FAF in this experiment), the IM operation is complete, and the IM information 

was removed from the CGD and EFB displays. At this time the flight crew configured the aircraft 

for landing, and slowed to the final approach speed. 

Other IM procedures enabled controllers to suspend, amend, or terminate an IM operation. The 

flight crew could also notify ATC if they desired to terminate the IM operation. 
 22 

3 Experiment Design 

3.1 Objectives and Scope 

The objectives of this experiment were to assess the acceptability and system performance of three 

IM operation types in realistic, high-density arrival scenarios and to identify implementation issues 

for real-world operations. In particular, IMAC was intended as a risk reduction activity to explore 

the new IM operations (CAPTURE and MAINTAIN), prototype a new avionics design, and 

investigate changes to the IM algorithm design, in preparation for an ATD-1 flight demonstration 

planned for 2017. Therefore, the anticipated capabilities and limitations of the air traffic system at 

that time were modeled in this experiment. 

For simulation efficiency and experimental control, during each scenario only one-half of the 

aircraft were controlled by controllers and flown by pilots (all aircraft east of Denver, or all aircraft 

west of Denver). The aircraft on the opposite side of the airfield were autonomously flown aircraft 

that did not interact with the controllers, pilots, or human flown aircraft. These autonomous aircraft 

were not included in any analysis discussed in this paper, and were only present in the simulation 

to provide additional traffic on the controllers’ and pilots’ displays. Thus the experimental scope 

was focused on arrivals to a single runway, and the more complex system-wide events affecting 

multi-runway operations were not explored. 

3.2 Assumptions 

To retain sufficient focus on the experiment’s objectives and obtain statistically significant data, 

several simplifying assumptions were made and are listed below. 

• Reception of ADS-B data from another aircraft was limited to 120 nmi or less. 

• No positional error was added to either ADS-B or radar data. 

• The TBFM runway load-balancing function was turned off. 

• Controllers were restricted to landing aircraft on the originally assigned runway. 

• Aircraft go-arounds or missed approaches were not included. 

• Aircraft emergencies or events requiring priority landing were not included. 

• Changes to the airport configuration were not included. 

• Dynamic convective weather requiring re-routing was not included. 

3.3 Performance Goals and Hypotheses 

The experiment was designed to meet the objectives by evaluating a subset of the ATD-1 Measures 

of Performance (MOP, ref. 19) consisting of: 

• Success rate of PBN operations (MOP 3.2.2) 

• Percentage of controller-interrupted FIM operations (MOP 3.3.3) 

• Percentage of flight deck-interrupted FIM operations (MOP 3.4.3) 

• Flight crew acceptability of FIM operations (MOP 3.4.4) 

• Flight crew workload of FIM operations (MOP 3.4.5) 

• FIM spacing goal conformance (MOP 3.4.1) 

• Inter-arrival spacing error (MOP 3.3.6) 
 23 

• Controller acceptability of ATD-1 operations (MOP 3.3.4) 

• Controller workload of ATD-1 operations (MOP 3.3.5) 

All of the hypotheses and performance goals are based on the ATD-1 MOP Specification, and are 

listed in Table 4. The right column of Table 4 lists the section of this document that contains the 

analysis and results for that performance goal or hypothesis. For the four performance goals, the 

limited sample size based on resource constraints was insufficient for statistical hypothesis testing. 

Instead, the MOP performance goals were assessed using descriptive statistics based on the data 

observed in the experiment, and so conclusions cannot be drawn for the larger population. 

Hypotheses for five of these metrics were determined a priori, and the experiment was designed 

to enable the evaluation of these hypotheses via statistical hypothesis testing. 

Additional metrics of interest include the number of aircraft separation violations, the operational 

efficiency of the CMS tools and procedures, ASTAR spacing algorithm performance, aircraft 

schedule deviation at the FAF and Meter Fix, and operational issues with implementing IM 

procedures. These metrics were investigated using exploratory analysis and results are discussed 

in Section 5. 

Table 4. Correlation of Performance Goals and Hypotheses to Section of Results 

# Performance Goals Section 

of MOP 

Section of 

Results 

1 The percentage of IM arrivals that have spacing errors at the ABP 

within 10 seconds will be ≥ 95% 3.4.1 5.3.1 

2a The inter-arrival spacing error of 68% of IM operations 

following either a TSAS or an IM operation will be ≤ 8 seconds 3.3.6 5.4.2 

2b 

The inter-arrival spacing error of 68% of TSAS operations 

following a TSAS operation or an IM operation will be ≤ 12 

seconds 

3.3.6 5.4.2 

3 The mean controller acceptability rating of ATD-1 operations 

will be ≥ ‘8’ on a scale of 1 – 10 3.3.4 5.7.1 

4 

The mean controller workload rating for Mental Demand, 

Physical Demand, Time Pressure, and Effort will be ≤ ‘5’, and 

for Success and Frustration will be ≤ ‘3’ on a scale of 1 – 7 

3.3.5 5.7.2 

# Hypotheses Section 

of MOP 

Section of 

Results

1 The percentage of uninterrupted PBN operations within the 

TRACON using ATD-1 tools will be > 70% 3.2.2 5.4.2 

2 The percentage of IM operations terminated by ATC prior to the 

Planned Termination Point will be < 30% 3.3.3 5.5.1 

3 The percentage of IM operations terminated by the flight crew 

prior to the Planned Termination Point will be < 30% 3.4.3 5.6.1 

4 The mean flight crew acceptability rating of IM operations will 

be ≥ ‘5’ on a scale of 1 – 7 3.4.4 5.8.1 

5 The mean flight crew workload ratings of IM operations will be 

≤ ‘3’ on a scale of 1 – 10 3.4.5 5.8.2 
 24 

3.4 Test Matrix 

The experiment utilized the 1x5 test matrix shown in Table 5 to assess the acceptability and 

performance of three different types of IM operations. Although hypothesis tests were not 

conducted to formally compare the operation types, differences were investigated using 

exploratory analysis. Two replicates of each scenario were conducted – one with the Captain as 

the Pilot Monitoring (PM) and the First Officer as the Pilot Flying (PF), and the other replicate 

with the roles reversed. Therefore, each flight crew flew a total of ten scenarios during data 

collection. 

Table 5. IMAC Experiment Test Matrix 

BASELINE CAPTURE CROSS MAINTAIN MIXED 

Scenario A 

Scenario B 

Scenario C 

Scenario D 

Scenario E 

Scenario F 

Scenario G 

Scenario H 

Scenario I 

Scenario J 

A within-subject design was employed, and the run order for the BASELINE, CAPTURE, 

CROSS, and MAINTAIN scenarios was partially counterbalanced using a Latin square design 

(ref. 18). No IM operations were conducted in BASELINE scenarios (TSAS only operations), and 

IM operations of the type corresponding to the cell name were conducted in the three remaining 

scenarios. 

These eight scenarios were followed by the two MIXED scenarios in which the controllers could 

issue a clearance for any of the three IM operations to any suitably equipped aircraft, based on 

their preference and judgment. The MIXED scenarios were intended to emulate a more dynamic 

air traffic control environment, thereby providing the subject controllers a more realistic 

operational context in which to explore the usability and acceptability of the different IM clearance 

types and their corresponding operation. 

Data were collected and analyzed from two groups of subjects, each consisting of four retired air 

traffic controllers and six two-person crews of commercial airline pilots, for a total of eight subject 

controllers and 24 subject pilots. In addition, each group also had two confederate controllers and 

six confederate pilots that assisted the research team in creating realism in the high-density arrival 

operations. 

3.5 Independent Variable 

The independent variable in the experiment was the type of TSAS or IM operation conducted: 

BASELINE, CAPTURE, CROSS, MAINTAIN, and MIXED. The baseline condition (i.e., TSAS 

tools and operations with no IM operations) was included to allow for comparison of system 

performance with and without IM operations. The CAPTURE, CROSS, and MAINTAIN 

scenarios provided the ability to individually assess each of the three IM alternative clearance 

types. In the MIXED scenarios, controllers had the flexibility to issue a clearance for any of the 

three IM operations. 
 25 

3.6 Dependent Measures 

This section briefly describes six categories of dependent measures analyzed in the experiment. A 

seventh category, case studies, illustrates various salient issues or conclusions observed 

independent of the primary metrics. Some of the calculations used in the analysis are further 

described in the ATD-1 Measures of Performance Specification document (ref. 19). 

3.6.1 IM Algorithm Performance 

The three main IM algorithm performance metrics analyzed as part of the experiment were: the 

IM spacing error at the PTP (typically the FAF), the frequency of IM speed changes, and the 

number of IM speed reversals. These metrics focused on verifying that the IM operations provided 

acceptable performance. Analysis and results for these dependent measures are presented in 

Section 5.3. 

The first metric (and Goal #1), the spacing goal conformance at the PTP, is the difference in time 

between the ASG and the difference in the time between when the Target and the Ownship crossed 

the PTP. The spacing goal conformance at the PTP is calculated using data from the ASTAR 

spacing algorithm for every IM aircraft that successfully conducted IM operations to the PTP. The 

minimum success criteria for this metric was for at least 95% of the spacing errors to be within 10 

seconds of the assigned spacing goal (ref. 19, para 3.4.1). 

The second metric was the number and frequency of IM speed changes commanded by ASTAR. 

The frequency of IM speed changes is the number of speed changes commanded by ASTAR 

divided by the total number of minutes that IM operations were conducted. The number and 

frequency of speed changes do not include the initial speed commanded by ASTAR at the 

beginning of an IM operation. The number and frequency of IM commanded speed changes 

includes published speed changes, since the pilot is still responsible for recognizing the new speed 

command and manually entering it into the aircraft. The criteria is for the speed change frequency 

to be less than two speed changes per minute. This criteria is based on a heuristic that was 

developed based on previous IM research (ref. 11). 

The third metric in the IM algorithm performance section was the number of IM commanded 

speeds that reversed direction, that is, commanded an increase to the aircraft’s speed where the 

typical (and operationally preferred) speed command during arrival operations is to slow down. 

3.6.2 Air Traffic System Performance 

The three system performance metrics analyzed for the experiment were: the inter-arrival spacing 

error at the FAF, the PBN success rate, and slot marker deviation in the TRACON. These air traffic 

system performance and results are presented in Section 5.4. 

The first metric (and Goal #2a and #2b), is the inter-arrival spacing error at the FAF, which is a 

measure of how well aircraft achieved their scheduled time intervals at the FAF. The inter-arrival 

spacing error at the FAF is calculated using the following equation: 

(ଵି௜ܣܵܶ − ௜ܣܵܶ) − (ଵି௜ܣܶܣ − ௜ܣܶܣ) = Error Spacing arrival – Inter

where ATA୧ is the actual time of arrival (ATA) of the aircraft of interest, ATA୧ିଵis the actual time 

of arrival of the preceding aircraft that arrived at the FAF, STA୧ is the scheduled time of arrival of 
 26 

the aircraft of interest, and STA୧ିଵ is the scheduled time of arrival of the preceding aircraft that 

arrived at the FAF. The inter-arrival spacing error at the FAF was calculated for all aircraft in the 

simulation using state data from the aircraft and schedule information from TBFM. For all IM 

operations except for the MAINTAIN operation, the difference between the scheduled times of 

arrival (ܵܶܣ ௜− ܵܶܣ௜ିଵ) was used to establish the ASG, and was provided to the flight crew in the 

IM clearance. For the MAINTAIN operation, the spacing goal was calculated by the IM software 

and was the interval between the Ownship and Target aircraft when the Target was within ADSB range and the flight crew pressed the EXECUTE button. 1 The criteria for this metric was for 

68% of the non-IM aircraft to have inter-arrival spacing errors less than or equal to 12 seconds, 

and for 68% of the IM aircraft to have an inter-arrival spacing error less than or equal to 8 seconds 

(ref. 19, para 3.3.6). 

The second metric (and Hypothesis #1), the success rate of PBN operations (that is, both non-IM 

and IM operations), is a measure of how consistently aircraft stayed on their planned routes when 

in the TRACON, and it is measured as the percentage of aircraft that were not vectored in the 

TRACON (ref. 20). The criteria for this metric is that less than 30% of aircraft receive vectors off 

path in the TRACON (ref. 19, para 3.2.2). This ATD-1 Project defined metric and criteria was 

established to provide continuity to previous ATD-1 research, and may not be appropriate for 

evaluating real-world operations. 

The third metric, the slot marker deviation, is the root mean square (RMS) of the distance between 

the center of the slot marker and corresponding aircraft’s position throughout the arrival and 

approach operation. Since the slot markers are only displayed in the TRACON, only the data 

accrued when the aircraft of interest was in the TRACON was used for this metric. This metric 

was established by the research team to provide a method to compare how TRACON controllers 

interacted with aircraft conducting IM operations and those not conducting IM operations, and 

may not be appropriate for evaluating real-world operations. 

3.6.3 Controller Objective Performance 

The two controller performance metrics analyzed for the experiment were: percentage of 

controller-interrupted IM operations, and schedule deviation at the FAF and Meter Fix. Analysis 

and results for these dependent measures are presented in Section 5.5. 

The first metric (and Hypothesis #2), the percentage of controller-interrupted IM operations, is the 

number of IM operations that were either canceled or suspended by an air traffic controller. The 

criteria for this metric is that air traffic controllers should interrupt less than 30% of the IM 

operations (ref. 19, para 3.3.3). This ATD-1 Project defined metric and criteria was established to 

provide continuity to previous ATD-1 research, and may not be appropriate for evaluating realworld operations. 

The second metric, the schedule deviation, is the difference between an aircraft’s scheduled time 

of arrival and the actual time of arrival at the FAF and TRACON meter fix. This metric was 

 

1

 Since the ASG is used in the CAPTURE and CROSS clearances is derived from the TMA-TM schedule and issued 

as part of the IM clearance, the inter-arrival spacing error will be the same as the spacing goal conformance metric 

presented in section 4.6.1 (with the exception of small differences caused by sampling from different data sources). 

The inter-arrival spacing error and spacing goal conformance for the MAINTAIN clearance will be different, since 

the ASG is not derived from the TMA-TM schedule. 
 27 

established by the research team to provide a method to compare how TRACON controllers 

interacted with aircraft conducting IM operations and those not conducting IM operations, and 

may not be appropriate for evaluating real-world operations. 

3.6.4 Flight Crew Objective Performance 

The five flight crew performance metrics analyzed for the experiment were: percentage of flight 

crew interrupted IM operations, flight crew reaction time to changes in the IM commanded speed, 

flight crew conformance to the IM speeds, missed altitude constraints, and unstable approaches. 

The flight crew performance analysis and results are presented in Section 5.7. 

The first metric (and Hypothesis #3), the percentage of IM operations terminated by the flight 

crew, is the number of IM operations that were initiated successfully, but then terminated by the 

flight crew (but not due to controller instruction) prior the PTP. The criterion for this metric is for 

flight crew to terminate less than 30% of the IM operations (ref. 19, para 3.4.3). This ATD-1 

Project defined metric and criteria was established to provide continuity to previous ATD-1 

research, and may not be appropriate for evaluating real-world operations. 

The second metric, flight crew reaction time, was the time it took from when the IM software 

commanded a new speed until the flight crew began to set that value in the MCP speed window as 

recorded in the data log. This ATD-1 Project defined metric was used to provide continuity to 

previous ATD-1 research. 

The third metric, flight crew conformance to the IM speed, measured the RMS of the difference 

between the IM instantaneous speed and the aircraft’s actual airspeed. This metric was established 

by the research team to provide a method to assess how closely pilots decelerated the aircraft 

compared to the speed calculated by the spacing algorithm, and may not be appropriate for 

evaluating real-world operations. 

The fourth metric, altitude constraints missed by the flight crew, analyzed the number of waypoints 

where the aircraft did not meet an altitude crossing constraint. These events were examined by 

simulator type and by IM operation type to understand their impact on the missed constraints. This 

metric was established by the research team to provide a method to assess how difficult the pilots 

found the use of VNAV speed while conducting IM operations. 

The fifth metric, not achieving a stabilized approach at 1000 feet above the ground (specified in 

Section 5.7.5), was examined by simulator type. This metric was established by the research team 

to provide a method to assess how difficult the pilots found the use of VNAV speed while 

conducting IM operations. 

3.6.5 Controller Subjective Assessments 

Subject questionnaires were developed using Lime Survey (ref. 35). A background questionnaire 

was completed by subjects prior to the start of the experiment on the first day. Post-run 

questionnaires were issued at the end of the final training run and immediately following all 

experimental runs thereafter. Upon completion of the final post-run questionnaire, a postexperiment questionnaire was completed by all subjects. Researchers remained on-hand to clarify 

questions relating to the questionnaires, if asked. All the questionnaires themselves are available 

in Appendix E. 
 28 

The primary controller subjective assessments from post-run and post-experiment surveys 

discussed in the body of this document (Section 5.6) are: the acceptability of ATD-1 operations, 

the workload of ATD-1 operations, and comments about the ATD-1 ConOps and IM operations. 

The entire results and controller assessments are in Appendix F.3 and F.5. 

The first metric (and performance goal #3), the controller acceptability rating of the ATD-1 

operation, used the Controller Acceptance Rating Scale (CARS, ref. 21). The criteria for this 

metric is for the mean rating to be equal to or greater than ‘8’. (ref. 19, para 3.3.4). 

The second metric (and Performance goal #4), the controller workload rating, used a measurement 

derived from the NASA Task Load Index (TLX) scale (ref. 19, para 3.3.5). The criteria for this 

metric is the mean for Mental Demand, Physical Demand, Time Pressure, and Effort will be equal 

to or less than ‘5’, and for Success and Frustration will be equal to or less than ‘3’. 

3.6.6 Flight Crew Subjective Assessments 

Subject questionnaires were developed using Lime Survey (ref. 35). A background questionnaire 

was completed separately by each subject pilot (two questionnaires per crew) prior to the start of 

the experiment on the first day. Separate post-run questionnaires were also issued to each subject 

pilot at the end of the final training run, and immediately following all experimental runs thereafter. 

Upon completion of the final post-run questionnaire, a post-experiment questionnaire was 

individually completed by all subject pilots. (This Section is intentionally labeled “flight crew” to 

imply that the simulators were flown as two-person crews, and each pilot was assigned specific 

PF or PM duties per scenario.) Researchers remained on-hand to clarify questions relating to the 

questionnaires, if asked. The questionnaires are available in Appendix E. 

The primary flight crew subjective assessments from post-run and post-experiment surveys 

discussed in the body of this document (Section 5.8) are: the acceptability of ATD-1 operations, 

the workload of ATD-1 operations, comments about situation awareness, comments about the 

ATD-1 ConOps and IM operations, and comments about the intuitiveness and usefulness of the 

IM interface and displays. The entire results and flight crew assessments are in Appendix F.4 and 

F.6. 

The first metric (and Hypothesis #4), the flight crew acceptability of the IM operation, was 

measured using a 7-point Likert scale. The criteria for this metric is for the mean rating to be equal 

to or greater than ‘5’. Additional analysis was conducted to confirm different aspects of the flight 

crew level of acceptance of the IM operation. 

The second metric (and Hypothesis #5), the flight crew workload of the IM operation, was 

measured using a 10-point Modified Cooper-Harper scale (ref. 22). 2 The criteria for this metric is 

the mean rating to be equal to or less than a ‘3’. 

3.7 Case Studies 

Five different types of case studies were conducted to identify and communicate more complex 

behaviors and events observed separately from the above analysis, and to provide a better 

 

2

 The Modified Cooper-Harper workload scale was used for pilots and the TLX scale was used for air traffic controllers 

to maintain consistency with the methodologies used in previous ATD-1 research. 
 29 

understanding of IM algorithm performance and the impact of various controller and pilot actions 

that occurred during the simulation. The case studies described in Section 5.9 include: 

• loss of separation between aircraft; 

• an example of undesirable propagation of IM speed changes through an arrival stream; 

• an example of desirable IM algorithm performance; 

• variations to the normal IM clearance and operations; and 

• the impact of the Target aircraft not flying the published speed. 

3.8 Environment and scenario description 

The experiment environment was the airspace and instrument arrival procedures into KDEN. 

KDEN was specifically chosen since the published STARs connected to either to a RNP or ILS 

approach, a requirement for ATD-1 operations in general, and IM in particular. Although 

becoming more common, few other major airports currently have this characteristic. 

3.8.1 KDEN Instrument Arrival and Approach Procedures 

Part of the FAA’s contribution to the IMAC experiment was having Denver air traffic control 

experts come to LaRC to brief the research team on arrival operations and real-world challenges 

at KDEN. In addition, they arranged a site visit to the Denver ARTCC and TRACON facilities for 

some of the research team members. The discussions with the controllers during this visit were 

invaluable in making the scenarios as realistic as possible, as was the documentation of standard 

operating procedures they provided (ref. 23, paragraphs 3-4-7 and 3-4-8). 

From these discussions and documentation, STAR/approach/runway combinations were selected 

for the scenarios. The north-flow combinations are shown in Figure 7 and Table 6, and the southflow combinations are shown in Figure 8 and Table 7. (Note: the WAHUU1 STAR does not 

connect to runway 17R when landing to the south, therefore this arrival to approach combination 

was not used in the experiment.) Both north and south flow landings were used in the experiment 

to reduce the possibility of learning effect by the subject controllers and pilots by having a wide 

range of scenarios in the experiment. 

Two characteristics incorporated into the scenarios to facilitate the research of the ATD-1 ConOps 

and IM procedures that do not exist yet in current day operations at KDEN are: 

• the simultaneous ILS and RNP approach operations to the same runway, and 

• simultaneous RNP approaches to parallel runways (incorporated to create a visually more 

complex and complete appearance of airport operations, but had no impact on data 

collection or results since the arrival streams did not interact). 

To facilitate the software development of a consistent experimental environment, the 1408 (18 

September 2014) navigation database was used for the KDEN RNAV STARs and RNP Z 

approaches. This ensured that the flight management systems in the three different aircraft 

simulators matched the controller waypoints and graphics as well as the TBFM adaptation needed 

to create the arrival schedule. 
 30 

Figure 7. Arrivals into KDEN for north-flow landing. 

Table 6. STARs and Approaches by Sector for North-Flow Landing 

Sector STAR Meter Fix Common 

Waypoint 

Approach 

AR1 (SW) LDORA2 LARKS LDORA ILS 35L 

AR1 (SW) TELLR2 POWDR TELLR ILS 35L 

AR2 (NW) FRNCH3 TOMSN HIMOM RNP Z 35L 

AR2 (NW) MOLTN3 RAMMS HIMOM RNP Z 35L 

AR3 (NE) ANCHR2 LANDR DOGGG RNP Z 35R 

AR3 (NE) KOHOE2 SAYGE DOGGG RNP Z 35R 

AR4 (SE) PURRL2 DANDD PURRL ILS 35R 

AR4 (SE) BOSSS2 QUAIL BOSSS ILS 35R 
 31 

Figure 8. Arrivals into KDEN for south-flow landing. 

Table 7. STARs and Approaches by Sector for South-Flow Landing 

Sector STAR Meter Fix Common 

Waypoint 

Approach 

AR1 (SW) PEEKK3 LARKS CLFFF RNP Z 16L 

AR1 (SW) CREDE3 POWDR CLFFF RNP Z 16L 

AR2 (NW) KAILE2 TOMSN KAILE ILS 16L 

AR2 (NW) TSHNR2 RAMMS TSHNR ILS 16L 

AR3 (NE) KIPPR2 LANDR KIPPR ILS 17R 

AR3 (NE) WAHUU1 SAYGE n/a n/a 

AR4 (SE) JAGGR3 DANDD QWIKE RNP Z 17R 

AR4 (SE) ZPLYN3 QUAIL QWIKE RNP Z 17R 
 32 

3.8.2 KDEN Airport and Landing Runways 

As mentioned in the preceding section, the discussion with controllers in the Denver FAA facilities 

and their standard operating procedures (ref 23, paragraphs 3-4-7 and 3-4-8) led to the selection of 

specific runways for this experiment. These runways are typical configurations for dual instrument 

approaches. The runways used during north-flow landing scenarios (35L and 35R) are shown in 

orange in Figure 9, and the runways used during south-flow scenarios (16L and 17R) are shown 

in purple. 

Figure 9. KDEN runways used for landing during IMAC. 
 33 

3.8.3 Truth and Forecast Wind 

Wind and wind error were not independent variables in this simulation; therefore, one truth and 

one forecast wind was used throughout the experiment. Since only one wind was used, it had to 

not only emulate winds typically observed at Denver, but also allow for aircraft landing either to 

the north or the south. Each simulator provided the subject pilots a simulated ACARS data link 

message containing the descent forecast winds, which the flight crew loaded via a button press 

into the FMS. Once the forecast descent wind information was loaded in the FMS, a button 

becomes available on the IM equipment (Figure 69). Pressing this button loaded the same descent 

forecast winds into the IM equipment. The flight crew were given a text version of the airport 

surface winds, which they entered manually into the surface forecast wind data field of the IM 

interface. 

To ensure a representative wind condition was used, Denver airspace wind data over a one year 

period was analyzed, and the following characteristics were incorporated into the one truth and 

one forecast wind field used in the IMAC experiment: 

• winds west of Denver were very strong from the west and varied little by altitude; 

• winds east of Denver were strong from the west and varied significantly by altitude; 

• surface winds at the Denver airport was a light cross wind from the west. 

Particular values from the wind fields include: 

• Average speed of truth wind: 67 knots @ FL350, 32 knots @ 10,000’, 14 knots @ 5000’ 

• Average speed of forecast wind: 69 knots @ FL350, 16 knots @ 10,000’, 13 knots @ 5000’ 

• Truth winds at KDEN: 279/71 @ FL350, 206/19 @ 10,000’, 023/12 @ 5000’ 

• Forecast winds at KDEN: 277/69 @ FL350, 322/15 @ 10,000’, 328/11 @ 5000’ 

 
 34 

3.8.4 Training Scenarios 

A training program was conducted that began with several part-task runs, which consisted of each 

simulator type operating in a stand-alone mode to allow subjects to gain familiarity with their 

particular simulator and the basic mechanics of conducting the TSAS (controllers) and IM (pilots) 

operations. Once basic proficiency was obtained, the training then progressed to four scenarios 

that covered each of the four arrival directions and two landing runway configurations. These four 

“connected runs” were similar to the data collection scenarios discussed in the next section. A 

description of the training scenarios is shown in Table 8. 

Table 8. Description of Training Scenarios 

Scena

rio 

Controllers & Non-Subject Pilots Subject Pilots 

Arrival

(E or W)

Land

(N or S)

Arrival 

(E or W)

Land

(N or S)

Part-task training runs (controllers and pilots independent in their respective simulator) 

M1 normal operations 

(BASELINE) E N (35R) normal ops, ILS W N (35L) 

RNP & configure W N (35L) 

M2 normal operations 

(BASELINE) W S (16L) IM, ILS E S (17R) 

IM, RNP E S (17R) 

Connected training runs (controllers and pilots together in the same interactive run) 

M3 IM CROSS Captain PM E N (35R) 

M4 any IM First Officer PM W S (16L) 

M5 IM CROSS Captain PM W N (35L) 

M6 any IM First Officer PM E S (17R) 

 
 35 

3.8.5 Data Collection Scenarios 

Ten scenarios were developed for the three IM operation types (Table 9). For each scenario, the 

location of the subject aircraft within the arrival sequence of aircraft conducting IM operations 

was intentionally varied, and their route geometry with the Target (i.e., in-trail, merge in the 

ARTCC, or merge in the TRACON) was 1) intentionally set in order to ensure a variety of IM 

operation types appropriate for that clearance, and 2) to make each scenario appear very different 

to controllers and pilots. Different arrival directions and landing runways were used to minimize 

the learning effect for the controllers and pilots. Each pilot of the two-person crew flew equal 

number of scenario runs as the PM and PF. 

Table 9. Description of Data Collection Scenarios 

Scenario Operation Geometry Arrival Land PM

A BASELINE n/a W N First Officer 

B BASELINE n/a E S Captain 

C CAPTURE In-trail W S First Officer 

D CAPTURE In-trail E N Captain 

E CROSS In-trail & merge E S First Officer 

F CROSS In-trail & merge W N Captain 

G MAINTAIN In-trail E N First Officer 

H MAINTAIN In-trail W S Captain 

I MIXED In-trail & merge E N First Officer 

J MIXED In-trail & merge W S Captain 

The call signs for the six aircraft conducting IM operations were kept the same for each crew for 

all data collection scenarios to simplify data analysis and to make it easier to graphically display 

the status of the aircraft. The call signs of the remaining aircraft were intentionally varied in each 

scenario to minimize any learning effect for controllers. 

To provide data to the FAA on call sign confusion and the use of a call sign as a third party in a 

controller issued instruction, some of the remaining call signs were also intentionally selected to 

be similar to each other, for example actual callsigns used during the experiment included 

UAL461, UAL561 and UAL651 (United Airlines), and ASH3708 and ASH5708 (Air Shuttle). To 

provide data to the FAA about controllers issuing instructions using the aircraft data tag, a range 

of call signs were used that were both common and uncommon, for example, LOF for Water Ski 

(Trans States Airlines) and EJM for Jet Speed (Executive Jet Management). A complete list of 

airlines used in this experiment is in Appendix D. 

 
 36 

3.8.6 Example of Aircraft Location at Scenario Start 

Each scenario contained aircraft arriving from all four directions, and landing on one of two 

runways. To simplify data analysis and accommodate lab space limitations, only aircraft in half of 

the airspace in each scenario were controlled by air traffic controllers, while the aircraft in the 

other half of the airspace were flown by completely automated simulated aircraft. This required 

the aircraft in each half of the airspace to land on the runway closest to their direction of arrival, 

which was facilitated by turning off the TBFM runway load-balancing function. There were no 

departures from KDEN in the scenario; however, there were overflight aircraft to create more 

realistic visual presentations on the controller workstations and cockpit traffic displays. 

The initial location of aircraft during data collection varied from scenario to scenario, and an 

example is shown in Figure 10. The blue icons were the six aircraft flown by subject pilots, and 

the green aircraft on the same side of the airspace (the east) were flown by MACS confederate 

pilots. All of the aircraft arriving from the east land on the eastern runway. The green aircraft on 

the opposite side of the airspace (the west) were automated simulated aircraft landing on the 

parallel runway to the west, and they did not interact with any of the aircraft arriving from the east. 

The orange aircraft were overflights not landing at Denver (flown by MACS pilots) and were 

added to create a more realistic environment for the controllers and more realistic cockpit traffic 

displays for pilots. There were no aircraft departing Denver in any of the scenarios. 

Figure 10. Example of aircraft location at scenario start. 
 37 

3.9 Controller and Pilot Participants 

3.9.1 Qualification and Currency Requirements 

Each week of data collection employed two confederate controllers, four subject controllers, six 

confederate pilots, and twelve subject pilots, for a total of eight subject controllers and 24 subject 

pilots across the entire experiment. The following sections describe the qualification and currency 

requirements for each of the four categories. 

3.9.1.1 Confederate Controllers 

The “Ghost” and “Tower” ATC positions were manned by confederate controllers, that is, retired 

professional controllers who assisted the research team in creating high-density arrival operations, 

and from whom no data was collected. Both positions were manned by the same two individuals 

throughout the experiment, ensuring a high quality and consistent performance. 

3.9.1.2 Confederate Pilots 

The six confederate pilots operated the Multi-Aircraft Control System (MACS) pilot stations, and 

the rank ordered qualifications for them were: 

• Required: Private Pilot Certificate with an Instrument Rating 

• Required: actual flight time flown within the past three years 

• Desired: commercial aircraft flight experience 

• Desired: previous experience using the MACS pseudo-pilot stations 

The confederate pilots met all requirements, were a mix of active and retired pilots, and most had 

flown commercial aircraft. One requirement not specified was experience with the operation of an 

FMS, in particular the various modes of VNAV. A significant portion of the operational issues by 

the MACS aircraft were caused by the few pilots that did not have FMS and VNAV flight mode 

experience. 

3.9.1.3 Subject Controllers 

The four controller subjects within each of the two groups operated two ARTCC and two 

TRACON stations. Controllers worked the same position throughout their time as a participant. 

The rank ordered qualifications for ARTCC controllers were: 

• Required: previous certification in a sector that fed arrivals to one of the major US airports 

• Desired: retired less than three years or are technically proficient training instructors 

• Desired: experience with RNAV Standard Terminal Arrival Routes 

• Desired: experience with Traffic Management Advisor or time-based metering 

• Desired: previous experience with arrival operations to KDEN 

The rank ordered qualifications for TRACON controllers were: 

• Required: previous job experience at a TRACON that served one of the major US airports 

• Required: previous certification on Feeder or Final positions 

• Desired: retired less than three years or are technically proficient training instructors 
 38 

• Desired: experience with RNAV Standard Terminal Arrival Routes 

• Desired: experience with Terminal metering 

• Desired: experience with RNP approach procedures 

The eight subject controllers ranged from 54 to 60 years in age, with a median of 25 years of 

experience. While six of the eight controllers had been retired for more than three years, four of 

them were active instructors which significantly helped their ability to learn and use the ATD-1 

tools and procedures. Their meeting of the desired qualifications varied significantly. Four of the 

controllers had experience with RNAV arrivals, five had experience with RNP approaches, and 

none had experience with operations into KDEN. See Appendix F.1 for a more detailed summary 

of subject controller backgrounds. 

One of the ARTCC controllers had been retired for greater than five years and had not had 

experience with time-based metering into busy TRACON sectors. His performance and ratings of 

the ATD-1 concept and procedures were significantly lower than the other controllers. 

3.9.1.4 Subject Pilots 

The twelve subject pilots within each of the two groups were paired into two-person crews and 

flew one of the three flight simulators (see Sections 4.1.3, 4.2 and 4.3). Crews remained paired 

throughout training and data collection, and they flew the same simulator in the same position 

throughout the experiment. 

The rank ordered qualifications for subject pilots were: 

• Required: currently or within the prior five years have flown the B-777, B-747-400, B767-400, B-757, or B-737NG (to align with simulator type) 

• Desired (all): experience flying RNP operations 

All the pilots met the required qualification, with the exception of one senior Captain who still 

served as an instructor at a major airline. They ranged from 39 to 68 years in age, with an average 

of 34 years and 17,000 hours of military and commercial flight experience. Every pilot had 

experience with both RNAV arrivals and RNP approaches, with the frequency ranging from almost 

daily to only several times per year. Half of the pilots had previous experience with IM operations, 

either at NASA Langley or NASA Ames in simulation experiments. See Appendix F.2 for a more 

detailed summary of subject pilot backgrounds. 

3.9.2 Protocol for Training and Data Collection 

To the maximum extent possible, the pilots were scheduled with other pilots from the same airline 

and in the position they normally occupied. This procedure was intended to minimize adverse 

effects from differing standard operating crew procedures or crew resource management 

principles. The PF and PM duties varied for each run, with each pilot acting as PF on half of the 

scenarios and PM on the other half. 

Pilots completed a computer-based training program prior to arriving at NASA Langley, while the 

controllers did not receive any prior training or information. All controllers and pilots participated 

in one week of training, followed approximately a month later by one week of data collection (each 

week began Tuesday morning at 8 am and ended Friday by 2 pm). 
 39 

The schedule and events for the training week were: 

• Tuesday: welcome orientation, academics (ATD-1 concept of operations, IM procedures, 

local Denver procedures, etc.), part-task training runs, two training runs 

• Wednesday: five training runs, post-run questionnaires, end of day debrief 

• Thursday: five training runs, post-run questionnaires, end of day debrief 

• Friday: two training runs, post-run and post-experiment questionnaires, debrief 

The schedule and events for the data collection week were: 

• Tuesday: refresher training, four training runs 

• Wednesday: four data collection runs, post-run questionnaires, end of day debrief 

• Thursday: four data collection runs, post-run questionnaires, end of day debrief 

• Friday: two data collection runs, post-run and post-experiment questionnaires, debrief 

Since this experiment employed a training protocol different from previous research (bringing in 

the subjects for an entire week of training, then bringing them back later for refresher training and 

data collection), a cursory analysis of the results was done to determine the impact this approach 

had. Post-analysis of controller and pilot subjective acceptability and workload rating from the 

data collection scenarios was assessed as a function of sequence order to provide an indication of 

whether the training was adequate. That is, if the acceptability and workload ratings as the data 

collection runs progressed remained fairly consistent, it can be inferred there was not significant 

learning still occurring during data collection runs. Appendix H contains the results of that 

analysis, and relatively constant acceptability and workload ratings indicates that the training was 

likely sufficient. 
 40 

4 Facilities and Software 

This section describes those simulator facilities and their software capabilities. Multiple simulators 

were used at NASA LaRC to conduct the IMAC human-in-the-loop experiment. 

4.1 Air Traffic Operations Laboratory (ATOL) 

The Air Traffic Operations Laboratory (ATOL) is a NASA LaRC research facility used to evaluate 

contemporary and future ATM concepts and procedures, while maintaining appropriate 

compatibility with real-world national airspace infrastructure and aircraft avionics system 

architectures. It consists of multiple configurable spaces for controllers and pilots. The ATC spaces 

can be established as en route and terminal facilities (with the appropriate hardware and displays). 

The pilot spaces support a range of different aircraft simulator types (Figure 11). 

The MACS controller stations were configured for ARTCC operations in ATC Room A and 

TRACON operations in ATC Room B. The TBFM schedule display was located in ATC Room 

A, as were the Ghost controller and Tower controllers. 

The aircraft simulator types in the ATOL included dual-crew and single-crew Aircraft Simulation 

for Traffic Operations Research (ASTOR) desktop simulators in Pilot Room A, and single-crew 

MACS pseudo-pilot simulators in Pilot Room B. 

Figure 11. Controller and pilot room configuration in ATOL for the IMAC experiment. 
 41 

4.1.1 MACS Controller Stations 

MACS is a NASA Ames designed research tool that provides both air traffic control and pseudopilot functionality for air traffic management simulations (ref. 15). MACS can be configured to 

provide high-fidelity emulations of both en-route controller workstations and terminal-area 

controller workstations, controllable either by regular keyboard and mouse inputs, or by using 

specialized controller keyboard/trackball input devices. Additionally, the workstations are 

configurable to include prototype controller tools. 

Figure 12 shows the setup of MACS controller workstations configured for ARTCC operations. 

Controllers were provided a headset foot-pedal and a hand-held switch that actuated the Voice 

over Internet Protocol (VoIP) communication (the controller had the option of using either device). 

A computer mouse and controller specific keyboard set allow the controller to interact with the 

MACS display. 

In the experiment, MACS provided four ATC stations used by subject controllers: north and south 

ARTCC positions, and Feeder and Final TRACON positions. MACS was also used for two 

confederate controller stations: a Ghost controller used to hand off aircraft to the ARTCC 

controllers at the appropriate time, and a Tower controller to clear aircraft to land and ensure proper 

performance of the MACS pseudo-aircraft simulators at touchdown. 

Figure 12. MACS workstations for controllers. 

 
 42 

Figure 13 shows a MACS emulation of an En Route Automation Modernization (ERAM) 

workstation for an ARTCC controller, and Figure 14 shows a MACS emulation of a Standard 

Terminal Automation Replacement System (STARS) workstation for a TRACON controller. 

Figure 13. MACS display for ARTCC controller workstation.

Figure 14. MACS display for TRACON controller workstation. 
 43 

4.1.2 Time Based Flow Management (TBFM) 

TBFM is an advanced air traffic control decision support tool for sequencing and scheduling 

aircraft developed by NASA Ames (ref. 16). The Traffic Graphic User Interface uses a timeline 

format to represent sequence information including delay values for aircraft with ETAs up to 

ninety minutes from the meter fix (Figure 15). Using foreknowledge of near-future traffic 

conditions, controllers can use this information to mitigate periods of heavy traffic flow by 

comparing expected demand against estimated capacity for the facility. 

Using the outcome from previous human-in-the-loop experiments (ref. 16 and ref. 20), an 

additional spacing buffer of 0.3 nmi was added to the minimum aircraft separation required for 

wake vortex as specified in the air traffic control regulation (ref. 10, paragraphs 4-5-1 and 5-5-4). 

This additional buffer accounts for a range of things that cause error in achieving the TBFM 

calculated spacing between aircraft: the variance of controller technique, variance in flight crew 

technique in decelerating the aircraft, and TBFM uses a single final approach speed for all turboclass aircraft whereas those aircraft have a range of final approach speeds. This turbo-class specific 

speed (135 knots) is used to convert the distance based criteria in ref. 10 into the ASG time shown 

on the controller workstations for the controller to issue to the flight crew as part of the IM 

clearance. 

 

Figure 15. TBFM screen and display. 

 
 44 

4.1.3 Aircraft Simulation for Traffic Operations Research (ASTOR) 

The ASTOR is a computer-based simulator that can be configured as a two-crew or single-crew 

pilot station, with either two or three monitors, and emulates a commercial transport category 

aircraft. The ASTOR simulators were used in order to obtain greater pilot participation in the 

simulation experiment than would have be possible with only the full-scale simulators, and both 

the high-fidelity DTS and IFD full-scale simulators were used to ensure realistic flight crew 

interactions. 

For this experiment, there were six two-crew stations (Figure 16). Two single-crew stations were 

used only for training. 

Figure 16. Three-monitor ASTOR station for a two-person flight crew.

The two-crew ASTOR stations consist of three touch sensitive display screens that contain dual 

Primary Flight Displays, dual Navigation Displays, an Engine Indication and Crew Alerting 

System, and dual EFBs and CGDs (both specifically configured for IMAC). All three simulators 

in this experiment were configured with a left and right EFB, where data entered by one pilot was 

visible on the other EFB once data entry in a given field is completed by pressing the ENTER 

bezel button or soft-key, or by selecting a different data field. 

Thrust control, flaps, speed brakes, and radio tuning are also displayed and manipulated via an 

externally connected mouse or by touching the screen itself. Pilots used a headset and foot pedal 

actuator to transmit and receive radio communications. The single-crew ASTOR stations have the 

same capabilities as the two-crew station, except the right-side display and associated dual-display 

options are removed. 

The ASTOR displays and controls are based on current commercial cockpit avionics. ASTOR 

aircraft performance is generated by a six degree of freedom real-time aerodynamic and engine 

model. ASTOR also includes a research prototype Flight Management System (FMS) with 

trajectory generation providing lateral and vertical path guidance using an ARINC 424 navigation 

database. Autopilot and flight director systems are incorporated, along with an ARINC 429 digital 

data bus. 

 
 45 

4.1.4 MACS Pseudo-Pilot Station 

MACS ‘pseudo-pilot’ functionality enables individual confederate pilots to control a large number 

of aircraft. The ‘pseudo-pilot’ performs frequency changes and check-ins, clearance read back, 

multiple aircraft monitoring, and clearance input so that the experiment subject pilots flying the 

other simulators see and hear a realistic high-density flight environment. Pilot workstations are 

configured using a combination of generic input devices and contemporary glass cockpit displays 

to accept and quickly enter both standard and non-standard clearance commands for multiple 

aircraft. The display setup consists of nine control fields: Aircraft State Panel, Control Display 

Unit (CDU), FMS Route Panel, Aircraft List, FMS VNAV Panel, Map Display, Primary Flight 

Display, MCP, and Pilot Hand-off Panel. 

Six MACS ‘pseudo-pilot’ stations were used during the experiment, each one paired one-to-one 

with one of the six controller stations. 

Figure 17. MACS workstation for a single pseudo-pilot. 

 
 46 

4.2 Development and Test Simulator (DTS) 

The DTS (Figure 18) is a full-scale, high-fidelity, fixed-base simulator representative of a large 

generic commercial transport category aircraft. It features Boeing 757-200 subsystem panels, a 

Boeing 767 center aisle stand with throttle quadrant, Honeywell Pegasus flight management 

computer, Research Mode Control Panel, dual Collins Business radio tuning units, and dual EFBs. 

(Note: the EFBs in both the DTS and IFD were Astronautic devices used to display the NASAcreated IM displays described in Appendix B.) 

The DTS is driven by a high-fidelity B757-200 aerodynamic mathematical model. There are three 

Smiths Industries Boeing 737 Multifunction Control Display Units – two located in the normal 

forward outboard sections of the aisle stand and a third in the aft center section of the aisle stand 

for use by the researcher. The overhead panel was not populated. 

Cockpit displays are incorporated in four 17-inch liquid crystal display screens and include dual 

Primary Flight Displays, dual Navigation Displays, an Engine Indication and Crew Alerting 

System, and dual CGDs to support IM operations. The stand-by altimeter, airspeed indicator, and 

attitude indicator are located forward of the throttle quadrant. Pilots control the simulator by using 

dual back-driven side-stick controllers and dual rudder pedals. 

The simulator’s out-the-window visual system provided a 210 degree horizontal by 45 degree 

vertical field-of-view. The visual scene used for this experiment was the Denver International 

Airport local flight environment in day visual meteorological conditions (VMC). 

Figure 18. The Development and Test Simulator (DTS). 
 47 

4.3 Integration Flight Deck (IFD) 

The IFD (Figure 19) is a full-mission, full-scale, high-fidelity Boeing 737-800 flight deck 

simulator with a full suite of flight deck panels replicating aircraft functionality. The forward panel 

consists of six ARINC D-sized display monitors which provide fully programmable heads-down 

displays. Displays include dual Primary Flight Displays, dual Navigation Displays, an Engine 

Indication and Crew Alerting System, dual EFBs, and dual CGDs. A Boeing 737 Mode Control 

Panel is positioned above the forward panel and directly overhead is the B737 NextGen forward 

overhead panel. 

The Control Aisle Stand hosts a Boeing 737 Dual Auto-throttle System and two tunable navigation 

and communication radios. Guidance and navigation flight management is interfaced through three 

Smiths Industries Aerospace Color Boeing 737 MCDUs, also on the aisle stand. A General Electric 

Flight Management Computer facilitates operations within the cockpit simulator. The IFD has dual 

hydraulic wheel/columns and dual digital rudder pedals. 

The simulator’s out-the-window visual system provided a 200 degree horizontal by 40 degree 

vertical field-of-view. The visual scene used for this experiment was the Denver International 

Airport local flight environment in day VMC. 

Figure 19. The Integration Flight Deck (IFD). 

 
 48 

The IFD can be operated on either a fixed base or a motion base, and for this experiment, the 

motion base was used. The cockpit motion base (Figure 20) is comprised of a 6 degree-of-freedom, 

76” stroke, hydraulically actuated Stewart Platform motion base synergistic motion platform for 

aerospace research (ref. 17). 

Commercial airline pilots and subject matter experts were used to establish the approximate feel 

of light to moderate turbulence, which resulted in 0.7 feet per second of motion. The IFD 

experienced the light to moderate turbulence throughout all scenarios to assess the impact it had 

on flight crews when entering information into the EFB. 

Figure 20. IFD on the cockpit motion base. 

 
 49 

5 IMAC Experiment Results 

5.1 Caveat to Experiment Results 

Two caveats to the research results were caused by intentional design decisions early in the 

experiment build-up phase that had unintended consequences when subsequent software capability 

and functionality was not completed in time for data collection. These two decisions and their 

impact on the IMAC results are described below. 

1) The ATD-1 ConOps was expanded to include three types of IM operations; however, it was 

assumed based on previous research that en route controller’s display should recommend a 

specific IM operation depending on the relative geometry of the IM and Target aircraft. The 

functionality and logic to show the most appropriate IM operation, and just the data relevant 

for that specific clearance, was not complete in time for this experiment; therefore, the IM 

displays for the en route controller were not tailored to the specify IM clearance type, and 

always showed all the IM data. This had the following repercussions: 

• The controller had to mentally track which scenario run was being conducted, then issue 

the appropriate IM clearance with only the relevant IM data. Therefore the ARTCC 

controller workload ratings for IM operations may be higher than if the intended 

functionality and logic had been implemented. 

• The MIXED scenarios, intended to represent more challenging and realistic operations as 

well as a mix of different IM operations, ended up being almost exclusively CROSS 

operations, the IM operation type implied by the complete IM information on their 

displays. As a result, the controller feedback about the usefulness of different IM operation 

types may not be as insightful compared to the intended system, although good comments 

were made by both en route and terminal area controllers. 

2) Requiring the flight crew to enter the Ownship cruise Mach and cruise/Mach descent speed 

into the IM software was deemed a minor additional workload to the crew and a task that could 

be done internally by the IM software. As a consequence, the display of these data fields on 

the crew interface to the IM software (the EFB) was removed. However, when this new 

software capability was not completed prior to data collection, the experiment commenced 

without a mechanism by which the IM spacing software could obtain the Ownship Mach speed, 

meaning an IM commanded speed could not be calculated if the Ownship was still in the Mach 

regime. (Based on the selected Mach and calibrated airspeed selected by the flight crew, the 

transition altitude ranged between 28,000 and 33,000’ mean sea level.) 

• When the flight crew expressed uncertainty or lack of understanding about the IM 

operation, the single greatest contributor to this issue was when a long delay occurred 

between entering the IM clearance into the EFB (the aircraft was typically above the 

transition altitude) and when the IM commanded speed would appear on the EFB (only 

when the aircraft descended below the Mach/airspeed transition altitude). 

• Some of the operationally undesirable behavior of the MAINTAIN operation occurred 

because the spacing error was calculated and set when the flight crew pressed the 

EXECUTE button; however, the operation itself did not commence until the aircraft had 

descended into the calibrated airspeed regime (the IM commanded speed was suppressed 

in the Mach regime). If there was an airspeed difference between the Target and Ownship 

in the Mach regime, a spacing error is created which can trigger an IM commanded speed 

increase or decrease when the software does engage. 
 50 

5.2 Preconditioning of Data 

Prior to analysis, the collected data was preconditioned, that is, aircraft removed from analysis, 

using the following criteria: 

• Autonomous aircraft (arriving from the opposite direction and landing on the parallel 

runway to the subject controlled aircraft) had all data removed. 

• The first two aircraft in each scenario had all data removed. 

• All aircraft landing after the final subject-piloted aircraft landed (scenario terminated one 

minute after last subject-piloted aircraft landed) had all data removed. 

• All aircraft with simulation artifacts were removed (see Appendix G); however, in some 

cases the qualitative data (surveys) were retained and included in the analysis. 

This resulted in 229 valid flights analyzed during the two week IMAC test period, of which the 

subject pilots flew 79 IM operations and 34 non-IM operations, and the confederate pilots flew 

116 non-IM operations. Appendix G contains additional details on the preconditioning of the data. 

5.3 IM Algorithm Performance Results 

5.3.1 IM Spacing Error 

Performance goal #1 states the minimum success criteria for the IM spacing error is for at least 

95% of the errors to be within 10 seconds of the ASG (ref. 19, para 3.4.1). Results are shown in 

Figure 21 and Table 10 below, with all values expressed in seconds since all IM clearances were 

issued using time-based spacing. The PTP in all IM operations was the FAF. The CAPTURE and 

MAINTAIN operations do not have an ABP, and therefore those ABP boxplots are not shown, 

and the table cells are shown as “n/a.” Also, the ABP and PTP were both set as the FAF in the 

CROSS clearances; therefore, the ABP cell is shown as “see PTP cell.” The ABP was different 

than the PTP for four of the clearances issued during the MIXED scenarios; therefore, the values 

in the two cells associated with the MIXED scenario are different. 

Performance goal #1 was not met for any of the IM operations. The green dashed lines in Figure 

21 represent the 10 second success criteria. Figure 21 is shown without the 31 second MAINTAIN 

and 37 second MIXED outliers to allow a smaller y-axis scale to improve visual clarity. 

 

Figure 21. Graphs of IM spacing error at the ABP and PTP. 
 51 

Table 10. IM Spacing Error at ABP and PTP by Scenario Type in Seconds 

Scenario CAPTURE CROSS MAINTAIN MIXED 

 Mean SD Mean SD Mean SD Mean SD 

Achieve-by Point (ABP) n/a see PTP cell n/a -0.3 10.0 

Planned Termination Point (PTP) 0.1 5.4 1.0 6.2 -5.2 9.1 -2.6 11.1 

N 20 24 12 23 

Performance Goal #1 

(95% IM ≤ 10 s) 

NO 

(18, or 90%) 

NO 

(21, or 87.5%) 

NO (1)

(9, or 75.0%) 

NO (2)

(17, or 73.9%) 

Note (1): The MAINTAIN operation had a large standard deviation at the PTP due to a 31 second 

outlier caused by the Target aircraft flying at an extremely slow speed, which in turn caused the 

IM aircraft’s spacing software to be speed limited. That is, the 15% bound around the published 

speed prevented the IM aircraft from flying slow enough to null the spacing error (see Section 

A.3.1 for a detailed description of the speed control logic). Removing the outlier data resulted in a 

mean of -2.6 seconds and standard deviation of 4.8 seconds. 

Note (2): 22 of the 23 IM clearances issued during the MIXED scenarios were for CROSS 

operations. The ABP was different than the PTP in four of the 22 CROSS operations (see Section 

5.7.4.1). A review of the data indicates that one of those four CROSS operations was an outlier 

with a -37 seconds (early) spacing error due to control law behavior described as a case study in 

Section 5.9.5, and is the third example of loss of aircraft separation described as a case study in 

Section 5.9.1. Removing the outlier data resulted in mean of -1.0 second and standard deviation of 

8.4 seconds. The research team postulates the high incidence of CROSS clearances issued during 

the MIXED scenarios were due to 1) most of the Target and IM aircraft were on merging routes, 

and 2) the controller displays showed all IM information (a caveat noted in Section 5.1), leading 

the controller to issue a CROSS clearance (the only clearance type to require all IM information). 

5.3.2 Frequency of IM Speed Changes during IM Operations 

ARTCC and TRACON controllers issue speed instructions as required to ensure a safe and 

efficient arrival flow, and do not attempt to have those instructions coincide with speed changes 

on the published procedure. The consequence of the Target aircraft decelerating at a point other 

than the published waypoint is that ASTAR perceives this as two speed changes (a deceleration 

prior to the waypoint and then back on speed at the waypoint), which in turn causes ASTAR to 

issue multiple IM speeds to the flight crew. This behavior is indicative of a system without accurate 

Target aircraft intent data being shared with the IM equipment. 

Figure 22 shows the IM speed change frequency (rate) observed for each of the scenarios types 

where IM operations were conducted. The MAINTAIN scenarios had the lowest mean speed 

change frequency, likely because the MAINTAIN operation started without any spacing error. 

Both the CAPTURE and Mixed scenarios had the highest speed change frequency (Table 11). The 

frequency of IM speed changes during this experiment is similar to results obtained during 

previous ATD-1 research experiments (ref. 11). 
 52 

Figure 22. Graph of IM of speed change frequency by scenario type. 

Table 11. IM Speed Change Frequency per Minute by Scenario Type 

CAPTURE CROSS MAINTAIN MIXED 

Mean SD Mean SD Mean SD Mean SD 

0.70 0.27 0.62 0.18 0.55 0.22 0.70 0.17 

Of note is the six subject piloted aircraft had an approximate mean frequency of 0.25 speed changes 

per minute during TSAS scenarios. 

Further analysis of the frequency of IM commanded speed changes for all IM operation types was 

conducted by examining the time between consecutive speed changes as a function of the 

remaining distance-to-go to the runway threshold (Figure 23). The histogram on the y-axis shows 

that the time between consecutive speed changes became smaller as the IM aircraft approached to 

the runway, indicating that the speed change frequency increased. The histogram on the x-axis 

shows that the number of speed changes increased as the IM aircraft approached the runway. 
 53 

Figure 23. Scatterplot of time between all IM speed changes by distance-to-go. 

The number of IM speed changes by IM operation type as a function of the remaining distance-togo was also plotted (Figure 24). The CROSS algorithm commanded a greater percentage of speed 

changes when the aircraft was close to the ABP, whereas the CAPTURE and MAINTAIN 

algorithm speed changes were more evenly distributed along the arrival. 

Figure 24. Histogram of IM speed changes by IM operation type by distance-to-go. 

In summary, two conclusions can be drawn from the data. First, Figure 24 shows that the highest 

localized frequency of IM speed changes occurring during CROSS operations when the aircraft 

were on final (within 20 nmi of the runway). During these CROSS operations, the TBO speed 

control law was being used (CAPTURE and MAINTAIN use the CTD law), indicating that further 

research and refinement is needed to reduce the number of changes generated by the TBO speed 

control law, particularly when the aircraft is close to the ABP. 

Secondly, the CAPTURE operations had more speed changes than the CROSS or MAINTAIN 

operations (Table 11), and had a slightly higher average number of IM speed reversals than CROSS 

operations, and much greater than MAINTAIN operations (Figure 25). Since the CAPTURE and 

MAINTAIN operations use the same CTD speed control law (the only difference is that the 

0 20 40 60 80

400

300

200

100

0

Distance-To-Go (nmi)

Time Betwwen Speed Changes (sec)

0 15 30 45 60 75 90

48

36

24

12

0

0 15 30 45 60 75 90

48

36

24

12

0

CAPTURE

Distance-To-Go (nmi)

Frequency

CROSS

MAINTAIN Mixed
 54 

MAINTAIN operation starts with zero spacing error), it can be hypothesized that the larger number 

of the speed reversals in CAPTURE operations occur during the ‘capture’ phase of flight. 

5.3.3 Number of IM Speed Reversals per IM Operation 

The number and magnitude of IM commanded speed reversals (that is, a speed increase during an 

arrival operation where typically only speed decreases are expected) were analyzed and are shown 

in Figure 25. The data indicate there was an average of one to two speed reversals per IM operation, 

with the MAINTAIN operations having the smallest number of speed reversals and the CAPTURE 

operation having the largest number of speed reversals. Since the MAINTAIN and CAPTURE 

operations used the same speed control law, this suggests that many of the additional speed 

reversals observed during the CAPTURE operation occurred when the IM aircraft was capturing 

the assigned spacing goal. 

One particular behavior that appears to have contributed to the speed reversals was air traffic 

controllers decreasing the Target aircraft’s altitude in order to absorb the delay required for that 

Target aircraft. In these cases, the altitude difference between the Target and IM aircraft causes a 

difference in their ground speeds, in turn causing ASTAR to issue a slower IM commanded speed 

(the same end effect described in Section 5.3.2 although the root cause is different). When the 

Target aircraft subsequently rejoins the published altitude profile, the Target’s ground speed 

returns to the expected speed profile, causing ASTAR to increase the IM commanded speed; i.e., 

a speed reversal. The case study in Section 5.9.5 describes an example of this behavior. 

Another contributing factor unique to the speed reversals in CROSS operations is the Target 

aircraft slowing at a location different than what ASTAR expects (at waypoints specified on the 

published procedure). This causes the spacing algorithm to calculate a ground speed deviation, in 

turn causing a decrease to the IM commanded speed. When the Target subsequently rejoins the 

published speed profile, the ground speed deviation returns to zero which causes ASTAR to 

increase the IM commanded speed; i.e., a speed reversal. A more thorough description of the cause 

for ASTAR speed reversals are discussed in ref. 24 and 25. 

Figure 25. Number of IM speed reversals per flight by scenario type. 

Of note is that during the BASELINE (TSAS only) scenarios, there were four instances where the 

controllers issued a speed increase to one of the six subject piloted aircraft during the arrival 

operation. 
 55 

5.4 Air Traffic System Performance Results 

5.4.1 Inter-Arrival Spacing Error at the Final Approach Fix 

Performance goal #2a states the inter-arrival spacing error of 68% of IM operations following 

either a TSAS or an IM operation will be ≤ 8 seconds, and performance goal #2b states the interarrival spacing error of 68% of TSAS operations following a TSAS operation or an IM operation 

will be ≤ 12 seconds. 

Figure 26 and Table 12 list the results for both non-IM and IM operations by scenario type, as well 

as if the performance goal was met or not. The scenarios with IM operations were sub-divided into 

two categories: “non-IM” and “IM”. The non-IM category was any aircraft flown by either a 

subject pilot or pseudo-pilot that was not conducting an IM operation at the PTP (the FAF), and 

the IM category was any aircraft flown by subject pilots that was conducting an IM operation at 

the PTP. 

Figure 26. Graph of inter-arrival spacing error at Final Approach Fix. 

Based on the data in Table 12, the performance goal #2a was met for the CAPTURE and CROSS 

scenarios. The performance goal #2b was also met for the CAPTURE scenario. The results indicate 

that aircraft conducting a CAPTURE operation had approximately the same mean inter-arrival 

spacing error as aircraft conducting non-IM operations; however, the standard deviation of the 

aircraft conducing CAPTURE operations (6.1 seconds) was smaller than the standard deviation of 

non-IM operations (11.3 seconds). Similarly, aircraft conducting a CROSS operation had a smaller 

standard deviation (6.2 seconds) than non-IM operations (14.4 seconds). These results suggest that 

IM helps aircraft achieve more precise spacing at the FAF. 
 56 

Table 12. Inter-Arrival Spacing Error Data at Final Approach Fix 

Scenario BASELINE CAPTURE CROSS MAINTAIN MIXED 

Type Operation Non-IM Non-IM IM Non-IM IM Non-IM IM Non-IM IM 

Mean (sec) 4.1 -1.2 -1.5 2.4 -0.1 -1.2 -7.3 3.9 -3.3 

SD (sec) 19.2 11.3 6.1 14.4 6.2 17.5 13.8 14.3 11.2 

N (1) 33 23 20 20 24 29 12 22 23 

% of ops ≤ 8 s 80% 83% 42% 65% 

Performance Goal 

#2a met? 

(68% IM ≤ 8 s) 

 YES YES NO (2) NO (3)

% of ops ≤ 12 s 61% 83% 55% 59% 77% 

Performance Goal 

#2b met? 

(68% TSAS ≤ 12 s) 

NO YES NO NO YES 

Note (1): The N value of 206 is less than the Appendix G value of 229 operations because the first 

aircraft of each of the 20 scenarios did not have an aircraft in front of it to calculate an inter-arrival 

error, and the inter-arrival error for four other aircraft could not be calculated because their 

preceding aircraft were excluded from data analysis. 

Note (2): the MAINTAIN clearance issued by the ARTCC controller is based on their visual 

perception of a desirable spacing at that time and is not expected to align with the TBFM schedule. 

Note (3): 22 of the 23 IM arrivals during the MIXED scenarios were CROSS operations, and four 

of those 22 CROSS operations were with the ABP not the same as the PTP (see Section 5.7.4.1). 

One of those four CROSS operations was a 37 seconds early outlier due to control law behavior 

(Section 5.9.5), and is also described in a loss of separation case study (Section 5.9.1.3). Without 

that outlier, the MIXED mean is -1.0 second, and the standard deviation is 8.4 seconds. 

The MAINTAIN operations were controlling to a time-based spacing goal calculated by the IM 

spacing software, and this spacing interval was determined by the ARTCC controller while the 

aircraft were at cruise altitude. Therefore, there is no expectation that 30 minutes later the 

MAINTAIN operation would achieve the time value calculated by the TBFM software for spacing 

at the FAF (note: the TRACON controllers were instructed and given training to terminate the IM 

operation if the ARTCC assigned spacing was not appropriate for the arrival operation within the 

TRACON). The standard deviation of inter-arrival spacing error in the MAINTAIN scenarios was 

also smaller for the IM operations than the non-IM operations. 

The MIXED operations are comprised of all three IM clearance types issued at the discretion of 

the controllers. Although the inter-arrival spacing error mean and standard deviation in the MIXED 

scenarios indicate a slight improvement of IM compared to non-IM operations, the difference was 

smaller than the differences observed for the CAPTURE and CROSS operations. Since the 

MIXED scenarios were predominantly composed of CROSS operations, it is hypothesized that the 

degradation in performance was due to the fact that the MIXED scenarios were always conducted 

last and an outlier where the inter-arrival spacing error was -37 seconds. 
 57 

5.4.2 PBN Success Rate 

Hypothesis #1 states the percentage of uninterrupted PBN operations with ATD-1 tools will be 

greater than 70%. A successful PBN operation was defined by ATD-1 as an aircraft that was not 

vectored within the TRACON airspace. 

To determine the results of this metric, the latitudes and longitudes of all the aircraft in the 

simulations were plotted and then visually inspected to determine the number of aircraft that were 

vectored in the TRACON. Figure 27 shows the latitude and longitude of the paths flown by all of 

the aircraft in this experiment, grouped by the landing runway, with the small circles depicting the 

locations of the TRACON meter fix waypoints. Any off path vector between the circle and landing 

runway indicates a failure of the PBN operation. 

A visual inspection of the plots in Figure 27 did not reveal any cases where aircraft were vectored 

off path while in the TRACON. Furthermore, there were no post-run comments by controllers or 

pilots that indicated vectors had been issued inside the TRACON. Thus, the success rate of PBN 

operations in the TRACON during the experiment was 100%, and hypothesis #1 was true. 

While the lack of vectors within the TRACON is a positive result, a definitive conclusion about 

the efficacy of ATD-1 operations cannot be established since a range of different traffic density 

scenarios and different wind conditions could not be performed as part of this experiment. 

Figure 27. The latitude and longitude of all aircraft for all four landing runways. 

Latitude (deg)

Longitude (deg)

-105 -104.5 -104 -103.5 -103 -102.5 -102 -101.5 -101 -100.5 -100

Latitude (deg) 36

37

38

39

40

41

42 DEN35R
 58 

5.4.3 Slot Marker Deviation in the TRACON 

The CMS tools provide terminal controllers with a graphical depiction of the location an aircraft 

should be when it is on schedule, referred to as a slot marker. The RMS of the slot marker deviation 

is a measurement of the proximity of an aircraft from its slot marker. The goal of this analysis is 

to determine how well the IM aircraft conformed to the slot marker compared to non-IM aircraft. 

The results shown in Figure 28 and Table 13 indicate that the IM aircraft in the CAPTURE scenario 

had less slot marker deviation (Mean = 0.9, standard deviation = 0.6) than the non-IM aircraft 

(Mean = 1.1, standard deviation = 0.4). Conversely, the IM aircraft in the CROSS scenario had 

greater slot marker deviation (Mean = 1.3, standard deviation = 0.6) than the non-IM aircraft 

(Mean = 1.0, standard deviation = 0.7). Additionally, the IM aircraft in the Mixed scenario, which 

primarily consisted of CROSS operations, had greater slot marker deviation (Mean = 1.3, standard 

deviation = 0.6) than the non-IM aircraft (Mean = 0.9, standard deviation = 0.6). This data suggest 

that the CAPTURE algorithm conforms slightly better to the CMS tools than non-IM operations 

(BASELINE) as tested in this experiment, although the values for the CROSS and MAINTAIN 

operations also appear to be close to the BASELINE values. Further work is needed to determine 

if this conclusion can be generalized to other wind and traffic scenarios. 

Figure 28. The RMS of the slot marker deviation in the TRACON. 

Table 13. RMS Data for Slot Marker Deviation in TRACON 

Scenario BASELINE CAPTURE CROSS MAINTAIN MIXED 

Operation Non-IM Non-IM IM Non-IM IM Non-IM IM Non-IM IM 

Mean (nmi) 1.1 1.1 0.9 1.0 1.3 1.2 1.2 0.9 1.3 

SD (nmi) 0.5 0.4 0.6 0.7 0.6 0.5 0.4 0.6 0.6 

N (1) 37 28 20 24 24 33 13 26 23 

Note (1): the total N value of 228 is one operation less than described in Appendix G due to one 

aircraft not having a TRACON Meter Fix assigned. 
 59 

5.5 Controller Objective Performance Results 

5.5.1 Percentage of Controller-Interrupted IM Operations 

Hypothesis #2 states the percentage of IM operations terminated by ATC (i.e., a vector or speed 

instruction was issued to the flight crew conducting that operation) prior to the Planned 

Termination Point will be < 30%. 

Of the 79 IM operations conducted, 14 (18%) of the operations were interrupted by ARTCC or 

TRACON controllers by issuing speed instructions. Statistical analysis using the one-sample 

proportion test found that significantly less than 30% of IM operations were interrupted by ATC 

(p = 0.001), therefore the criteria for this hypothesis was met. The 14 interrupted events occurred 

during CAPTURE (3), CROSS (1), and MAINTAIN (10) operations, therefore the percentage of 

controller-interrupted IM operations designed for arrival operations (CROSS and CAPTURE) was 

4% (3 of 79). Post-analysis of the controller post-run and post-experiment surveys indicate the 

reasons for the cancellations included controller unease, controller confusion, controller desire to 

expedite spacing, miscommunication, algorithm performance (too aggressive in capture phase), 

and the spacing goal did not match the schedule (MAINTAIN operations only). 

There were five instances where the controller issued a speed command to supersede the IM 

commanded speed, but did not verbally state “Suspend IM” as specified by the ATD-1 procedures 

and practiced during training. This led to confusion in the cockpit whether IM spacing was still in 

effect or not, as receiving the speed instruction alone was contrary to training. Both controllers and 

pilots had received training for this possibility, and stated they understood that training during the 

post-experiment debrief session. 

In summary, this analysis indicates that generally the controllers felt the CROSS and CAPTURE 

IM operations (designed for arrival operations) behaved appropriately and interacted well with 

non-IM operations. However, the delay from when the controller issued the IM clearance until the 

IM operation commenced presented challenges, and to a lesser degree the phraseology did as well. 

5.5.2 Schedule Deviation 

The schedule deviation results discussed in this section differ from the inter-arrival spacing error 

results in Section 5.4.1 in that this section compares the aircraft’s actual time of arrival with the 

scheduled time of arrival. These results are typically not as operationally significant as the interarrival spacing error discussed earlier, where the Final controller is focused on optimum spacing 

between aircraft and not attempting to meet a specific time for each aircraft. Therefore there is no 

hypothesis associated with this metric. 

5.5.2.1 Schedule deviation at the Final Approach Fix 

The schedule deviation is the difference in time between an aircraft’s scheduled time of arrival and 

its actual time of arrival at the FAF. Results for the 229 flight operations in IMAC are shown in 

Figure 29 and Table 14, grouped by scenario type and separated by non-IM and IM operations. 

• Note: whether an aircraft is conducting an IM operation or not is determined by whether 

the IM software is in the PAIRED state and the flight crew are flying the IM speed. 

Therefore, the number of IM operations may be different at different waypoints. For 
 60 

example, the different number of IM operations in the MAINTAIN scenario in Table 14 

and Table 15 indicates six (6) aircraft had their IM operation suspended or canceled 

between entry into the TRACON at the Meter Fix and the FAF. 

The results indicate that the schedule deviation was similar for BASELINE, CAPTURE, and 

CROSS operations, as reflected in similar means and standard deviations. For the MAINTAIN 

operation, the mean schedule deviation at the FAF for IM operations was less than for non-IM 

operations, however the standard deviation of the IM operations was substantially greater. This 

was in part due to the spacing goal being visually assigned by the ARTCC controller and not being 

associated with the scheduled time of arrival estimated by the ground software. 

Figure 29. The schedule deviation at Final Approach Fix. 

Table 14. Schedule Deviation Data at Final Approach Fix 

Scenario BASELINE CAPTURE CROSS MAINTAIN MIXED 

Operation Non-IM Non-IM IM Non-IM IM Non-IM IM Non-IM IM 

Mean (sec) -7.9 -7.7 -4.7 -8.9 -8.9 -14.2 0.4 -1.3 -4.6 

SD (sec) 17.3 10.4 10.0 10.1 9.7 17.2 29.4 8.5 13.6 

N (1) 39 28 20 24 24 33 12 26 23 

Note (1): there were 90 IM operations conducted within in the TRACON during this experiment, 

therefore the N value of 79 for all IM operations at the FAF indicates 11 IM operations were 

canceled in the TRACON prior to the FAF (all by controllers). 

 
 61 

5.5.2.2 Schedule Deviation at the Meter Fix 

The goal of an IM operation is to achieve or maintain a single assigned spacing goal. However, 

the TBFM and CMS ground tools provide air traffic controllers with guidance to achieve the 

schedule at several different meter points along an aircraft’s route. The schedule deviation at the 

meter fix (the entry into the TRACON) was examined to determine if the schedule deviation of 

IM aircraft was worse than the schedule deviation of non-IM aircraft. 

The results indicate that there were differences between the schedule deviation of IM aircraft and 

the schedule deviation of non-IM aircraft at the meter fix; however, the differences are relatively 

small and are not necessarily problematic (Figure 30 and Table 15). 

Figure 30. The schedule deviation at the TRACON Meter Fix. 

Table 15. Schedule Deviation Data at TRACON Meter Fix 

Scenario BASELINE CAPTURE CROSS MAINTAIN MIXED 

Operation Non-IM Non-IM IM Non-IM IM Non-IM IM Non-IM IM 

Mean (sec) 9.3 -8.8 -2.9 -1.7 -0.2 -7.5 4.8 1.0 -4.4 

SD (sec) 21.9 14.5 15.5 24.6 NA 17.6 17.1 20 27.3 

N (1) (2) 37 25 23 47 1 26 20 42 7 

Note (1): the N value of 228 is one less operation than shown in Table 14 due to the aircraft not 

having a TRACON Meter Fix assigned, which occurred when they initialized inside the TRACON. 

Note (2): there were a total of 90 IM operations in the TRACON, therefore the N value of 51 

operations in Table 15 indicates that 39 of those IM operations were initiated just inside the 

TRACON boundary, due to all the criteria for initiation of those IM operations not being satisfied 

until within the TRACON. 
 62 

5.6 Controller Subjective Assessment Responses 

5.6.1 Controller Acceptability of ATD-1 Operations 

Performance goal #3 states the mean controller acceptability rating of ATD-1 operations (from the 

CARS on the controller post-run survey), will be ≥ ‘8’, where a rating of ‘8’ on the CARS indicates 

that the system has mildly unpleasant deficiencies, is acceptable, and minimal compensation is 

needed to meet desired performance. 

A boxplot of the subjective assessment data is shown in Figure 31. There were only two CARS 

ratings less than ‘8’ (indicated by the dashed green line), indicating that, in general, controllers 

found the ATD-1 operations to be acceptable. 

The first of the two ratings below ‘8’ was a Feeder controller who rated the acceptability of 

operations during a MAINTAIN scenario as ‘7’ due to excessive IM aircraft spacing. (Note: even 

though it was part of their training, it is unclear from the survey whether or not the Feeder 

controller understood that a MAINTAIN clearance issued by an ARTCC controller is not intended 

to meet the TBFM schedule.) The second rating below ‘8’ was a ‘7’ given by a Final controller 

during a MIXED scenario, who reported that several IM aircraft did not adequately slow down on 

final. Additional analysis is in Appendix F.3, questions #10 - #16. 

MIXED

MAINTAIN

CROSS

CAPTURE

BASELINE

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

10

9

8

7

6

5

4

3

2

1

CARS Ratings

Figure 31. Controller acceptability of ATD-1 operations. 

The controller post-experiment ratings of acceptability of IM operations aligns with the post-run 

ratings. On a scale of 1 – 7 with ‘4’ being “Acceptable” and ‘7’ being “Completely Acceptable”, 

the mean rating from the eight subject controllers was 6.6, 6.4, and 5.9 for CAPTURE, CROSS, 

and MAINTAIN, respectively. Complete data is in Appendix F.5, question #7. 
 63 

5.6.2 Controller Assessment of Workload of ATD-1 Operations 

Performance goal #4 states the mean controller rating (from the controller post-run survey using a 

NASA TLX with a 7-point Likert scale as defined in reference 19) for Mental Demand, Physical 

Demand, Time Pressure, and Effort will be ≤ ‘5’, and for Frustration and Success will be ≤ ‘3’. A 

7-point scale ranging from ‘1’ = “Very Low” to ‘7’ = “Very High” was used for the Mental 

Demand, Physical Demand, Time Pressure, Effort, and Frustration subscales, and ‘1’ = “Good” to 

‘7’ = “Poor” was used for the Success subscale. 

Descriptive statistics for controller workload ratings are shown in Table 16. There were no TLX 

ratings above ‘5’ for Mental Demand, Physical Demand, Time Pressure, and Effort during all 

operations; two ratings above ‘3’ for Frustration; and one rating above ‘3’ for Success. These 

results indicate that controllers generally found the workload of ATD-1 operations acceptable. 

There was one TLX rating of ‘5’ for Success during a BASELINE scenario by one of the Final 

controllers who reported that an aircraft on final slowed too much, the controller tried to speed it 

up but was too late, and the following aircraft had to be sent around. That same Final controller 

also rated his Frustration as ‘4’ during a MAINTAIN scenario indicating that he felt the IM aircraft 

didn’t always slow as he expected. One of the Center controllers also provided a TLX rating of ‘4’ 

for Frustration during a CAPTURE scenario commenting that he was still adapting to using time 

over mileage for separation and still learning the sequence for issuing various clearances. 

Accompanying plots and additional analysis are in Appendix F.3, questions #4 - #9. 

Table 16. Descriptive Statistics for Controller Workload Ratings (1 of 3) 

Operation TLX Position N Mean SD Min Median Max 

BASELINE 

Mental 

Demand 

Center 8 1.9 1.4 1 1 4 

Feeder 4 2.3 0.5 2 2 3 

Final 4 2.3 1.9 1 1.5 5 

Physical 

Demand 

Center 8 1.5 0.9 1 1 3 

Feeder 4 1.5 0.6 1 1.5 2 

Final 4 1.0 0.0 1 1 1 

Time 

Pressure 

Center 8 1.6 1.2 1 1 4 

Feeder 4 2.0 0.8 1 2 3 

Final 4 1.8 1.0 1 1.5 3 

Effort 

Center 8 1.9 1.4 1 1 4 

Feeder 4 3.0 1.2 2 3 4 

Final 4 2.3 1.9 1 1.5 5 

Frustration 

Center 8 1.1 0.4 1 1 2 

Feeder 4 2.0 0.0 2 2 2 

Final 4 1.5 1.0 1 1 3 

Success 

Center 8 1.0 0.0 1 1 1 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 2.3 1.9 1 1.5 5 

 
 64 

Table 22. Descriptive Statistics for Controller Workload Ratings (2 of 3) 

Operation TLX Position N Mean SD Min Median Max 

CAPTURE 

Mental 

Demand 

Center 8 2.6 1.6 1 2 5 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 2.0 0.8 1 2 3 

Physical 

Demand 

Center 8 2.0 1.1 1 2 4 

Feeder 4 1.5 0.6 1 1.5 2 

Final 4 1.0 0.0 1 1 1 

Time 

Pressure 

Center 8 2.3 1.3 1 2 4 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 1.3 0.5 1 1 2 

Effort 

Center 8 2.5 1.4 1 2 5 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 1.5 0.6 1 1.5 2 

Frustration 

Center 8 1.6 1.1 1 1 4 

Feeder 4 1.3 0.5 1 1 2 

Final 4 1.0 0.0 1 1 1 

Success 

Center 8 1.0 0.0 1 1 1 

Feeder 4 1.3 0.5 1 1 2 

Final 4 1.0 0.0 1 1 1 

CROSS 

Mental 

Demand 

Center 8 2.1 1.4 1 1.5 4 

Feeder 4 2.0 0.8 1 2 3 

Final 4 2.0 0.8 1 2 3 

Physical 

Demand 

Center 8 2.0 1.1 1 2 4 

Feeder 4 1.5 0.6 1 1.5 2 

Final 4 1.0 0.0 1 1 1 

Time 

Pressure 

Center 8 1.9 0.8 1 2 3 

Feeder 4 1.8 0.5 1 2 2 

Final 4 1.5 0.6 1 1.5 2 

Effort 

Center 8 2.1 1.1 1 2 4 

Feeder 4 2.3 1.0 1 2.5 3 

Final 4 2.0 0.8 1 2 3 

Frustration 

Center 8 1.1 0.4 1 1 2 

Feeder 4 1.0 0.0 1 1 1 

Final 4 1.0 0.0 1 1 1 

Success 

Center 8 1.0 0.0 1 1 1 

Feeder 4 1.3 0.5 1 1 2 

Final 4 1.0 0.0 1 1 1 

 
 65 

Table 22. Descriptive Statistics for Controller Workload Ratings (3 of 3) 

Operation TLX Position N Mean SD Min Median Max 

MAINTAIN 

Mental 

Demand 

Center 8 1.9 1.1 1 1.5 4 

Feeder 4 1.8 0.5 1 2 2 

Final 4 2.5 0.6 2 2.5 3 

Physical 

Demand 

Center 8 1.6 0.9 1 1 3 

Feeder 4 1.5 0.6 1 1.5 2 

Final 4 1.3 0.5 1 1 2 

Time 

Pressure 

Center 8 1.8 1.2 1 1 4 

Feeder 4 1.5 0.6 1 1.5 2 

Final 4 1.3 0.5 1 1 2 

Effort 

Center 8 1.9 1.1 1 1.5 4 

Feeder 4 2.8 0.5 2 3 3 

Final 4 2.0 0.8 1 2 3 

Frustration 

Center 8 1.1 0.4 1 1 2 

Feeder 4 1.0 0.0 1 1 1 

Final 4 1.8 1.5 1 1 4 

Success 

Center 8 1.0 0.0 1 1 1 

Feeder 4 1.5 0.6 1 1.5 2 

Final 4 1.5 1.0 1 1 3 

MIXED 

Mental 

Demand 

Center 8 2.1 1.5 1 1.5 5 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 2.0 0.8 1 2 3 

Physical 

Demand 

Center 8 1.8 1.2 1 1 4 

Feeder 4 1.3 0.5 1 1 2 

Final 4 1.0 0.0 1 1 1 

Time 

Pressure 

Center 8 2.0 1.3 1 1.5 4 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 1.0 0.0 1 1 1 

Effort 

Center 8 2.0 1.3 1 1.5 4 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 1.8 1.0 1 1.5 3 

Frustration 

Center 8 1.0 0.0 1 1 1 

Feeder 4 1.0 0.0 1 1 1 

Final 4 1.0 0.0 1 1 1 

Success 

Center 8 1.0 0.0 1 1 1 

Feeder 4 1.3 0.5 1 1 2 

Final 4 1.0 0.0 1 1 1 

 
 66 

5.6.3 Controller Comments about ATD-1 ConOps and IM Operations 

Controller post-experiment comments about IM operations varied greatly based on what position 

the controller occupied (ARTCC, Feeder, Final), and sometimes the comments were counter to 

comments made by other controllers. A complete listing of their comments is in Appendix F.5, 

and a synthesis of those comments by question number is below: 

• #8: Rate the impact of the addition of IM operations on expediting traffic flow. 

- “Adds a little to your workload, but spacing is more efficient.” 

- “Allowed for more attention complying with metering duties.” 

• #9: When IM aircraft were in your sector, how acceptable was it for the controller to be 

responsible for maintaining standard separation between aircraft. 

- “Increased vigilance required to scan aircraft from separate routes increased workload.” 

- “The MAINTAIN operations were the easiest and required the least work, but 

frequently resulted in less than optimum spacing and had to be canceled in the 

TRACON.” 

• #12: Describe any changes you would make to the IM operation. 

- “Using the arrival sequence numbers as a reference made it easy to determine who the 

IM aircraft was paired with.” 

• #13 & #14: What additional information would your like for IM aircraft and Target aircraft? 

- Most controllers found the displays acceptable as currently configured. 

- “The meter list and ATD-1 specific information in the aircraft data block was helpful.” 

- “Slot markers, spacing cone and speed advisories were useful.” 

• #15 & #18: Rate the operational acceptability of the IM phraseology, and suggest any 

improvements for clarity or completeness. 

- Overall the controllers rated the IM phraseology as acceptable and clear. 

- “Report paired” should be part of ATC’s acknowledgement of the flight crew’s read 

back of the IM clearance, not the part of the initial issuing of the IM instruction. 

o For example, Delta 1415 read back correct, report paired

• #16 & #17: Describe any confusion using the Target call sign in the IM instruction, and rate 

the operational acceptability of it. 

- All eight responses stated it was acceptable to use the Target call sign when issuing an 

IM clearance. 

- Seven of the eight responses stated there was no confusion due to using the Target’s 

call sign in a voice instruction to the IM aircraft, with the remaining response noting 

that the Target call sign occasionally had to be spelled phonetically. 

- The “report paired” should be moved to ATC’s acknowledgement of the flight crew’s 

read back of the IM instruction, and not as part of the initial issuing of the instruction. 

• #19: Describe any challenges to implementing IM operations. 

- “Using data link to transmit IM clearances would improve/increase the delivery of said 

clearances and reduce misunderstandings.” 

- “Adjusting the scanning technique. Currently most scanning is altitude, speed, and then 

time when determining separation. With IM it was time, altitude, and then speed.” 

In summary, there was a wide range of responses that did not appear to be related to the role 

(ARTCC or TRACON) or experience of the controller. 

 
 67 

5.7 Flight Crew Objective Performance Results 

5.7.1 Percentage of Flight Crew Interrupted IM Operations 

Hypothesis #3 states the percentage of IM operations terminated by the flight crew (i.e., pressing 

the CANCEL button) prior to the Planned Termination Point will be less than 30%. 

Of the 79 IM operations, there were no IM operations terminated by the flight crew, and therefore 

the criteria for this hypothesis was met (p < 0.0005). However, there were several instances where 

the subject pilots verbally commented after the scenario that they would have terminated the IM 

operation if it had been a real-world operation. Examples of rationale they provided include: an 

IM speed of 210 knots commanded at FL210 (traced to a software error); the IM aircraft perceived 

to be too close to the preceding aircraft (see Section 5.9.1); and instances of high frequency of IM 

speed changes and speed reversals (see Sections 5.8.1.8 and 5.9.2). 

5.7.2 Flight Crew Reaction Time to IM Speed Command Change 

The flight crew reaction time to IM commanded speed changes is defined as the seconds between 

the new IM commanded speed being first displayed and the flight crew beginning to enter that 

speed in the MCP speed window (Table 17 and Figure 32). The overall mean of 4.0 seconds and 

standard deviation of 3.0 seconds is better than the flight crew reaction time observed in previous 

IM research simulation (for example, a mean of 10.5 seconds and standard deviation of 9.2 seconds 

in ref. 11). The research team have several possible theories to be explored in future research, but 

no evidence to postulate the reason for the faster flight crew reaction time to IM commanded speed 

changes. 

Reaction time when blocked by simulator type shows a somewhat shorter reaction time in the fullscale high-fidelity simulators (the DTS and IFD) when compared to the medium-fidelity 

simulators (the ASTORs). This was done to validate results obtained from the ASTOR, where the 

flight crew had to time-share a single input device to both respond to ATC and enter the speed in 

the MCP. The 1.4 second difference in flight crew mean reaction time to an IM speed change 

indicates the ASTOR is slightly more challenging to operate than the full-scale simulators, 

however the time difference is not operationally significant. 

The reaction time was also examined to determine if it was different at lower altitudes (less than 

11,000 feet) where the flight crew must begin to configure the aircraft for landing. The results in 

Table 17 show a negligible difference by altitude. 

Table 17. Flight Crew Reaction Time to IM Commanded Speed Change in Seconds 

Simulator or Altitude Band N Mean SD N’ 

All simulators, all altitudes 783 4.0 3.0 36 

• ASTOR only 536 4.3 3.3 22 

• DTS only 142 3.4 2.2 4 

• IFD only 105 2.9 1.9 10 

• Above 11,000’ MSL 469 3.9 2.9 5 

• Below 11,000’ MSL 314 4.0 3.2 31 
 68 

The N’ value in the far right column indicates the number of times a second speed change happened 

prior to the flight crew responding to the first speed command, which is an indirect measure of the 

frequency of the IM speed changes and the ability of the crew to respond based on their other 

cockpit tasks. Approximately 1% of the speed changes above 11,000’ MSL (prior to the crew 

beginning to configure the aircraft for landing) were not responded to prior to the next speed 

change, while below 11,000’ MSL a non-response was recorded for 10.0% of the speed changes. 

This is operationally significant and reemphasizes the importance of reducing the frequency of 

changes to the IM speed command, particularly in the higher workload environment created when 

the crew configures the aircraft. 

Figure 32. Scatter plot of pilot reaction time in seconds by altitude by simulator type. 

Note: a 40 second outlier from an ASTOR simulator during a CROSS operation was removed from 

Figure 32 for clarity (but the value retained in calculations and Table 17). 

5.7.3 Flight Crew Conformance to the IM Speed 

The RMS of the difference between the IM instantaneous airspeed (see Section B.4.2 for a detailed 

description) and the aircraft’s actual airspeed was analyzed as a method of determining the 

similarity/dissimilarity of the three different simulator types while conducting the IM operation. 

The 1.2 knot difference in the mean and standard deviation by simulator type is operationally 

insignificant, indicating that in terms of the flight crew maintaining the appropriate speed, the 

simulators are comparable. Based on Boeing provided documentation of auto-throttle behavior, 

and pilot responses on the surveys, the speed conformance is well within the range of expected 

flight operations. 

Table 18. RMS of Aircraft Deviation from IM Commanded Speed in Knots 

Simulator N Mean SD 

All simulators 79 7.70 3.19 

• ASTOR only 53 7.70 2.94 

• DTS only 13 8.88 4.10 

• IFD only 13 7.70 2.94 

0

5000

10000

15000

20000

25000

30000

35000

0 5 10 15 20 25

MSL Altitude (feet)

Time for pilot to begin setting new IM commanded speed in MCP (in seconds)

ASTOR

DTS

IFD
 69 

5.7.4 Missed Altitude Constraints 

Based on pilot and controller subject matter expert feedback during previous IM simulations, there 

was concern that IM operations may be more susceptible to missed altitude constraints than nonIM operations. The primary reason for this concern is that the aircraft’s FMS trajectory is not 

updated when a new IM speed is provided. This can place aircraft into a less than ideal energy 

state that requires the flight crew to use drag devices to maintain the aircraft’s initial vertical path. 

The objective of this analysis was to try and infer whether IM operations increased the likelihood 

that pilots would miss an altitude constraint. 

A missed altitude constraint was counted every time the aircraft crossed an altitude-constrained 

waypoint in excess of 200 feet above or below the constraint associated to that waypoint. The 200 

feet value was selected to be consistent with the methodology used for aircraft data tags on 

controller displays that show an aircraft’s altitude. Additionally, if the aircraft’s lateral position 

was not within 2 nmi of the altitude-constrained waypoint, it was assumed that the aircraft was 

vectored and that the altitude constraint did not apply to that aircraft. 

During the experiment, 110 arrival operations (Table 19) containing 1,240 altitude-constrained 

waypoint crossings (Table 20) were flown by ASTOR, DTS, and IFD subject pilots. 23 of the 110 

(21%) flights had at least one missed altitude constraint, and during these flights, 32 of the 1240 

(2.6%) waypoints with altitude constraints were missed. In all of these cases, the aircraft crossed 

the altitude constraint too high. The missed altitude constraints do show a correlation to simulator 

type, with the ASTOR desktop simulators having a substantially higher percentage of flights and 

waypoint crossings with missed altitude constraints compared to the DTS and IFD simulators. 

Table 19. Arrival Operations with Missed Altitude Constraints 

Simulator Type ASTOR DTS IFD TOTAL 

Total number of all flight operations (1) 74 19 17 110 

Number of flights with ≥ 1 missed altitude constraint 20 2 1 23 

Percent of flights with ≥ 1 missed altitude constraint 27% 11% 5.9% 20.9% 

Note (1): the 110 operations in this table is less than the 113 operations listed in Appendix G since 

three additional arrival operations were excluded from this specific analysis due to being vectored. 

Table 20. Waypoints with Missed Altitude Constraints 

Simulator Type ASTOR DTS IFD TOTAL 

Number of waypoint crossings with altitude constraint 844 207 189 1240 

Number of waypoints with missed altitude constraint 29 2 1 32 

Percent of waypoints with missed altitude constraint 3.4% 1.0% 0.6% 2.6% 

Subset of waypoints during non-IM operations 2 1 0 3 

Subset of waypoints during IM operations 27 1(2) 1 29 

Note (2): the DTS crew did foresee their inability to meet the upcoming altitude constraint due to 

the IM speed change (slower speed caused the aircraft to decrease the descent rate), therefore 

requested an altitude waiver from air traffic which the controller granted. 
 70 

The missed altitude constraints were also analyzed to determine if there was a correlation by airline 

or by crew. No trend was found by airline or subject group; however, the results by crew indicate 

that 17 of 32 (53%) of the missed altitude constraints were caused by just 2 of 12 (17%) of the 

crews. These two crews, both flying the ASTOR simulators but in different subject groups, had all 

17 missed altitude constraints occur while conducting IM operations, and none when conducting 

non-IM operations. This provides strong evidence that the single mouse-driven interface of the 

ASTOR simulator was challenging or even prevented independent actin by those flight crews when 

compared to the full-scale DTS and IFD simulators. This ASTOR interface hypothesis is also 

corroborated by the unstable approach data (next section) where ASTOR pilots had a higher rate 

of unstable approaches than the DTS or IFD. Anecdotally, it also appears that some flight crews 

were less experienced at predicting the change in the aircraft’s vertical trajectory resulting from 

the pending speed change. 

The magnitude of the altitude miss were categorized into small (less than 400 ft.) or large (greater 

than 400 ft.) events (Figure 33). One-third of the non-IM operations (1 of 3) and approximately 

half of the IM operations (15 of 29) missed an altitude constraint by greater than 400 ft. 

Figure 33. The magnitude of missed altitude constraints by simulator type. 

Next, the missed altitude constraints were examined to determine if they were influenced by the 

scenario type (i.e., BASELINE, CAPTURE, CROSS, MAINTAIN, MIXED). Table 21 shows the 

percentage of flights that contained at least one missed altitude constraint for each scenario type, 

and as a percentage of the total number of altitude constraints encountered in each scenario type. 

The BASELINE scenario (non-IM operation) had the smallest percentage of aircraft with at least 

one missed altitude constraint (8.3%), and the smallest percentage of waypoints with missed 

altitude constraints (0.7%). However, the CAPTURE scenario also had relatively few missed 

constraints. It is unclear why the difference in missed altitude constraints between the different IM 

scenarios exists. 

 

0

500

1000

1500

2000

2500

3000

3500

ASTOR DTS IFD

Altitude Violation Magnitude (ft)
 71 

Table 21. Number and Percent of Missed Altitude Constraints by Scenario Type 

Operation Type BASELINE CAPTURE CROSS MAINTAIN MIXED 

Total number of flight operations (1) 24 19 24 20 23 

Number of flights with a missed altitude 2 2 4 5 10 

Percent of flights with a missed altitude 8.3% 10.5% 16.7% 25% 44% 

Number of waypoints with altitude constraints 272 241 257 223 247 

Number of waypoints with missed altitude 2 4 5 8 13 

Percent of waypoints with a missed altitude 0.7% 1.7% 1.9% 3.6% 5.3% 

Finally, video recordings were also examined for missed altitude constraint events to get more 

insight into why they occurred. It was found that 23 out of 32 of these IM and non-IM events 

occurred relatively soon after the flight crew received a speed change from IM or air traffic control 

(i.e., within 8 nmi prior to a waypoint with an “at” or “at or below” constraint), suggesting that 

speed changes immediately prior to altitude constrained waypoints is a significant pre-condition 

to the missed altitude constraint event. The primary factor contributing to these events continues 

to appear to be the ASTOR single mouse-driven interface that precluded independent action by 

the flight crew (responding to air traffic control, entering a speed, deploying the speed brake, etc.). 

However, based on discussions during the week-long training session and comments made during 

debrief, that some flight crews had rarely or never used the VNAV speed mode, and were 

challenged to predict the change to the aircraft’s vertical trajectory based on a pending speed 

change with enough accuracy to determine if speed brakes were required to meet an altitude 

constraint. Furthermore, they stated the new requirement to maintain vertical path with power and 

speed brake added to their mental workload and required additional head-down time.3

 

3

 Specifically, when flying in VNAV speed mode in the aircraft types used in this experiment, the 

aircraft uses pitch to achieve speed, and the pilots are required to use throttle and drag devices to 

maintain the aircraft’s vertical path. The VNAV speed mode provides protection against crossing 

a waypoint at too low an altitude (the mode will switch to VNAV path); however, there is no 

automatic protection against crossing a waypoint at an altitude higher than the constraint. Within 

this experiment, all of the missed altitude events were above the altitude constraint – where VNAV 

speed mode does not provide protection and the pilot must apply sufficient drag if required. This 

particular root cause occurred in both IM and non-IM operations. 

It should be noted that the IM research teams at NASA Langley and elsewhere have known that 

the VNAV speed mode is not commonly used by all flight crews, and that managing the aircraft’s 

vertical path and correcting with either throttle or speed brake can be high workload. Therefore, 

mitigations were included in the experimental design: (1) each group of subject pilots was selected 

based on their being qualified for that particular simulator type; (2) all subject pilots were given a 

dedicated one-hour classroom discussion about VNAV during the week-long training, and (3) they 

all were given hands-on practice conducting VNAV speed mode operations in the simulator they 

were to fly during this experiment. 
 72 

To summarize this analysis, 21% of the IM and non-IM operations by subject pilots had at least 

one missed altitude constraint. Of these events, 91% occurred during IM operations, 87% were 

committed by the ASTOR simulators, 53% of them were from just two crews (both operating an 

ASTOR desktop simulator), and 72% percent of the missed altitude constraints occurred when the 

flight crew received a speed change from the IM software or air traffic control when less than 8 

nmi prior to an altitude constrained waypoint. 

The conclusion from this missed altitude constraint analysis is the single ASTOR interface 

prevented independent action by the flight crew, which was the primary cause for the 

disproportionate number of missed altitude events occurring by the ASTOR simulators. 

Anecdotally, some of the flight crew did not have familiarity using the VNAV speed mode, and 

did not appear to predict the change in the aircraft’s vertical trajectory due to a speed change with 

enough accuracy to adhere to published altitude constraints. 

5.7.5 Unstable Approaches 

A stabilized approach (ref. 34, chapter 15) is characterized as the aircraft having a constant angle 

and constant rate of descent while meeting the following criteria no later than when descending 

through 1,000 feet above the ground: 

• ± 10 knots of the final approach speed, 

• landing gear down and flaps set to the desired landing position, 

• engines spooled for the aircraft configuration and speed, and 

• on lateral path and on vertical profile. 

A rudimentary analysis was conducted using the first three criteria when the aircraft descended 

through 1000 feet above the airport elevation to determine if the IM procedures impacted how the 

flight crew operated their aircraft during this critical stage of flight (Table 22). 

Table 22. Operations not Meeting Stabilized Approach Criteria 

ASTOR DTS IFD 

Number of all subject pilot flown operations 78 17 18 

• Total number of non-IM operations 25 4 5 

• Number (%) of unstable non-IM approaches 12 (48%) 1 (25%) 0 (0%) 

• Total number of IM operations 53 12 13 

• Number (%) of unstable IM approaches 9 (17%) 2 (17%) 1 (8%) 

The following conclusions were drawn from the data shown in Table 22: 

• Achieving a stabilized approach appears to have been a more significant challenge in the 

ASTOR simulator as compared to the full-scale DTS and IFD simulators. A significant 

factor is believed to be the use of a single computer-mouse interface, time-shared by the 

two pilots, that was used by one crew member to respond to ATC instructions on the radio, 

and the other crew member making changes to the aircraft’s heading and airspeed. In other 

words, the single mouse-interface prevented normally independent action by the two pilots. 
 73 

• The IM operations had a lower rate of unstable approaches than non-IM (current day) 

operations. The ASTAR spacing algorithm is specifically designed to assist the flight crew 

in achieving a stabilized approach; however, it cannot be conclusively shown from these 

results that the algorithm design itself caused the improved performance. It can be stated 

that the IM operation procedure does not appear to increase the likelihood of an unstable 

approach compared to current day operations. 

5.8 Flight Crew Subjective Assessment Responses 

For the flight crew post-run questionnaires, a sample size of N = 48 responses was anticipated for 

each scenario type; however, simulation errors during four flights resulted in N = 46 and N = 42 

for the CAPTURE and MAINTAIN scenarios, respectively. The impacted data includes Table 23 

through Table 32, Table 34, Table 52, and Table 53. 

For flight crew acceptability and workload metrics discussed in this section, statistical analysis 

was performed using the Wilcoxon signed rank test, a nonparametric test appropriate for analyzing 

ordinal data (ref. 27). There were no statistically significant differences between mean responses 

for the PF and PM for any of the pilot post-run questionnaire items (p ≥ 0.100), and therefore data 

were combined for all subsequent analyses. 

5.8.1 Flight Crew Acceptability 

Hypothesis #4 states the mean flight crew acceptability rating of IM operations, using a 7-point 

Likert rating scale (from ‘1’ = “Completely Unacceptable” to ‘7’ = “Completely Acceptable”), 

will be ≥ ‘5’ (i.e., top one-third of the 1-to-7 scale, where a rating of ‘5’ indicates “Slightly Agree”). 

Nine different measures are analyzed in this section (listed from highest to lowest importance) to 

provide in-depth assessment of the flight crew acceptability ratings of IM operations. All nine 

measures used in this research indicate that in general, the flight crew found the IM operations to 

be acceptable during high-density arrival operations. 

5.8.1.1 Post-Run Acceptability of Overall IM Operation 

Descriptive statistics associated with the ratings of overall acceptability of IM are shown in Table 

23 and Figure 34. For all five operations, the mean acceptability ratings were statistically greater 

than ‘5’ (p ≤ 0.006), indicating that flight crews found all IM operation types to be acceptable on 

average. (The ‘5’ criteria is indicated by a dashed green line in Figure 34.) 

Table 23. Descriptive Statistics for Flight Crew Acceptability Ratings 

Operation N Mean SD Min Median Max 

BASELINE 48 6.8 0.6 5 7 7 

CAPTURE 46 5.6 1.4 1 6 7 

CROSS 48 5.7 1.1 2 6 7 

MAINTAIN 42 5.8 1.5 1 6 7 

MIXED 48 5.6 1.6 1 6 7 
 74 

Baseline Capture Cross Maintain Mixed

7

6

5

4

3

2

1

Operation

Acceptability Ratings

Figure 34. Flight crew acceptability of operations by IM operation type. 

There were five pilot post-run acceptability ratings of ‘1.’ One of the pilots felt the speed changes 

were too large, and reported receiving commands of 230 to 290 and then to 240 within 45 seconds 

during a CAPTURE scenario. During a MAINTAIN scenario, both pilots in one flight crew 

reported that they received a speed decrease at FL290 from 280 to 230 kts, then a 210 kt speed 

command at FL210, which was soon followed by the controller cancelling IM and assigning a 

speed of 260 kts (the 210 knot speed instruction was due to a software error discovered after data 

collection). Another pilot reported IM speed commands of 180 to 130 and then to 160 while on 

final (due to the ground speed term phase out described in Section 5.9.5), and that pilot stated the 

IM speed changes need to be small and more frequent rather than large speed reductions and speed 

reversals. 

Two pilots provided acceptability ratings of ‘2.’ During a CROSS scenario, one pilot attempted to 

enter the clearance twice in the left EFB and received ‘UNABLE’ and ‘TGT BAD ROUTE’ 

messages; on the third try he used the right EFB at the observer’s suggestion and was able to pair 

(traced to a software logic error). Another pilot reported receiving four speed reductions in 30 

seconds from 180 to 140 on final, while the aircraft’s minimum speed is 144 kts. 

For the eight acceptability ratings of ‘3,’ pilots reported receiving too many speed commands, 

large speed reductions resulting in being unable to meet altitude constraints, speed reversals, and 

commanded speeds below the aircraft minimum speed. Another pilot reported confusion due to 

receiving an assigned speed from ATC without explicitly being told to suspend IM operations, 

while also getting a frequency handoff at the same time. Additional analysis is in Appendix F.4. 

5.8.1.2 Post-Run Acceptability of IM Operation by Phase of Flight 

The flight crew also rated the acceptability of IM operations by phase of flight (from ‘1’ = 

“Completely Unacceptable” to ‘7’ = “Completely Acceptable”). In general, the flight crew rated 

the acceptability of IM operation types very high (7) when in the higher-segment (> FL180). The 

CROSS and CAPTURE operations were rated very high (7) in the mid-level segment (11,000 – 

18,000’ MSL) and high (6) in the lower-level segment (Surface – 11,000’). The MAINTAIN 

operation was rated slightly differently by the flight crew, with the mid-level rated high (6) and 

the lower-level rated very high (7). Additional analysis is in Appendix F.4. 
 75 

5.8.1.3 Post-experiment acceptability of IM procedures 

The mean flight crew acceptability rating (scale of 1 to 7, with 7 being “Completely Acceptable”) 

of the three IM operation types on the post-experiment survey is shown in Table 24, with a 

summary of flight crew comments immediately below it. All the comments are available in 

Appendix F.6, question #8. 

Table 24. Descriptive Statistics for Pilot Operational Acceptability of IM Procedures 

Operation N Mean SD Min Median Max 

CAPTURE 24 6.4 0.9 4 7 7 

CROSS 24 6.3 0.8 5 6.5 7 

MAINTAIN 24 5.9 1.5 1 6 7 

• Pilots individually provided nearly identical ratings of acceptability for CAPTURE, 

CROSS, and MAINTAIN. There were a couple of exceptions where the acceptability of 

the MAINTAIN operation was rated very low. 

• When the MAINTAIN clearance was issued in the Mach regime, the time delay until the 

IM operation initiated in the CAS regime caused some pilots to believe the IM software 

was ineffective or not working correctly. 

• Getting the clearance early from ATC made it easier to follow IM procedures. 

• Acceleration/Decelerations were sometimes too large and may affect passenger comfort. 

• Excessive broadcast time for some clearances, a suitable solution would be communication 

using data link instead of voice. 

5.8.1.4 Post-Run Acceptability of IM Procedures 

Descriptive statistics associated with the acceptability ratings are shown in Table 25 and Figure 

35. For all IM procedures, the mean acceptability ratings were statistically greater than ‘5’ (p < 

0.0005). Therefore, flight crews found the IM procedures to be acceptable overall. 

Three pilots rated the acceptability of the IM procedures as ‘1,’ one pilot rated it as ‘2,’ and four 

pilots rated it as ‘3.’ Several factors reported by the pilots may have contributed to these lower 

ratings: being unable to meet altitude constraints due to IM commanded speed reductions, too 

many speed changes especially on final, large speed changes, speed reversals, and speed reductions 

resulting in configuring early followed by speed increases. One pilot who rated the IM procedures 

during the MIXED scenario as ‘2’ felt that things went well until inside the FAF, at which point 

they were closing on the Target aircraft and believed they would have executed a go-around in the 

real world. Another pilot who rated the IM procedures as ‘3’ during the MIXED scenario reported 

that after his aircraft was paired, ATC canceled the operation and then said to be prepared for a 

CAPTURE clearance, but did not provide direction on the desired speed. ATC then issued the 

second IM clearance without asking if the flight crew was ready to copy. Additional information 

is in Appendix F.4, question #18. 
 76 

Table 25. Descriptive Statistics for Flight Crew Acceptability of IM Procedures Ratings 

Operation N Mean SD Min Median Max 

CAPTURE 46 6.0 1.2 1 6 7 

CROSS 48 6.0 1.0 3 6 7 

MAINTAIN 42 6.0 1.4 1 6 7 

MIXED 48 6.0 1.3 2 6.5 7 

CAPTURE CROSS MA INTA IN MIXED

7

6

5

4

3

2

1

Operation

Acceptability Ratings

p y

Figure 35. Flight crew acceptability of IM procedure by IM operation type.

5.8.1.5 Post-Experiment Acceptability of IM Procedures 

The flight crew rated the acceptability (scale of 1 to 7, with 7 being “Completely Acceptable”) of 

the IM procedures on the post-experiment survey as a mean of 6.1 (standard deviation of 0.8). 

This indicates that the pilots found the IM procedures to be acceptable overall, which is consistent 

with the post-run acceptability ratings discussed in Section 5.8.1.4. Additional data is in Appendix 

F.6, question #9. 

5.8.1.6 Post-Run Completeness of IM Procedures 

Descriptive statistics associated with the completeness ratings are shown in Table 26 and Figure 

36. For all IM procedures, the mean completeness ratings were statistically greater than ‘5’ (p < 

0.0005). Therefore, flight crews found the IM procedures to be complete (that is, all crew actions 

required to conduct the IM operation were given during training) overall. 

One pilot rated the completeness of the IM procedures as ‘1,’ one pilot rated it as ‘2,’ and three 

pilots rated it as ‘3.’ One pilot felt there was limited information for the flight crew to know that 

‘TGT BAD ROUTE’ message meant the aircraft would not pair without further intervention from 

them. During a MAINTAIN scenario, one pilot reported that the speed commands were too large 

and the operation was canceled by ATC. Another pilot felt that receiving a speed command from 

ATC without direction to suspend IM caused confusion. A fourth pilot received a speed command 

just prior to the IM system auto-suspending, which was unacceptable for maintaining a stable 

approach. Another pilot also reported that the speed commands were too large on final. Additional 

analysis is in Appendix F.4, question #19. 
 77 

Table 26. Descriptive Statistics for Completeness of IM Procedures Ratings 

Operation N Mean SD Min Median Max 

CAPTURE 46 6.4 0.9 3 7 7 

CROSS 48 6.3 0.9 3 6 7 

MAINTAIN 42 6.3 1.2 1 7 7 

MIXED 48 6.3 1.1 2 7 7 

CAPTURE CROSS MAINTAIN MIXED

7

6

5

4

3

2

1

Operation

Completeness Ratings

Figure 36. Flight crew completeness of IM procedure by IM operation type.

Pilots were also asked if there were any steps missing from the IM procedures (Appendix F.4, 

question #20), if there were extra steps that were unnecessary (question #21), and if the steps were 

logical and easy to follow (question #22). The ratings are summarized below in Table 27. 

Comments from these questions are repeated or incorporated in comments elsewhere (for example, 

the high frequency and the location of IM speed changes). 

Table 27. Pilot Percentage Reporting Missing, Unnecessary and Logical Steps by Type 

Operation N 

Missing Steps 

(% No) 

Unnecessary Steps 

(% No) 

Logical Steps 

(% Yes) 

CAPTURE 46 93% 98% 89% 

CROSS 48 92% 98% 92% 

MAINTAIN 42 100% 95% 93% 

MIXED 48 94% 94% 92% 

5.8.1.7 Post-Run Acceptability of IM Commanded Speeds 

Descriptive statistics associated with the ratings of operational acceptability of IM commanded 

speeds are shown in Table 28 and in Figure 37. For the CAPTURE and MAINTAIN operations, 

the mean acceptability ratings were statistically greater than ‘5’ (p ≤ 0.026). For the CROSS 

operation and the MIXED scenario, the mean acceptability ratings were not statistically greater 

than ‘5’ (p = 0.062 and p = 0.086, respectively), although the median rating was ‘6.’ This indicates 

that flight crews found the IM commanded speeds to be operationally acceptable for the 

CAPTURE and MAINTAIN operations, but were uncomfortable with some of the commanded 

speeds during the CROSS operation and the MIXED scenario. 
 78 

Nine pilots rated the operational acceptability of the IM commanded speeds as ‘1,’ six pilots rated 

it as ‘2,’ and six pilots rated it as ‘3.’ They reported being unable to meet altitude constraints due 

to IM commanded speed reductions, numerous speed changes, large speed changes, speed 

reversals, and speed reductions resulting in configuring early followed by speed increases, and 

commanded speeds less than the aircraft minimum speed. Additional information is in Appendix 

F.4, question #15. 

Table 28. Descriptive Statistics of Operational Acceptability of IM Speeds 

Operation N Mean SD Min Median Max 

CAPTURE 46 5.4 1.7 1 6 7 

CROSS 48 5.2 1.5 1 6 7 

MAINTAIN 42 5.5 1.7 1 6 7 

MIXED 48 5.3 1.7 1 5.5 7 

CAPTURE CROSS MA INTA IN MIXED

7

6

5

4

3

2

1

Operation

Acceptability Ratings

Figure 37. Flight crew operational acceptability of IM speeds by IM operation type. 

5.8.1.8 Post-Run Acceptability of Frequency of IM Speed Changes 

Descriptive statistics associated with the ratings of acceptability of the frequency of IM speed 

changes are shown in Table 29 and Figure 38. For the CROSS operation, MAINTAIN operation, 

and MIXED scenario, the mean acceptability ratings were statistically greater than ‘5’ (p ≤ 0.007), 

indicating that flight crews found the frequency of IM speed changes to be acceptable. For the 

CAPTURE operation, the mean acceptability rating was not statistically greater than ‘5’ (p = 

0.072). Although the median rating for the CAPTURE operation was ‘6,’ the variability in the 

ratings was slightly higher than for the other scenarios (standard deviation = 1.6). 

Three pilots rated the acceptability of the frequency of IM speed changes as ‘1,’ four pilots rated 

it as ‘2,’ and eleven pilots rated it as ‘3.’ They reported being unable to meet altitude constraints 

due to IM commanded speed reductions, too many speed changes especially on final, large speed 

changes, speed reversals, and speed reductions resulting in configuring early followed by speed 

increases. Several pilots also commented that size and frequency of speed changes would be 

unacceptable with passengers onboard the aircraft. Additional information is in Appendix F.4, 

question #16. 
 79 

Table 29. Descriptive Statistics of Acceptability of Frequency of IM Speed Changes 

Operation N Mean SD Min Median Max 

CAPTURE 46 5.3 1.6 1 6 7 

CROSS 48 5.5 1.2 2 6 7 

MAINTAIN 42 5.6 1.3 1 6 7 

MIXED 48 5.5 1.5 1 6 7 

CAPTURE CROSS MAINTAIN MIXED

7

6

5

4

3

2

1

Operation

Acceptability Ratings

Figure 38. Flight crew acceptability of IM speed change frequency by IM operation type. 

5.8.1.9 Post-Run Acceptability of Head Down Time 

Descriptive statistics associated with the acceptability ratings of flight crew head down time are 

shown in Table 30 and Figure 39. For all IM procedures, the mean acceptability ratings were 

statistically greater than ‘5’ (p < 0.0005). Therefore, flight crews found the amount of head down 

time required of the PM to input information from the IM clearance(s) into the EFB to be 

acceptable. 

Six pilots rated the acceptability of the amount of head down time as ‘2,’ and one pilot rated it as 

‘3.’ They felt there was too much head down time required after pairing, especially below 10,000 

feet. Two pilots suggested more aural or visual alerting for CGD changes, and one pilot felt that 

cancelling and revising IM clearances below 10,000 feet should not be considered due to the 

amount of head down time required. Additional information is in Appendix F.4, question #17. 

Table 30. Descriptive Statistics for Acceptability of Amount of Head Down Time 

Operation N Mean SD Min Median Max 

CAPTURE 46 6.2 1.2 2 7 7 

CROSS 48 6.3 1.0 2 7 7 

MAINTAIN 42 6.3 1.1 2 7 7 

MIXED 48 6.2 1.1 2 6 7 
 80 

CAPTURE CROSS MAINTAIN MIXED

7

6

5

4

3

2

1

Operation

Acceptability Ratings

Figure 39. Flight crew acceptability of head down time by IM operation type.

In summary, nine different aspects of the acceptability of IM operations to pilots were assessed, 

and generally the pilots rated the IM procedure and actions required to accomplish it as acceptable. 

However, the frequency of the IM speed changes was rated as slightly less acceptable. 

5.8.2 Flight Crew Workload of IM Operations 

Hypothesis #5 states the mean flight crew workload rating of IM operations, measured using the 

Modified Cooper-Harper (MCH) subjective workload rating scale (ref. 22) on post-run surveys, 

will be less than or equal to ‘3’. 

A rating of ‘3’ on the MCH rating scale indicates that the instructed task is fair and/or has mild 

difficulty, and acceptable operator mental effort is required to attain adequate system performance. 

Descriptive statistics associated with the pilot workload ratings are shown in Table 31 and Figure 

40. For all five operations, the mean MCH workload ratings were statistically less than ‘3’ (p ≤

0.010). Therefore, overall, flight crews found the workload level experienced to be acceptable on 

average. 

There were two workload ratings of ‘10.’ The pilot who rated the BASELINE scenario as ‘10’ did 

not indicate any issues during the run. The pilot who rated his workload as ‘10’ during a 

MAINTAIN scenario reported receiving excessive speed reduction from the IM commanded 

speeds, and the operation was canceled. Of the three pilots who rated their workload as ‘9,’ one 

indicated too many speed changes causing the gear to be lowered, raised, and then lowered again. 

The other two pilots both indicated that large IM commanded speed reductions resulted in a failure 

to meet altitude restrictions during MAINTAIN scenarios. The twelve pilots who experienced 

workloads of ‘7’ and ‘8’ reported being unable to meet altitude constraints due to IM commanded 

speed reductions, numerous speed changes, large speed changes, speed reversals, and speed 

reductions resulting in configuring early followed by speed increases. Additional information is in 

Appendix F.4, questions #5 - #10. 

In summary, the flight crews in general found the workload of any IM operation type to be low. 

Workload was rated higher when frequent IM speed changes were commanded, and when a speed 

change caused the flight crew to not meet an altitude constraint. 
 81 

Table 31. Descriptive Statistics for Flight Crew Workload Ratings 

Operation N Mean SD Min Median Max 

BASELINE 48 1.4 1.3 1 1 10 

CAPTURE 46 2.3 1.4 1 2 8 

CROSS 48 2.4 1.9 1 2 9 

MAINTAIN 42 2.5 2.5 1 2 10 

MIXED 48 2.4 1.9 1 2 8 

BASELINE CAPTURE CROSS MA INTAIN MIXED

10

9

8

7

6

5

4

3

2

1

Operation

Pilot Workload Ratings

Figure 40. Flight crew perception of workload by IM operation type.

5.8.3 Flight Crew Ratings and Comments about Situation Awareness 

The pilots rated the situation awareness provided by the IM operation and displays (scale of 1 to 

7, with 1 being “Severely Degraded Situational Awareness” to 7 being “Greatly Improved 

Situational Awareness”) on the post-experiment survey as a Mean of 5.8 (standard deviation = 

1.6). Complete information is in Appendix F.6, question #10. A synopsis of the comments is: 

• Situation awareness was higher due to the ability to see aircraft with ID tags. 

• Task saturation near the FAF during an IM operation reduced situation awareness. 

• An aural warning for speed changes would reduce fixation on the speed command. 

• EFB display and filters enhanced situation awareness of other aircraft on the arrival. 

• While the flight crews’ awareness was enhanced regarding surrounding aircraft, the IM 

operation and displays could distract from the crews’ awareness of the Ownship state. 

In summary, the cockpit display of traffic shown on the EFB enhanced the flight crews’ situation 

awareness, while the IM operation itself did not generally impact awareness, except when frequent 

IM speed changes occurred close to the FAF. 
 82 

5.8.4 Flight Crew Comments about ATD-1 ConOps and IM Operations 

5.8.4.1 Controller Retains Separation Responsibility While the Flight Crew Space 

The pilots rated the acceptability of the controller retaining separation responsibility while the 

flight crew ensures proper spacing during IM operation (scale of 1 to 7, with 1 being “Completely 

Unacceptable” to 7 being “Completely Acceptable”) on the post-experiment questionnaire as a 

Mean of 6.4 (standard deviation = 0.8). Complete information is in Appendix F.6, question #11. 

A synopsis of the comments is: 

• “As long as the controller controls the IM pairing message to keep proper spacing.” 

• “As long as the controller controls the IM pairing message to keep proper spacing. We 

have to be able to trust that we have enough spacing.” 

5.8.4.2 IM Commanded Speed Causing Unexpected or Undesired Behavior 

Nineteen of the 24 (79%) pilots reported in the post-experiment survey that the IM commanded 

speeds at some time caused unexpected or undesired behavior. Complete information is in 

Appendix F.6, question #12. A synopsis of the comments is: 

• Multiple large speed changes, especially near the FAF (termination point) are undesired 

especially when affecting passenger comfort. 

• Certain FMS vertical operational modes when following the IM commanded speed 

eliminate altitude protection. 

• On occasion the commanded speed was below landing speed or above over-speed limits. 

• Large speed changes reduced situation awareness, resulting in missed altitude restrictions. 

• The aircraft had to be configured early to meet speed commands. 

5.8.4.3 Desired Changes to the IM Operation 

Overall the flight crews indicated the IM operation was acceptable, but had a detailed list of items 

that need to be addressed before being implemented in current-day operations. Complete 

information is in Appendix F.6, question #13. A synopsis of the comments is: 

• Data link is a highly desirable enabling component for IM. 

• IM operations need to engage as soon as the flight crew enters the information, regardless 

of Mach or calibrated airspeed segment, and regardless of prior or after a waypoint with a 

speed constraint. 

• Consideration should be given for an algorithm catered to aircraft specific performance. 

• Minimize the speed changes. 

• No IM speed changes (or operation) within 2 miles of the FAF to allow crew to focus on 

configuring the aircraft and achieving a stabilized approach for landing. 

• The FAST/SLOW indicator should be integrated into the speed tape. 

• An aural or visual alerting system needs to be incorporated to reduce head-down time. 
 83 

5.8.4.4 Flight Crew Comments on IM Phraseology 

Pilots were asked if they experienced any issues with the IM phraseology. The full list of comments 

is in Appendix F.6, with a summary of those comments directly below and Table 32 indicating the 

IM operation type they occurred in. Complete information is in Appendix F.4, question #24. A 

synopsis of the comments is: 

• Received IM clearance prior to receiving ‘descend via’ clearance. 

• ATC issued speed commands after the aircraft paired, but did not issue suspend IM 

command. 

• ATC told flight crew to “discontinue” spacing rather than “suspend” or “cancel.” 

• ATC used call sign for Target aircraft when issuing IM clearance, and the flight crew of 

the IM aircraft did not know the corresponding three letter identifier. 

• ATC issued MAINTAIN clearance and included the Target’s route. 

• ATC issued MAINTAIN clearance and included spacing goal. 

• ATC issued MAINTAIN clearance but did not specify miles or seconds. 

• There was confusion among the pilots as to whether or not they needed to advise the next 

controller that they were paired. 

• Slightly more than half of the pilots felt the phonetic identifier should be used for the 

Target’s call sign during the IM clearance instruction. 

Table 32. Pilot Percentage Reporting Issues with IM Phraseology by IM Operation Type 

Operation N % Reporting Phraseology Issue 

CAPTURE 46 4% 

CROSS 48 4% 

MAINTAIN 42 31% 

MIXED 48 10% 

The type of third party ID used by the controller to issue the IM clearance is shown in Table 33. 

Three pilots reported that both the call sign and alpha-numeric identifier were used in the same 

clearance. In addition, six pilots received a clearance with the Target aircraft’s call sign and had 

to query ATC for the three letter identifier. Complete information is in Appendix F.4, question 

#25. 

Table 33. Type of Third Party ID Used to Issue the IM Clearance 

Third Party ID N 

Call sign (e.g., Brickyard 123) 158 

Alpha-numeric (e.g., R P A 123) 11 

Phonetic (e.g., Romeo Papa Alpha 123) 4 

Other 14 
 84 

5.8.4.5 Post-Run Acceptability of the Use of Voice Communications 

The acceptability of the use of voice communications was measured using a 7-point Likert rating 

scale (‘1’ = “Completely Unacceptable” to ‘7’ = “Completely Acceptable”). Descriptive statistics 

associated with the operational acceptability ratings are shown in Table 34 and Figure 41. For all 

types of operations, the mean acceptability ratings were statistically greater than ‘5’ (p < 0.0005). 

This indicates that flight crews found the use of voice communications to provide the IM 

clearances to be acceptable. 

One pilot rated the acceptability of the use of voice communications as ‘2,’ commenting that ATC 

informed him that his microphone was breaking up, and that there seemed to be a lot of pilots 

stepping on each other on the radio (simulator limitation due to software-based communication 

system). Four pilots provided ratings of ‘3,’ and reported that there was confusion regarding the 

terminology to suspend IM, and also that cancelling IM should be followed by speed instructions 

from ATC. Additional information is in Appendix F.4, question #13. 

Table 34. Descriptive Statistics for Acceptability of the Use of Voice Communications 

Operation N Mean SD Min Median Max 

BASELINE 48 6.6 0.7 4 7 7 

CAPTURE 46 6.3 1.2 2 7 7 

CROSS 48 6.3 0.9 4 7 7 

MAINTAIN 42 6.1 1.2 3 7 7 

MIXED 48 6.1 1.2 3 7 7 

Baseline Capture Cross Maintain Mixed

7

6

5

4

3

2

Operation

Acceptabililty Ratings

Figure 41. Flight crew acceptability of use of voice communications by IM operation type. 

5.8.4.6 Post-Experiment Acceptability of the Use of Voice Communications 

The flight crew rated the operational acceptability of the IM phraseology (scale of 1 to 7, with 7 

being “Very Clear”) in the post-experiment survey. Complete information is in Appendix F.6, 

question #19, and a synopsis of the comments is below. 
 85 

Table 35. Pilot acceptability of Voice Comm during ATD-1 operations 

IM Phraseology N Mean SD Min Median Max 

CAPTURE clearance 24 6.9 0.3 6 7 7 

CROSS clearance 24 6.9 0.3 6 7 7 

MAINTAIN clearance 24 6.5 1.5 1 7 7 

Reporting “IM Paired” 24 6.9 0.3 6 7 7 

Check in with TRACON 24 6.7 0.7 4 7 7 

Amendments to IM clearance 24 6.5 0.9 4 7 7 

IM clearance Suspension 24 6.0 1.5 2 7 7 

IM clearance Resume 24 6.1 1.3 3 7 7 

IM Status reporting Unable 24 6.0 1.6 2 7 7 

• Off-nominal operations may need further development because things become ‘confusing 

when deviations occur’. 

• In a high density environment with voice communication, issuing and reading back IM 

clearances may create unintended workloads 

• Confusion for pilots regarding whether IM is suspended or canceled when given a speed 

command by ATC but not explicitly stating IM has been canceled or suspended. 

• Cancellation and reissuance of the IM clearance was ‘tedious’ in the voice environment. 

5.8.4.7 Post-Experiment Confusion about Target or Ownship Call Sign 

Four of 24 (16.6%) pilots responded on the post-experiment questionnaire that they experienced 

confusion during the IM clearance instruction about the Target versus Ownship call sign. Another 

three (12.5%) reported not hearing or noticing the issue, and the remaining 17 (70.8%) reported 

they did not experience any confusion throughout the experiment. Complete information is in 

Appendix F.6, question #20. A synopsis of the comments is: 

• “Heard it happen more than once. It took additional read-backs to get fixed.” 

• “Specifically on call signs that I don't frequently hearing.” 

• “It's a minor distraction, but also aids in situation awareness, as you get to hear who is 

following whom.” 

5.8.4.8 Post-Experiment Acceptability of Using Target Call Sign in IM Clearance 

The acceptability of the using the Target’s call sign in the IM clearance was also measured using 

a 7-point rating scale (‘1’ = “Completely Unacceptable” to ‘7’ = “Completely Acceptable”) via 

the electronic post-experiment questionnaire. The mean acceptability ratings were statistically 

greater than ‘5’ (p < 0.0005), indicating the flight crews found the use of the Target call sign in an 

IM clearance to be acceptable. Complete information is in Appendix F.6, question #21. A synopsis 

of the comments is: 

• Training may be necessary to familiarize crews with lesser known airline call signs. 

• Controller should issue the Target call sign using the phonetic or alpha-numeric alphabet. 
 86 

5.8.4.9 Post-Experiment Suggestions to Improve IM Phraseology 

Overall the flight crew reported the phraseology was complete and clear, however would require 

more training and practice to become proficient. Complete information is in Appendix F.6, 

question #22. A synopsis of the comments is: 

• Data link communication is optimal. 

• “United 123" for the usual ATC communications, and “Uniform Alfa Lima 123” for the 

IM Target until we get the clearances by data link. 

• Add a quick reference page on the EFB for call sign and ICAO code. 

• ATC should not have to communicate “report paired”. This should be a required action on 

the part of the pilots. 

• Standardize clearance between CROSS, MAINTAIN, and CAPTURE. Give Target route 

even on MAINTAIN clearance. 

• There is a need for clear delineation between paired/suspend/resume/cancel events that 

both the controller and pilot can understand. 

5.8.4.10 Post-Experiment Comments about Challenges to IM Implementation 

Overall many of the flight crew responded that the IM operation and procedures would be fairly 

easy to implement, however there were several issues highlighted that need refinement prior to 

implementation in current-day operations. Complete information is in Appendix F.6, question #24. 

A synopsis of the comments is: 

• Overall the concept and procedures are straight-forward, and training would be easy and 

effective if included in the simulator. However, implementing IM operations by only 

providing a training bulletin will probably be insufficient. 

o “With proper simulator training I would probably rate this as Very Easy. If training 

were just a bulletin, then it could be slightly to moderately difficult.” 

o “It would require training and several LOFTs [line-oriented flight training events] to 

completely train line crews. Integrated training may be helpful but difficult to do.” 

o “I think this is overall very easy to learn.” 

o “Training would be required including at least one simulator flight.” 

• Issues that need to be refined include impact on VNAV PATH operations when making 

speed changes, especially close to altitude constrained waypoints. 

o “High rates of distractions trying to slow a Boeing 737 with constant speed changes 

close to the airport. It is relative easy if IM is terminated further out from FAF. I think 

there are too many additive conditions and increased task loading with IM procedures. 

Errors may not be caught by crews trying to comply close to airport.” 

o “The IM speed changes near altitude restrictions in the terminal arrival area interfere 

with configuration changes.” 

o “Large speed changes may be difficult to accomplish. Each crew would also need to 

understand the importance of making a "gradual" adjustment, not a really quick 

adjustment when commanded.” 
 87 

5.8.4.11 Post-Experiment Comments about Most and Least Useful IM Operation 

The responses to these questions were wide-ranging, resulted in no overall conclusion, and 

frequently offered competing suggestions. Complete information is in Appendix F.6, question #25 

& #26. A synopsis of the comments is: 

• All operations seemed acceptable to pilots, and the response of most useful IM operation 

was approximately evenly split between the three IM types. 

• All IM operation types were rated as the least useful, however 9 of the 24 responses was 

the MAINTAIN as the least useful. The rationale for the MAINTAIN being the least useful 

varied widely, and in several cases displayed an incorrect understanding of the operation. 

5.8.5 Flight Crew Comments about IM Interface and Displays 

5.8.5.1 Intuitiveness of Entering Information into the EFB 

The flight crew rated the intuitiveness of entering IM clearance information into the EFB (scale of 

1 to 7, with 7 being “Completely Intuitive”) on the post-experiment questionnaire a Mean of 6.7 

(standard deviation = 0.6). Complete information is in Appendix F.6, question #14. A synopsis of 

the comments is: 

• Every EFB page should have a touch screen HOME option. Several pilots reported at times 

being confused on how to reach the HOME page. 

• The ENTER button should appear in same location on every page. 

• This is very easy, requiring minimal training to master. 

None of the pilots that flew the IFD with the constant, light to moderate turbulence, expressed any 

difficulty with data entry of information into the EFB to conduct the IM operation. 

5.8.5.2 The Usefulness of Display Elements on the EFB 

The flight crew rated the usefulness of the EFB display elements in the post-experiment survey, 

and those rated at ‘6’ or higher (on a scale of 1 – 7) are shown below. 

• IM commanded speed: Mean = 6.7 (standard deviation = 0.6) 

• IM status (i.e. ARMED, PAIRED): Mean = 6.5 (standard deviation = 0.7) 

• Ownship route: Mean = 6.1 (standard deviation = 1.6) 

• Target route: Mean = 6.1 (standard deviation = 1.3) 

Complete information is in Appendix F.6, questions #15 & #16. A synopsis of the comments is: 

• EARLY/LATE indicator was considered fairly useless to pilots. 

• FAST/SLOW indicator should be integrated onto the speed tape on the primary display. 

• Target GS/TRK/BRG options within the EFB increased situation awareness and was more 

useful than the EARLY/LATE indicator. 

• Revise the armed/waiting condition messages to make more intuitive. 

• Revise condition messages to either make more sense or else provide a training manual 

with the message description. 
 88 

• Add seconds to the range on the EARLY/LATE indicator (i.e. 45 s). Pilots want some 

indication of what the range values are. 

• Annunciation is desirable when changing from ARMED to AVAILABLE. 

• All information on the EFB should be replicated in the CGD. The EFB is needed for other 

things such as approach plates, airport diagrams, etc. 

• Add functionality to uplink the IM clearance via data link. 

• Filters improved situations awareness and were intuitive to use, but could be better 

presented on the navigation display which already hosts traffic display information 

(elements could be more fully integrated into the aircraft). 

5.8.5.3 The Effectiveness and Usefulness of Display Elements on the CGD 

The mean rating given by the flight crew during the post-experiment questionnaire of the 

effectiveness of the CGD in providing adequate information to conduct an IM operation was 6.3 

(standard deviation = 0.7). Complete information is in Appendix F.6, question #18. Comment 

highlights are: 

• Integrate the CGD into another display, allowing pilots to ‘stow’ display when not in use. 

• EFB commands more attention because it provides more situation awareness. 

• An alert needs to be in place when able to EXECUTE. 

• Pilot recognized it created more heads-down time during critical phases of flight. 

The CGD display elements that had a mean rating by the flight crew in the post-experiment survey 

of ‘6’ or higher included the IM commanded speed, Target aircraft call sign, and the IM status 

(i.e., ARMED, PAIRED). Descriptive statistics for the CGD effectiveness and usefulness of the 

display elements are shown in Table 36. Complete information is in Appendix F.6, questions #17. 

A synopsis of the comments is: 

• Have the FAST/SLOW indicator blink or turn red to indicate a large correction is needed. 

• Incorporate aural alerts for speed changes to reduce head down time. 

• Important for this information to be in pilot’s forward field of view. 

• The FAST/SLOW indicator not very useful. 

• Replicate all messages from the EFB on the CGD. 

Table 36. Descriptive Statistics for CGD Effectiveness and Display Element Usefulness 

Effectiveness of CGD 

Display N Mean SD Min Median Max 

Effectiveness of CGD in providing 

adequate information for IM 24 6.3 0.7 5 6 7 

Usefulness of Display Elements on the CGD 

CGD Display Element N Mean SD Min Median Max 

Target Call sign 24 6.2 1.5 1 7 7 

IM Speed 24 6.9 0.3 6 7 7 

IM Status 24 6.3 1.3 2 7 7 

IM Message 24 5.5 1.7 1 6 7 

FAST/SLOW Indicator 24 5.1 2.1 1 6 7 
 89 

Other than the rating given to the EARLY/LATE indicator on the EFB (mean = 4.1, standard 

deviation = 3.0), the lowest rating given to any display element on either the EFB or CGD was the 

FAST/SLOW indicator. The FAST/SLOW indicator was intended to be a deceleration cue for the 

flight crew, since decelerating too quickly or too slowly will trigger additional IM speed changes. 

The low rating and the high number of speed changes due to poor deceleration is strong evidence 

for the need to improve the design and saliency of the deceleration cue provided to the flight crew. 

5.9 Case Study Results 

5.9.1 Loss of Separation between Aircraft 

A total of 229 flight operations were assessed in the experiment, of which the subject pilots flew 

79 IM operations and 34 non-IM operations, and the confederate pilots flew 116 non-IM 

operations. All operations were analyzed for loss of separation (LOS), using the air traffic control 

separation criteria defined in reference 10, paragraphs 4-5-1 and 5-5-4. 

There were four LOS events: the first one did not involve IM operations, the second and third 

involved IM software and displays causing the flight crew to fly speeds that resulted in a LOS, and 

the fourth one involved the leading aircraft, conducting an IM operation, to unexpectedly (from 

the controller’s point of view) decelerate and cause a LOS with the aircraft behind it. Therefore, 

the LOS frequency was 0.4% (1 of 229) for reasons not related to the IM software and procedures, 

and 1.2% (3 of 229) for reasons due to IM related software and procedures. 

5.9.1.1 Loss of Separation Event #1 

The first LOS event occurred at the FAF to runway 35R between two non-IM equipped aircraft. 

The controller had the lead aircraft on the RNP turn slightly behind the slot marker, while the 

trailing aircraft flying the ILS straight-in was in its slot marker. The wake vortex separation 

requirement for this aircraft pair was four miles in trail, and minimum separation of these aircraft 

at the FAF was 3.85 nmi. This LOS is attributed to the controller’s control technique and not taking 

action to maintain separation; the IM procedures were not a factor. 

5.9.1.2 Loss of Separation Event #2 

The second LOS event was between a Target and an IM aircraft conducting a CROSS operation, 

and two different separation criteria were not met: the 3 nmi criteria prior to both aircraft being 

established on final, and the 2.5 nmi reduced separation criteria once on final. 

The IM aircraft was issued a TBFM calculated ASG of 81 seconds, which was based on 2.8 nmi 

(2.5 nmi minimum separation plus a 0.3 nmi buffer) behind the Target aircraft at the FAF (both 

the ABP and PTP in this case). Prior to final, the minimum distance between the two aircraft was 

2.58 nmi (required minimum 3 nmi as shown in Figure 42), and the distance at the FAF was 2.4 

nmi (assigned minimum 2.9 nmi). Interestingly, the spacing error of the IM aircraft was zero at 37 

nmi to the runway, 33 seconds early at 11 nmi to the runway, and 16 seconds early at the FAF. 
 90 

A review of the data and video shows that the flight crew properly executed the IM procedures, 

that is they promptly set the proper value in the MCP speed window when a new IM commanded 

speed appeared, and the overwhelming majority of the time they were on speed according to the 

FAST/SLOW indicator. The EARLY/LATE indicator did consistently show the aircraft was very 

early, and the IM SPEED LIMITED message was continuously displayed, indicating the IM 

spacing algorithm wanted to fly a speed slower than the 15% bound around the published speed 

for that segment. This was due to the Target aircraft (also conducting an IM operation) being over 

60 seconds early with only 35 nmi to go to the runway, causing it to fly the remainder of the 

approach at the slowest speed that ASTAR would allow. Since the Target aircraft was flying the 

lowest allowable speed, the IM aircraft could not fly even slower to fix the early time error. 

Figure 42. Loss of separation case #2 between IM and Target aircraft. 

Several conclusions are: 

1) The spacing algorithm was intentionally designed to only consider the spacing at a 

specified waypoint, and safe separation criteria is not part of the algorithm’s consideration. 

In certain geometries and wind conditions, this will continue to present an operational 

safety issue, causing controllers to occasionally have to intervene and thereby possibly 

reducing their confidence in the IM operation. 

2) The large changes to the spacing error are attributed to the phase-out of the ground speed 

term prior (see Section 5.9.5). 

3) If the 2.5 nmi separation criteria for aircraft established on a RNP approach was the 

operational standard, the first loss of separation (3 nmi until established on final) would 

not have occurred. To generalize that statement, as RNP approaches become more 

prevalent, if the 2.5 nmi separation criteria is applied to aircraft on any portion of a RNP 

approach, it should simplify the process to ensure aircraft have the required separation. 
 91 

4) The IM aircraft was not within 15 seconds of the assigned spacing goal at the ABP/PTP, 

and the only IM message displayed was IM SPEED LIMITED. The IM software should 

have given the aircrew an indication that the spacing could not be achieved. 

5) The pilots had no indication that separation criteria was projected to be lost or it actually 

had been lost, and therefore continued the IM operation under the assumption that 

everything was progressing as intended. 

5.9.1.3 Loss of Separation Event #3 

The third LOS event occurred between a Target aircraft and an IM aircraft conducting a CROSS 

operation, again while intercepting final, but this time with the Target on the straight in and the 

IM aircraft flying the RNP curved approach (Figure 43). A significant contributing factor to this 

case was that the non-IM aircraft in front of the Target aircraft was slowed by ATC to 180 knots 

at 21 nmi to the runway, and then to 150 knots at 12 nmi to the runway. This meant the Target 

aircraft, also conducting an IM operation, already flying 30 knots slower than the published 

procedure, had to slow to 135 knots at 10 nmi from the runway to correct a 30 second early spacing 

error. Meanwhile, the IM aircraft continued to fly at or above the published speed from 60 nmi to 

25 nmi to the runway, changing the spacing error from 60 seconds late to 15 seconds early. The 

consequence of the IM software reacting slowly to the change in the Target’s airspeed meant the 

spacing error continued to increase, until at 10 nmi to the runway the IM aircraft was over 40 

seconds early and crossed the ABP (the FAF in this example) 37 seconds early. Given that the IM 

aircraft’s ASG was 79 seconds, the IM aircraft crossed the FAF only 42 seconds after the Target 

aircraft, or approximately 1.5 nmi in trail. 

Figure 43. Loss of separation case #3 between IM and Target aircraft. 
 92 

A review of the data and video shows that the flight crew properly executed the IM procedures, 

that is they promptly set the proper value in the MCP speed window when a new IM commanded 

speed appeared, and the overwhelming majority of the time they were on speed according to the 

FAST/SLOW indicator. The EARLY/LATE indicator showed the aircraft essentially on time until 

just prior to beginning the RNP turn to final, at which point the indicator showed the aircraft 

trending early and the IM SPEED LIMITED message was displayed. At no time did the software 

indicate to the crew that the operation was no longer feasible nor did it give any indication that the 

operation was unsafe. The flight crew stated “we need to go-around” as they crossed the FAF, 

however, they did not state their concern to the controller nor did they take any action. 

The primary conclusion from this case is the IM spacing software needs to be able to continuously 

calculate the feasibility of successfully completing the IM operation, and give unambiguous 

indications to the flight crew when it is no longer feasible. 

5.9.1.4 Loss of Separation Event #4 

The fourth LOS event was a non-IM aircraft in trail behind an aircraft conducting an IM operation, 

and the non-IM aircraft closed to within 2.57 nmi when both aircraft were turning onto final (3 

nmi criteria). This LOS was the result of a string of events that caused the leading aircraft 

(conducting the IM operation) to unexpectedly and quite substantially slow from 180 knots to 140 

knots at 14 nmi to the runway. The underlying cause for this change in IM commanded speed of 

the lead aircraft was the phase-out of the ground speed term prior to the ABP (a more detailed 

description is given in Section 5.9.5). Magnifying the issue in this particular event was that the 

aircraft preceding the lead aircraft also decelerated to the final approach speed much more rapidly 

than expected. The combination of the ground speed term phase-out of the lead aircraft, plus the 

rapid deceleration of the preceding aircraft, were the direct causes for this LOS event. 

While technically the controller has the responsibility to maintain separation between all aircraft, 

in this particular case the rapid and unexpected deceleration of the leading aircraft would not have 

happened if it had not been conducting an IM operation. Therefore the IM operation of the lead 

aircraft contributed substantially and directly to the LOS of the trailing non-IM aircraft. 

5.9.2 Undesirable Propagation of Speed Changes 

The IM algorithm performance results described in Section 5.3.2 (Frequency of IM Speed 

Changes) and Section 5.3.3 (Number of IM Speed Reversals) indicate that the CAPTURE 

operation exhibited the largest number of speed changes and the largest number of high-magnitude 

speed reversals, exceeding the values for the MAINTAIN operation even though the same statebased CTD speed control law was used. Since the main difference between the CAPTURE and 

MAINTAN operations is that the MAINTAIN operations begin with zero spacing error, the data 

suggest that the CAPTURE phase of the CTD speed control law caused additional speed changes 

and high magnitude speed reversals. The case study in Figure 44 is an example of how changes in 

speed propagate through a string of aircraft that were conducting CAPTURE operations. 

From left to right in Figure 44, the columns are data for three IM aircraft in their arrival sequence 

(DAL1605, AAL491, and FFT933). The top plot in each column contains colored lines for four 

speeds of that IM aircraft: green is the nominal trajectory airspeed, grey is the discrete IM 

commanded end speed (desired speed calculated by the spacing algorithm for the aircraft to 

decelerate to, and entered by the flight crew into the mode control panel speed window), red is the 
 93 

instantaneous IM speed (current estimated airspeed calculated by the spacing algorithm), and blue 

is the IM aircraft’s actual airspeed. The middle plot shows the call sign and speeds of the Target 

aircraft, with green the nominal trajectory (published) airspeed of the Target aircraft, and blue the 

Target aircraft’s actual airspeed. The bottom plot shows the spacing error in time (blue line 

corresponding to left axis) and the actual distance between the IM and Target aircraft (orange line 

corresponding to right axis). 

The middle plot of the left-most column in Figure 44 shows the speeds of the Target aircraft 

(UAL1082) for the first IM aircraft in the string (DAL1605). The Target aircraft was controlled 

by air traffic controllers using the TBFM schedule and CMS decision support tools. Prior to top of 

descent, the Target aircraft was instructed by ATC to descend 4,000 feet below the altitude shown 

on the published approach procedure to help absorb its required delay. Since the Target aircraft 

was at a lower altitude than the IM aircraft, its ground speed was lower as well. This, combined 

with the fact that the IM aircraft started with a 20 second early spacing error, caused ASTAR to 

command a speed of 250 knots (a very low speed for this phase of flight). Later in the arrival, at 

60 nmi to the runway, the first IM commanded speed increased from 250 knots to 290 knots. There 

were three factors that contributed to this speed increase: first, the lower altitude of the Target 

aircraft relative to the IM aircraft; the Target aircraft was instructed by the controller to increase 

its speed to 300 knots; and the initial 20 second early spacing error was nulled. This large speed 

reversal propagated through the string of aircraft. 

The state-based CTD speed control law in ASTAR calculates the commanded speed by adding the 

amount of speed control required to null the spacing error to the Target aircraft’s time-history 

speed (see Appendix A.3). Thus, the speed reversal propagated to the second IM aircraft (AAL491) 

in the string, shown in the middle column of Figure 44. The second IM aircraft in the string also 

started with a 20 second early spacing error, and again the speed control required to null the spacing 

error was added to the airspeed flown by the previous aircraft in the string, resulting in an even 

larger speed reversal. 

The third IM aircraft (FFT933) in the string also started out with a 20 second early spacing error 

further amplified the speed reversal, since the speed control required to null the spacing error was 

added to the airspeed flown by the second aircraft in the string. 

The main catalyst for the undesirable speed behavior observed in this case study was an altitude 

step down flown by the first Target aircraft that was not flown by the IM aircraft following that 

Target. It should be noted that this particular case study was one of the worst examples of this 

behavior and is not indicative of the typical behavior observed in this experiment or in previous 

research (although previous experiments also had isolated instances of poor IM spacing algorithm 

performance). Nevertheless, these large speed reversals will likely be operationally unacceptable, 

suggesting that additional work is needed to develop either algorithm requirements or procedures 

to prevent Target aircraft altitude step downs from causing large speed reversals when conducting 

CAPTURE or MAINTAIN operations. 
 94 

Figure 44. Non-ideal IM speed reversals of CTD control law during CAPTURE operatio

Note: this figure is read from left to right, and represents the arrival stream of four sequential aircraft: UAL1082 

(IM), AAL491 (IM), and FFT993 (IM). 
 95 

5.9.3 Desirable IM Spacing Algorithm Performance 

The case study in the previous section described the ASTAR13 algorithm exhibiting undesirable 

performance was not representative of the ASTAR13 behavior observed elsewhere in the 

experiment. For an IM spacing algorithm, similar to the one investigated in this experiment, the 

design goals for the spacing algorithm were for the following desirable characteristics to exist: 

• the spacing error should be smoothly and continuously nulled; 

• once the spacing error is close to being nulled it should remain close to zero; 

• the frequency of IM speed changes should be minimized; 

• the number of IM speed increases (reversals) should be minimized; and 

• the IM speed changes should be intuitive to pilots and controllers. 

This section illustrates desired ASTAR13 algorithm performance that occurred in each of the IM 

operation types. Figure 45 contains plots of three case studies where ASTAR13 exhibited the 

performance described above, that is, desirable IM speeds. The leftmost plot is from a CAPTURE 

operation, the middle plot is form a CROSS operation, and the rightmost plot is from a MAINTAIN 

operation. 

In each of the three plots, the black “CmdEnd” line in the top panel indicates the speed commanded 

by the IM spacing software. The frequency of IM speed changes is low, and there are few 

additional speed changes the flight crew must implement for the IM operation compared to the 

typical Baseline (TSAS only) operation. The blue “SpacingError” line in the bottom panel 

smoothly and continuously moves closer to zero, with the exception of the MAINTAIN operation 

where a 5 second spacing error accumulated within the final 17 nmi. 
 96 

 CAPTURE CROSS MAINT

Figure 45. Desired spacing algorithm performance by IM operation type. 

Note: this figure represents three separate and distinct IM operations, one for each IM operation type. 
 97 

5.9.4 Variations to Normal IM Clearances and Operations 

5.9.4.1 Achieve-By Point Dissimilar to the Planned Termination Point 

During the MIXED scenarios, the ARTCC air traffic controllers were given the latitude to choose 

which IM clearances to provide to the IM equipped aircraft. Furthermore, it was intended for some 

of the CROSS clearances to be issued with the ABP different than the PTP (which was to remain 

the FAF). Ideally, the ABP would be the waypoint where the routes of the IM aircraft and the 

Target aircraft merged. There were four instances (Table 37) where the ARTCC controller issued 

a CROSS IM clearance with the ABP prior to the PTP (which remained the FAF). 

The two CROSS clearances issued to aircraft arriving from the same sector correctly used the 

merge waypoint as the ABP (FFFAT for the ANCHR2 and KOHOE2 arrivals, and LONGS for 

the CREDE3 and PEEKK3 arrival), and correctly used the FAF as the PTP (FRONZ and LEETS). 

The other two CROSS clearances were issued to aircraft coming from different sectors (the IM 

aircraft on the PEEKK3, and the Targets on the KAILE2 and TSCHNR2) but they incorrectly used 

an ABP that was only on the IM aircraft’s route; therefore, the change from the TBO to the CTD 

speed control law did not happen until both aircraft were established on final. 

Only two valid IM operations with the ABP at the merge did not provide statistically meaningful 

data. However, it was noticed on three of the four operations that the transition of the IM software 

from the TBO to the CTD speed control law (which occurs once the aircraft are on the same route 

and past the ABP) created a spike in the calculated spacing error, in turn leading to a change in the 

commanded IM speed. Of note, the RTCA MOPS (ref. 4) has been updated since this software 

was written, and the discrete discontinuity between the two algorithms should no longer exist. 

Table 37. IM Clearances with ABP Dissimilar to the PTP 

Group - 

Scenario Simulator IM Call sign IM Route Target Call sign 

Target 

Route ABP PTP Valid

1 - I ASTOR1 AAL491 ANCHR2 CHQ3655 KOHOE2 FFFAT FRONZ Y 

3 - J ASTOR3 FFT933 PEEKK3 DAL1605 KAILE2 LONGS LEETS N 

3 - J ASTOR4 ASQ7044 PEEKK3 UPS702 TSHNR2 LONGS LEETS N 

3 - J IFD SWA3036 CREDE3 FFT933 PEEKK3 LONGS LEETS Y 

5.9.4.2 Issuing MAINTAIN Clearances in the TRACON 

During the MIXED scenarios, the TRACON controllers had the option to issue an IM clearance if 

their workload permitted. Since the TRACON workstations do not contain any IM information 

(i.e., IM equipped aircraft, the corresponding Target aircraft, and the assigned spacing goal), the 

expectation was some controllers may issue a MAINTAIN clearance if the aircraft were properly 

spaced on the same arrival procedure. Although this did not occur, several of the TRACON 

controllers stated that, given more practice and proficiency, they understood the utility of the 

procedure and believed it could be useful in arrival operations. This should be examined in future 

research once the concept of operations for this procedure is fully developed. 
 98 

5.9.5 Impact to ASTAR When Target Aircraft not at Published Airspeed 

5.9.5.1 ASTAR Ground Speed Term and Phase-Out Logic 

During the design of the trajectory based control law in ASTAR13, the decision was made to 

inhibit the ground speed term when the IM aircraft is close to the achieve-by point. The rationale 

for this design decision was the expectation that the Target aircraft would finish absorbing delay 

prior to the ABP and would fly speeds that were close to the published speeds. However, there 

were route designs and TSAS adaptation designs in the IMAC simulation experiment that allow 

the Target aircraft to absorb delay much closer to the ABP than originally anticipated. 

If the Target aircraft is absorbing delay close to the ABP (i.e., not flying speeds close to the 

published speeds), the trajectory-based speed control law will provide commanded speeds that 

cause the spacing error to diverge from zero to a steady state error value. This problem occurs 

because ASTAR only uses proportional control when the ground speed term is turned off. 

Therefore, there must be a certain amount of spacing error present for the IM aircraft to match the 

traffic aircraft’s speed deviation. 

This section begins with analytic descriptions of the steady state error of the ASTAR13 trajectorybased control law with the ground speed term either active or inactive. A case study from this 

experiment is used to show how this problem could impact real-world operations. The case study 

is followed by a discussion on potential methods that could be used to minimize the steady state 

error. 

The ground speed term in the ASTAR13 trajectory-based speed control law is inhibited when the 

IM aircraft is close to the achieve-by point. An analysis of the steady state error was conducted for 

the case where the ground speed term in ASTAR13 is active and the case where the ground speed 

term is inactive. As is often the case with analytical analyses, several simplifying assumptions are 

made within this analysis: 

• the conversion from airspeed to ground speed is not modeled; 

• various filters and heuristics used by ASTAR are not modeled; 

• the IM aircraft’s nominal speed is assumed to be constant; 

• the Target aircraft’s speed and nominal speed are assumed to be constant; and 

• the IM aircraft’s speed (ݒூெ) is equal to the IM commanded speed (ݒ௠௖ௗ) (i.e., the IM 

aircraft flies its commanded speed exactly) 

The ASTAR13 trajectory-based speed control law calculates the spacing error using the time-togo of the IM and Target aircraft along their predicted 4D trajectories. The time-to-go for each 

aircraft is simply the difference between their ETAs at the achieve-by point and the current time. 

The spacing error is then defined as the difference between the IM aircraft’s time-to-go to the ABP 

(ܶܶܩூெ), and the Target aircraft’s time-to-go to the ABP (ܶܶܩ்ீ்(, minus the ASG issued by 

ATC (Δ). 

(1) Δ −) ݐ)்ீ்ܩܶܶ − (ݐ)ூெܩܶܶ = (ݐ)݁
 99 

The amount of speed control required to null the spacing error is computed using a proportional 

control law with a ground speed term added to compensate for differences between the Target 

aircraft’s predicted ground speed and actual ground speed. The speed control is added to the IM 

aircraft’s nominal speed to generate an IM commanded speed (ݒ௠௖ௗ). 

(2) (்ீ்௠௢௡ݒ − ்ீ்ݒ)ௌ݇ீ +) ݐ)݁௣݇ + ಾ௡௢௠಺ݒ = (ݐ)ௗ௖௠ݒ

Here, ݒ௠௢௡ூெ is the nominal 4D trajectory ground speed of the IM aircraft, ݇௣ is the proportional 

gain, ݁(ݐ (is the spacing error, ݇ீௌ is the gain for the ground speed term, ݒ்ீ் is the ground speed 

of the Target aircraft, and ݒ௠௢௡்ீ் is the nominal 4D trajectory ground speed of the Target aircraft. 

The ground speed term is fully active until the IM aircraft is 40 nmi from the achieve-by point, 

and linearly decreases to zero when the IM aircraft is 20 nmi from the achieve-by point. 

Using the assumptions described above, the estimated time-to-go of the IM aircraft and the Target 

aircraft is simply their respective distance-to-go divided by the nominal speed. The spacing error 

and time derivative of the spacing error are described in equations 3 and 4, respectively. 

(ݐ)ூெܩܶܦ = (ݐ)݁

ூெ௡௢௠ݒ

(ݐ)்ீ்ܩܶܦ −

்ீ்௠௢௡ݒ

− Δ (3)

(ݐ)ௗ௖௠ݒ = (ݐ)ሶ݁

ூெ௡௢௠ݒ

்ீ்ݒ −

்ீ்௠௢௡ݒ

(4)

Two cases are examined in the following sub-sections: a case where the ground speed term is fully 

active (݇ீௌ = 1) and a case where the ground speed term is not active (݇ீௌ = 0). For both cases, 

a differential equation for the spacing error is determined, a Laplace transformation is used to 

convert the spacing error to the frequency domain, and the final value theorem is used to determine 

the steady state error that will occur as time approaches infinity. 

5.9.5.1.1 Ground Speed Term Fully Active (ࡿࡳ࢑(૚ = 

The ground speed term in ASTAR13 is fully active when the ground speed gain, ݇ீௌ, is equal to 

one. A differential equation for the spacing error is obtained by combining equations 2 and 4. 

(்ீ்௠௢௡ݒ − ்ீ்ݒ) + (ݐ)݁௣݇ + ಾ௡௢௠಺ݒ = (ݐ)ሶ݁

ூெ௡௢௠ݒ

்ீ்ݒ −

்ீ்௠௢௡ݒ

(5)

௣݇ − (ݐ)ሶ݁

ூெ௡௢௠ݒ

்ீ்௠௢௡ݒ − ்ீ்ݒ = (ݐ)݁

ூெ௡௢௠ݒ

+

்ீ்ݒ − ்ீ்௠௢௡ݒ

்ீ்௠௢௡ݒ

(6)

 
 100 

Next, equation 6 is converted to the frequency domain using a Laplace transformation, and the 

resulting expression is rearranged. 

௣݇ − (ݏ)ܧݏ

ூெ௡௢௠ݒ

்ீ்௠௢௡ݒ − ்ீ்ݒቆ) = ݏ)ܧ

ூெ௡௢௠ݒ

+

்ீ்ݒ − ்ீ்௠௢௡ݒ

்ீ்௠௢௡ݒ

ቇ

1

(7) ݏ

= (ݏ)ܧ

൬

்ீ்௠௢௡ݒ − ்ீ்ݒ

ூெ௡௢௠ݒ

+

்ீ்ݒ − ்ீ்௠௢௡ݒ

்ீ்௠௢௡ݒ

൰

௣݇ − ଶݏ

ூெ௡௢௠ݒ

ݏ

(8)

 

The steady state error is the error that will occur as time approaches infinity, and can be calculated 

using the Final Value Theorem, which states that the steady state error is the lim

 Equation .)ݏ)ܧݏ ଴→௦

9 is the result of applying the Final Value Theorem to equation 8, and equation 10 is a rearranged 

expression of the steady state error. 

݁௦௦ =

൬

்ீ்௠௢௡ݒ − ்ீ்ݒ

ூெ௡௢௠ݒ

+

்ீ்ݒ − ்ீ்௠௢௡ݒ

்ீ்௠௢௡ݒ

൰

− ݇௣

ூெ௡௢௠ݒ

(9)

 

݁௦௦ = 1

݇௣

்ீ்ݒ ቆ

்ீ்௠௢௡ݒ

− 1ቇ ቆ ݒ௠௢௡ூெ

்ீ்௠௢௡ݒ

− 1ቇ (10)

 

Equation 10 can be used to attain intuition on the factors that influence the steady state error when 

the ground speed term is fully active. First, the steady-state error will be equal to zero whenever 

the Target aircraft’s speed is equal to its nominal speed and whenever the nominal speed of the IM 

aircraft is equal to the nominal speed of the Target aircraft. Since the proportional gains in ASTAR 

range from 0.375 to 1.5, the steady state error will be small for all operationally relevant speed 

deviations. As an example, consider the extreme case where the Target aircraft’s speed is twice as 

large as the Target aircraft’s nominal speed and the IM aircraft’s nominal speed is twice as high 

as the Target aircraft’s nominal speed; a case that is not expected to occur in an operational 

environment. In this case, the steady state error will be equal to1/݇௣, indicating that steady state 

error is not problematic when the ground speed term is fully active. 

 
 101 

5.9.5.1.2 Ground Speed Term Not Active (ࡿࡳ࢑(૙ = 

When the IM aircraft is 40 nmi from the achieve-by point, ݇ீௌ linearly decreases from a value of 

one to a value of zero when the IM aircraft is 20 nmi from the achieve-by point. This section 

examines the steady state error for the case where the ground speed term is equal to zero, causing 

ASTAR13 to revert to proportional control. A differential equation for the spacing error is 

calculated by combining equations 2 and 4, and simplifying. 

(ݐ)݁௣݇ + ಾ௡௢௠಺ݒ = (ݐ)ሶ݁

ூெ௡௢௠ݒ

்ீ்ݒ −

்ீ்௠௢௡ݒ

(11)

௣݇ − (ݐ)ሶ݁

ூெ௡௢௠ݒ

்ீ்ݒ − ்ீ்௠௢௡ݒ = (ݐ)݁

்ீ்௠௢௡ݒ

(12)

Next, equation 12 is converted to the frequency domain using a Laplace transformation, and the 

resulting expression is rearranged. 

௣݇ − (ݏ)ܧݏ

ூெ௡௢௠ݒ

்ீ்ݒ − ்ீ்௠௢௡ݒቆ) = ݏ)ܧ

்ீ்௠௢௡ݒ

ቇ

1

(13) ݏ

= (ݏ)ܧ

൬

்ீ்ݒ − ்ீ்௠௢௡ݒ

்ீ்௠௢௡ݒ

൰

௣݇ − ଶݏ

ூெ௡௢௠ݒ

ݏ

(14)

Similarly to the case where the ground speed term was active, the steady state error is calculated 

using the Final Value Theorem, which states that the steady state error is the lim

 Equation .)ݏ)ܧݏ ଴→௦

15 shows the steady state error calculated using the Final Value Theorem, and equation 16 is a 

rearranged expression of the steady state error. 

݁௦௦ =

൬

்ீ்ݒ − ்ீ்௠௢௡ݒ

்ீ்௠௢௡ݒ

൰

− ݇௣

ூெ௡௢௠ݒ

(15)

்ீ்௠௢௡ݒ − ்ீ்ݒቆ = ௦௦݁

݇௣

ቇ ݒ௠௢௡ூெ

்ீ்௠௢௡ݒ

(16)
 102 

Equation 16 can be examined to gain intuition on the factors that influence the steady state error. 

First, the steady state error will be lower when ݇௣ is large and larger when ݇௣ is small. Within 

ASTAR, ݇௣ ranges from a value of 0.375 when the aircraft is far from the achieve-by point to 1.5 

when the aircraft is close to the achieve-by point. When the Target aircraft’s nominal speed is 

equal to the IM aircraft’s nominal speed, ݁௦௦ = ൫ݒ்ீ் − ݒ௠௢௡்ீ்൯/݇௣. Therefore, the steady state 

error will be large when the Target aircraft has a large ground speed deviation and when ݇௣ is 

large. 

5.9.5.2 Example of Ground Speed Phase-Out Causing Negative Impact 

Figure 46 and Figure 47 show a case study from this experiment where large Target aircraft speed 

deviations close to the achieve-by point caused the spacing error to trend toward a large steady 

state error value. The middle plot in Figure 46 shows the Target aircraft’s airspeed (blue line) 

relative to the Target aircraft’s nominal airspeed profile expected by ASTAR (green line). Within 

the last 30 nmi of the IM operation, the Target aircraft flew speeds substantially lower than the 

nominal speed profile. Figure 47 shows that the ground speed compensation (red line) was phased 

out between a distance-to-go of 40 nmi and 20 nmi from the achieve-by point. During this period 

of time, the spacing error increased from a value of zero when the IM aircraft was 36 nmi from the 

runway threshold to -35 seconds when the IM aircraft was 11 nmi from the runway threshold. 

When the IM aircraft had a distance-to-go of 20 nmi to the runway threshold, the IM speed limited 

flag was set, indicating that the speed control was limited because the spacing error was large 

enough to generate speed control greater than 15% of the nominal speed. By this time, the spacing 

error had grown to -29 seconds, and continued to grow until it reached a value of -35 seconds. At 

the very end, the spacing error decreased to -15.7 seconds because the Target aircraft started flying 

a speed that was closer to its nominal speed. 

The values in this case study can be input into equation 16 to determine if the spacing error was 

close to the theoretical steady-state error value when the IM aircraft was 20 nmi from the runway 

threshold. It should be noted that the values are unlikely to match exactly, since ASTAR contains 

several features that were not modeled in the analysis. These features include speed limiting, 

filters, and heuristics to prevent ASTAR from commanding unnecessary speed commands to 

pilots. Furthermore, the dynamic system may not reach its steady-state value, since the inputs are 

constantly changing. 

When the IM aircraft was 20 nmi from the achieve-by point, the proportional gain (݇௣) was 

approximately equal to one, the Target aircraft’s ground speed (ݒ்ீ் (was equal to 209 knots, the 

Target aircraft’s nominal ground speed (ݒ௠௢௡்ீ் (was equal to 248 knots, and the IM aircraft’s 

nominal ground speed (ݒ௠௢௡ூெ) was equal to 250 knots. Plugging these number into equation 16 

gives a steady-state error value of -39 seconds. The IM aircraft had a spacing error as large as-29 

seconds when it was 20 nmi from the runway threshold and had a maximum spacing error of -35 

seconds, suggesting that the data matches the analytic model reasonably closely. 
 103 

Figure 46: Case study of spacing error when Target has a large ground speed deviation. 

Figure 47: Target aircraft ground speed compensation term. 
 104 

5.9.5.3 Possible Alternatives to Current Ground Speed Phase-Out Approach 

During the design of the trajectory based control law in ASTAR13, the decision was made to 

inhibit the ground speed term when the IM aircraft is close to the achieve-by point. When the 

Target aircraft is absorbing delay close to the achieve-by point (i.e., not flying speeds close to the 

published speeds), the trajectory based control law will provide commanded speeds that cause the 

spacing error to diverge from zero to a steady state error value. An analytic analysis was conducted 

to characterize that steady state error when ASTAR’s ground speed term was fully active and when 

it was inhibited. The results of the analytic analysis showed that the steady state error was not 

problematic when the ground speed term was fully active; however, it can be problematic when 

the ground speed term is inhibited. A case study from this human-in-the-loop simulation was used 

to show the impact that the steady state error can have. The remainder of this section discusses 

potential solutions to this problem. 

One potential solution to this problem is to keep the ground speed term fully active until the Target 

aircraft crosses the achieve-by point. However, there are challenges with this approach. First, it is 

unclear what to do with the ground speed term after the Target aircraft crosses the achieve-by 

point. When the Target aircraft crosses the achieve-by point, its crossing time is recorded. The IM 

aircraft will then attempt to cross the achieve-by point Δ seconds after the Target aircraft, where Δ

is the value of the assigned spacing goal. When this occurs, the Target aircraft’s ground speed is 

no longer relevant; however, removing the ground speed term could cause the IM commanded 

speed to increase when the IM aircraft is close to the achieve-by point. Increasing the commanded 

speeds close to the achieve-by point could be detrimental to pilot acceptability, particularly when 

the achieve-by point is the final approach fix. A second problem is that the Target aircraft often 

matches the nominal speed during the deceleration to the final approach fix. As a result, the ground 

speed term could go from a large value to a much smaller value, causing ASTAR to command a 

speed increase just prior to the final approach fix. 

Another possible solution is to derive the Target aircraft’s current airspeed from its altitude, ground 

speed, and the wind forecast, and use that derived airspeed to update the Target aircraft’s nominal 

speed profile. For instance, if the Target aircraft slows early, it could be assumed that the Target 

aircraft will stay at its current speed until its speed intersects the nominal speed profile. Using this 

method, the Target aircraft’s time-to-go calculations would be based off of the now modified 

nominal speed profile, causing the spacing error to change. There are, however, challenges with 

this approach as well. First, the current ASTAR13 trajectory-based control law is continually 

adjusting the commanded speed to null the spacing error. Therefore, assuming that the Target 

aircraft will continue to fly a particular speed may not be a good assumption when the Target 

aircraft is also conducting an IM operation. Additionally, small Target aircraft speed changes could 

cause the spacing error to change significantly if the speed segment that the Target aircraft is on is 

very long. 

Further research and development is needed to determine the best way of eliminating the steady 

state error problem. 

 
 105 

6 Conclusions and Lessons Learned 

This section contains: (1) a high-level summary of the experiment results; (2) a table of results for 

the Performance Goals and Hypotheses; (3) conclusions related to the ATD-1 ConOps, 

Procedures, and Phraseology; (4) conclusions about the IM Cockpit Displays; (5) conclusions 

about the IM Spacing Software; (6) and Lessons Learned through practice, observation, or 

discussions with the subject and confederate participants (or due to lack of sufficient data for 

statistical analysis). 

6.1 General 

In general, the ATD-1 concept and IM procedures demonstrated substantial promise in this 

experiment and met many important performance criteria, but the current instantiation of IM in a 

busy voice environment has several critical issues that must be resolved prior to being 

implemented in real-world operations (described in subsequent sections). 

6.2 Meeting Performance Goals and Hypotheses 

The ATD-1 ConOps, particularly the IM procedure as tested during the IMAC experiment, was 

successful when evaluated against the performance goals and hypotheses described in Table 38. 

Table 38. Summary of Performance Goals and Hypotheses 

# Performance Goals Results 

1 The percentage of IM spacing errors at the ABP within 10 

seconds will be ≥ 95% NO 

2a The inter-arrival spacing error of 68% of IM operations 

following either a TSAS or IM operation will be ≤ 8 seconds

YES (capture, cross) 

NO (maintain, mixed) 

2b The inter-arrival spacing error of 68% of TSAS operations 

following another TSAS or IM operation will be ≤ 12 seconds

YES (capture, mixed) 

NO (TSAS, maintain) 

3 The mean controller acceptability rating of ATD-1 operations 

will be ≥ ‘8’ on a scale of 1 – 10 YES 

4 

The mean controller (workload) rating for Mental Demand, 

Physical Demand, Time Pressure, and Effort will be ≤ ‘5’, and 

for Success and Frustration will be ≤ ‘3’ on a scale of 1 – 7 

YES 

# Hypotheses Results

1 The percentage of uninterrupted PBN operations within the 

TRACON and using ATD-1 tools will be > 70% YES 

2 The rate of IM operations terminated by ATC prior to the 

Planned Termination Point will be < 30% YES 

3 The rate of IM operations terminated by the flight crew prior 

to the Planned Termination Point will be < 30% YES 

4 The mean flight crew acceptability rating of IM operations 

will be ≥ ‘5’ on a scale of 1 – 7 YES 

5 The mean flight crew workload ratings of IM operations will 

be ≤ ‘3’ on a scale of 1 – 10 YES 
 106 

6.3 ATD-1 ConOps, Procedures, and Phraseology 

The IM procedures achieved the ATD-1 ConOps goal of greater aircraft delivery precision at the 

FAF compared to TSAS operations (Table 12). 

For ARTCC controllers, aircraft behavior during IM operations was comparable to non-IM 

operations, and they rated the ATD-1 operations, workload, and phraseology as acceptable. For 

TRACON controllers, non-IM and IM operations exhibited similar performance in terms of 

schedule deviation at the Meter Fix, and they rated the ATD-1 operations, workload, and 

phraseology as acceptable; however, a concern was identified that the MAINTAIN operation as 

conducted in this experiment was generally not suitable and frequently had to be canceled. 

(Sections 5.7.1, 5.7.2, and 5.7.3). 

Those controllers’ qualitative responses are corroborated by the use of speed control alone in the 

TRACON (Section 5.4.2), and the low percentage of IM operations interrupted by controllers (4% 

for CROSS and CAPTURE, Section 5.5.1). Although the arrival scenarios were designed to be at 

the airport’s maximum throughput rate, it is recognized that other wind conditions or aircraft 

distributions would most likely have required the use of vectors and increased the frequency of IM 

operation cancellation. 

For the pilots of IM capable aircraft, the acceptability, workload, and head down time required to 

conduct IM operations was rated as acceptable and comparable to non-IM operations, although the 

higher frequency of speed changes and speed reversals during IM operations was identified as was 

not desirable (Section 5.8). 

A list of issues and conclusions about the ATD-1 ConOps, procedures, and phraseology, and if 

appropriate possible methods to address them, are described below. 

• The interaction of the different operations (TSAS and IM), within the same arrival stream, 

appears to be generally compatible and acceptable. This conclusion is based on the two 

operations having similar schedule deviation at the TRACON Meter Fix (Section 5.5.2.2) 

and the FAF (Section 5.5.2.1), similar values for variation from the CMS slot marker 

(Section 5.4.3), and the low rate of controller canceled IM operations (Section 5.5.1) and 

pilot canceled IM operations (Section 5.6.1). There were no qualitative controller 

comments (Section 5.7.3) or pilot comments (Section 5.8) about this particular finding. 

• The use of voice communication to issue all IM clearance types was rated as acceptable by 

controllers and pilots (Sections 5.6.3, 5.8.4.5, and 5.8.4.6). However, it became very 

challenging when controllers had to quickly issue multiple IM clearances in a voice 

environment during high-density arrival operations. 

o The use of data link is seen as the most viable method to address this issue. The 

ground software should automatically present to the controller on their screen a 

viable IM clearance, which the controller reviews and then can elect to send or not 

send via uplink using a single button push. The flight crew acknowledges the IM 

clearance via a down link message using a button push, loads the clearance via 

button push (reducing probability of error), and notifies ATC the IM operation has 

commenced via a down link with a single button push. 

• Issuing the Target aircraft’s identification using its call sign within the IM clearance 

generally did not cause confusion for the pilots (Sections 5.8.4.7, 5.8.4.8, and 5.8.4.9). In 

the post-experiment survey, controllers and pilots agreed with the philosophy given in 
 107 

training that issuing the Target’s aircraft identification using the phonetic alphabet instead 

of the call sign (for example, Delta Alpha Lima 324 instead of Delta 324) potentially 

provides two benefits: it clearly differentiates the Target aircraft from the aircraft the 

clearance is addressed to, and it gives an unambiguous and precise description of the 

aircraft identifier shown in the cockpit when the call sign is unfamiliar or the call sign is 

not similar to the identifier (for example, BAW is the identifier seen by controllers and 

pilots on their displays, but the call sign for British Airways is Speed Bird). For these two 

reasons, slightly more than half of the flight crew reported that they preferred the phonetic 

identifier be used for the Target aircraft (Section 5.8.4.4). Despite the simulation 

environment intentionally having unusual call signs and similar sounding numbers, none 

of the controllers and only four of the pilots reported any issues with the Target call sign 

being used in the IM clearance, and overwhelmingly the Target call sign was used since 

there was so little confusion. When the pilots did query the controllers, they generally 

responded by using the phonetic spelling of the Target. 

o Controllers should typically use the phonetic identifier of the Target aircraft in the 

IM clearance, while having the discretion to use the Target call sign if appropriate. 

• Issuing the MAINTAIN clearance was relatively easy since it had the least amount of data 

to convey (Section 5.6.3); however, both controllers (Section 5.6.1) and pilots (5.8.1.3) 

rated the acceptability of the operation itself the lowest of any operation since the spacing 

interval calculated by the IM software when the aircraft was at altitude was frequently not 

desirable when the aircraft entered the TRACON or was on final. 

o If the next sector has a very different operational characteristic (such as in this 

experiment where the ARTCC sector did not involve many altitude changes but the 

TRACON Feeder had aircraft descending), one option could be to issue the 

MAINTAIN clearance with the PTP as the last waypoint within that controller’s 

sector. 

• Using VNAV speed (manually dialing the IM speed into the MCP) to fly an IM operations 

results in a somewhat higher workload (Section 5.8.2) and removes some of the altitude 

protection offered in VNAV Path (Section 5.7.4). Anecdotally this contributed to the IM 

operation’s significantly higher percentage of missed altitude constraints as compared to 

non-IM operations (the single mouse-driven computer interface to the ASTOR simulators 

was the primary cause). Furthermore, some of the pilots reported that the additional task of 

maintaining vertical path required additional head-down time. 

o At least two options are available. First, the avionics manufacturer could make 

VNAV speed perform more like VNAV path in terms of deviation from the original 

calculated trajectory. And second, for a longer-term solution the spacing software 

could be integrated into the FMS to eliminate the need for manual entry. 

6.4 IM Cockpit Displays 

In general, the pilots found their cockpit interfaces and displays useful, in particular the IM speed 

on the EFB and especially on the CGD. Although they also reported the amount of head down 

time required to conduct IM was acceptable (Section 5.8.1.9), it was noted that the IM operation 

did frequently require more head down time during critical phases of flight (on final and 

configuring the aircraft for landing). Other cockpit display issues identified by pilots, and potential 

methods to address them, are listed below. 
 108 

• To many pilots, the FAST/SLOW indicator could be confusing and received the lowest 

usefulness rating of all display elements (Section 5.8.5). For example, if the flight crew 

decelerated faster than the spacing algorithm expected and slowed below the IM 

commanded end speed, the FAST/SLOW indicator will display ‘Too Slow’ (aircraft slower 

than the IM instantaneous speed), yet the aircraft’s airspeed may be at or below the IM 

commanded end speed. Consequently, if pilots compared the IM commanded end speed to 

the aircraft’s speed (which many did), they could not understand why the IM equipment 

was telling them they were slow. However, by the end of the experiment, many of the 

subject pilots found the display useful, and their ability to decelerate the aircraft in a manner 

expected by the spacing algorithm substantially improved. For the pilots that were not able 

to incorporate the FAST/SLOW indicator into their decision making process, they had a 

greater challenge decelerating the aircraft (frequently due to overuse of speed brake or 

configuring the aircraft too early), which in turn caused subsequent IM speed increases. 

o The FAST/SLOW indicator should be redesigned to be more intuitive, and one 

method of accomplishing this could be to change the location from the CGD to the 

speed tape on the primary flight display. 

• The EARLY/LATE indicator was problematic for many flight crews and received the 

lowest usefulness rating of any display on the EFB (Table 73). Many considered it useless 

and essentially ignored it through most of the experiment (Sections 5.8.5.2 and 5.9.1.3). 

Unlike the FAST/SLOW indicator (once correctly understood), the EARLY/LATE 

indicator does not provide unambiguous information that enables the flight crew to take 

action. It was difficult to determine when the circle reached the scale limit, and even when 

it was clear that it had reached the limit, no action was required since the limit was not 

related to the infeasibility of successfully completing the IM operation. 

o The EARLY/LATE indicator should include values on the scale, and consideration 

should be given to displaying on that indicator the fore and aft locations of where 

the IM operation is no longer considered feasible, and have accompanying pilot 

procedures to notify ATC that the IM operation is no longer feasible. 

• An issue reported by some pilots was the need to devote attention to watching for a change 

to the IM commanded end speed, which detracted from their overall situation awareness 

(Sections 5.8.3, 5.8.4.3, and 5.8.5.3). If the aircraft decelerated at a slower rate than 

anticipated by the spacing algorithm, then the IM software could command additional 

speed decreases to compensate. 

o Aural alerts could be added as a back-up to the visual cues that the IM speed 

changed, thereby allowing flight crew to devote more attention to duties other than 

watching for a change to the IM commanded end speed. For example, retain the IM 

commanded end speed in steady reverse video when the speed change occurs and 

in flashing video if no action occurs within 10 seconds, but then also trigger a chime 

to sound after 15 seconds of no pilot response. 
 109 

6.5 IM Spacing Software 

In general, the spacing algorithm met the performance metrics defined in the performance goals 

and hypotheses, in particular the primary metric of delivery precision of the CROSS and 

CAPTURE operations. 

• In both the CROSS and CAPTURE scenarios, aircraft conducting IM operations were very 

precise in achieving the ASG (mean of 1 second or less, and standard deviation of 6.2 

seconds or less), (Section 5.3.1). 

• In both the CROSS and CAPTURE scenarios, aircraft conducting IM operations had a 

mean schedule spacing error and standard deviation less than half that of aircraft not 

conducting IM operations (Section 5.4.1). 

In general, the behavior of the IM spacing algorithm rated as acceptable by both controllers and 

pilots. However, listed below are four key IM spacing software issues that need to be resolved 

prior to real-world implementation. 

• Quantitative data and survey data indicated that the IM spacing algorithm commanded a 

higher frequency of speed changes (Section 5.3.2) and more speed reversals (Section 5.3.3) 

as compared to non-IM (TSAS) operations. The highest frequency of IM speed command 

changes (Figure 24) occurred 1) while the aircraft was on final (within twenty nmi of the 

FAF), and 2) when that aircraft used the TBO speed control law to conduct a CROSS 

operation (Section 5.3.2). It was further noted that the ‘capture’ phase of the CTD speed 

control law also appears to have a higher frequency of speed change than the ‘maintain’ 

phase. Pilot comments also indicated that the frequency, magnitude, and reversal of speed 

commands was problematic, and should be improved prior to IM being deployed into realworld operations (Sections 5.8.1.1, 5.8.1.3., 5.8.1.4, 5.8.1.6, 5.8.1.7, and 5.8.1.8). 

Furthermore, commanding a second IM speed prior the first speed being set in the MCP 

speed window is not ideal, particularly when configuring the aircraft for landing (Section 

5.6.2; 1% of IM speed changes above 11,000’ MSL, but 10% below 11,000’ MSL). The 

high frequency of speed changes, especially when close to the FAF when configuring the 

aircraft for landing, was also cited as an issue by the pilots in their survey responses 

(Section 5.8.4.3) 

o The consequences of these issues were slightly increased pilot workload, reduced 

fuel efficiency, a concern for passenger comfort, and the pilot’s trust in IM. 

o Causes included: lack of shared aircraft intent data, controllers issue speed change 

instructions when needed and not at published waypoints (as the trajectory 

generator used by ASTAR expects), the phase out of the ground speed term prior 

to the ABP (see next bullet), the methodology used to calculate the Target aircraft’s 

trajectory (the Target’s expected published speed versus its actual speed) was 

problematic, the methodology used by the IM software to resolve the spacing error 

is different than the ground software, and some flight crews had difficulty 

decelerating the aircraft in a fashion expected by the spacing algorithm (triggering 

further speed changes to compensate for the increase in spacing error). 

• During CROSS operations and when the IM aircraft was within 40 nmi of the ABP, the 

ground speed term in the TBO speed control law was phased out. As a result, ASTAR 

controlled the aircraft to a large steady state error value (Section 5.9.5). This behavior 

resulted in less than ideal speed commands that included very slow speed commands on 
 110 

final approach and speed reversals close to the ABP. Phasing out the ground speed term 

was also a contributing factor to one of the loss of separation events (Section 5.9.1). 

• During CAPTURE operations when air traffic controllers instructed the Target aircraft to 

descend to an altitude lower than the published procedure (to absorb required delay), the 

resulting lower Target ground speed caused the IM software to command a speed decrease 

for the IM aircraft. When the Target aircraft later rejoined the altitude profile defined by 

the published procedure, the Target ground speed differential no longer exists, causing a 

large reversal of the IM commanded end speed (Section A.3.2). 

o The consequences of using the CTD speed control law (where the Target aircraft’s 

estimated airspeed is the base speed, and the speed control is added to the base 

speed) was that large speed reversals propagate through the arrival stream. 

• Flight crews conducting IM operations using the VNAV speed mode (manually setting the 

controller issued or IM commanded speed in the MCP) had a more difficult time meeting 

the high end of an altitude constrained waypoint when a slower speed was issued to the 

flight crew within 8 nmi of that waypoint (Section 5.7.4 and 5.8.1.4). 

o The consequences was a higher frequency of missed altitude constraints for IM 

operations when compared to non-IM operations (4.4% versus 0.5%). 

o Causes include: the challenges ASTOR pilots had time sharing a mouse-driven 

interface to manipulate the desktop simulator; the VNAV speed mode on many 

aircraft uses pitch to control airspeed, in turn causing the aircraft to not meet the 

high side of an altitude constrained waypoint when decelerating; the IM operation 

requires the flight crew to use the VNAV speed mode for a longer duration 

compared to current day operations; and the IM operation has a higher speed change 

frequency compared to current day operations. 

6.6 Lessons Learned 

The following items are derived from lessons learned and best practices the IM research team 

developed over the course of this experiment and from discussions with the controllers and pilots. 

Although there is only anecdotal evidence for these issues, the information in this sub-section does 

provide additional information about this research experiment. 

ATD-1 ConOps, Procedures, and Phraseology 

• Some pilots became confused when a long delay occurred between entering the IM 

clearance and the IM operation commencing (Section 5.8.1.3). This occurred when one of 

the criteria to commence the IM operation had not been met, for example, being outside of 

the Target aircraft’s ADS-B range, being in the Mach segment of the arrival procedure, etc. 

o The consequence of this delay between data entry and IM operation start was that 

some pilots expressed low confidence in the IM concept and software. 

o Recommendations for future research to remove the delay include: 

 The flight crew enters Ownship Mach speed (as in previous IM research), 

and the software estimates the Target’s Mach speed. 

 Controllers issue the IM clearance after all criteria has been met (previously 

considered but not implemented based on the FAA’s modernization plan). 
 111 

 If the IM aircraft is not within ADS-B range of the Target aircraft, the 

clearance should include the Target aircraft’s scheduled time of arrival. 

• The design of the arrival procedure should support the use of speed as a method of timing 

control (speed constraints in the middle of the aerodynamic range of the stream class for 

that altitude to allow speed increases and decreases), and the magnitude of the speed 

decrease should be no larger than approximately 30 knots (to preclude the flight crew from 

assuming there is a need to aggressively slow the aircraft). 

• A few controllers stated that future research should explore the ramifications of issuing a 

clearance for an operation that continues into another controller’s sector, in particular, the 

CROSS operation whether the IM aircraft is in the PAIRED mode or not. 

• A few controllers made the following verbal statements about the IM phraseology: 

o The CAPTURE and MAINTAIN clearances may be short enough not to require a 

preparatory instruction. 

o The “REPORT PAIRED” should be included in the controller’s acknowledgement 

of the flight crew’s read back, and not as part of the initial clearance. 

Pilot and Controller Displays 

• The pilots flying the IFD simulator did not report any issues with entering the Ownship 

and IM clearance information into the EFB while in conditions of light to moderate 

turbulence. 

• The ARTCC workstation should display only the specific IM operation deemed to be most 

suitable at one time, and only those elements needed for that clearance should be displayed. 

• The TRACON workstation should indicate which aircraft are IM capable in its data tag 

and whether or not an IM clearance has been issued to that aircraft or not. 

IM Spacing Software 

• When flight crews decelerated the aircraft at a rate faster than expected by the IM software, 

it was frequently in response to large changes to the IM commanded speed (>30 knots on 

arrival and >20 knots on approach). In these situations the pilots extended the speed brake 

more than required, or lowered the gear early to facilitate rapidly achieving the new speed. 

o Mitigating this issue was the genesis of the FAST/SLOW indicator in previous IM 

research. 

o The consequence of decelerating the aircraft faster than the algorithm’s prediction 

is an increase in spacing error, in turn causing an IM speed increase (reversal). 

o A possible cause is hypothesized to be the subject pilots’ instinctual assumption 

that a large speed change is indicative that something is not going as planned, 

therefore an immediate and substantial reaction is required. 

• When flight crews decelerated the aircraft at a rate slower than expected by the IM 

software, it was typically inattention by the flight crew (long response time to set the new 

IM commanded speed in the MCP) or insufficient use of speed brake (not achieving the 

new IM commanded speed in a timely fashion). 

o Mitigating this issue was the genesis of the FAST/SLOW indicator in previous IM 

research. 
 112 

o The consequence of decelerating the aircraft slower than the algorithm’s prediction 

is that it requires subsequent changes of even slower IM commanded speeds. 

• The FAST/SLOW indicator behaved differently for speed changes at a published waypoint 

with a speed constraint versus a deceleration commanded by the IM software. This 

difference in methodology to calculate the FAST/SLOW value was covered in training. 

o The consequence was that the pilots became confused, and believed either the 

display or the calculations by the software were incorrect. 

o The cause is due to the spacing algorithm not having a look-ahead function enabling 

it to predict the next speed change. 

• The “IM Speed Limited” message was not well understood by many pilots. 

o The consequence was that most pilots typically ignored the message since there 

was no action to take, and it did not seem to add to the pilots overall situation 

awareness. 

Simulators 

• The ASTOR simulator provides great capability and utility, but there are a few challenges 

that need to be addressed in future hardware and software upgrades. In particular, the single 

mouse interface to operate the aircraft and communicate with the controller causes a slight 

delay in response to IM commanded speeds (Section 5.7.2), and a significant increase in 

missed altitude constraints (Section 5.7.4) and unstable approaches (Section 5.7.5). 

 
 113 

7 Recommendations for Future Research 

This section contains areas of research and development that should occur prior to real-world 

implementation of the ATD-1 ConOps, or that would benefit the IM procedures and operations. 

Concept of Operations and Procedures 

• Develop an alternative concept where the controller issues a two-part clearance to the IM 

aircraft that includes a STA to the waypoint that the IM and Target merge at, then 

CAPTURE the ASG behind the Target aircraft. This would require an advanced FMS if 

the merge point occurs while the aircraft are descending. 

• Explore the impact of setting the ABP as the waypoint where the Target and Ownship 

routes merge, in particular, if this approach is more robust to ensuring desired separation 

between aircraft when on the same route. 

• Safe separation between the IM and Target aircraft should be considered by the spacing 

algorithm. Furthermore, preserving safe separation with the aircraft behind the aircraft 

conducting the IM operation should also be considered. 

• Issuing a MAINTAIN clearance (the shortest clearance and most intuitive operation) in the 

TRACON should be explored. This would also require development of the IM cockpit 

interface to minimize the number of button presses to display only those aircraft that meet 

the in-trail criteria for the operation (to minimize head down time). 

• Develop an integrated avionics solution (locating the spacing software within the FMS) to 

enable the use of VNAV Path and auto-throttles. An issue previously identified with this 

approach that must also be resolved is how to keep the flight crew in the decision making 

cycle. 

• The use of data link to issue the IM clearance should reduce workload, reduce the 

probability of data entry error, and allow for more complex operations. 

Controller and Cockpit Displays 

• The controller decision support tools should identify when the ASG is not valid throughout 

the entire arrival operation, and either adjust that value appropriately or preclude issuing 

an IM clearance altogether in those situations. 

• Provide more IM information on ARTCC and TRACON controller workstations to reduce 

the voice communication required and increase the opportunity for conducting IM 

operations. Areas to explore include: 

o For ARTCC workstations: 

 Only display the one IM clearance most suitable for that aircraft geometry 

 Only display the clearance when it is feasible (within ADS-B range, etc.) 

 Only display the data elements relevant to that particular IM clearance 

o For TRACON workstations: 

 Display element in the aircraft data tag indicating which aircraft are IM 

capable and what the IM status is 
 114 

IM Spacing Software 

• Explore the benefit of providing the IM aircraft with improved wind information and more 

accurate route information (i.e., a trajectory that includes the delay calculated by TBFM). 

Problems that currently degrade the accuracy of the calculation by the airborne spacing 

software include: 

o Not knowing TBFM’s calculated delay for the IM aircraft and Target aircraft means 

there can be a large difference between what the spacing algorithm expects (the 

speeds on the published procedure) and what the aircraft actually flies. 

o Lack of knowledge about the wind field of the Target aircraft, especially when the 

Ownship aircraft is not in-trail with the Target aircraft. 

• How the spacing algorithm resolves the spacing error should be explored. Options include: 

o Align the methodology closer to the technique used by the ground system scheduler 

(apportion as much delay to the Final sector, any remaining delay to the Feeder 

sector, then any remaining delay to the ARTCC) 

o Resolve the spacing error more strategically than the proportional algorithms that 

have been investigated to date. For example, a spacing algorithm could be 

developed that uses optimal control techniques to create plan a new speed profile 

to the ABP that resolves the spacing error. 

 
 115 

References . 

1. Baxley, B., et al., Air Traffic Management Technology Demonstration-1 Concept of Operations 

(ATD-1 ConOps), Version 3.0, NASA/TM-2016-219213, June 2016 

2. RTCA, Minimum Operational Performance Standards (MOPS) for Aircraft Surveillance Applications 

(ASA) System, DO-317B, June 2014 

3. RTCA, Safety, Performance and Interoperability Requirements Document for Airborne Spacing – 

Flight Deck Interval Management (ASPA-FIM), DO-328, June 2011 

4. RTCA, Minimum Operational Performance Standards (MOPS) for Flight-deck Interval Management 

(FIM), DO-361, Sept 2015 

5. Ging, A., Wang, E., ATD-1 Flight Deck Interval Management (FIM) System Requirements Document 

(SRD) v3.1, https://teams.share.nasa.gov/arc/avsys/nextgen/aegis/default.aspx, NASA Ames Research 

Center, August 2015 

6. Baxley, B., et al., Cockpit Interfaces, Displays, and Alerting Messages for the Interval Management 

Clearances (IMAC) Experiment, NASA/TM-2015-218775, July 2015 

7. Hicok, D., and Barmore, B., Concept of Operations for Interval Management Arrivals and Approach, 

AIAA-2016-1609, AIAA GNC Conference, San Diego, CA, January 2016 

8. Federal Aviation Administration, FAA Aerospace Forecast, Fiscal Years 2015-2035

9. H.R.2115, Vision 100 – Century of Aviation Reauthorization Act, 

https://www.congress.gov/bill/108th-congress/house-bill/2115, December 2003 

10. Federal Aviation Administration, JO 7110.65W, Air Traffic Control, December 2015 

11. Baxley, B., et. al., Experiment Description and Results for Arrival Operations Using Interval 

Management with Spacing to Parallel Dependent Runways (IMSPiDR), NASA/TP 2013-217998, June 

2013 

12. Abbott, T., An Overview of a Trajectory-Based Solution for En Route and Terminal Area SelfSpacing: Fifth Revision, NASA/CR 2015-218680, Febuary 2015 

13. Abbott, T., An Overview of a Trajectory-Based Solution for En Route and Terminal Area SelfSpacing: Seventh Revision, NASA/CR 2015-218794, August 2015 

14. Williams, D., Time-Based Self-Spacing Techniques Using Cockpit Display of Traffic Information 

during Approach to Landing in a Terminal Area Vectoring Environment, NASA/TM 84601, 1983 

15. Prevot, T., et al., An Overview of Current Capabilities and Research Activities in the Airspace 

Operations Laboratory at NASA Ames Research Center, AIAA-2014-2860, AIAA ATIO Conference, 

Atlanta, GA, June 2014 

16. Swenson, H., et al., Design and Evaluation of the Terminal Area Precision Scheduling and Spacing 

System, 9th USA/Europe ATM Seminar (ATM2011), Berlin, Germany, June 2011 

17. Bryant, R. Gupton, L., Martinez, D., and Carrelli, D., Mitigating Motion Base Safety Issues - The NASA 

LaRC CMF Implementation, AIAA-2005-6107, San Francisco, CA, August 2005 

18. Dean, A., and Voss, D., Design and Analysis of Experiments, Springer-Verlag New York, Inc., New 

York, NY., 1999 

19. Johnson, M., and Ging, A., ATD-1 Measures of Performance Specification (MOPS), Rev D, 

https://teams.share.nasa.gov/arc/avsys/nextgen/aegis/default.aspx, NASA Ames Research Center, 

October 2014 
 116 

20. Robinson, J., Thipphavong, J., and Johnson, W., Enabling Performance-Based Navigation Arrivals: 

Development and Simulation Testing of the Terminal Sequencing and Spacing System, ATM2015, 

Lisbon, Portugal, June 2015 

21. Lee, K. K., Kerns, K., Bone, R., Nickelson, M. Development and Validation of the Controller 

Acceptance Rating Scale (CARS): Results of Empirical Research, 4th USA/Europe Air Traffic 

Management R&D Seminar, Santa Fe, NM, December 3-7, 2001 

22. Wierwille, W., and Casali, J., A valid rating scale for global mental workload measurement. In 

Proceedings of the Human Factors Society 27th Annual Meeting, Norfolk, VA., 1983 

23. Federal Aviation Administration, D01 TRACON 7110.K, Terminal Air Traffic Control Positions of 

Peration and Standard Operaiton Procedures (SOP), February 2014 

24. Swieringa, K., Underwood, M., Barmore, B., and Leonard, R., An Evaluation of a flight Deck Interval 

Management Algorithm including Delayed Target Trajectories, AIAA-2014-3148, AIAA ATIO 

Conference, Atlanta, GA, June 2014 

25. Swieringa, K., The String Stability of a Trajectory-Based Interval Management Algorithm in the 

Midterm Airspace, AIAA-2015-2278, AIAA ATIO Conference, Dallas, TX, June 2015 

26. Swieringa, K., Wilson, S., and Shay, R., An Evaluation of Retrofit Flight Deck Displays for Interval 

Management, AIAA-2014-2023, AIAA ATIO Conference, Atlanta, GA, June 2014 

27. Hollander, M., and Wolfe, D., Nonparametric Statistical Methods. John Wiley & Sons, New York, NY, 

2nd edition, 1999 

28. Witzberger, et al., NextGen Technologies on the FAA's Standard Terminal Automation Replacement 

System, IEEE/AIAA DASC2014, Colorado Springs, CO, October 2014 

29. Thipphavong, et al., Evaluation of the Terminal Sequencing and Spacing System for PerformanceBased Navigation Arrivals, IEEE/AIAA DASC2013, Syracuse, NY, Oct 2013 

30. Jung, et al., Assessing Relation between Performance of Schedule-Based Arrival Operation and 

Schedule Nonconformance, AIAA Aviation2014, Atlanta, GA, June 2014 

31. Callantine, et al., System-Level Performance Evaluation of ATD-1 Ground-Based Technologies, AIAA 

Aviation2014, Atlanta, GA, June 2014 

32. Martin, et al., Flight Deck Crew Experiences Flying Profile Descents During Metering Operations, 5th

Applied Human Factors and Ergonomics Conf., Krakow, Poland, July 2014 

33. Wilson, et al., ATD-1 Research and Procedural Testing of Routes, NASA/TM-2015-218707, May 2015 

34. Federal Aviation Administration, FAA-H-8083-3A, Airplane Flying Handbook, 2004 

35. LimeSurvey, https://www.limesurvey.org/ 

36. Baxley, B., et al., Operational Concept for Flight Crews to Participate in Merging and Spacing of Aircraft, 

AIAA-2006-7722, AIAA ATIO Conference, Wichita, KS, Sept 2006 
Appendix A: IM Software 117 

Appendix A: IM Software 

A.1 IM Software Components 

The IM system consists of the following IM software and IM hardware components shown in 

Figure 48. All the components shown, except for the 4D trajectory and the dynamically updated 

data, are discussed in Appendix A, and the IM displays and how the flight crew enter the IM 

information into the IM system are discussed in Appendix B. 

Figure 48. Relationship of IM software, ASTAR algorithm, and speed control laws. 

A.2 ASTAR Algorithm 

A.2.1 ASTAR Overview and History 

The basic goal of an airborne spacing algorithm is to provide an airspeed to the flight crew, which 

if flown, nulls the spacing error. Research in the mid-1980s explored constant time delay (or timehistory) techniques, where the spacing error is calculated by determining the time elapsed between 
Appendix A: IM Software 118 

when the Target aircraft crossed a specific point and when the IM aircraft crossed that same point. 

The spacing error at any given point is simply the difference between the elapsed time and the 

assigned spacing goal. The IM speed is a summation of the Target aircraft’s current speed and the 

speed needed to null the current spacing error, and can only be calculated when both aircraft are 

on the same route (in-trail with each other). 

In the early 2000’s, trajectory-based techniques were developed, where the spacing algorithms 

calculate the ETA for each aircraft at the achieve-by point (ABP) and then compare the difference 

in ETAs to the assigned spacing goal to determine the current spacing error. The IM speed is then 

defined as the IM aircraft’s expected speed on that segment plus the speed compensation used to 

null the current spacing error. This type of algorithm relaxes the requirement of the aircraft being 

in-trail, but requires additional Target and IM aircraft route information to calculate the trajectory 

for both aircraft from their current positions to a common ABP. 

The NASA-developed Airborne Spacing for Terminal Arrival Routes (ASTAR) uses detailed route 

information for both aircraft to allow spacing to begin at any time the Target aircraft’s route can 

be communicated to the IM aircraft. This allows for multiple turns, planned altitude changes, and 

planned speed changes prior to the common point, and for a much larger range between aircraft at 

the start of the operation. In a mature Next Generation Air Transportation System (NextGen) 

environment, the Target aircraft’s route information would be delivered by a data link message 

from air traffic control. In the interim, however, published RNAV arrival routes and instrument 

approaches can provide sufficiently accurate information for airborne spacing. Since the spacing 

algorithm is continually running and providing up-to-date speed guidance, any trajectory 

prediction errors will eventually appear as spacing errors and are corrected. Previous human-inthe-loop simulations have demonstrated that the ASTAR algorithm is able to precisely deliver 

aircraft to the ABP and that the speeds produced by the algorithm are generally acceptable to pilots 

(ref. 11). Each of these human-in-the-loop simulations assumed an advanced airspace environment 

with controller-pilot data link communications used to transmit IM clearances to the flight deck, 

and that the IM aircraft will have access to detailed information of the Target aircraft’s intended 

trajectory. 

When the ATD-1 project began, the focus of IM research at NASA switched from a future 

environment that included controller-pilot data link communications to the use of IM in the 

midterm airspace environment. Since controller-pilot data link communications are not expected 

to be available in the midterm National Airspace System, IM clearances are provided using voice 

communications and the intended trajectories of the IM and Target aircraft are assumed to be 

published Standard Terminal Arrival Routes (STARs). In preparation for the ATD-1 flight 

demonstration, several simulations were conducted to examine the integration of IM with TMATM and CMS. 

An earlier version of the ASTAR algorithm, ASTAR11, did not perform well when used with 

TMA-TM and the CMS tools. TMA-TM uses the predicted trajectory of each aircraft along their 

projected OPDs to compute their ETAs to a series of scheduling waypoints. If there is a conflict at 

one of the scheduling waypoints, TMA-TM often delays aircraft to resolve the conflict. With the 

advent of flex scheduling, aircraft can also be advanced in certain circumstances. Since ASTAR 

uses the published STARs as the estimate of the Target aircraft’s intended trajectory, the speeds 

expected by ASTAR and TMA-TM do not always match. The ASTAR11 algorithm was not 

designed to compensate for large speed differences between the Target aircraft’s actual speed and 

the published speeds that were used by ASTAR11 to predict the Target aircraft’s ETA. The result 
Appendix A: IM Software 119 

was that the ASTAR11 algorithm exhibited a large steady state error and undesirable closure rates 

with the Target aircraft when it was absorbing delay (an example of this type of behavior is 

described in Section 5.9.5). 

In 2013, NASA’s ASTAR algorithm was updated to mitigate the previously described problems 

and improve compatibility with TMA-TM and CMS; this version of ASTAR was called 

ASTAR12. The main modification was a ground speed term that was added to the ASTAR 

algorithm to compensate for discrepancies between the Target aircraft’s actual speeds and 

published speeds. The ground speed term essentially enables the IM aircraft to match the Target 

aircraft’s speed deviation and then correct for the spacing error using the proportional control term. 

The ground speed term also prevents steady-state errors from occurring when the Target aircraft 

is not flying its expected speed, reducing undesirable closure rates between the IM and Target 

aircraft. Several batch simulations were conducted to investigate the performance of the ASTAR12 

algorithm with this new ground speed term functionality (ref. 24 and 25). 

A.2.2 ASTAR13 in the IMAC Experiment 

In 2015, NASA’s ASTAR algorithm was updated to support new IM operations that are described 

in the IM industry standards (ref. 3 and ref. 4). These standards define five different IM operation 

types: Capture then Maintain (CAPTURE), Achieve-by then Maintain (CROSS), Maintain Current 

Spacing (MAINTAIN), Final Approach Spacing (SPACE), and IM Turn (TURN). Prior versions 

of ASTAR only supported the Achieve-by portion of the CROSS operation and did not support 

the other IM operations, whereas ASTAR13 supports all of these operation types except for IM 

Turn. In order to support the additional operation types, a new state-based Constant Time Delay 

(CTD) speed control law was added to ASTAR. The trajectory-based speed control law used in 

ASTAR12 is also used in ASTAR13. 

An algorithmic description of each of the three IM operations used in this experiment is as follows: 

• Achieve-by then Maintain (CROSS): 

o The Ownship and Target aircraft can be on the same or different routes. 

o ATC assigns a specific spacing interval in either time or distance. 

o The TBO control law is used until the Ownship aircraft crosses the ABP, after 

which the CTD control law is used until the PTP. 

• Capture then Maintain Spacing (CAPTURE): 

o The Ownship and Target aircraft must be on the same route. 

o ATC assigns a specific spacing interval in either time or distance. 

o The CTD control law is used to achieve the ATC assigned spacing interval. 

• Maintain Current Spacing (MAINTAIN): 

o The Ownship and Target aircraft must be on the same route. 

o When the flight crew initiates the operation, the IM avionics measures the current 

spacing interval and uses that value as the ASG. 

o The CTD control law is used to maintain the avionics measured spacing interval. 
Appendix A: IM Software 120 

A.3 Speed Control Laws 

A.3.1 TBO Speed Control Law 

The TBO speed control law in ASTAR13 is designed to support IM operations both when the IM 

and Target aircraft are in-trail and when they are on merging routes. The spacing error is calculated 

using the time-to-go of the IM and Target aircraft along their predicted 4D trajectories. The timeto-go of each aircraft is simply the difference between their respective ETAs at the ABP and the 

current time. For time-based operations4, the spacing error is defined as the difference between the 

IM aircraft’s time-to-go (ܶܶܩூெ) to the ABP and the Target aircraft’s time-to-go to the ABP 

(ܶܶܩ்ீ் (minus the spacing goal assigned by air traffic control (Δ). 

Δ −) ݐ)்ீ்ܩܶܶ − (ݐ)ூெܩܶܶ = (ݐ)݁

The amount of speed control required to null the spacing error is computed using a proportional 

controller with a ground speed term added to compensate for differences between the Target 

aircraft’s predicted ground speed and actual ground speed. The speed control is added to the IM 

aircraft’s nominal speed to generate the IM commanded speed (ݒ௠௖ௗ(ݐ((. 

ቁ)ݐ)೅ಸ௡௢௠೅ݒ − (ݐ)೅ಸ௘௥೅ௗݒቀ ௌ݇ீ +) ݐ)݁௣݇ + (ݐ)ூெ௡௢௠ݒ = (ݐ)ௗ௖௠ݒ

Here, ݒ௠௢௡ூெ(ݐ (is the nominal 4D-trajectory airspeed of the IM aircraft; ݇௣ is the proportional 

gain, ݁(ݐ (is the spacing error; ݇ீௌ is the gain for the ground speed term; ݒௗ௘௥೅ಸ೅(ݐ (is the airspeed 

derived from the Target aircraft’s ground speed, wind forecast, and altitude; and ݒ೅௠௢௡ಸ೅(ݐ (is the 

nominal 4D-trajectory airspeed of the Target aircraft. A graphical depiction of the ASTAR13 TBO 

speed control law is shown in Figure 49. 

The IM avionics used in the ATD-1 flight demonstration will likely be a retrofit implementation 

that is not connected with the aircraft’s flight management system, requiring the flight crew to 

monitor for IM speed changes and manually enter them into the aircraft’s mode control panel speed 

window in order to close the control loop. To reduce the number of speed changes that the flight 

crew is required to respond to, speed commands are discretized into either five or ten knot 

increments prior to being displayed to the flight crew. 

Gain scheduling is implemented to ensure that the appropriate amount of speed control is used 

throughout the arrival. The proportional gain ranges from a value of 0.375 when the IM aircraft is 

far from the ABP to 1.5 when the IM aircraft is within 5 nmi of the ABP. The ground speed gain 

is set to one until the IM aircraft is 40 nmi from the ABP. Between 40 nmi and 20 nmi from the 

ABP, the ground speed gain is linearly decreased to zero, and remains at zero until the ABP. 

Additionally, the ground speed gain is set to zero whenever the Target aircraft’s ground speed is 

faster than expected. This is done to reduce the number of instances where the TBO speed control 

law commands a speed increase during an arrival and to increase conformance with the controller 

tools, which primarily resolve conflicts at meter points by delaying aircraft. 

 

4

 Since all of the IM clearances evaluated in the IMAC human-in-the-loop simulation were time-based, the discussion 

of the TBO and CTD speed control laws is limited to time-based operations. 
Appendix A: IM Software 121 

Additional filtering is applied to the Target aircraft’s ground speed differential to prevent 

undesirable speed changes from occurring. The Target aircraft’s ground speed is filtered using a 

first order low pass filter to remove high frequency variability. The time constant of the filter 

changes from 60 seconds when the IM aircraft is more than 30 nmi from the ABP to 30 seconds 

when the IM aircraft is at the ABP. 

To prevent the TBO speed control law from commanding unacceptable speeds, the commanded 

speed is limited to be within ±15% of the TBO nominal profile speed, which is derived from the 

STAR that the IM aircraft is on. Furthermore, the commanded speed is limited to conform to any 

airspace speed restrictions. 

Figure 49. Diagram of the TBO speed control law. 

A.3.2 CTD Speed Control Law 

The state-based CTD speed control law was added to ASTAR to support the maintain phase of the 

CROSS operation, the CAPTURE operation, and the MAINTAIN operation. Unlike the TBO 

speed control law, the CTD speed control law can only be used when the IM and Target aircraft 

are on the same route. 

For time-based operations, the Measured Spacing Interval (ܯܵܫ (is calculated by determining the 

difference between the IM and Target aircrafts’ actual times of arrival at a particular point along 

their common path. The spacing error at any given point is simply the difference between the MSI 

and the ASG. 

Δ − ܫܵܯ = (ݐ)݁

The amount of speed control required to null the spacing error is computed using a proportional 

controller, and the Target aircraft’s estimated time-history airspeed. The Target aircraft’s timehistory airspeed is estimated using the IM aircraft‘s sensed winds, the ground speed, and the 

altitude of the Target aircraft when it was at the IM aircraft’s current along path position. The 
Appendix A: IM Software 122 

speed control is added to the Target aircraft’s derived time-history airspeed to generate the IM 

commanded speed. 

(ݐ)݁௣݇ + (ܫܵܯ − ݐ)೅ಸ௘௥೅ௗݒ = ௗ௖௠ݒ

Here, ݒௗ௘௥೅ಸ೅(ݐ − ܯܵܫ (is the Target aircraft’s estimated time-history airspeed, ݇௣ is the 

proportional gain, and ݁(ݐ (is the spacing error. Similarly to the TBO speed control law, the pilots 

are required to manually enter the commanded speeds into their aircraft’s mode control panel to 

close the control loop. Additionally, the speed commands are discretized into either five or ten 

knot increments, limited to be within ±15% of the TBO nominal profile speed, and limited to 

conform to any airspace speed restrictions. A graphical depiction of the ASTAR13 CTD speed 

control law is shown in Figure 50. 

Gain scheduling is implemented to ensure that the appropriate amount of speed control is used 

throughout the arrival. The gains used in the CTD speed control law range from 0.5 to 1.5, and 

depend on the IM aircraft along-path distance to the PTP and the magnitude of the spacing error. 

When the IM aircraft is within 7.5 nmi of the PTP, the gains are increased to obtain a more precise 

spacing interval at the PTP. The gains are decreased as the magnitude of the spacing error increases 

because fast-time simulations prior to this simulation showed that decreasing the gain resulted in 

improved string stability and speed behavior. 

The IM standards specify that the IM CAPTURE rate should be a minimum of three seconds per 

minute, which is equivalent to speed control equal to 5% of the Target aircraft’s time-history 

ground speed. There were some cases where the gains were not high enough to meet the capture 

rate specified in the IM standards; therefore, an override function was implemented5

. Whenever 

the speed control (݇௣݁(ݐ ((is less than 5% of the IM aircraft’s nominal TBO airspeed and the 

spacing error is greater than 20 seconds, the magnitude of the speed control is increased so that it 

is equal to 5% of the IM aircraft’s nominal TBO airspeed6. 

Additional heuristics are used to minimize the number of speed commands provided to the flight 

crew. One of these heuristics is a function that uses the Target aircraft’s time-history information 

to estimate the speed that the Target aircraft had at the end of a long deceleration. This results in 

fewer speed commands when the Target aircraft has a large speed change. For example, the 

algorithm can command a single thirty knot deceleration instead of three consecutive ten knot 

decelerations. 

 

5

 The decision to add the override function was made because there was not enough time before the IMAC human-inthe-loop experiment to complete another design revision of the CTD algorithm. Future design iterations should 

examine modifying the proportional gains so that the override function is not needed. 

6

 In order to meet the capture rate in the IM standards, the speed control magnitude must be equal or greater than 5% 

of the Target aircraft’s time-history ground speed. The IM aircraft’s nominal TBO airspeed was used in the override 

function because it is a more stable value than the Target aircraft’s time-history ground speed and was found to result 

in improved speed behavior. However, it should be noted that the use of the IM aircraft’s nominal TBO airspeed 

results in cases where the IM aircraft will not meet the desired capture rate (i.e., when there is a large tailwind). 
Appendix A: IM Software 123 

Figure 50. Diagram of the CTD speed control law. 

A.4 ASTAR Logic Wrapper 

This part of Appendix A contains four sections that provide greater detail on the IM states, the 

criteria to transition between states, the IM alert levels, and the IM messages. 

A.4.1 IM States 

The IM state logic used in this experiment consisted of seven states. These states were: 

1. OFF/TERMINATE: IM not active, or prior to all information being entered into the EFB; 

2. ARMED: information entered, but not all requirements have been met to conduct IM; 

3. AVAILABLE: all requirements met, but the flight crew has not initiated the IM operation; 

4. PAIRED: the IM operation is in effect, and the flight crew flies the IM commanded speed; 

5. SUSPENDED-ARMED: the IM operation was previously in effect, but all requirements 

are no longer met to conduct the operation; 

6. SUSPENDED-AVAILABLE: the IM operation was previously in effect, and all 

requirements are met to resume the operation; and 

7. UNABLE: an unrecoverable error has occurred. 

Figure 51 illustrates the IM states used in the IM application, the manual actions taken by the flight 

crew (pressing the ARM, EXECUTE, SUSPEND, RESUME, and CANCEL buttons), and 

transitions done automatically by the IM application software (to the AVAILABLE, 

SUSPENDED-ARMED, UNABLE, and TERMINATE states). This IM state logic was designed 

so that the OFF and TERMINATE states as defined in reference 4 are the same. 
Appendix A: IM Software 124 

Figure 51. IM state logic flow diagram. 

A.4.2 Transition between IM States 

The following sections describe how the IM software transitions between states. If a transition is 

not listed, for example from AVAILABLE to SUSPSENDED-AVAILABLE, then that transition 

did not exist for this experiment. 

 OFF/TERMINATE State 

• Transition from OFF/TERMINATE to ARMED State 

o The transition from OFF/TERMINATE to ARMED occurs when the flight crew 

presses the ARM IM HOME bezel button or soft-key (R1) on the IM clearance home 

page. For this button to be visible, all the Ownship and IM clearance information must 

be entered. 

o The transition from OFF/TERMINATE to ARMED is shown by a solid blue up arrow 

labeled “ARM” in Figure 51. 
Appendix A: IM Software 125 

 ARMED State 

• Transition from ARMED to OFF/TERMINATE State 

o The transition from ARMED to OFF/TERMINATE occurs one of two possible ways: 

 The flight crew manually presses the CANCEL IM button on the EFB (shown by a 

solid red line labeled “CANCEL” in Figure 51), or 

 The Ownship crosses the Planned Termination Point, causing the software to 

automatically terminate the IM operation (the dotted black line labeled “software 

auto-transition”). 

When the IM software transitions to the OFF/TERMINATE state, all IM clearance information is 

cleared, but the Ownship information is retained. 

The transition from any other state to the OFF/TERMINATE state is always for the two reasons 

listed above, therefore all subsequent sections will state “Previously described.” 

• Transition from ARMED to AVAILABLE State 

o The transition from ARMED to AVAILABLE occurs automatically when all of the 

criteria for the AVAILABLE state are met (listed below). 

o The transition from ARMED to AVAILABLE is shown by a black line in Figure 51. 

• Transition from ARMED to UNABLE State 

o The transition from ARMED to UNABLE occurs automatically when any of the 

following criteria are met: 

 The IM software or system fails, 

 The IM navigation database is not current (not implemented in IMAC), 

 The IM software cannot calculate the route for the Ownship, or 

 The IM software cannot calculate the route for the Target aircraft. 

o The transition from ARMED to UNABLE is shown by the dashed orange line in Figure 

51. 

The transition to the UNABLE state is the exactly the same from any other state, therefore all 

subsequent sections will state “Previously described.” 

 AVAILABLE State 

• The criteria to transition to the AVAILABLE state is all of the following criteria are met: 

o A valid ADS-B track file exists for the Target aircraft, 

o The Ownship aircraft is on its route, 

o The Target aircraft is on its route, 

o The Ownship has passed a speed constrained waypoint (CROSS clearance only), 

o The Target aircraft has passed a speed constrained waypoint (CROSS clearance only), 

o The assigned spacing goal is feasible, and 

o Both aircraft are in the airspeed segment of the route (approximated as below FL290). 
Appendix A: IM Software 126 

• Transition from AVAILABLE to OFF/TERMINATE State 

o Previously described. 

• Transition from AVAILABLE to ARMED State 

o The transition from AVAILABLE to ARMED occurs one of two possible ways: 

 The flight crew manually amends the Ownship information or the IM clearance 

(shown by a solid turquoise line in Figure 51), or 

 When any of the criteria required for the AVAILABLE state (defined above) is no 

longer met (shown by a black down arrow in Figure 51). 

• Transition from AVAILABLE to PAIRED State 

o The transition from AVAILABLE to PAIRED occurs when the flight crew presses the 

EXECUTE button, and is shown by the solid blue up arrow labeled “EXECUTE” in 

Figure 51. 

• Transition from AVAILABLE to UNABLE State 

o Previously described. 

 PAIRED State 

• Transition from PAIRED to OFF/TERMINATE State 

o Previously described. 

• Transition from PAIRED to ARMED State 

o The transition from PAIRED to ARMED occurs when the flight crew modifies any 

Ownship information or IM clearance information. (Note: the Target ID cannot be 

modified, rather it requires the IM operation to be canceled and then the new clearance 

be entered.) 

o The transition from PAIRED to ARMED is shown by a solid turquoise line Figure 51. 

• Transition from PAIRED to SUSPENDED-AVAILABLE State 

o The transition from PAIRED to SUSPENDED-AVAILABLE occurs when the flight 

crew manually suspends the IM operation by pressing the SUSPEND button on the 

EFB. 

o The transition from PAIRED to SUSPENDED-AVAILABLE is shown by a solid limegreen line in Figure 51. 
Appendix A: IM Software 127 

• Transition from PAIRED to SUSPENDED-ARMED State 

o The transition from PAIRED to SUSPENDED-ARMED occurs automatically when 

any of the criteria required for the AVAILABLE state (Section B.2.3) is no longer met. 

o The transition from PAIRED to SUSPENDED-ARMED is shown by the thin solid 

black line in Figure 51. 

• Transition from PAIRED to UNABLE State 

o Previously described. 

 SUSPENDED-AVAILABLE State 

• Transition from SUSPENDED-AVAILABLE to OFF/TERMINATE State 

o Previously described. 

• Transition from SUSPENDED-AVAILABLE to ARMED State 

o The transition from SUSPENDED-AVAILABLE to ARMED occurs when the flight 

crew modifies any Ownship information or IM clearance information. (Note: the Target 

ID cannot be modified, rather it requires the IM operation to be canceled and then the 

new clearance be entered.) 

o The transition from SUSPENDED-AVAILABLE to ARMED is shown by a solid 

turquoise line in Figure 51. 

• Transition from SUSPENDED-AVAILABLE to SUSPEND-ARMED State 

o The transition from SUSPENDED-AVAILABLE to SUSPENDED-ARMED occurs 

automatically when any of the criteria required for the AVAILABLE state (Section 

B.2.3) is no longer met. 

o The transition from SUSPENDED-AVAILABLE to SUSPENDED-ARMED is shown 

by a thin solid black line in Figure 51. 

• Transition from SUSPENDED-AVAILABLE to PAIRED State 

o The transition from SUSPENDED-AVAILABLE to PAIRED occurs when the flight 

crew presses the RESUME button. 

o The transition from SUSPENDED-AVAILABLE to PAIRED is shown by a solid limegreen line in Figure 51. 

• Transition from SUSPENDED-AVAILABLE to UNABLE State 

o Previously described. 
Appendix A: IM Software 128 

 SUSPENDED-ARMED State 

• Transition from SUSPENDED-ARMED to OFF/TERMINATE State 

o Previously described. 

• Transition from SUSPENDED-ARMED to ARMED State 

o The transition from SUSPENDED-ARMED to ARMED occurs when the flight crew 

modifies any Ownship information or IM clearance information. (Note: the Target ID 

cannot be modified, rather it requires the IM operation to be canceled and then the new 

clearance be entered.) 

o The transition from SUSPENDED-ARMED to ARMED is shown by a solid turquoise 

line in Figure 51. 

• Transition from SUSPENDED-ARMED to SUSPENDED-AVAILABLE State 

o The transition from SUSPENDED-ARMED to SUSPENDED-AVAILABLE occurs 

automatically when all of the criteria required for the AVAILABLE state (Section 

B.2.3) is met. 

o The transition from SUSPENDED-ARMED to SUSPENDED-AVAILABLE is shown 

by a thin solid black line in Figure 51. 

• Transition from SUSPENDED-ARMED to UNABLE State 

o Previously described. 

 UNABLE State 

• Transition from UNABLE to the OFF/TERMINATE State 

o Previously described. 
Appendix A: IM Software 129 

A.4.3 Alert Levels 

A structured alert hierarchy that aligns with other cockpit display philosophies was used for the 

IM displays. IM displays and messages are assigned the lowest possible alert level and do not 

change levels if the pilot does not take action. No audio tones are used to alert the pilot of a message 

during the IM operation since none of the IM messages meet the criteria for a warning level alert 

as described in Table 39. 

Table 39. Alert Levels and Characteristics 

 Alert Characteristics 

Alert 

Level Criteria Aural Visual Tactile 

 Display Color 

3 

Warning 

Emergency 

operational or aircraft 

systems conditions 

which require 

immediate corrective 

or compensatory 

action by the crew. 

ATTENTION or 

DISCRETE aural 

alert. 

Time critical alerts 

and annunciations 

may be 

supplemented by 

voice message. 

• Alpha-numeric alert 

messages. 

Red Stick 

shaker 

2 

Caution 

Abnormal operational 

or aircraft systems 

conditions which 

require immediate 

crew awareness and 

subsequent corrective 

or compensatory 

crew action. 

None 

• Alpha-numeric alert 

messages. 

• Flashing reverse video for 

IM speed (indicates a 

change in the IM speed 

greater than 10 seconds ago; 

returns to normal video 

when speed set in MCP). 

White None 

1 

Advisory 

Operational or 

aircraft systems 

conditions which 

require crew 

awareness and may 

require crew action. 

None 

• Alpha-numeric alert 

messages. 

• Reverse video for IM speed 

(indicates a change in the 

IM speed less than 10 

seconds ago; returns to 

normal video when speed set 

in MCP). 

White None 

0 

Info 

Operational or 

aircraft systems 

conditions which 

require flight deck 

indication. 

None • Discrete lights, alphanumeric readout, or icon. White None 

 
Appendix A: IM Software 130 

A.4.4 IM Alerting Messages 

The IM alerting messages are categorized from highest to lowest alert level outlined in Table 39 

(caution, advisory, information), with no IM messages reaching the criticality of a warning level. 

A maximum of three messages are shown on either the EFB or CGD. 

Only messages that enabled the pilot to take direct action based on the information in that message 

were shown in the pilot’s primary field of view on the CGD. All other messages provide an 

explanation for the current IM software state, and are only shown on the EFB outside of the 

primary field of view. 

Table 40. IM Message Criteria and the Pilot Response 

Message Criteria Pilot Response Alert 

Level Location 

AIRCRAFT 

TOO FAST 

The aircraft is faster than 0.02 Mach or 10 

kt above the IM instantaneous speed for 

more than 10 seconds. 

Reduce throttle and/or 

deploy speed brake Caution 

EFB 

and 

CGD 

AIRCRAFT 

TOO SLOW 

The aircraft is slower than 0.02 Mach or 10 

kt below the IM instantaneous speed for 

more than 10 seconds. 

Increase throttle and/or 

retract speed brake Caution 

EFB 

and 

CGD 

IM SYS FAIL 

A failure of the IM software or hardware 

has occurred, or the Ownship data is not 

valid. 

Notify ATC unable to 

initiate the IM 

operation 

Caution EFB 

only 

IM DB NOT 

CURRENT 

Navigation database (DB) used by IM 

system is not current, therefore IM speed 

will not be calculated. This check occurs 

when the IM application is initially selected 

(prior to entry of either Ownship data or IM 

clearance data). 

Notify ATC unable to 

initiate the IM 

operation 

Caution EFB 

only 

OWNSHIP 

BAD ROUTE 

A valid Ownship traffic record and an 

Ownship route definition exists, but the 

calculated trajectory cannot be calculated, 

or it does not meet speed or altitude 

constraints. 

Either notify ATC 

unable to initiate the 

IM operation, or notify 

ATC must terminate 

the IM operation 

Caution EFB 

only 

TGT BAD 

ROUTE 

A valid Target traffic record and a Target 

route definition exists, but the calculated 

trajectory cannot be calculated, or it does 

not meet speed or altitude constraints. 

Either notify ATC 

unable to initiate the 

IM operation, or notify 

ATC must terminate 

the IM operation 

Caution EFB 

only 

OWNSHIP 

OFF ROUTE 

A valid Ownship traffic record exists and 

the Ownship’s calculated trajectory is valid, 

but the Ownship is greater than 2 nautical 

miles laterally or greater than 8000 feet 

vertically from the intended flight path. 

Either update route in 

IM application if 

required (any state 

except PAIRED), or 

notify ATC must 

suspend IM operation 

(if in PAIRED state) 

Advisory EFB 

only 
Appendix A: IM Software 131 

Message Criteria Pilot Response Alert 

Level Location 

TGT OFF 

ROUTE 

A valid Target traffic record exists and the 

Target’s calculated trajectory is valid, but 

the Target is greater than 2 nautical miles 

laterally or greater than 8000 feet vertically 

from the intended flight path. 

No action (any state 

except PAIRED), or 

notify ATC must 

suspend IM operation 

(if in PAIRED state) 

Advisory EFB 

only 

TGT DATA 

LOST 

The ADS-B track file for the Target aircraft 

has been lost or removed (data invalid or no 

longer received for a longer time period 

than the Air Traffic Computer allows). 

No action (any state 

except PAIRED), or 

notify ATC must 

suspend IM operation 

(if in PAIRED state) 

Advisory EFB 

only 

SPC GOAL 

TOO LARGE 

The assigned spacing goal cannot be 

attained, even with the Ownship flying its 

slowest possible speed profile. 

Notify ATC unable to 

initiate or must 

terminate the IM 

operation 

Advisory EFB 

only 

SPC GOAL 

TOO SMALL 

The assigned spacing goal cannot be 

attained, even with the Ownship flying its 

fastest possible speed profile. 

Notify ATC unable to 

initiate or must 

terminate the IM 

operation 

Advisory EFB 

only 

IM SPEED 

LIMITED 

The software calculated speed is being 

limited by one of: 

• regulatory (i.e., 250 knots or less < 

10,000’) 

• airframe (Mmo, Vmo, etc.) 

• more than 15% difference from the 

published speed 

No action Info EFB 

only 

NO 

DESCENT 

WIND 

No forecast descent wind is entered, either 

via data comm or manually. Enter wind if available Info EFB only 

NO 

SURFACE 

WIND 

No forecast surface wind is entered. Enter wind if available Info EFB 

only 

WAITING 

OWN WPT 

Ownship on the specified route, however 

has not yet passed a speed constrained 

waypoint. 

No action Info EFB 

only 

WAITING 

TGT WPT 

Target on its specified route, however the 

aircraft has not yet passed a speedconstrained waypoint. 

No action Info EFB 

only 

WAITING 

TGT DATA 

ADS-B data has never been received from 

the Target aircraft. No action Info EFB 

only 

MANUALLY 

SUSPENDED 

IM operation was manually suspended by 

the pilot. No action Info EFB 

only 

 
Appendix B: IM States and Transitions 132 

Appendix B: IM Displays and Data Entry 

The entirety of Appendix B is derived from reference 6, Cockpit Interfaces, Displays, and Alerting 

Messages for the Interval Management Clearances (IMAC) Experiment. For more detailed 

information, please refer to this document. 

B.1 EFB Overview 

Each pilot could manually interface with and configure their respective EFB (shown in Figure 52) 

to: 1) enter Ownship information, 2) enter the IM clearance, 3) select when to commence, suspend, 

resume, and terminate the IM operation, 4) filter the traffic data shown on the display, and 5) 

display additional information about aircraft and routes. Although the displays were independently 

operated and controlled by the respective pilot, the data entered by one pilot would be reflected in 

the other EFB (caveat: the filter options only applied to the EFB being manipulated). 

The EFBs present information and may have selectable data fields based on what page has been 

selected. The “Ownship and Wind” entry page allowed pilots to enter the IM equipped aircraft’s 

destination, route of flight, and forecast descent winds. The “IM Clearance” entry page allowed 

pilots to specify the IM clearance type, and enter the data unique to that clearance. The “IM Home” 

page presented the following information to the flight crew: 1) mode status of the IM software, 2) 

the IM commanded speed, 3) the Ownship and IM clearance information if previously entered, 4) 

an icon uniquely identifying the Target aircraft, 5) an indication of the aircraft’s current airspeed 

relative to the spacing algorithm’s expected speed, and 6) an indication of the aircraft’s alongtrack position relative to the desired location. 
Appendix B: IM States and Transitions 133 

Figure 52. Legend for the IM Home page on an EFB. 

A) Title of IM page (changes to state function of the page) 

B) Soft-key and bezel button to enter Ownship and forecast wind information 

C) Text box that displays Ownship information 

D) FAST/SLOW indicator 

• PAIRED state: displayed 

• all other states: not displayed 
Appendix B: IM States and Transitions 134 

E) IM commanded speed 

• OFF and TERMINATE states: nothing displayed 

• ARMED, SUSPENDED-ARMED, and UNABLE states: white dashes 

• AVAILABLE and SUSPENDED-AVAILABLE: value shown in grey 

• PAIRED state: value shown in green 

F) IM state 

• OFF and TERMINATE states: nothing displayed 

• ARMED, SUSPENDED-ARMED, and UNABLE states: white text 

• AVAILABLE and SUSPENDED-AVAILABLE: white text 

• PAIRED state: green text 

G) IM alert message box (applicable messages shown in white) 

H) EARLY/LATE indicator (shown when within 30 nautical miles of ABP or during a 

maintain phase of the IM operation) 

I) Soft-key and bezel button to enter or modify the IM clearance information 

J) Non-selectable text box that displays the IM clearance information 

K) Area where Ownship aircraft filter annunciations are displayed 

L) Area where Target aircraft filter annunciations are displayed 

M) Soft-key and bezel button to cancel IM (present in all states except OFF/TERMINATE) 

N) Soft-key and bezel button to display FILTERS page 

O) Soft-key and bezel button to suspend the IM 

P) Ownship location 

• ASTOR, DTS, and IFD: single solid white triangle 

Q) Traffic and Target location 

• Traffic: 

o ASTOR: single-white hollow chevron 

o DTS and IFD: single-cyan hollow chevron 

• Target in the ARMED, AVAILABLE, or SUSPENDED state: 

o ASTOR: single-white hollow chevron surrounded by white chevron 

o DTS and IFD: single-cyan hollow chevron surrounded by a white chevron 

• Target in the PAIRED state: 

o ASTOR, DTS, IFD: single-white hollow chevron surrounded by a green 

chevron 

 
Appendix B: IM States and Transitions 135 

B.2 CGD Overview 

The CGD (Figure 53) cannot be manually interfaced with, rather it is a repeater display in the 

pilot’s optimal primary field of view that automatically provides the subset of the critical 

information from the EFB needed to conduct the IM operation. Unlike the EFBs that could be 

individually manipulated and tailored by the respective pilot, the two CGDs always showed 

identical information. 

Figure 53. Legend for the IM displays on a CGD. 

A) FAST/SLOW indicator 

• PAIRED state: displayed 

• all other states: not displayed 

B) IM commanded speed 

• OFF and TERMINATE states: nothing displayed 

• ARMED, SUSPENDED-ARMED, and UNABLE states: white dashes 

• AVAILABLE and SUSPENDED-AVAILABLE: value shown in grey 

• PAIRED state: value shown in green 

C) Text box that displays the IM clearance information once entered by the crew 

D) IM state 

• OFF and TERMINATE states: nothing displayed 

• ARMED, SUSPENDED-ARMED, and UNABLE states: white text 

• AVAILABLE and SUSPENDED-AVAILABLE: white text 

• PAIRED state: green text 

E) IM alert message box (applicable messages shown in white) 

 
Appendix B: IM States and Transitions 136 

B.3 IM Displays by State 

B.3.1 OFF/TERMINATE State 

No data is shown on the EFB or CGD in the OFF/TERMINATE state prior to the flight crew 

entering any data into the IM application (Figure 54). If the Ownship information has been entered 

into the IM application, it will be visible when the display is in the OFF/TERMINATE state 

(Figure 55). 

Figure 54. EFB and CGD in the OFF state with no Ownship information. 
Appendix B: IM States and Transitions 137 

Figure 55. EFB and CGD in the OFF state with Ownship information. 

 
Appendix B: IM States and Transitions 138 

B.3.2 ARMED State 

In the ARMED state, all IM information is shown on the EFB except for the FAST/SLOW 

indicator, the EARLY/LATE indicator, and the IM commanded speed (Figure 56). Up to three IM 

alerting messages (prioritized by alert level) are displayed on the EFB, and any traffic for which a 

valid ADS-B data file exists are shown as single white chevrons. The IM state, the IM clearance 

type, and the IM clearance information are shown on the CGD, but the IM commanded speed and 

the FAST/SLOW indicator are not shown. No alerting messages are shown on the CGD in the 

ARMED state. 

Figure 56. EFB and CGD in the ARMED state. 

 
Appendix B: IM States and Transitions 139 

B.3.3 AVAILABLE State 

In the AVAILABLE state, the IM information is shown in white on the EFB, the IM commanded 

speed is shown in grey, and the FAST/SLOW and EARLY/LATE indicators are not displayed 

(Figure 57). The white Target aircraft chevron is outlined with a second white chevron. The IM 

state and IM clearance information are shown in white on the CGD, while the IM commanded 

speed is shown in grey and the IM clearance type in green. The FAST/SLOW indicator is not 

displayed. 

Figure 57. EFB and CGD in the AVAILABLE state. 

 
Appendix B: IM States and Transitions 140 

B.3.4 PAIRED State 

In the PAIRED state, the IM state and commanded speed are shown in green. The IM clearance 

type and information are shown cyan on the EFB, and in green and white on the CGD (Figure 58). 

The FAST/SLOW indicator is shown on the EFB and CGD anytime the IM algorithm is in the 

PAIRED state. 

Figure 58. EFB and CGD in the PAIRED state. 

 
Appendix B: IM States and Transitions 141 

B.3.5 SUSPENDED-AVAILABLE State 

In the SUSPENDED-AVAILABLE state, the IM state is shown in white and the commanded speed 

in grey on the EFB and CGD, while the FAST/SLOW and EARLY/LATE indicators are removed 

(Figure 59). The Target aircraft chevron is outlined with a second white chevron (instead of 

outlined in green in the PAIRED state). 

Pressing the RESUME bezel button or soft-key (R8) causes the software to transition to the 

PAIRED state. This is the only time the RESUME button is visible and the function available.

Figure 59. EFB and CGD in the SUSPENDED-AVAILABLE state. 

 
Appendix B: IM States and Transitions 142 

B.3.6 SUSPENDED-ARMED State 

In the SUSPENDED-ARMED state, the IM information is visible but the IM speed, the 

FAST/SLOW indicator, and the EARLY/LATE indicators are removed. Up to three IM alerting 

messages (prioritized by alert level) are displayed on the EFB. All traffic on the EFB is shown as 

a single white chevron. The IM state and the IM clearance information are shown in white on the 

CGD, the IM clearance type in green, and the IM commanded speed and the FAST/SLOW 

indicator are removed. 

Figure 60. EFB and CGD in the SUSPENDED-ARMED state. 

 
Appendix B: IM States and Transitions 143 

B.3.7 UNABLE State 

In the UNABLE state, the Ownship information and IM state remain visible, however the 

FAST/SLOW indicator, the EARLY/LATE indicator, and the IM commanded speed are removed 

from the EFB and CGD displays (Figure 61). All traffic on the EFB is shown as a single white 

chevron (traffic not depicted). 

Figure 61. EFB and CGD in the UNABLE state. 

 
Appendix B: IM States and Transitions 144 

B.4 IM Display Sub-Elements 

B.4.1 Change to IM Command Speed 

When a change occurs to the IM commanded speed, the colors of the green text and black 

background are reversed to provide a salient cue to the flight crew (Figure 62). The reverse video 

is maintained until the flight crew sets the IM commanded speed in the MCP speed window. If the 

correct speed is not set within 10 seconds, the normal and reverse video configuration are 

alternated at approximately 1 Hz until the speed is set in the MCP. 

Figure 62. EFB and CGD with IM speed in reverse video. 

 
Appendix B: IM States and Transitions 145 

B.4.2 FAST/SLOW Indicator 

The FAST/SLOW indicator displays the aircraft’s current airspeed and the IM instantaneous 

speed. It is intended to allow pilots to quickly compare the relationship between the two speeds 

and then take appropriate action. The solid white triangle is the aircraft’s current airspeed, and the 

hollow green triangle is the IM instantaneous speed which is displayed as the reference speed (i.e., 

it remains fixed in the middle of the vertical FAST/SLOW display). The IM instantaneous speed 

takes the discrete IM commanded speed, adds compensation for the delay due to pilot recognition 

and reaction time, then estimates the deceleration rate of the aircraft to produce a smooth and 

continuous value. When the aircraft’s airspeed does not match the IM instantaneous airspeed the 

pilots are expected to move the throttle in the direction towards the green reference speed in the 

center of the display. 

In the example shown in Figure 63, the IM commanded speed changes from 270 to 240 knots. 

From left to right, the first panel shows the IM commanded speed of 270 knots (the green 

numbers), and the aircraft’s current airspeed matches the IM instantaneous speed (solid white 

triangle aligns with the hollow green triangle). The second panel shows the IM commanded speed 

just changed to 240 knots, and the IM instantaneous speed has not changed yet (triangles still 

aligned). In the third panel, the spacing algorithm has calculated that the aircraft should have begun 

to decelerate, however since the aircraft itself is still at 270 knots, the aircraft is faster than expected 

(shown by the solid white triangle moving towards the FAST end of the scale). This display is 

intended to graphically present a cue to the pilots that they should pull the throttles aft, which 

would slow the aircraft and resolve the error calculated by the ASTAR algorithm. The final panel 

shows that when the difference between the aircraft’s current airspeed and the IM instantaneous 

speed is greater than 10 knots, the number of that difference and a message (AIRCRAFT TOO 

FAST) is added to the display to increase the saliency of the information being presented to the 

flight crew. 

Figure 63. FAST/SLOW indicator sequence during IM speed change. 

B.4.3 EARLY/LATE Indicator 

The EARLY/LATE indicator provides the flight crew an awareness of their ability to meet the 

assigned spacing goal within the expected tolerance and is not used to actively control the aircraft. 

The EARLY/LATE indicator is only presented on the side-mounted EFB and only shown when 

the aircraft is within 30 nautical miles of the ABP (CROSS operation only) or during the 

MAINTAIN phase of any IM operation (between the ABP and PTP in a CROSS operation, or 

anytime during a CAPTURE and a MAINTAIN operation). 
Appendix B: IM States and Transitions 146 

From 30 nautical miles prior to the ABP until the aircraft is within 210 seconds of the ABP, the 

scale of the display is ± 2 minutes, with tick marks at zero, +60, and -60 seconds. The “bug” is 

located at the current spacing error value and is a circle with a diameter equal to 20 seconds (left 

panel of Figure 64). 

From 210 seconds prior to the ABP until the Termination Point, the scale is ±45 seconds with tick 

marks at +30, +15, 0, -15, and -30 seconds. The “bug” is located at the current spacing error value 

and is a circle with a diameter equal to 20 seconds (right panel of Figure 64). The 20 second 

diameter of the circle is the same as the IM error tolerance for a maintain operation described in 

the IM MOPS. Thus, if the center mark is not within the circle, the spacing error is greater than 

the ±10 second tolerance. 

 

Figure 64. Progress Indicator with 2 minute (left) and 45 second (right) scales. 

B.4.4 Canceling the IM Operation 

When the pilot presses the CANCEL IM bezel button or soft-key (L8 in Figure 62), a page is 

displayed on the EFB that requires the flight crew to confirm their intention to cancel the IM 

operation (left panel of Figure 65). The YES bezel button and soft-key (R8) is on the opposite side 

of the EFB from the previous CANCEL IM button to prevent accidental termination of the IM 

operation. 

Pressing the YES bezel button or soft-key (R8) terminates the IM operation and returns the IM 

equipment to the OFF/TERMINATE state (right panel of Figure 62). Pressing the NO bezel button 

or soft-key (L8) returns the IM display to the previous IM state and page. 

The right panel of Table 69 illustrates the EFB after an IM operation has been manually canceled 

by the flight crew or automatically canceled by the software (at the Planned Termination Point). 
Appendix B: IM States and Transitions 147 

 

Figure 65. EFB display to confirm cancellation of IM operation. 

 
Appendix B: IM States and Transitions 148 

B.5 Flight Crew Data Entry into IM Interface 

B.5.1 Navigation to the IM Home Page 

When the EFB is first powered on or the MENU bezel button is pressed, the MAIN MENU page 

is displayed (left panel of Figure 66). Pressing the APPLICATIONS MENU (L2) bezel button or 

touch screen soft-key causes the list of cockpit-based procedures to be displayed to the flight crew 

(right panel of Figure 66). Pressing the INTERVAL MANAGEMENT (L1) bezel button or softkey causes the IM Home page to be displayed (Figure 67). 

 

Figure 66. Main Menu and Applications Menu pages. 

 
Appendix B: IM States and Transitions 149 

B.5.2 Ownship and Wind Data 

Pressing the OWNSHIP AND WINDS bezel button or soft-key (L1) on the IM Home page (left 

panel of Figure 67) causes the OWNSHIP AND WIND ENTRY page to be displayed (right panel 

of Figure 67). The flight crew enter the Ownship information as soon as feasible (typically prior 

to receiving an IM clearance from ATC). 

Mandatory entries are indicated by boxes (e.g., four characters must be entered for the destination 

airfield), and non-mandatory entries are indicated by dashes (e.g., descent forecast wind) in the 

right panel of Figure 67. If the optional descent forecast wind information is not entered, a message 

to the pilots is triggered. 

Pressing the DESTINATION AIRPORT bezel button or soft-key (L1) causes that data field to be 

highlighted in green and causes the keypad to be displayed as well (left panel of Figure 68). 

 

Figure 67. IM Home and Ownship Entry pages. 
Appendix B: IM States and Transitions 150 

When a data field is active (indicated by the green hue and underscore cursor), any value touched 

on the keypad is entered into that field up to the maximum available characters limit. In this 

example, the flight crew will select the four-letter identifier of their destination airport, causing the 

display to change to the right panel of Figure 68. 

Once the four letters have been typed into the data field, the flight crew may either press the 

ENTER soft-key adjacent to (L5), in which case no data field is active (i.e. green), or else press 

the bezel button or soft-key to activate the next field of intended data entry. 

The flight crew must enter the destination airport prior to entering either the Ownship arrival route 

or Target route information. To minimize the likelihood of data entry error by the flight crew, the 

Ownship and Target route information is entered by the crew using a selectable list of options (not 

feasible for airport and forecast winds). 

 

Figure 68. Airport data field selected and airport entered. 

 
Appendix B: IM States and Transitions 151 

Once the descent forecast wind is received via ACARS, the bezel button and soft-key (R6) to autoload the wind direction and speed at four different altitudes is shown (left panel of Figure 69). The 

time stamp of the message is shown on the second line of the data field. 

Pressing the LOAD DEC FCST WIND bezel button or soft-key causes 1) the data from the 

ACARS message to be auto-loaded into the appropriate data fields, 2) the time of the message to 

be displayed in the page title, 3) activates the next data field (surface wind direction), and 4) causes 

the keypad to appear for manual data entry (right panel of Figure 69). 

 

Figure 69. Descent Forecast page prior to and after data entry. 

 
Appendix B: IM States and Transitions 152 

Figure 70 illustrates the IM Home page with the Ownship information entered. The three Ownship 

data elements shown on the upper left blue box are 1) the next waypoint, 2) the arrival procedure, 

and 3) the instrument procedure. 

Any Ownship data may be modified by pressing the OWNSHIP & WINDS bezel button or softkey (R1) on the IM Home page to return to the OWNSHIP AND WIND ENTRY page. All three 

data fields (airport, route, and winds) can be changed using the same procedure that was used for 

the initial data entry. 

Pressing the IM CLEARANCE bezel button or touch screen soft-key (R1) brings up the initial IM 

CLEARANCE ENTRY page (Figure 71). 

Figure 70. IM Home page with Ownship data entered. 
Appendix B: IM States and Transitions 153 

B.5.3 IM Clearance Entry Page 

This section uses the CROSS clearance type as the example since it has the most data elements, is 

the most complex of the clearances implemented for this experiment, and is the clearance type 

used in all previous IM research at NASA Langley and Ames Research Centers. 

To minimize the probability of data entry error by the flight crew, whenever feasible, the IM 

clearance information is entered by the crew using a selectable list of options. Exceptions include 

the assigned spacing goal and time periods when the Target aircraft is outside ADS-B range of the 

IM aircraft. 

The left panel of Figure 71 illustrates the initial IM CLEARANCE ENTRY page, displayed when 

the flight crew presses the IM clearance bezel button or soft-key in Figure 70, but no clearance 

data has been entered yet. 

Pressing the CLEARANCE TYPE bezel button or soft-key (L1 on left panel of Figure 71) causes 

the clearance type sub-page to be displayed (right panel of Figure 71). The clearance types are 

listed in alphabetical order from top to bottom, and any type that has been implemented is shown 

in grey, while those not implemented are shown in cyan (non-selectable). Pressing the appropriate 

bezel button or soft-key (L2 for CROSS clearance in this example) causes a page to appear that 

displays only those data elements relative to that clearance type (Figure 72). 

 

Figure 71. Initial IM clearance entry page and IM clearance type sub-page. 

 
Appendix B: IM States and Transitions 154 

The left panel of Figure 72 illustrates the required data elements for the IM CROSS clearance type, 

shown top to bottom in the order that the controller is expected to issue the clearance. The 

Achieve-By and Terminate At soft-keys auto-populate with the Ownship’s final approach fix 

waypoint. 

Pressing the ARM IM HOME bezel button or soft-key (R1 in the right panel of Figure 72) returns 

the display to the IM Home page. If the clearance is fully input, then the IM software will also 

transition to the ARMED state (Figure 73). 

 

Figure 72. IM CROSS clearance page with and without data entered. 
Appendix B: IM States and Transitions 155 

Figure 73 illustrates the IM Home page with both Ownship and IM clearance information 

displayed, and the software has transitioned from the OFF to the ARMED state. A CANCEL IM 

function at (L8) in gray is now visible and selectable, while an EXECUTE function is visible at 

(R8) in cyan, but not selectable (turns white in the AVAILABLE state). Up to three status 

messages, listed by priority, may appear immediately below the IM state. 

Figure 73. IM Home page with all data entered. 

 
Appendix B: IM States and Transitions 156 

B.6 IM Displays in DTS and IFD 

The previous sub-section of this Appendix used pictures of the software-based EFB and CGD 

generated by the ASTOR simulator. This sub-section of the Appendix provides graphics generated 

by the DTS and IFD simulators which use hardware-based EFBs. The bezel buttons, while not 

depicted, function and are located identically to the ASTOR EFB. 

Two Astronautics EFBs for both the DTS and IFD were positioned on three-axis rotating mounts 

next to each pilot and co-pilot station, allowing rotation from portrait to landscape mode. The DTS 

used a software emulated CGD located next to the navigation display, while the IFD used a 

hardware-based CGD. 

Shown in Figure 74 is an example of an IM CROSS clearance display in the DTS and IFD, similar 

to Figure 72 for the ASTOR. 

Figure 74. IM CROSS clearance page in DTS and IFD. 

 
Appendix B: IM States and Transitions 157 

Shown in Figure 75 is an example of an IM CAPTURE and an IM MAINTAIN clearance display 

in the DTS and IFD. 

 

Figure 75. IM CAPTURE and MAINTAIN clearance pages in DTS and IFD. 

Figure 76 illustrates the ARMED and AVAILABLE states in the DTS and IFD, and Figure 77 

illustrates the PAIRED and the SUSPENDED-AVAILABLE states in the DTS and IFD. 

 
Appendix B: IM States and Transitions 158 

 

Figure 76. ARMED and AVAILABLE states in the DTS and IFD. 

 

Figure 77. PAIRED and SUSPENDED-AVAILABLE states in the DTS and IFD.
Appendix B: IM States and Transitions 159 

Figure 78 illustrates the confirm cancellation page in the DTS and IFD, similar to Figure 65 for 

the ASTORs. 

Figure 78. The confirm cancel message in the DTS and IFD. 

Figure 79 illustrates the CGD as displayed in the DTS and IFD in the AVAILABLE state (Figure 

57 for the ASTORs) and PAIRED state (Figure 62 for the ASTORs). 

 

Figure 79. The CGD showing AVAILABLE and PAIRED in the DTS and IFD.

 
Appendix C: Procedures and Phraseology 160 

Appendix C: Procedures and Phraseology for IM Operations 

C.1 Pilot Procedures for IM Operations 

C.1.1 Nominal IM Procedures 

During nominal current-day and IM operations, the flight crews were instructed to 1) achieve and 

maintain the assigned lateral and vertical path, 2) achieve and maintain the appropriate airspeed, 

and 3) achieve and maintain the proper spacing. 

1) The procedures to achieve and maintain the proper vertical path were: 

• Verify VNAV speed is the active mode; 

• Ensure aircraft starts a descent at Top of Descent (TOD) Point; 

• Use drag and thrust as necessary to maintain vertical path within ±400 feet; and 

• Monitor the aircraft to ensure it stays on path and all restrictions are met. 

2) The procedures to achieve and maintain the proper airspeed were: 

• Observe and announce IM Speed changes and mode changes on CGD/EFB; 

• Determine if airspeed is safe and acceptable for current conditions 

o If not, see “Other IM Flight Crew Procedures” section below for action 

• Set IM commanded speed in speed window on MCP; 

o Note: speed changes are highlighted for 10 seconds, then flash if not set in MCP 

• Maintain ±10 knots of IM instantaneous speed during speed changes; 

• Configure aircraft as necessary to maintain IM commanded speed. 

NOTE: The IM spacing algorithm is designed to comply with all procedural constraints (e.g., 250 

knots or less when at or below 10,000’ MSL), and the flight crew is responsible for determining 

that the IM speed is operationally acceptable for the current flight conditions. If those two criteria 

are met, the priority of what speed the crew should fly is: 

1. Controller-assigned airspeed 

2. IM assigned airspeed 

3. Published procedure airspeed 

4. Airline standard operating procedure airspeed 

Note: The FAST/SLOW indicator can be used as a secondary display for deceleration or 

acceleration rate guidance. This display is particularly important when close to the ABP. 

3) The procedures to achieve or maintain the proper spacing were: 

• Ensure no alert messages on EFB or CGD (See Appendix A.4.4 for messages); 

• Notify ATC when initially spacing behind Target aircraft; 

• Notify each new ATC check in with “Paired with”; and 

• Notify ATC if no longer IM spacing. 
Appendix C: Procedures and Phraseology 161 

C.1.2 Off-Nominal IM Procedures 

1) The controller amends the ASG: 

• Press IM GOAL and enter the new value into the system; 

• The other pilot confirms the new value has been correctly entered; 

• Assess the new IM commanded speed for acceptability; and 

• If acceptable, notify ATC of PAIRED status and IM speed; or 

• If unacceptable or infeasible, notify ATC unable to conduct IM operation. 

2) The controller suspends, resumes, or cancels the IM operation: 

• Press the SUSPEND, RESUME or CANCEL button (Figure 52) as appropriate; and 

• Notify ATC of new status (for example, RESUME with IM speed). 

Note: If the controller suspends or cancels the IM operation, that instruction should also include 

either clearance to resume the arrival procedure as published, or a specific heading and speed for 

the flight crew to fly. 

3) The flight crew must suspend or cancel the IM operation: 

• Continue to fly current airspeed and notify ATC; and 

• Comply with ATC instructions. 

Note: The ATC response can range from “advise when able to resume spacing” to “cancel IM and 

resume the published approach.” 

NOTE: When an IM operation is canceled or suspended, the IM commanded speed is removed 

from the cockpit displays. In this situation, the priority of what speed the crew should fly is: 

1. Controller-assigned airspeed 

2. Last valid IM airspeed 

3. Published procedure airspeed 

4. Airline standard operating procedure airspeed 

C.2 Controller – Pilot Phraseology for IM Operations 

This Appendix contains examples of the phraseology used during IMAC to conduct IM operations. 

The controllers and pilots in this experiment were trained and instructed to use their judgment 

whether to use the Target’s call sign, i.e., Southwest3033, or to use the phonetic pronunciation, 

i.e., Sierra Whiskey Alpha three zero three three. The alpha-numeric format, i.e., S-W-A-3-0-3-3, 

was not part of the training for IM phraseology, however some controllers and pilots did 

occasionally use it. 

Whenever an aircraft call sign (for example SWA3033) is shown in this document, it should be 

understood that the controllers and pilots verbally said Southwest3033 or Sierra Whiskey Alpha 

three zero three three. 
Appendix C: Procedures and Phraseology 162 

C.2.1 Preparatory Call 

Controllers were instructed to issue a preparatory call to the flight crew prior to issuing the IM 

clearance to ensure the crew was prepared and could write down the IM instruction. 

ATC: (Call sign), CLEARANCE AVAILABLE, ADVISE WHEN READY TO COPY. 

Crew: (Call sign) READY TO COPY. 

C.2.2 IM Clearance Instruction 

An example of a controller-issued IM CROSS clearance in which the PTP is not specified, 

therefore the IM software sets the PTP to a default of the FAF (in this example FRONZ is both the 

ABP and PTP). 

ATC: (Call sign), FOR INTERVAL SPACING, CROSS FRONZ 120 SECONDS BEHIND 

SWA3033 ON THE ANCHR2 ARRIVAL. 

An example of a controller-issued IM CROSS clearance in which the ABP and PTP are not the 

same, and therefore the controller does need to issue them both. 

ATC: (Call sign), FOR INTERVAL SPACING, CROSS CTFSH 120 SECONDS BEHIND 

November Alpha Sierra Alpha zero niner ON THE ANCHR2 ARRIVAL, TERMIANTE 

AT DOGGG. REPORT PAIRED. 

An example of a controller-issued IM CAPTURE clearance. 

ATC: (Call sign), FOR INTERVAL SPACING, CAPTURE 120 SECONDS BEHIND 

SWA3033. REPORT PAIRED. 

An example of a controller-issued IM maintain clearance is: 

ATC: (Call sign), FOR INTERVAL SPACING, MAINTAIN CURRENT TIME BEHIND 

SWA3033. REPORT PAIRED. 

C.2.3 IM Operation Commencing 

The flight crew will notify the controller when commencing the IM operation and will include the 

initial IM speed. The controller will update the aircraft data tag as described in Section 2. 

Crew: (Call sign), NASA06 PAIRED BEHIND United254. AIRSPEED IS 290 KNOTS. 

ATC: (Call sign) ROGER. 

C.2.4 Amendment to IM Clearance 

Any field of the IM clearance can be modified except for the Target call sign. Shown here is a 

change to the ASG from 120 seconds in the previous sub-section to 115 seconds. 

ATC: (Call sign), AMENDMENT TO YOUR CLEARANCE. ADVISE WHEN READY TO 

COPY. 
Appendix C: Procedures and Phraseology 163 

Crew: (Call sign) READY TO COPY. 

ATC: (Call sign), SPACE 115 SECONDS BEHIND Sierra Whiskey Alpha three zero three 

three. 

Crew: (Call sign), SPACE 115 SECONDS BEHIND Sierra Whiskey Alpha three zero three 

three. 

C.2.5 Suspending the IM Operation 

ATC: (Call sign), SUSPEND INTERVAL SPACING, SLOW TO two-three-zero KNOTS. 

Crew: (Call sign), SUSPEND INTERVAL SPACING, SLOW TO two-three-zero KNOTS. 

C.2.6 Resuming the IM Operation 

ATC: (Call sign), RESUME INTERVAL SPACING BEHIND Delta six-two-two. REPORT 

PAIRED. 

Crew: (Call sign), RESUME INTERVAL SPACING BEHIND Delta six-two-two. REPORT 

PAIRED. 

C.2.7 Cancelling the IM Operation 

The IM operation can be canceled by the controller or the flight crew. In this example, the flight 

crew notifies the controller the IM operation has been suspended due to the Target aircraft no 

longer being on the expected trajectory, and the controller elects to terminate the IM operation. 

Once the controller determines that the IM operation will be canceled without the expectation of 

allowing the aircraft to pair at a later time, the keyboard command “FI” is entered. This removes 

all IM indicators from the data-tag, and the flight is now controlled as a non-IM operation. 

Crew: (Call sign), INTERVAL SPACING SUSPENDED, TARGET OFF PATH. 

ATC: (Call sign), CANCEL INTERVAL SPACING, MAINTAIN two-one-zero KNOTS. 

Crew: (Call sign), CANCEL INTERVAL SPACING, MAINTAIN two-one-zero KNOTS. 

C.2.8 Check-in with subsequent controllers 

Not all controllers have IM information available on their workstations, and therefore until it is 

available, the ATD-1 ConOps instructs the flight crew to append the IM information to the initial 

check-in with each subsequent controller. 

Crew: (Call sign) LEAVING one-niner thousand, DESCENDING VIA THE TELLR2 

ARRIVAL, PAIRED BEHIND Delta Alpha Lima one-two-eight. 

ATC: (Call sign), ROGER.

 
Appendix D: Denver ATC Site Visit 164 

Appendix D: Airlines Operating into Denver International Airport 

Based on the KDEN site visit, Table 41 is a list of some of the airlines that operate into KDEN and 

their corresponding ID (or data tag) and voice call sign. 

Table 41. Airline IDs and Call Signs Used in IMAC 

ID Call sign Airline

AAL American American Airlines 

ACA Air Canada Air Canada 

ANZ New Zealand Air New Zealand 

ASA Alaska Alaska Airlines 

ASQ Acey Express Jet 

ASH Air Shuttle Mesa 

BAW Speed Bird British Airways 

CHQ Chautauqua Chautauqua 

DAL Delta Delta 

DLH Lufthansa Lufthansa 

EJM Jet Speed Executive Jet Management 

FDX FedEx FedEx 

FFT Frontier Frontier 

GJS Lindbergh GoJet Airlines 

GLA Lakes Air Great Lakes 

JBU JetBlue JetBlue 

LOF Water Ski Trans States Airlines 

LYM Key Lime Key Lime Air 

NKS Spirit Wings Spirit Airlines 

RPA Brickyard Republic 

SKW Skywest SkyWest 

SWA Southwest Southwest 

TCF Mercury Shuttle America 

UAL United United 

UPS UPS United Parcel 

VIR Virgin Virgin Atlantic 

WJA West Jet West Jet 

 
Appendix E.1: Controller Background Questionnaire 165 

Appendix E: Questionnaires 

E.1 Controller Background Questionnaire 

1. Controller ID: 

2. What is your current age? 

3. Did you work en route ATC? 

4. [If answered ‘Yes’ to en route] Please list the facilities where you worked en route and 

the number of years at each. 

5. Did you work TRACON? 

6. [If answered ‘Yes’ to TRACON] Please list the facilities where you worked TRACON, 

the number of years at each, and the position(s) held. 

7. Did you work Tower? 

8. [If answered ‘Yes’ to Tower] Please list the facilities where you worked Tower and 

the number of years at each. 

9. On what date (MM/YYYY) did you retire? 

10. What qualifications have you held? 

Please choose all that apply: 

 Certified Professional Controller (CPC) 

 Front Line Manager (FLM) 

 Staff Specialist 

 Traffic Management Coordinator (TMC) 

 Other: 

11. Describe your experience and currency with RNAV arrivals. 

12. Describe your experience and currency with RNP approaches. 

13. Describe your experience with any previous IM experiments and please provide the 

dates and locations. 

 
Appendix E.2: Pilot Background Questionnaire166 

E.2 Pilot Background Questionnaire 

1. Pilot ID: 

2. What is your current age? 

3. With which airline do you have the most experience? 

4. How many years of military/commercial flight experience do you have? 

5. How many hours of commercial, multi-engine flight experience do you have? 

6. What type(s) of aircraft have you flown? 

7. On what date (MM/YYYY) did you most recently fly a commercial aircraft? 

8. What qualifications have you held (instructor, standards captain, etc.)? 

9. Describe your experience and currency with RNAV arrivals. 

10. Describe your experience and currency with RNP approaches. 

11. Describe your experience with any previous IM experiments and please provide the 

dates and locations. 

 
Appendix E.3: Controller Post-Run Questionnaire 167 

E.3 Controller Post-Run Questionnaire 

1. Controller ID: 

2. Please select the scenario you just completed from the list below: 

M6 

Scenario A 

Scenario B 

Scenario C 

Scenario D 

Scenario E 

Scenario F 

Scenario G 

Scenario H 

Scenario I 

Scenario J 

3. Which position did you work in this scenario? 

Center North 

Center South 

Feeder 

Final 

4. [TLX 1] How mentally demanding was the scenario? 

Very 

Low Very 

High 

1 2 3 4 5 6 7 

5. [TLX 2] How physically demanding was the scenario? 

Very 

Low Very High 

1 2 3 4 5 6 7 

 

6. [TLX 3] How hurried or rushed was the pace of the scenario? 

Very 

Low Very High 

1 2 3 4 5 6 7 

 
Appendix E.3: Controller Post-Run Questionnaire 168 

7. [TLX 4] How hard did you have to work to accomplish your level of performance? 

Very 

Low Very High 

1 2 3 4 5 6 7 

8. [TLX 5] How insecure, discouraged, irritated, stressed, and annoyed were you? 

Very 

Low Very High 

1 2 3 4 5 6 7 

9. [TLX 6] How successful were you in accomplishing what you were asked to do? 

Good Poor 

1 2 3 4 5 6 7 

** For controller post-run questions #11 through #16, the italicized number in square brackets (not 

shown on survey to controllers) is the number associated with the rating shown in Figure 31. ** 

10. [CARS 1] Was the system safe in this last run? 

Yes 

No 

11. [CARS 2] [If controller responded ‘Yes’ to CARS 1] Did the system function 

adequately, so that you had a tolerable workload? 

Yes 

No [1] 

12. [CARS 3] [If controller responded ‘No’ to CARS 2] Given that you think the 

system’s performance was not adequate, were the operations controllable? 

Barely controllable, needs extreme amounts of controller compensation [2] 

Marginally controllable, needs considerable controller compensation [3] 

Controllable, needs some controller compensation [4] 

13. [CARS 4] [If controller responded ‘Yes’ to CARS 2] Is the system here satisfactory 

without improvement? 

Yes 

No 
Appendix E.3: Controller Post-Run Questionnaire 169 

14. [CARS 5] [If controller responded ‘No’ to CARS 4] How much did you have to 

compensate for the system to make the operations work? (In these situations “compensate” 

means how much did you have to work to counterbalance or offset less desirable actions 

from the system?) 

Extensive compensation required to maintain adequate performance [5] 

Considerable compensation required to achieve adequate performance [6] 

Moderate compensation required to maintain adequate performance [7] 

15. [CARS 6] [If controller responded ‘Yes’ to CARS 4] How close to a desired level of 

performance was the system in this scenario? 

Moderate controller compensation needed to reach desired performance [8] 

Minimal controller compensation required to reach desired performance [9] 

Desired ATC system performance with no controller correction [10] 

16. [CARS 7] [If controller responded ‘No’ to CARS 1] Please describe any events you 

saw in the last run that were unsafe. 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

17. To what degree did you modify your technique in order to use the CMS tools? 

Not at all Moderately Completely

1 2 3 4 5 6 7 

Please comment on how and why you modified your technique: [mandatory if rated 4-7] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

18. To what degree did you modify your technique in order to use the IM tools? 

Not 

at all Moderately Completely

1 2 3 4 5 6 7 

Please comment on how and why you modified your technique: [mandatory if rated 4-7] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 
Appendix E.3: Controller Post-Run Questionnaire 170 

19. Did you have to change the way you worked to manage the IM aircraft? 

Please choose all that apply: 

 Changed my general scan of the IM aircraft (compared with non-IM aircraft) 

 Changed the way I monitored separation of the IM aircraft (compared with non-IM 

aircraft) 

 Changed the way I monitored the IM aircraft spacing (compared with non-IM aircraft) 

 Issued different types of clearances to the IM aircraft (compared with non-IM aircraft) 

 Issued more clearances to the IM aircraft (compared with non-IM aircraft) 

 Less voice communication with the IM aircraft, including speed instructions or vectors 

(compared with non-IM aircraft) 

 More voice communication with the IM aircraft, including speed instructions or vectors 

(compared with non-IM aircraft) 

 Changed the way I coordinated with other controllers (compared with non-IM aircraft) 

 Considered different types of solutions to IM aircraft problems (compared with non-IM 

aircraft) 

 Other (If none, comment “None”): 

Please comment on your choice here: 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

20. Please rate the acceptability of the speeds flown by the IM aircraft. 

Completely 

Unacceptable Completely Acceptable 

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

 
Appendix E.3: Controller Post-Run Questionnaire 171 

21. Did you issue any of the clearance types listed below? Please note the aircraft call 

sign (if able) as well as where and why you took this action in the box on the right. 

Please choose all that apply and provide a comment: [mandatory comment for each chosen] 

 Suspended IM spacing 

 Resumed IM spacing 

 Canceled IM spacing 

 Decided not to issue a IM clearance 

 Aircraft never paired ______________________________ 

 Other (If none, comment “None”) 

22. [If Center] Were there any issues with the IM clearance phraseology? 

Yes 

No 

Please comment on your choice here: [mandatory comment if “Yes”] 

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________ 

23. [If Center] What type of third party ID was used to issue the IM clearance? Please 

note the aircraft call sign (if able) and describe any issue(s) in the box on the right. 

Please choose all that apply and provide a comment: [mandatory comment for each chosen] 

 Call sign (e.g., Brickyard 123) 

 Alpha-numeric (e.g., R P A 123) 

 Phonetic (e.g., Romeo Papa Alpha 123) 

 Other: 

Please describe any issues with the IM clearance phraseology here: 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

24. Describe any unusual or unexpected event(s). Please include time, location, and 

aircraft call signs if able. 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________
Appendix E.4: Pilot Post-Run Questionnaire 172 

E.4 Pilot Post-Run Questionnaire 

1. Pilot ID: 

2. Please select the scenario you just completed: 

M6 

Scenario A 

Scenario B 

Scenario C 

Scenario D 

Scenario E 

Scenario F 

Scenario G 

Scenario H 

Scenario I 

Scenario J 

3. Please select your position during the scenario: 

Captain 

First Officer 

4. Please select your role during the scenario: 

Pilot Flying 

Pilot Monitoring 

5. [MCH 1] Even though errors may be large or frequent, can instructed task be 

accomplished most of the time? 

Yes 

No 

6. [MCH 2] [If pilot responded ‘Yes’ to MCH 1] Are errors small and inconsequential? 

Yes 

No 

7. [MCH 3] [If pilot responded ‘No’ to MCH 2] Given that major deficiencies exist and 

system redesign is strongly recommended, please choose one of the following ratings: 

Major difficulty / maximum operator mental effort is required to bring errors to moderate 

level 

Major difficulty / maximum operator mental effort is required to avoid large or numerous 

errors 
Appendix E.4: Pilot Post-Run Questionnaire 173 

Major difficulty / intense operator mental effort is required to accomplish task, but 

frequent or numerous errors persist 

8. [MCH 4] [If pilot responded ‘Yes’ to MCH 2] Is mental workload level acceptable? 

Yes 

No 

9. [MCH 5] [If pilot responded ‘No’ to MCH 4] Given that mental workload is high 

and should be reduced, please choose one of the following ratings: 

Minor but annoying difficulty / moderately high operator mental effort is required to 

attain adequate system performance 

Moderately objectionable difficulty / high operator mental effort is required to attain 

adequate system performance 

Very objectionable but tolerable difficulty / maximum operator mental effort is required 

to attain adequate system performance 

10. [MCH 6] [If pilot responded ‘Yes’ to MCH 4] Given that mental workload level was 

acceptable, please choose from one of the following: 

Very easy / highly desirable / operator mental effort is minimal and desired performance 

is easily attainable 

Easy, desirable / operator mental effort is low and desired performance is attainable 

Fair, mild difficulty / acceptable operator mental effort is required to attain adequate 

system performance 

11. [Acceptability] Please rate the overall acceptability of IM during the scenario you 

just completed. 

 Completely Completely 

 Unacceptable Acceptable 

 1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

 
Appendix E.4: Pilot Post-Run Questionnaire 174 

12. Please rate the acceptability of IM during each segment of flight during the scenario 

you just completed. 

Completely 

Unacceptable Completely Acceptable 

 1 2 3 4 5 6 7 

> 18,000 ft. (cruise, initial descent) ○ ○ ○ ○ ○ ○ ○

18,000-11,000 (descent, approach 

check) 

○ ○ ○ ○ ○ ○ ○

11,000-6,000 (TRACON, low 

altitude merge) 

○ ○ ○ ○ ○ ○ ○

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

__________________________________________________________________________

13. Please rate the acceptability of the use of voice communications to provide the IM 

clearance(s) during the scenario you just completed. 

 Completely Completely 

 Unacceptable Acceptable 

 1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

14. Please rate the overall effectiveness with which relevant information, including 

operational plans, decisions, and changes in aircraft state were communicated between 

yourself and your crew member. 

 Completely Completely 

 Ineffective Effective 

 1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________
Appendix E.4: Pilot Post-Run Questionnaire 175 

15. Please rate the operational acceptability of the IM commanded speeds. 

 Completely Completely 

 Unacceptable Acceptable 

 1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

16. Please rate the acceptability of the frequency of IM speed changes. 

 Completely Completely 

 Unacceptable Acceptable 

 1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

17. Please rate the acceptability of the amount of head down time required of the Pilot 

Monitoring to input information from the IM clearance(s) into the EFB. 

Completely 

Unacceptable Completely 

Acceptable 

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

 
Appendix E.4: Pilot Post-Run Questionnaire 176 

18. Please rate the acceptability of the IM procedures for the events in this scenario. 

Completely 

Unacceptable Completely Acceptable 

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

19. Please rate the completeness of the IM procedures for the events in this scenario. 

Not at all 

complete Somewhat complete Very complete 

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

20. Were there any missing steps in the IM procedures? 

Yes 

No 

Please comment on your choice here: [mandatory comment if Yes] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

21. Did the IM procedures contain extra steps that were unnecessary? 

Yes 

No 

Please comment on your choice here: [mandatory comment if Yes] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

 
Appendix E.4: Pilot Post-Run Questionnaire 177 

22. Were the IM procedural steps logical and easy to follow? 

Yes 

No 

Please comment on your choice here: [mandatory comment if No] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

23. Did you take any of the actions listed below? Please note where and when (if able), 

as well as why you took this action in the box on the right. 

Please choose all that apply and provide a comment: [mandatory comment for each chosen] 

 Suspended IM spacing 

 Resumed IM spacing 

 Canceled IM spacing 

 Aircraft never paired ______________________________ 

 Other (If none, comment “None”) 

24. Were there any issues with the IM phraseology? 

Yes 

No 

Please comment on your choice here: [mandatory comment if Yes] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

25. What type of third party ID was used to issue the IM clearance? Please note the 

aircraft call sign (if able) and describe any issues in the box on the right. 

 Call sign (e.g., Brickyard 123) 

 Alpha-numeric (e.g., R P A 123) 

 Phonetic (e.g., Romeo Papa Alpha 123) 

 Other 

Please describe any issues with the IM clearance phraseology here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________
Appendix E.4: Pilot Post-Run Questionnaire 178 

26. Were there any issues with using a touch-screen device during this scenario? 

Yes 

No 

Please comment on your choice here: [mandatory comment if Yes] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

27. Describe any unusual or unexpected events. Please include time, location, and 

aircraft call signs if able. 

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________ 

 
Appendix E.5: Controller Post-Exp. Questionnaire 179 

E.5 Controller Post-Experiment Questionnaire 

1. Controller ID: 

2. Which position did you work during the simulation? 

Center North 

Center South 

Feeder 

Final 

3. Describe any anomalies or inconsistencies in the simulation that affected your 

performance. 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

4. Please rate how well the training prepared you to use the following: 

Not At 

All 

Prepared 

 Moderately 

Prepared 

 Very 

Prepared

 1 2 3 4 5 6 7 

simulator ○ ○ ○ ○ ○ ○ ○

airspace ○ ○ ○ ○ ○ ○ ○

metering concept ○ ○ ○ ○ ○ ○ ○

CMS tools ○ ○ ○ ○ ○ ○ ○

IM operation ○ ○ ○ ○ ○ ○ ○

phraseology ○ ○ ○ ○ ○ ○ ○

5. Please describe how the academic and hands-on training can be improved. 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

 
Appendix E.5: Controller Post-Exp. Questionnaire 180 

6. Please rate the operational acceptability of the procedures for using the CMS tools. 

Completely 

Unacceptable Completely Acceptable 

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

7. Please rate the operational acceptability of the procedures for IM operations. 

Completely 

Unacceptable Completely Acceptable 

 1 2 3 4 5 6 7 

CAPTURE clearance ○ ○ ○ ○ ○ ○ ○

CROSS clearance ○ ○ ○ ○ ○ ○ ○

MAINTAIN clearance ○ ○ ○ ○ ○ ○ ○

Please comment on your choice here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

8. Please rate the impact of the addition of IM operations on expediting the traffic flow 

during this simulation. 

Much Less 

Expeditious No Impact Much More Expeditious

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________
Appendix E.5: Controller Post-Exp. Questionnaire 181 

9. When IM aircraft were in your sector, how acceptable was it for you to be 

responsible for maintaining safe / standard separation between these aircraft, while the 

flight crew was managing the spacing on a CAPTURE clearance. 

Completely 

Unacceptable Moderately Acceptable 

 Completely 

Acceptable 

 1 2 3 4 5 6 7 

IM aircraft spacing 

behind IM aircraft on the 

same route 

○ ○ ○ ○ ○ ○ ○

IM aircraft spacing 

behind a IM aircraft on a 

different route 

○ ○ ○ ○ ○ ○ ○

IM aircraft spacing 

behind a CMS aircraft on 

the same route 

○ ○ ○ ○ ○ ○ ○

IM aircraft spacing 

behind a CMS aircraft on 

a different route 

○ ○ ○ ○ ○ ○ ○

Please comment on your choices here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

 
Appendix E.5: Controller Post-Exp. Questionnaire 182 

When IM aircraft were in your sector, how acceptable was it for you to be responsible for 

maintaining safe / standard separation between these aircraft, while the flight crew was 

managing the spacing on a CROSS clearance. 

Completely 

Unacceptable Moderately Acceptable 

 Completely 

Acceptable 

 1 2 3 4 5 6 7 

IM aircraft spacing 

behind IM aircraft on the 

same route 

○ ○ ○ ○ ○ ○ ○

IM aircraft spacing 

behind a IM aircraft on a 

different route 

○ ○ ○ ○ ○ ○ ○

IM aircraft spacing 

behind a CMS aircraft on 

the same route 

○ ○ ○ ○ ○ ○ ○

IM aircraft spacing 

behind a CMS aircraft on 

a different route 

○ ○ ○ ○ ○ ○ ○

Please comment on your choices here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

 
Appendix E.5: Controller Post-Exp. Questionnaire 183 

When IM aircraft were in your sector, how acceptable was it for you to be responsible 

for maintaining safe / standard separation between these aircraft, while the flight crew 

was managing the spacing on a MAINTAIN clearance. 

Completely 

Unacceptable Moderately Acceptable 

 Completely 

Acceptable 

 1 2 3 4 5 6 7 

IM aircraft spacing 

behind IM aircraft on the 

same route 

○ ○ ○ ○ ○ ○ ○

IM aircraft spacing 

behind a IM aircraft on a 

different route 

○ ○ ○ ○ ○ ○ ○

IM aircraft spacing 

behind a CMS aircraft on 

the same route 

○ ○ ○ ○ ○ ○ ○

IM aircraft spacing 

behind a CMS aircraft on 

a different route 

○ ○ ○ ○ ○ ○ ○

Please comment on your choices here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

 
Appendix E.5: Controller Post-Exp. Questionnaire 184 

10. [If Center] How useful were the controller tools for managing the aircraft? 

Please choose the appropriate response for each item: 

 Not useful 

at all 

(ignored) 

Somewhat 

useful 

Very useful 

(essential) 

 1 2 3 4 5 6 7 

Delay in the meter list ○ ○ ○ ○ ○ ○ ○

Delay countdown timer ○ ○ ○ ○ ○ ○ ○

IM designators ○ ○ ○ ○ ○ ○ ○

Runway identifier ○ ○ ○ ○ ○ ○ ○

IM-specific meter list information ○ ○ ○ ○ ○ ○ ○

All-aircraft meter list information ○ ○ ○ ○ ○ ○ ○

Please comment on your choices here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

 
Appendix E.5: Controller Post-Exp. Questionnaire 185 

11. [If Feeder or Final] How useful were the CMS tools for managing the aircraft? 

Please choose the appropriate response for each item: 

 Not useful 

at all 

(ignored) 

Somewhat 

useful 

Very useful 

(essential) 

 1 2 3 4 5 6 7 

Speed advisories ○ ○ ○ ○ ○ ○ ○

Slot markers ○ ○ ○ ○ ○ ○ ○

Early/late indicators ○ ○ ○ ○ ○ ○ ○

Delay countdown timer ○ ○ ○ ○ ○ ○ ○

Timelines ○ ○ ○ ○ ○ ○ ○

IM designators ○ ○ ○ ○ ○ ○ ○

Spacing cones (bats) ○ ○ ○ ○ ○ ○ ○

Please comment on your choices here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

12. Describe any changes you would make to the IM operation. If appropriate, please 

comment on the IM and Target aircraft being in-trail vs. on merging routes, the delay 

between the issuance of the IM clearance and the initiation of the IM operation, etc. 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

13. What changes would you make to the displays for the IM aircraft? What additional 

information (if any) would you like with respect to the IM aircraft and how would you like 

it displayed? 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________
Appendix E.5: Controller Post-Exp. Questionnaire 186 

14. What information (if any) would you like available with respect to the Target 

aircraft and how would you like it displayed? 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

15. Please rate the operational acceptability of the IM phraseology used during the 

simulation. 

Not at all 

clear 

Moderately 

clear 

Very 

clear 

 1 2 3 4 5 6 7 N/A 

CAPTURE clearance ○ ○ ○ ○ ○ ○ ○ ○

CROSS clearance ○ ○ ○ ○ ○ ○ ○ ○

MAINTAIN clearance ○ ○ ○ ○ ○ ○ ○ ○

Reporting “IM paired” ○ ○ ○ ○ ○ ○ ○ ○

Check in with TRACON ○ ○ ○ ○ ○ ○ ○ ○

Amendments to IM 

clearance 

○ ○ ○ ○ ○ ○ ○ ○

IM clearance suspension ○ ○ ○ ○ ○ ○ ○ ○

IM clearance resumption ○ ○ ○ ○ ○ ○ ○ ○

IM engagement status 

reporting “Unable” 

○ ○ ○ ○ ○ ○ ○ ○

Please comment on your choices here: [mandatory comment if any ratings of 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

 
Appendix E.5: Controller Post-Exp. Questionnaire 187 

16. Did you experience any confusion due to the use of the Target aircraft’s call sign in 

the IM clearance? 

Yes 

No 

Please comment on your choice here and indicate what type of third party ID was used: 

[mandatory comment if Yes] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

17. Please rate the operational acceptability of using the Target aircraft’s call sign in 

the IM clearance. 

Completely 

Unacceptable Completely Acceptable 

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rating 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

18. Describe any suggestions to improve the clarity and/or completeness of the IM 

clearance phraseology. 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

19. Describe the challenges (if any) you perceive to the operational implementation of 

IM operations. 

______________________________________________________________________________

______________________________________________________________________________

_____________________________________________________________________ 

20. In order to issue an IM clearance in the TRACON, what information would you 

need and how would you like it displayed? 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 
Appendix E.5: Controller Post-Exp. Questionnaire 188 

21. Of the IM clearances and operations you experienced in this experiment 

(CAPTURE, CROSS, MAINTAIN), which one do you believe is the most useful, and when 

would you use it? 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

22. Of the IM clearances and operations you experienced in this experiment 

(CAPTURE, CROSS, MAINTAIN), which one do you believe is the least useful, and why? 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

23. Do you have any additional comments about the experiment? 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

 
Appendix E.6: Pilot Post-Exp. Questionnaire 189 

E.6 Pilot Post-Experiment Questionnaire 

1. Pilot ID: 

2. Describe any anomalies or inconsistencies in the simulation that affected your 

performance. 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

3. Was the workload required to operate the simulator much less than, the same as, or 

greater than the workload required to fly an actual aircraft? 

Much 

More 

Moderately 

More 

Slightly 

More 

The 

Same 

Slightly 

Less 

Moderately 

Less 

Much 

Less 

Workload 

Required 1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

4. Please rate how well the training prepared you to fly the simulator. 

Not At 

All 

Prepared 

 Moderately 

Prepared 

 Very 

Prepared 

1 2 3 4 5 6 7 

Please describe how simulator training can be improved: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

 
Appendix E.6: Pilot Post-Exp. Questionnaire 190 

5. Please rate how well the training prepared you to enter information into and 

interpret the information presented on the EFB. 

Not At 

All 

Prepared 

 Moderately 

Prepared 

 Very 

Prepared 

1 2 3 4 5 6 7 

Please describe how EFB training can be improved: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

6. Please rate how well the training prepared you to interpret the information 

presented on the CGD in the forward field of view. 

Not At 

All 

Prepared 

 Moderately 

Prepared 

 Very 

Prepared 

1 2 3 4 5 6 7 

Please describe how CGD training can be improved: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

7. Please rate how well the training prepared you to conduct an IM operation. 

Not At 

All 

Prepared 

 Moderately 

Prepared 

 Very 

Prepared 

1 2 3 4 5 6 7 

Please describe how IM operations training can be improved: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 
Appendix E.6: Pilot Post-Exp. Questionnaire 191 

8. Please rate the operational acceptability of the procedures for IM operations. 

Completely 

Unacceptable Completely Acceptable 

 1 2 3 4 5 6 7 

CAPTURE clearance ○ ○ ○ ○ ○ ○ ○

CROSS clearance ○ ○ ○ ○ ○ ○ ○

MAINTAIN clearance ○ ○ ○ ○ ○ ○ ○

Please comment on your choice here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

9. Please rate the overall acceptability of the IM procedures. 

Completely 

Unacceptable Completely 

Acceptable 

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

10. Please rate the impact of conducting an IM operation on your overall situational 

awareness during the arrival operations. 

Severely 

Degraded 

Situational 

Awareness 

 No 

Impact 

Greatly 

Improved 

Situational 

Awareness

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________
Appendix E.6: Pilot Post-Exp. Questionnaire 192 

11. How acceptable was it for you to be responsible for achieving the assigned spacing 

interval while the controller retained responsibility for the separation of aircraft? 

Completely 

Unacceptable Completely Acceptable 

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

12. Did following the IM commanded speed ever cause unexpected or undesired 

behavior? 

Yes 

No 

Please comment on your choice here: [mandatory comment if Yes] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

13. Describe any changes you would make to the IM operation. If appropriate, please 

comment on the IM and Target aircraft being in-trail vs. on merging routes, the delay 

between the issuance of the IM clearance and the initiation of the IM operation, etc. 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

14. Please rate the intuitiveness of entering IM clearance information into the EFB. 

 

Completely 

Unintuitive Completely Intuitive 

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 
Appendix E.6: Pilot Post-Exp. Questionnaire 193 

15. Please rate the usefulness of the following elements on the EFB display. 

Please choose the appropriate response for each item: 

 Not useful 

at all 

(ignored) 

Somewhat 

useful 

Very useful 

(essential) 

 1 2 3 4 5 6 7 

IM commanded speed ○ ○ ○ ○ ○ ○ ○

IM status (i.e., ARMED, 

SUSPENDED, PAIRED) 

○ ○ ○ ○ ○ ○ ○

IM messages ○ ○ ○ ○ ○ ○ ○

FAST/SLOW indicator ○ ○ ○ ○ ○ ○ ○

EARLY/LATE indicator ○ ○ ○ ○ ○ ○ ○

What changes would you make to the above display elements? [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________
Appendix E.6: Pilot Post-Exp. Questionnaire 194 

16. Please rate the usefulness of the following elements on the EFB display. 

Please choose the appropriate response for each item: 

Not useful 

at all (ignored) 

Somewhat 

useful 

Very useful 

(essential) 

 1 2 3 4 5 6 7 N/A 

Altitude filter ○ ○ ○ ○ ○ ○ ○ ○

Bearing and Range ○ ○ ○ ○ ○ ○ ○ ○

Target Route ○ ○ ○ ○ ○ ○ ○ ○

Merge Point / Way Point ○ ○ ○ ○ ○ ○ ○ ○

Target Ground Speed and Track ○ ○ ○ ○ ○ ○ ○ ○

Terminate Point ○ ○ ○ ○ ○ ○ ○ ○

Ownship Route ○ ○ ○ ○ ○ ○ ○ ○

What changes would you make to the above display elements? [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________ 

___________________________________________________________________________

___________________________________________________________________________ 
Appendix E.6: Pilot Post-Exp. Questionnaire 195 

17. Please rate the usefulness of the following elements on the CGD display (forward 

field of view). 

Please choose the appropriate response for each item: 

 Not useful 

at all 

(ignored) 

Somewhat 

useful 

Very useful 

(essential) 

 1 2 3 4 5 6 7 

Target aircraft call sign ○ ○ ○ ○ ○ ○ ○

IM commanded speed ○ ○ ○ ○ ○ ○ ○

IM status (i.e., ARMED, 

SUSPENDED, PAIRED) 

○ ○ ○ ○ ○ ○ ○

IM messages ○ ○ ○ ○ ○ ○ ○

FAST/SLOW indicator ○ ○ ○ ○ ○ ○ ○

What changes would you make to the above display elements? [mandatory comment] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________
Appendix E.6: Pilot Post-Exp. Questionnaire 196 

18. Please rate the effectiveness of the CGD in providing adequate information to 

conduct an IM operation. 

Completely 

Ineffective Completely Effective 

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

19. Please rate the operational acceptability of the IM phraseology used during the 

simulation. 

Not at 

all clear 

Moderately 

clear 

Very 

clear 

 1 2 3 4 5 6 7 N/A 

CAPTURE clearance ○ ○ ○ ○ ○ ○ ○ ○

CROSS clearance ○ ○ ○ ○ ○ ○ ○ ○

MAINTAIN clearance ○ ○ ○ ○ ○ ○ ○ ○

Reporting “IM paired” ○ ○ ○ ○ ○ ○ ○ ○

Check in with TRACON ○ ○ ○ ○ ○ ○ ○ ○

 Amendments to IM 

clearance 

○ ○ ○ ○ ○ ○ ○ ○

IM clearance suspension ○ ○ ○ ○ ○ ○ ○ ○

IM clearance resumption ○ ○ ○ ○ ○ ○ ○ ○

 IM engagement status 

reporting unable 

○ ○ ○ ○ ○ ○ ○ ○

Please comment on your choice here: [mandatory comment if any ratings of 1-3] 

___________________________________________________________________________

___________________________________________________________________________ 
Appendix E.6: Pilot Post-Exp. Questionnaire 197 

20. Did you experience any confusion when hearing your aircraft call sign used as the 

Target aircraft in an IM clearance issued to another aircraft? 

Yes 

No 

Did not hear this occur 

What was the impact (if any) and how quickly was it resolved? [mandatory comment if Yes] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

21. Please rate the operational acceptability of using the Target aircraft’s call sign in 

the IM clearance. 

Completely 

Unacceptable Completely Acceptable 

1 2 3 4 5 6 7 

Please comment on your choice here: [mandatory comment if rated 1-3] 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

22. Describe any suggestions to improve the clarity and/or completeness of the IM 

clearance phraseology. 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

23. How difficult do you think it would be for a typical crew to learn and integrate the 

IM spacing procedures into their current daily operational flight procedures? 

Very 

Difficult 

Moderately 

Difficult 

Slightly 

Difficult Neutral Slightly Easy 

Moderately 

Easy 

Very 

Easy 

1 2 3 4 5 6 7 

24. Describe the challenges (if any) you perceive to the operational implementation of 

IM operations. 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________
Appendix E.6: Pilot Post-Exp. Questionnaire 198 

25. Of the IM clearances and operations you experienced in this experiment 

(CAPTURE, CROSS, MAINTAIN), which one do you believe is the most useful, and when 

would you use it? 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

26. Of the IM clearances and operations you experienced in this experiment 

(CAPTURE, CROSS, MAINTAIN), which one do you believe is the least useful, and why? 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________ 

27. Do you have any additional comments about the experiment? 

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

___________________________________________________________________________

 
Appendix F.1: Controller Background Results 199 

Appendix F: Questionnaire Results 

F.1 Controller Background Questionnaire Results 

Participating controllers had a mean age of 57.4 years, standard deviation = 1.9. Groups 1 and 3 

had similar mean ages (group 1: mean = 57.3, standard deviation = 2.5; group 3: mean = 57.5, 

standard deviation = 1.3). 

Fifty percent of the participating controllers had worked en route ATC, and they had an average 

of 24 years (standard deviation = 6.2) of experience at en route facilities. Five of the eight (62.5%) 

participating controllers had worked TRACON with a mean experience of 23.8 years (standard 

deviation = 9.4) at TRACON facilities. Five of the eight (62.5%) participating controllers had 

worked in a Tower facility for an average of 7.4 years (standard deviation = 6.1). 

All controller subjects were retired with an average of 67.3 months (5 years, 7.3 months) (standard 

deviation = 30.3 months, or 2 years, 6.3 months) since retirement. Mean retirement times of group 

1 and group 3 differed by 31.5 months (group 1: mean = 51.5, standard deviation = 35.4; group 3: 

mean = 83, standard deviation =14.7). 

All controllers were Certified Professional Controllers (CPC). Additionally, 25% of the controllers 

were Front Line Managers (FLM), and 25% of the controllers were Traffic Management 

Coordinators (TMC). One controller was a Staff Specialist, and one controller was a Controllerin-Charge (CIC) and an on-the-job instructor. 

Seventy-five percent of controller subjects stated they used RNAV arrivals during their operational 

career. The remaining 25% had not used them operationally but had knowledge of RNAV arrivals 

and used them in instruction. Five of the eight (62.5%) controller subjects had little or no 

operational experience with RNP approaches. The remaining three controllers had operational and 

instructional experience with RNP approaches. Only one controller had participated in a previous 

IM experiment. 

 
Appendix F.2: Pilot Background Results 200 

F.2 Pilot Background Questionnaire Results 

The 24 participating pilots had a mean age of 57.5 years (standard deviation = 7.1). Groups 1 and 

3 had similar mean ages (group 1: mean = 58.4, standard deviation = 8.3; group 3: mean = 56.5, 

standard deviation = 6.0). 

Fifty percent of pilots reported they had the most experience with United Airlines. The other 50% 

reported a variety of other air carriers. The mean military and commercial flight experience of both 

groups in years was 34.4, standard deviation = 9.0. Groups 1 and 3 had similar mean years of 

experience (group 1: mean = 33.6, standard deviation = 9.0; group 3: mean = 35.2, standard 

deviation = 9.4). The mean commercial, multi-engine flight experience was 17382.3 hours 

(standard deviation = 6346.42). The difference between groups 1 and 3 was not operationally 

significant (group 1: mean = 18575, standard deviation = 7315.9; group 3: mean = 16189.58, 

standard deviation = 5252.61). Participating pilots listed experience in a wide variety of aircraft, 

reporting between 3 and 21 different aircraft (median = 8). 

Ninety-two percent of participating pilots had flown a commercial aircraft within the previous 4 

years, and 75% had flown a commercial aircraft in the last two months. 66.7% of participating 

pilots reported experience as an instructor, and 50% reported experience as a standards captain or 

line check airman. 

All pilots reported high levels of experience and currency with RNAV arrivals. Multiple pilots 

described their experience with RNAV arrivals as “extensive” and many comments describe daily 

use. All pilots report training or experience with RNP approaches, but 41.6% reported infrequent 

use, as little or less than one real-world RNP approach per year. Fifty percent of pilots had 

participated in a previous IM experiment. 
Appendix F.3: Controller Post-Run Results 201 

F.3 Controller Post-Run Questionnaire 

The controller post-run questionnaire was issued after each data collection run. Questions 1-3 were 

administrative and are not reported in this document. 

F.3.1 TLX Questions 

Questions 4-9 were adapted from the NASA Task Load Index (TLX) to assess controller workload 

using a 7 point Likert scale from “Very Low” to “Very High” for the Mental Demand, Physical 

Demand, Time Pressure, Effort, and Frustration subscales, and from “Good” to “Poor” for the 

Success subscale. These questions and the rating are defined in the ATD-1 Measures of 

Performance Specification (ref. 19). Response summaries, plots, and descriptive statistics can be 

found in the following pages, which provide additional detail for Section 5.6.2.

 
Appendix F.3: Controller Post-Run Results 202 

4. [TLX 1] How mentally demanding was the scenario? 

Mean Mental Demand responses were all less than or equal to 2.5 (standard deviation ≤ 1.6). There 

were no responses above ‘5’ on the Mental Demand subscale. All but one response above ‘3’ were 

given by the same Center controller. The additional ‘5’ response was due to an aircraft slowing 

excessively in the BASELINE operation, which forced the Final controller to instruct the following 

aircraft to go around. 

Mixed

Maintain

Cross

Capture

Baseline

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

7

6

5

4

3

2

1

TLX Ratings

Figure 80. Controller perception of mental demand by station by IM operation type. 

Table 42. Descriptive Statistics for Controller Perception of Mental Demand. 

Operation Position N Mean SD Min Median Max 

BASELINE 

Center 8 1.9 1.4 1 1 4 

Feeder 4 2.3 0.5 2 2 3 

Final 4 2.3 1.9 1 1.5 5 

Total 16 2.1 1.3 1 2 5 

CAPTURE 

Center 8 2.6 1.6 1 2 5 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 2.0 0.8 1 2 3 

Total 16 2.3 1.3 1 2 5 

CROSS 

Center 8 2.1 1.4 1 1.5 4 

Feeder 4 2.0 0.8 1 2 3 

Final 4 2.0 0.8 1 2 3 

Total 16 2.1 1.1 1 2 4 

MAINTAIN 

Center 8 1.9 1.1 1 1.5 4 

Feeder 4 1.8 0.5 1 2 2 

Final 4 2.5 0.6 2 2.5 3 

Total 16 2.0 0.9 1 2 4 

MIXED 

Center 8 2.1 1.5 1 1.5 5 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 2.0 0.8 1 2 3 

Total 16 2.0 1.2 1 2 5 

 
Appendix F.3: Controller Post-Run Results 203 

5. [TLX 2] How physically demanding was the scenario? 

Physical Demand response means for all operations and positions were less than or equal to 2.0 

(standard deviation ≤ 1.2). All Physical Demand responses above ‘2’ were given by the same 

Center controller identified in question 4. Three ‘4’ responses were given. 

Figure 81. Controller perception of physical demand by station by IM operation type. 

Table 43. Descriptive Statistics for Controller Perception of Physical Demand. 

Operation Position N Mean SD Min Median Max 

BASELINE 

Center 8 1.5 0.9 1 1 3 

Feeder 4 1.5 0.6 1 1.5 2 

Final 4 1.0 0.0 1 1 1 

Total 16 1.4 .7 1 1 3 

CAPTURE 

Center 8 2.0 1.1 1 2 4 

Feeder 4 1.5 0.6 1 1.5 2 

Final 4 1.0 0.0 1 1 1 

Total 16 1.6 0.9 1 1 3 

CROSS 

Center 8 2.0 1.1 1 2 4 

Feeder 4 1.5 0.6 1 1.5 2 

Final 4 1.0 0.0 1 1 1 

Total 16 1.6 0.9 1 1 4 

MAINTAIN 

Center 8 1.6 0.9 1 1 3 

Feeder 4 1.5 0.6 1 1.5 2 

Final 4 1.3 0.5 1 1 2 

Total 16 1.5 0.7 1 1 3 

MIXED 

Center 8 1.8 1.2 1 1 4 

Feeder 4 1.3 0.5 1 1 2 

Final 4 1.0 0.0 1 1 1 

Total 16 1.4 0.9 1 1 4 

 

Mixed

Maintain

Cross

Capture

Baseline

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

7

6

5

4

3

2

1

TLX Ratings

y
Appendix F.3: Controller Post-Run Results 204 

6. [TLX 3] How hurried/rushed was the pace of the scenario? 

All means were less than or equal to 2.3 (standard deviation ≤ 1.3). All but 6 responses provided 

were less than or equal to ‘3’; those higher responses were again provided by the controller 

identified in questions 4 and 5. Nine ‘3’ responses were provided. 

Mixed

Maintain

Cross

Capture

Baseline

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

7

6

5

4

3

2

1

TLX Ratings

Figure 82. Controller perception of time pressure by station by IM operation type. 

Table 44. Descriptive Statistics for Controller Perception of Time Pressure. 

Operation Position N Mean SD Min Median Max 

BASELINE 

Center 8 1.6 1.2 1 1 4 

Feeder 4 2.0 0.8 1 2 3 

Final 4 1.8 1.0 1 1.5 3 

Total 16 1.8 1.0 1 1 4 

CAPTURE 

Center 8 2.3 1.3 1 2 4 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 1.3 0.5 1 1 2 

Total 16 1.9 1.1 1 1.5 4 

CROSS 

Center 8 1.9 0.8 1 2 3 

Feeder 4 1.8 0.5 1 2 2 

Final 4 1.5 0.6 1 1.5 2 

Total 16 1.8 0.7 1 2 3 

MAINTAIN 

Center 8 1.8 1.2 1 1 4 

Feeder 4 1.5 0.6 1 1.5 2 

Final 4 1.3 0.5 1 1 2 

Total 16 1.6 0.9 1 1 4 

MIXED 

Center 8 2.0 1.3 1 1.5 4 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 1.0 0.0 1 1 1 

Total 16 1.7 1.1 1 1 4 

 
Appendix F.3: Controller Post-Run Results 205 

7. [TLX 4] How hard did you have to work to accomplish your level of performance? 

The mean Effort response was 2.1, standard deviation = 1.1. All means were less than or equal to 

2.3 (standard deviation ≤ 1.4). One of the two ‘5’ responses and seven of the nine ‘4’ responses 

were from the same Center controller identified in questions 4-6. The other ‘5’ response was again 

due to the unexpected go around mentioned in question 4. 

Figure 83. Controller perception of effort by station by IM operation type. 

Table 45. Descriptive Statistics for Controller Perception of Effort. 

Operation Position N Mean SD Min Median Max 

BASELINE 

Center 8 1.9 1.4 1 1 4 

Feeder 4 3.0 1.2 2 3 4 

Final 4 2.3 1.9 1 1.5 5 

Total 16 2.3 1.4 1 2 5 

CAPTURE 

Center 8 2.5 1.4 1 2 5 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 1.5 0.6 1 1.5 2 

Total 16 2.1 1.2 1 2 5 

CROSS 

Center 8 2.1 1.1 1 2 4 

Feeder 4 2.3 1.0 1 2.5 3 

Final 4 2.0 0.8 1 2 3 

Total 16 2.1 1.0 1 2 4 

MAINTAIN 

Center 8 1.9 1.1 1 1.5 4 

Feeder 4 2.8 0.5 2 3 3 

Final 4 2.0 0.8 1 2 3 

Total 16 2.1 1.0 1 1 4 

MIXED 

Center 8 2.0 1.3 1 1.5 4 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 1.8 1.0 1 1.5 3 

Total 16 1.9 1.1 1 1.5 4 

 

Mixed

Maintain

Cross

Capture

Baseline

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

7

6

5

4

3

2

1

TLX Ratings
Appendix F.3: Controller Post-Run Results 206 

8. [TLX 5] How insecure, discouraged, irritated, stressed, and annoyed were you? 

All means were less than or equal to 2.0 (standard deviation ≤ 1.5). Two ‘4’ responses were given, 

one by the controller identified in previous questions in the CAPTURE operation, one by another 

controller in the MAINTAIN operation. 

Figure 84. Controller perception of frustration by station by IM operation type.

Table 46. Descriptive Statistics for Controller Perception of Frustration. 

Operation Position N Mean SD Min Median Max 

BASELINE 

Center 8 1.1 0.4 1 1 2 

Feeder 4 2.0 0.0 2 2 2 

Final 4 1.5 1.0 1 1 3 

Total 16 1.4 0.6 1 1 3 

CAPTURE 

Center 8 1.6 1.1 1 1 4 

Feeder 4 1.3 0.5 1 1 2 

Final 4 1.0 0.0 1 1 1 

Total 16 1.4 0.8 1 1 4 

CROSS 

Center 8 1.1 0.4 1 1 2 

Feeder 4 1.0 0.0 1 1 1 

Final 4 1.0 0.0 1 1 1 

Total 16 1.1 0.3 1 1 2 

MAINTAIN 

Center 8 1.1 0.4 1 1 2 

Feeder 4 1.0 0.0 1 1 1 

Final 4 1.8 1.5 1 1 4 

Total 16 1.3 0.8 1 1 4 

MIXED 

Center 8 1.0 0.0 1 1 1 

Feeder 4 1.0 0.0 1 1 1 

Final 4 1.0 0.0 1 1 1 

Total 16 1.0 0.0 1 1 1 

Mixed

Maintain

Cross

Capture

Baseline

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

7

6

5

4

3

2

1

TLX Ratings
Appendix F.3: Controller Post-Run Results 207 

9. [TLX 6] How successful were you in accomplishing what you were asked to do? 

The mean Success response was 1.2, standard deviation = 0.6. All means were less than or equal 

to 2.3 (standard deviation ≤ 1.9). There was one ‘5’ response in the BASELINE condition, by the 

same Final controller mentioned in question 4, in response to the event where one aircraft slowed 

excessively on final forcing the following aircraft to go around. All other responses were less than 

or equal to ‘3’. 

Figure 85. Controller perception of success by station by IM operation type. 

Table 47. Descriptive Statistics for Controller Perception of Success. 

Operation Position N Mean SD Min Median Max 

BASELINE 

Center 8 1.0 0.0 1 1 1 

Feeder 4 1.8 1.0 1 1.5 3 

Final 4 2.3 1.9 1 1.5 5 

Total 16 1.5 1.1 1 1 5 

CAPTURE 

Center 8 1.0 0.0 1 1 1 

Feeder 4 1.3 0.5 1 1 2 

Final 4 1.0 0.0 1 1 1 

Total 16 1.1 0.3 1 1 2 

CROSS 

Center 8 1.0 0.0 1 1 1 

Feeder 4 1.3 0.5 1 1 2 

Final 4 1.0 0.0 1 1 1 

Total 16 1.1 0.3 1 1 2 

MAINTAIN 

Center 8 1.0 0.0 1 1 1 

Feeder 4 1.5 0.6 1 1.5 2 

Final 4 1.5 1.0 1 1 3 

Total 16 1.3 0.6 1 1 3 

MIXED 

Center 8 1.0 0.0 1 1 1 

Feeder 4 1.3 0.5 1 1 2 

Final 4 1.0 0.0 1 1 1 

Total 16 1.1 0.3 1 1 2 

Mixed

Maintain

Cross

Capture

Baseline

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

Final

Feeder

Center

7

6

5

4

3

2

1

TLX Ratings
Appendix F.3: Controller Post-Run Results 208 

F.3.2 CARS Questions 

Questions 10-16 used the Controller Acceptance Rating Scale (CARS) (ref. 21). A box plot of the 

CARS ratings is shown in Figure 86, and a summary of the controller post-run workload CARS 

ratings is shown in Table 48. 

 

Figure 86. Controller post-run acceptability (CARS) ratings by operation type. 

Table 48. Descriptive Statistics for Controller Acceptability Ratings. 

Operation Position N Mean SD Min Median Max 

BASELINE 

Center 8 9.1 0.6 8 9 10 

Feeder 4 8.5 0.6 8 8.5 9 

Final 4 8.5 0.6 8 8.5 9 

CAPTURE 

Center 8 9.1 0.6 8 9 10 

Feeder 4 8.8 0.5 8 9 9 

Final 4 9.0 0.0 9 9 9 

CROSS 

Center 8 9.1 0.6 8 9 10 

Feeder 4 8.5 0.6 8 8.5 9 

Final 4 8.8 0.5 8 9 9 

MAINTAIN 

Center 8 9.3 0.5 9 9 10 

Feeder 4 8.0 0.8 7 8 9 

Final 4 8.8 0.5 8 9 9 

MIXED 

Center 8 9.5 0.5 9 9.5 10 

Feeder 4 8.5 0.6 8 8.5 9 

Final 4 8.5 1.0 7 9 9 

 
Appendix F.3: Controller Post-Run Results 209 

F.3.3 Additional Questions 

Questions 17 – 19 asked controllers about changes to technique to use the new tools. Question 20 

asked controllers about the acceptability of speeds flown by the IM aircraft. 

17. To what degree did you modify your technique in order to use the CMS tools? 

This question asked controllers to rate the degree they modified their technique to use the CMS 

tools on a scale from 1 – 7 where ‘1’ was “Not at all,” ‘4’ was “Moderately,” and ‘7’ was 

“Completely.” The mean rating was 1.5 (standard deviation = 0.7). 

Figure 87. Modification of controller technique to use CMS tools. 

18. To what degree did you modify your technique in order to use the IM tools? 

This question asked controllers to rate the degree they modified their technique to use the IM tools 

on a scale from 1 – 7 where ‘1’ was “Not at all,” ‘4’ was “Moderately,” and ‘7’ was “Completely.” 

The mean rating was 1.5 (standard deviation = 0.7). 
Appendix F.3: Controller Post-Run Results 210 

Figure 88. Modification of controller technique to use IM tools. 

19. Did you have to change the way you worked to manage the IM aircraft? 

Table 49 displays the number of controllers of the 80 total respondents who chose each item. Of 

the 35 controllers who responded “Other” to question 19, 32 responded “None” or that no IM 

aircraft were in that scenario. Of the three other responses, two also responded yes to question 19 

item 7: “More voice communication with the IM aircraft, including speed instructions or vectors 

(compared with non-IM aircraft).” 

Table 49. Responses to How Controllers Changed Working with IM Aircraft. 

Response (compared with non-IM aircraft) N

Changed my general scan of the IM aircraft 10 

Changed the way I monitored separation of the IM aircraft 10 

Changed the way I monitored the IM aircraft spacing 12 

Issued different types of clearances to the IM aircraft 10 

Issued more clearances to the IM aircraft 18 

Less voice communication with the IM aircraft, including speed instructions or vectors 8 

More voice communication with the IM aircraft, including speed instructions or vectors 17 

Changed the way I coordinated with other controllers 0 

Considered different types of solutions to IM aircraft problems 5 

Other (If none, comment “None”) 35 

 
Appendix F.3: Controller Post-Run Results 211 

20. Please rate the acceptability of the speeds flown by the IM aircraft. 

This question asked controllers to rate the acceptability of the speeds flown by the IM aircraft on 

a scale from 1 - 7 where ‘1’ was “Completely Unacceptable” and ‘7’ was “Completely 

Acceptable.” The mean acceptability rating was 6.1 (standard deviation = 1.6). 

Figure 89. Pilot accept ability ratings of speeds flown by IM aircraft. 

21. Did you issue any of the clearance types listed below? 

Table 50 includes the number of controllers who used each clearance type. 

Of the five controllers who suspended an IM operation, two commented they did it by mistake and 

it was not needed. 

One of the two Resume clearances was a controller who suspended an aircraft after the pilot “said 

he needed to suspend,” and resumed IM when the pilot “said he could resume IM spacing.” 

Of the five Cancel clearance responses, two commended that the Cancel clearance was in response 

to incorrect application of the Maintain clearance. One controller commented “three aircraft had 

to be canceled,” another controller commented that an aircraft on downwind had to be canceled 

for excessive spacing, and another commented, “it was asking for a slower speed that the [aircraft] 

was unable to maintain.” 

There were no cases where a controller decided not to issue an IM clearance, but four of the 

controllers responded that the IM aircraft did not pair in their airspace. 

One response to the “Other” item was that the controller issued no IM clearances, while all other 

comments were “None.” 
Appendix F.3: Controller Post-Run Results 212 

Table 50. Clearance Types Issued. 

Clearance N

Suspended IM spacing 5 

Resumed IM spacing 2 

Canceled IM spacing 5 

Decided not to issue an IM clearance 0 

Aircraft never paired 4 

Other (If none, comment “None”) 65 

22. [If Center] Were there any issues with the IM clearance phraseology? 

Of the 40 Center controller responses, all but one controller responded “No.” The only controller 

who responded “Yes” commented that there were no issues. 

23. [If Center] What type of third party ID was used to issue the IM clearance? Please 

note the aircraft call sign (if able) and describe any issue(s) in the box on the right. 

Table 51 lists the types of third party call signs used to issue IM clearances. Note: the 25 

“Other” responses all either stated no IM clearance issued or “none.” See Table 33 in 

Section 5.8.4.4 for responses to the same question as reported by pilots. 

Table 51. Type of Third Party ID Used to Issue the IM Clearance. 

Third Party ID N

Call sign (e.g., Brickyard 123) 18 

Alpha-numeric (e.g., R P A 123) 0 

Phonetic (e.g., Romeo Papa Alpha 123) 0 

Other 22 

24. Describe any unusual or unexpected event(s). 

Most comments were “none” or a phrase with the same meaning. One controller mentioned a 

situation with an aircraft that had to cancel IM, also mentioned in question 21, and another 

controller mentioned an aircraft out of parameters for a Maintain clearance. 

 

 
Appendix F.4: Pilot Post-Run Results 213 

F.4 Pilot Post-Run Questionnaire Results 

The controller post-run questionnaire was issued after each data collection run. Questions 1 – 4 

were administrative and are not reported in this document. 

F.4.1 Modified Cooper-Harper ratings 

Questions 5 – 10 were used to provide Modified Cooper-Harper (MCH) subjective workload 

ratings. The figures in this section provide additional detail for Section 5.8.2, where these workload 

ratings are discussed. See Table 31 for a summary of these workload ratings by operation. 

1 2 3 4 5 6 7 8 9 10

40

30

20

10

0

Pilot Workload Ratings

Frequency

Figure 90. Flight crew perception of workload during BASELINE operation.

1 2 3 4 5 6 7 8 9 10

40

30

20

10

0

Pilot Workload Ratings

Frequency

Figure 91. Flight crew perception of workload during CAPTURE operation.
Appendix F.4: Pilot Post-Run Results 214 

1 2 3 4 5 6 7 8 9 10

40

30

20

10

0

Pilot Workload Ratings

Frequency

Figure 92. Flight crew perception of workload during CROSS operation.

1 2 3 4 5 6 7 8 9 10

40

30

20

10

0

Pilot Workload Ratings

Frequency

Figure 93. Flight crew perception of workload during MAINTAIN operation.

1 2 3 4 5 6 7 8 9 10

40

30

20

10

0

Pilot Workload Ratings

Frequency

Figure 94. Flight crew perception of workload during MIXED operation.
Appendix F.4: Pilot Post-Run Results 215 

F.4.2 Acceptability ratings 

Questions #11 - #18 asked the flight crew to respond to a range of acceptability issues. 

11. Please rate the overall acceptability of IM during the scenario you just completed. 

The figures in this section provide additional detail for Section 5.8.1.1, where these acceptability 

ratings are discussed. Ratings were given on a scale from ‘1’ as “Completely Unacceptable” to ‘7’ 

as “Completely Acceptable.” 

Most comments in response to this question included at least one of the following three concerns: 

• Speed commands were too frequent. 

• Speed commands required large control adjustments for compliance with the command 

itself or for altitude restriction compliance. 

• Speed commands were outside the performance capabilities of the aircraft. 

Selected comments: 

• “Was good until the turn to final. Received 4 speed reductions in about 30 seconds from 

180 to 140 - a speed below the aircraft's capability (final approach speed 144kts). The 

number of speed changes in rapid succession caused us to not set our missed approach 

altitude. This caused the aircraft to not leave 7000' at the FAF. The delay in starting down 

caused us to have a high sink rate approach down to 500'. The turn to final is a very bad 

place to have this distraction.” 

• “The IM speed changes above FL180 were a little too excessive. They are manageable 

from a pilot’s perspective, however the passenger comfort would be marginal due to the 

constant application of speed brakes and/or thrust changes.” 

• “Some speed reductions required full spoilers to make altitude restrictions.” 

• “Multiple speed changes in short period of time immediately after IM Pairing: in 1-2 

minutes had IM speed commands of 280-300-290-250.” 

• “IM speed changes were ignored around 9 mile final due to task prioritization in cockpit.” 

• “Approaching the FAF crew task loading gets fairly high. I failed to notice that the speed 

brakes were still deployed.” 

Figure 95. Flight crew acceptability of BASELINE operation.

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency
Appendix F.4: Pilot Post-Run Results 216 

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 96. Flight crew acceptability of CAPTURE operation.

Figure 97. Flight crew acceptability of CROSS operation.

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 98. Flight crew acceptability of MAINTAIN operation.

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency
Appendix F.4: Pilot Post-Run Results 217 

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 99. Flight crew acceptability of MIXED operation. 

12. Please rate the acceptability of IM during each segment of flight during the scenario 

you just completed. 

The figures in this section provide additional detail for Section 5.8.1.2, were given on a scale from 

‘1’ as “Completely Unacceptable” to ‘7’ as “Completely Acceptable.” 

In the cruise and initial descent phases of flight (>18,000 feet), three pilots gave IM acceptability 

ratings of ‘1’, echoing concerns from question 11 about excessive speed command frequency and 

size. One pilot gave a rating of ‘2’, commenting on poor setup pre-IM. Two pilots gave ratings of 

‘3’, again echoing concerns from question 11 about excessive speed commands. Five pilots gave 

ratings of ‘4’. 

In the mid-level segment (18,000 – 11,000 feet), one pilot rated acceptability as ‘1’, commenting 

about large speed command changes in a CAPTURE scenario, from 230 to 290 to 240 knots within 

45 seconds. Five pilots gave ratings of ‘2’, all commenting on large speed reductions, which caused 

two pilots to overshoot altitude crossing restrictions. Seven pilots gave ratings of ‘3’, commenting 

about large and frequent changes in the commanded speed which, again, made descending to meet 

altitude restrictions in the mid-level segment difficult or impossible. Twelve pilots (7.3% of all 

mid-level segment responses) gave a rating of ‘4’. 

In the lower-level segment (Surface – 11,000’), overall, pilots reported excessive speed command 

changes which caused many pilots to change their configuration at undesirable points in the 

approach. Two pilots rated acceptability as ‘1’, commenting on “major speed changes” nearing the 

FAF. Five pilots rated acceptability as ‘2’, with most pilots again commenting on the excessive 

speed changes on final. One of those five pilots rated acceptability in the lower segment as a ‘2’ 

for confusion in communication with the feeder approach controller over the suspension of a 

clearance. Seventeen pilots (10.4% of all lower-level segment responses) gave a rating of ‘4’. 
Appendix F.4: Pilot Post-Run Results 218 

Table 52. Descriptive Statistics for Flight Crew Acceptability Ratings. 

Operation Altitude N Mean SD Min Median Max 

CAPTURE 

> 18,000’ 46 6.2 1.2 3 7 7 

11,000 – 18,000’ 46 5.9 1.5 1 7 7 

Surface – 11,000’ 46 6.0 1.5 1 6 7 

CROSS 

> 18,000’ 48 6.4 1.2 1 7 1 

11,000 – 18,000’ 48 6.3 1.0 3 7 7 

Surface – 11,000’ 48 5.8 1.3 2 6 7 

MAINTAIN 

> 18,000’ 42 6.1 1.5 1 7 7 

11,000 – 18,000’ 42 5.8 1.5 2 6 7 

Surface – 11,000’ 42 6.3 1.1 1 7 7 

MIXED 

> 18,000’ 48 6.7 0.6 5 7 7 

11,000 – 18,000’ 48 6.2 1.4 2 7 7 

Surface – 11,000’ 48 5.3 1.6 1 6 7 

>18,000 18,000-11,000 11,000-6,000

7

6

5

4

3

2

1

Altitude (ft)

Acceptability Ratings

Figure 100. Flight crew acceptability of CAPTURE operation by phase of flight.

>18,000 18,000-11,000 11,000-6,000

7

6

5

4

3

2

1

Altitude (ft)

Acceptability Ratings

Figure 101. Flight crew acceptability of CROSS operation by phase of flight.
Appendix F.4: Pilot Post-Run Results 219 

>18,000 18,000-11,000 11,000-6,000

7

6

5

4

3

2

1

Altitude (ft)

Acceptability Ratings

Figure 102. Flight crew acceptability of MAINTAIN operation by phase of flight.

>18,000 18,000-11,000 11,000-6,000

7

6

5

4

3

2

1

Altitude (ft)

Acceptability Ratings

Figure 103. Flight crew acceptability of MIXED operations by phase of flight.

 
Appendix F.4: Pilot Post-Run Results 220 

13. Please rate the acceptability of the use of voice communications to provide the IM 

clearance(s) during the scenario you just completed. 

The figures in this section provide additional detail for Section 5.8.4.5, where these acceptability 

ratings are discussed. Ratings were given on a scale from ‘1’ as “Completely Unacceptable” to ‘7’ 

as “Completely Acceptable.” Eight pilots commented that data link communications would have 

helped avoid confusion and congestion in verbal communication. One pilot also commented, “Not 

all airline IDs are known to flight crews.” 

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 104. Flight crew acceptability of voice communication for BASELINE operation.

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 105. Flight crew acceptability of voice communication for CAPTURE operation.
Appendix F.4: Pilot Post-Run Results 221 

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 106. Flight crew acceptability of voice communication for CROSS operation.

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 107. Flight crew acceptability of voice communication for MAINTAIN operation.

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 108. Flight crew acceptability of voice communication for MIXED scenario. 

 
Appendix F.4: Pilot Post-Run Results 222 

14. Please rate the overall effectiveness with which relevant information, including 

operational plans, decisions, and changes in aircraft state were communicated between 

yourself and your crew member. 

Figure 109 summarizes responses to question 14, given on a scale from ‘1’ as “Completely 

Ineffective” to ‘7’ as “Completely effective.” Responses were consistent across all operations 

and most pilots reported no issues in cockpit communication, though one crew reported entering 

the wrong approach into the FMS and some crews were confused about specific speed 

assignments. 

Figure 109. Perceived flight crewmember communication effectiveness. 

15. Please rate the operational acceptability of the IM commanded speeds. 

The figures in this section provide additional detail for Section 5.8.1.7, where these acceptability 

ratings are discussed. Ratings were given on a scale from ‘1’ as “Completely Unacceptable” to ‘7’ 

as “Completely Acceptable.” Most pilots who gave ratings less than or equal to ‘3’ reported 

excessively large and frequent speed changes, which resulted in missed speed and altitude 

restrictions, undesirable and inefficient configuration changes, and speeds below the capabilities 

of the aircraft. In reference to the large and frequent speed command changes, one comment 

mentions, “We’d be slamming passengers around.” Pilots commented on frequent speed changes 

near the FAF as distracting and slow speeds further from the FAF as inefficient, requiring gear and 

flap extension. 

Selected comments: 

• “IM commanded a large speed reduction from 190 to 130 kts resulting in full configuration 

early. Then IM commanded a faster speed that did not require gear and full flaps. This 
Appendix F.4: Pilot Post-Run Results 223 

causes large fuel inefficiencies carrying more drag than necessary. Attempted to decelerate 

as slow as possible to mitigate this.” 

• “IM speeds were logical in hindsight but very little SA in real time.” 

• “I think the typical airline pilot would balk at such large speed changes approaching the 

FAF.” 

• “Received and IM speed reduction of 40 knots 4 miles prior to a hard altitude waypoint. It 

is challenging to meet both speed and altitude constraints in such a short distance.” 

• “Some of these were pretty significant in the initial descent phase, like we were chasing 

speeds. Commanded speed changes prior to altitude constraints were incompatible with 

VNAV use, had to resort to V/S or FLCH more than once. Speed brake usage was 

frequent.” 

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 110. Flight crew acceptability of IM commanded speeds for CAPTURE operation. 

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 111. Flight crew acceptability of IM commanded speeds for CROSS operation.
Appendix F.4: Pilot Post-Run Results 224 

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 112. Flight crew acceptability of IM commanded speeds for MAINTAIN operation.

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 113. Flight crew acceptability of IM commanded speeds for MIXED operation. 

 
Appendix F.4: Pilot Post-Run Results 225 

16. Please rate the acceptability of the frequency of IM speed changes. 

The figures in this section provide additional detail for Section 5.8.1.8, where these acceptability 

ratings are discussed. Ratings were given on a scale from ‘1’ as “Completely Unacceptable” to ‘7’ 

as “Completely Acceptable.” Comments associated with low ratings echo previous responses 

describing unacceptable frequencies of speed changes in the low-altitude segment. 

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 114. Flight crew acceptability of IM speed change frequency for CAPTURE ops.

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 115. Flight crew acceptability of IM speed change frequency for CROSS operation.
Appendix F.4: Pilot Post-Run Results 226 

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 116. Flight crew acceptability of IM speed change frequency for MAINTAIN ops.

1 2 3 4 5 6 7

40

30

20

10

0

Pilot Acceptability Ratings

Frequency

Figure 117. Flight crew acceptability of IM speed change frequency for MIXED operation. 

 
Appendix F.4: Pilot Post-Run Results 227 

17. Please rate the acceptability of the amount of head down time required of the Pilot 

Monitoring to input information from the IM clearance(s) into the EFB. 

The figures in this section provide additional detail for Section 5.8.1.9, where these acceptability 

ratings are discussed. Ratings were given on a scale from ‘1’ as “Completely Unacceptable” to ‘7’ 

as “Completely Acceptable.” 

Below are comments associated with ratings less than ‘4’: 

• “The more changes to IM speeds the more heads down time needed, not good.” 

• “Heads down time required for canceled and revised clearances below 10,000’ AGL 

are not acceptable and should not be considered a procedure ... aside from the normally 

high amount of heads down associated without aural and visual cues for CGD 

changes.” 

• “More speed adjustments on IM means more heads down time ... AURAL OR VISUAL 

HEADS UP alerting system will be essential.” 

• “Too much heads down after pairing.” 

• “Acceptable but this will have to be fixed in future iterations ... one more time AURAL 

and/or Visual alerting system for CGD changes.” 

1 2 3 4 5 6 7

25

20

15

10

5

0

Pilot Acceptability Ratings

Frequency

Figure 118. Flight crew acceptability of head down time to enter CAPTURE clearance.

1 2 3 4 5 6 7

25

20

15

10

5

0

Pilot Acceptability Ratings

Frequency

Figure 119. Flight crew acceptability of head down time to enter CROSS clearance.
Appendix F.4: Pilot Post-Run Results 228 

1 2 3 4 5 6 7

25

20

15

10

5

0

Pilot Acceptability Ratings

Frequency

Figure 120. Flight crew acceptability of head down time to enter MAINTAIN clearance.

1 2 3 4 5 6 7

25

20

15

10

5

0

Pilot Acceptability Ratings

Frequency

Figure 121. Flight crew acceptability of head down time to enter MIXED clearance.

 
Appendix F.4: Pilot Post-Run Results 229 

18. Please rate the acceptability of the IM procedures for the events in this scenario. 

The figures in this section provide additional detail for Section 5.8.1.4, where these acceptability 

ratings are discussed. Ratings were given on a scale from ‘1’ as “Completely Unacceptable” to ‘7’ 

as “Completely Acceptable.” One comment provides a good summary of responses to this 

question: “I don’t think the problem is a procedural one but rather a computer issue.” 

Figure 122. Flight crew acceptability of CAPTURE procedure.

1 2 3 4 5 6 7

25

20

15

10

5

0

Pilot Acceptability Ratings

Frequency

Figure 123. Flight crew acceptability of CROSS procedure. 

1 2 3 4 5 6 7

25

20

15

10

5

0

Pilot Acceptability Ratings

Frequency
Appendix F.4: Pilot Post-Run Results 230 

1 2 3 4 5 6 7

25

20

15

10

5

0

Pilot Acceptability Ratings

Frequency

Figure 124. Flight crew acceptability of MAINTAIN procedure.

1 2 3 4 5 6 7

25

20

15

10

5

0

Pilot Acceptability Ratings

Frequency

Figure 125. Flight crew acceptability of MIXED procedure. 

 
Appendix F.4: Pilot Post-Run Results 231 

F.4.3 Completeness. 

Questions #19 - #23 provide additional detail to the analysis of IM completeness in Section 5.8.1.6 

and any actions taken during the IM operation.

19. Please rate the completeness of the IM procedures for the events in this scenario. 

The figures in this section provide additional detail for Section 5.8.1.6., where these acceptability 

ratings are discussed. Ratings were given on a scale from ‘1’ as “Not at all complete” to ‘4’ as 

“Somewhat complete” to ‘7’ as “Very complete.” 

1 2 3 4 5 6 7

30

25

20

15

10

5

0

Completeness Ratings

Frequency

Figure 126. Flight crew perception of procedure completeness for CAPTURE operation.

1 2 3 4 5 6 7

30

25

20

15

10

5

0

Completeness Ratings

Frequency

Figure 127. Flight crew perception of procedure completeness for CROSS operation.
Appendix F.4: Pilot Post-Run Results 232 

1 2 3 4 5 6 7

30

25

20

15

10

5

0

Completeness Ratings

Frequency

Figure 128. Flight crew perception of procedure completeness for MAINTAIN ops.

1 2 3 4 5 6 7

30

25

20

15

10

5

0

Completeness Ratings

Frequency

Mixed

Figure 129. Flight crew perception of procedure completeness for MIXED operation.

 
Appendix F.4: Pilot Post-Run Results 233 

20. Were there any missing steps in the IM procedures? 

All comments associated with “Yes” responses: 

• “Suspend command not given when we were given an airspeed assignment.” 

• “Again, the IM system went from ARMED to SUSPENDED without either pilot putting 

into the EFB. Also, the IM commanded airspeed of 140 KIAS was below our minimum 

VRef speed.” 

• “Give a speed command if ATC cancels a clearance.” 

• “Limited information for crew to know ‘TGT BAD ROUTE’ would never pair without 

further intervention.” 

• “Recommend a procedure change to ignore speed changes within 1 mile of the FAF.” 

• “Was given a ‘capture’ clearance before receiving a descend-via clearance.” 

• “Was issued IM clearance then assigned a speed. Then we became paired. Unsure whether 

to fly assigned speed or IM clearance.” 

• “When ATC cancels an IM clearance, it would behoove them to give us an interim airspeed 

target until we get the new clearance issued, read back, entered and armed.” 

• “With no pilot action, CGD went from suspended to Paired”. 

21. Did the IM procedures contain extra steps that were unnecessary? 

All comments associated with “Yes” responses: 

• “Again, the IM commanded airspeed change from 270-280-260 noted above.” 

• “I didn't understand why IM commanded speed was slower than the speed that ATC wanted 

to the IF for the RNAV Z 16L.” 

• “Required gear extension too soon.” 

• “Should have just requested a speed change from 210-270 instead of 210-230-240-250-

260-270 in a short time period of about 20 seconds.” 

• “Too much talking. Need data link.” 

• “We put gear and flaps down to accommodate the speed reduction, then we raised flaps for 

the speed increase that was required. 

• “We were assigned 250kts by ATC while we were waiting to be paired. Once pairing was 

available there was some confusion as to whether or not we should execute the pairing 

because we were previously assigned a speed by ATC. We called ATC to confirm that we 

should pair and he said yes, comply with the previous pairing instructions.” 

22. Were the IM procedural steps logical and easy to follow? 

All comments associated with “No” responses: 

• “IM logic was tough to follow because of the scope of speed changes.” 

• “Large A/S and too many A/S changes on final in a short period of time. It seemed like it 

should have delayed the speed command in order to minimize the number of close in speed 

changes.” 

• “Limited information for crew to know ‘TGT BAD ROUTE’ would never pair without 

further intervention.” 
Appendix F.4: Pilot Post-Run Results 234 

• “Major speed reduction following suspension of IM spacing caused aircraft to be high on 

the path and was unable to meet an altitude restriction. 

• “IM commanded 250 knots then 210 knots before I could set 250 knots.” 

• “No issues here.” 

• “Not easy to follow approaching the FAF, received 3 speed up commands right at FAF we 

still needed to finish configuring the Aircraft. Got a little behind and didn't notice that the 

speed brake was still deployed. The PM caught it.” 

• “Not logical within 1 mile of FAF.” 

• “Not on short final. Spacing was too close and unacceptable.” 

• “Not sure why we got the discontinue spacing direction.” 

• “[Nothing significant to report.]” 

• “There was confusion on my part as to why ATC was commanding a speed faster than IM 

commanded speed.” 

23. Did you take any of the actions listed below (suspend, resume, cancel, not pair)? 

Most pilots (75%, 18/24) encountered SUSPEND. All but one response to “Not PAIR” were from 

the BASELINE operation, where no IM clearances were issued. Most “Other” responses were, 

“None” though two pilots reported ignoring IM speed commands after or within one mile of the 

FAF. 

- SUSPEND: 18 (Approximately 85% ATC, 10% software, < 5% by pilots) 

- RESUME: 3 

- CANCEL: 9 (Approximately 75% ATC, 25% flight crew due to IM speed changes) 

- Not PAIR: 7 

 
Appendix F.4: Pilot Post-Run Results 235 

F.4.4 Phraseology, Third-Party ID, and Unusual Occurrences

Questions #24 - #27 had the flight crew respond to issues about the IM phraseology, the use of 

third-party ID when issuing IM clearances, use of touch-screen devices, and unusual events. 

24. Were there any issues with the IM phraseology? 

Table 53. Percent of Pilots Who Reported an Issue with the IM Phraseology 

Operation N % Reporting Phraseology Issue 

CAPTURE 46 4% 

CROSS 48 4% 

MAINTAIN 42 31% 

MIXED 48 10% 

- CAPTURE: 

o “Was given a ‘capture’ clearance before receiving a ‘Descend Via’ clearance.” 

o “We were given ATC speed commands after paired, but the controller never used the 

"suspend" word. We were not expecting to be suspended, nor did we see a reason for 

it. Once the confusion was cleared up with the next controller, we resumed IM with no 

further IM issues.” 

- CROSS: 

o “They paired us behind Lufthansa 1434. This was not a choice on our EFB so we had 

to ask ATC what the 3 letter code for Lufthansa was. We would have never guessed 

that it was DLH. A suggestion would be to list the entire airline name as well as the 3 

letter code to help eliminate some of the confusion.” 

o “Communication system clipped clearance, I had to verify with controller. 130 seconds 

sounded like 103 seconds.” 

- MAINTAIN: 

o “The IM Maintain clearance included the Target's route, which should have been 

redundant.” 

o “Initial ATC clearance was to maintain 111 seconds behind, but changed to maintain 

current time.” 

o “I question if we need to check in with the next controller and tell him we are paired as 

he may need a little reminder.” 

o “ATC issued an incorrect IM clearance. We were maintain 96 seconds, this should have 

a capture clearance. You cannot input into the CGD a maintain seconds.” 

o “Given a MAINTAIN clearance to "maintain current spacing" with Target aircraft. Our 

assumption was that this meant ‘time’, not ‘miles’. However, after the fact, we request 

clarification from ATC.” 

o “Controller never used SUSPEND wording but used I believe "discontinue." 

o “Given an airspeed with the "S" word (suspend). We queried the controller if he wanted 

us to suspend and he quickly came back with an affirmative.” 

o On initial contact for an interval spacing clearance ATC should automatically provide 

the phonetic identifier for the Target aircraft ID and not assume we know every airline’s 

three letter identifier.” 
Appendix F.4: Pilot Post-Run Results 236 

- MIXED: 

o “The initial IM cross clearance did not have the Target’s route, which turned out to not 

be the same as ours. This was clarified with a radio call.” 

o “Received initial IM crossing clearance from ATC, then received an ATC assigned 

speed before pairing, then paired and flew IM speeds. Later ATC expressed confusion 

whether we were flying IM speeds or ATC assigned speeds. (Pairing radio call was 

made). Confusion could exist when having simultaneous ATC clearances. Is this a 

potential threat/conflict area that needs further clearance verification procedures?” 

o “We had to ask from the get go our aircraft to follow’s proper name for IM input.” 

o “Not with us, but another controller/aircraft stumble over cancel maintain IM. 

Controller did not know what speed to assign to the canceled IM aircraft.” 

25. What type of third party ID was used to issue the IM clearance? Please note the 

aircraft call sign (if able) and describe any issues in the box on the right. 

The type of third party ID used to issue the IM clearance is shown in Table 54 as reported by the 

pilots. During three flights, pilots reported that both the call sign and alpha-numeric identifier were 

used in the same clearance. In addition, pilots received a clearance with the Target aircraft’s call 

sign and had to query ATC for the three letter identifier during six flights. Of the 24 subject pilots, 

four reported experiencing confusion when their aircraft call sign was used as the Target aircraft 

in an IM clearance issued to another aircraft, 17 did not experience confusion due to this, and three 

did not hear this occur. Five pilots suggested that the phonetic identifier always be used for the 

Target aircraft during IM clearance issuance. In addition, seven pilots mentioned issues with 

obscure call signs not being issued with the phonetic identifier, or suggested modifying the EFB 

to include the full call sign for Target aircraft. 

Table 54. Type of Third Party ID Used to Issue the IM Clearance 

Third Party ID N 

Call sign (e.g., Brickyard 123) 158 

Alpha-numeric (e.g., R P A 123) 11 

Phonetic (e.g., Romeo Papa Alpha 123) 4 

Other 14 

26. Were there any issues with using a touch-screen device during this scenario? 

Twenty-six of the 290 responses indicated an issue with a touch screen device, however they all 

referred to simulation artifacts and not flight crew interaction with the IM equipment (which was 

the intent of the question). 

27. Describe any unusual or unexpected events. Please include time, location, and 

aircraft call signs if able. 
Appendix F.4: Pilot Post-Run Results 237 

Approximately 35% of comments were “none” or similar. Below is a selection of other comments 

in response to this question, excluding issues previously noted: 

• “MAINTAIN IM clearance was not executable above FL290, but did not show that it was 

awaiting anything.” 

• “A large speed change 16 mile final with everything showing normal fast/slow meter and 

Early/late showing normal. A compression buffer should be written mileage or time to 

FAF. Aircraft should not have to fully configure on a 16 mile final.” 

• ATC assigned speed 230 w/o clarifying our IM clearance. Everything was resolved, but 

resulted in extra communication and crossing TWNSN +500ft high. 

• “CROSS clearance. Our Target aircraft was -5000 ft. when we went active IM. Immediate 

speed reduction to 230 with gradual increase to 250. Is there any algorithm for IAS versus 

altitude? It would seem we are erroneously slowed and then accelerated as we approached 

the altitude of the Target aircraft.” 

• “Early/Late indicator was blank until well into approach (ARCHY- 13000 ft.). Use of speed 

braked needed to attain desired speed changes, drove us 2300+ below VNAV path (but IM 

seems to do better being low than being high).” 

• “Just before LONGS CGD gave us 210 kts. Noticed EFB had an IM SPEED LIMITED 

note and we were showing a little early. It didn’t make sense that the CGD could not slow 

us a little more at that point to keep from having the later rapid series of speed reductions 

with a final assigned speed below the aircraft capability.” 

• “Speed change algorithm could not hack the ‘slowing in descent’ speed change. Started 

out with arrows balanced, but soon got a ‘too fast’ cue with speed exceeding 20 kts fast on 

indicator. PF reverted to V/S 0 to capture the commanded speed, then reengaged VNAV 

and used speed brakes to make altitude constraints.” 

 
Appendix F.5: Controller Post-Exp. Results 238 

F.5 Controller Post-Experiment Questionnaire Results 

Post-experiment questionnaires were completed by all participants after completion of the last 

post-run questionnaire. 

3. Describe anomalies or inconsistencies in the simulation that affected your 

performance. 

Comments other than “none”: 

• “When the aircraft were given direct a fix and then took too long to start turning. Some 

planes started down right away and other planes on the same arrival waited to the last 

second to descend.” 

• “The only think I would like to see changed is the leader line position on the west side. 

When you put the leader at 9 o’clock it covers the tag.” 

• “Some of the airspace maps need additional markings to aid the controller. All arrival maps 

need to depict the fix on the RNAV Z approached where there is a 210 speed restriction. 

This allows aircraft that are late to fly faster but gives the reference point where 210 kts is 

needed. The maps for the 16L scenarios do not have the runway threshold markings that 

were updated from my previous week on the other flows.” 

• “They were consistent, sometimes the mac pilot couldn’t respond fast enough.” 

4. Please rate how well the training prepared you to use the 1) simulator, 2) KDEN 

airspace, 3) metering concept, 4) CMS tools, 5) IM operation, and 6) phraseology: 

This question asked controllers to rate their training on a scale from ‘1’ as “Not at all prepared,” 

to ‘4’ as “Moderately Prepared,” to ‘7’ as “Very Prepared.” The only ‘1’ rating was given for the 

“Simulator” training, and that controller did not clarify in his comment in question 5, only 

commenting on phraseology training, which that controller rated ‘3’. The same controller rated 

CMS Tool training and IM Operation training ‘4’. Table 55 and Figure 130 show summaries of 

responses to this question. 

Table 55. Descriptive Statistics for Post-Experiment Controller Training Ratings 

Training Type N Mean SD Min Median Max 

Simulator 8 6.1 2.1 1 7 7 

Airspace 8 6.3 0.7 5 6 7 

Metering Concept 8 6.1 0.8 5 6 7 

CMS Tools 8 6.0 1.1 4 6 7 

IM Operation 8 6.1 1.1 4 6.5 7 

Phraseology 8 5.8 1.6 3 6.5 7 
Appendix F.5: Controller Post-Exp. Results 239 

Figure 130. Controller post-experiment ratings of training by type of training. 

5. Please describe how the academic and hands-on training can be improved. 

All comments: 

• “Providing incoming controllers copies of the arrival routes in advance.” 

• “Send all the info in an email to controllers. That way they can get an idea of what is 

coming and prepare better. Also run 1 problem and let an expert show you how it is done. 

They can show you all the things you need to know and how to do it efficiently.” 

• “Knowing more about the altitudes available would be beneficial if and or when you need 

to vector to salvage a sequence.” 

• “Emphasize that speed advisories are just that, advisories, and if something else is required 

to meet the schedule then take action and get the aircraft in the slot markers, then resume 

following the advisories.” 

• “It was comprehensive.” 

• “I would suggest not switching controllers week to week. I would suggest having the same 

controllers so they become more familiar with the operation.” 

• “Have the instructors ask more often if there are any questions. At times you are not sure 

if you are to speak.” 

• “I am not sure; phraseology could maybe be covered better.” 

 
Appendix F.5: Controller Post-Exp. Results 240 

6. Please rate the operational acceptability of the procedures for using the CMS tools. 

On a scale from ‘1’ as “Completely Unacceptable” to ‘7’ as “Completely Acceptable, all 8 

responses were greater than or equal to ‘5’. The mean response was 6.3 (standard deviation = 0.9). 

Figure 131. Controller operational acceptability of CMS tool procedures. 

All comments: 

• “Being able to compare ETA vs STA was beneficial to evaluate the controllers’ actions.” 

• “I like the concept. It is a good tool.” 

• “I have a lot of experience using the CMS tools so my opinion is biased. I think they work 

very well. Some little nuances will be learned over time to make them even more effective.” 

• “The tools worked well, but sometimes you could not trust the speeds on final.” 

• “Good with a few normal problems.” 

• “No complaints.” 

• “None.” 

• “Phraseology was different but once you learned it, it was ok.” 

 
Appendix F.5: Controller Post-Exp. Results 241 

7. Please rate the operational acceptability of the procedures for IM operations. 

Controllers were asked to rate operational acceptability of the procedures for each type of IM 

operation, CAPTURE, CROSS, and MAINTAIN. On a scale from ‘1’ as “Completely 

Unacceptable” to ‘7’ as “Completely Acceptable.” 

CAPTURE CROSS MAINTAIN

7

6

5

4

3

2

1

Acceptability Ratings

Center

Feeder

Final

Figure 132. Controller acceptability of IM procedures for each clearance type. 

Table 56. Descriptive Statistics for Controller Acceptability Ratings of IM Procedures 

Operation N Mean SD Min Median Max 

CAPTURE 8 6.6 1.1 4 7 7 

CROSS 8 6.4 1.2 4 7 7 

MAINTAIN 8 5.9 2.1 2 7 7 

 
Appendix F.5: Controller Post-Exp. Results 242 

8. Please rate the impact of the addition of IM operations on expediting the traffic flow 

during this simulation. 

Controllers were asked to rate the impact of IM operations on expediting the traffic flow during 

the simulation on a scale from ‘1’ as “Much Less Expeditious,” to ‘4’ as “No Impact” to ‘7’ as 

“Much more Expeditious.” The mean response was 5.6 (standard deviation = 1.2). 

Figure 133. Impact of IM operations on expediting traffic flow. 

All comments: 

• “Allowed for more attention complying with metering duties.” 

• “Adds a little to your workload but spacing is more efficient.” 

• “Only on a maintain clearance do I see a huge benefit. The crossing and capture clearances 

seem to waste space from time to time. Maybe the time needs to be reduced or use 

mileage.” 

• “If you observe the timelines, the IM aircraft were quite often ahead of schedule and wellspaced.” 

• “Aircraft were run closer and speed adjustments were used for spacing.” 

• “It appears that there is as much pressure on the airport as if you were metering.” 

• “I did not see where it expedited traffic more than the non-IM aircraft on final.” 

• “It was somewhat more expedited.” 

 
Appendix F.5: Controller Post-Exp. Results 243 

Questions 9.1 – 9.3. 

Questions 9.1 – 9.3 used a rating scale of ‘1’ as “Completely Unacceptable,” to ‘4’ as “Moderately 

Acceptable,” to ‘7’ as “Completely Acceptable.” 

Note: for questions 9.1, 9.2, and 9.3, one Final controller consistently rated the clearance types 

significantly lower than all the other controllers, with responses of ‘5’ for 9.1, ‘4’ for 9.2, and ‘3’ 

for 9.3. This controller had TRACON experience, but no experience with RNAV arrivals or RNP 

approaches. The controller had been retired for approximately 5 years, which was typical for this 

controller population. 

9.1. When IM aircraft were in your sector, how acceptable was it for you to be responsible 

for maintaining safe / standard separation between these aircraft, while the flight crew was 

managing the spacing on a CAPTURE clearance. 

Table 57 summarizes responses to this question. Each “same route” situation had one ‘5’ rating 

and seven ‘7’ ratings. Each “different route” situation had one ‘5’ rating, two ‘6’ ratings, and five 

‘7’ ratings. The ‘5’ ratings were given by a Final controller.

Table 57. Descriptive Statistics for Acceptability of Controller Responsibility for 

CAPTURE Clearance 

Situation N Mean SD Min Median Max 

IM behind IM on the same route 8 6.8 0.7 5 7 7 

IM behind IM on a different route 8 6.5 0.8 5 7 7 

IM behind CMS on the same route 8 6.8 0.7 5 7 7 

IM behind CMS on a different route 8 6.5 0.8 5 7 7 

All comments (not listed are two comments stating, “No problems”): 

• “I utilized information available in data block and matched that with trailing aircraft.” 

• “CAPTURE works well as long as you do not go below the given time.” 

• “Controller increased vigilance scanning of aircraft that were merging from separate routes 

(workload increased).” 

• “Speeds on final were not always acceptable.” 

• “Different routes caused a delay in pairing which increased scanning to come back to the 

Target aircraft to see if they would capture and pair.” 

• “Separation has to be maintained regardless of the types of aircraft equipment.” 

• “Sometimes you have to rely on good old ATC.” 

 
Appendix F.5: Controller Post-Exp. Results 244 

9.2. When IM aircraft were in your sector, how acceptable was it for you to be responsible 

for maintaining safe / standard separation between these aircraft, while the flight crew was 

managing the spacing on a CROSS clearance. 

Table 58 summarizes responses to this question. Each “same route” situation had one ‘4’ rating 

and seven ‘7’ ratings. Each “different route” situation had one ‘4’ rating, two ‘6’ ratings, and five 

‘7’ ratings. The ‘4’ ratings were given by the same Final controller identified in question 9.1. 

Table 58. Descriptive Statistics for Acceptability of Controller Responsibility for CROSS 

Clearance 

Situation N Mean SD Min Median Max 

IM behind IM on the same route 8 6.6 1.1 4 7 7 

IM behind IM on a different route 8 6.4 1.1 4 7 7 

IM behind CMS on the same route 8 6.6 1.1 4 7 7 

IM behind CMS on a different route 8 6.4 1.1 4 7 7 

Relevant comments are listed below. Two comments were excluded: “Same as in the previous 

question,” and, “No problems.” The same as in previous question was, “I utilized the information 

available in the data block and matched that with trailing aircraft.” 

• “Cross fixes are located in TRACON airspace, so en route controllers did not know when 

pairing took place. Therefore, spacing requirements had to be standard separation 

clearances.” 

• “Crossing clearances always worked well.” 

• “This didn’t adversely impact my operation at all.” 

• “Having sight of the aircraft involved in the clearance is helpful.” 

• “Cross clearances seemed to take longer to get paired.” 

• “Again, without assigning a speed on final they had to be watched a lot.” 

 
Appendix F.5: Controller Post-Exp. Results 245 

9.3. When IM aircraft were in your sector, how acceptable was it for you to be responsible 

for maintaining safe / standard separation between these aircraft, while the flight crew was 

managing the spacing on a MAINTAIN clearance. 

Table 59 summarizes responses to this question. Each “same route” situation had one ‘3’ rating 

and seven ‘7’ ratings. Each “different route” situation had one ‘3’ rating, one ‘6’ rating, and five 

‘7’ ratings. The ‘3’ ratings were given by the same Final controller identified in questions 9.1 and 

9.2. 

Table 59. Descriptive Statistics for Acceptability of Controller Responsibility for 

MAINTAIN Clearance 

Situation N Mean SD Min Median Max 

IM behind IM on the same route 8 6.5 1.4 3 7 7 

IM behind IM on a different route 8 6.1 1.5 3 7 7 

IM behind CMS on the same route 8 6.5 1.4 3 7 7 

IM behind CMS on a different route 8 6.1 1.5 3 7 7 

All comments (excluded is one comment stating, “no problems”): 

• “MAINTAIN was a beautiful thing on the same routes. Aircraft on different routes took 

too long to occur, until you got the same routing established.” 

• “MAINTAIN clearances worked well, even with aircraft changing speed in descent.” 

• “In most cases, the separation spacing ended up being excessive, so I was never worried 

about losing separation. I was always concerned about too much spacing.” 

• “For aircraft on the same route, the clearance can be issued on a time available basis. 

Aircraft on different routes required me to wait for the aircraft to be on the same route.” 

• “I thought MAINTAIN clearances were the easiest and required less controller input.” 

• “The MAINTAIN clearance was a little less acceptable.” 

 
Appendix F.5: Controller Post-Exp. Results 246 

Questions 10 – 11. 

Question 10 and question 11 used a rating scale from ‘1’ as “Not useful at all (ignored)” to ‘4’ as 

“Somewhat useful” to ‘7’ as “Very useful (essential).” 

10. [If Center] How useful were the controller tools for managing the aircraft? 

All but one usefulness rating was ‘7’. The one ‘6’ rating was given for the “Delay in the meter list” 

item. Table 60 summarizes the responses to this question. 

Table 60. Descriptive Statistics for Usefulness of Controller Tools for Managing Aircraft 

Controller tool N Mean SD Min Median Max 

Delay in the meter list 4 6.8 0.5 6 7 7 

Delay countdown timer 4 7.0 0.0 7 7 7 

IM designators 4 7.0 0.0 7 7 7 

Runway identifier 4 7.0 0.0 7 7 7 

IM-specific meter list information 4 7.0 0.0 7 7 7 

All-aircraft meter list information 4 7.0 0.0 7 7 7 

All comments: 

• “Comparing ETA/STA was useful. Timer absolutely must have. Designators must have. 

Runway ID helpful if there’s difference on the arrival due to flow but TRACON is going 

to tell them what to expect on initial communications.” 

• “I used them all a lot.” 

• “The lists and data block info worked out well.” 

• “The meter list was very helpful especially the all.” 

• “It helped a lot. To me the more info I get to see the better.” 

11. [If Feeder or Final] How useful were the CMS tools for managing the aircraft? 

The CMS tool with the poorest mean ratings were the delay countdown timer and spacing cones, 

though both of these items received also received ratings of ‘7’ from different controllers. 

Table 61 summarizes the responses to this question. There was one ‘1’ rating, for the spacing 

cones by a controller who did not know they were available.
Appendix F.5: Controller Post-Exp. Results 247 

Table 61. Descriptive Statistics for Usefulness of CMS Tools for Managing Aircraft 

CMS tool N Mean SD Min Median Max 

Speed advisories 4 6.0 1.4 4 6.5 7 

Slot markers 4 6.3 1.5 4 7 7 

Early/late indicators 4 5.5 1.7 4 5.5 7 

Delay countdown timer 4 4.3 2.1 2 4 7 

Timelines 4 6.5 1.0 5 7 7 

IM designators 4 6.8 0.5 6 7 7 

Spacing cones (bats) 4 4.0 2.9 1 4 7 

All comments: 

• “Did not use the bats or the delay count down timer functions.” 

• “Was not aware that spacing cones were available so my answer indicated is only to fill 

the slot. I do feel they would be useful as a tool especially when there is a wake turbulence 

component in the sequence.” [This controller answered ‘1’ for the usefulness of the spacing 

cones, as indicated above.] 

• “Speed advisories help to provide a reference point.” 

• “I liked the slot markers best, and the speed advisories let me know the trend.” 

• “The speed advisories let me know the trend. Slot markers I liked the best. Spacing cone 

helped on final” 

12. Describe any changes you would make to the IM operation. If appropriate, please 

comment on the IM and Target aircraft being in-trail vs. on merging routes, the delay 

between the issuance of the IM clearance and the initiation of the IM operation, etc. 

Relevant comments: 

• “Try and create some way for IM aircraft to pair with aircraft on merging routes prior to a 

common fix. If IM aircraft can see the Target aircraft, then it seems it should be pair-able.” 

• “In trail is easy. On merging routes crossing seems to lead to less possible confusion. The 

delay in IM initiation usually did not cause a problem. But you must be aware of it to figure 

it in.” 

• I think the program tries to “chase” the cone too much. If you follow all the speed 

advisories, you would be issuing speeds constantly. Pick a fix, for example on the 

downwind and project ahead to that point. I did my speed assignments like that and it 

seemed to work well.” 

• “I was very happy from a Final [Controller] perspective of how the IM aircraft operated 

and were displayed. Using the sequence numbers as a reference made it easy to determine 

who the aircraft was paired with. I am not a fan of the MAINTAIN clearance in the terminal 

airspace. I think once the IM aircraft enters the TRACON airspace, another IM clearance 

type needs to be issued or the MAINTAIN clearance canceled.” 

• “I would put the requirement to say “report paired” with the acknowledgement of the 

original IM clearance. Adding it to the original clearance at the end is just too much 

phraseology.” 
Appendix F.5: Controller Post-Exp. Results 248 

13. What changes would you make to the displays for the IM aircraft? What additional 

information (if any) would you like with respect to the IM aircraft and how would you like 

it displayed? 

Most controllers found the displays acceptable as currently configured. Included below is the 

only comment which described a change: 

• “Different identifier for maintain clearance. This clearance should hold the spacing 

assigned and the others will deteriorate at some point.” 

14. What information (if any) would you like available with respect to the Target 

aircraft and how would you like it displayed? 

Most controllers found the current Target information to be acceptable as currently displayed. 

Included below are the only comments which described a change: 

• “If you could click a button and have the altitudes, speeds and all the fixes on the STAR 

that would be good.” 

• “As a TRACON controller I do not care about the Target aircraft. I am not going to the 

trouble of issuing an IM clearance in my airspace.” 

15. Please rate the operational acceptability of the IM phraseology used during the 

simulation. 

All ‘4’ ratings (the only ratings less than 5) were provided by one Center controller who did not 

comment about the reasons for those ratings. Not all controllers rated all phraseology. 

Table 62. Descriptive Statistics for Controller Operational Acceptability of IM Phraseology 

Phraseology N Mean SD Min Median Max 

CAPTURE clearance 5 6.2 1.3 4 7 7 

CROSS clearance 5 7.0 0.0 7 7 7 

MAINTAIN clearance 7 7.0 0.0 7 7 7 

Reporting “IM paired” 8 6.5 1.1 4 7 7 

Check in with TRACON 4 6.8 0.5 6 7 7 

Amendments to IM clearance 7 6.3 1.1 4 7 7 

IM clearance suspension 8 6.4 0.8 4 4 7 

IM clearance resumption 8 6.4 1.1 4 7 7 

IM clearance status reporting “Unable” 6 6.4 0.8 5 6.5 7 

Three controllers commented that the phraseology was clear and worked well, or commented on 

their position to explain why they did not issue certain clearances. Other comments: 
Appendix F.5: Controller Post-Exp. Results 249 

• “Unable should only be used if it cannot be accomplished, not to signal that it is not doable 

from the moment and then later the aircraft pairs up.” 

• “When unable given from an aircraft they need to give a reason. That would avoid some 

possible confusion.” 

16. Did you experience any confusion due to the use of the Target aircraft’s call sign in 

the IM clearance?

Seven of the eight controllers responded “No.” The controller that responded “Yes” commented 

“I had to use phonetics on some aircraft for their call signs.” 

17. Please rate the operational acceptability of using the Target aircraft’s call sign in the 

IM clearance. 

All 8 responses gave a rating of ‘7’ on a scale from “1” as “Completely Unacceptable” to ‘7’ as 

“Completely Acceptable.” Some controllers commented the use of the Target aircraft’s call sign 

in the IM clearance was not only acceptable but necessary. Selected comments: 

• “I like the verification of who they [the pilots] are paired with.” 

• “As long as the other aircraft can see the other call sign, I thought this was completely 

acceptable.” 

• “Got to have it.” 

18. Describe any suggestions to improve the clarity and/or completeness of the IM 

clearance phraseology. 

Two controllers had suggestions to improve the IM clearance phraseology. One controller 

suggested that the relative position of the Target aircraft to the IM aircraft should be called to 

expedite locating that aircraft. Another controller suggested removing the “report paired” 

instruction out of the initial clearance and adding it to a follow-on acknowledgment, giving the 

example, “US1415 read back correct, report paired.” 

19. Describe the challenges (if any) you perceive to the operational implementation of IM 

operations. 

Controllers described the challenges of implementing IM operations. Multiple controllers 

mentioned trust of the system as a possible challenge. Selected comments: 

• “Using a data link to transmit IM clearances would improve/increase the delivery of said 

clearances and reduce misunderstandings.” 

• “Controllers might think it will be like time metering that did not work before. But advise 

them this is different and better.” 

• “Training and trusting will be the biggest hurdle. Once the trust is gained that the computer 

is making the correct sequencing decisions and that the IM aircraft really do what is 

necessary, things will go smoothly.” 

• “Adjusting the scanning technique. Currently most scanning is altitude, speed, and then 

time when determining separation. With IM it was time, altitude, and then speed.” 
Appendix F.5: Controller Post-Exp. Results 250 

20. In order to issue an IM clearance in the TRACON, what information would you need 

and how would you like it displayed? 

Two of four controllers commented that no change was necessary from how information was 

provided. The other two controllers’ comments are as follows: 

• “I would need to know what aircraft are IM capable, and those that are capable need a 

designator in the data block.” 

• “I do not think issuing IM clearances in the TRACON is feasible, too little airspace and 

time.” 

21. Of the IM clearances and operations you experienced in this experiment (CAPTURE, 

CROSS, MAINTAIN), which one do you believe is the most useful, and when would you use 

it? 

ARTCC controllers gave the following comments, two of four preferring MAINTAIN: 

• “The MAINTAIN clearance. I would use it to sequence in-trail aircraft.” 

• “CROSS is the easiest to use to increase good spacing. CAPTURE could be the most 

beneficial to close the gaps in spacing. MAINTAIN just keeps it safer than monitoring it 

all the time.” 

• “CAPTURE because spacing can be created.” 

• “MAINTAIN and I would use it for in-trail spacing requirements.” 

TRACON controllers gave the following comments, three of four preferring CAPTURE: 

• “CAPTURE. I would use it whenever possible as it frees up my time for other operations 

and reduces transmissions.” 

• MAINTAIN. All the time, due to the fact that the spacing will hold” 

• "CAPTURE from what I saw. As a final controller, if it was not issued already, I doubt I 

would use it.” 

• “Yes CAPTURE.” 

22. Of the IM clearances and operations you experienced in this experiment (CAPTURE, 

CROSS, MAINTAIN), which one do you believe is the least useful, and why? 

ARTCC controllers gave the following comments, two of four describing the CAPTURE clearance 

as least useful:

• “CAPTURE, because it is not implemented until both aircraft are on same route.” 

• “The MAINTAIN clearance is the least useful. Only because you already have a spacing 

lineup and speeds done normally.” 

• “CROSS if it includes fixes outside the sector.” 
Appendix F.5: Controller Post-Exp. Results 251 

TRACON controllers gave the following comments, describing a mix of least-useful clearances. 

• “CROSS and CAPTURE clearances, since they both seem to have a higher percentage of 

deterioration on final.” 

• “MAINTAIN. You really do not know what the spacing is going to end up being, so it is 

not an operation I would use if there was any volume of traffic.” 

• “MAINTAIN. One aircraft off, many others off.” 

• “Cross seemed a little iffy from my limited point of view.” 

23. Do you have any additional comments about the experiment? 

• “Possibly have all aircraft in one sector while the other corner just meters.” 

• “It increases accuracy without controller adjustments, and will allow for better spacing. It 

will close the gaps in spacing.” 

• “I think there should be more situations like the pilot error in the last scenario. This gives 

you a better feel for how it would work in the real world. Nothing is fail proof. Controllers 

have to think and adjust on the fly. So put some of those type things in various scenarios.” 

• “When a scenario is being conducted to the west side of the airport i.e. 16L. Most 

controllers will leader the aircraft to the West (the number 4 position). When you do this 

and when you take a handoff the leader extension does not work so the data block ends up 

right on the target. A change needs to be made to mirror an East (6) direction selection.” 

 
Appendix F.6: Pilot Post-Exp. Results 252 

F.6 Pilot Post-Experiment Questionnaire Results 

Post experiment questionnaires were completed by all participants after completion of the last 

post-run questionnaire. 

2. Describe any anomalies or inconsistencies in the simulation that affected your 

performance. 

Most of the 24 responses to this question could be grouped into the following four issues: 

• ASTOR: only one crewmember at a time could use the mouse/touch screen, which affected 

timeliness of performance when workload should be shared by both pilots. 

• DTS and IFD: autopilot/VNAV performed inconsistently. 

• ASTOR, DTS, and IFD: communications were sometimes garbled and difficult to 

understand. 

• ASTOR, DTS, and IFD: aircraft performance was not as expected, especially deceleration 

rates. 

Selected comments, with simulator used: 

• ASTOR: “The only real simulation issue that affected our performance was the vertical 

path tracking would for no apparent reason, with no commanded speed changes etc., go off 

scale at times from a steady state condition. We would catch it and of course do what was 

needed to get back on profile, but it did not seem natural for a large mass aircraft to zing 

off the profile indications.” 

• ASTOR: “The ASTOR touch screen not being capable of handling two operator inputs at 

the same time. The EFB on a couple occasions failed to mirror the corresponding EFB.” 

• ASTOR: “Deceleration rates at final configuration required a mental adjustment to [reduce 

how quickly drag was added]. It would not likely happen in the real world.” 

• DTS: “On one simulation autopilot did not follow flight director commands resulting in 

descent below a minimum altitude.” 

• IFD: “Simulator was very difficult to slow - especially at altitude. This was far worse than 

the actual aircraft. The simulator would change speeds unbeknownst to the pilots and not 

commanded by the automation. When resetting Missed approach altitude, the simulator 

would often drop out of Vertical Navigation Mode and go into Vertical Speed Mode. If not 

caught immediately, this caused a high descent profile after the Final Approach Fix.” 

• IFD: “The simulator VNAV performance is not correct. This may influence the results for 

evaluating IM procedures.” 

• IFD: “The FMS does not react well to speed changes during descent on path. There were 

times that I felt it had not adjusted top of descent for winds.” 

 
Appendix F.6: Pilot Post-Exp. Results 253 

3. Was the workload required to operate the simulator much less than, the same as, or 

greater than the workload required to fly an actual aircraft? 

Pilots were asked to compare the workload required to operate the simulator to the workload 

required to fly an actual aircraft on a scale from ‘1’ as “Much More” to ‘4’ as “The Same” to ‘7’ 

as “Much Less.” 

Table 63. Descriptive Statistics for Perceived Workload of Simulator vs. Actual Aircraft 

Simulator N Mean SD Min Median Max 

ASTOR 16 4.6 1.3 3 5 7 

DTS 4 5 1.2 1 2.5 7 

IFD 4 3 2.2 1 2.5 6 

Total 24 4.4 1.5 1 5 7 

The comments discussed characteristics that increased the workload when operating the simulator 

compared to a real aircraft, including the mouse/touch screen issues (ASTOR only), the increased 

monitoring of instruments, and the unfamiliar/unexpected aircraft performance. A characteristic 

that decreased the workload when operating the simulator was the reduced number of distractions 

and tasks in the simulator environment compared to the real world. 

Selected comments, grouped by simulator: 

• ASTOR: “A number of real-world tasks and distractions are absent from the simulator.” 

• ASTOR: “Slightly more considering the fact that you had to operate through the use of a 

mouse and inputs at the same time, i.e. an ATC frequency change while trying to configure 

the aircraft would knock the other crew member off of the screen, thus delaying the 

operation.” 

• DTS: “Looking for sim anomalies will increase workload, but there are real world 

distractions that will make up the difference. I thought workload was as good as a 

simulation can get. ATC simulation was very realistic.” 

• IFD: “In VNAV, when slowing down and deploying the speed brakes, the simulator would 

not raise the nose much, if at all to reduce the speed - even though it was going below to 

well below the VNAV profile. In VNAV speed, the aircraft is supposed to pitch up to 

capture the speed and usually causes you to go high on the VNAV descent profile. But the 

simulator seemed to do the opposite. This would force the PF to come out of VNAV and 

use VS to get the simulator to pitch the nose up and slow. In at least one occasion, this 

caused us to bust an altitude on the approach - since VS has no altitude protection.” 

• IFD: “Simulator was very difficult to slow - especially at altitude. This was far worse than 

the actual aircraft. The simulator would change speeds unbeknownst to the pilots and not 

commanded by the automation. When resetting Missed approach altitude, the simulator 

would often drop out of VNAV and go into VS. If not caught immediately, this caused a 

high descent profile after the FAF.” 
Appendix F.6: Pilot Post-Exp. Results 254 

4. Please rate how well the training prepared you to fly the simulator. 

This question asked pilots to rate how well the training prepared them to fly the simulator on a 

scale from ‘1’ as “Not At All Prepared” to ‘4’ as “Moderately Prepared” to ‘7’ as “Very Prepared.” 

Table 64. Descriptive Statistics for Perceived Preparedness by Training 

Simulator N Mean SD Min Median Max 

ASTOR 16 5.9 1.1 4 6 7 

DTS 4 6.0 1.4 4 6.5 7 

IFD 4 7.0 0.0 7 7 7 

Total 24 6.1 1.1 4 6.5 7 

Overall, the comments and ratings indicate the training was well received and worked well, 

especially the reinforcement of information with hands-on training. A few themes emerge from 

the comments, indicating the benefit of verbal debriefs, the benefit of sending training information 

early, and the desire for checklists (which were not provided). 

Selected comments, grouped by simulator: 

• ASTOR: “The CBT training gave a very good idea of the IM procedures. Flying the 

simulator was just a matter of hands on practice.” 

• ASTOR: “Maybe a bit more should be provided on the Fast/Slow indicator, and how it 

should be used.” 

• ASTOR: “Stress the importance of proper radio calls both for the cockpit and controllers 

with regard to IM. This is especially important for suspend/cancel clearances and 

subsequent follow on commands from ATC.” 

• DTS: “Lack of checklists of any kind led to us missing items on set-ups, etc. In one case, 

we did not verify the correct landing runway, and this most likely had a negative effect on 

the experiment data, even though it had nothing to do with IM. I STRONGLY suggest you 

get the checklist card from someone and we use that (I can adjust to another company 

checklist, it is probably good enough).” 

• DTS: “The guide was missing a few diagrams which would have made initial 

understanding of the speed trend display easier to use and interpret. All other issues were 

thoroughly covered in the verbal debrief of the training session.” 

• IFD: “Training was good and complete.” 

 
Appendix F.6: Pilot Post-Exp. Results 255 

5. Please rate how well the training prepared you to enter information into and interpret 

the information presented on the EFB. 

Pilots were asked to rate how well the training prepared them to use the EFB on a scale from ‘1’ 

as “Not At All Prepared” to ‘4’ as “Moderately Prepared” to ‘7’ as “Very Prepared.” 

Table 65. Descriptive Statistics for Perceived Preparedness from EFB Training 

N Mean SD Min Median Max 

24 6.3 0.9 4 7 7 

Overall, pilots commented they felt well trained, and there were no ratings below ‘4.’ Pilots felt 

the early distribution of training material allowed them the necessary time to learn the material, 

and the on-site training clarified and reinforced concepts. 

Selected comments: 

• “Training was good, allowing a week of operation prior to data collection was good. I 

received the manual prior to attending the first week and that was a big help as well.” 

• “Needed some clarification to get totally up to speed. Mostly changes that did not make it 

to the training manual.” 

• “Go over which FILTERS (or other features) might be useful and which ones do not work 

at all. Data input is fairly intuitive, well done on that.” 

• “Training was good. Need to caution users that only the PM should make data entries in 

EFB.” 

• “On line CBT and manuals were good. Best training was the "hands-on" training 

simulations. Experience working system was the best training.” 

• “Very little hands-on required before I felt completely proficient on using the EFB for IM.” 

• “I learned more as I went along. I found the SA very good using the filters.” 

• “Training was great.” 

6. Please rate how well the training prepared you to interpret the information presented 

on the CGD in the forward field of view. 

Pilots were asked to rate how well the training prepared them to use the EFB on a scale from ‘1’ 

as “Not At All Prepared” to ‘4’ as “Moderately Prepared” to ‘7’ as “Very Prepared.” 

Table 66. Descriptive Statistics for Perceived Preparedness from CGD Training 

N Mean SD Min Median Max 

24 6.0 1.1 3 6 7 

Multiple pilots expressed difficulty understanding how to interpret the fast/slow trend indicator, 

some expressing the desire for more training. Ratings were more mixed than other training 

questions but only two ratings were below ‘5’. 

 
Appendix F.6: Pilot Post-Exp. Results 256 

Selected comments: 

• “I feel changes need to be made to the CDG: specifically, the fast/slow and early/late scales. 

Fast/slow needs more detail to increase its usefulness and the early/late scale should be 

eliminated. I do not think it serves any purpose.” 

• “Like most procedures, you can only explain so much before you have to see it in practice. 

The training was adequate to allow understanding of the actual presentation.” 

• “Nothing can adequately train to proficiency on that. Even after 6 days of use, it is still not 

a very helpful display.” 

• “The Fast/Slow cue interpretation was still fuzzy... Only during data collection phase were 

we comfortable with what it was saying.” 

• “The fast/slow display was not explained initially but on 2nd day we had additional 

training. At that time, I understood it.” 

7. Please rate how well the training prepared you to conduct an IM operation. 

Pilots were asked to rate how well the training prepared them to use the EFB on a scale from ‘1’ 

as “Not At All Prepared” to ‘4’ as “Moderately Prepared” to ‘7’ as “Very Prepared.” 

Table 67. Descriptive Statistics for Perceived Preparedness from IM Training 

N Mean SD Min Median Max 

24 6.5 0.8 5 7 7 

Overall, participants felt well prepared for IM operations. Simulation provided the best 

understanding of IM. Multiple pilots suggested video of the IM operation be shown prior to 

simulation training. 

Selected comments: 

• “Concepts were well understood walking in the door. Perhaps some more time spent on 

the limitations of the desktop simulator as compared to the real aircraft, which would 

steepen the learning curve (might also reduce the whining).” 

• “Exceptions to the speed change algorithm (hard constraints, etc.) are still confusing, will 

cause consternation on the line.” 

• “Sequential slides or a video of the acceleration/deceleration, the speed trends (with a large 

deviation), and response technique to correct the delta and then to capture VNAV path 

deviation by proper application of power (below path) or speed brakes (above path).” 

• “Maybe some time compressed videos of actual operation could help pull it together.” 

• “Maybe you could make a short "live" video of someone conducting an experiment?” 

 
Appendix F.6: Pilot Post-Exp. Results 257 

8. Please rate the operational acceptability of the procedures for IM operations. 

This section provides additional detail for Section 5.8.1.3. Pilots rated operational acceptability of 

the procedures for IM operations on a scale from ‘1’ as “Completely Unacceptable” to ‘7’ as 

“Completely Acceptable.” 

Table 68. Descriptive Statistics for Pilot Operational Acceptability of IM Procedures 

Operation N Mean SD Min Median Max 

CAPTURE 24 6.4 0.9 4 7 7 

CROSS 24 6.3 0.8 5 6.5 7 

MAINTAIN 24 5.9 1.5 1 6 7 

Selected comments are included below. Multiple pilots mentioned data link communications as a 

way to make IM clearances more efficient. Multiple pilots also commented on MAINTAIN as 

ineffective or confusing. 

• “All operations are easy to accomplish with the given procedure. And if large 

acceleration/deceleration deviations are corrected and the vertical path is corrected, then 

the commanded speed changes are minimized.” 

• “It is still spooky to have someone turn in front of you when you do not have a lot of 

options to evade a bad turn. Likewise, it seems poor practice to have two airplanes turn 

onto final for parallel runways at the same distance to go and at the same time.” 

• “No issues in simulation, but: 1) Data link is needed for high density operations, and 2) 

software needs to be improved for higher altitude operations and not just below FL290.” 

• “The CAPTURE operation seems to work well. The CROSS operation seems to also work 

well. The MAINTAIN clearance from ATC was confusing. I could see my Target’s ground 

speed was greater than my own and therefore I was unsure how to maintain "current 

spacing" when the IM device would not pair immediately.” 

• “Operationally it has potential in a high density environment. I would expect MAINTAIN 

and CAPTURE clearances to be used the most.” 

• “Overall, some of the cross clearances seemed to result in some excessive decelerations 

which, in turn, put us high on several crossing restrictions.” 

• “Intuitive, however I believe the constraints are too narrow, requiring too many changes 

that are out of line with passenger comfort.” 

• “From an operational standpoint the IM clearances were succinct and to the point.” 

• “The MAINTAIN clearance did not seem to work correctly. A large delay was incurred 

waiting for Pairing. Isn’t logical.” 

• “All three [clearances] are quite acceptable with a good ATC partnership. The set-up at 

high altitude is key to getting a functional IM descent and approach onto the merge.” 

• “CROSS clearances seem to be the most problematic. Every one that we did required a 

speed reduction which could only be obtained in landing configuration and caused gear 

down and full flaps as much as 10 miles from the FAF.” 

• “Must clarify procedures after IM clearance is issued and before pairing if ATC issues a 

speed to fly.” 
Appendix F.6: Pilot Post-Exp. Results 258 

• “MAINTAIN failed due to speed differences between aircraft at different altitudes on 

descent. This caused a jamming up of aircraft and excessively slow speed for trailing 

aircraft. I understand this was caused by initiating the MAINTAIN IM too early.” 

• “MAINTAIN operations seemed to have less speed changes than the others. CROSS would 

have been my next choice. Those two words are also something a pilot is used to hearing.” 

• “As long as we received the clearances early, it was very easy to follow the procedures.” 

• “CROSS seemed to give very aggressive speed changes that required a lot of drag and 

speed changes.” 

• “In the MAINTAIN clearance, there was a bit of difficulty with maintaining the required 

IM speed. The IM commanded speed was asking for a speed near the top limit of our speed 

tape. In fact, if we flew the speed requested, we would have been past the upper limit speed 

bars.” 

• “Voice is still tedious compared to text (CPDLC or ACARS).” 

9. Please rate the overall acceptability of the IM procedures. 

This section provides additional detail for Section 5.8.1.5. Pilots rated the overall acceptability of 

IM procedures on a scale from ‘1’ as “Completely Unacceptable” to ‘7’ as “Completely 

Acceptable.” 

Table 69. Descriptive Statistics for Pilot Overall Acceptability of IM Procedures 

N Mean SD Min Median Max 

24 6.1 0.8 5 6 7 

Selected comments: 

• “The instructions were clear. However, we need data link to make this effective.” 

• “Overall the IM procedures seem to work out for spacing aircraft. I do, however, feel that 

the termination point of IM should be about 3 miles prior to the FAF. Several times I was 

in violation of Company procedures when trying to comply with IM commands and a stable 

approach with check list completed. This portion of IM should be revised. Distractions and 

additive conditions should not be induced by IM so near both the airport and the 1000 ft. 

AGL point.” 

• “Procedures were very logical and understandable.” 

• “I think the procedures would work better when the specific aircraft and its’ performance 

are programed into the software.” 

• “The speed changes in certain segments of the arrival at times conflicted with the crossing 

alt. crossing restrictions.” 

• “I do not think the typical airline pilot will blindly follow IM speed commands. They will 

always modify to smooth speed commands or for their personal preferences. Or will simply 

say unable when large speed commands occur near altitude constraints.” 

• “The issues are added workload for the crew, especially right before (2 miles) the FAF. At 

this time the speed is high and usually I receive speed changes at this time. It is easy for 

the crew to become saturated with the additive conditions and mistakes can be made. I 

highly recommend the procedures be changed to ignore all IM instructions 1 mile before 
Appendix F.6: Pilot Post-Exp. Results 259 

the FAF. This will allow the crew to complete configuring and slowing as in normal 

operations.” 

• “IM clearances are easy to understand, load, and fly. The only one that I would rate as 

unacceptable was the CROSS IM.” 

• “Completely acceptable unless it requires a speed change below VRef that requires the 

aircraft to drop the gear.” 

• “Procedurally I like IM. It is pretty straight forward for a pilot.” 

• “Acceptable in a simulation environment, but needs a lot of work for the dynamic real 

world.” 

• “Most of the IM procedures presented no problems. However, there were times when the 

IM speeds were exceeding the normal operating limit speeds of our aircraft.” 

• “Need to train to correct techniques for speed changes (when to use speed brakes vice not) 

and the relationship to speed vs altitude. This will be your biggest challenge to getting 

acceptance on the line.” 

10. Please rate the impact of conducting an IM operation on your overall situational 

awareness during the arrival operations. 

This section provides additional detail for results discussed in Section 5.8.3. Pilots rated the impact 

of an IM operation on situational awareness using a scale from ‘1’ as “Severely Degraded 

Situational Awareness” to ‘4’ as “No Impact” to ‘7’ as “Greatly Improved Situational Awareness.” 

Figure 134. Pilot perceived impact of IM operations on overall situational awareness. 

Table 70. Descriptive Statistics for Pilot Perceived Impact of IM on Situational Awareness 

N Mean SD Min Median Max 

24 5.8 1.6 2 7 7 

Selected comments: 
Appendix F.6: Pilot Post-Exp. Results 260 

• “This is probably one of the strongest selling points of IMAC as it gives such a broader 

view of what will happen down the road. Also opens up an operational partnership between 

ATC and pilots.” 

• “You know exactly who you are sequenced behind and know where he is, and frequently 

you knew who he was sequenced behind.” 

• “Improves situational awareness, but increases workload.” 

• “I began looking at Target aircraft and estimating whether our performance would meet 

the desired target times. Having access to other aircraft IDs on the Navigation Display was 

a great boost in situation awareness.” 

• “You can anticipate the changes by better knowing exactly which A/C you are following 

from the onset of the arrival and approach.” 

• “It gives you the big picture as you can track and follow all the other aircraft that are 

arriving at the same airport which allows for much greater situational awareness.” 

• “Using IM allowed me to focus on the aircraft around me and try myself to predict what it 

might do next. I agree that it can cause a little more heads down activity.” 

• “When there were many speed changes or clearances given just before handoffs, or speeds 

assigned that did not make sense or were outside the capability of the aircraft, situational 

awareness was significantly degraded. On the IM approaches where we paired early and 

speed changes were minimal and logical, we maintained a high level of situational 

awareness.” 

• “It is a great thing to know how you fit in with the aircraft around you. I believe it improves 

safety.” 

• “Helped build scan to look at other traffic.” 

• “It really was a great tool for our approaches and remaining in the loop.” 

• “Greatly improved my situational awareness to traffic, but tended to distract from my own 

aircraft state.” 

11. How acceptable was it for you to be responsible for achieving the assigned spacing 

interval while the controller retained responsibility for the separation of aircraft? 

Pilots rated the acceptability of IM responsibilities on a scale from ‘1’ as “Completely 

Unacceptable” to ‘7’ as “Completely Acceptable.” One pilot gave a rating of ‘4’ but commented 

“No problem overall.” Two pilots gave ratings of ‘5,’ commenting, “Confusion can arise between 

controller assigned speeds prior to ‘pairing’ and IM speeds after pairing,” and, “Overall acceptable. 

However, with multiple speed changes in a short period of time requires more heads down. It can 

at times cause conflicts with VNAV descent profiles.” All other ratings were greater than or equal 

to ‘6’. 
Appendix F.6: Pilot Post-Exp. Results 261 

Figure 135. Pilot acceptability of IM responsibilities. 

Table 71. Descriptive Statistics for Pilot Acceptability of IM Responsibilities 

N Mean SD Min Median Max 

24 6.4 0.8 4 7 7 

Overall, most pilots found the responsibility delegation very acceptable, which is reflected in most 

comments. Additional selected comments: 

• “The pilot controls the airplane, thus it follows that he is in tune with the operational 

capabilities and limitations of his airplane and how to maintain or adjust the spacing in the 

most efficient and operationally feasible manner. There may be MELs, or CDLs that limit 

some types of operation, high gross weights, very low weights, and many other factors that 

impact operational execution that are and should always be under pilot control.” 

• “As long as the controller controls the IM pairing message to keep proper spacing. We 

have to be able to trust that we have enough spacing” 

• “I never felt unsafe, except for a couple of events whereby we were requested to maintain 

a speed which exceeded our aircraft’s operating limit speeds (either too fast or too slow).” 

12. Did following the IM commanded speed cause unexpected or undesired behavior? 

Of 24 responses, 19 pilots (79%) responded “Yes” and 5 pilots (21%) responded “No.” A number 

of common issues emerged from the comments associated with “Yes” responses and are discussed 

in Section 5.8.4.2. All comments from “Yes” respondents are included below, grouped by common 

issue: 
Appendix F.6: Pilot Post-Exp. Results 262 

a) Large speed changes, especially near FAF or crossing restrictions, which reduces 

situational awareness, increases workload, and is unconformable to passengers: 

• “Was unable to meet altitude constraints on several occasions when IM made large speed 

changes approaching hard altitude constraints.” 

• "There were times the speed changes were very extreme and this would have caused 

undesired passenger comfort with the constant use of speed brakes.” 

• “Slowing sometimes made it difficult to meet altitude constraints.” 

• “A lot would have given the passengers a very rough ride. A lot of pilots would have not 

accepted the IM due to having to fly using aggressive speed changes.” 

• “Biggest issues were when IM reduced speed while trying to make hard altitude 

constraints. If we are on path and anywhere close to a constraint and IM gives us a slowdown, it will take a concerted effort to manage all parameters. Without due diligence (or 

relief), altitude deviations are very likely.” 

• “Yes slowing from 220 to 180 in less than a minute then speeding back up to 220 in less 

than a minute- so in the course of 120 seconds we had multiple speed commands, but at 

the end of all those we were back to the original speed, and the range to the Target was 

unchanged, and the early late indicator was pegged to "on time" throughout all the speed 

command changes.” 

• “Yes when it sometimes made it impossible to make an altitude constraint.” 

• “Too many speed changes close to the FAF would in the real world be rejected.” 

• “Initially some of the IM speeds had huge fluctuations as you approached the FAF. We 

did not have these on the last 6-7 runs.” 

• “Too many speed changes near the FAF could create a conflict during actual operations. 

Sometimes speed changes caused us to miss our altitudes on the arrival.” 

b) Certain FMS modes eliminate altitude protection: 

• The aircraft climbed when asked to speed up while in a descent via the arrival. We had to 

use FLCH and then back to VNAV. The altitude protections are lost when out of VNAV 

as well as putting the aircraft in a position to possibly not being able to meet the crossing 

restrictions. 

c) Commanded speed below or above safe speed limitations: 

• “Speeds requested at times were above ability of aircraft and below.” 

• “IM on several occasions called for me to slow below my aircraft final target airspeed with 

landing flaps. I ignored this command. I would ignore it in the aircraft. Deviations below 

target on final approach should only be momentary and not commanded by an IM device.” 

• “Never expected to see speeds above placarded speed (340kts) or below final approach 

speed (140kts). Those anomalies must be fixed.” 

• “On one occasion the IM speed command was the clacker [over-speed alarm] speed. It 

increased from 280 knots up to 330 knots. This is not acceptable. On several occasions the 

commanded speed was below target speed, this is also not acceptable.” 
Appendix F.6: Pilot Post-Exp. Results 263 

• “Undesired....yes. In maintaining a high speed on a "Maintain" IM spacing clearance (i.e. 

in trail), we were descending on the arrival and (I believe it was the sim or most likely, my 

error) our aircraft did descend below a crossing restriction altitude, as our aircraft’s descent 

rate was exceeding 4000 fpm. This became sort of like a high speed descent.” 

• “The Maintain clearances caused us to approach minimum safe airspeeds at high altitudes 

- 210 knots at FL230 in one case.” 

d) The aircraft had to be configured early to meet speed commands: 

• “Required to configure for landing well before it is operationally efficient to do so.” 

• “When it required us to drop the gear to fly the commanded speed.” 

• “Drastic [airspeed] decreases and increases causing us to dirty earlier than normal and then 

raising the flaps to accommodate the [airspeed] increase. 

• “There were times where we had to configure the aircraft much further out than we liked 

to achieve a commanded speed (gear out, so we could slow and extend more flaps). This 

is not fuel efficient.” 

13. Describe any changes you would make to the IM operation. If appropriate, please 

comment on the IM and Target aircraft being in-trail vs. on merging routes, the delay 

between the issuance of the IM clearance and the initiation of the IM operation, etc. 

All comments are included below. A synopsis of these comments can be found in Section 5.8.4.3. 

• “A data link command would be useful. If the EFB would auto-select Target filters for 

range/bearing, Target ID/tag, Target route and merge point it would increase situational 

awareness, especially for those pilots who do not normally select those filters.” 

• “A CROSS clearance that results in the Target turning onto final in front of you should 

have a fairly large spacing. If it works reliably for a number of years, then it might be 

reduced. IM spacing could begin earlier and use smaller or fewer speed changes if it did 

not wait for both to pass a fix with speed and altitude constraints.” 

• “Need data link communications for instructions–PERIOD. Also need to clean up software 

for operations above FL290. I really have no issues with following someone coming in on 

another STAR or any other in trail or merging procedures. What I saw worked well.” 

• “Need data link! Need data link! Need data link!” 

• “Allow the MAINTAIN clearance to pair immediately otherwise there is confusion with 

"maintain current spacing".” 

• “It seemed to take a long time before pairing occurred. I would like to see IM pairing 

happen sooner after system is armed. I am certain ATC would appreciate that also.” 

• “An initial Target bearing would be helpful, or an anticipated intercept point (ideally 

displayed on the EFB). Sometimes it seemed like 5min, sometimes 30sec until you were 

expected to comply, i.e. pair.” 

• “I think that the IM pairing could happen a little sooner which would allow the aircraft to 

maintain intervals easier and perhaps reduce the speed changes required to maintain the 

spacing. Also, specific aircraft performance and a consideration for passenger comfort 

should be considered in future versions of the IM.” 
Appendix F.6: Pilot Post-Exp. Results 264 

• “The only change that I would suggest is the FAST/SLOW indictor being integrated 

differently by designing it as a speed tape to make it more informatively relevant.” 

• “Again, I would try to find a way to minimize the speed changes as much as possible for 

passenger comfort.” 

• “CROSS clearances seem to take an excessive amount of time to ‘pair’. This often resulted 

in uncertainty by the pilots if the programming had been done correctly. I think the typical 

airline pilot would probably cancel and reprogram trying to speed up the ‘pairing’.” 

• “Recommend changes to the procedures to stop following IM speed one mile outside the 

FAF. The MAINTAIN operation incurred a long delay while waiting for pairing capture. 

I did not experience this with the CROSS or CAPTURE clearance. At least the delay was 

logical.” 

• “1) From the descent point to the proposed point of Pairing it may be valuable to assign 

standard descent speeds across all fleets. 2) This pays dividends for aircraft arriving from 

different sectors on different arrivals. 3) Phonetic identifiers need to be given on initial 

spacing interval clearance. 4) An aural or visual alerting system needs to be incorporated 

to allow for more heads up time. Currently too much time spent head down during the 

ARMED phase and while PAIRED monitoring for speed changes on the CGD.” 

• “If the pairing could occur earlier, possibly the frequent and excessive speed changes could 

be avoided.” 

• “Voice commands that specifically transfer responsibility for IM clearance. If ATC issues 

a speed and then pairing occurs, it is unclear if the pilot can then follow IM commands 

without clearance from ATC.” 

• “MAINTAIN clearances - not sure how they fit into the controller regime. I understand 

that they should be issued below FL290, but that may still cause excessively slow IM 

commands. Routinely, the MAINTAIN times were around 150 seconds, when the desired 

times were around 80-100 seconds.” 

• “If the IM aircraft is off scale there should be an indicator as to where he will appear when 

in range.” 

• “I think the time between issuance of the clearance to the initiation of the IM operation 

should be shorter. In trail aircraft seemed like a much smoother operation than merging 

routes.” 

• “I would limit the amount a speed changes within a predetermined mileage/time of the 

FAF. The system should know the type and operational ability of both the Ownship and 

Target aircraft. ATC should know what IM has been assigned to the Ownship aircraft.” 

• “MAINTAIN seemed to be the smoothest, because you knew the slowdown points, you 

knew what to expect, and the speed changes mostly seemed logical.” 

• “It seems to work well merging aircraft. I am not sure why there were times when we would 

be paired, but the note said IM limited. This seems to throw a wrench into the plan, if you 

cannot fly the speed that ATC needs due to limitations.” 

• “A better phase of flight logic for speed changes.” 

• “I thought it worked quite well, once you got through a few IM operations.” 

• “Earlier clearances the better. However, since IM would not pair up until past constrained 

waypoints, controllers need to understand that we will not get paired until passing those 

points. Some ATC controllers were impatient on waiting until that waypoint passage.” 
Appendix F.6: Pilot Post-Exp. Results 265 

14. Please rate the intuitiveness of entering IM clearance information into the EFB. 

This section provides additional information for Section 5.8.5.1. Of 24 responses, only one rating 

was less than ‘6’, which was a ‘5’ rating given with the comment, “I think every EFB page should 

have a touch screen home option. A few times I was confused on how to reach the home page.” 

Most ‘6’ and ‘7’ ratings were associated with comments saying, “No issues,” “Very intuitive,” or 

a similar phrase. Some comments had suggestions for improvement; and are listed below: 

• “Very easy to do, except that the full call sign should be displayed so as to reduce 

unessential radio communications.” 

• “The Enter button is usually on the lower right of the screen, except for the call sign manual 

entry. It would be helpful to always have the enter button in the same place.” 

• “Somehow avoid the manual inputs for IM aircraft. Maybe show all aircraft that will be 

arriving over a certain point in a certain window. That way entering an incorrect IM aircraft 

would not be possible.” 

Figure 136. Pilot rated intuitiveness of entering IM clearance information into the EFB. 

Table 72. Descriptive Statistics for Pilot Intuitiveness of Entering IM Information 

N Mean SD Min Median Max 

24 6.7 0.6 5 7 7 
Appendix F.6: Pilot Post-Exp. Results 266 

15. Please rate the usefulness of the following elements on the EFB display. 

This section and the next (question 16) provide additional detail for the synopsis in Section 5.8.5.2. 

Pilots rated EFB display elements on a scale from ‘1’ as “Not useful at all (ignored)” to ‘4’ as 

“Somewhat useful” to ‘7’ as “Very useful (essential).” IM commanded speed and IM status had 

no ratings less than ‘5’. IM messages and the FAST/SLOW indicator had one rating each less than 

‘4’. The EARLY/LATE indicator had the lowest mean rating (4.1, standard deviation = 3.0) and 8 

ratings less than ‘4’. 

Table 73. Descriptive Statistics for Usefulness Ratings of EFB Display Elements 

EFB Display Elements N Mean SD Min Median Max 

IM commanded speed 24 6.7 0.6 5 7 7 

IM status (i.e. PAIRED) 24 6.5 0.7 5 7 7 

IM messages 24 5.5 1.3 2 6 7 

FAST/SLOW indicator 24 5.3 1.8 1 6 7 

EARLY/LATE indicator 24 4.1 3.0 1 4 7 

All comments are listed below: 

• “We were given command speed changes that were sometimes contrary to the 

EARLY/LATE indicator. For example, a speed up when we were indicating early and 

range to the Target was in acceptable parameters with no signs of closure or opening to the 

Target (referencing the Target range/bearing selection).” 

• “Include an aural alert when the IM speed changes or IM status changes.” 

• “Just a note, that once the IM software is set up and engaged I really rely on the CGD for 

most operational data. I do use Target name data and position on the display for situational 

awareness.” 

• “Never did get the hang of the FAST/SLOW indicator. The EARLY/LATE indicator 

helped me to understand if I was about to get another speed change, albeit the opposite of 

what I was anticipating.” 

• “I found that my own interpretation of Target groundspeed track and bearing worked better 

that the early/late indicator. I could predict when speed changes would occur and anticipate 

configuration changes to comply with IM.” 

• “EARLY/LATE was least useful. It could be eliminated.” 

• “I liked the early/late indicator. Get rid of the FAST/SLOW indicator...integrate that onto 

the speed tape on the EHSI.” 

• “I would leave the EARLY/LATE feature on so as to increase the pilot’s situational 

awareness. However, using that feature has been a little misleading on what the next 

command would be (i.e., we are early, but it asks us to speed up). A little confusing.” 

• “The EARLY/LATE indicator is not useful, and was not relative in this operation.” 

• “The EARLY/LATE indicator was good for situational awareness, but not very useful for 

anything else.” 

• “ARMED waiting condition messages could be more intuitive.” 

• “You could ditch the EARLY/LATE indicator; no one would notice.” 

• “[Status] messages and FAST/SLOW on EFB are nice but not necessary...more applicable 

to the CGD.” 
Appendix F.6: Pilot Post-Exp. Results 267 

• “Some messages did not make sense. Maybe produce a list of what the messages were 

telling us and why we were getting them.” 

• “Blinking should occur when it changes status. Add seconds to the range of the 

EARLY/LATE indicator (i.e. 45s).” 

• “CGD needs some sort of positive annunciation when changing from ARMED to 

AVAILABLE [or else it] will be easily missed in the real world.” 

• “Maybe with more experience using IM messages the FAST/SLOW indicator and 

EARLY/LATE indicator will be brought into the pilot’s scan more.” 

• “I found the early/late indicator the least useful.” 

• “If the EARLY/LATE indicator it is just a predicting tool, and there is nothing we can do 

to change it, then get rid of it.” 

• “I would like to see all these indicators on the CGD so I do not have to look back at the 

EFB once data is entered and confirmed. We will need the EFB for other things like 

approach plates, airport diagrams, unusual approach briefing pages (like RNV (RNP) 

approaches), etc...” 

• “The FAST/SLOW indicator should be less sensitive.” 

• “If the IM messages are clear in meaning, it would help situational awareness.” 

• “IM messages are good advisories on status of the IM operation. EARLY/LATE is okay to 

know, but not actionable. All others are MUST HAVE.” 

16. Please rate the usefulness of the following elements on the EFB display. 

Pilots rated the usefulness of EFB display elements on a scale from ‘1’ as “Not useful at all 

(ignored)” to ‘4’ as “Somewhat useful” to ‘7’ as “Very useful (essential).” Note: Some pilots 

responded “N/A” to certain elements, which were not included in the analysis for Table 74. 

Table 74. Descriptive Statistics for Usefulness Ratings of EFB Display Elements 

EFB Display Elements N Mean SD Min Median Max 

Altitude Filter 20 4.4 2.5 1 4.5 7 

Bearing and Range 23 4.9 2.4 1 6 7 

Target Route 23 6.1 1.3 2 7 7 

Merge Point/ Way Point 21 5.0 1.9 1 5 7 

Target Ground Track and Speed 22 5.2 2.4 1 6.5 7 

Terminate Point 22 4.8 2.5 1 6 7 

Ownship Route 22 6.1 1.6 2 7 7 

Comments apart from “none” or similar are listed below: 

• “When using a CROSS clearance, the EFB should auto select bearing/range, Target route, 

and merge point. When paired, display the Target on the navigation display.” 

• “I would have a data link block with IM uplinked clearance!” 

• “The bearing [filter output] was always incorrect.” 
Appendix F.6: Pilot Post-Exp. Results 268 

• “I like the filters, they improved my situational awareness. It would be helpful if the 

navigation display also displayed id targets similar to TCAS displays.” 

• “Seeing other aircraft helped increase situational awareness by better understanding what 

traffic was doing around you.” 

• “The bearing [output] did not work, but could be estimated (very useful). The above N/A’s 

did not seem to work [Target Route, Merge Point/Way Point, Terminate Point].” 

• “It is intuitive to use and helpful with getting and maintaining situational awareness.” 

• “It is likely with more experience many of these features would be useful.” 

• “Sometimes less is better and there seemed to be filters that did not work in the simulator... 

the operation worked fine without those things.” 

• “More info on what the Target aircraft is doing.” 

• “I liked all the filters.” 

• “The altitude and bearing and range filters are not needed.” 

• “The above things marked useful were [Altitude, Target Route, Waypoint, Ownship 

Route], but it would be better on the ND where we have our TCAS information.” 

• “Perhaps a magenta line or hashed line depicting the Target’s route could be useful.” 

17. Please rate the usefulness of the following elements on the CGD display. 

This section provides additional detail for Section 5.8.5.3. Pilots rated CGD display elements on a 

scale from ‘1’ as “Not useful at all (ignored)” to ‘4’ as “Somewhat useful” to ‘7’ as “Very useful 

(essential).” 

Table 75. Descriptive Statistics for Usefulness Ratings of CGD Display Elements 

CGD Display Elements N Mean SD Min Median Max 

Target Aircraft Call Sign 24 6.2 1.5 1 7 7 

IM Commanded Speed 24 6.9 0.3 6 7 7 

IM Status (i.e. PAIRED) 24 6.3 1.3 2 7 7 

IM Messages 24 5.5 1.7 1 6 7 

FAST/SLOW Indicator 24 5.1 2.1 1 6 7 

All comments (excluding “None” or similar) are included below: 

• “Speed, status, and FAST/SLOW give the most info quickly and are needed in the forward 

field of view.” 

• “A tone to call attention to changes would be helpful.” 

• “Positive alert for change in status of ARMED to AVAILABLE.” 

• “Maybe with the FAST/SLOW indicator, if you overcompensate more than the desired rate 

the + or - speed would change to an alert color of some sort.” 

• “Target call sign is important. Fast/slow is not. I would change the Execute light when IM 

becomes available to flash or change color.” 
Appendix F.6: Pilot Post-Exp. Results 269 

• “Would like to see all messages that were on the EFB on the CGD. Also, would like to see 

the Early/Late indicator. Never noticed if the aircraft’s call sign was on the CGD. As 

mentioned above, once the EFB IM information is entered and confirmed, I would like to 

never have to refer back to the EFB for IM information. It is easier to keep the CGD in my 

scan than the EFB.” 

• “The IM speed, IM status are very useful. I probably looked at CGD display 30% of the 

time.” 

• “FAST/SLOW indicator...now that I understand better that it is a TREND indicator and 

not a STATE indicator, I am better at using it. However, it still has behaviors that are not 

well understood (like big deviations that occur almost instantly after speed changes).” 

• “When there is a large deviation indicated by a numeric value, make the fast/slow indicator 

value blink or in red to garner attention that the acceleration/deceleration rate needs to be 

corrected in order for smoother IM command speed changes.” 

• “Useful as they enhance SA.” 

• “Looked at the IM clearance for the call sign. Used the commanded speed all the time, 

never really viewed the paired, messages or fast/slow because we used the EFB.” 

• “Fast/Slow indicator needs a delay to allow the crew to enter the commanded speed. Plus 

some more detail to better describe the desired speed/rate.” 

• “Display works well and is pilot’s field of view which is essential.” 

• “Again, FAST/SLOW indicator was good for SA but not very useful for anything else.” 

18. Please rate the effectiveness of the CGD in providing adequate information to conduct 

an IM operation. 

This section provides additional detail for Section 5.8.5.3 where summary statistics and a synopsis 

of the following comments can be found. 

Figure 137. Pilot ratings of CGD effectiveness. 
Appendix F.6: Pilot Post-Exp. Results 270 

All comments are included below: 

• “Looked at the IM clearance for the call sign. Used the commanded speed all the time, 

never really viewed the paired, messages or fast/slow because we used the EFB.” 

• “This is somewhat artificial here ...I get the requirement for forward field of view in the 

cockpit, but reading the EFB was far more effective in the lab.” 

• “Outside my normal field of view and flow.” 

• “It is a workable solution to conducting IM spacing. In the 777 the same information could 

be one of the selectable displays. Not only is there little space to put an extra display, this 

would let pilots put away the IM display when it is not needed.” 

• “It is very effective, however, improvements can always be made.” 

• “It would be helpful if the information could propagate back and forth with the FMS.” 

• “Would be nice to have an execute button on the CGD so that we would not need to go 

back to the EFB to execute.” 

• “If it were located in a different place, closer to the top of the instrument panel with a larger 

presence it would that much more effective.” 

• “Sometimes, the EFB commands more attention because of situational awareness.” 

• “Was good, but would like to see more info on it. If some pilots want less on it, you could 

have a declutter switch that would eliminate certain unessential things.” 

• “Maybe add range bearing and delta groundspeed when paired.” 

• “No real changes at this time.” 

• “Once paired I rarely looked at EFB. The CGD was placed properly next to airspeed tape.” 

• “Very effective and essential. I think when the execute button becomes active something 

should flash in the cockpit because unless you are looking right at the CGD it is hard to see 

the greyed out IM commanded airspeed.” 

• “It worked well.” 

• “Only issue is that it does keep our heads down more in critical phases of flight.” 

19. Please rate the operational acceptability of the IM phraseology used during the 

simulation. 

This section provides additional detail for Section 5.8.4.6, where a synopsis of all comments can 

be found. Mean operational acceptability of IM phraseology ratings were all greater than or equal 

to ‘6’ on a scale from ‘1’ as “Not at all clear” to ‘4’ as “Moderately clear” to ‘7’ as “Very clear.” 

Table 76. Descriptive Statistics for Pilot Operational Acceptability of IM Phraseology 

Phraseology N Mean SD Min Median Max 

CAPTURE clearance 24 6.9 0.3 6 7 7 

CROSS clearance 24 6.9 0.3 6 7 7 

MAINTAIN clearance 24 6.5 1.5 1 7 7 

Reporting “IM Paired” 24 6.9 0.3 6 7 7 

Check in with TRACON 24 6.7 0.7 4 7 7 

Amendments to IM clearance 24 6.5 0.9 4 7 7 

IM clearance Suspension 24 6.0 1.5 2 7 7 

IM clearance Resume 24 6.1 1.3 3 7 7 

IM Status reporting Unable 24 6.0 1.6 2 7 7 
Appendix F.6: Pilot Post-Exp. Results 271 

One pilot rated the Maintain clearance ‘1’ and commented, “The ‘maintain current spacing’ 

clearance is confusing when one cannot make an instant input to pair with the IM device.” Another 

pilot rated the maintain clearance ‘3’ with the comment, “The most confusing clearance was the 

Maintain clearance. It did not follow the normal format, did not state report paired. Seemed that 

information was missing.” All other ratings for this phraseology were greater than ‘4’. 

One pilot rated the IM clearance suspension phraseology ‘2’ with the comment, “Early on, we 

were paired, then given speeds by ATC - the same speed we were flying. So did he mean to give 

us the speed or did he mean for us to suspend IM or cancel IM. Pilots really need to hear the 

‘Suspend’ or ‘Cancel’ word before/as ATC assigns us a speed. Then it is very clear.” Another pilot 

rated this phraseology ‘3’ with the comment, “Some confusion between controllers and crews 

about what action should follow on after suspension, sometimes confusion over suspend versus 

cancel. Need a more positive communication between both parties.” All other ratings were greater 

than or equal to ‘4’. One pilot gave a ‘4’ rating and commented, “We had a couple of times when 

it was not clear if we were suspending ourselves or the controller was supposed to suspend us. I 

think it was more to do with the controller not knowing if we were paired or not, possibly because 

we had just been handed off to another controller.” 

One pilot rated the IM clearance resume phraseology as ‘3’, but did not comment on it. All other 

ratings for this phraseology were greater than or equal to ‘4’. 

One pilot rated the IM unable phraseology as ‘2’ and another pilot rated this phraseology as ‘3’ 

but neither commented on the rating. 

20. Did you experience any confusion when hearing your aircraft call sign used as the 

Target aircraft in an IM clearance issued to another aircraft? 

This section provides additional detail for Section 5.8.4.7. Of 24 responses, 4 pilots (16.6%) 

responded “Yes,” 17 pilots (70.8%) responded “No,” and 3 pilots (12.5%) responded “Did not 

hear this occur.” All comments associated with “Yes” responses are included below: 

• “Heard it happen more than once. It took additional read-backs to get fixed.” 

• “Specifically on call signs that I do not frequently hear.” 

• “We had issues when given just a call sign not in IM software form. We learned to ask 

rather quickly for that information as we knew how important it was. At the very least I 

would issue clearances by the phonetic alpha numeric rather than ‘Jetspeed 23’.” 

• “It is a minor distraction, but also aids in situational awareness, as you get to hear who is 

following whom.” 

21. Please rate the operational acceptability of using the Target aircraft’s call sign in the 

IM clearance. 
Appendix F.6: Pilot Post-Exp. Results 272 

This section provides additional detail for Section 5.8.4.8 where a synopsis of the comments can 

be found. All ratings were greater than or equal to ‘4’ on a scale from ‘1’ as “Completely 

unacceptable” to ‘7’ as “Completely acceptable.” 

Figure 138. Pilot operational acceptability of Target aircraft’s call sign in IM clearance. 

Table 77. Descriptive Statistics for Pilot Acceptability of Using Target Call Sign in IM 

Clearance 

N Mean SD Min Median Max 

24 6.1 1.0 4 6 7 

All comments in response to question 21: 

• “For the major carriers it is fine as it is well known. But for everyone else, the IM clearance 

should include either the ID tag/the call sign.” 

• “I would suggest giving the IM clearance with the phonetic (Romeo Poppa Alfa) call sign, 

which reduces the confusion that would result from the usual call sign, and aids reliability 

in Target entry.” 

• “I would assign by phonetic alpha numeric from the get go.” 

• “It is very clear and acceptable. You have to know for situational awareness.” 

• “Since most flight crews do not know all the Target identifiers, ATC should always issue 

IM with those IDs versus an airline name.” 

• “It works well, but as I said earlier, it would be helpful to have the full call sign on the 

Target ID page.” 

• “There were several times when we had to manually input the Target aircraft call sign and 

it was unclear what the three letter identifier was. My suggestion is using the entire airline 

name as well as the three letter identifier in the EFB.” 

• “Possibly a threat, but had no occurrences across clearances. Longer clearances definitely 

clutter the radios and there were multiple instances of blocked communications.” 
Appendix F.6: Pilot Post-Exp. Results 273 

• “No problem unless the Target aircraft was out of range of ADS-B. If I needed to manually 

enter the Target I had to query ATC to get correct Target ID.” 

• “The three letter abbreviations are not common knowledge to most pilots.” 

• “It is essential in my opinion.” 

• “It is important and would have it said phonetically to prevent mistakes.” 

• “Worked well.” 

• “Well known call signs are fine, but some obscure ones will be a big problem. ATC needs 

to anticipate this.” 

22. Describe any suggestions to improve the clarity and/or completeness of the IM 

clearance phraseology. 

This section provide additional detail for Section 5.8.4.9. All comments are listed below: 

• “Use data link messaging.” 

• “‘United 123’ for the usual ATC communications, and Uniform Alfa Lima 123 for IM 

Target ... until we get the clearances by data link.” 

• “1) Need data link. 2) Clearance should be via phonetic alpha numeric if by voice, and data 

block characters if by message uplink.” 

• “Worked for me.” 

• “Change the Maintain clearance terminology. It is confusing to me. See previous comments 

on how to comply with current spacing.” 

• “Since most flight crews do not know all the Target identifiers, ATC should always issue 

IM with those IDs versus an airline name. Data link clearances avoid this problem.” 

• “No improvement needed.” 

• “Reporting ‘paired’ should be a required report that does not have to be communicated by 

ATC.” 

• “Standardize the clearance between Cross, Maintain, and Capture. Give the Target aircraft 

route even with the Maintain clearance and include the "report paired" statement.” 

• “Standardization that will come with time, education, and evolution of the product.” 

• “ATC should give name and three letter abbreviation of the Target aircraft in all 

clearances.” 

• “Regarding airspeed issuance by ATC: ATC must notify the pilot when he may resume the 

IM clearance. It should not be assumed.” 

• “Stress more adherence to the suspend/resume/cancel events.” 

• “Common call signs were no problem, but unusual ones should be given phonetically.” 

• “Use phonetic spelling all the time to avoid mistakes or excessive radio talk.” 

• “If ATC gives a paired aircraft a speed assignment, preface it with ‘Suspend/Cancel IM 

and fly XXX speed’. 

• “Like it as it is.” 

• “Keep it simple stupid. I thought the phraseology was completely within reason in a normal 

cockpit operating environment.” 

• “Well known call signs are fine, but some obscure ones will be a big problem. ATC needs 

to anticipate this.” 
Appendix F.6: Pilot Post-Exp. Results 274 

23. How difficult do you think it would be for a typical crew to learn and integrate the 

IM spacing procedures into their current daily operational flight procedures? 

The mean pilot rating was between “Neutral” and “Slightly Easy” (Mean = 4.4, standard deviation 

= 1.5) on a scale from ‘1’ as “Very Difficult” to ‘4’ as “Neutral” to ‘7’ as “Very Easy.” There was 

no comment entry associated with this question. 

Figure 139. Responses to difficulty of integration into daily flight procedures. 

Table 78. Descriptive Statistics for Difficulty Integrating IM into Current Flight 

Procedures 

N Mean SD Min Median Max 

24 4.4 1.5 1 5 7 

24. Describe the challenges (if any) you perceive to the operational implementation of IM 

operations. 

This section lists all responses to this question to provide additional detail for Section 5.8.4.10 

where a synopsis of these comments is available. 

• “1) Confusion of call sign ID tag - radio congestion. 2) Proper use of the fast/slow 

deceleration indicator. 3) Proper use of power and speed brakes to maintain path when in 

VNAV speed. 4) Preferred Target filters that enhance situational awareness, especially on 

cross/merge clearances (Target id/tag, range/bearing, Target route, merge point, Target 

groundspeed).” 

• “Some training will be required, which is an expense to management.” 

• “I rated Neutral as an overall answer. With proper simulator training I would probably rate 

this as Very Easy. If training were just a bulletin, then it could be slightly to moderately 

difficult.” 

• “We need data link to reduce the chatter!” 
Appendix F.6: Pilot Post-Exp. Results 275 

• “High rates of distractions trying to slow a Boeing 737 with constant speed changes close 

to the airport. It is relative easy if IM is terminated further out from FAF. I think there are 

too many additive conditions and increased task loading with IM procedures. Errors may 

not be caught by crews trying to comply close to airport.” 

• “Like anything new it would be developing trust in the concept.” 

• “The long lead time typically experienced between training and actually conducting the 

operation (think PRM approaches ...9-12months). Most crews will forget what they 

learned.” 

• “It would require training and several LOFTs [line-oriented flight training events] to 

completely train line crews. Integrated training may be helpful but difficult to do.” 

• “The IM speed changes near altitude restrictions in the terminal arrival area interfere with 

configuration changes.” 

• “I think this is overall very easy to learn.” 

• “Like any new procedure, the knowledge imparted will fade if not used on a regular basis.” 

• “Training would be required including at least one simulator flight. Cost will be high; 

airlines will not implement this until mandated. The training will be easy to integrate into 

initial and upgrade training; the problem will be with the rest of the pilots. We might be 

able to add it to add it to the Advanced Qualification Program annual training.” 

• “For the end user the challenge will be the time involved to learn it and then continued use 

thereafter to hone it.” 

• “Like TCAS, it will require use and experience to gain the trust of line pilots.” 

• “Not to over compensate speed changes.” 

• “All pilots are able to comply with IM.” 

• “Depends on training offered. Depends on the pilot’s own abilities, and ability to study 

manuals. It also depends on the age of the pilot.” 

• “Training and the pervasiveness of it [IM operations conducted]. If only one or two airports 

are doing it, it will be more difficult.” 

• “Large speed changes may be difficult to accomplish. Each crew would also need to 

understand the importance of making a "gradual" adjustment, not a really quick adjustment 

when commanded.” 

• “The negative effect on VNAV PATH operations. Airlines are trending STRONGLY 

towards use of VNAV, and this will take them a step backwards in that regard. The 

potential or risk of altitude deviations on hard constraints is going to be elevated 

significantly, and this is going to be the biggest challenge to fleet introduction.” 

25. Of the IM clearances and operations you experienced in this experiment (CAPTURE, 

CROSS, MAINTAIN), which one do you believe is the most useful, and when would you use 

it? 

This section lists all responses to this question to provide additional detail for Section 5.8.4.11 

where a synopsis of these comments is available. 

• “It is not the decision of the pilot to select the type of clearance. Separation and arrival rate 

is an ATC function. However, the IMAC tools enable CROSS and CAPTURE 

functionality that does not exist now. MAINTAIN exists in the sense that a pilot can see 
Appendix F.6: Pilot Post-Exp. Results 276 

his interval visually or on TCAS, so CAPTURE & CROSS are new and would be useful 

with all the developing software & displays.” 

• “I think Capture is most useful and could be used to line up arrivals along a corridor.” 

• “I would say Cross, Capture, and then Maintain in that order, but really all have their uses 

depending on traffic and airspace.” 

• “All worked, but Maintain was the easiest to understand and input.” 

• “CAPTURE or CROSS. I think those both work well. Works well with LNAV less so 

during the RNP approach.” 

• “CAPTURE since most arriving aircraft in a given sector are using the same arrival and it 

seems that presently ATC gives many speed changes. This could alleviate some of their 

problems.” 

• “CAPTURE ...en route to a feeder fix.” 

• “Maintain seems to work the best with the least amount of large speed changes.” 

• “Cross and Capture.” 

• “CROSS is useful on arrivals. This could also be very useful when crossing the tracks on 

the Atlantic.” 

• “All.” 

• “All can be useful, but mainly for the controllers. Airline management will like the 

increased flow rates into the major airports.” 

• “CROSS is likely the most useful, but all three hold their own importance.” 

• “I like and would use all of the IM, but the cross clearance was the most problematic.” 

• “CAPTURE would be used frequently.” 

• “CROSS, because it is the most definitive.” 

• “All good.” 

• “Probably CROSS as it gives the controller a great tool to sequence the landing traffic.” 

• “MAINTAIN or CROSS, two common phrases pilots hear. It would be used when a time 

is used behind an aircraft or fix.” 

• “MAINTAIN is the easiest for the pilots, but ATC has to have the aircraft in the proper 

position to start off with. So the most useful overall may be the CAPTURE IM clearance.” 

• “MAINTAIN seems pretty easy and it would assure proper clearance and spacing in any 

phase of the approach.” 

• “Capture during nice days with long straight in approaches.” 

• “Cross IM clearance would be the most useful, because a typical controller clearance is to 

cross a fix at a certain altitude. Adding the time element is a no brainer. Plus, it gives you 

great situational awareness.” 

• “Maintain is easiest to issue and implement. Use Maintain when you need action NOW. 

Others will work for more deliberate situations.” 

26. Of the IM clearances and operations you experienced in this experiment (CAPTURE, 

CROSS, MAINTAIN), which one do you believe is the least useful, and why? 

This section lists all responses to this question to provide additional detail for Section 5.8.4.11 

where a synopsis of these comments is available. 
Appendix F.6: Pilot Post-Exp. Results 277 

• “MAINTAIN - The pilot can currently see his interval in front of him on TCAS (IMC) or 

visually (VMC) and do that now, albeit with a bit less precision, but it can be done, as it is 

a linear problem.” 

• “Maintain may be the least useful. It is almost redundant.” 

• “Maintain as ATC paired speed control can do that job.” 

• “Not sure the CROSS was as effective as the others.” 

• “Maintain. See previous comments” [Previous Comment: “Capture or cross. I think those 

both work well. Works well with LNAV less so during the RNP approach.”] 

• “MAINTAIN. This could be considered as CAPTURE clearance.” 

• “CROSS...seems to depend on too many other pieces of the puzzle falling into place.” 

• “Capture, as the Target A/C may have speed changes that would make your speed changes 

more dramatic.” 

• “All were essential to the IM operation.” 

• “MAINTAIN seemed to cause the most problems.” 

• “The least useful is probably the CROSS clearance as I do not think it will be used by the 

controllers very often.” 

• “CAPTURE... seems redundant with CROSS at times.” 

• “I like and would use all of them, but the CROSS clearance was the most problematic. The 

excessive speed reductions, shortly after the aircraft converge on the crossing fix are 

unacceptable.” 

• “All are useful depending on the situation.” 

• “MAINTAIN does not work as advertised.” 

• “All good. No problems noted.” 

• “MAINTAIN. It is the easiest but not the most useful.” 

• “CAPTURE. Too many speed changes and not a term most heard by pilots.” 

• “CROSS would be the hardest to control because of the different variables each aircraft is 

experiencing. One aircraft’s sixty knot tail wind might be the other aircraft’s sixty knot 

head wind. Though we put the forecast winds in our FMCs, they are not all that accurate 

for each piece of the sky we are flying through.” 

• “CAPTURE seems the vaguest and hardest to visualize.” 

• “CROSS. Requires too many speed changes and based on different aircraft operations is 

the hardest to make work.” 

• “Maintain. Seems like it could possibly set up a scenario whereby you may exceed an 

aircraft limitation more easily or miss a crossing restriction on an arrival (had such an event 

occur during this experiment).” 

• “All have their use. Keep them.” 

27. Do you have any additional comments about the experiment? 

• “Nothing more. All captured in the above comments. Great staff and design of 

experiments.” 

• “This would be ideal if we could get clearances via data link!!” 

• “Very interesting and I can see it as a useful tool. Do not let operators undertrain for its 

introduction.” 
Appendix F.6: Pilot Post-Exp. Results 278 

• “It was very informative and I appreciate the invite. I would be available if you required 

my assistance in the future.” 

• “Great experiment. When ADS-B is fully operational IM can be a very useful tool. 

Improvement of situational awareness is important if nothing else.” 

• “Great concept and I hope it comes to pass. Experiment was very well organized.” 

• “No, nice job by all involved!” 

• “Very well done, I learned a lot and would be interested to take part again in the future.” 

• “Overall, the experiment was great and I was happy to have the opportunity to participate!” 

• “The real time ATC arrival process is so dynamic. I can count on one hand the number of 

times ATC allows aircraft to fly the entire arrival routing unencumbered by changes!” 

• “I think that time and money might be well spent improving the 737 Simulator to more 

accurately model the IM operations. At time the bad VNAV response interfered. Enjoyed 

the experience.” 

• “Exciting advance for aviation and ATC going forward...opening situational awareness in 

the terminal area is long overdue...if the cockpit setup can be redesigned to allow for more 

heads up time this will be awesome. Nice job.” 

• “Had a great time, glad to help.” 

• “Enjoyed working with you all. Looking forward to coming back.” 

• “The team has worked especially hard to accomplish this experiment. My congratulations 

to those involved. It has been a great experience. Thanks again, [NAME DELETED] (can 

I say who I am?) Cheers!” 

• “Hope to see this operational before I retire.” 

• “I think it was very well conducted.” 

• “An excellent program and much need change to ATC.” 

• “Great personnel here. Really enjoyed it and hope to be back. Please, please put some 

plastic down in the briefing room so we can sip our coffee in the morning. For us west 

coast guys, it is 5am - our body-clock time. But the donuts almost made up for the lack of 

coffee. Thanks - a pleasure!!!” 

• “I enjoyed participating, and hope it turns out to be very successful and useful in safety and 

fuel efficiency. Also, in creating less ATC delays.” 

• “I think it is a great concept and can lead to much greater situational awareness for pilots.” 

• “I am really excited about the potential that IM can bring to the national airspace and our 

operations. However, it will not be a simple training with PowerPoint and a couple runs in 

the simulator. This will require INTEGRATION into existing training tracks with constant 

use to get proficient.” 

 
Appendix G: Data Removed from Analysis 279 

Appendix G: Data Removed from Analysis 

This Appendix lists the criteria used to remove specific aircraft from data analysis, and fell into 

two categories. The first category was defined during the experiment design phase, and all 

quantitative and qualitative data aircraft were removed if any of the criteria were met. The three 

criteria in this category were: 

1) All aircraft flying autonomously (not controlled by human ATC nor flown by a human 

pilot) were removed (that is, all the aircraft arriving into Denver on the opposite side of the 

airspace and landing on the parallel runway to all the human controlled and flown aircraft). 

2) The first two aircraft (always flown by pseudo-pilots) to land in every run were removed 

due to the low-workload and potentially unrealistic behavior for high-density operations. 

3) Aircraft landing after the last subject-piloted aircraft landed, since the scenario was 

terminated at that point and data from aircraft flown by pseudo-pilots was not analyzed. 

After the data had been filtered based on the criteria above, the remaining aircraft operations were 

examined for operationally unrealistic behavior or simulation artifacts that impacted the arrival 

operation (ref. 19, para 3.1.1). These aircraft are identified in Table 79. For these aircraft, all 

quantitative data was removed, but the qualitative data was included in the analysis (surveys from 

the ASTOR, DTS, and IFD pilots). [Note: for some data analysis, in particular the inter-arrival 

spacing error (Section 5.4.1), if an aircraft was removed due to one of the criteria above, the data 

point for that aircraft was removed, as well as for the following aircraft.] 

Summarizing the number of flights, below is the number of operations for various categories: 

• Total number of aircraft in IMAC: 422 

• First two aircraft in each scenario: -40 

• All aircraft after the last IM aircraft landed: -142 

• Other aircraft with operationally unrealistic or simulation artifacts (Table 79): -11 

• Number of aircraft operations remaining in the data analysis: 229 

o Total number of subject operations (ASTOR, DTS, IFD; non-IM & IM) 113 

o Number of subject IM operations (ASTOR, DTS, IFD) 79 

o Number of subject non-IM operations (ASTOR, DTS, IFD) 34 

 
Appendix G: Data Removed from Analysis 280 

Table 79. Aircraft Removed From Quantitative Analysis 

Group Scenario Call sign 

ID (type) 

Target 

Aircraft Notes: 

1 B SWA154 

(MACS) n/a Odd behavior: descended to 10,000 then climbed to ~14,000’, drastic speed changes 

1 G FFT70 

(MACS) n/a Target changed route when transition to different MACS station, triggering off route, in turn causing IM of SWA3036 to go to SUSPENDED. 

1 G SWA3036 

(IFD) FFT70 

IM operation for SWA3036 was SUSPENDED because of software issue 

with Target. Therefore the subsequent IM aircraft (DAL1605) data is also 

excluded from analysis. (1)

1 G DAL1605 

(DTS) SWA3036 Data and results were good, however to be consistent, run was removed 

from analysis since the Target aircraft was removed. 

3 B AAL1691 

(MACS) n/a (MACS) Aircraft terminated (removed from simulation) prior to FAF for unknown reason. IM operation not being conducted. 

3 B UPS896 

(MACS) n/a No spacing data available since lead aircraft removed from simulation. IM operation not being conducted. 

3 C FDX1162 

(MACS) n/a Aircraft flying 360 knots at FL200. 

3 C ASQ7044 

(ASTOR) FDX1162 Data removed due to unrealistic Target speed. (1)

3 H SWA3036 

(IFD) UPS702 Software bug that caused IFD to terminate, in turn causing ASTAR to 

switch to UNABLE prior to FAF. (1)

3 H ASQ7044 

(ASTOR) SWA3036 Removed since Target was removed. (1)

3 I DAL1605 

(DTS) n/a 

Programmed incorrect runway, causing significant overshoot on final. The 

DTS was not a Target for any other IM operation, therefore no additional 

data was lost. 

Note (1): Qualitative data from pilot post-run surveys were also removed for these four aircraft. 

 
Appendix H: Effectiveness of Training 281 

Appendix H: Effectiveness of Training 

This Appendix analyzes mean controller acceptability, mean pilot acceptability, and mean pilot 

workload as a function of run order, to provide a limited validation of the effectiveness of the 

training provided. Data is shown for the two groups of participants described in Section 5.2. 

A visual inspection of the acceptability and workload ratings during the data collection scenarios 

as a function of run order seem to be fairly constant, potentially indicating that the training sessions 

were sufficient to achieve an adequate level of understanding and proficiency prior to commencing 

data collection. 

Figure 140. Mean controller acceptability of IM operations by scenario run order. 

1 2 3 4 5 6 7 8 9 10

10

9

8

7

6

5

4

3

2

1

Run Number

Mean CARS

1

3

Group

J

I H F

E

G B

A

C D

I J C G

H

D B A E F
Appendix H: Effectiveness of Training 282 

Figure 141. Mean pilot acceptability of IM operations by scenario run order. 

Figure 142. Mean pilot workload of IM operations by scenario run order. 

 

1 2 3 4 5 6 7 8 9 10

7

6

5

4

3

2

1

Run Number

Mean Pilot Acceptability

1

3

Group

J

I

F

H

E

B

G

A

D

C

J G I

C

H

F

E

A B

D

1 2 3 4 5 6 7 8 9 10

10

9

8

7

6

5

4

3

2

1

Run Number

Mean Pilot Workload

1

3

Group

J F I

H

E

B

G

A

D

C

J I G H C

F

E

A

D B
Appendix I: Oculometers 283 

Appendix I: Oculometers 

An oculometer is an electro-optical infrared tracker which measures pupil diameter and corneal 

reflection relative to the center of the pupil to determine eye direction. 

For this experiment, the IFD was equipped with two linked digital Smart Eye Oculometer systems, 

each comprised of six small Basler acA640-100gm cameras and four flashes that operate in the 

infrared spectrum (outside perceptible range). The Smart Eye system uses image processing with 

a 60Hz sampling rate to detect head and eye location and rotation to calculate Point-of-Gaze (POG) 

for each pilot (software version 6.1). Optimal camera-to-eye distance typically falls within 30-300 

cm depending on lens adjustment. A predetermined region of the flight-deck environment, called 

an Area of Interest (AOI), is reported when a POG falls within its boundaries. The system reports 

head position and gaze direction. Quality metrics are used to select appropriate data for further 

analysis. Analyses of POG data determines the time taken for a subject to reorient attention to 

relevant events. Analyses of AOI data characterizes the distribution of visual attention. 

Figure 143. A Smart Eye oculometer camera. 

Video recording consists of two feeds, oculometer data and diagnostics data. The left and right 

systems for each data type are merged into a single feed. 

The oculometer system can provide the following raw data in real-time: 

• Gaze vectors for each crew and each eye 

• Head and eye position (each eye) for each crew 

• Eyelid closure distance for each eye for each crew 

• Pupil size for each eye for each crew 

Just prior to data collection, the researcher with oculometer expertise and human factors 

background was moved to another project, resulting in the data being collected during the 

experiment but not analyzed in time for this document. 
REPORT DOCUMENTATION PAGE

Standard Form 298 (Rev. 8/98)

Prescribed by ANSI Std. Z39.18

Form Approved 

OMB No. 0704-0188

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data

sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other

aspect of this collection of information, including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, Directorate for Information

Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other

provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY) 2. REPORT TYPE 3. DATES COVERED (From - To)

4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER 

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

6. AUTHOR(S)

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION

 REPORT NUMBER

10. SPONSOR/MONITOR'S ACRONYM(S)

11. SPONSOR/MONITOR'S REPORT 

 NUMBER(S)

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

12. DISTRIBUTION/AVAILABILITY STATEMENT

13. SUPPLEMENTARY NOTES

14. ABSTRACT

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF:

a. REPORT b. ABSTRACT c. THIS PAGE

17. LIMITATION OF 

 ABSTRACT

18. NUMBER

 OF 

 PAGES 

19a. NAME OF RESPONSIBLE PERSON

19b. TELEPHONE NUMBER (Include area code)

 (757) 864-9658

 

 

NASA Langley Research Center 

Hampton, VA 23681-2199

National Aeronautics and Space Administration 

Washington, DC 20546-0001

Unclassified - Unlimited 

Subject Category 03 

Availability: NASA STI Program (757) 864-9658

NASA-TP-2016-219362

L-20691

01-12-2016 Technical Publication

STI Help Desk (email: help@sti.nasa.gov)

U U U UU

Air traffic; Concept operations; Human-in-the-loop; Interval management

 

Human-in-the-Loop Assessment of Alternative Clearances in Interval 

Management Arrival Operations

Baxley, Brian T.; Wilson, Sara R.; Swieringa, Kurt A.; Johnson, William C.; Roper, 

Roy D.; Hubbs, Clay E.; Goess, Paul, A.; Shay, Richard F.

 330693.04.10.07.07

302

NASA

Interval Management Alternative Clearances (IMAC) was a human-in-the-loop simulation experiment conducted to explore the Air Traffic Management (ATM) 

Technology Demonstration (ATD-1) Concept of Operations (ConOps), which combines advanced arrival scheduling, controller decision support tools, and aircraft 

avionics to enable multiple time deconflicted, efficient arrival streams into a high-density terminal airspace. Interval Management (IM) is designed to support the ATD-1

concept by having an “Ownship” (IM-capable) aircraft achieve or maintain a specific time or distance behind a “Target” (preceding) aircraft. The IM software uses IM 

clearance information and the Ownship data (route of flight, current location, and wind) entered by the flight crew, and the Target aircraft’s Automatic Dependent 

Surveillance–Broadcast state data, to calculate the airspeed necessary for the IM-equipped aircraft to achieve or maintain the assigned spacing goal.