NASA/CR—2010-216390

Identification and Characterization of Key Human Performance

Issues and Research in the Next Generation Air Transportation

System (NextGen)

Paul U. Lee

San Jose State University, San Jose, CA

Tom Sheridan

San Jose State University Foundation, San Jose, CA

James L. Poage

JLP Performance Consulting, Lexington, MA

Lynne Martin

San Jose State University Foundation, San Jose, CA

Kimberly Jobe

San Jose State University Foundation, San Jose, CA

Chris Cabrall

San Jose State University Foundation, San Jose, CA

June 2010
NASA STI Program...in Profile

Since it’s founding, NASA has been dedicated

to the advancement of aeronautics and space

science. The NASA scientific and technical

information (STI) program plays a key part in

helping NASA maintain this important role.

The NASA STI program operates under the

auspices of the Agency Chief Information

Officer. It collects, organizes, provides for

archiving, and disseminates NASA’s STI. The

NASA STI program provides access to the

NASA Aeronautics and Space Database and its

public interface, the NASA Technical Report

Server, thus providing one of the largest

collections of aeronautical and space science

STI in the world. Results are published in both

non-NASA channels and by NASA in the

NASA STI Report Series, which includes the

following report types:

• TECHNICAL PUBLICATION. Reports of

completed research or a major significant

phase of research that present the results of

NASA Programs and include extensive data or

theoretical analysis. Includes compilations of

significant scientific and technical data and

information deemed to be of continuing

reference value. NASA counterpart of peerreviewed formal professional papers but has

less stringent limitations on manuscript length

and extent of graphic presentations.

• TECHNICAL MEMORANDUM. Scientific

and technical findings that are preliminary or

of specialized interest, e.g., quick release

reports, working papers, and bibliographies

that contain minimal annotation. Does not

contain extensive analysis.

• CONTRACTOR REPORT. Scientific and

technical findings by NASA-sponsored

contractors and grantees.

• CONFERENCE PUBLICATION. Collected

papers from scientific and technical conferences,

symposia, seminars, or other meetings sponsored

or co-sponsored by NASA.

• SPECIAL PUBLICATION. Scientific, technical,

or historical information from NASA programs,

projects, and missions, often concerned with

subjects having substantial public interest.

• TECHNICAL TRANSLATION. Englishlanguage translations of foreign scientific and

technical material pertinent to NASA’s mission.

Specialized services also include creating custom

thesauri, building customized databases, and

organizing and publishing research results.

For more information about the NASA STI

program, see the following:

• Access the NASA STI program home page at

http://www.sti.nasa.gov

• E-mail your question via the Internet to

help@sti.nasa.gov

• Fax your question to the NASA STI Help Desk at

(301) 621-0134

• Phone the NASA STI Help Desk at

(301) 621-0390

• Write to:

NASA STI Help Desk

NASA Center for AeroSpace Information

7121 Standard Drive

Hanover, MD 21076-1320
NASA/CR—2010-216390

Identification and Characterization of Key Human Performance

Issues and Research in the Next Generation Air Transportation

System (NextGen)

Paul U. Lee

San Jose State University, San Jose, CA

Tom Sheridan

San Jose State University Foundation, San Jose, CA

James L. Poage

JLP Performance Consulting, Lexington, MA

Lynne Martin

San Jose State University Foundation, San Jose, CA

Kimberly Jobe

San Jose State University Foundation, San Jose, CA

Chris Cabrall

San Jose State University Foundation, San Jose, CA

National Aeronautics and Space Administration

Ames Research Center

Moffett Field, California 94037

June 2010
Acknowledgements

This research was funded by the NASA Airspace Systems program via NASA

Research Announcement Award (No. NNX08AE86A). We are grateful to our

NASA Technical Monitor, Immanuel Barshi, and our project sponsors, Rob Fong

and Jorge Bardina, for their interest and support throughout the project. We are also

thankful to the NASA Ames Airspace Operations Laboratory who provided

simulation videos of Separation Assurance concept that was used integrally in the

cognitive walkthrough portion of the report. We would like to thank Lissa Webbon

for her assistance in manuscript preparation. Lastly, we would like to remember

Kevin Corker, who was the original Principal Investigator for this research, for

setting the guidelines for us to follow in this project as well as his many

contributions to the NextGen air traffic management field.

The use of trademarks or names of manufacturers in the report is for accurate reporting and does not

constitute an official endorsement, either expressed or implied, of such products or manufacturers by

the National Aeronautics and Space Administration.

Available from:

NASA Center for AeroSpace Information NASA Center for Aerospace Information

7121 Standard Drive 7121 Standard Drive

Hanover, MD 21076-1320 Hanover, MD 21076-1320

(301) 621-0390 (301) 621-0390
v

Table of Contents

Acronyms and Abbreviations...........................................................................................................xi

Executive Summary..........................................................................................................................1

Key Human Performance Issues for Separation Assurance..........................................................2

Key Human Performance Issues for Airspace Super Density Operations.....................................3

Key Human Performance Issues for Traffic Flow Management...................................................4

Key Human Performance Issues for Dynamic Airspace Configuration........................................5

Integration of Multiple Research Focus Areas.............................................................................5

Conclusion..................................................................................................................................6

1. Overview......................................................................................................................................9

2. Background ................................................................................................................................ 11

3. Technical Approach.................................................................................................................... 12

3.1 Identification of General Human Factors Related to NextGen ............................................. 12

3.2 Human Performance Issues in the Context of the NextGen.................................................. 14

3.3 Identification and Prioritization of Key Human Performance Issues for Further Study

 and Mitigation.................................................................................................................... 15

3.4 Suggestions for Further Exploration of Key Human Performance Issues through

Cognitive Walkthroughs and HITL Simulations.................................................................. 16

4. Separation Assurance.................................................................................................................. 17

4.1 Human Roles and Technology Assumptions for Ground-based Automated Separation

Assurance ........................................................................................................................... 17

4.2 Key Human Performance Issues for ASA............................................................................ 18

4.2.1 Distribution of Roles and Responsibilities between Humans and Automation............. 18

4.2.2 Brittleness and Human Operator Trust in the Automation and Decision

Support Systems......................................................................................................... 19

4.2.3 Demand for Workload, Attention, Situation Awareness, and Coordination ................. 19

4.2.4 Dealing with Emergency, Safety Critical, and Off-Nominal Events............................ 20

4.2.5 Human Operator Characteristics and Training ............................................................ 20

4.3 Main Drivers of the Human Performance Issues for ASA.................................................... 21

4.4 Cognitive Walkthrough of ASA .......................................................................................... 21

5. Airspace Super Density Operations............................................................................................. 23

5.1 Human Roles and Technology Assumptions for ASDO....................................................... 24

5.1.1 Distribution of Roles and Responsibilities between Humans and Automation............. 25

5.1.2 Demand for Workload, Attention, Situation Awareness, and Coordination ................. 25

5.1.3 Dealing with Emergency, Safety Critical, and Off-Nominal Events............................ 26

5.1.4 Human Operator Characteristics and Training ............................................................ 27

5.2 Main Drivers of the Human Performance Issues for ASDO................................................. 28

6. Traffic Flow Management........................................................................................................... 28

6.1 Human Roles and Technology Assumptions for TFM ......................................................... 29

6.2 Key Human Performance Issues for TFM ........................................................................... 29

6.2.1 Reliance on the Automation and Decision Support Tools............................................ 30
vi

6.2.2 Demand for Workload, Attention, and Situation Awareness ....................................... 31

6.2.3 Communication and Coordination .............................................................................. 31

6.2.4 Dealing with Emergency, Safety Critical, and Off-Nominal Events............................ 32

6.2.5 Human Operator Characteristics and Training ............................................................ 32

6.3 Main Drivers of the Human Performance Issues for TFM ...................................................33

7. Dynamic Airspace Configuration................................................................................................33

7.1 Human Roles and Technology Assumptions for DAC.........................................................34

7.2 Key Human Performance Issues for DAC ...........................................................................34

7.2.1 Reliance on the Automation and Decision Support Tools............................................35

7.2.2 Demand for Workload, Attention, and Situational Awareness.....................................35

7.2.3 Dealing with Emergency, Safety Critical, and Off-Nominal Events............................35

7.2.4 Human Operator Characteristics and Training ............................................................36

7.3 Main Drivers of the Human Performance Issues for DAC ...................................................36

8. Integrated DAC-SA ....................................................................................................................37

8.1 Human Roles and Technology Assumptions for Integrated ASA and DAC .........................38

8.2 Key Human Performance Issues for Integrated ASA and DAC............................................38

8.2.1 Distribution of roles and responsibilities between humans and automation .................39

8.2.2 Demand for workload, attention, and situational awareness........................................39

8.2.3 Dealing with emergency, safety critical, and off-nominal events.................................40

9. Conclusion..................................................................................................................................41

10. References................................................................................................................................43

Appendix A: Human Factors Reference List...................................................................................45

A.1 Attention Allocation...........................................................................................................45

A.2 Decision-making and Mental Modeling..............................................................................46

A.3 Communication..................................................................................................................48

A.4 Memory .............................................................................................................................49

A.5 Workload ...........................................................................................................................50

A.6 Interaction with Automation/Decision Support Tools and Displays....................................52

A.7 Organizational Factors .......................................................................................................54

A.8 Potential Errors and Recovery from Errors or System Failures ...........................................55

A.9 Personnel Selection, Certification, and Training .................................................................56

Appendix B: Human Performance Factors in NextGen ...................................................................59

B.1 Attention Allocation ...........................................................................................................59

B.2 Decision-making and Mental Modeling ..............................................................................60

B.3 Communication ..................................................................................................................61

B.4 Memory..............................................................................................................................62

B.5 Workload ...........................................................................................................................63

B.6 Interaction with Automation, Decision Support Tools, and Displays...................................64

B.7 Organizational Factors........................................................................................................66

B.8 Potential Errors and Recovery from Human Errors or System Failures...............................66

B.9 Potential Selection, Certification, and Training...................................................................67

B.10 References........................................................................................................................68

Appendix C: Human Performance Issues in the Context of the NextGen Concepts and

Operations..................................................................................................................71

C.1 Use of Automation and DSTs.............................................................................................72
vii

C.2 Human Decision-making and Workload .............................................................................73

C.3 Attention, Situation Awareness, and Memory.....................................................................73

C.4 Communication and Coordination ......................................................................................74

C.5 Organization, Selection, Job Qualification, and Certification Factors..................................75

C.6 References..........................................................................................................................75

Appendix D: Separation Assurance.................................................................................................77

D.1 Overview Description of SA Covered in the Report ...........................................................77

D.2 Summary of Required Technology for Ground-Based ASA................................................78

D.3 Summary of Human Roles for ASA ...................................................................................78

D.3.1 ASA Manager Roles and Responsibilities for ASA....................................................78

D.3.2 Controller Roles and Responsibilities for Unequipped Aircraft Flying in Same

Airspace as ASA-equipped Aircraft ..........................................................................80

D.3.3 Pilot Roles and Responsibilities for ASA...................................................................80

D.4 Human Factors for ASA.....................................................................................................81

D.4.1 Attention Allocation ..................................................................................................82

D.4.2 Decision-making and Mental Modeling .....................................................................82

D.4.3 Communication .........................................................................................................83

D.4.4 Memory.....................................................................................................................83

D.4.5 Workload...................................................................................................................84

D.4.6 Interaction with Automation, Decision Support Tools, and Displays..........................84

D.4.7 Organizational Factors...............................................................................................86

D.4.8 Potential Errors and Recovery from Human Errors or System Failures ......................86

D.4.9 Potential Selection, Certification, and Training..........................................................87

D.5 Human Operations Issues, Design Issues, and Countermeasures for ASA...........................87

D.5.1 Use of Automation and DSTs....................................................................................87

D.5.2. Human Decision-making, Workload, and Error Recovery.........................................90

D.5.3 Attention, Situation Awareness, and Memory ............................................................93

D.5.4 Communication and Coordination .............................................................................95

D.5.5 Organization, Selection, Job Qualification, and Certification Factors.........................97

D.6 Summary of Key Human Performance Issues for ASA.......................................................99

D.6.1 Use of Automation and Decision Support Tools for ASA ..........................................99

D.6.2 Human Decision-making and Workload for ASA ....................................................100

D.6.3 Attention, Situational Awareness, and Memory for ASA .........................................102

D.6.4 Communication and Coordination for ASA .............................................................102

D.6.5 Organization, Selection, Job Qualification, and Certification Factors for ASA.........103

D.7 References........................................................................................................................103

Appendix E: Cognitive Walkthrough of Automated Separation Assurance ...................................105

E.1 Purpose of Cognitive Walkthrough ...................................................................................105

E.2 Cognitive Walkthrough Approach ....................................................................................106

E.3 Description of Separation Assurance Concept Studied in Walkthrough.............................107

E.4 Study Method ...................................................................................................................108

E.4.1 Walkthrough Generation ..........................................................................................108

E.4.2 Participants..............................................................................................................110

E.4.3 Equipment and Stimulus Events...............................................................................111

E.4.4 Procedure.................................................................................................................111

E.5 Results and Discussion .....................................................................................................112
viii

E.5.1 Time Pressure ..........................................................................................................112

E.5.2 Shift of Control........................................................................................................113

E.5.3 Responsibility ..........................................................................................................113

E.5.4 Trust in the Automation ...........................................................................................113

E.5.5 Controllers' Expertise...............................................................................................114

E.5.6 Summary and Recommendations of Content Results................................................114

E.6 Summary Discussion of the Walkthrough as a Method .....................................................116

E.7 References........................................................................................................................117

Addendum to Appendix E. Detailed Analysis of Questions Asked of Human

Subjects in Walkthrough........................................................................................119

Addendum E.1 Answers Related to Use of Automation and DSTs......................................120

Addendum E.2 Answers Related to Decision-Making and Workload..................................122

Addendum E.3 Answers Related to Attention, Situation Awareness, and Memory..............125

Addendum E.4 Answers Related to Communication and Coordination ...............................127

Addendum E.5 Answers Related to Organization, Selection, Job Qualification, and

Certification Factors ........................................................................................128

Appendix F: Airspace Super Density Operations ..........................................................................129

F.1 Overview Description of ASDO........................................................................................129

F.2 Summary of Required Technology for ASDO...................................................................130

F.3 Summary of Human Roles for ASDO ...............................................................................130

F.3.1 Designated Traffic Manager Roles and Responsibilities for ASDO ..........................131

F.3.2 Controller Roles and Responsibilities for ASDO ......................................................132

F.3.3 Pilot Roles and Responsibilities for ASDO...............................................................133

F.4 Human Factors for ASDO.................................................................................................133

F.4.1 Attention Allocation.................................................................................................133

F.4.2 Decision-making and Mental Modeling....................................................................134

F.4.3 Communication........................................................................................................135

F.4.4 Memory ...................................................................................................................136

F.4.5 Workload .................................................................................................................137

F.4.6 Interaction with Automation, Decision Support Tools (DSTs), and Displays............137

F.4.7 Organizational Factors .............................................................................................138

F.4.8 Potential Errors and Recovery from Human Errors or System Failures.....................138

F.4.9 Potential Selection, Certification, and Training.........................................................139

F.5 Human Operations Issues, Design Issues, and Countermeasures for ASDO.......................140

F.5.1 Use of Automation and DSTs...................................................................................140

F.5.2 Human Decision-making and Workload...................................................................143

F.5.3 Attention, Situation Awareness, and Memory...........................................................145

F.5.4 Communication and Coordination ............................................................................148

F.5.5 Organization, Selection, Job Qualification, and Certification Factors........................150

F.6 Summary of Key Human Performance Issues for ASDO...................................................151

F.6.1 Use of Automation and Decision Support Tools.......................................................152

F.6.2 Human Decision-making and Workload...................................................................153

F.6.3 Attention, Situational Awareness, and Memory........................................................154

F.6.4 Communication and Coordination ............................................................................155

F.6.5 Organization, Selection, Job Qualification, and Certification Factors........................156

F.7 References ..................................................................................................................156
ix

Appendix G: Traffic Flow Management .......................................................................................157

G.1 Overview Description of TFM..........................................................................................158

G.2 Summary of Required Technology for TFM.....................................................................158

G.3 Summary of Human Roles for TFM .................................................................................158

G.3.1 National and Regional Traffic Flow Manager Roles and Responsibilities for TFM ..159

G.3.2 AOC Roles and Responsibilities for TFM................................................................161

G.3.3 Controller Roles and Responsibilities for TFM........................................................161

G.3.4 Pilot Roles and Responsibilities in TFM..................................................................162

G.4 Human Factors for TFM...................................................................................................162

G.5 Human Operations Issues, Design Issues, and Countermeasures for TFM ........................164

G.5.1 Questions for Use of Automation and DSTs ............................................................164

G.5.2 Questions for Human Decision-making and Workload.............................................166

G.5.3 Questions for Attention, Situation Awareness, and Memory ....................................168

G.5.4 Questions for Communication and Coordination......................................................170

G.5.5 Questions for Organization, Selection, Job Qualification, and Certification Factors.172

G.6 Summary of Key Human Performance Issues for TFM.....................................................173

G.6.1 Use of Automation and DSTs..................................................................................173

G.6.2 Human Decision-making and Workload ..................................................................174

G.6.3 Attention, Situation Awareness, and Memory ..........................................................175

G.6.4 Communication and Coordination ...........................................................................176

G.6.5 Organization, Selection, Job Qualification, and Certification Factors.......................177

Appendix H: Dynamic Airspace Configuration.............................................................................179

H.1 Overview Description of DAC .........................................................................................179

H.2 Summary of Required Technology for DAC.....................................................................179

H.3 Summary of Human Roles for DAC.................................................................................180

H.3.1 Airspace Manager Roles and Responsibilities for DAC ...........................................180

H.3.2 Controller Roles and Responsibilities for DAC........................................................181

H.3.3 Pilot Roles and Responsibilities for DAC ................................................................182

H.4 Human Factors for DAC ..................................................................................................182

H.4.1 Attention Allocation ................................................................................................183

H.4.2 Decision-making and Mental Modeling ...................................................................183

H.4.3 Communication .......................................................................................................184

H.4.4 Memory...................................................................................................................185

H.4.5 Workload.................................................................................................................185

H.4.6 Interaction with Automation, Decision Support Tools, and Displays........................186

H.4.7 Organizational Factors.............................................................................................186

H.4.8 Potential Errors and Recovery from Human Errors or System Failures ....................187

H.4.9 Potential Selection, Certification, and Training........................................................187

H.5 Human Operations Issues, Design Issues, and Countermeasures for DAC ........................188

H.5.1 Questions for Use of Automation and DSTs ............................................................188

H.5.2 Questions for Human Decision-making and Workload.............................................190

H.5.3 Questions for Attention, Situation Awareness, and Memory ....................................193

H.5.4 Questions for Communication and Coordination......................................................195

H.5.5 Questions for Organization, Selection, Job Qualification, and Certification Factors.196

H.6 Summary of Key Human Performance Issues for DAC ....................................................197

H.6.1 Use of Automation and DSTs..................................................................................198
x

H.6.2 Human Decision-making and Workload ..................................................................198

H.6.3 Attention, Situation Awareness, and Memory ..........................................................199

H.6.4 Communication and Coordination ...........................................................................200

H.6.5 Organization, Selection, Job Qualification, and Certification Factors.......................200

Appendix I: Integrated DAC-SA...................................................................................................203

I.1 Overview Description of Integrated ASA and DAC...........................................................203

I.2 Summary of Human Roles and Responsibilities for Integrated ASA and DAC...................203

I.3 Human Factors for Integrated ASA and DAC ....................................................................206

I.3.1 Attention Allocation..................................................................................................206

I.3.2 Decision-making and Mental Modeling.....................................................................207

I.3.3 Communication.........................................................................................................208

I.3.4 Memory ....................................................................................................................208

I.3.5 Workload ..................................................................................................................209

I.3.6 Interaction with Automation, Decision Support Tools, and Displays .........................209

I.3.7 Organizational Factors ..............................................................................................209

I.3.8 Potential Errors and Recovery from Errors or System Failures..................................210

I.3.9 Personnel Selection, Certification, and Training........................................................210

I.4 Human Operations Issues, Design Issues, and Countermeasures for Integrated ASA

and DAC...........................................................................................................................210

I.5 Key Human Performance Issues for Integrated ASA and DAC ..........................................210

I.5.1 Use of Automation and DSTs....................................................................................210

I.5.2 Human Decision-making and Workload....................................................................211

I.5.3 Attention, Situation Awareness, and Memory............................................................213

I.5.4 Communication and Coordination.............................................................................214

I.5.5 Organization, Selection, Job Qualification, and Certification Factors ........................214
xi

Acronyms and Abbreviations

3-D................ three-dimensional (three

dimensions of space)

4-D................four-dimensional (three

dimensions of space plus time)

AAC.............. Advanced Airspace Concept

AAL.............. American Airlines

ADS-B .......... automatic dependent

surveillance-broadcast

ANSP ............ air navigation service provider

AOC.............. airline operations center

AOL.............. Airspace Operations

Laboratory (NASA Ames

Research Center)

ARTCC ......... Air Route Air Traffic Control

Center

ASA .............. Automated Separation

Assurance

ASDO............ Airspace Super Density

Operations

ASQ .............. Atlantic Southeast Airlines

(callsign: Acey)

ATC .............. air traffic control

ATM ............. air traffic management

ATWIT.......... Air Traffic Workload Input

Technique

BTA ..............Jetlink

CD&R........... conflict detection & resolution

CDA.............. continuous descent approach

CDTI............. cockpit display of traffic

information

CSPA ............ closely spaced parallel

approaches

CSPR............. closely spaced parallel runways

CW................ cognitive walkthrough

DAC ..............Dynamic Airspace

Configuration

D-side ............data side controller

DST ...............decision support tool

ERAM ..........En Route Automation

Modernization

FAA...............Federal Aviation

Administration

FDMS............flight deck merging & spacing

FLG ...............Flagship/Pinnacle Airlines

FMS...............flight management system

HAA ..............high altitude airspace

HF..................human factors

HITL..............human in the loop simulation

IFR ................instrument flight rules

JPDO .............Joint Planning and

Development Office

LCD...............liquid crystal display

LOFT.............Line Oriented Flight Training

LoS................loss of separation

MSAT............mid term separation assurance

tool

NAS...............national air space

NASA TLX....NASA task load index

NASA............National Aeronautics and

Space Administration

NextGen.........Next Generation Air Traffic

Management System

NWA .............Northwest Airlines

PM.................prospective memory

RFA...............research focus area

RNAV............area navigation

RNP...............required navigation

performance
xii

R-side ............radar side controller

SA .................separation assurance

SAGAT ......... Situation Awareness Global

Assessment Technique

SSAT.............short term separation assurance

tool

SUA ..............special use airspace

SWA ............. Southwest Airlines

SWIM............system wide information

management

TBO .............. trajectory based operations

TCAS ............ Traffic Alert and Collision

Avoidance System

TFM .............. traffic flow management

TMA ............ Traffic Management Advisor

TMI............... traffic management initiatives

TMU ............. traffic management unit

TOC .............. transfer of communication

TRACON ...... Terminal RAdar CONtrol

TSAFE .......... Tactical Separation Assisted

Flight Environment
1

Abstract

This report identifies key human-performance-related issues associated with

Next Generation Air Transportation System (NextGen) research in the NASA

NextGen-Airspace Project. Four Research Focus Areas (RFAs) in the NextGenAirspace Project – namely Separation Assurance (SA), Airspace Super Density

Operations (ASDO), Traffic Flow Management (TFM), and Dynamic Airspace

Configuration (DAC) – were examined closely. In the course of the research, it was

determined that the identified human performance issues needed to be analyzed in the

context of NextGen operations rather than through basic human factors research. The

main gaps in human factors research in NextGen were found in the need for accurate

identification of key human-systems related issues within the context of specific

NextGen concepts and better design of the operational requirements for those

concepts. By focusing on human-system related issues for individual concepts, key

human performance issues for the four RFAs were identified and described in this

report. In addition, mixed equipage airspace with components of two RFAs were

characterized to illustrate potential human performance issues that arise from the

integration of multiple concepts.

Executive Summary

This report identifies and characterizes key human performance-related issues associated with Next

Generation Air Transportation System (NextGen) research in the NASA NextGen-Airspace Project.

Four Research Focus Areas (RFAs) in the NextGen-Airspace Project—namely Separation

Assurance (SA), Airspace Super Density Operations (ASDO), Traffic Flow Management (TFM),

and Dynamic Airspace Configuration (DAC)—were examined. In the course of our research, we

concluded that the identified human performance issues need analysis in the context of NextGen

operations rather than through basic human factor research. We also concluded that the work to

address human factors research and develop potential countermeasures lie in the accurate

identification of key human-systems related issues, study of the issues in the context of the concept

operations, and a better design of the operational requirements for the concept to address the issues.

A review of the NextGen concepts and related human factors research identified nine general human

factors categories: 1) attention allocation; 2) decision-making and mental modeling; 3)

communication; 4) memory; 5) workload; 6) interaction with automation, decision support tools

(DSTs), and displays; 7) organizational factors; 8) potential errors and recovery from errors and

failures; and 9) potential selection, certification, and training. These human factors categories were

used as the basis to generate human performance questions that cover safe and efficient human

performance under NextGen operations. Forty-six questions were generated; each question was
2

analyzed for each RFA and each question was rated as high, medium, or low regarding its priority to

the particular RFA. The analysis of these questions in the context of individual RFAs allowed us to

determine the key issues for each RFA and recommend how to mitigate these issues in concept

research/design. The key human performance issues for the four RFAs, as well as the potential

countermeasures (i.e., recommendation of next steps in concept refinement or research), are

identified and described in this report.

Key Human Performance Issues for Separation Assurance

In the Separation Assurance RFA, we examined an Automated Separation Assurance (ASA) concept

that utilizes an automated conflict detection and resolution system to provide separation assurance

for ASA-equipped aircraft in nominal situations and a human operator (whom we call “ASA

Manager”) to provide an additional layer of service and safety by handling conflicting pilot requests

and short-term weather events. Under these assumptions, the key ASA human performance issues

revolved around the following two areas:

1. Trust in automation and distribution of roles where automation is central. A concept that

requires automation to perform a central role in conflict situations that are both time and

safety-critical raises a series of issues related to trust and reliability of the automation. The

task distribution and coordination is especially important if ASA airspace allows mixed

equipage with controllers managing aircraft without ASA-equipage. Key issues to resolve

upfront are 1) how to prevent confusion in task execution and coordination by having clear

roles and responsibilities; and 2) how to build in layers of human and automation functions to

provide redundancy necessary for safety into the system.

2. Human operator manages exception cases and is “out-of-the-loop.” If automation handles the

nominal situations and the ASA Manager is brought “in-the-loop” when his/her assistance is

necessary, the available time for decision-making plays a critical role. Key issues to resolve

are 1) human factors research to understand how much time and information are needed to

make safety-critical decisions; 2) smart DST/automation design that can alert and/or display

appropriate information to the human to bring him/her in-the-loop; 3) research and design to

develop efficient coordination procedures between the ASA Manager, pilots, and automation

when action is needed in time-critical situations; 4) human factors research to understand how

high workload affects time-critical decision-making and how low workload affects vigilance

and its impact on safety; and 5) how to train and maintain skill set for the ASA Manager on

the safety-critical separation-related tasks when automation handles the nominal cases and

takes away the opportunity to practice those skills.

Some of these issues were selected to further explore by conducting a cognitive walkthrough of

traffic scenarios and soliciting feedback from controllers about five key human performance issues:

1) ASA Manager trust in the automation; 2) difficulty forming situation awareness when s/he is

brought in-the-loop of the traffic situation; 3) potential high workload with high traffic levels; 4)

having enough time to make decisions once s/he is brought in-the-loop; and 5) clarity of delineation

of roles and responsibilities.

Four scenarios with difficult short-term conflict situations had been captured in a movie under

another project and were shown to six controller participants. One hundred and thirty six questions

related to human factors and human performance issues were asked of the participants and analyzed

for common themes. The feedback suggested that issues 1, 2 and 3 were not problematic: 1)
3

participants trusted the automation since such trust was a precondition to controlling traffic in the

ASA environment; 2) they had no problem forming traffic situation awareness in short-term conflict

situations; and 3) workload in general did not pose a problem. However, participants reported

different feedback for the remaining two issues: 4) the decision-making timeframe of three minutes

was potentially too short, and 5) the roles and responsibilities between human and automation

needed to be better delineated.

The walkthrough demonstrated that once the key issues are identified, methods such as walkthrough

or human-in-the-loop (HITL) simulations can answer and bring insight into the validity of the

identified issues.

Key Human Performance Issues for Airspace Super Density Operations

In the Airspace Super Density Operations RFA, many operational concepts—such as Flight-Deck

Merging & Spacing (FDMS), Closely-Spaced Parallel Spacing (CSPA), Optimized Profile Descent

(OPD), precise arrival/departure routing that meets Required Navigation Performance (RNP), etc.—

are turned on or off depending on the traffic situation. Under these assumptions, the key underlying

human performance issues revolved around the following three areas:

1. Shifting roles among multiple operational modes that turn on and off. An operational

environment with different operational modes and distributed responsibilities across the Traffic

Manager (assumed human operator to manage arrival and departure routes), supervisor,

controller, pilot, and ground/flight-deck automation needs clear delineation of roles and a clear

decision authority in case of disagreement. The roles can shift over time as the operational

modes turn on or off and there are different combinations of operational modes. These roles

and responsibilities should be a part of the concept design and/or refinements. Methods need to

be developed to train future controllers to work under multiple operational modes with many

possible error and failure states.

2. Workload, situation awareness, and coordination under multiple modes of operation.

Possibility of multiple modes of operations requires attention to workload, situation awareness,

and coordination. The issues can be resolved by: 1) good DST and display design to smartly

identify and highlight different operational modes and off-nominal/emergency situations

without adding clutter; 2) human factors research to determine workload impact and possible

modal confusion with multiple operational modes; and 3) human factors research to measure

peak workload in relation to traffic complexity under different combinations of operational

modes and under mixed equipage.

3. Emergency and safety-critical situations under multiple operational modes. A systematic look

at emergency and safety-critical situations is needed in a diverse set of possible operational

modes. In addition to the training for errors and failures and DST design for off-nominal/

emergency situations, the following issues should be resolved: 1) recovery strategies should be

developed for known possible errors and failures; 2) research on modal confusion should be

applied to prevent possible human errors; and 3) potential high workload resulting from

multiple operational modes and emergency/safety-critical situations should be mitigated with

better procedures and tools designs, as well as more refined human factors research.
4

Key Human Performance Issues for Traffic Flow Management

In the Traffic Flow Management RFA, the National Traffic Flow Manager is expected to use

advanced DSTs and algorithms that gather the best traffic and weather predictions and estimate

demand/capacity in National Airspace System (NAS) in order to suggest alternative optimal traffic

schedules. Starting from these constraints and initial schedule alternatives, the National Traffic Flow

Manager (e.g., Command Center) and Regional Traffic Flow Manager (e.g., Traffic Management

Unit) must then coordinate with airline dispatchers to negotiate user-preferred routes and input them

into a scheduler that can take them, as well as the best traffic and weather predictions, to build the

schedules. The coordination must also occur whenever predictions and schedules change

significantly enough to require a re-plan. Under these assumptions, the key underlying human

performance issues revolve around the following three areas:

1. Use of sophisticated DSTs based on factors that might not be fully understood. Traffic Flow

Manager in NextGen needs to be able to utilize potentially sophisticated algorithms and DSTs

that provide schedule solutions that are based on factors that the person might not fully

understand (e.g., probabilistic weather prediction, a complex set of airport throughput

constraints that affect the schedule, algorithms that optimizes over metrics that are not easily

translatable in non-mathematical terms, etc.). His/her ability to utilize such DSTs hinges on

many factors such as 1) designing DSTs and algorithms that can provide complex information

in easily understandable format to the human operators; and 2) having a selection process that

identifies persons that can understand the complex set of constraints and advanced machine

systems in the future air traffic system and a training process that educates them to fully utilize

the DSTs and understand the intentions of the algorithms/automation.

2. Complex coordination. The process of selecting the appropriate schedules; coordinating and

negotiating the solutions with other ATM personnel and with the airlines; and replanning/coordinating whenever the schedule changes significantly based on changes in

weather or traffic information involves significant and complex coordination as a central part

of the activity. Therefore considerable research and development is needed to 1) leverage past

research in collaborative decision-making and conduct focused studies on specific aspects of

coordination in TFM; 2) develop a set of coordination procedures for both normal and offnominal situations; and 3) develop DSTs that intelligently support non-verbal coordination via

a ground-to-ground data communication system that is integrated into their tools.

3. Workload and potential cognitive tunneling under significant route planning and offnominal/emergency situations. A significant re-planning due to severe changes in the traffic

situations is difficult in terms of workload, situation awareness, and coordination in current

operations and is expected to be even more complex in NextGen. Given the complex traffic

problem and potential overload of information, the Traffic Flow Manager may fixate on one

traffic problem while neglecting others. Efforts are needed to 1) design the automation to

recognize and trigger attention for the changing traffic situations, especially discrete offnominal/emergency situations (e.g., decision by a Tower facility to shut down an airport) and

synchronize its information with the Traffic Flow Manager; 2) design DSTs that provide only

relevant information without clutter to human operators; 3) conduct research on attention and

decision-making strategies that can help with the design; 4) develop and evaluate coordination

mechanisms and procedures for use in off-nominal/emergency situations; and 5) assess the

workload impact of significant schedule changes.
5

Key Human Performance Issues for Dynamic Airspace Configuration

In the Dynamic Airspace Configuration RFA, we expect that demand-capacity balance can be

achieved by selectively managing the airspace capacity in conjunction with managing the traffic

demand via airspace reconfiguration. We assume that a National “Airspace Manager” exists to

determine the daily airspace configuration using advanced DSTs and algorithms that gathers best

traffic and weather predictions and estimates demand/capacity in NAS in order to suggest alternative

airspace configurations to best optimize NAS efficiency. Once the airspace configuration is set, we

expect airspace reconfiguration to be done at a local level in which a Regional Airspace Manager

assesses traffic demand and airspace complexity a few hours in advance to either identify sectors

that may become over and under capacity or to anticipate significant route changes due to

weather/other events that are expected to shift future demand on airspace. The airspace changes are

then expected to be coordinated with supervisors and controllers to carry out the reconfiguration

plan. The coordination must also occur whenever the predictions and schedules change significantly

enough to require a re-plan. Under these assumptions, the key underlying human performance issues

revolve around following two issues, which are similar to those under TFM:

1. Use of sophisticated DSTs based on factors that might not be fully understood. Analogous to

TFM, the Airspace Manager in NextGen needs to be able to utilize potentially sophisticated

algorithms and DSTs that provide airspace reconfigurations based on factors that the person

might not fully understand. His/her ability to do that hinges on many factors such as 1)

designing DSTs and algorithms that can provide complex information in an easily

understandable format to the human operators; and 2) forming a process to identify and train a

person with appropriate aptitude for understanding the complex set of constraints in ATM and

advanced algorithms and automation.

2. Workload under significant route planning and off-nominal/emergency situations. The impact

of airspace design and reconfiguration on the controller’s workload, situation awareness, and

coordination needs to be examined. The issues can be resolved by 1) good DST and display

design to smartly display, coordinate, and propagate new airspace configuration; 2)

development of operational procedures for coordination during a boundary change; 3) human

factors research to assess workload impact in relation to airspace design, traffic complexity,

and other factors during a boundary change; and 4) potential safety impact of conflict-related

or other safety-critical events that occur during a boundary change.

Integration of Multiple Research Focus Areas

In order to march towards an integrated NextGen operation in the future, individual concept

elements need to be integrated into a larger NAS-wide concept. Within the NASA NextGenAirspace project, concepts in the Research Focus Areas (RFAs) should be combined to examine the

operational implications and the impact on the human-system performance. Some potential

combinations of the existing RFAs are:

• DAC-TFM: The traffic demand/capacity imbalance can be resolved by 1) Airspace Manager

reconfiguring the airspace to balance average traffic load across sectors, and 2) Traffic Flow

Manager re-routing or delaying the few remaining aircraft to manage transitory peaks in the

traffic.

• TFM-ASDO: National Traffic Flow Manager determines the national schedule and propagates

it to a metroplex area. Regional Traffic Flow Manager aligns it with a local schedule and plans
6

arrival and departures routes to the metroplex accordingly and coordinates the solution with the

Terminal controllers/supervisors.

• DAC-SA: One class of airspace that can be reconfigured to balance controller workload is ASA

airspace with ASA Managers vertically stacked above non-ASA airspace with controllers. Due

to weather, airspace reconfiguration is needed across the ASA and non-ASA airspace classes,

resulting in mixed equipage situations during the transition of airspace reconfiguration.

In our report, DAC-SA combinations have been examined at a cursory level to demonstrate how key

human performance issues that arise from integrations of multiple concepts can be examined. Our

partial analysis did not have details that a full analysis would have if a more careful examination of

the assumptions about the integrated concept and a closer look at the possible issues were

completed.

Conclusion

Our study identified nine general human factors categories that are accepted categories in human

factor research and which were assessed over each of the four RFAs in the NASA NextGenAirspace project. We generated 46 questions that address human performance from the operational

systems design perspective. Each question was analyzed for each RFA, and each question was rated

as high, medium, or low regarding its priority to the particular RFA. The questions rated high for

each RFA have been aggregated and covered in this summary. Key human performance issues were

similar across the RFAs but the details of how the issues played out, who were impacted by the

issues, and the potential countermeasures differed somewhat for each RFA, thereby supporting our

initial hypothesis that key human performance issues should be identified and mitigated for the

individual concepts.

Our approach to analyze across all instances of the concepts is time and labor intensive. Despite the

large volume of analyses in our full report, we were able to cover only few of the concepts in each of

the RFAs and had to omit others. We also completed only a limited analysis of the integration of

multiple concepts. Clearly more study is needed.

Future work should include the analyses of the other concepts in the RFAs. Once the key human

performance issues are identified, more resources should be designated to run multiple walkthroughs

on a selected set of concepts that are of particular interest to the program managers and decisionmakers. Further analysis is needed in assessing key issues related to integration of multiple concepts.

Operational assumptions need to be defined for combined concepts since there are few details of the

integrated NextGen system. Any information that can improve the technological and operational

assumptions about a concept will significantly improve the final outcome of the analyses.

Many of the recommendations for the countermeasures included the following set of steps: 1) further

define the concept; 2) develop prototypes of the tools and operational procedures; and 3) evaluate

the concept via walkthroughs or HITL simulation. These steps are inherently a design process, in

which many decisions about the tasks, procedures, algorithms, displays, and technological

assumptions need to be made in order to evaluate the concept. These steps are integral to designing

and conducting HITL simulations, and yet the reports that describe the outcome of the simulations

focus solely on the results and analyses. Documenting design decisions and their rationale could
7

provide invaluable insights into NextGen concept refinements since they offer valuable “hands-on”

experience of a world that has yet to be created.

In summary, we have outlined a method of analysis to identify human performance issues related to

NextGen concepts that can be turned into a set of next steps to address the potential gaps in the

concept. We hope that other NextGen concept areas can utilize the general human factors issues and

questions that we have generated (see full report and Appendices) and apply them in their respective

concepts to identify key human performance issues and define future research and design to address

the issues.
8
9

1. Overview

This report describes identification and characterization of key human performance-related issues

associated with Next Generation Air Transportation System (NextGen) research in the National

Aeronautics and Space Administration (NASA) NextGen-Airspace Project. Four Research Focus

Areas (RFAs) in the NextGen-Airspace Project were examined: Separation Assurance (SA);

Airspace Super Density Operations (ASDO); Traffic Flow Management (TFM); and Dynamic

Airspace Configuration (DAC).

The objectives for this research are to:

1. Identify human factors issues related to NextGen operational contexts

2. Identify gaps in existing human factors research as it relates to needs in NextGen concepts and

provide potential countermeasures for system design and operation

3. Delineate particular human performance issues that are critical for further study and mitigation

4. Discuss methods for refining operational requirements related to human performance issues

An additional supporting objective was added to support the first:

5. Understand the operational contexts of NextGen technologies as they might influence human

task performance

In the course of our research, we concluded that the identified human performance issues need

analysis in the context of NextGen operations rather than through basic human factor research. We

also concluded that the gaps in human factors research and potential countermeasures lie in the

accurate identification of key human-systems related issues, study of the issues in the context of the

concept operations, and a better design of the operational requirements for the concept to address the

issues.

Existing human factor research knowledge was used to guide our analyses. We have expanded prior

research and analysis of human-system requirements, both in general and for aviation systems, to

identify specific human performance issues associated with NextGen. The outcome of this report

provides the NextGen-Airspace Project with the basis to conduct a thorough exploration of and
10

develop mitigations for human factors issues to be addressed in realizing the NextGen vision. The

report also supports the following two NextGen-Airspace Project milestones:

1. AS 1.2.02: Synthesis of human factors and operational literature

2. AS.1.7.01: Develop initial system-level conops. Leverage JPDO NGATS ConOps (now called

the Operational Concept for the Next Generation Air Transportation System [NextGen

ConOps]) and expand development as required to support Airspace Systems Program research

and concept development

Before we discuss the content of this report, we will comment briefly on its organization. We have

performed the following tasks: 1) the identification of the general human factors categories; 2)

preparation of descriptions of the four RFAs; 3) examination of the potential human performance

issues for each RFA; and 4) conducted a cognitive walkthrough of one RFA. Each activity merits a

full report by itself and writing in detail about all our activities in a single report would likely inhibit

readability. Therefore, we have chosen to keep the report brief by providing a summary of our

results and to attach the details of our analyses as appendices. Our overall report organization is: 1)

an Executive Summary that summarizes the findings and recommendations regarding mitigating

human performance concerns that is suitable for those wanting a overview of human factor issues in

the NextGen-Airspace Project; 2) the report that provides information for decision-making on what

human factor issues should be addressed through further research and concept design; and 3)

detailed appendices aimed at researchers conducting concept design and human analysis on specific

RFAs. The attached appendices are:

Appendix A: Human Factors Reference List. An extensive list of general and applied human

factors literature related to aviation

Appendix B: General Human Factors Categories. Descriptions of nine general human factors

categories that we identified as relevant to NextGen. Potential metrics and the

method of measurement for each category are also discussed. Analysis of the nine

categories of human factor issues for each RFA is reported in the RFA-specific

appendices (Appendices D, F, G, H, and I).

Appendix C: Human Performance-Related Questions that Are Relevant to NextGen. A list of 46

questions related to human factors that should be addressed during concept design.

Analysis of these 46 questions for each RFA is presented in the RFA-specific

appendices.

Appendix D: Separation Assurance (SA). Describes the SA concept, technology assumptions,

human operators and their tasks, the nine human factors categories in the context

of SA, identification and prioritization of the key human performance issues, and

recommendations for potential countermeasures.

Appendix E: Cognitive Walkthrough of SA. Contains method and results of a cognitive

walkthrough of a ground-based Automated Separation Assurance concept under

SA Research Focus Area.

Appendix F: Airspace Super Density Operations. Contains the same information for ASDO as

the Appendix D does for SA.

Appendix G: Traffic Flow Management. Contains the same information for TFM as the

Appendix D does for SA.
11

Appendix H: Dynamic Airspace Configuration. Contains the same information for DAC as the

Appendix D does for SA.

Appendix I: Integrated DAC-SA. Explores human performance issues and countermeasures

when two RFAs (i.e., DAC and SA) are integrated and considered together.

Contains the same information for the two integrated RFAs as Appendix D

does for SA.

2. Background

NextGen is a major effort being conducted by the Joint Planning and Development Office (JPDO),

of which the Federal Aviation Administration (FAA) and NASA are major participants, to plan and

implement an air transportation system for the period 2010-2025 that will accommodate an expected

doubling of air traffic in the United States (JPDO, 2007; FAA, 2008). The European Community

(Eurocontrol) has a very similar project, Single European Sky ATM Research (SESAR) (2009).

In NextGen, automation is expected to play a greater role in managing complexity, and the human

roles in the NextGen operations are expected to migrate to more strategic management and decisionmaking. A scalable Air Traffic Management (ATM) system will be achieved through distributing

decision-making across different human operators (e.g., controllers, pilots, traffic managers,

dispatchers, etc.) and through automation (e.g., automatic conflict detection and resolution, DSTs for

traffic flow management and other activities, etc.) (JPDO, 2007). New surveillance, new digital

communication, better prediction of traffic and weather, and the ability to fly pre-planned 4-

dimensional trajectories (4D, 3 in space and one in time) are expected to be the technology enablers

for increasing the system capacity and efficiency.

NASA’s NextGen-Airspace Project is one of NASA’s efforts to support NextGen and is addressing

the demand/capacity imbalance. The NextGen-Airspace Project is grouped into seven RFAs to

address different technical objectives. Some RFAs focus strictly on capabilities, such as trajectory

prediction for Trajectory-Based Operations (TBO) in NextGen. Other RFAs, such as ASDO and

DAC, focus on NextGen concepts. We examined four concept-related RFAs (i.e. SA, ASDO, TFM,

and DAC) in the NextGen-Airspace project (Swenson, Barhydt, & Landis, 2006; SLDAST, 2007).

These four RFAs match the JPDO ATM operations—Separation Management, Trajectory

Management, Flow Contingency Management, and Capacity Management, respectively—that are

shown in Figure 1.

The four RFAs in the NASA NextGen-Airspace project can be linked to related activities in the

FAA, which focus on concepts/projects that can be implemented in the near- to mid-term timeframe.

These projects include High Altitude Airspace (e.g., Rowe, Borowski, Wendling & DeSenti, 2003),

Big Airspace (FAA, 2007), Time-Based Flow Management (e.g., Stalnaker, et al., 2008) and MultiSector Planner (e.g., Lee, et al., 2006). NASA and the FAA have formed Research Transition Teams

(RTTs) and are working together to provide transition paths for taking promising concept research to

implementation by aligning the near- to mid-term research/implementation plans from the FAA with

the mid- to far-term research by NASA on related topics.
12

Figure 1. Different ATM operations across different time horizons of decision-making.

3. Technical Approach

JPDO reports on NextGen suggest that roles and responsibilities of humans must change with the

introduction of new automation and operational procedures. In separation management, for example,

controllers may rely on automation to manage separation and become automation supervisors and

managers by exception. Pilots, using cockpit displays of traffic information (CDTIs) as well as

displays of weather, may be authorized to self-separate to a much greater extent than currently.

For these examples and for other NextGen operations, the relationship between human and

automation tools, as well as among human operators, changes significantly and thus requires

identification and examination of human-system interactions/issues for the NextGen concepts. This

report captures many of these interactions and human performance issues related to NextGen. In the

following chapter, we describe our general approach.

3.1 Identification of General Human Factors Related to NextGen

To begin, we identified generic human factors issues that are likely to be related to NextGen

operational contexts by reviewing existing human factors literature related to aviation. The compiled

reference list of human factors literature is in Appendix A.

When the generic human factors issues identified in the literature were applied to NextGen

operations described in various documents from NASA, JPDO, and the FAA, we found that many

general human factors issues covered in the literature are applicable to NextGen operations. We
13

believe that the following generic human factors categories play a critical role in a majority of the

NextGen concepts:

1. Attention allocation

2. Decision-making and mental modeling

3. Communication

4. Memory

5. Workload

6. Interaction with automation, decision support tools, and displays

7. Organizational factors

8. Potential errors and recovery from human errors or system failures

9. Potential selection, certification, and training

A more detailed description of the nine human factors categories is given in Appendix B. In

Appendix B, each category is discussed to provide an understanding of the category and its

implications for safe and efficient human performance in NextGen operations. The discussion is for

the benefit of researchers who may be less familiar with general human factors research. At the end

of the discussion of each human factor category in Appendix B, critical terms, common means of

measurement, and comments on our analysis of that particular human factor category are described

in italics. The measurements are the ones that can be taken during human-system evaluations such

as a human-in-the-loop (HITL) simulation. References related to the categories are listed at the end

of Appendix B.

Most of the nine human factor categories apply to the each of the NextGen-Airspace concepts, but

their importance differs based on the individual concept. For each of the four RFAs (i.e., SA, ASDO,

TFM, and DAC), the descriptions of concepts in RFA, the concepts’ benefit mechanisms, the

enabling technologies, and the general technology assumptions were studied. Since some

formulation of tasks humans will conduct is needed for an assessment of human factor issues, we

then developed a specific set of automation tool assumptions, operational procedures, and task

descriptions for each RFA and generated human-system interactions for each concept instantiation.

Based on this, the roles of the human operators and automation were generated. Since the concepts

are in the research stage, the exact final human roles are not known but we tried to develop roles at a

general level that would apply to however the concept might evolve.

The nine human factors categories were studied in the context of the concept instantiations as

developed for the RFAs as described above. These human factors were used as basis to ask the 46

human performance questions that apply to NextGen operations. The analysis of these questions in

the context of individual RFAs allowed us to determine the key human performance issues for each

RFA and to recommend how to mitigate these issues in concept research/design. The key human

performance issues for the four RFAs, as well as the potential countermeasures (i.e., recommended

next steps in concept refinement or research), are summarized beginning in Chapter 4 and are

covered in detail in Appendices D through I.
14

3.2 Human Performance Issues in the Context of the NextGen

Examination of the nine human factors categories in the context of a specific concept provides a

level of specificity regarding human factors that has been lacking in many prior efforts in

NextGen. The 46 questions we developed were used to analyze the NextGen-Airspace concepts

from the perspective of a concept design that needs to address human factors concerns. The 46

questions provide an appropriate level of detail to examine the nine human factors issues

categories as they apply to the NextGen-Airspace concepts. The 46 questions reflect what

designers should be addressing to ensure that human performance and human-system interactions

are adequately considered.

We will not present all 46 questions in this report. The full set of the 46 questions related to

NextGen human performance issues is described in Appendix C. Below is a sample of the questions

grouped into five human performance groupings. These questions or topics point directly to potential

approaches and countermeasures to mitigate the human performance issues of concern.

1. Use of automation and Decision Support Tools (DSTs)

• What is the distribution of roles and responsibilities across different human operators and

automation?

• Who has decision making authority when there are conflicting decisions or advice given by

some combination of automation and/or other operators?

2. Human decision-making and workload

• Is there a sufficient time for decision-making for critical situations, especially off-normal

situations?

• What is the workload impact when there is transitioning between airspace configurations or

different modes of operations?

3. Attention, situation awareness, and memory

• If the human operator is passively monitoring and suddenly there is a need to get back into

the loop, how long does it take to do this?

• Are there situations that will tax working memory capacity (e.g., immediate recollection of

highly detailed instructions received by voice communication or from a visual display)?

4. Communication and coordination

• Who or what has final authority? How will an end to negotiations be determined?

• Is the communication information that is shared among multiple operators appropriate

regarding content, detail, and type of format for each particular operator?

5 Organization, selection, job qualification, and certification factors

• What will be organizational structure, interactions between organizational units, and

delegation of authority to units?

• What skills, abilities, and traits are needed by personnel to perform the given tasks?
15

Recovery from potential human errors or system failures was applicable across the five human

performance question groupings above, and therefore the questions related to error recovery were

distributed across the categories.

The questions were compiled from reviewing prior work from this report’s authors and other

NextGen human factors researchers (e.g. Sheridan, et al., 2006; Willems and Koros, 2007; Andre, et

al., 2008; Funk, et al., 2009) and expertise gained from previous HITL simulation activities related

to NextGen concept evaluation (e.g., Prevot, et al., 2009; Kopardekar, et al., 2009).

3.3 Identification and Prioritization of Key Human Performance Issues for

Further Study and Mitigation

The key human performance issues were identified by applying the 46 questions in Appendix C to

each of the four RFAs, namely SA, ASDO, TFM, and DAC. When the human performance issues

were examined across the individual RFAs, it was clear that mitigation approaches and

countermeasures were not related so much to fundamental human factors or human-system

interaction research. Rather, the needs arose mostly from under-specification of NextGen concepts,

of operational procedures, and of supporting automation tools.

It seemed that most of the mitigation needs could be properly addressed after the concepts were

further developed with an accompanying intent to seriously engage human performance issues.

Mitigation of human performance issues will require refinement of the automation and decision

support tools (DSTs). There were a few instances where the analyses identified a particular

technology or assumption as being critical to a particular concept and therefore made the concept

vulnerable to a potential failure of that technology/assumption. These weak links were identified and

described.

Our approach to find key human performance issues was to answer the human performance-related

questions as described above for each of the four RFAs and to give a priority rating to each

question. The priority ratings are expressed as high, medium, or low. A question rated high

suggests needs for research and/or development on the particular question. To assign the priority

rating, we considered the:

• need for human factor research since the answer to the question is not readily evident for the

particular concept

• need for further concept design/definition to address the human factor question

• criticality for safety of concept

• criticality to achieving the desired benefit of concept

Once the questions were answered and given the priority ratings, we re-examined the questions that

were rated high and looked for common themes and drivers for the issues. Recommendations/

countermeasures were generated for both the questions and the common themes to provide

suggestions on next steps for research and development of the individual concept. The

recommendations include the following: 1) more information needed on the concept definition; 2)

more human factors research needed to understand some unanswered questions related to humansystem interaction; 3) operational procedures and decision support tools need to be prototyped for

further evaluation; and 4) either walkthroughs and/or HITL study is needed to evaluate the human-
16

system interactions. These general recommendations were tailored for each RFA to describe the

types of research/development that was needed to address specific issues.

3.4 Suggestions for Further Exploration of Key Human Performance Issues

through Cognitive Walkthroughs and HITL Simulations

The overall method of analyses described above can be used for identification, evaluation, and

prioritization of the human performance issues in NextGen. It outlines a set of generic human factors

that are related to NextGen, applies these factors to specific RFA concepts, identifies human

performance issues related to operations within the concepts, and then identifies mitigation strategies

and countermeasures. Concepts, operational procedures, and/or DSTs can be refined to address the

identified issues from the analysis.

A shortcoming in such a method is that the types of issues that could be examined are still quite

coarse. To examine the detailed interactions between human operators and their displays, DSTs,

automation, etc., a more detailed specification of tasks and information requirements is necessary.

An effective method of examining human-systems interactions and the related human performance

issues in more detail is to use cognitive walkthroughs.

To demonstrate how a cognitive walkthrough can provide a more detailed analysis to compliment

the coarser analyses, one of the RFAs, namely Separation Assurance, was chosen for further

examination. A detailed description of the Separation Assurance cognitive walkthrough is in

Appendix E. The Appendix presents both the method and the results of the study. In the

walkthrough, a number of controller participants watched a video, which had been made in another

project, of traffic scenarios with automation separated aircraft and gave feedback on the five

groupings human factors questions described above. The outcome of the walkthrough provided more

insights into which of the human performance issues identified in the general analyses resulted in

significant challenges and which ones were less problematic. The results of the walkthrough are

presented in Section 4.4 and in more detail in Appendix E.

A finer level of NextGen concept development, detailed refinement of operational requirements, and

extensive evaluation of human performance issues can be best accomplished with a HITL

simulation. Performing an example HITL simulation was outside the scope of this study. A

precondition of running a HITL simulation is developing tool prototypes, operational procedures,

and roles/responsibilities definitions of an instantiated concept. A HITL simulation can provide an

evaluation of human-human and human-automation interactions and other human performance

issues. While there are laboratory studies and conceptual modeling activities that are ongoing within

the nine separate human factor categories, adding to this literature on basic human factors is not

where NextGen human factors research emphasis should be. It should be squarely in studying the

interaction of human factors issues in the context of various operational innovations of NextGen.

We have outlined above our approach and method for identification of key human performance

issues and recommendation of mitigation strategies. In the following sections, we summarize the

results of this approach in identifying human performance issues related to the four RFAs in

NextGen-Airspace Project.
17

4. Separation Assurance

The NextGen-Airspace’s Separation Assurance (SA) research aims to substantially increase airspace

capacity (e.g., 2–3X) by utilizing trajectory-based technologies and human-automation concepts that

can safely maintain separation assurance for the increased traffic. The main requirement for adding

automation to separation assurance is to increase capacity by reducing the controller workload

bottleneck that results from monitoring aircraft separation by visual and cognitive analysis of a

traffic display and issuing control clearances to the pilots using voice communication. This

controller workload bottleneck limits how many aircraft a controller can handle in his/her sector and

hence limits sector capacity. Although advanced DSTs (e.g., data link trajectory-based advisory

information on conflict detection and resolution) have helped reduced the controller workload and

increased the sector capacity, a fundamental change in the way that separation assurance is provided

seems necessary to provide two- to three-fold increases in capacity.

The SA research is focused on developing automated separation assurance technology that can

generate robust automated resolution trajectories that are safe and efficient and thereby reduce

controller workload. SA under NextGen can potentially be provided under several modes: Conflict

Detection and Resolution (CD&R) provided by controller as today (e.g., classic airspace); fully

automated ground-based CD&R, which we will call Automated Separation Assurance (ASA) (e.g.,

the fully automated Advance Airspace Concept [AAC]); automated ground-based conflict detection

and ground-based automation recommending conflict resolutions to controller (e.g., AAC as a

Decision Support Tool); and self-separation (e.g., separation maintained by the pilots using flight

deck automation). Within these modes of SA, there could be mixed equipage where some aircraft

are equipped for full operation in an automated SA mode (either by ground and/or flight deck

automation) and other aircraft are not so equipped and would have SA provided by a controller.

In there SA modes, overall human-system operations need be evaluated by examining the roles and

responsibilities of controllers, pilots, and automation in different service-provider-based and

operator-based concepts of operations. Research is also needed to identify human and/or automation

failures, such as missed conflict alerts and data link failure, in order to design safe recovery modes.

4.1 Human Roles and Technology Assumptions for Ground-based Automated

Separation Assurance

In order to limit the overall scope, we picked the ground-based Automated Separation Assurance

(ASA) concept that utilizes automated conflict detection and resolution system to provide separation

assurance for ASA-equipped aircraft in nominal situations and a human operator (whom we call

“ASA Manager”) to provide an additional layer of service and safety by handling conflicting pilot

requests and short-term weather events.

The ground-based technology provides routine conflict detection and resolution (CD&R) for aircraft

without a human operator. The ASA also deals with routine pilot requests for routine changes

without a human operator. A human operator will become involved if the automation cannot

provide a conflict-free route to meet a pilot request and to deal with pop-up weather. ASA has a

medium-term (e.g., between 3 to 12 minute timeframe from Loss of Separation [LoS]) CD&R

capability that probes along the flight trajectories and automatically resolves conflicts. ASA also

has an independent tactical conflict resolution system (e.g., Tactical Separation Assisted Flight

Environment; TSAFE) that separates aircraft in the short timeframe (e.g., between 1 and than 3
18

minutes from LoS). An independent Traffic alert and Collision Avoidance System (TCAS) provides

the final layer of safety by advising coordinated collision avoidance maneuvers to the flight deck

(e.g., less than 1 minute from LoS).

Aircraft flying under ASA are assumed to have data link capability to uplink 4-D trajectories. The

aircraft can also downlink aircraft trajectory intent (e.g., data link and/or ADS-B). Aircraft are

expected to fly 4D trajectories within a certain Required Navigation Performance (RNP). Weather

information is expected to be available for both ground and air via common weather database.

Three human roles are described in this section on ASA:

1. ASA Manager: provides support for equipped aircraft flying under ASA

2. Controller: manages and provides separation for aircraft that cannot fly under ASA. In this

operational context, Controller is only involved in mixed equipage airspace.

3. Pilot: executes clearances issued by either the automation or a human operator. When

necessary, the pilot coordinates the maneuvers with an ASA Manager or a Controller. Pilot

can also request re-routes.

Both a dedicated ASA and mixed equipage with some ASA equipped aircraft and some unequipped

aircraft were assessed. Two mixed equipage cases were examined of the ASA Manager handling

unequipped aircraft (similar to a Controller) as well as handling ASA equipped aircraft, and of the

ASA Manager dealing only with SA equipped aircraft with a separate Controller handling

unequipped aircraft. Roles and responsibilities were delineated for the three human operators, and

these form the basis for identifying human performance issues. The ASA human operators’ roles and

responsibilities are listed in Appendix D.3.1.

4.2 Key Human Performance Issues for ASA

In this section, key human performance issues and potential countermeasures are summarized for the

ASA concept. These issues and countermeasures were derived by: 1) answering the 46 human

performance-related questions in Appendix C for ASA; 2) identifying the key issues from the

questions; and 3) consolidating issues with common threads together. The details of the

identification and prioritization of the key human performance issues and recommendations for

potential countermeasures, as well as the descriptions of the nine human factors categories in the

context of ASA, are reported in Appendix D.

The main drivers from ASA for the human performance issues are the central role of the automation

in the concept, the ASA Manager becoming involved in time and safety-critical situations at the last

minute, and the presence of mixed equipage. The human performance issues that were rated high in

our prioritization are presented in the following sub-sections. Other issues rated medium and low

can be found in Appendix D.

4.2.1 Distribution of Roles and Responsibilities between Humans and Automation

Issue: Roles and responsibilities are not clearly delineated in the SA concept.

• There needs to be a clear transition from when the automation is handling the traffic and when

the human takes over, such as when automation is responsible for nominal separation and the

ASA Manager handles off-nominal pilot requests and conflict situations.
19

• There needs to be a clear transition from one algorithm to another if two automation

capabilities handle a conflict situation (i.e., a strategic conflict resolver and a short-term

tactical conflict resolver).

• A clear delineation of roles is necessary if there are two human operators handling different

traffic in mixed equipage.

• There needs to be a clear decision-making authority during short-term conflict or other critical

situations when multiple humans and automation can take action to solve the same traffic

situation.

Potential Countermeasure/Next Step: A systematic analysis of all combinations of human

and automation roles should be cataloged, and a select set of promising combinations

should be explored further by concept refinement, cognitive walkthroughs, or HITL

simulation.

4.2.2 Brittleness and Human Operator Trust in the Automation and Decision Support Systems

Issue: Given that the SA concept requires a high level of dependence on automation in a safetycritical environment, error in the automation can lead to a brittleness of the concept unless sufficient

layers of automation and human redundancy are built in.

• SA automation will have layers of automation redundancy (e.g., capabilities like strategic

conflict resolver [like Advanced Airspace Concept, AAC], short-term tactical conflict resolver

[like Tactical Separation Assisted Flight Environment, TSAFE], and Traffic Collision

Avoidance System [TCAS]). The presence of the ASA Manager to handle situations that the

automation cannot is another level of redundancy.

• The ASA Manager might not be able to act as a useful back-up if s/he has little time to act.

• It seems that someone will be required to monitor ASA Manager workload, stress, etc., in ways

that Front Line Managers monitor controllers today, but it is unclear what that role will be.

• The human operator needs to trust the information that the automation provides and the

solutions that it comes up with during time and safety critical situations.

Potential Countermeasure/Next Step: A human-system analysis is needed to chart out the

chain of human and automation functions that should be developed to build redundancy in

the SA concept. A series of walkthroughs should be conducted to step through various offnominal situations to see if the redundant functions of humans and automation will be

sufficient to recover from the situation. HITL simulations should be conducted when the

procedures to handle off-nominal situations have been narrowed.

4.2.3 Demand for Workload, Attention, Situation Awareness, and Coordination

Issue: Peak workload can be an issue for the ASA Manager if s/he is given a large airspace with an

assumption that automation takes care of most of the tasks during nominal operations but during offnominal traffic and/or weather events many aircraft require the ASA Manager’s attention at the same

time.

• Issues for attention, situation awareness, and memory are mainly driven by low workloads for

the ASA Manager while SA automation is providing Separation Assurance but needing to

become involved when SA automation cannot handle a pilot request or other situation.

• Once the ASA Manager is alerted of an off-nominal conflict, request, or emergency situations,

s/he needs to assess with whom to coordinate before taking action.
20

• Coordination might be necessary with multiple pilots who may not have full awareness of the

situation as well.

• If both ASA-equipped and unequipped aircraft can fly in the same airspace, the specific

interaction between ASA Managers and Controllers needs to be designed.

• Peak workload could be a problem for the Controller during mixed operations if s/he needs to

maneuver unequipped aircraft around ASA-equipped aircraft during off-nominal traffic and/or

weather situations.

Potential Countermeasure/Next Step: Each of the issues mentioned above should be

examined, and initial designs of the concept instantiations, procedures, and tool prototypes

developed. Then concept-focused walkthroughs or HITL simulations should be conducted

to evaluate the feasibility of the designs, procedures, and roles.

4.2.4 Dealing with Emergency, Safety Critical, and Off-Nominal Events

Issue: Since automation handles safety-critical responsibility in this concept, its potential failure

needs to be examined and sufficiently mitigated within the concept. All potential failure modes that

can be identified should be examined to see if there can be adequate recovery.

• If the human is expected to intervene, there needs to be a set of indicators that would act as

diagnostics for the automation failure to provide information to the human.

• Similarly when an emergency or off-nominal traffic situation occurs (e.g., data link failure), the

automation needs to be “aware” of the off-nominal situation and act accordingly.

• Procedures also need to be built to recover from the situation.

Potential Countermeasure/Next Step: The above issues are difficult concept-related

questions. A list of off-nominal, safety-critical situations should be generated and each of

the situations should be further analyzed through a cognitive walkthrough or functional

analysis to explore potential concept refinements that can mitigate the issue. Once a set of

mitigating designs are developed, procedures, tools, and traffic situations should be

developed and tested to evaluate the efficacy of the solutions.

4.2.5 Human Operator Characteristics and Training

Issue: If the ASA Manager handles the off-nominal conflict, request, or emergency situations but the

nominal situations are handled by the automation, the following may occur:

• The human may not have enough exposure to the traffic situation to effectively recognize and

resolve the situation in a time-critical manner.

• Finding a training procedure to keep their monitoring and decision skills sharp when the

occurrence is rare will be a challenge.

Potential Countermeasure/Next Step: A systematic analysis should be conducted to

determine which human operator skills that exist today will likely become degraded in the

future through being used rarely. In particular, the skills for the ASA Manager that are

critical to the job but may not be often used should be compiled. Once the skills are

cataloged, recurrent training exercises for rare and off-nominal events for the ASA

Manager should be developed.
21

4.3 Main Drivers of the Human Performance Issues for ASA

Examining the issues identified above, the key human performance issues revolve around the

following two areas:

1. Trust in automation and distribution of roles where automation is central. A concept that

requires automation to perform a central role in conflict situations that are both time and

safety-critical raises a series of issues related to trust and reliability of the automation. The task

distribution and coordination is especially important if ASA airspace allows mixed equipage

with controller-managed aircraft without ASA-equipage. Key issues to resolve upfront are: 1)

how to prevent confusion in task execution and coordination by having clear roles and

responsibilities; and 2) how to build in layers of human and automation functions to provide

redundancy and safety into the system.

2. Human operator manages exception cases and is “out-of-the-loop.” If automation handles the

nominal situations and the ASA Manager is brought “in-the-loop” when his/her assistance is

necessary, the available time for decision-making plays a critical role. Key issues to resolve

are: 1) human factors research to understand how much time and information are needed to

make safety-critical decisions; 2) smart DST/automation design that can alert and/or display

appropriate information to the human to bring him/her in-the-loop; 3) research and design to

develop efficient coordination procedures between the ASA Manager, pilots, and automation

when action is needed in time-critical situations; 4) human factors research to understand how

high workload affects time-critical decision-making and how low workload affects vigilance

and its impact on safety; and 5) how to train and maintain the skill set for the ASA Manager on

the safety-critical separation-related tasks when automation handles the nominal cases and

takes away the opportunity to practice those skills.

4.4 Cognitive Walkthrough of ASA

In the previous section, key human performance issues were raised for ASA. Some of these issues

were selected to be further explored by conducting a cognitive walkthrough of traffic scenarios and

soliciting feedback from controllers. The cognitive walkthrough described below is an illustration of

a method to begin to find answers to questions that need influence concept design. This section

summarizes the method and the results of the walkthrough. The details of the walkthrough are

reported in Appendix E.

A number of the issues raised in the previous section were studied in the walkthrough, but other

issues could not be included because there was not enough time and resources to run a longer study.

The walkthrough was focused on the ASA Manager in pure ASA operations (i.e., no mixed equipage

and no controller). Based on the issues identified in the previous section, five key human

performance issues were selected for further examination:

1. ASA Manager could have limited or low trust in the conflict detection and resolution tools.

2. ASA Manager may have difficulty forming situation awareness when s/he is brought in-theloop of the traffic situation.

3. ASA Manager could have high workload when traffic levels are high.

4. ASA Manager may not have enough time to make decisions once s/he is brought in-the-loop.
22

5. The delineation of roles and responsibilities between ASA Manager and automation is not

clear.

One hundred and thirty six questions related to human factors and human performance issues were

asked and analyzed for common themes. The questions were aligned with both the nine human

factors categories and the issues identified in the section 4.2.

Four scenarios with difficult short-term conflict situations were captured in a video under another

previous project. The video was shown to six controller participants. Each of the four events was

shown three times to each participant. First, the event was played in real time. Participants were

free to ask questions or make comments as they watched. Second, the event was stepped through.

The video was forwarded to each breakdown step and then paused at a point just as the action for

that step had happened. Figure 2 shows an example of the way the problem looked at as it was

paused when the participants were asked the questions.

Figure 2. Snapshot of the movie playback in the walkthrough as a resolution is sent to the

American Airlines 140.

The overall results based on the participant feedback suggested that the first three issues listed above

were not problematic:

1. Participating controllers trusted the automation for safety-critical events. Since the concept

would not work without trust in the automation, participants accepted trust as a precondition

to controlling traffic in the ASA environment.

2. They had no problem forming traffic situation awareness in short-term conflict situations by

using their existing knowledge of conflict monitoring in the current operations.
23

3. Workload in general did not pose a problem. In general, the number of situations in which

they needed to intervene did not impose high enough workload to reach the limits of

controllers’ workload capacity.

However, participants reported a different story for the remaining two issues. Participants reported:

4. Given the assumption that TSAFE resolves the conflict but the ASA Manager can provide

redundant safety and assist putting aircraft back on-track once s/he is alerted of the situations

at 3 minutes prior to the LoS, the decision-making timeframe to assist short-term conflict

resolution and recovery was potentially too short.

5. Given the ASA Manager’s role to provide assistance to the automation during off-nominal

situations, the roles and responsibilities between human and automation could be better

delineated.

Further insights were gained from the walkthrough on the topics of time pressure, shift of control

authority, responsibility, trust/transparency/level of automation, controllers’ expertise, mental

models, rule-based decision-making, task prioritization under time pressure, conscious and

involuntary attention allocation, etc. These results are reported in detail in Appendix E.

Overall, the walkthrough demonstrated that once the key issues are identified, methods such as

walkthrough or human-in-the-loop (HITL) simulations can answer and bring insight into the validity

of the identified issues. It is noted that in issue 2, the controller participants reported that they had no

problem forming traffic situation awareness in short-term conflict situations by using their existing

knowledge of conflict monitoring in the current operations. However under the future ASA concept,

automation will perform conflict monitoring, and the ASA Manager may not accumulate, and at

least not keep up-to-date, the knowledge of conflict monitoring that today’s controllers have. This is

an example of an issue that warrants further study.

5. Airspace Super Density Operations

Airspace Super Density Operations (ASDO) enables high-efficiency trajectory-based operations to

meet NextGen demands in super-dense and regional/metroplex airspace while minimizing

environmental impacts. ASDO includes developing: 1) technologies to simultaneously sequence

and de-conflict aircraft leading to trajectory management for aircraft in terminal and extended

terminal (as necessary) airspace; 2) precision spacing and merging capabilities to reduce workload

and spacing variance between aircraft in terminal and extended terminal airspace; 3) methods for

optimizing terminal airspace resource utilization among interconnected airportals; and 4) ASDO

technologies and operations that are robust to weather and other disturbances.

In ASDO, many operational concepts—such as Flight-Deck Merging & Spacing (FDMS),

Closely-Spaced Parallel Spacing (CSPA), Optimized Profile Descent (OPD; formerly called

Continuous Descent Approaches [CDA]), precise arrival/departure routing that meets Required

Navigation Performance (RNP), etc.—can co-exist in the same airspace and are envisioned to

be turned on or off depending on the traffic situation. The list of operations under ASDO is

described in Appendix F.1.
24

5.1 Human Roles and Technology Assumptions for ASDO

Compared to other RFAs, ASDO assumes the existence of relatively low technology for nearer-term

implementations. For example, CDA, and RNP routes are possible with advanced Flight

Management System (FMS), RNAV, and Automatic Dependent Surveillance-Broadcast (ADS-B)

for precision navigation and surveillance. FDMS can be achieved with ADS-B and advanced cockpit

displays and automation. CSPR operations with better wake vortex handling require RNAV/RNP

capability and wake sensing and prediction technology. Far-term applications of ASDO will likely

require advanced decision support tools (e.g., further applications of CDA would be possible with

DSTs to enable managing denser traffic levels over the larger descent profiles required for CDA),

net-centric flight information exchange, and probabilistic weather forecast products. Data link is

assumed for high altitude airspace where the initial descent occurs for arrivals. Data link allows for

flexible routes that can be constructed and uplinked to the flight crew without prior published routes.

Data link should also be useful in the terminal airspace to dynamically construct 4D trajectories, but

it might not be available in the mid-term timeframe.

An overview of ASDO might begin with aircraft are assigned a 4D arrival profile at the top of

descent. A local scheduler with a pre-defined time window determines when an aircraft can enter

super density airspace. The 4D arrival profile follows a CDA or other optimized profile descent to

maximize fuel efficiency and minimize noise. During the descent, the aircraft equipped with ADS-B

and flight deck automation/displays can merge and space behind another equipped aircraft and

reduce the inter-arrival spacing buffer by actively managing the spacing interval at a tight tolerance.

Three human roles were analyzed for ASDO:

1. Traffic Manager: manages and works with DSTs to plan the arrival and departure routes.

Currently these routes extend beyond terminal airspace but may fall exclusively in the

terminal area if its boundaries are extended. The “Traffic Manager” described here may have

different names and may span multiple positions depending on the situational context and the

facility. For example, there may be separate Arrival Flow Manager and a Departure Flow

Manager who focus solely on the arrival or the departure stream. For the purpose of this

section, we will use a generic Traffic Manager term for this function.

2. Controller: manages arrival and departure traffic in and near the terminal airspace. Their

main tasks include separation management, maintaining throughput, and monitoring

conformance to 4D trajectories along RNP requirements and CDAs. They may also support

ground-based separation assurance and FDMS, as well as dealing with aborted approaches,

dealing with anomalies, assisting aircraft in changing trajectories in response to weather and

other factors, working with pilots and Traffic Managers to return an aircraft that deviated for

some reason to the arrival stream, etc.

3. Pilot: reviews and executes 4D trajectories communicated via data link. S/he manages spacing

and/or separations during FDMS and performs CSPR operations when appropriate.

Roles and responsibilities for these three operators were developed to form the basis for the analysis

of HF issues. Further details of the human operators’ roles and responsibilities are outlined in

Appendix F.3.1. The human roles for ASDO may change if there are changes to the capabilities that

could be implemented independently.
25

In this section, key human performance issues and potential countermeasures are summarized for

ASDO. The details of the identification and prioritization of the key human performance issues and

recommendations for potential countermeasures, as well as the descriptions of the nine human

factors in the context of ASDO, are reported in Appendix F.

The main drivers from ASDO for HF issues are the increased use of automation and Decision

Support Tools, the high number of operational modes within ASDO (e.g., CDAs, FDMS, dynamic

and multiple metering fixes, etc.) that will operate in different combinations as they are individually

turned on and off, high traffic levels, and the needed precision in operations.

The human performance issues rated high in our analysis for ASDO (see Appendix F) are listed

below. Other issues rated medium and low can be found in Appendix F.

5.1.1 Distribution of Roles and Responsibilities between Humans and Automation

Issue: With the many operational concepts that can be switched on and off, and hence operation in

different combinations, as part as ASDO (e.g., CDA, FDMS, etc.), the distribution of roles and

responsibilities across the Traffic Manager, Supervisor, Controller, pilot, and ground/flight-deck

automation needs to be delineated. Decision authority needs to be defined along with roles and

responsibilities. Examples of roles and responsibilities needing attention are:

• An aircraft in ground-based separation that transitions to self-spacing or self-separation may

shift responsibilities between Controller, pilot, and their respective DSTs multiple times within

the same flight.

• The decision authority may become problematic during self-spacing operations - a problem if

the pilots create separation errors during self-spacing task and the separation responsibility

ultimately falls on the Controller.

• If either the pilots or the Controllers may be relying heavily on automation for performing the

separation tasks and cannot perform the tasks without it, the actual decision authority between

humans and automation may be blurred.

Potential Countermeasure/Next Step: Functional analysis conducted to outline the roles

and responsibilities of humans and automation in the system. Human factors research

should be conducted to test potential problem areas. Some basic human factors research

related to human-automation interactions and collaborative decision-making can examine

how the tasks and responsibilities might be best distributed in a complex human-machine

system. Applied research that tests specific concept instantiations via HITL simulation

would also be useful.

5.1.2 Demand for Workload, Attention, Situation Awareness, and Coordination

Issue: The Traffic Manager and Controller will deal with many operational concepts that can be

switched on and off as part of ASDO (e.g., CDA, FDMS, etc.), and this will lead to demands on

workload, attention, situation awareness, and coordination.

• Some of the key issues involve how a single human operator handles multiple operational

modes required for the traffic density and many the concepts of ASDO.

• DSTs need to be designed to support the human operators performing their roles under the

numerous concept elements that will be part of ASDO.
26

- Displays may need to present more information than can reasonably be displayed in one

place and result in screen clutter.

- Human interactions with automation and DSTs need to be given attention in this complex

work environment.

• Monitoring peak workload is a key issue since current use of aircraft count to measure potential

overload may no longer apply in complex mixed airspace that may also include transitioning

from one operational mode to another (e.g., switching between CDA and non-CDA and

between FDMS and non-FDMS operations).

• In mixed operations, a key question is whether one or more human operators (e.g., pilots,

Controllers) can perform different functions for the different modes of operations (e.g., FDMS,

CDA, etc.).

• To make timely decisions, operators will need to have an awareness of the situation and enough

resources to attend to critical information. Examples of awareness issues are:

- knowing the mode in which they are operating under normal operations with the many

different possible operational mode combinations (e.g., what ASDO modes are on and

what is off, and what is the runway configuration since the runway configuration may

change more frequently than today)

- memory of actions that apply in each particular operating mode

- memory of the situation in a previous mode where the mode has just changed

- need for emergency trigger events in all the operational modes

- aids to help the human operators respond to emergencies given the different and changing

operating modes in which the human operators may be working

• If data link is to be used for safety-critical messages such as clearances to resolve conflicts, the

visual attention demands of data link in an environment that potentially has high visual clutter

should be carefully examined.

Potential Countermeasure/Next Step: Each of the issues mentioned above should be

examined, and initial designs of the concept instantiations, procedures, and tool prototypes

developed. This includes examining peak workload situations, workload impact of

transitioning between operational modes, workload under off-nominal situations, traffic

complexity, workload impacts in a mixed equipage environment, and means of

communication for sharing information. Then concept-focused walkthroughs or HITL

simulations should be conducted to evaluate the feasibility of the designs, procedures, and

roles.

5.1.3 Dealing with Emergency, Safety Critical, and Off-Nominal Events

Issue: A recurring theme in ASDO is whether operators will have enough time to make decisions in

critical situations, both normal and off-nominal.

• Because of the complexity of ASDO operations, many types of human error and system failures

must be anticipated and recovery strategies considered.

• While standard responses would be preferable, it is likely that each ASDO concept will give

rise to different types of problems and therefore detection and responses to off-nominal events

will have to be handled on a case-by-case basis.
27

• Operators need to know the mode in which they are operating under normal operations with the

many different possible operational mode combinations.

• Need for emergency trigger events and aids to help the human operators respond to

emergencies, given the different operating modes in which the human operators may be

working.

• Possible mode confusion in which an aircraft is assumed to be operating in one separation

mode (e.g., tight spacing with FDMS) when it is in fact in another (e.g. low equipage aircraft

that requires larger spacing buffer).

• Negotiation needed between Controller and pilot to react to separation violation or significant

deviation with the different modes that may be operational under ASDO.

• High load on operators’ working memory while monitoring multiple aircraft under different

conditions.

• With so many simultaneous activities, operators will have to be prepared to be interrupted

often while they are completing tasks; interruptions can place a high load on prospective

memory (you have to remember to go back to the activity you were working on).

Potential Countermeasure/Next Step: Emergency or off-nominal events under ASDO

should be identified within all the operation modes that will be part of ASDO (e.g., FDMS,

CDA, etc.). Responsibilities for automation and for different human operators need to be

allocated for monitoring and resolving. Then means to alert the human operators and for

humans to respond can be designed. The potential designs for humans to be alerted about

and to deal with emergences can be validated and refined using HITL simulations.

Coordination roles in emergences can also be studied in HITL simulations. Those

emergencies allocated to automation should also be studied to see if they can be identified

and solved by automation.

5.1.4 Human Operator Characteristics and Training

Issue: In ASDO, there are numerous operational modes and a need for precision in timing for

actions. Thus:

• Human operators need the ability to handle a variety of operational modes

• Training is needed for performance under the different operational modes

• Organizational structure and distribution of roles under ASDO needs to be clarified to cover

operators who are responsible for determining the route structure and operational modes (we

call that person “Traffic Manager”) and operators who monitor the conformance of the traffic

and manage normal control functions

Potential Countermeasure/Next Step: Given that human operators in ASDO might often

work in mixed operations, how operators can be trained in such operations needs to be

studied. If issues arise that indicate that the number of operations in ASDO is beyond

human abilities even after proper training, redesign of operational procedures and the

airspace should occur early in the concept development. Current methods of training

should be cataloged and steps should be taken to see how they can be modified to handle

ASDO operations. Feedback from Subject Matter Experts should be solicited to assess if it

would be possible for Traffic Managers, Controllers, and pilots to learn and operate in this

environment.
28

5.2 Main Drivers of the Human Performance Issues for ASDO

Examining the issues identified above, the key human performance issues revolve around the

following three areas:

1. Shifting roles among multiple operational modes that turn on and off. An environment with

different operational modes and distributed responsibilities across the Traffic Manager

(assumed human operator to manage arrival and departure routes), supervisor, controller,

pilot, and ground/flight-deck automation needs clear delineation of roles and a clear decision

authority in case of disagreement. The roles can shift over time as the operational modes turn

on or off. These roles and responsibilities should be a part of the concept design and/or

refinements. Methods need to be developed to train future controllers on the multiple

operational modes with many possible error and failure states.

2. Workload, situation awareness, and coordination under multiple modes of operation.

Possibility of multiple modes of operations requires attention to workload, situation

awareness, and coordination. The issues can be resolved by (1) good DST and display design

to smartly identify and highlight different operational modes and off-nominal/emergency

situations without adding clutter; (2) human factors research to determine workload impact

and possible modal confusion with multiple operational modes; and (3) human factors

research to measure peak workload in relation to traffic complexity in mixed equipage and

operational modes.

3. Emergency and safety-critical situations multiple operational modes. A systematic look at

emergency and safety-critical situations is needed in a diverse set of possible operational

modes and combinations of these modes. In addition to the training for errors and failures and

DST design for off-nominal/emergency situations, the following issues should be resolved:

(1) recovery strategies should be developed for known possible errors and failures; (2)

research on modal confusion should be applied to prevent possible human errors; (3) potential

high workload resulting from multiple operational modes and emergency/safety-critical

situations should be mitigated with better procedures and tools design, as well as more refined

human factors research.

6. Traffic Flow Management

Traffic Flow Management (TFM) identifies and resolves imbalances in the demand and supply of

NAS resources, such as airspace and runways, by scheduling and routing aircraft. This includes: 1)

assimilating weather into TFM by exploring the impact of weather (convective and non-convective)

on the capacity of NAS resources; 2) exploring the roles and responsibilities of airspace users on

TFM decision making; 3) developing and exploring the utility of both aggregate (i.e., flow-based)

and aircraft-level flow control models; and 4) developing heuristic and optimization approaches for

developing optimal flow control strategies (e.g., rerouting, departure control, airborne flight delays).

Under NextGen TFM, more precision is expected in the planned and actually flown trajectories.

Higher fidelity in the trajectory prediction, along with the increased precision in the weather

prediction will allow a more optimal schedule, which in turn will increase the airspace capacity.

Automation for TFM will be primarily in the form of decision support tools (DST) to support the

Traffic Flow Managers in planning aircraft trajectories.
29

6.1 Human Roles and Technology Assumptions for TFM

The general types of technology assumed for TFM are 4D trajectory-based operations, Performance

Based Services, and ground-based automated separation assurance. It is further assumed that TFM

will operate along with DAC and ASDO, as planning trajectories is part of these concepts as well.

Specific technologies assumed include data link, ADS-B to provide accurate position data, and

System-Wide Information Management (SWIM). En Route Automation Modernization (ERAM)

and Traffic Management Advisor (TMA) are assumed for use by Air Traffic Control. Aircraft

equipage is assumed for Cockpit Display of Traffic Information (CDTI), Area Navigation (RNAV),

and RNP.

The human roles analyzed for TFM are:

• TFM functions at two levels:

1. National Traffic Flow Managers: at one national TFM facility with that will perform

strategic flight trajectory management by setting a NAS-wide schedule based on:

- advanced traffic and weather prediction tools

- user preferred routes from Airline Operations Centers (AOCs)

2. Regional Traffic Flow Managers: at regional TFM facilities that provide tactical TFM to

adjust the strategic solutions at a local level to account for more local or late occurring

weather and traffic demand changes. They may also negotiate with AOCs for updated user

preferred routes.

• AOC: coordinates with the National and/or Regional Traffic Flow Managers to negotiate their

preferences on the routes and the schedule. Assuming that smart DSTs will be available to the

schedulers, coordination may be done through the scheduling DST by having the AOCs input

the preferred scheduled slots and the routes directly into the DST.

• Controller: concerned with monitoring and managing an assigned airspace. For TFM, s/he

executes any Traffic Management Initiatives (TMIs) and monitors aircraft for its conformance

to the schedule. The Controller will have different duties depending on the Separation

Assurance (SA) mode in the airspace, for example in classic airspace the term “Controller”

may be used and in automated ground-based SA, the term “ASA Manager.” We use only the

term “Controller” for this basic function throughout the TFM discussion.

• Pilot: coordinates with AOC to request and fly user-preferred routes that are optimal to the

airline. S/he conforms to any TMIs that are being executed by the Controller.

Human operators’ roles and responsibilities for TFM were developed to provide the basis for

analyzing the HF issues. Further details of the human operators’ roles and responsibilities are

outlined in Appendix G.3.1.

6.2 Key Human Performance Issues for TFM

The main drivers from TFM for HF issues are the strong reliance on the automation and DSTs,

coordination complexity, and potential issues with situation awareness and workload due to possible

short, intensive time periods for planning reroutes (e.g., due to sudden weather). The details of the

identification and prioritization of the key human performance issues and recommendations for
30

potential countermeasures, as well as the descriptions of the nine human factors in the context of

TFM, are reported in Appendix G.

The high priority human performance questions for TFM are described below. Other issues rated

medium and low can be found in Appendix G.

6.2.1 Reliance on the Automation and Decision Support Tools

Issue: The Traffic Flow Managers will need to trust and rely on the automation’s (i.e., DST’s)

recommendations and parameter information since there will be:

• an assumption that the automation can provide an optimized schedule that takes trajectory and

weather predictions fully into account in ways that humans cannot

• more trajectories with increased planning complexity.

• large number of reroutes to plan in a short time, and perhaps accompanied by changes in

weather and airspace configurations

• TFM function could breakdown without the ability to process information and make decisions

quickly

Potential Countermeasure/Next Step: Situations for large numbers of complex reroutes in a

short time, and with rapidly changing weather and changing airspace configurations, need

to be defined. Human Factors research is needed to understand how humans process

complex information such as the TFM functions in contrast to the automation solution. An

analysis is needed on how the Traffic Flow Manger will judge and interpret aircraft

trajectories and conditions to plan, make decisions on, and implement the trajectories as

recommended by the DST. Cognitive walkthroughs and HITL simulations should be

conducted to test DST designs and to understand how the Traffic Flow Managers will

interpret information and make decisions.

Issue: Information display under TFM will likely be complex since:

• DSTs will propose scheduling and traffic flow options based on complex parameters such as

probabilistic information (e.g., weather, and even future traffic levels) and uncertain

information (e.g., flight trajectories in a six hour timeframe that can change due to human

interventions)

• flight environments will likely be more dynamic than today (e.g., Dynamic Airspace

Configuration, different modes of separation assurance, and different modes of operation under

Airspace Super Density Operations)

Potential Countermeasure/Next Step: Situations for which Traffic Flow Managers will

need to make decisions, types of decisions they will need to make, and types of information

they will need for the decisions should be identified. Potential information displays can

then be prototyped. The displays can be refined via walkthrough or HITL simulation.

How to present probabilistic information (e.g., weather) and consequent recommend

trajectories needs research.
31

6.2.2 Demand for Workload, Attention, and Situation Awareness

Issue: Traffic Flow Mangers and Controllers could have problems with demand for workload,

attention, and situation awareness. Traffic Flow Managers could have problems if there are:

• need to handle an excessive number of reroutes in a short time period

• related to this, the need for strategies to allocate attention during task execution and to

maintain situation awareness when airspace and routes are less structured under NextGen

operations

• mixed equipage aircraft with different equipage and different performance characteristics that

affect trajectory planning and make it more complex for the Traffic Flow Manager to plan

routes

• potential for mental fixation or cognitive “tunneling” that occurs due to a highly complex

traffic problem in one area consuming all of the attentional resources while other areas are

being ignore

There are issues for Controllers if they reroute numerous aircraft in a short time period while

monitoring for and resolving conflicts and conducting handoffs.

Potential Countermeasure/Next Step: Situations that will drive the information display

design need to be defined. Smart DST design is needed to filter the traffic information to

highlight the relevant information while reducing the overall workload related to traffic

assessment to avoid cognitive tunneling. The ability of Traffic Flow Managers, AOCs,

Controllers, and pilots to handle multiple complex traffic situations and possible strategies

for these operators to handle the situations should be researched from both basic research

on attention/decision-making and by evaluating prototype tools with Traffic Flow

Managers in HITL evaluation.

6.2.3 Communication and Coordination

Issue: Coordination under TFM could be highly complex and potentially excessive.

• Coordination linkages include between AOC and National and Regional TFM, National TFM

and regional TFM, pilots and AOC, pilots and Controllers, Controllers and TFM, and, if DAC

exists, TFM and DAC.

• How these various parties will exchange information on possible routes needs study, such as

the means (e.g., voice, data communications, or both) and content, detail, and type of format

for each particular operator.

Who has final authority after all this coordination is an issue.

• Authority changes among National TFM, Regional TFM, controllers, and pilots depending on

the scale of the traffic problem and the safety criticality of the traffic situation.

Potential Countermeasure/Next Step: Situations that will drive complex needs for

coordination and negotiation among the different operators should be defined.

Research in collaborative decision-making should be leveraged to design effective

coordination. Operational procedures and the corresponding roles (including

who has authority) need to be designed for the coordination. DSTs that can

facilitate the coordination via data communication need to be prototyped. The

ability for Traffic Flow Managers, AOCs, Controllers, and pilots to handle these

situations, and possible strategies for these operators to handle the situations can
32

be investigated with HITL simulations. Displays of information and appropriate

means of communication can also be studied with HITL simulations.

6.2.4 Dealing with Emergency, Safety Critical, and Off-Nominal Events

Issue: The need for any indictors of emergency or off-nominal events during Traffic Flow

Management activities needs to be studied for Traffic Flow Management and Controllers since this

is a safety issue:

• A high number of aircraft to reroute, complex trajectories, and changing airspace

configurations could lead to the Traffic Flow Manager only noticing emergency situations with

the assistance of automated alerts

• It is recommended that having the automation also make recommendations for resolving the

emergency be explored

• Coordination needs among Regional and National Traffic Flow Managers, Controllers, AOCs,

and pilots during emergencies also needs study

Potential Countermeasure/Next Step: Emergency or off-nominal events under TFM should

be identified. Responsibilities for automation and for different human operators need to be

allocated for monitoring and resolving. Then, means to alert the human operators and for

humans to respond can be designed. The potential designs for humans to be alerted about

and to deal with emergences can be validated and refined using HITL simulations.

Coordination roles in emergences can also be studied in HITL simulations. Those

emergencies allocated to automation should also be studied to see if they can be identified

and solved by automation.

6.2.5 Human Operator Characteristics and Training

Issue: Traffic Flow Managers and Controllers will need to be comfortable using displays and

working with DSTs and working under different airspace configurations during a shift as well as

from one day to the next. Thus, the skills, abilities, and traits needed by the human operators are

questions:

• Traffic Flow Mangers will need to be able to assess traffic situations and deal with trajectories

that are more complicated than today.

• An issue is whether Controllers will have the needed skills to move to a Traffic Flow Manager

position in a TMU as they move to TMU positions today.

• Other issues are how to train for work in changing conditions and how to conduct on-the-job

training when conditions change.

Potential Countermeasure/Next Step: The types of tasks human operators will conduct

under TFM and the skills, abilities, and traits needed for these tasks should be identified.

Then, types of personnel needed should be defined. HITL simulations may help with these

definitions and how to identify these needs in humans. Career paths should be examined

as to possible career progressions for people with particular characteristics. How to train

should be studied. Similar work situations in other industries should be examined to learn

how these organizations conduct training. Training experts will need to be involved. It

may help to conduct prototype training sessions to learn about and refine training

methods.
33

6.3 Main Drivers of the Human Performance Issues for TFM

Examining the issues identified above, the key human performance issues revolve around the

following three areas:

1. Use of sophisticated DSTs based on factors that might not be fully understood. The Traffic

Flow Manager in NextGen needs to be able to utilize potentially sophisticated algorithms and

DSTs that provide schedule solutions based on factors that the person might not fully

understand (e.g., probabilistic weather prediction, a complex set of airport throughput

constraints that affect the schedule, algorithms that optimize over metrics that are not easily

translatable in non-mathematical terms, etc.). His/her ability to do that hinges on many factors

such as (1) designing DSTs and algorithms that can provide complex information in easily

understandable format to the human operators; and (2) selection and training processes that

identify a person that can understand the complex set of constraints in air traffic system and

advanced machine systems and that can train them to fully utilize the DSTs and understand the

intentions of the algorithms/automation.

2. Complex coordination. The process of selecting the appropriate schedules; coordinating and

negotiating the solutions with other ATM personnel and with the airlines; and replanning/coordinating whenever schedule changes are needed based on changes in weather or

traffic information involves significant and complex coordination as a central part of the

activity. Therefore considerable research and development is needed to: 1) leverage past

research in collaborative decision-making and conduct focused studies on specific aspects of

coordination in TFM; 2) develop a set of coordination procedures for both normal and offnominal situations; and 3) develop DSTs that intelligently support non-verbal coordination via

a ground-to-ground data communication system that is integrated into their tools.

3. Workload and potential cognitive tunneling under significant route planning and offnominal/emergency situations. A significant re-planning due to severe changes in the traffic

situation is difficult in terms of workload, situation awareness, and coordination in current

operations and is expected to be even more complex in NextGen. Given the complex traffic

problem and potential overload of information, the Traffic Flow Manager may fixate on one

traffic problem while neglecting others. Efforts are needed to: 1) design the automation to

recognize and trigger the changing traffic situations, especially discrete offnominal/emergency situations (e.g., decision by a Tower facility to shut down an airport) and

synchronize its information with Traffic Flow Manager; 2) design DSTs that can remove

clutter and provide only relevant information to human operators; 3) conduct research on

attention, and decision-making strategies that can help with the design; 4) develop and evaluate

coordination mechanisms and procedures in case of off-nominal/emergency situations; and 5)

assess the workload impact of significant schedule changes.

7. Dynamic Airspace Configuration

Dynamic Airspace Configuration (DA increases capacity through strategic airspace organization and

dynamic allocation of airspace structures and controller resources. This includes: 1) develop overall

classifications of airspace; 2) identify new classes of airspace for NextGen operations; 3) develop

adaptable airspace with algorithms and technologies for dynamically changing airspace to
34

accommodate demand; and 4) develop generic airspace characteristics to increase interchangeability

of facilities and controllers.

7.1 Human Roles and Technology Assumptions for DAC

The DAC concept addresses changing sector boundaries in real-time to adjust new air traffic demand

to the workload constraints of the Controllers. DAC algorithms formulate optimal sector boundaries

for a particular traffic demand and the results are presented to a human operator (we will call this

operator “Airspace Manager”). Automation for DAC will be primarily in the form of decision

support tools (DST) to support the airspace reconfiguration. The Airspace Manager uses the

algorithms and DSTs to plan the new airspace configuration and decides when to change the

configuration.

Three human roles were analyzed for DAC:

1. Airspace Manager: makes decisions in real-time on when and how to change the airspace

configuration. The Airspace Manager is assumed to be part of the Traffic Management Unit

(TMU); it is expected that duties of the Airspace Manager will include both TFM and airspace

management unless the DAC duties prove to be large enough to preclude this.

2. Controller: monitors and manages an assigned airspace. During sector reconfiguration, s/he

needs to transfer ownership of aircraft to their proper sectors after the boundary change by

coordinating with the neighboring sector Controllers.

3. Pilot: executes clearances issued by automation or a human operator and participates in

handoffs. When necessary, pilot coordinates the maneuvers with a Controller. The pilot can

also request reroutes.

Further details of the human operators’ roles and responsibilities are outlined in Appendix H.3.1.

The ground technology assumed for DAC includes automated Decision Support Tools to aid the

Airspace Manager in selecting airspace configurations and to support implementing a change in

airspace configuration. Controllers will use DSTs in managing the airspace for which they are

responsible. These DSTs will have capabilities to preview the upcoming airspace boundary changes

on their displays. It is assumed that the boundary changes can be propagated digitally to all human

operators who need the information, including Airspace Manager, Controllers, Front Line Managers,

Traffic Flow Managers, etc.

Aircraft equipage depends on the equipage requirements for a particular airspace. However, data

link is considered to be a prerequisite for DAC both in the air and on the ground. Aircraft need to

have data link to receive 4D trajectories, FMS to process 4D trajectories, and also RNP in certain

airspace types. ADS-B would provide more location precision of aircraft and hence add benefits to

DAC, but it is not assumed as necessary. Aircraft flying in self-separation airspace will be equipped

with automation for use when flying under self-separation.

7.2 Key Human Performance Issues for DAC

The main drivers from DAC for HF issues are reliance on the automation and DSTs, complexity of

situations in which DAC may operate, and working in changing airspace configurations. The details

of the identification and prioritization of the key human performance issues and recommendations
35

for potential countermeasures, as well as the descriptions of the nine human factors in the context of

DAC, are reported in Appendix H.

The high priority human performance questions for DAC are described below. Other issues rated

medium and low can be found in Appendix H.

7.2.1 Reliance on the Automation and Decision Support Tools

Issue: The Airspace Manager will need to trust and rely on the automation’s (i.e., DST’s)

recommendations and parameter information for DAC to provide benefits since there is extensive

information to process to make airspace configuration decisions. Such extensive information to

process can occur if there are:

• several sectors or airspace units involved

• large number of aircraft and complex flight patterns

• complex configuration changes, such as splitting and combining sectors

Potential Countermeasure/Next Step: The type of airspace reconfigurations to be made

under DAC need to be defined, and then the capabilities of the automation designed. An

analysis is needed on how the Airspace Manager will judge and interpret boundary change

impacts of airspace reconfigurations recommended by the DST. Cognitive walkthroughs

and HITL simulations can examine the robustness of the decision support tools for

supporting Airspace Mangers in a timely manner in all the situations they are expected to

encounter.

7.2.2 Demand for Workload, Attention, and Situational Awareness

Issue: Controllers could have problems with demand for workload, attention, and situation

awareness if they need to handle:

• an excessive number of conflicts while conducting handoffs during an airspace configuration

transition

• mixed equipage flying in airspace units under going reconfiguration resulting in a more

complex reconfiguration execution situation

Potential Countermeasure/Next Step: Situations for Controllers where a large number of

conflicts may occur while conducting handoffs for airspace reconfigurations should be

identified. The ability for a Controller to handle these situations and possible strategies

for the Controller to manage the demand can be investigated with HITL simulations. More

knowledge is needed about the DAC concept regarding mixed equipage to obtain an

understanding of Airspace Manger HF issues for this situation. Then, walkthroughs and

HITL simulations should be conducted to understand the limits for human operators and

DSTs to handle mixed equipage and to refine procedures.

7.2.3 Dealing with Emergency, Safety Critical, and Off-Nominal Events

Issue: The need for any alerts for Airspace Mangers and Controllers to indicate emergency or offnominal events during airspace configuration change needs to be studied:

• for aircraft near the boundary of the new or old airspace configuration

• if not all aircraft trajectories have been checked for compatibility with the configuration change
36

• if an aircraft is deviating from its planned path during an airspace configuration change

• more complex situations with changing airspace configurations than today

For this issue, it is recommended that:

• whether automation can make recommendations for resolving the emergency be explored

• coordination needs between the Airspace Manager, Controller, and pilot during emergencies be

studied

Potential Countermeasure/Next Step: Emergency or off-nominal events under DAC should

be identified. Responsibilities for automation and for human operators need to be allocated

for both monitoring and resolving. Then, means to alert the human operators and aids for

humans to respond can be designed. The potential designs for humans to be alerted about

and to deal with emergencies can be validated and refined using HITL simulations.

Coordination roles in emergences can also be studied in HITL simulations. Those

emergencies allocated to automation should also be studied to see that they can be

identified and solved by automation.

7.2.4 Human Operator Characteristics and Training

Issue: Airspace Managers and Controllers will need to be comfortable using displays and working

with DSTs and working under different airspace configurations during a shift as well as from one

day to the next. Thus the skills, abilities, and traits needed by the human operators are questions

needing attention.

• Airspace Mangers in a TMU will need to be able to assess traffic situations that are more

complicated than today.

• An issue is whether Controllers will have the needed skills to move to an Airspace Manager

position in a TMU as they move to TMU positions today.

• How to train for work in changing conditions and how to conduct on-the-job training when

conditions change are also issues.

Potential Countermeasure/Next Step: The types of tasks human operators will conduct and

the skills, abilities, and traits needed for these tasks should be identified. Then, types of

personnel needed should be defined. HITL simulations may help define what skills and

abilities are needed and how to identify these needs in humans. Career paths should be

examined as to possible career progressions for people with particular skills, abilities, and

traits. How to train needs to be studied. Similar work situations in other industries should

be examined to learn how these organizations conduct training. Training experts will need

to be involved. It may help to conduct prototype training sessions to learn about and refine

training methods.

7.3 Main Drivers of the Human Performance Issues for DAC

Examining the issues identified above, the key human performance issues revolve around the

following two areas:

1. Use of sophisticated DSTs based on factors that might not be fully understood. Analogous to

TFM, the Airspace Manager in NextGen needs to be able to utilize potentially sophisticated

algorithms and DSTs that provide airspace reconfigurations based on factors that the person

might not fully understand. His/her ability to do that hinges on many factors such as: 1) DSTs

and algorithm design that can provide complex information in an easily understandable
37

format to the human operators; and 2) processes to identify and train a person with

appropriate aptitude for understanding the complex set of ATM constraints and advanced

algorithms and DSTs,.

2. Workload under intense route planning and off-nominal/emergency situations. The impact of

airspace design and reconfiguration on the controller’s workload, situation awareness, and

coordination needs to be examined. The issues can be resolved by: 1) good DST and display

design to smartly display, coordinate, and propagate new airspace configurations; 2)

development of operational procedures for coordination during a boundary change; 3) human

factors research to assess workload impact in relation to airspace design, traffic complexity,

and other factors during a boundary change; and 4) potential safety impact of conflict-related

or other safety-critical events that occur during a boundary change.

8. Integrated DAC-SA

This report focuses on identifying key human performance issues for four individual Research Focus

Areas (RFAs). Concepts from more than one RFA are very likely to operate simultaneously in an

integrated manner. There will then be additional human performance issues for such integrated

operation of RFAs. This chapter presents a representative assessment of human performance issues

for one integrated pair of RFAs, namely Automation Separated Airspace (ASA) and Dynamic

Airspace Configuration (DAC).

For this examination of integrated ASA and DAC concepts, we assume two airspaces: 1) one is an

airspace where controllers provide flight management and where DAC operates; and 2) the other

with Automated Separation Assurance (ASA) and immediately above the former airspace. When

DAC operates in the first airspace, sectors in the first airspace may expand into the second airspace.

The first airspace is assumed to be a High Altitude Airspace (HAA) that is envisioned to have data

link equipped aircraft performing Trajectory-Based Operations (TBO) in a relatively simple traffic

environment (e.g., mostly level flight with little local knowledge needed). Controllers are assumed

to provide flight management in the High Altitude Airspace. This High Altitude Airspace is felt to

be a likely airspace in which DAC will operate, and thus is taken as part of the integrated analysis

presented in this chapter.

Above High Altitude Airspace, we assume a ground-based automated-separation airspace that

handles a high volume of aircraft without controllers providing active separation management. We

assume that the Automated Separation Assurance (ASA) concept covered in Chapter 4 operates in

this airspace. We further assume that this ASA airspace is above the High Altitude Airspace.

The simplest form of DAC is probably to change the lateral airspace boundaries in response to

changing traffic demand or weather. In this case, the airspace units being changed are of the same

type, (i.e., sectors reconfigured in the High Altitude Airspace do not intrude into the ASA airspace

above High Altitude Airspace). A more complex form of DAC, but likely needed to meet a range of

traffic and weather situations, is to change the boundaries between the different classes of airspace

above and below each other. In this case, High Altitude Airspace sectors would be allowed to

expand vertically into ASA airspace. Examples of conditions that might cause such changes in

vertical airspace boundaries include traffic demand and mixes, weather tops, and winds/ride

conditions. It is this more complex DAC application where High Altitude Airspace sectors expand
38

vertically into the ASA airspace above it that is assumed for this analysis of integrated ASA and

DAC concepts.

8.1 Human Roles and Technology Assumptions for Integrated ASA and DAC

As stated above, to illustrate analyzing human performance issues under integrated concepts, we will

examine sectors in High Altitude Airspace being dynamically expanded into and out of Automation

Separated Airspace above the High Altitude Airspace. For this analysis, we designed the scenario

such that there are no mixed equipage in either HAA and ASA airspaces in steady state but when the

vertical boundary between the two airspaces needs to move up or down due to weather (or other

traffic situations), mixed equipage conditions occur during the transition period.

Four human roles were analyzed for integrated ASA and DAC: the Airspace Manager (for DAC),

ASA Manager (i.e., “controllers” for ASA airspace), HAA Controller (we will use the term HAA

Controller to indicate that the Controller is managing aircraft in the High Altitude Airspace), and

pilot:

1. Airspace Manager: makes decisions in real-time on when and how to change the airspace

configuration, both laterally and vertically. For vertical reconfiguration, s/he assesses

whether there are capacity constraints in HAA due to weather and excess capacity available in

ASA to decide on a vertical configuration change. S/he coordinates/negotiates the change

with other Traffic Flow Managers, Front Line Managers, ASA Managers, and AOCs.

2. ASA Manager: Once the Airspace Manager coordinates with the ASA Manager about the

vertical reconfiguration, the ASA Manager sends route clearances to the aircraft to move them

out of the what will become the new HAA airspace. S/he also monitors the transition airspace

if unequipped aircraft have already entered the airspace in the midst of the ASA-equipped

aircraft.

3. HAA Controller: monitors and manages an assigned HAA airspace. During sector

reconfiguration, s/he needs to transfer ownership of aircraft to their new proper HAA sectors

after the boundary change by coordinating with the neighboring sector Controllers. If the

aircraft enters the transition airspace such that it is mixed equipage with ASA-equipped

aircraft during the transition, the HAA Controller manages the unequipped aircraft on their

4D trajectories and keeps them safely away from the ASA-equipped aircraft.

4. Pilot: executes clearances issued by automation or a human operator and participates in

handoffs. For the pilots of the ASA-equipped aircraft, they can coordinate with the AOC to

request user-preferred reroutes away from the transition airspace. The reroutes would be

reviewed and approved by the ASA ground automation or the ASA Manager.

Further details of the human operators’ roles and responsibilities under integrated ASA and DAC are

outlined in Appendix I.2.

The descriptions of and assumed technology for ASA and DAC were described in Chapters 4 and 7,

respectively, and will not be repeated here.

8.2 Key Human Performance Issues for Integrated ASA and DAC

The main drivers from integrated ASA and DAC for HF are: 1) the large number of different

automation and human operators involved; 2) increased complexity of operations during transition,
39

including mixed equipage; and 3) workload, attention, and time pressure during the airspace

transition.

The details of the identification and prioritization of the key human performance issues and

recommendations for potential countermeasures, as well as the descriptions of the nine human

factors in the context of Integrated ASA/DAC, are reported in Appendix I.

8.2.1 Distribution of Roles and Responsibilities between Humans and Automation

Issue: Roles and responsibilities need to be clearly delineated between human operators and

automation for handling the ASA-equipped aircraft during implementation of the reconfiguration to

expand High Altitude Airspace into the ASA airspace:

• The trajectory changes for ASA-equipped aircraft are handled by the ASA Manager vs.ASA

automation will need to be determined.

• For the trajectory changes handled by the ASA automaton, either the ASA Manager, Traffic

Manager, or some automated transfer of data from the DAC planning DST will need to enter

the trajectory changes for ASA-aircraft into the ASA automation.

Roles with HAA Controllers, Airspace Managers, and any DSTs need to be clear:

• The manner in which HAA Controllers might use DSTs under Trajectory Based Operations

(TBO) and data link will need to be determined.

• Roles between Airspace Manager and the HAA Controllers in finalizing new trajectories and

communicating them to HAA aircraft will need to be delineated.

Potential Countermeasure/Next Step: A systematic analysis of the steps to handle transition

of ASA-equipped aircraft and HAA aircraft to their new airspaces should be conducted and

promising procedures should be explored. Further refinement can be performed through

cognitive walkthroughs and HITL simulations.

8.2.2 Demand for Workload, Attention, and Situational Awareness

Issue: The potential for high demands on attention demands and on time for all human operator

positions to gain situation awareness should be investigated:

• Mixed equipage flying in the transition airspace as the HAA airspace moves into ASA airspace

complicates the situation, as different aircraft will have different equipage and different

performance characteristics. It will be more complex for the Airspace Manger to plan an

airspace reconfiguration. It may also be more complex for HAA Controllers to handle

handoffs associated with the reconfiguration in the short time required. Any interactions

between ASA Managers and HAA Controllers need to be worked out.

• Possible excessive number of aircraft to reroute and for an excessive number of conflicts (and

handoffs if applicable) to resolve during an airspace configuration transition.

• Time for human operators to perform their tasks with the complex situations (e.g., mixed

equipage, numerous conflicts to resolve during transition, number handoffs, etc.) under

airspace reconfiguration. Time is inversely linked to ease of decision-making. The concept

design is such that the human operators may need to face many situations in which they will

have a short time to assess the traffic situation and react.

• More tasks for human operators. HAA Controllers will be changing aircraft trajectories which

they did not do under DAC (discussed in Chapter 7). HAA Controllers will also be monitoring
40

for and resolving conflicts (and perhaps conducting handoffs) while rerouting aircraft in

expanded High Altitude Airspace sectors, ASA Managers will be overseeing ASA-equipped

aircraft moving from the new High Altitude Airspace and dealing with ASA duties (See

Section 4.4), and the Airspace Manager will be planning a complex airspace reconfiguration.

All operators may need strategies to allocate attention.

• There will likely also be situations of low workload for the human operators when there is no

airspace reconfiguration and then workload will quickly spike as they need to become involved

in a complex reconfiguration.

Potential Countermeasure/Next Step: Situations that may drive high demands on human

operators as discussed above need to be defined as well as situations where there maybe

high demands following a period of low demand. Some specific topics to study are: the

concept regarding mixed equipage to obtain an understanding of HF issues for this

situation; identification of all tasks human operators will perform; any changes or

refinement of the concepts’ designs should be double checked to see if they pose potential

adverse impact on the human operator’s ability to make time critical decisions; what is the

time needed for operators to get back in loop when low workload is followed by high

workload; means and time to gain situation awareness under such situations; remembering

appropriate responses with perhaps recommendations from automation; and triggers for

events that indicate an emergency or anomaly and automation aids that remind operators

about actions. The ability for the operators to handle these situations and possible

strategies to handle the demand can be investigated with cognitive walkthroughs and HITL

simulations.

8.2.3 Dealing with emergency, safety critical, and off-nominal events

Issue: The need to study alerts and coordination to handle emergency or off-nominal events during

airspace configuration changes.

• Alerts for emergency or off-nominal events, particularly for aircraft near the boundary of the

new or old airspace configurations, if not all aircraft trajectories have to be checked for

compatibility with the configuration change, and whether an aircraft is deviating from its

planned path during an airspace configuration change.

• Coordination for responding to emergency situations is complicated in that there is mixed

equipage which may involve coordination among pilots, ASA Managers, and Controller(s). It

may also need to be determined if coordination has to involve automation or if the decision can

be made solely among humans. These parties may have confusion as to whom they need to

coordinate with in different situations. It may also be unclear who would have the final

decision authority. All decisions must occur with clear and efficient coordination, as many

decisions are likely to be both time and safety critical.

• Aids for remembering or deciding on actions and on coordination needs, including with whom

to coordinate, for responding to emergency or off-nominal events need to be checked for

appropriateness and effectiveness.

Potential Countermeasure/Next Step: Emergency or off-nominal events for which the

human operators need to be aware and coordinate should be identified. Responsibilities for

automation and for human operators need to be allocated for monitoring and resolving

conflicts. Then, means to alert the human operators can be designed. Regarding

coordination, research in collaborative decision-making are useful in designing a sensible
41

coordination mechanism under ASA and DAC mixed operations - as well as a catalog of

observations on how the current ATM system achieves minimal and efficient coordination

in similar situations. Full team configuration, that includes all humans and automations

with potential decision-making authority, needs to be addressed and coordination

procedures and supporting DSTs need to be developed for the possible combinations of

people/automation that require coordination. Those emergencies allocated to automation

should also be studied to see if they can be identified and solved by automation. The

potential designs of automation and procedures for alerts, coordination, and how to deal

with emergences in general can be validated and refined using cognitive walkthroughs and

HITL simulations.

9. Conclusion

Our study identified nine general human factors categories that are accepted categories in human

factors research. These were assessed for each of four RFAs in the NASA NextGen-Airspace

project. To develop countermeasures for human performance issues, we generated 46 questions that

address human performance from the operational systems design perspective. Each question was

analyzed for each RFA and each question was rated as high, medium, or low regarding its priority to

the particular RFA. This summary section reviews the questions rated high across the four RFAs.

The questions are grouped in this summary into high level common themes.

Key human performance issues were similar across the RFAs but the details of how the issues

played out, who were impacted by the issues, and the potential countermeasures differed somewhat

among the RFAs, thereby supporting our initial hypothesis that key human performance issues

should be identified and mitigated within the context of each individual concept. Below is an

integrated summary of human performance issues and analysis needs that cuts across the RFA

results.

1. Distribution of Roles and Responsibilities between Humans and Automation. Critical issues are

driven by the variety of and changing operation modes and time pressure for decision-making:

• roles and responsibilities under the variety of operational modes

• transition of responsibility under evolving events

• clear decision-making authority during short-term conflicts or other critical situations

2. Brittleness and Human Operator Trust in the Automation and Decision Support Systems. Critical

issues are driven by the variety of and changing operational modes:

• layers of redundancy

• short time to act

• increased complexity of operations

• large number of flights

3. Demand for Workload, Attention, and Situational Awareness. Critical issues are driven by the

changing operational modes and short time for decisions:

• low workload then the need for humans to act quickly
42

• numerous and complex actions under time constraints

• complex coordination under time constraints

• awareness of complex situations

• more dynamic flight environments

• single human operator handling multiple operations

• mixed equipage operations

• roles of DSTs, displays, and human-automation interactions

• knowing mode in which operating and how to respond in that mode

• visual attention demands when using data communications

4. Dealing with Emergency, Safety Critical, and Off-Nominal Events. Critical issues are driven by

the possible failures in a large number of and changing operational modes, complexity of

operations, and short time for decisions:

• alert indicators

• reminders of modes in which operating and appropriate actions

• standard versus different responses to emergencies in different concepts

• negotiation and coordination needed in short time periods

• demand on human operator’s working memory and prospective memory

• high number of aircraft and changing airspace configurations

5. Human Operator Characteristics and Training. Critical issues are driven by the numerous and

changing operational modes, use of automation and DSTs, and the need for precision in timing

for actions:

• ability to work in a variety of and changing operational modes

• ability to use automation and DSTs

• ability to assess and deal with traffic situations that are more complicated than today

• training for performance under different operational modes

• skill needs for career paths

• training for human operators to keep their monitoring and decision skills sharp when event

occurrence is rare

• how to train for work in changing conditions and how to conduct on-the-job training when

conditions change

One of the challenges of this project has been that, despite the large volume of analyses conducted,

we were able to cover only few of the concepts in each of the RFAs. We also completed only a

limited assessment on the integration of multiple concepts. Clearly more work is needed.

Future work should include the analyses of the other concepts in the RFAs. Once the key

human performance issues are identified, more resources should be placed to run multiple

walkthroughs on a selected set of concepts that are of particular interest to program managers

and decision-makers. More work is clearly needed in assessing key issues related to integration
43

of multiple concepts. Operational assumptions need to be developed for the combined concepts

since there are few details of the integrated NextGen system. Any information that can improve

the technological and operational assumptions about the concept will significantly improve the

final outcome of the analyses.

Many of the recommendations for the countermeasures include the following steps: 1) further define

the concept; 2) develop prototypes of the tools and operational procedures; and 3) evaluate the

concept via walkthroughs or HITL simulation. These steps are inherently part of the design process,

in which many decisions about the tasks, procedures, algorithms, displays, and technological

assumptions need to be made in order to evaluate the concept. These decisions about the tasks,

procedures, algorithms, displays, and technological assumptions are an integral part of constructing

HITL simulations and can provide useful information for the concept designers; yet the reports that

describe the outcome of the simulations usually focus solely on the results and analyses and do not

include a record of these decisions that could aid concept design. Documenting design decisions and

their rationale could provide invaluable insights into NextGen concept refinements since they offer

valuable “hands-on” experience of a future world in which the concept will operate and has yet to be

created.

Our work has outlined a method of analysis to identify human performance issues related to

NextGen concepts that can be turned into a set of next steps to address the potential human

performance gaps in the concepts. We hope that other NextGen concept areas could utilize the

general human factors and issues questions that we have generated and apply them in their

respective concepts to identify key human performance issues and define future research and design

to address the issues.

10. References

Andre, A.D., Hooey, B.L., & Foyle, D.C. (2007). ASDO Seamless Transition Airspace Information

Requirements and Procedural Integration: Gap Analysis, Human Centered Systems Laboratory

(HCSL) HCSL Technical Report (HCSL-07-04). Moffett Field, CA.

Eurocontrol (2009). European ATM Master Plan,

http://www.eurocontrol.int/sesar/public/subsite_homepage/homepage.html. Brussels, Belgium.

Federal Aviation Administration (2007). Integrated arrival/departure control service (Big Airspace)

concept validation. Washington, DC: FAA Office of Air Traffic Organization Operations

Planning.

Federal Aviation Administration (2008). Federal Aviation Administration flight plan 2009-2013:

Charting the path for the next generation. Washington, DC: Author.

Funk, K., Mauro, R., & Barshi, I. (2009). NextGen Flight Deck Human Factors Issues. Proceedings

of the 15th International Symposium on Aviation Psychology, Dayton, OH, April 27 - 30,

2009.

Joint Planning and Development Office (2007). Concept of Operations for the Next Generation Air

Transportation System, Version 2.0, Washington, DC.
44

Kopardekar, P., Lee, P. U., Smith, N., Lee, K., Prevôt, T., Mercer, J., Homola, J., & Mainini, M.

(2009). Feasibility of Mixed Equipage Operations in the Same Airspace. Proceedings of the

8th USA/Europe Air Traffic Management Research and Development Seminar, Napa, CA.

Lee, P. U., Corker, K., Smith, N., Prevôt, T., Martin, L., Mercer, J., Homola, J., & Guneratne, E.

(2006). A Human-in-the-loop Evaluation of Two Multi-Sector Planner Concepts: Multi-D and

Area Flow Manager, Proceedings of the 25th Digital Avionics Systems Conference, Portland,

OR.

Prevot, T., Homola, J., Mercer, J., Mainini, M., & Cabrall, C. (2009). Initial evaluation of NextGen

Air/Ground Operations with ground-based Automated Separation Assurance. Eighth

USA/Europe Air Traffic Management Research & Development Seminar ATM 2009, June,

Napa, California, USA.

Rowe, D.W., Borowski, M., Wendling, V.S., & DeSenti, C.T. (2003). Redesigning High Altitude

Airspace in the National Airspace System: modeling, simulation, and assessment. Journal of

Air Traffic Control, Vol. 45, no. 2, pp 17-23. Apr-June 2003.

Sheridan, T. B., Corker, K. M., & Nadler, E. D. (2006). Report of a workshop on human-automation

interaction in NGATS. (DOT-VNTSC-NASA-06-02.)

SLDAST: System-Level Design, Analysis, and Simulation Tools (2007). NASA NextGen Air

Transportation System ATM-Airspace Project SLDAST Technical Integration Report,

November, 26. Moffett Field, CA.

Stalnaker, S., Ostwald, P., Beaton, E., Levin, K. & Stapleton S. (2008). Integrated Time-based Flow

Management Concept. The 26th Congress of International Council of the Aeronautical

Sciences (ICAS), Anchorage, AK, September 14 - 19.

Swenson, H., Barhydt, R., & Landis, M. (2006), Next Generation Air Transport System (NGATS)

Air Traffic Management (ATM)- Airspace Project, Moffett Field, CA.

Willems, B. & Koros, A. (2007). Advanced concept of the National Airspace System of 2015:

Human factors considerations for air traffic control (DOT/FAA/TC-TN07/21). Atlantic City

International Airport, NJ: Federal Aviation Administration William J. Hughes Technical

Center.
45

Appendix A: Human Factors Reference List

A.1 Attention Allocation

Bolstad, C. A., Cuevas H. M., Gonzalez, C., & Schneider, M. (2005). Modeling shared situation

awareness. Proceedings of the 14th Conference on Behavior Representation in Modeling and

Simulation (BRIMS). Los Angeles, CA.

Bolstad, C. A., Foltz, P., Franzke, M., Cuevas, H. M., Rosenstein, M., & Costello, A. M. (2007).

Predicting situation awareness from team communications. Proceedings of the 51st Annual

Meeting of the Human Factors and Ergonomics Society. Santa Monica, CA: HFES.

Caldwell, J. A. (1997). Fatigue in the aviation environment: An overview of the causes and effects

as well as recommended countermeasures. Aviation, Space and Environmental Medicine, 68,

932-938.

Dinges, D. F., Graeber, R. C., Rosekind, M. R., Samel, A. & Wegmann, H. M. (1996) Principals and

guidelines for duty and rest scheduling in commercial aviation. (NASA Tech Memorandum

110404). Moffett Field, CA: NASA Ames Research Center.

Dinges, D. F., Pack, F., Williams, K., Gillen K. A., Powell, J. W., Ott, G. E., Aptowicz, C. & Pack.

A. I. (1997). Cumulative sleepiness, mood disturbance, and psychomotor vigilance

performance decrements during a week of sleep restricted 4-5 hours per night. Sleep, 20, 267-

277.

Endsley, M. (1995). Measurement of situation awareness in dynamic systems. Human Factors, 37,

65-84.

Foltz, P. W., Bolstad, C. A., Cuevas, H. M., Franzke, M., Rosenstein, M., & Costello, A. M. (2008).

Measuring situation awareness through automated communication analysis. In M. Letsky, N.

Warner, S. M. Fiore, & C. Smith (Eds.), Macrocognition in teams: Human factors in defence,

pp. 259-273. Aldershot, UK: Ashgate.

Histon, J. M. & Hansman, R. J. (2008). Mitigating complexity in air traffic control: The role of

structure-based abstractions. Retrieved May 1, 2009, from http://hdl.handle.net/1721.1/42006

Jeannot, E., Kelly, C. & Thompson, D. (2003). The development of situation awareness measures in

ATM systems. Brussels: Eurocontrol.

Kaber, D. B., Perry, C. M., Segall, N., McClernon, C. K., & Prinzell III, L. J. (2006). Situation

awareness implications of adaptive automation for information processing in an air traffic

control-related task. International Journal of Industrial Ergonomics, 36(5), 447-462.

Kramer, A., Wiegman, D. and Kirlik, A. (2007). Attention: From Theory to Practice. NY: Oxford.

Landry, S. J., Sheridan, T. B., & Yufik, Y. M. (2001). A methodology for studying cognitive

groupings in a target tracking task. IEEE Transactions on intelligent transportation systems,

2(2), 92-100.

Lau, N., Skraaning Jr., G., Jamieson, G. A., & Burns, C. M. (2008). Enhancing operator task

performance during monitoring for unanticipated events through ecological interface design. In

Proceedings of the Human Factors and Ergonomics Society 52nd Annual Meeting.
46

Loukopoulos, L. D., Dismukes, R. K., & Barshi, I. (2009). The Multitasking myth: Handling

complexity in real-world operations. Burlington, VT: Ashgate.

McFarlane, D. C. (2002). Comparison of four primary methods for coordinating the interruption of

people in human-computer interaction. Human-Computer Interaction, 17, 63-139.

Monk T. H., Buysse, D., Reynolds III, C., Berga, S., Jarrett, D., Begley, A., & Kupfer, D. (1997).

Circadian rhythms in human performance and mood under constant conditions. Journal of

Sleep Research, 6(1), 9-18.

Monan, W. P. (1978). Distraction - A human factor in air carrier hazard events. (Technical

Memorandum No. 78608). Moffett Field, California: Ames Research Center.

Moray, N. & Inagaki, T. (2000). Attention and complacency. Theoretical Issues in Ergonomics

Science, 1, 354–365.

Niessen, C. & Eyferth, K. (2001). A model of the air traffic controller’s picture. Safety Science,

37(2-3), 187-202.

Pattyn, N., Neyt, X., Henderickx, D., & Soetens, E. (2008). Psychophysiological investigation of

vigilance decrement: Boredom or cognitive fatigue? Physiology & Behavior, 93, 369-378.

Posner, M. I., Snyder, C. R. R., & Davidson, D. J. (1980). Attention and the detection of signals.

Journal of Experimental Psychology: General, 109, 160-174.

Reynolds, T. G., Histon, J. M., Davison, H. J., & Hansman, R. J. (2002). Structure, intent, &

conformance monitoring in ATC. Workshop on ATM System Architectures and CNS

Technologies, Capri, Italy, September 2002.

Sollenberger, R. L. & Stein, E. S. (1995). A simulation study of air traffic controllers’ situation

awareness. In Proceedings of an International Conference: Experimental Analysis and

Measurement of Situation Awareness (pp. 211-217). Daytona Beach, FL.

Sheridan, T. (2002). Humans and automation. New York: Wiley.

Sperling, G. (1960). The information in brief visual presentations. Psychological Monographs, 74

(Whole number 11).

Vortac, O. U., Edwards, M. B., Fuller, D. K., & Manning, C. A. (1994). Automation and cognition

in air traffic control: An empirical investigation. Applied Cognitive Psychology, 7, 631-651.

Wickens, C. (1980). The structure of attentional resources. In R. Nickerson (Ed.), Attention and

performance VIII (pp. 239-257). Mahwah, NJ: Erlbaum.

A.2 Decision-making and Mental Modeling

Bailey, L.L., Willems, B.F., & Peterson, L.M. (2001). The effects of workload and decision-support

automation on enroute R-Side and D-Side communication exchanges. (DOT/FAA/AM-01/20).

Department of Transportation, Federal Aviation Administration, Office of Aerospace

Medicine, Washington, D.C.

Bisantz, A. & Seong, Y. (2001). Assessment of operator trust in and utilization of automated

decision-aids under different framing conditions. International Journal of Industrial

Ergonomics, 28, 85–97.
47

Brehmer, B. (1987). The psychology of risk. In W.T. Singleton & J. Hovden (Eds.), Risk and

Decisions (pp. 25-39). New York: John Wiley & Sons.

Cohen, M.S. (1993). The naturalistic basis of decision biases. In G. Klein, J. Orasanu, R.

Calderwood & C. Zsambok (Eds.), Decision making in action: Models and methods (pp. 51-

99). Norwood, NJ: Ablex.

Cohen, M., Parasuraman, R., & Freeman, J. T. (1998). Trust in decision aids: A model and its

training implications. In Proceedings of the Command and Control Research and Technology

Symposium (pp. 255–265). Washington, DC: Department of Defense, Command and Control

Research Program.

D’Arcy, J., & Della Rocca, P.S. (2001). Air traffic control specialist decision making and strategic

planning - A field survey. (DOT/FAA/CT-TN01/05). William J. Hughes Technical Center,

Atlantic City International Airport: NJ.

Fiske, S.T. & Taylor, S.E. (1991). Social Cognition (2nd Ed). New York: McGraw-Hill.

Flin, E. Salas, M. Strub & Martin, L. (Eds.), Decision making under stress: Emerging themes and

applications. Aldershot, UK: Ashgate.

Foyle, D. & Hooey, B. (2008). Human performance modeling in aviation. Boca Raton, FL: CRC

Press.

Hollenbeck, J., Ilgen, D., Phillips, J. & Hedlund, J. (1994). Decision risk in dynamic two-stage

contexts: Beyond the status-quo. Journal of Applied Psychology, 79(4), 592-598.

Klein, G. (1993). Sources of error in naturalistic decision making. In Proceedings of the Human

Factors and Ergonomics Society 37th Annual Meeting, 1, 368-371. Santa Monica, CA: Human

Factors and Ergonomics Society.

Lanier, R. C. & Coppenbarger, R. (2004). Human factors in the design and development of an

advanced decision support tool. In The 23rd Digital Avionics Systems Conference, Oct. 24-28,

2004, 5.b.2 – 5.1-9, vol. 1.

Means, B., Salas, E., Crandall, B. & Jacobs, T.O. (1993). Training decision makers for the real

world. In G. Klein, J. Orasanu, R. Calderwood, & C. Zsambok (Eds.), Decision making in

action: Models and methods (pp. 51-99). Norwood, NJ: Ablex.

Metzger, U. & Parasuraman, R. (2005). Automation in future air traffic management: Effects of

decision aide reliability on controller performance and mental workload. Human Factors,

47(1), 35-49.

Mogford, R. H. (1994). Application of research techniques for documenting cognitive processes in

air traffic control: Sector complexity and decision making. (DOT/FAA/CT-TN94/3).

Department of Transportation Federal Aviation Association Technical Center, Atlantic City,

NJ.

Moray, N. (1997). Models of models of ....mental models. In T. Sheridan and T. Van Lunteran

(Eds.), Perspectives on the human controller (pp. 271-285). Mahwah, NJ: Erlbaum.

Payne, J. W., Bettman, J. R. & Johnson, E. J. (1988). Adaptive strategy selection in decision making.

Journal of Experimental Psychology: Learning, Memory and Cognition, 14(3), 534-552.
48

Rasmussen, J. (1983). Skills, rules and knowledge: Signals, signs and symbols, and other

distinctions in human performance models. IEEE Transactions on Systems, Man and

Cybernetics, SMC-132557-266.

Rasmussen, J. (1985). The role of hierarchical knowledge representation in decision making and

system management. IEEE Transactions on Systems, Man and Cybernetics, 2, (SMC - 15)

234-243.

Simon, H. A. (1979). Rational decision making in business organizations. The American Economic

Review, 69 (4), 493-513.

Tversky, A. & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science,

185(4157) 1124 – 1131.

 Woods, D. D., Johannesen, L. J., Cook, R. I. & Sarter, N. B. (1994). Behind human error: Cognitive

systems, computers, and hindsight. Wright-Patterson Air Force Base, Ohio: Crew Systems

Ergonomics Information Analysis Center.

Zsambok, C. & Klein, G. (1997). Naturalistic Decision Making. Hillsdale, NJ: Lawrence Erlbaum.

A.3 Communication

Bailey, L. L., Willems, B. F., & Peterson, L. M. (2001). Effects of workload and decision-support

automation on en route R-side and D-side communication exchanges. (DOT/FAA/AM-01/20.)

Coppenbarger, R. A., Kanning, G., & Salcido, R. (2001). Real-time data link of aircraft parameters

to the Center-TRACON Automation System (CTAS). Presented at the 4th USA/Europe ATM

R&D Seminar, Santa Fe, NM, 3-7 Dec. 2001.

Hellenberg, J. R. & Wickens, C. D. (2001). Effects of data link modality and display redundancy on

pilot performance: An attentional perspective. International Journal of Aviation Psychology,

13, 189-210.

Latorella, K. (1998). Effects of modality on interrupted flight deck performance: Implications for

data link. In Proceedings of the 42nd Annual Meeting of the Human Factors and Ergonomics

Society (pp. 87–91). Santa Monica, CA: Human Factors and Ergonomics Society.

McMillan, D. (1998) “...Say Again?...” Miscommunications in air traffic control. Master’s Thesis,

Queensland University of Technology.

Metzger, U. & Parasuraman, R. (2006). Effects of automated conflict cuing and traffic density on air

traffic controller performance and visual attention in a data link environment. The International

Journal of Aviation Psychology, 16(4), 343-362.

Prichett, A. R. & Hansman, R. J. (1997). Pilot non-conformance to alerting system commands

during closely spaced parallel approaches. MIT Aeronautical Systems Laboratory Report,

ASL-97-2, January, 1997.

Rantanen, E. M. & Kokayeff, N. K. (2002). Pilot error in copying air traffic control clearances. In

Proceedings of the Human Factors and Ergonomics Society, 2002, pp. 145-149.

Rognin, L. & Blanquart, J. P. (2001). Human communication, mutual awareness and system

dependability: Lessons learnt from air-traffic control studies. Reliability Engineering and

System Safety, 71(3), 327-336.
49

Shannon, C. (1949). Communication in the presence of noise. In Proceedings of the Institute of

Radio Engineers, 37, 10-22.

Sharples, S., Stedmon, A., Cox, G., Nicholls, A., Shuttleworth, T., & Wilson, J. (2007). Flightdeck

and air traffic control collaboration evaluation (FACE): Evaluating aviation communication in

the laboratory and field. Applied Ergonomics, 38(4), 399-407.

Schleicher, D. R., Jones, E., Dow, D., & Coppenbarger, R. A. (2002). Improved lateral trajectory

prediction through en route air-ground data exchange. AIAA Aircraft Technology, Integration

and Operations (ATIO) Conference, Los Angeles, CA, 1-3 Oct. 2002

Sollenberger, R. L., McAnulty, D. M., & Kerns, K. (2003). The effect of voice communications

latency in high density, communications-intensive airspace (DOT/FAA/CT/TN03/04). William

J. Hughes Technical Center, Atlantic City International Airport: NJ.

Tufte, E. (1983). The visual display of quantitative information. Cheshre, CT: Graphics Press.

A.4 Memory

Brandimonte, M., Einstein, G. O., & McDaniel, M. A. (1996). Prospective memory: Theory and

applications. Mahwah, New Jersey: Erlbaum.

Cardosi, K. (2005). Runway safety. In B. Kirwan, M. Rodgers, & D. Schafer (Eds.), Human factors

impacts in air traffic management. Burlington, VT: Ashgate.

Dismukes, R. K. (2007). Prospective memory in aviation and everyday settings. In M. Kliegel, M.

A. McDaniel, & G. O. Einstein (Eds.), Prospective memory: Cognitive, neuroscience,

developmental, and applied perspectives. Mahwah, NJ: Erlbaum.

Dismukes, R. K. (2006) Concurrent task management and prospective memory: Pilot error as a

model for the vulnerability of experts. In Proceedings of the Human Factors and Ergonomics

Society 50th Annual Meeting (pp. 903-114). San Francisco, CA: HFES.

Gromelski, S., Davidson, L., & Stein, E. S. (1992) Controller memory enhancement: Field facility

concepts and techniques. (DOT/FAA/CTTN92/7). William J. Hughes Technical Center,

Atlantic City International Airport: NJ.

Gronlund. S. D., Ohrt, D. D., Dougherty, M. R. Perry, J. L., & Manning, C. A. (1998). Role of

memory in air traffic control. Journal of Experimental Psychology: Applied, 4(3), 263-280.

Latorella, K. A. (1999). Investigating interruptions: Implications for flightdeck performance

(Technical Memorandum No. 209707). Moffett Field, California: NASA Ames Research

Center.

Li, S., Y. W., Blandford, A., Cairns, P. & Young, R. M. (2008). The effect of interruptions on

postcompletion and other procedural errors: An account based on the activation-based goal

memory model. Journal of Experimental Psychology: Applied, 14(4), 314-328.

McDaniel, M. A., & Einstein, G. O. (2007). Prospective memory: An overview and synthesis of an

emerging field. Los Angeles, CA: Sage.

McFarlane, D. C. (2002). Comparison of four primary methods for coordinating the interruption of

people in human-computer interaction. Human-Computer Interaction, 17, 63-139.
50

Monk, C. A., Trafton, J. G., & Boehm-Davis, D. A. (2008). The effect of interruption duration and

demand on resuming suspended goals. Journal of Experimental Psychology: Applied, 14(4),

299-313.

Norman, D.A. (1981). Categorization of action slips. Psychological Review, 88, 1-14.

Reason, J. T., & Mycielska, K. (1982). Absent-minded? The psychology of mental lapses and

everyday errors. Englewood Cliffs, NJ: Prentice Hall.

Stein, E. S. & Garland, D. (1993). Air traffic controller working memory: Considerations in air

traffic control tactical operations (DOT/FAA/CTTN93/37). Atlantic City, NJ: Federal Aviation

Administration.

Stone, M., Dismukes, R. K., & Remington, R. W. (2001). Prospective memory in dynamic

environments: Effects of load, delay, and phonological rehearsal. Memory, 9(3), 165-176.

Vingelis, P. H., Schaeffer, E., Stringer, P., Gromelski, S., & Ahmed, B. (1990). Air traffic controller

memory enhancement: Literature review and proposed memory aids (DOT/FAA/CTTN90/38), Atlantic City, NJ: DOT/FAA Technical Center.

Vortac, O. U., Edwards, M. B., Fuller, D. K., Manning, C. A. (1993). Air traffic controller working

memory: Considerations in air traffic control tactical operations (Rep. No.

DOT/FAA/CTM93/97). Atlantic City, NJ: Federal Aviation Administration.

Vortac, O. U., Edwards, M. B., & Manning, C. A. (1995). Functions of external cues in prospective

memory (Final Report) (DOT/FAA/AM-95/9). Civil Aeromedical Inst., Oklahoma City, OK.

A.5 Workload

Ahlstrom, U. & Friedman-Berg, F. (2005). Subjective workload ratings and eye movement activity

measures (DOT/FAA/CT-05/32). Atlantic City International Airport, NJ: William J. Hughes

Technical Center, Atlantic City International Airport: NJ.

Aigoin, G. (2001). Air traffic complexity modeling. Master Thesis. ENAC, Toulouse, France.

Arad, B.A., (1964). The controller load & sector design. Journal of Air Traffic Control, 13(31), 13-

31.

Athènes, S., Averty, P., Puechmorel, S., Delahaye, D., & Collet, C. (2002) Complexity and

controller workload: Trying to bridge the gap. International Conference on Human-Computer

Interaction in Aeronautics, Massachusetts Institute of Technology: Cambridge, MA, USA.

Averty, P. Athènes, S., Collet, C. & Dittmar, A. (2005). Mental workload in ATC an index

constructed from field text. Aviation Space Environmental Medicine 75(4), 333-341.

Baddeley, A. D. & Hitch, G. J. (1974). Working Memory. In G. Bower, (Ed.), Advances in Learning

and Motivation. Vol. 8, pp. 47-90 New York: Academic Press.

Bagheri, N. & Jamieson, G. A. (2004). Considering subjective trust and monitoring behavior in

assessing automation-induced “complacency.” In Proceedings of the Human Performance,

Situation Awareness and Automation Technology Conference, 2004.
51

Bailey, L. L., Willems, B. F., & Peterson, L. M. (2001). Effects of workload and decision-support

automation on En route R-side and D-side communication exchanges (DOT/FAA/AM-01/20).

Atlantic City, NJ: Federal Aviation Administration.

Baron, S., & Corker, K. (1989). Engineering-based approaches to human performance modeling. In

G. McMillan, D. Beevis, E. Salas, M. Strub, R. Sutton, & L. Van Breda (Eds.), Applications of

Human Performance Models to Systems Design, Plenum Publishing.

Cao, A., Chintamani, K. K., Pandya, A. K., & Ellis, R. D. (2009). NASA TLX: Software for

assessing subjective mental workload. Behavior Research Methods, 41(1), 113-117.

Cardosi, K. M & Murphy, E.D. (1995). Human factors in the design and evaluation of air traffic

control systems (DOT/FAA/RD-95/3). Volpe Research Center, Cambridge MA. US

Department of Transportation.

Chatterji, G.B. & Sridhar, B. (2001). Measures for air traffic controller workload prediction. First

AIAA Aircraft Technology, Integration and Operations Forum. Los Angeles, CA.

Corker, K. (2005). Computational Human Performance Models and Air Traffic Management. In B.

Kirwan, M. Rodgers and D. Schafers (Eds.) Human Factors Impacts in Air Traffic

Management, pp. 317-350. Aldershot, UK: Ashgate Publishing.

Corker, K., Howard, M. & Mooij, M. (2005). Adaptation and adoption of technologies: Examination

of the User Request Evaluation Tool (URET). (SJSUHAIL 2005- 2442-01). San Jose State

University, CA.

Costa, G. (1993). Evaluation of workload in air traffic controllers. Ergonomics, 36(9), 1111-1120.

Gao, J., & Lee, J. D. (2006). Extending the decision field theory to model operators’ reliance on

automation in supervisory control situations. IEEE Transactions on Systems, Man, and

Cybernetics, 36, 943–959.

Gingell, R., Corker, K., & Weber, E. (2005). The feasibility of measuring capacity in a real-time

ATM simulation independent of subjective controller workload measurement. USA/Europe

International Symposium in Air Traffic Control, Baltimore, MA.

Hart, S. G. & Staveland, L. E. (1988) Development of NASA-TLX (Task Load Index): Results of

empirical and theoretical research. In P. A. Hancock and N. Meshkati (Eds.) Human mental

workload. Amsterdam: North Holland Press.

Hopkin, V.D. (1994). Human performance implications of air traffic control automation. In M.

Mouloua & R. Parasuraman (Eds.) (1996). Human Performance in Automated Systems:

Current Research and Trends. Hillsdale, New Jersey: Erlbaum.

Hooey, B. M. & Wickens, C. (1993). Workload in transition: Implications for individual and team

performance. Washington, DC: National Academy Press.

Lee, P. U. (2005). A non-linear relationship between controller workload and traffic count. In

Human Factors and Ergonomics Society Annual Meeting Proceedings, Human Performance

Modeling (pp. 1129-1133).

Lee, P. U., Mercer, J., & Callantine, T. (2007). Examining the moderating effect of workload on

controller task distribution. In Proceedings of the 7th International Conference, EPCE 2007,

Beijing, China, July 22-27.
52

Manning, C. A., Mills, S. H., Fox, C. M., & Pfleiderer, E. (2000). Investigating the validity of

performance and objective workload evaluation research (POWER). Paper presented in the

third USA/Europe Air Traffic Management R & D Seminar, Naples, Italy.

Manning, C. A. & Pfliederer. E. M. (2006). Relationship of sector activity and sector complexity to

air traffic controller taskload (DOT/FAA/AM-06/29). Oklahoma City, OK: Civil Aerospace

medical Institute Federal Aviation Administration.

Metzger, U., & Parasuraman, R. (2005). Automation in future air traffic management: Effects of

decision aid reliability on controller performance and mental workload. Human Factors, 47(1),

35-49.

Metzer, U. & Parasuraman, R. (2006). Effects of automated conflict cuing and traffic density on air

traffic controller performance and visual attention in a data link environment. The International

Journal of Aviation Psychology, 16(4), 343–362.

Mogford, R. H., Guttman, J. A., Morrow, S. L., & Kopardekar, P. (1995). The complexity construct

in air traffic control: A review and synthesis of the literature (DOT/FAA/-CT TN95/22).

Atlantic City, New Jersey: Federal Aviation Administration.

Niedringhaus, W. P. & Gisch, A. H. (1983). Automated en route air traffic control algorithmic

specifications: Sector workload probe (MTR-83W152). McLean, VA.

Sperandio, J. C. (1971). Variations of operators' strategies and regulating effects on workload.

Ergonomics, 14, 571-577.

Stein, E.S. (1985). Air traffic controller workload: An examination of workload probe (FAA/CTTN90/60). Atlantic City, New Jersey: Federal Aviation Administration.

A.6 Interaction with Automation/Decision Support Tools and Displays

Andre, A., & Cutler, H. (1998). Displaying uncertainty in advanced navigation systems. In

Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting (pp. 31-35).

Santa Monica, CA: HFES.

Bagheri, N. & Jamieson, G. A. (2004a). A sampling model to ascertain automation-induced

complacency in multi-task environments. In Proceedings of the IFIP Working Conference on

Human Error, Safety and System Development (pp. 131-145), 2004.

Bagheri, N. & Jamieson, G. A. (2004b). The impact of context-related reliability on automation

failure detection and scanning behavior. IEEE International Conference on Systems, Man and

Cyperbetics, pp. 212-217, 2004.

Bailey, N., & Scerbo, M. W. (2007). Automation-induced complacency for monitoring highly

reliable systems: The role of task complexity, system experience, and operator trust.

Theoretical Issues in Ergonomics Science, 8, 321–348.

Cardosi, K. (2003). Human factors integration challenges in the terminal radar approach control

(TRACON) environment (DOT/FAA/AR-02/127). Cambridge, MA: John A. Volpe National

Transportation Systems Center.

Consiglio, M., Hoadley, S., Wing, D., Bazley, B., & Allen, B. D. (2008). Impact of pilot delay and

non-responsiveness on the safety performance of airborne separation. Paper presented at the
53

26th Congress of International Council of the Aeronautical Sciences (ICAS), 14 – 19

September 2008, Anchorage, Alaska.

Dennis, G. D. (2003). T-SAFE: Building a trusted computing base for air traffic control software.

Master’s Thesis, MIT, 2003.

Dixon, S., Wickens, C. D., & McCarley, J. M. (2007). On the independence of reliance and

compliance: Are false alarms worse than misses? Human Factors, 49, 564–572.

Dzindolet, M. T., Peterson, S. A., Pomranky, R., Pierce, L. G., & Beck, H. P. (2003). The role of

trust in automation reliance. International Journal of Human-Computer Studies, 6, 697–718.

Della Rocco, P., Panjwani, G., Friedman-Berg, F. Kopardekar, P., & Hah, S. (2003). Collocation of

User Request Evaluation Tool, Traffic Management Advisor, and controller pilot data link

communications: An initial human factors evaluation (DOT/FAA/CT-TN02). William J.

Hughes Technical Center, Atlantic City International Airport: NJ.

Farrell, S. & Lewandowsky, S. (2000). A connectionist model of complacency and adaptive

recovery under automation. Journal of Experimental Psychology: Learning, Memory, and

Cognition, 26(2), 395-410.

Goillau, P., Kelly, C., Boardman, M., Jeannot, E. (2003). Guidelines for trust in future ATM

systems-Measures. Retrieved May 1, 2009 from

http://www.hf.faa.gov/Portal/techrptdetails.aspx?id=1295

Kelly, C., Boardman, M., Goillau, P., & Jeannot, E. (2003). Guidelines for trust in future ATM

Systems: A literature review. HRS/HSP-005-GUI-01.

Krois, P., Mogford, R., & Rehmann, J. (2003). FAA/NASA human factors for evolving

environments: Human factors attributes and technology readiness levels (DOT/FAA/AR03/43). Washington, DC: Federal Aviation Administration.

Lau, N., Skraaning Jr., G., Jamieson, G. A., & Burns, C. M. (2008). Enhancing operator task

performance during monitoring for unanticipated events through ecological interface design. In

Proceedings of the Human Factors and Ergonomics Society 52nd Annual Meeting.

Lee, J. D., & See, K. A. (2004). Trust in automation: designing for appropriate reliance. Human

Factors, 46(1), 50-80.

Lee, K. K., Kerns, K., Bone, R., & Nickelson, M. (2001). Development and validation of the

Controller Acceptance Rating Scale (CARS): Results of empirical research. Presented at the

4th USA/Europe Air Traffic Management Research and Development Seminar (ATM-2001),

Santa Fe, NM, 3-7 Dec. 2001.

Madhavan, P., & Wiegmann, D. A. (2007). Similarities and differences between human-human and

human-automation trust. Theoretical Issues in Ergonomics Science, 8, 277–301.

Miller, C., & Parasuraman, R. (2007). Designing for flexible interaction between humans and

automation: Delegation interfaces for supervisory control. Human Factors, 49, 57–75.

Molloy, R., & Parasuraman, R. (1996). Monitoring an automated system for a single failure:

Vigilance and task complexity effects. Human Factors, 38, 311–322

Nadler, E. (2005). Human factors integration challenges in the Traffic Flow Management (TFM)

environment (DOT/FAA/AR-05/40). Cambridge, MA: DOT.
54

Neumann, P. G. (2006). Risks of untrustworthiness. In Proceedings of the 22nd Annual Computer

Security Applications Conference, pp. 321-328.

Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2008). Situation awareness, mental workload,

and trust in automation: Viable, empirically supported cognitive engineering constructs.

Journal of Cognitive Engineering and Decision Making, 2(2), 140-160.

Prichett, A. R. & Hansman, R. J. (1997). Pilot non-conformance to alerting system commands

during closely spaced parallel approaches. MIT Aeronautical Systems Laboratory Report,

ASL-97-2, January, 1997.

Shorrock, S. T. (2006). Errors of perception in air traffic control. Safety Science, 45(8), 890-904.

Vincente, K. J. & Rasmussen, J. (1992). Ecological interface design: Theoretical foundations. IEEE

Transactions on Systems, Man and Cybernetics, 22(4), 589-606.

Walton, D., Quinn, C., & Atkins, S. (2002). Human factors lessons learned from a surface

management system simulation. Presented at the AIAA Aircraft Technology, Integration and

Operations (ATIO) Conference, Los Angeles, CA, 1-3 Oct. 2002.

Willems, B., & Koros, A. (2007). Advanced concept of the National Airspace System of 2015:

Human factors considerations for air traffic control (DOT/FAA/TC-TN07/21). Atlantic City

International Airport, NJ: Federal Aviation Administration William J. Hughes Technical

Center.

Xing, J. (2007). Information complexity in air traffic control displays (DOT/FAA/AM-07/26).

Oklahoma City, OK: FAA Civil Aerospace Medical Institute.

A.7 Organizational Factors

Bhadra, D. & Schaufele, R. (2007). Probabilistic forecasts for aviation traffic at FAA’s commercial

terminals: Suggested methodology and an example. Transportation Research Record: Journal

of the Transportation Research Board, 2007, 37-46.

Booker, P. (2005). Air traffic control automation: For humans or people? Human Factors and

Aerospace Safety, 5(1), 23-42.

Erzberger, H. (2004). Transforming the NAS: The Next Generation air traffic control system. In

Proceedings of the International Congress of the Aeronautical Sciences (ICAS), August 30,

2004, Yokohama, Japan.

Flin, R. Mearns, K., O’Connor, P., & Bryden, R. (2000). Measuring safety climate: Identifying the

common features. Safety Science, 34, 177-192.

Hollnagel, E., Woods, D. D., & Leveson, N. (2006). Resilience engineering: Concepts and precepts.

Burlington, VT: Ashgate.

Iordanova, B. N. (2003). Air traffic knowledge management policy. European Journal of

Operational Research, 146(1), 83-100.

Ishutkina, M. A. & Hansman, J. R. (2009). Analysis of the interaction between air transportation

economic activity: A worldwide perspective (ICAT-2009-2). MIT International Center for Air

Transportation (ICAT). Department of Aeronautics & Astronautics, Massachusetts Institute of

Technology, Cambridge: MA.
55

Johnson, C. W., Kirwan, B., Licu, A., & Stastny P. (2009). Recognition primed decision making and

the organisatinal response to accidents: Überlingen and the challenges of safety improvement

in European air traffic management. Safety Science, 47(6), 853-872.

Kirwan, B. (2007). Safety informing design. Safety Science, 45(1-2), 155-197.

Klein, H. K. (2005). System development in the federal government: How technology influences

outcomes. Policy Studies Journal, 28(2), 313-328.

Marais, K., & Leveson, N. (2003). Archetypes for organisational safety. In Proceedings of the

Second Workshop on the Investigation and Reporting of Incidents and Accidents (IRIA 2003),

NASA/CP-2003-212642. Retrieved May 14, 2009, from

http://shemesh.larc.nasa.gov/iria03/p01-marais.pdf

Mozdzanowska, A. (2008). System transition: Dynamics of change in the US air transportation

system. Master’s Thesis, MIT, May, 2008.

Poole, R. (2007b). The urgent need to reform the FAA’s air traffic control system. Reason

Foundation.

Reason, J. (1997). Managing the risks of organizational accidents. Burlington, VT: Ashgate.

Vicente, K. (1999). Cognitive work analysis. Mahwah, NJ: Erlbaum.

Zemrowski, K. M. (2008). Impacts of increasing reliance on automation in air traffic control

systems. In Systems Conference 2008 2nd Annual IEEEE, pp 1-6.

A.8 Potential Errors and Recovery from Errors or System Failures

Andrews, J. W., Welch, J. D., & Erzberger, H. (2006). Safety analysis for advanced separation

concepts. Air Traffic Control Quarterly, 14(1), 5-24.

Averty, P., Guittet, K., & Lezaud, P. (2008). An ordered Logit model of air traffic controllers'

conflict risk judgment. Air Traffic Control Quarterly, 16(2), 101-125.

Cerone, A., Connelly, S., & Lindsay, P. (2008). Formal analysis of human operator behavioural

patterns in interactive surveillance systems. Software and Systems Modeling, 7(3), 273-286.

Danaher, J. W. (1980). Human error in ATC system operations. Human Factors, 22, 535–545.

Di Nocera, F., Fabrizi, R., Terenzi, M., & Ferlazzo, F. (2006). Procedural errors in air traffic control:

Effects of traffic density, expertise, and automation. Aviation, Space, and Environmental

Medicine, 77(6), 639-643.

Gordon, R., Shorrock, S.T., Pozzi, S. & Boschiero, A. (2004). Using human error analysis to help to

focus safety analysis in ATM simulations: ASAS separation. Paper presented at the 2004

Human Factors and Ergonomics Society Conference, Australia, 22nd– 25th August, 2004.

Li, S., Y. W., Blandford, A., Cairns, P. & Young, R. M. (2008). The effect of interruptions on

postcompletion and other procedural errors: An account based on the activation-based goal

memory model. Journal of Experimental Psychology: Applied, 14(4), 314-328.

Isaac, A., Shorrock, S. T., & Kirwan, B. (2002). Human error in European air traffic management:

The HERA project. Reliability Engineering & System Safety, 75(2), 257-272.
56

Kanse, L. & Van der Schaaf, T. W. (2000). Recovery from failures: Understanding the positive role

of human operators during incidents. In Proceedings of the Human Factors and Ergonomics

Society Europe Chapter Annual Meeting 2000, Maastricht, Netherlands, Nov. 1-3, pp. 367-

379.

Kontogiannis, T. & Malakis, S. (2009). A proactive approach to human error detection and

identification in aviation and air traffic control. Safety Science, 47(5), 693-706.

Merket, D. C., Bergondy, M., & Cuevas-Mesa, H. (1997, March). Making sense out of teamwork

errors in complex environments. Paper presented at the 18th Annual Industrial/OrganizationalOrganizational Behavior Graduate Student Conference, Roanoke, VA.

Redding, R. E. (1992). Analysis o operational errors and workload in air traffic control. In

Proceedings of the 36th Human Factors and Ergonomics Society Annual Meeting: Training,

pp. 1321-1325.

Rodgers, M., Mogford, R., & Mogford, L. (1998). The relationship of sector characteristics to

operational errors (ReportNo.DOT/FAA/AM-98/14). Washington, D.C.: Federal Aviation

Administration Office of Aviation Medicine.

Sheridan, T. B., & Parasuraman, R. (2000). Human vs. automation in responding to failures: An

expected-value analysis. Human Factors, 42, 403-407.

Shorrock, S. T. (2006). Errors of perception in air traffic control. Safety Science, 45(8), 890-904.

Shorrock, S. T. (2006). A framework for managing system disturbances and insights from air traffic

management. Ergonomics, 49(12-13), 1326-1344.

Shorrock, S. T. & Kirwan, B.(2002). Development and application of a human error identification

tool for air traffic control. Applied Ergonomics, 33(4), 319-336.

Subothic, B., Ochieng, W. Y., & Straeter, O. (2007). Recovery from equipment failures in ATC:

Determination of contextual factors. Reliability Engineering and Systems Safety, 92(7), 858-

870.

A.9 Personnel Selection, Certification, and Training

Bolczak, R. & Celio, J. (2005). Accommodating ATC system evolution through advanced training

techniques. Presented at the AIAA 5th Aviation, Technology, Integration, and Operations

Conference (ATIO), 26-28 September 2005, Arlington, VA.

Broach, D. (1997). Designing selection tests for the future National Airspace System architecture

(DOT/FAA/AM-97/19). Washington DC: Federal Aviation Administration, Office of Aviation

Medicine.

Broach, B. (2005). A singular success: Air traffic control specialist selection 1981–1992. In B.

Kirwan, M. Rodgers, & D. Schafer (Eds.), Human factors impacts in air traffic management

(pp. 177-206). Burlington, VT: Ashgate.

Ellis, K. H. & McDonell, P. (2003) On-the-job training in air traffic control. In I. M. A. Henley

(Ed.), Aviation education and training, adult learning principles and teaching strategies (pp.

367-387). Aldershot, UK: Ashgate Publishing Limited.
57

Forest, L. M. & Hansman, R. J. (2006). The future oceanic ATC environment: Analysis of mixed

communication, navigation, and surveillance equipage. Air Traffic Control Quarterly, 14(2),

117-138.

Groppe, M. & Bui, M. (2008). Study of cockpit’s perspective on human-human interactions to guide

collaborative decision making design in air traffic management. In the First International

Conference on Advances in Computer-Human Interaction (pp.107-113).

Jorna, P.G.A. (2000). ATM human factors and human resources considerations. Air & Space

Europe, 2(5), September-October, pp. 58-64.

Kirwan, B. (2001). The role of the controller in the accelerating industry of air traffic management.

Safety Science, 37, 151-185.

Major, L., Johannsson, H., Davison, H., Hvannberg, E. T., & Hansman, R. J. (2004). Key humancentered transition issues for future oceanic air traffic control systems. HCI-Aero 2004,

Toulouse, France.

Manning, C. A. & Broach, D. (1998) Identifying ability requirements for operators of future

automated air traffic control systems (DOT/ FAA-AM-92-26). Washington, DC: Federal

Aviation Administration, Office of Aviation Medicine.

Manning, C. & Stein, E. (2005). Human factors methodologies: Measuring air traffic controller

performance in the 21st century. In B. Kirwan, M. Rodgers, & D. Schafer (Eds.), Human

factors impacts in air traffic management (pp. 283-316). Burlington, VT: Ashgate.

Manzey, D., Bahner, J. E., & Hueper, A.-K. (2006). Misuse of automated aids in process control:

Complacency, automation bias, and possible training interventions. Proceedings of the Human

Factors and Ergonomics Society 50th Annual Meeting (pp. 220–224). Santa Monica, CA:

Human Factors and Ergonomics Society.

Metzger, U. & Parasuraman, R. (2001). The role of the air traffic controller in future air traffic

management: An empirical study of active control versus passive monitoring. Human Factors,

43(4), 519-528.

Sheridan, T. B. (2006). Next Generation air transportation system: Human-automation interaction

and organizational risks. Paper presented at the Resilience Engineering Symposium, Juan les

Pins, France, November 8-10.

Sheridan, T. B., Corker, K. M., & Nadler, E. D. (2006). Report of a workshop on human-automation

interaction in NGATS. (DOT-VNTSC-NASA-06-02.)

Voller, L. & Fowler, A. (2005). Human factors and human resources: Human factors longitudinal

study to support the improvement of air traffic controller training. In B. Kirwan, M. Rodgers,

& D. Schafer (Eds.), Human factors impacts in air traffic management, pp. 153-176.

Burlington, VT: Ashgate.

Verma, S., Kozon, T., Cheng, V., & Ballinger, D. (2007). Changes in roles/responsibilities of air

traffic control under precision taxiing. Paper presented at the Digital Avionics Systems

Conference (DASC), IEEE/AIAA 26th, 21-25 Oct. 2007, 6.B.2-1-6.B.2-12.

Willems, B., & Koros, A. (2007). Advanced concept of the National Airspace System of 2015:

Human factors considerations for air traffic control (DOT/FAA/TC-TN07/21). Atlantic City

International Airport, NJ: Federal Aviation Administration William J. Hughes Technical

Center.
58
59

Appendix B: Human Performance Factors in NextGen

In this report, we reviewed the existing human factors literature in the context of the NextGen

operations proposed by JPDO, NASA, and FAA. The references to the human factors literature were

compiled and included in the Appendix A. The review suggests that the basic human factors related

to NextGen concepts and human performance issues are known and relatively constant. However,

the factors that are critical to each concept differ depending on the underlying assumptions about the

operational procedures and the tool capabilities.

Based on the review, we have identified nine categories of basic human factors which are described

below. Satisfaction of these conditions may be thought of as “requirements” from a behavioral

perspective for any human operator of a complex human-machine system including NextGen.

Below, each category is discussed to provide an understanding of the category and its implications

for safe and efficient human performance in NextGen operations. The discussion is for the benefit

of researchers who may be less familiar with general human factors research. Appendix A provides

further references for researchers who would like further information on human factors research.

These nine categories will be addressed for each of the four RFAs assessed for human factor issues.

At the end of each subsection, critical terms, common means of measurement, and comments on our

analysis of the category are described in italics. The measurements are the ones that can be taken

during human-system evaluations, such as a HITL simulation.

B.1 Attention Allocation

NextGen will place critical demands on attention and its allocation to many tasks, some routine and

some not. This refers to allocation of either or both cognitive and visual effort, the assumption being

that a human can only give primary attention to one thing at a time, though keeping other salient task

requirements in memory. There can be cognitive effort (thinking) without visual attention, and

visual attention (gaze, focus) without cognitive effort. Attention allocation here is taken to mean not

only allocation or focus of cognitive and/or visual attention within each given task, but also

awareness of the broader “situation” (situation awareness). Both of these terms are associated with

monitoring, which is prevalent in just about all NextGen operations. Both visual gaze/scanning and

cognitive attention can be voluntary or can be involuntarily “grabbed” by visual or auditory stimuli

(e.g., alarms, alerts). Rational attention allocation must be based on some combination of temporal

urgency, time required to observe and act, and relative importance.

In addition to the cognitive construct of situation awareness are the neurophysiological constructs of

arousal and alertness, which generally depict the organism’s readiness to sense stimuli, and

vigilance, usually referring to a person’s capability to attend over a prolonged period of time.

Several cognitive factors affect attention in a negative sense: Distraction is normally considered to

be a disturbance to rational attention allocation, refocusing visual or cognitive attention to something

other than what is the appropriate set of stimuli. Fatigue, drowsiness, emotional stress and boredom

reduce a person’s attention capacity; there are well-established countermeasures for each. All four

are also related to mental workload, which, because of its importance to ATM task design is here

treated separately below. Some of the recommended readings on this topic are:
60

• Endsley (1995) posited three stages of situation awareness: 1) the observer perceives all salient

elements of the environment; 2) the observer comprehends the meaning of the situation; and 3)

the observer predicts the effects of potential actions on the situation.

• Wickens (1980) postulated a multi-dimensional cognitive resource model, with one dimension

being either perception (visual or auditory) or response (manual or speech), and a second

dimension being either spatial or verbal processing. We might add the distinction between

analogic vs. symbolic inputs or outputs.

• Kramer, Wiegman and Kirlik (2007) provide a good review of attention concepts applied to

systems design.

Means of Measurement

Attention is measured experimentally in simulation by detecting errors of omission and commission,

such as not following a proper procedure or not being aware of the surrounding situation. Eye

movement instrumentation can be used to measure visual attention to different elements on a

display. Situation awareness can be measured by the Situation Awareness Global Assessment

Technique (SAGAT) process in which the experimenter interrupts a simulation without warning and

asks the experimental subjects about the status of particular variables, or what will happen next, etc.

(Endsley, 1995).

B.2 Decision-making and Mental Modeling

Decisions can be automatic, skill-based, and not necessarily conscious; this is characteristic of

much of piloting and air traffic control for experienced operators. Some decisions are

characteristic of sensory-motor skills that are continuous (or continual) in time and space or

learned skills that are performed without great conscious effort. Alternatively decisions can be

made by learned rules: “if X then Y, else Z,” usually referred to as rule-based behavior

(Rasmussen, 1983). Finally, decisions can be made by reflection on stored knowledge and

innovation: so-called knowledge-based behavior. Both rule-based and knowledge-based behavior

can be said to depend on some mental model, a hypothetical internal representations of “what

would happen if...” and what to do in that case.

Rational decision-makers consider both probability and consequences, but evidence suggests that

human decision-making is more a matter of quasi-rational satisficing (Simon, 1979). Satisficing is

the process of arriving at a decision that is acceptable, though not necessarily optimal. (Determining

the unique optimal is likely to take time and resources that are unavailable to humans and agreement

on what is optimal is unlikely in any case.) We are especially concerned in NextGen with decisionmaking under a modicum of time stress, which typically limits the decision-maker to skill- or rulebased behavior, and addition of emotional stress almost certainly results in habituated satisficing.

Rational decision-making is based on maximizing some explicit “goodness” criterion (Sheridan,

2002). The criterion most often cited is the product of probability of a desirable outcome from the

decision and the magnitude of the worth (payoff, value) of that outcome. An equivalent criterion is

the minimization of the product of probability of an undesirable outcome of the decision and the

magnitude of the negative worth (cost) of that outcome. Outcome worth is scaled with respect to a

subjective utility, which is a nonlinear function of dollars, time, inconvenience, effect on reputation

or other social factors, etc. Such a decision criterion is risk-neutral, as compared to a risk-averse

criterion wherein the decision maker prefers, for example, a low-cost, high probability outcome to a
61

high-cost, low probability outcome both of which have the same probability-cost product. For

example, people gladly buy insurance to avoid low-probability, high cost outcomes, where on

average over time insurance company profits and the buyer loses. Most people are risk averse, but

not close to the most risk-averse min/max criterion, where, when choosing among alternatives, they

would select the one with the least possible cost regardless of the probability. People tend to

overestimate probability of low probability events, especially if they have high consequences, and

underestimate probability of frequent common events. The risk-averse nature of human decisionmaking plays a significant role in air traffic system in which catastrophic events with low probability

may be monitored better than more probable events with less catastrophic outcome.

The idea of a mental model, which might be described as a cognitive representation of how

something works or how a series of events plays out, has received considerable interest as humancomputer interaction has become more prevalent (Moray, 1997). Unfortunately, eliciting these

relationships in quantitative form is not possible because only subjective reporting is available.

Quantitative models of human decision-making, on the other hand, are prevalent and found in the

literature under many headings in addition to what is discussed above and including, for example,

probability estimation, signal detection, continuous control decision, formal game playing, etc.; too

many to review here. Foyle and Hooey (2008) review a variety of decision performance models

applied to aviation.

Means of Measurement

Decision errors may be noted by HITL simulation observers with respect to bad timing,

inappropriate configuration ordered, inappropriate controller assignments, or improper intent

based on incorrect mental model or perception. The latter often occur because of careless

habituation (unusual circumstances require something other than a standard response).

B.3 Communication

This includes both receiving (sensing, perceiving, and understanding) and sending (message is

correctly formatted, correctly transmitted, and comprehensible to the receiver). We include here

communication with other people as well as communication with computers. The modes of

communication include voice, data-link typing and reading of text, and graphical presentations or

interactions (e.g., with maps). Safe air traffic management is totally dependent on effective

communication.

The most common formal model of communication is that of Shannon (1949), where information in

a message is defined by the number of bits in the message set that is reduced to certainty (zero bits)

after the message is received. Formally this is the average of the logarithm of probability of each

specific message in an ensemble (analogous to the number of binary “cuts” that need to be made to

reduce the set of possible messages to certainty). The degree to which a complex message

(requiring narrowing from of a large set of possibilities) is sent but with the receiver seeing little

variety from which to choose (i.e., much of the variety is lost en route) is called equivocation. The

degree to which a simple message is sent (selection from of a very small set) but appears complex to

the receiver (appearing to require disambiguation from a much larger message set including added

variety not originally there) is called noise. Both equivocation and noise detract from effective

communication and are easily quantified if joint sent-received probabilities are known over time.
62

Such communication models have been used extensively to characterize human capabilities in

sensory and motor skills. It should be emphasized that for Shannon information is equivalent to the

inverse probability (surprise) of a message, and has nothing to do with the worth or usefulness or

meaning of the message.

Design of visual and auditory displays for detection, readability, and understandability has a large

effect on communication performance, as does the specification of speech terminology to be used in

various operating procedures. Display design continues to be an art (Tufte, 1983).

In NextGen, data link communication is expected to replace or supplement voice communication to

modify and communicate route trajectories. Researchers have investigated the effects of data link on

controller performance (e.g., Farley, et al., 1998; Hansman and Davison, 2001; Lozito, Verma,

Martin, Dunbar, and McGann, 2003). Some of the human factors challenges that have been

identified include longer transaction times, difficulties in reviewing data link communications, loss

of “party-line” information that are available for situation awareness when pilots share a common

radio frequency, and potential “head-down” time required for text-based data system.

Means of Measurement

Communication errors of terminology use, omitting required messages or other miscommunications

are measured experimentally in HITL simulations, or can be noted in preliminary walkthroughs in

training trials. Quantitative measure of communication (information transmission) requires knowing

the matrix of conditional probabilities of messages sent and messages received.

B.4 Memory

We are concerned with human operator’s memory in both aviation operations and in training. In

operations this includes capability of operators to remember instructions regarding what to do and

when to do it (Vortac, et al., 1993; Stein and Garland, 1993; Stone, et al., 2001; Dismukes, 2006,

2007; Cardosi, 2005). This includes both prospective memory (self-initiated remembering to

remember an intention) and retrospective memory (recall of past events, actions, or knowledge). It

also includes printed and computer-based memory aids. Memory can also refer to group memory

(someone in a closely interacting group can recall what is needed) or to human-plus-machine

memory, in the sense of memory aids being readily available (on a computer display or in print). In

training the questions is: what information/procedures, etc., need to be remembered (as opposed to

provided in situ), and how best to instill that memory.

A distinction is also made between immediate and long-term memory, where the time ranges are

separated by roughly one hour. Immediate memory is supported by associations in the ambient

operational context which is expected to be more or less continually the same as when the stimuli

were first observed, whereas with long term memory the context will have changed and

associational mechanisms been interrupted numerous times.

Memories are plastic in the same sense that other brain faculties are plastic, and can be influenced by

external suggestion, peer pressure, and by one’s own wishful thinking.
63

Means of Measurement

Forgetting errors of omission or commission may be recorded in HITL simulations. Memory for

procedures can be tested in training exercises. Memory issues include remembering when to do an

action, what action to take, and how to perform the action. Memory is particularly important under

NextGen since more precision is involved in operations involving more aircraft, and actions need to

be performed correctly at the current time.

B.5 Workload

The concern for workload in aviation is almost entirely with mental workload as contrasted to

physical (energy) workload. Workload is most problematic during abrupt transitions from low to

high workload such as when circumstances suddenly change from passive monitoring of normal

events and automatic control to having to recognize, diagnose and intervene in a critical unexpected

abnormal situation. Sustained high workload can occur occasionally even in expected operations, but

this should be anticipated and means should be engineered to plan ahead and/or “buy time.” Very

low mental workload can occur when automation is performing all essential tasks, and the operator

becomes bored or drowsy, which exacerbates the ability to intervene when it becomes necessary to

do so.

Interest in mental workload more or less began in aviation when the DC 9-80 was being recertified

and both the airlines and the FAA alleged that 3-person crews were no longer needed, whereas the

pilot union claimed otherwise and cited pilot workload as their reason. From this controversy

emerged the idea of a multi-attribute subjective scale that includes at least frequency and duration of

separate tasks (“busyness”), complexity of tasks including overlap and time from cue to appropriate

action, and affect (emotional stress) (Sheridan and Simpson, 1979). The NASA Task Load Index

(TLX) scale is a refinement of multi-attribute workload scales (Hart and Staveland, 1988) now

widely used and available as computer software. The subjective scale continues to be the standard,

since objective scales have limitations (see below).

Effective mental workload is also a function of the degree to which necessary cognitive resources

overlap in time, as noted above under Attention. Experience and expectation tend to make any given

task easier and therefore lower workload. One strategy used by experienced pilots and controllers is

to anticipate tasks so as to “plan and work ahead,” and to even out the pace of activities (“speed up

on the straight-a-ways, slow down on the curves”). Unexpected events can catch an operator “off

guard,” and require significant time to “get one’s bearings” and decide what to do. Such events may

become more prevalent in NextGen when operators monitoring automated systems are alerted and

suddenly called upon to intervene.

Means of Measurement

The preferred means to measure workload is with a subjective rating scale such as NASA TLX or

FAA Air Traffic Workload Input Technique (ATWIT; Stein, 1985). Communication time has

sometimes been used as an objective measure of workload. Secondary task measures of workload

such as counting backwards or target tracking (the worse one does on the secondary task the

greater the load of the primary task) are only for experiments because they interact with the

operation being tested. Observer workload ratings and physiological workload indices (e.g., heart

rate and heart rate variability, respiration rate, galvanic skin response, pupil diameter, etc.) are

known to pose high intra- and inter-subject variability.
64

Since there will be considerable changes in operations under NextGen (e.g., Dynamic Airspace

Configuration, Continuous Descent Approach, etc.) and considerably more use of automation than

today, workload issues may occur under NextGen in a different ways than today.

B.6 Interaction with Automation, Decision Support Tools, and Displays

“Automation” can mean complete closed-loop automatic control or, more broadly, it can mean

instrumentation which senses physical variables, transforms that information into a form

understandable to a human operator, and possibly provides advice/suggestions for action but leaves

the final decision and triggering of action to the human. In the majority of NextGen concepts (with

the exception of Automated Separation Assurance and other far-term NextGen concepts), we are

concerned with the latter, so-called decision support tools (DSTs). Subsumed under this heading are

several issues. One issue is (1) allocation of functions between humans and automation: how to

achieve a best synthesis between what humans do best and what computers do best. This is usually

not a matter of “either human or machine,” but of varying degrees of machine-aided sensing,

analysis, presentation, and decision. Also subsumed are (2) conventional display design factors, and

(3) issues of the human understanding the automation: knowing what it can do and cannot do, and

what to expect from it. Major design efforts are also warranted to (4) make displays and controls

ecological, meaning interaction with them is easy and natural and corresponds to expectation.

Finally there are (5) issues regarding human trust of the automation. If a decision support tool is not

trusted, it will be ignored; or if over-trusted it may be used improperly.

Trust in automation is a somewhat new human factors concern, and not a topic one would find in

any traditional engineering discipline. Automation that is not trusted will not be used, and if it is

over-trusted it may be depended upon when it is not trustworthy (Lee and See, 2004; Kirlik, 2006)

As automation is added to the any system, the role of the human changes from being a direct or inthe-loop controller to that of a supervisor, whose new functions are to plan what to ask the

automation to do, teach or program the desired action together with “if...then...else...”

contingencies; activate the automation; monitor its performance on the assigned task; detect any

discrepancies or failures, in which case it is necessary to intervene and reprogram or abort the

operation; and learn from experience so that further use is best managed.

It is useful at the design stage to consider what level of automation is appropriate to provide

(Sheridan, 2002). At the lowest level the automation merely provides “canned” information as

requested, such as an airport diagram or a weather report. At a next higher level there may be

flexibility built in to advise the user on what action to take as a function of measured variables or

further information input by the user. At a still higher level, the automation might execute that

action at the user’s initiation. Additional sophistication might enable the automation to request

permission to take actions it deems appropriate, and ultimately to take actions if the user does not

object within some time limit, or even to take actions independent of the user. There have been

various forms of such scales, and shades of independence between the particular levels described

above. While the aircrew is somewhat accustomed to being a supervisor (“flight manager” of the

FMS), that role is new to the NextGen controller.
65

There are multiple variables that characterize the capability of automation. Some of these are:

• robustness, or variety of functions the automation can perform

• range of environmental variables within which it functions well

• operating speed following command entry or other trigger

• control accuracy

• capability for delegation of authority

• recovery of authority under given conditions

• ability to operate in parallel with command entry

• variety of display mode alternatives

• feedback provided on currency and confidence in information provided

• feedback provided on success of control execution

A somewhat different attribute of automation is its usability by the human operator or supervisor,

where the variables include:

• appropriate sensory mode for display of information

• visibility and audibility of information sufficiently above absolute threshold

• message sufficiently above differential threshold (lack of display clutter and noise)

• appropriately analogic or symbolic display

• text or graphics or what combination

• clarity or information re expectations, language, culture, etc.

• appropriate redundancy in displays

• ecological naturalness re affordances (tasks)

• interactive etiquette

• time required to read or operate

• opportunities for mode error

• ability to recover from mis-commands

• easy intervention to modify and reprogram

• transparency as to what it has done, is doing, and will do

• distracting features

• engendering appropriate level of trust

• assistance in situation awareness

Means of Measurement

Evaluation of (1) above can be by user rating, observer rating, or focus group appraisal; (2) above,

which includes the many standard display criteria (e.g., size, brightness, contrast, text readability,

graphics understandability, clutter, pagination, etc.) should be evaluated by a human factors

professional who is familiar with the individual tasks and perhaps scores the various factors on a

checklist; (3) and (4) can be measured off-line by written questionnaires; (5) should be evaluated by

an observer in a simulation, or by errors recorded in performing a simulated part-task using the

DST.
66

B.7 Organizational Factors

Organizational factors involve lines of authority and relative responsibilities of and to different

people (and in rare cases to machines where humans may be subservient to automation). Though

collaborative decision-making by aircrews and ground personnel is emphasized in NextGen

planning, it is well appreciated that time-critical decisions should not be made by committee

deliberations: lines of authority and responsibility should go together and be clear for both normal

and off-normal operations. Policies must be set for what requirements aircraft must meet to be

authorized to enter certain airspace at given times or perform certain maneuvers such as continuous

descent approaches or paired approaches, etc.

The organizational structure determines the cognitive work of the component actors. Cognitive

work analysis has become a popular human factors topic in recent years. Vicente (1999) provides a

good review.

Currently experts in safety and risk analysis of human-machine systems emphasize the perils of

managerial complacency (that the organization is regarded to be safe because failures have not

recently occurred), or lack of surveillance of organizational readiness for unexpected adverse events.

They promote resilience engineering, a focus on the blunt end (management policy and safety

culture) as contrasted to the sharp end (where operator error is most readily apparent) (Hollnagel, et

al., 2006).

The high degree of instrumentation and communication in NextGen will mean that there will be less

need for humans to make subjective judgments regarding safety events, which should help promote

safety culture and obviate the “blame game” and under-reporting that have been so divisive in

current air traffic management.

The great challenge in NextGen is to provide air traffic management services that are efficient in

both capacity and cost and yet, given all the necessary automation, provide fulfilling work for the

personnel.

Means of Measurement

Measures of organization effectiveness include whether personnel at all levels share common goals,

whether communication across divisions at the same level and between levels functions well, and

whether cost-benefit tradeoffs are articulated, are realistic and openly observable. A “safety

culture” should be a common goal. One criterion of organizational effectiveness is whether

organizations can are prepared major safety challenges when they come.

B.8 Potential Errors and Recovery from Human Errors or System Failures

Human factors often take a negative perspective on human performance. That is, an error (by a

human operator or a system designer) can be defined as a failure to meet some (arbitrarily defined)

standard of performance. Such an event – whether one’s own error, that of another person or a

hardware or software failure – can pose the need to take recovery action. While potential for human

error and system failure must be considered and minimized in design, we may expect that such

errors and failures will always occur. Thus the primary concern is with modes of recovery when
67

errors/failures do happen (Reason, 1990). Options for recovery from errors/failures that can be

anticipated should be apparent and must be sufficiently easy and timely to implement.

There are differing opinions as to whether counting of past errors and incidents contributes

significantly to prevention of future errors, as precursor circumstances tend to be unique, and errorcounting tends to focus on the sharp-end, as noted above. Incidents tend to be grossly under-reported

for the obvious reason of people’s reluctance to self-incriminate. Accidents (events associated with

deaths, injuries, or significant property damage) are rare and seldom occur in the same way, so

accident statistics tend to be of limited substantive use unless they recur in the same way (though

political pressures demand elaborate accident investigations in any case).

In NextGen concept evaluations, anticipation of errors is not an activity that planners and technology

adopters wish to engage in. Planning and simulation trials generally are focused on nominal

scenarios and procedures. Off-nominal and emergency procedures are easily put off, presumably

until sufficient evidence accumulates, but that may be too late to fix design flaws. Although it is

difficult to anticipate errors in future air traffic systems, off-nominal and emergency situations that

can be anticipated should be tested and mitigated as an integral part of the NextGen concept design.

Means of Measurement

Human errors and failures can be observed by human observers and/or measured by

instrumentation provided they are sufficiently well defined. It is particularly important to record offnominal errors and failures during HITL simulation exercises as they indicate vulnerabilities in

hardware and software design, procedures, and human behavior, and suggest needs for redesign

prior to actual operations and requirements for training afterward after equipment is in operation.

B.9 Potential Selection, Certification, and Training

Many air traffic controllers are now retiring and replacements are being trained. There are somewhat

different expectations for NextGen controller personnel, mainly having to do with ability to

understand and work with automation, think strategically, and “manage” in contrast to being in-theloop tactical controllers (though that need has certainly not gone away).

Controllers are known to be a proud and disciplined lot, conscious of their great responsibility. They

are also experts of air traffic domain, capable of being quick and decisive in safety-critical

situations. There is danger in substituting individuals who may be intelligent but ill-suited for the

tasks for those who are dependable and trustworthy and have sound judgment in both routine and

non-routine activities.

However, NextGen does require controllers with new skills as more automation and DSTs play a

larger role in ATM systems. Up to now hardware and software have been certified relatively

independently of controller and pilot certification, though usability by human operators is becoming

more and more recognized as an essential requirement. In NextGen, because of the great extent to

which both pilots and controllers will be dependent on decision support systems, the usability factor

will increase in importance.

NextGen should increase training requirements, particularly with regard to understanding roles and

responsibilities and having a good understanding of capabilities and limitations of the automation

and decision support tools. Training should include initial training in various normal and off-normal
68

operations as well as refresher training. Both concept training and procedural training can be done

as now with some combination of reading, lecture, part-task hardware simulation, and relatively

high-fidelity HITL simulation.

The extensive use of automation will pose further training requirements to ensure that ANSP, as well

as pilots, know:

• What function(s) automation is designed to do

• What input data the automation uses to do this

• What output variable(s) are controlled, and normally to what accuracy

• What is the range for key input and output variables within which the automation is competent

to perform its job

• Whether the automation has the capability to alert/alarm whenever these ranges are exceeded

• What approximately is the algorithm (or heuristics, or control law) used by the automation to

transform the input to the output

• What is the reliability of the automation within the alleged and competent range.

Means of Measurement

Measurement in the case of job qualification and certification factors will be difficult until selection

and training criteria can be set, and then candidates and trainees can be tested against those

criteria.

B.10 References

Cardosi, K. (2005). Runway safety. In B. Kirwan, M. Rodgers, & D. Schafer (Eds.), Human factors

impacts in air traffic management. Burlington, VT: Ashgate.

Dismukes, R. K. (2006) Concurrent task management and prospective memory: Pilot error as a

model for the vulnerability of experts. In Proceedings of the Human Factors and Ergonomics

Society 50th Annual Meeting (pp. 903-114). San Francisco, CA: HFES.

Dismukes, R. K. (2007). Prospective memory in aviation and everyday settings. In M. Kliegel, M.

A. McDaniel, & G. O. Einstein (Eds.), Prospective memory: Cognitive, neuroscience,

developmental, and applied perspectives. Mahwah, NJ: Erlbaum.

Endsley, M. (1995). Measurement of situation awareness in dynamic systems. Human Factors, 37,

65-84.

Farley, T., Hansman, R., Endsley, M., Amonlirdviman, K., & Vigeant-Langlois, L (1998). The

Effect of Shared Information on Pilot/Controller Situation Awareness and Re-Route

Negotiation. FAA / European ATM Conference.

Foyle, D. & Hooey, B. (2008). Human performance modeling in aviation. Boca Raton, FL: CRC

Press.

Hansman, R. & Davison, H. (2001). Effect of shared information on pilot / controller and controller/

controller interactions. In G. Donohue & Zellweger, A. (Eds), Air transportation systems

engineering, Ch14, pp205-225. Reston, VA:AIAA.

Hart, S. G. & Staveland, L. E. (1988) Development of NASA-TLX (Task Load Index): Results of

empirical and theoretical research. In P. A. Hancock and N. Meshkati (Eds.)
69

Human mental workload. Amsterdam: North Holland Press.

Hollnagel, E., Woods, D.D., & Leveson, N. (2006). Resilience engineering: Concepts and precepts.

Burlington, VT: Ashgate.

Kirlik, A. (2006). Human-Technology Interactions: Methods and models for cognitive engineering

and human-computer interaction. New York, NY: Oxford University Press.

Kramer, A., Wiegman, D. and Kirlik, A. (2007). Attention: From Theory to Practice. NY: Oxford.

Lee, J. D., & See, K. A. (2004). Trust in automation: designing for appropriate reliance. Human

Factors, 46(1), 50-80.

Lozito, S., Verma, S., Martin, L., Dunbar, M., & McGann, A. (2003). The impact of voice, data

link, and mixed air traffic control environments on flight deck procedures. Air Traffic Control

Quarterly, 11, 4, 293-310.

Moray, N. (1997). Models of models of ....mental models. In T. Sheridan and T. Van Lunteran

(Eds.), Perspectives on the human controller (pp. 271-285). Mahwah, NJ: Erlbaum.

Rasmussen, J. (1983). Skills, rules and knowledge: Signals, signs and symbols, and other

distinctions in human performance models. IEEE Transactions on Systems, Man and

Cybernetics, SMC-132557-266.

Reason, J. (1990). Human Error. Cambridge, UK: Cambridge University Press.

Shannon, C. (1949). Communication in the presence of noise. In Proceedings of the Institute of

Radio Engineers, 37, 10-22.

Sheridan, T. & Simpson, R. (1979). Toward the Definition and Measurement of the Mental

Workload of Transport Pilots. FTL Report 79.Cambridge, MA: MIT Flight Transportation

Laboratory.

Sheridan, T. (2002). Humans and automation. New York: Wiley.

Simon, H. A. (1979). Rational decision making in business organizations. The American Economic

Review, 69, 4 (Sep., 1979), pp. 493-513.

Stein, E. S. & Garland, D. (1993). Air traffic controller working memory: Considerations in air

traffic control tactical operations (DOT/FAA/CTTN93/37). Atlantic City, NJ: Federal Aviation

Administration.

Stein, E.S. (1985). Air traffic controller workload: An examination of workload probe (FAA/CTTN90/60). Atlantic City, New Jersey: Federal Aviation Administration.

Stone, M., Dismukes, R. K., & Remington, R. W. (2001). Prospective memory in dynamic

environments: Effects of load, delay, and phonological rehearsal. Memory, 9 (3), 165-176.

Tufte, E. (1983). The visual display of quantitative information. Cheshre, CT: Graphics Press.

Vicente, K. (1999). Cognitive work analysis. Mahwah, NJ: Erlbaum.

Vortac, O. U., Edwards, M. B., Fuller, D. K., Manning, C. A. (1993). Air traffic controller working

memory: Considerations in air traffic control tactical operations (Rep. No.

DOT/FAA/CTM93/97). Atlantic City, NJ: Federal Aviation Administration.

Wickens, C. (1980). The structure of attentional resources. In R. Nickerson (Ed.), Attention and

performance VIII (pp. 239-257). Mahwah, NJ: Erlbaum.
70
71

Appendix C: Human Performance Issues in the Context of the

NextGen Concepts and Operations

Part of the effort in our identification and characterization of human factor issues is to identify gaps

in and potential countermeasures for operational requirements in the NASA RFAs and NextGen

concepts regarding these human factor issues. In the course of our research, we also concluded that

the gaps in human factors research and potential countermeasures lie in the accurate identification of

key human-systems related issues, study of the issues in the context of the concept operations, and a

better design of the operational requirements for the concept to address the issues.

In order to simplify identifying gaps in and potential countermeasures for operational requirements

in the NASA RFA and NextGen concepts, the nine human factor categories were combined into five

groupings explicitly related to human performance from the operational systems perspective. The

nine basic human factor categories were organized into the five categories of human performance:

1. Use of automation and Decision Support Tools (DSTs)

2. Human decision-making and workload

3. Attention, situation awareness, and memory

4. Communication and coordination

5. Organization, selection, job qualification, and certification factors

We formulated a series of questions related to human performance issues that is specific enough to

be used in concept design. We have made an assessment for each question as it relates to each of the

four RFAs studied in this report. Not all questions apply to all RFAs, and we state when a question

is not critical to a RFA.

Recovery from potential human errors or system failures was applicable across the five categories

above, and therefore the questions related to error recovery were distributed across the categories.

The questions generated in each of the above five human performance categories point directly to

potential approaches and countermeasures to mitigate the human performance issues of concern.

The questions were compiled from reviewing prior work from the report’s authors and other

NextGen human factors researchers (e.g., Sheridan et al., 2006; Willems and Koros, 2007) and

expertise gained from HITL simulation activities related to NextGen concept evaluation (e.g.,

Prevot, et al., 2009; Lee, et al., 2007).

After discussing how each question relates to a particular RFA, a priority rating has been assigned to

each question. The rating is expressed as high, medium, and low and indicates the need for HF

research for the particular question. To assign the priority rating, we consider the:

• need for HF research since the answer to the question is not readily evident for the particular

concept

• need for further concept design to address the HF question

• criticality for safety for concept

• criticality to achieving the desired benefit of concept.

The questions are categorized and listed on the following pages.
72

C.1 Use of Automation and DSTs

1. What is the distribution of roles and responsibilities across different human operators and

automation? Is the responsibility clear for each task? Does the concept clearly delineate the

functions by the different agents (human or automation) either by time or by procedures? If not

(meaning two agents can potentially act on a same aircraft at the same time), does the concept

clearly identify an expedient coordination/negotiation mechanism to clarify the intent of multiple

agents in the time constraint given for the problem? How is intent communicated? Will the

human and automation “trade” control and responsibility of the tasks (i.e., hand-off the tasks

between them? Or will there be expectations for “sharing” tasks (where the human does part and

the automation simultaneously does part)? What are the mechanisms for switching tasks between

them? Do automation and human evaluate the operations in a similar manner?

2. Who has decision making authority when there are conflicting decisions or advice given by some

combination of automation and other operators? Is there a clear trigger event for any one

operator to decide and act? Is there enough time to solve the problem under such circumstances?

Are there procedures to do this? Does the operator get to choose how long to wait if s/he must act

or does the automation determine this for them?

3. Is automation allowed to execute tasks without the human operator in-the-loop or is automation

providing DST functions with operator in-the-loop? Can the automation tell whether the human

operator is in the loop or not? If the operator is not in the loop, can they jump in when they need

to? If the automation is responsible will the operator lose motivation to pay sufficient attention?

I.e. what is the time constraint that the operator has to solve the problem? How critical is the

solution (e.g., safety critical)? Is the trigger event salient? Is the operator (e.g., controller, pilot)

being out-of-the-loop feasible for a given concept?

4. Are there any cases where the human operator will need to become involved, such as when there

is an exception to nominal situations? If so, how and when does the human operator know there

is an exception and s/he needs to act? What information does the automation need to present to

the human in the exception cases? How much advanced notice does the human need?

5. What is the proper degree of trust by humans in the automation? For example, too much may

lead to complacency and too little to disuse or misuse. How much reliance on the automation is

necessary for the concept to work?

6. Do human operators have the appropriate amount of information on their displays to do their

jobs successfully within the time constraints that they have? For example, do they have enough

information but not so much that they are overloaded and break down? When should the DST

recommend solutions and when just provide information?

7. If automation/DSTs fails, will human operators realize it and recover? Will the automation

properly indicate failure?

8. Will the DSTs recognize a problem and help in emergency situations? Are the DSTs smart

enough to indicate when an unusual situation is beyond their ability to be useful and should not be

relied upon?
73

C.2 Human Decision-making and Workload

1. Does a single human operator handle multiple operations? Or will different operators handle

different functions in the same airspace?

2. Is there complex mixed equipage airspace? Can the combined human operators and automation

handle the complex mixed equipage airspace?

3. Is there a sufficient timeframe for decision-making for critical situations, especially off-normal

situations?

4. Can the human operator attend to multiple alternatives in support of optimization? How many

options should be presented?

5. Are there peak workload situations that should be moderated? What metrics are appropriate?

6. Can human operators perform all the functions assigned to them? If not, can the function be

7. What is the workload impact when there is transitioning between airspace configurations or

different modes of operations (e.g., from CDA to FDMS)?

8. If there are mixed surveillance sources with different precisions, can the human operators utilize

the more precise information available for some of the aircraft and the less precise surveillance

information on other aircraft?

9. Are there appropriate layers of human operator redundancies built into the concept so that some

operators monitor others to anticipate if help is needed (e.g., Area Sup monitors controller

workload and TMU monitors and controls traffic demand with anticipation of congestion)? Is the

automation able to monitor the operator for signs of stress or workload? Does the human

supervisor have special displays to indicate their level of stress?

10. In responding to emergencies or recovering from error, are there standard responses generated

across different modes of operation or does it need to be different for each mode? If different is

the operator likely to be confused about which mode is appropriate?

11. How brittle is the concept? How flexibly can the operations adapt to off-normal situations? How

easy is it for the concept to fall apart under off nominal situations, and can any such situations be

anticipated? How are fail-soft modes and resilience built into the system?

C.3 Attention, Situation Awareness, and Memory

1. Are there any excessive demands on attention (too many things at once with too much accuracy

required)?

2. What is the time required to gain situational awareness to perform a normal task?

3. If the human operator is passively monitoring and suddenly there is a need to get back into the

loop, how long does it take to do this? Is there enough time to do this for the given concept? What

is the workload involved? Is it a feasible amount workload?

4. Can automation complement or replace human monitoring function by calling attention to salient

and critical events?

5. Are there recognizable indicators of emergency and critical safety events? Are there situations

where alertness of human operator to new information and indicators may be a problem?

6. Are the modes of operations (e.g., automated, self, or controller separation assurance) or

airspace configuration always clear to the human operators (i.e., pilot and/or controller) in both
74

nominal and off-nominal situations? Are there too many alternative modes such that the human

operator is likely to respond as though in one mode when actually in another?

7. Are there situations where mental fixation and “cognitive tunneling” may be a problem?

8. What strategies are used to allocate attention during task execution? How does the human

operator know when to attend to the automation?

9. How do the human operators maintain situation awareness of the traffic when airspace and

routes are less structured in NextGen operations? Do they need a way to anchor their mental

models of the air traffic situations?

10. Are there situations where prospective memory is particularly important (e.g., remembering to

make a particular response at a future time)?

11. Are there situations that will tax working memory capacity (e.g., immediate recollection of highly

detailed instructions received by voice communication or from a visual display)?

C.4 Communication and Coordination

1. Is the coordination overly complex (e.g., too many players, cumbersome degree of back and forth,

distracting)? Do the coordination procedures create problems for the human operators (e.g., delay

in getting the attention of another person, explaining the problem, or getting agreement on what to

do)?

2. What impact do time constraints have on flexibility for coordination? How much time is allowed

for negotiation, if any? How many modification attempts are acceptable?

3. Who or what has final authority? How will an end to negotiations be determined?

4. How are the trajectory changes negotiated via data link? Do they need accompanying voice

communication to express the intent? If both modalities are needed, how are the procedures of

that interaction managed?

5. Is communications via data link, voice, or both? If data link is called for, can the concept cope with

the expected delay and the difficulty of sharing one’s intent via data link? Are there

circumstances where only voice or data link communication should be used?

6. Are the underlying technological assumptions for communications and information sharing

adequate? For example, can information be sent and received with the needed speed, appropriate

encryption, and via the assumed network?

7. Is the information that is shared among multiple operators appropriate regarding content, detail,

and type of format for each particular operator?

8. How important is participating in “party line” communications (i.e., a voice frequency shared

among several persons) as is used in current operations for situation awareness under NextGen?

Should there be analogous mechanism in data link communication?

9. How will visual attention demands of data link communication affect other tasks that require

visual attention?

10. How are the emergency responses coordinated between human operators?
75

C.5 Organization, Selection, Job Qualification, and Certification Factors

1. What will be organizational structure, interactions between organizational units, and delegation

of authority to units? For example under Traffic Flow Management, organizational relationships

will need to be assigned among national flow management, regional traffic management units,

and sector controllers. What organizational/cultural impediments must be overcome in the

transition to the concept?

2. What skills, abilities, and traits are needed by personnel (e.g., ability to use automation, and to

work more as a manager of operations that are handled mostly by automation with occasional

involvement by the operator)?

3. How difficult will it be for human operators to develop an understanding of the capabilities and

limitations of the computer-based models they will employ?

4. What additional training is needed for the human operators to perform the necessary operations

for the given concept? How do they train to understand the automation systems, including failure

detection?

5. How will the human operators train for the more diverse and dynamic airspace and route

structures that will exist in NextGen operations? Currently controllers rely on developing

expertise in a stable airspace that they work in over a period of years. And, how do they train for

varying, dynamic operations.

6. Should human operators train to maintain skills that are only needed when the automation fails?

How do human operators maintain skills that may only be needed for rare events – e.g., how do

operators train to monitor for conflicts if automation fails.

C.6 References

Lee, P. U., Mercer, J., & Callantine, T. (2007). Examining the moderating effect of workload on

controller task distribution. In Proceedings of the 7th International Conference, EPCE 2007,

Beijing, China, July 22-27.

Prevot, T., Homola, J., Mercer, J., Mainini, M., & Cabrall, C. (2009). Initial evaluation of NextGen

Air/Ground Operations with ground-based Automated Separation Assurance. Eighth

USA/Europe Air Traffic Management Research & Development Seminar ATM, June, Napa,

California, USA.

Sheridan, T. B., Corker, K. M., & Nadler, E. D. (2006). Report of a workshop on humanautomation interaction in NGATS. (DOT-VNTSC-NASA-06-02.)

Willems, B., & Koros, A. (2007). Advanced concept of the National Airspace System of 2015:

Human factors considerations for air traffic control (DOT/FAA/TC-TN07/21). Atlantic City

International Airport, NJ: Federal Aviation Administration William J. Hughes Technical

Center.
76
77

Appendix D: Separation Assurance

The Separation Assurance (SA) research aims to substantially increase airspace capacity (e.g., 2-3X)

by utilizing trajectory-based technologies and a human-automation operating concept that can safely

maintain separation assurance for the increased traffic. The main approach to increase capacity is to

reduce the controller workload bottleneck that results from monitoring aircraft separation by visual

and cognitive analysis of a traffic display and issuing control clearances to the pilots using voice

communication. Although advanced DSTs (e.g., data link trajectory-based advisory information on

conflict detection and resolution) have helped reduced the controller workload and increased the

sector capacity, a fundamental change in the way that separation assurance is provided seems

necessary in order to provide two- to three-fold increases in capacity.

The SA research is focused on developing automated separation assurance technology that can

generate robust automated resolution trajectories that are safe and efficient. In addition, the overall

human-system operations are evaluated by examining the roles and responsibilities of controllers,

pilots, and automation in different service-provider-based and operator-based concepts of operations.

Research is also needed to identify human and/or automation failures, such as missed conflict alerts

and data link failure, in order to design safe recovery modes.

Separation Assurance under NextGen can potentially be provided under several modes: Conflict

Detection and Resolution (CD&R) provided by controller as today (e.g., classic airspace); fully

automated ground-based CD&R, which we will call Automated Separation Assurance (ASA) (e.g.,

fully automated Advance Airspace Concept [AAC]); automated ground-based conflict detection and

ground-based automation recommending conflict resolutions to controller (e.g., AAC as a Decision

Support Tool); and self-separation (e.g., separation maintained by the pilots using flight deck

automation). Within these modes of SA, there could be mixed equipage where some aircraft are

equipped for full operation in an automated SA mode (either by ground and/or flight deck

automation) and other aircraft are not so equipped and would have SA provided by a controller.

In order to limit the overall scope, this report only examines fully automated, ground-based

Separation Assurance (ASA), with and without mixed equipage. ASA analyzed in this paper is

based to a large extent on AAC.

D.1 Overview Description of SA Covered in the Report

ASA is an automated, ground-based system for providing Conflict Detection and Resolution

(CD&R) capabilities. Human performance issues for the following applications of ASA are

analyzed in this report:

• segregated ASA airspace with only aircraft equipped for accepting and executing 4-D

trajectories that have been uplinked by ground-side automation in a sector. In this scenario, the

controller functions as an “ASA Manager.” With much of the CD&R functions handled by

automation, the ASA Manager can handle more aircraft that today’s controller.

• ASA CD&R capability exists for all aircraft in the sector but some aircraft do not have the

equipage to accept and fly 4-D trajectories that have been uplinked by ground-side automation,

creating a mixed equipage environment in the sector. ASA detects, resolves, and uplinks the

clearances to equipped aircraft, and a traditional Controller function provides CD&R to the

unequipped aircraft. Two cases of this mode are considered: (i) one person providing services
78

for both ASA and traditional controller functions for unequipped aircraft, and (ii) separate

persons providing services for ASA-equipped and for unequipped aircraft.

D.2 Summary of Required Technology for Ground-based ASA

The ground-based technology for fully automated ASA assumes that ASA provides routine CD&R

for aircraft without human ANSP. ASA also deals with routine pilot requests for routine changes

without an ANSP. In this concept, flight deck automation that can generate efficient conflict-free

trajectories may be sent as pilot requests that can be approved by the ground automation without

human ANSP involvement. A human ANSP will become involved if ASA cannot provide a conflictfree route to meet a pilot request and to deal with pop-up weather. ASA has a medium-term CD&R

capability that probes along the flight trajectories (e.g., between 3 to 12 minute timeframe) from

Loss of Separation (LoS), as well as an independent tactical conflict resolution system (e.g., Tactical

Separation Assisted Flight Environment, or TSAFE) that separates aircraft in the short timeframe

(e.g., less than 3 minutes) from LoS, An independent Traffic alert and Collision Avoidance System

(TCAS) provides the final layer of safety by advising coordinated collision avoidance maneuvers to

the flight deck.

Aircraft flying under ASA are assumed to have data link capability to uplink 4-D trajectories. The

aircraft can also downlink aircraft trajectory intent (e.g., data link and/or ADS-B). Aircraft are

expected to fly 4D trajectories within a certain Required Navigation Performance (RNP). Weather

information is expected to be available for both ground and air via common weather database.

D.3 Summary of Human Roles for ASA

Three human roles are described in this section on ASA:

1. ASA Manager: provides support for equipped aircraft flying under ASA

2. Controller: manages and provides separation for aircraft that cannot fly under ASA. In this

operational context, Controller is only involved in mixed equipage airspace.

3. Pilot: executes clearances issued by either the automation or a human operator. When

necessary pilot coordinates the maneuvers with an ASA Manager or a Controller. Pilot can

also request re-routes.

The roles and responsibilities assumed for these three humans are listed in the following subsections.

D.3.1 ASA Manager Roles and Responsibilities for ASA

The assumed ASA Manager roles and responsibilities are as follows:

1. Respond to pilot requests for re-routes that cannot be handled by the ground-side automation (e.g.,

provide conflict-free requested re-routes).

Normal pilot requests for re-routes can be handled by the ground-side automation which checks

the routes for conflicts and other potential problems and issues them as route clearances if the

routes are problem-free. If the automation cannot properly process the requests or rejects them

erroneously, the pilot can coordinate with the ASA Manager to plan the trajectory changes.
79

2. Resolve conflicting pilot requests.

If the automation receives multiple pilot requests that conflict with each other and cannot arrive at

a reasonable solution that satisfies both pilots, the pilots can coordinate with the ASA Manager to

plan a coordinated set of trajectory changes.

3. Monitor aircraft in short-term conflicts and return aircraft onto path to its original destination if

necessary following operation of short-term conflict resolver (e.g., TSAFE).

TSAFE will resolve short-term conflicts (e.g., less than 3 minutes prior to LoS). TSAFE

resolution is a simple tactical maneuver to prevent an immediate loss of separation which

involves a simple heading maneuver to keep the conflicted aircraft on divergent paths. This

maneuver may put an aircraft off its trajectory and the ASA Manager may need to return the

aircraft on its original path trajectory if the automation cannot. Workload permitting, the ASA

Manager may also work in conjunction with TSAFE to ensure separation between aircraft (e.g.,

TSAFE issues a heading change for lateral separation while the ASA Manager issues a vertical

clearance to keep vertical separation as well).

4. Monitoring and control of separation of equipped aircraft that have reverted to unequipped status

because of on-board equipment failure or other reasons.

The ASA Manager may need to step in and provide separation assurance for aircraft that changed

to unequipped status. S/he may need to determine if the aircraft can continue to its original

destination or must land quickly. If the aircraft can be returned to equipped status, the ASA

Manager can take control of the aircraft until s/he can safely put it back into equipped status.

5. Assist pilots in handling emergencies and other abnormal situations.

Possible situations include medical emergencies, failure of critical equipment on aircraft, security

situations requiring aircraft to land immediately, etc. The aircraft will need to notify the ASA

Manager who may need to take control and maintain safety of the aircraft. The DST support

needed to manage such an aircraft in the ASA environment needs to be determined.

6. Other potential functions for the ASA Manager include: 1) monitoring local movement of weather

for pop-up cells, Special Use Airspace (SUA), and other capacity limiting factors; and 2) taking

immediate actions for local traffic disturbances and/or coordinating a joint plan of action with

other ANSP operators manage problems.

Additional tasks for ASA Manager under mixed equipage with a Controller handling unequipped

aircraft in the same airspace

7. The ASA Manager coordinates with the Controller (who manages unequipped aircraft) as needed

when planning reroutes for equipped aircraft. This will occur when the ASA Manager is

responding to pop-up weather, pilot requests for re-routes that cannot be handled by the groundside automation, and conflicting pilot requests.

8. The ASA Manager coordinates with the Controllers as needed when assisting equipped pilots in

handling emergencies, failed ASA equipment or other abnormal situations. If an equipped

aircraft changes its status to unequipped, the ASA Manager may need to transfer control of the

aircraft to the Controller.
80

Additional tasks for ASA Manager under mixed equipage with ASA Manager handling both

equipped and unequipped aircraft (ASA Manager acts as a Controller for unequipped aircraft)

9. ASA will conduct tasks listed above under Items 1 through 6 and the tasks listed below for

Controllers under items 1 through 3.

D.3.2 Controller Roles and Responsibilities for Unequipped Aircraft Flying in Same Airspace

as ASA-equipped Aircraft

The Controller is involved in ASA only under mixed equipage when the Controller is responsible for

CD&R and other functions for unequipped aircraft. The following are roles for the Controller under

mixed equipage. These roles are similar to today’s controller except when interacting with the ASA

Manager and being aware of equipped aircraft flying under ASA as needed.

1. The Controller has normal responsibilities such as handoffs, separation management, and pilot

requests of unequipped aircraft. Conflict detection and resolution are aided by advanced DSTs.

Transfer-of-communication is automated. Pilot check-ins may be omitted.

2. When conflicts occur between an ASA-equipped and an unequipped aircraft, the Controller is

responsible to move unequipped aircraft whenever possible.

3. The Controller keeps unequipped aircraft on their 4-D trajectories by issuing voice clearances to

the pilots who then update their FMS to fly along the trajectories.

Keeping an unequipped aircraft along its 4-D trajectory is key in allowing the automation to

detect the unequipped aircraft in a mixed-equipage airspace. The Controller is likely to need

advanced DSTs and procedures to keep unequipped aircraft on their trajectories while executing

lateral and vertical maneuvers. Procedures and tools are needed to absorb delays and avoid

weather cells and/or SUAs.

4. Issue a safety alert to an equipped aircraft if the controller is aware that the equipped aircraft is in

a position/altitude that, in his/her judgment, places it in unsafe proximity to terrain, obstructions,

or other aircraft.

5. Assist pilots of unequipped aircraft to handle emergencies and other abnormal situations in the

midst of potentially dense ASA-equipped traffic.

D.3.3 Pilot Roles and Responsibilities for ASA

The roles of pilots are first presented for dedicated ASA airspace with fully equipped aircraft,

followed by pilot tasks in mixed equipage:

Tasks for pilots flying ASA equipped aircraft in dedicated ASA airspace

1. There will likely be no or little role for pilots during handoffs from an ASA airspace to another.

Future transfer-of-communication technology (TOC) will likely allow the ground system to

automatically uplink the new frequency to the cockpit, which may be uploaded to flight deck

radio automatically or manually entered by the flight crew.
81

2. Request re-routes (e.g., around weather, efficient user-preferred routes, etc.).

For strategic re-route requests with route changes into the future (e.g., greater than 20 minutes),

the requests are likely to be coordinated with TMU, AOC, or both. Once the coordination is done,

the actual request from the pilot needs to go either to the ASA automation and/or ASA Manager

for review. If ASA automation finds the request is infeasible (e.g., ASA automation cannot

resolve possible conflicts), the ASA Manager will become involved. This may involve

negotiations between the pilot and the ASA Manager; there is likely to be a negotiation protocol

between the pilot and ASA Manager.

3. Work with ASA Manager during short-term conflict situations.

During short-term conflict situations, the ASA Manager may need to intervene to ensure safety

and/or to return the aircraft to its original path. In short-term conflict situations, TSAFE normally

resolves conflicts and send the resolution via data link. The pilot follows TSAFE instructions to

avoid the near-term conflict but the pilot also needs to be available to the ASA Manager who may

want to coordinate further for additional clearances or to return the aircraft to its original route.

The pilot will work with the ASA Manager closely in this safety-critical situation.

4. Work with ASA Manager as needed to handle emergencies and other abnormal situations.

Tasks for pilots flying unequipped aircraft in mixed equipage airspace

1. Participate in handoffs, including change of frequency for communications. Perform all other

functions under normal Controller-managed airspace.

2. Execute trajectory changes specified by Controller.

Receive maneuver intended to keep the aircraft on its 4-D trajectories from Controller via voice

communication, review maneuver, communicate acceptance to Controller of maneuver via voice

communication, and execute maneuver.

3. Work with Controller as needed to handle emergencies and other abnormal situations.

D.4 Human Factors for ASA

In this paper we take two perspectives to identify and characterize human performance issues for the

individual Research Focus Areas: one is the perspective of the human operators performing the

required tasks and the other the perspective is what impact the tasks have on the human operators in

the operational context. Section D.4 addresses the perspective of the human by examining the 9

factors of human performance presented in Chapter 2 (e.g., attention allocation, memory, workload,

etc.). Section D.5 will take the perspective of the tasks and the resulting human performance issues

as described in Section 4.2.

The human tasks in Section D.3 were examined for the ability of humans to perform the tasks, both

from a safety viewpoint and from the perspective of performing the tasks in a manner that the

benefits of ASA would be achieved. The human performance issues identified are described below

for the ASA Manager, who works with ASA-equipped aircraft; sector controller, who works with

aircraft that are not equipped as in today’s classic airspace; and the pilot flying either ASA-equipped

aircraft or unequipped aircraft.
82

D.4.1 Attention Allocation

Attention allocation, situation awareness, and monitoring will be discussed for the ASA Manager

first, and then for any differences that result from mixed equipage. The ASA Manager monitors

traffic flows on a larger scale for the designated airspace, as well as the upstream region for weather

and changes in traffic to conform to weather. The ASA Manager is not involved in CD&R

performed by ASA. However, if a separation violation occurs where TSAFE and/or TCAS acts, the

ASA Manger may work to return the aircraft to its original trajectory. There are likely to be

situation awareness issues for the ASA Manger to gain awareness of the location of other traffic

when returning the aircraft to its original trajectory.

It is noted that under mixed equipage, the ASA Manager needs to coordinate with the Controller in

some situations, such as emergencies. Human factors questions related to the necessary

coordination and situational awareness for the ASA Manager and the Controller need to be

addressed.

Under mixed equipage, the Controller monitors potential separation violations of unequipped aircraft

with all traffic. S/he must also be generally aware of traffic flows in the sector being monitored, for

example that aircraft are not deviating from assigned trajectories and meeting metering constraints.

Visual scanning must occur more often when there are weather constraints. These are the same

tasks that the Controllers perform today. Monitoring and maintaining proper situation awareness of

equipped aircraft is necessary to safely maneuver the unequipped aircraft, which poses an interesting

human factors challenge on how best to allocate the Controller’s attention effectively.

The pilot of an ASA-equipped aircraft should have a general awareness about which frequency that

s/he is on, what type of traffic is around him/her (e.g., by observing the CDTI), and who s/he should

contact in case of an emergency. If a separation violation occurs and a resolution is uplinked from

the ground the pilot must execute and monitor the aircraft avoidance maneuver, and subsequently

follow the commands for getting back into the traffic flow. If ground automation asks for a

trajectory modification due to weather, the pilot monitors that the aircraft complies (unless the pilot

rejects and a negotiation with the ground ensues). Protocols for negotiating with ground automation

will need to be developed, as will protocols for switching from negotiations with ground automation

to with the ASA Manager when the ground automation cannot resolve the negotiations.

The human factors for the pilot flying an unequipped aircraft in mixed equipage airspace are similar

to those today.

D.4.2 Decision-making and Mental Modeling

When the ASA Manager needs to facilitate a short-term conflict resolution or to accommodate pilot

requests, a quick decision is needed once the awareness of the traffic situation is gained. A good

understanding of the traffic and the actions of the automation will be a prerequisite for sound

decision-making. Frequent occurrence of the situation and the actions by the ASA Manager will be

needed for the decision to be skill-based (i.e., can be done without conscious decisions).

The ASA Manager should also anticipate when current trajectories cannot be sustained due to

unpredictable weather and use some decision criterion for when to intervene. S/he may also

participate in making rerouting decisions for major weather constraints. S/he should also monitor the
83

traffic level in his/her airspace and request from the TMU a lower traffic level for her/his designated

airspace if s/he decides it is necessary.

The Controller needs to update his/her decision-making skills to match the requirements of NextGen

operations (e.g., TBO airspace, data link equipage, etc.). In a mixed ASA airspace, s/he needs to also

develop skill-based decision-making capability related to mixed conflict situations between

equipped and unequipped aircraft. The Controller needs a good understanding of the automation and

its actions under such situations.

With respect to ground-based automatic detection and resolution of either separation conflicts or

weather obstructions to the planned trajectory, the pilot must decide whether to accept the resolution

or suggest another resolution. If the latter, the ASA Manager controller must evaluate the pilot’s

request and decide whether to accept it. The same decision responsibilities would pertain if an

aircraft deviates from the planned trajectory.

D.4.3 Communication

The ASA Manager’s imposition of changes in trajectory because of weather or need to resolve

conflicting pilot requests would normally be up-linked to the aircraft and communicated to the pilot

via digital communication. If necessary, either party can resort to audio voice communication, e.g.,

if data communication fails of if interchange between people is urgent. (The latter is a significant

reservation about data-link communication.) Such changes also need to be communicated to salient

Controllers if there is mixed equipage.

For the Controller dealing with unequipped aircraft under mixed equipage, communication between

Controller and pilot regarding separation conflicts, coping with weather constraints, or deviations

from planned trajectories will be via voice if data-link is not available.

Any system outage or failure would be broadcast via data-link to all parties who are involved

currently or will be in the near future.

The ideal of cooperative air traffic management is still to be defined, but generally will require

communication inter- and intra-party by voice or data-link. Conventional telephone communication

may be used when other communication means are busy or there is no urgency.

There is planned NextGen communication network for System-Wide Information Management

(SWIM) that will be available by data-link. However particular uses of this network have not been

determined, for example when such information will be presented involuntarily (i.e., “pushed”),

require voluntary request (i.e., “pulled”), or be available to only certain parties on a restricted basis.

D.4.4 Memory

Because the ASA Managers dealing with equipped aircraft have a longer horizon than the today’s en

route controller for conducting strategic planning, there will naturally be a need for them to

remember properties of a situation previously planned.

In coping with multiple aircraft for weather rerouting or conformance to traffic flow commands,

ASA Managers, and the Controllers dealing with unequipped aircraft under mixed equipage, have to
84

recall which aircraft is which (retroactive memory). Tag (flight object) information should be

available to reinforce correct recall. If ASA Managers and Controllers need to cut off interaction

with one aircraft to attend to some more pressing need, they require prospective memory to get back

to and complete the earlier interaction.

Getting reroute information via data-link should relieve pilots of many memory problems that arise

when receiving information via voice communication and having to retain that information. The

latter problems have been evident in current ATM operations. The ASA concept should not pose

additional memory burdens on the pilot.

D.4.5 Workload

During normal operations in less busy times, the ASA Manager should have diminished workload in

terms of active involvement, and therefore it may be difficult to sustain the attention required for

monitoring. For this reason the ASA Manager could possibly become inattentive due to underload.

Mostly, if traffic increases as expected in NextGen and sectors are made larger, etc. the ASA

Manager workload may be the same or only slightly different when all operations are normal - the

automation will be taking over much of the work and prompt the ASA Manager when critical events

need attention. In off-normal situations the workload may increase substantially if there are multiple

simultaneous demands on the ASA Manager’s attention: to negotiate and ensure proper modification

of trajectories for weather, resolving conflicting pilot requests, returning an aircraft to its original

trajectory after the activation of TSAFE, interaction (and confusion) with the automation, or some

combination of these. The ASA Manager’s workload will change in nature from today’s controller,

with less telephone conferencing and much greater use of computer models.

The tasks of the Controller in dealing with unequipped aircraft under mixed equipage are similar to

those of today’s controllers, and workload due to tasks should be similar to today. The extent of the

Controller’s workload will depend on the amount of unequipped aircraft for which the Controller is

responsible.

The pilot in ASA-equipped aircraft should see a reduction in workload having to do with trajectory

control, primarily due to automation, but a compensating increase due to increased traffic.

D.4.6 Interaction with Automation, Decision Support Tools, and Displays

Implementation of ASA will cause interaction with automation at a level much greater than in

today’s NAS. This is the case for the ASA Manager, the Controller under mixed equipage, and the

pilot.

ASA automation performs most routine CD&R for equipped aircraft. The ASA Manager has

automation available to help monitor and make decisions for events for which they are responsible,

such as traffic levels in their designated airspace and pop-up weather. Further, there is a need for

specialized automation for executing “what would happen if....” models that take current traffic

situations as initial conditions and project ahead under various assumed conditions.

The Controller handling unequipped aircraft in mixed equipage has automation that detects conflicts,

suggests resolutions, facilitates communication with the pilot, and enables the proposed rerouting to

be uploaded directly to the aircraft FMS for aircraft equipped with data link. Controller automation
85

should detect and display weather patterns, make recommendations for rerouting traffic around the

weather, and facilitate negotiation.

ASA sends trajectories to FMS on ASA equipped aircraft via data link. The pilot can evaluate and

communicate acceptance, or rejection, to ASA. The short-term conflict resolution system, such as

TSAFE, is an integral part of the ASA concept. Flight deck automation may also display weather

information in a highly refined “go/no-go” form (not probabilistic) and indicate deviation from

assigned trajectory.

A major HF issue regarding automation design is the appropriate level of automation, where

different levels may be characterized on an ordinal scale with categories such as:

1. A system limited to giving the human information without ability to exercise actual control

(lowest level of automation, probably characteristic of SWIM).

• Discrete alarm/alert?

• Alarm/alert with explanation of why?

• Alarm/alert with explanation of why and advice as to how to resolve the problem?

• Unsolicited advice automatically linked not of immediate need but linked to phase of

flight or similar trigger?

• Solicited advice from SWIM?

2. A system that initially gives advice (one of above) but can be requested to provide further

advice, such as how to take control action (e.g., characteristic of future weather displays).

3. A system that initially gives advice (one of above) but can also execute control actions on

demand.

4. A system that can be asked by pilot or ANSP to execute control actions on his/her (own

programmed) demand (e.g., autopilot, flap setting, etc.).

• Execute now

• Execute at some future time, programmed with given if/then rules by pilot or ANSP

5. A system programmed pre-flight that executes control actions without human intervention

unless modified by some authority (e.g., engine controls).

• Advises pilot/ANSP what it is doing or has done

• Does not advise pilot/ANSP what it is doing or has done

For any of the above levels, automation designers should consider human usability criteria such as

whether the:

• display is easily visible or audible

• appropriate sensory mode is employed

• appropriate degree of redundancy is used in the displays

• display is appropriately either analog or symbolic or a combination

• display-control interaction is natural (ecological “affordance”)
86

• automation is immune to mode error (e.g., where there are multiple modes and user can

inadvertently expect operation in one mode when it is set to another mode)

• automation does not produce false expectations (what is it doing now, what will it do next?)

• intervention and reset are easily performed as necessary, but with proper guards

• automation is transparent (versus opaque) as to what it is doing

• it minimizes both transient and sustained workload

• it promotes situation awareness

• it is relatively immune to distraction

• it engenders an appropriate level of trust

D.4.7 Organizational Factors

It is imperative that roles and responsibilities of the ASA Manager, Controllers, pilot be clear. The

NextGen Concept of Operations emphasizes cooperative decision-making, but one person should be

designated to have final authority under normal conditions. For very abnormal and/or emergency

conditions, of course, the pilot has final authority.

There must also be clear lines of accountability and reporting for safety-related events and decisions

taken. Currently ATM can be characterized by extensive under-reporting, in terms of incomplete

reports and events that should, under existing rules, be reported but are not. Those that are reported

often have so many critical details omitted that the reports are not useful for subsequent safety

analysis. ASA, combined with ADS-B and weather surveillance and digital communication, will

allow for much data to be automatically measured and reported that could obviate the need for much

subjective reporting by ANSP.

Procedures should be readily available, either on the SWIM network or the electronic flight bag. It

is expected that routine procedures will be well learned through practice, but that is not true of

procedures for off-normal events. Experience from nuclear power, military, and hospital contexts

suggests that the presentation of procedures must be professionally human-factored, with text that is

simple and straightforward accompanied by appropriate graphics, and vetted by the prospective

users for clarity and understandability.

Managers need to be supportive of safety culture, communicating to subordinates that they are

serious about preparedness for unexpected events (see Section D.4.9 below).

D.4.8 Potential Errors and Recovery from Human Errors or System Failures

Anticipated human errors in ASA operations include the following examples:

• The ASA Manager or the TMU fails to scan for ongoing critical traffic management events,

and calls for a new separation maneuver that is in conflict with an ASA conflict resolution that

has just been accepted by one or more pilots.

• The pilot disagrees with the ASA’s recommended SA resolution and delays in renegotiating the

trajectory, or fails to respond to the uploaded revised trajectory.

• The ASA Manager is heavily loaded in attending to one aircraft and delays giving attention to a

second aircraft in a time-critical ASA separation resolution.
87

A new approach to safety emphasizes resilience of the overall system. For ASA this implies an

automatic means to monitor that all the NAS facilities are functioning normally, and if not to ensure

that the salient personnel are notified immediately. It also implies that errors or critical situations

such as those cited above be anticipated by system designers and recovery procedures be developed.

D.4.9 Potential Selection, Certification, and Training

Human interaction with ASA typifies the need for new criteria for selection of controller personnel.

The candidates need to exhibit experience with, understanding of, or propensity to work comfortably

with, automation and computer-based decision support tools.

As such tools are developed, part-task simulations must be performed that include not only normal

but also off-normal events. The results, mostly subjective and anecdotal at the early stages, should

be fed into requirements for software and hardware interfaces, especially including design of

controller displays.

These same simulations, or a higher fidelity version of the same, might be used as part of the

certification process for decision support tools.

It is important that trainers engender in ANSP the right amount of trust in the ASA automation they

will come to depend upon. Too much trust would be manifest in controllers expecting the automation

to be cleverer than it actually is, such as having an ability to resolve conflicts that involve more

factors than were ever anticipated by the programmers. Too little trust would be manifest in ASA

Managers ignoring the ASA detection/resolution capability and trying to handle too many conflict

resolutions by their own observation, intuition and manual vectoring using voice radio.

D.5 Human Operations Issues, Design Issues, and Countermeasures for ASA

Section D.4 discussed Human Factor issues for ASA. This subsection takes the knowledge gained

from assembling these human factor issues and looks at the list of questions presented in Appendix

C with respect to SA. As discussed in Section 3.3 and Appendix C, these questions are addressed in

the context of the ASA concept at its current level of concept development (at the time of writing).

Not all of these questions will be design issues for ASA. However, each question will be covered

and comments will be made regarding how the question relates to ASA. Again as discussed in

Section 3.3 and Appendix C, a priority rating high, medium, and low has been assigned to each

question. These sections also discussed the criteria behind assigning a priority rating.

D.5.1 Use of Automation and DSTs

1. What is the distribution of roles and responsibilities across different human operators and

automation? Will the human and automation “trade” control and responsibility of the tasks?

What are the mechanisms for switching tasks between them?

Functions are not clearly delineated by time or procedures in the ASA concept. Two agents

(automation and human) can potentially work on a problem at the same time. Two levels of

automation will work on a problem for different time periods (e.g., ASA for a number of minutes

before TSAFE is activated at around three minutes) and then the ASA Manger is expected to

step in if the automation is unable to resolve the problem for the equipped aircraft. The ASA

Manager is also expected to step in when ASA cannot provide a conflict reroute in response to a
88

pilot request. In a mixed equipage environment, functions are further complicated by the

presence of controller-managed unequipped aircraft.

The timing and means of the transfer from automation to the operator is not defined. Intent of

what the automation is attempting in solving a problem is not communicated between

automation and human during transfer from automation to human; although, because the ASA

Manager has to check their solutions with ASA automation, intent is communicated from human

to automation. Identifying automation intent and having clear procedures for the human

stepping in needs to be provided. However, providing intent and transfer of function would

seem to be relatively easily handled with a display indication driven by explicit automation rules.

Clearly delineated functions are needed for the ANSP who is responsible for aircraft that do not

have the requisite level of equipage, and for those that report problems (off-nominal events).

Under ASA, the automation and the human will trade control and responsibility of some tasks

but it is not an equal trade. For example, TSAFE can prevent LoS then indicates the diverted

aircraft to the ASA Manager who is expected to re-place that aircraft back on its trajectory.

TSAFE indicates this shift of control by highlighting the datablock of the aircraft concerned.

The ASA Manager takes these tasks and any other impending LoS that they judge they need to

step in on. Their intervention is not explicitly indicated on the display. The automation does not

relinquish tasks but accepts inputs from the ASA Manager until the problem is solved.

The ASA tools and the human operator work very differently. The ASA tools complete pairwise comparisons of the trajectories of the aircraft in proximity, the human looks at the patterns

of the aircraft moving in their sector.

Priority: High

2. Who has decision making authority when there are conflicting decisions or advice given by some

combination of automation and other operators? Is there a clear trigger event for any one

operator to decide and act?

Since the ASA Manager is to step in if the automation cannot solve a situation, it is assumed that

the ASA Manager will break a tie between any conflicting solutions among automation and

operator. As the operator knows the operating rules of the automation, s/he is generally able to

generate solutions that do not conflict with those that may be issued by the automation. As noted

above, there are no clear triggers for the operator to step in and timing is often an issue.

However, it should not be difficult to define decision-making authority.

Priority: Medium to High

3. Is automation allowed to execute tasks without the human operator in-the-loop or is automation

providing DST functions with operator in-the-loop?

The ASA concept depends on automation executing most tasks for CD&R without the ASA

Manager in the loop. However, the automation is not aware whether the human is in the loop or

not and acts without regard or consideration of human actions. At times a DST aids the ASA

Manager to make decisions and take action, such as acting on a user request that the ASA

automation cannot handle. The ASA Managers can “jump in” whenever they want to, but all

specific conditions as to when they need to are not clearly defined. Accounting for operator

abilities and the need to get “up to speed” on a problem, human operators may need more than a

3 minutes warning if they are required to take action. Thus, a trigger or cue event needs to occur
89

at this point, not later. How the ASA Manager will acquire, and how long it will take to acquire,

awareness needs to be studied.

Priority: High

4. Are there any cases where the human operator will need to become involved, such as when there

is an exception to nominal situations? If so, how and when does the human operator know there

is an exception and s/he needs to act?

If automation wants a human to manage a LoS it needs to indicate this to the ASA Manager in

some way. The current development of TSAFE does not make this indication, the ASA

Manager is expected to watch the resolution process and step in when they judge that TSAFE is

not going to prevent the LoS. To do this, the ASA Manager would likely prefer enough warning

time (e.g., 4–5 minutes) but can probably work with less. TSAFE needs to flag the conflicted

aircraft concerned for the ASA Manager. The standard data block information should be enough

for the ASA Manager to work with, although they would feel better informed if they have some

other information as well, such as the aircraft flight path history, and the flight origin and

destination.

Priority: Medium to High

5. What is the proper degree of trust by humans in the automation? How much reliance on the

automation is necessary for the concept to work?

As the ASA Manager is out of the loop for routine separation assurance, the automation has to

function as intended (correctly) since there is no way the ASA Manager can independently

monitor traffic even if s/he did not trust the automation. Automation working properly and

clearly marking when a failure has occurred is critical for the ASA concept.

Priority: Medium to High

6. Do human operators have the appropriate amount of information on their displays to do their

jobs successfully within the time constraints that they have?

For ASA Managers to become involved, when ASA cannot resolve conflicting pilot requests and

under other circumstances, they will need to know the path, speed and altitude of the aircraft

involved. These three items of information should be easy to supply along with the route or

trajectory line that can be brought up through the data block’s icons. Additional information

(like destination) could be helpful, (this used to be an entry on the data block) and should be

easily obtained in ASA. Time for the ASA Manager to absorb this information is a concern.

Priority: Medium to High

7. If automation/DSTs fails, will human operators realize it and recover?

The range of failures needs to be defined, from a power outage to automation not calculating the

algorithms to automated separation not being provided. Since Separation Assurance is to be

automated, a failure in the computer operating to separate aircraft or providing an erroneous

separation result is a concern. Confidence of the ASA Manager in the ASA automation, and that

it is working properly, needs study as well as how the ASA Manager will know the ASA

automation is not operating properly. In these cases, the display should clearly indicate the

length of time to LoS (or conflict) and make the data block flash and turn red. These indications
90

or alerts are only displayed when time to LoS is less than three minutes. The ASA Manager

would be able to step in to take action, but anything less than three minutes warning is too short

a-time to expect them to do anything other than prevent a conflict.

Priority: High

8. Will the DSTs recognize a problem and help in emergency situations? Are the DSTs smart enough

to indicate when an unusual situation is beyond their ability to be useful and should not be relied

upon?

At the current level of investigation of the ASA concept, the tools have been presented with offnominal situations and their responses charted but they were not programmed to respond any

differently to emergency situations. In the few observed studies, the tools flagged the aircraft as

off-nominal (either off course or not on a 4D trajectory and it was the operators’ responsibility to

recognize, investigate and solve these problems.

Currently ASA tools do not have any ‘self awareness’, nor any awareness of the wider context.

They will issue a resolution, even if in the broader view this resolution is not the right solution.

In the initial AOL instantiation of ASA (Prevot, et al., 2009), TSAFE could become stuck in a

loop of continually reissuing updated resolutions if aircraft failed to respond or the situation was

changing rapidly (due, usually to the proximity of other aircraft).

Priority: Medium to High

D.5.2. Human Decision-making, Workload, and Error Recovery

1. Does a single human operator handle multiple operations?

There seems to be a tradeoff with sector size and number of functions that an ASA Manager can

handle. An ASA Manager can handle more functions in a smaller designated airspace of

responsibility, or fewer functions over a larger designated airspace. It is assumed that in

furthering the design of ASA the size of a sector that an ASA Manager oversees will be studied.

In the mixed equipage case, one Controller deals with a number of tasks similar to today’s

controllers. Sector size for a Controller under mixed equipage will need study.

Priority: Medium

2. Is there complex mixed equipage? Can the combined human operators and automation handle

the complex airspace?

The concern here is the specific interactions needed between ASA Managers and Controllers

under mixed equipage, and these will need to be studied and procedures formulated to handle the

interactions. Issues of determining airspace unit size for ASA Managers and Controllers are

mentioned in Item 1 above.

Priority: Medium to High

3. Is there a sufficient timeframe for decision-making for critical situations, especially off-normal

situations?

The timeframe of decision-making under ASA varies but is short. If the controller is busy with

other problems or TSAFE activates late, the ASA Manager has little time to react. A short

timeframe may not be large enough. Time is inversely linked to ease of decision-making. As
91

time reduces, the solution options also reduce, but rather than making things easier (less to

choose from) this makes the problem more complex if the easy straightforward solution options

are the first to drop away. However, if the human operator is not responsible for the maneuvers

made by automation, they may be able to assist in emergency situations to lessen their impact.

Timeframe for decision-making needs study.

Priority: High

4. Can the human operator attend to multiple alternatives in support of optimization? How many

options should be presented?

Yes. In the current day, the Controller is able to consider a range of solutions for each conflict.

The number of viable solutions varies depending on the particular situation. If solutions are

presented to the Controller to choose from, the absolute number that should be presented needs

to be determined by research.

Priority: Low

5. Are there peak workload situations that should be moderated? What metrics are appropriate?

Peak workload occurs when there are multiple conflicts that require TSAFE to step in. At this

point, the controller is trying to monitor, assess and intervene if necessary in multiple solutions.

Metrics to measure this peak could be a complexity value that counts the number of short-term

conflicts in a sector.

Priority: Medium to High

6. Can human operators perform all the functions assigned to them? If not, can the function be

distributed across multiple operators or can they be offloaded to the automation?

Under tests conducted on the AAC instantiation of the ASA concepts, the AAC Managers were

able to perform all their required separation assurance functions with 2x traffic in the sector sizes

tested (Prevot, et al., 2009). However, while the relationship between additional functions and

sector size/traffic levels has not yet been tested, it is clear that the balance has to be less of one if

there is more of the other (i.e., if sector size and therefore traffic count is large, ASA Manager

functions have to be more limited; if sector size is smaller with fewer aircraft to manage, ASA

Managers can take on more functions, like weather rerouting, as they do today).

Priority: Medium

7. What is the workload impact when there is transitioning between airspace configurations or

different modes of operations (e.g., from CDA to FDMS)?

The main issue for human operators here is likely to be any transition between dedicated ASA

airspace and mixed equipage airspace.

Priority: Low
92

8. If there are mixed surveillance sources with different precisions, can the human operators utilize

the more precise information available for some of the aircraft and the less precise surveillance

information on other aircraft?

In one subjective study (see Appendix E), Controllers preferred to move the aircraft for which

they had the most information because they could be most certain about the effects of their

solutions. A similar result was seen in another study with mixed equipage (Kopardekar, et al.,

2009). However, neither study assumed different separation standards based on the equipage and

had significant automation help for conflict detection. If different separation standards are

needed based on the surveillance source, there will likely be significant challenges.

Priority: Medium

9. Are there appropriate layers of human operator redundancies built into the concept so that some

operators monitor others to anticipate if help is needed?

Who will monitor the ASA Manager (for workload, stress, etc.) is unclear. However, it is

possible that future machines would be able to monitor the operator for signs of stress or

workload in addition to their main functions. How the ASA Supervisor fits into the larger

organization has not been defined. The chain for monitoring a separation assurance problem and

handing it off to a different agent to solve is more clear (i.e., ASA TSAFE human ANSP

TCAS human pilot).

Priority: High

10. In responding to emergencies or recovering from error, are there standard responses generated

across different modes of operation or does it need to be different for each mode? If different is

the operator likely to be confused about which mode is appropriate?

ASA does not respond to emergencies any differently than normal situations (see D.5.1.7.). This

means that the automation responds to all situations in the same way with no consideration of the

context. Although this would seem to reduce mode confusion, actually it could increase it, as

operators have no way to distinguish emergencies from standard conflict resolutions and have to

make this assessment for themselves, often based on information they have to search more

widely, or proactively, to gather.

Priority: Medium to High

11. How brittle is the concept? How flexibly can the operations adapt to off-normal situations? How

easy is it for the concept to fall apart under off nominal situations, and can any such situations be

anticipated? How are fail-soft modes and resilience built into the system?

ASA has back-up modes (TSAFE) and an airborne last defense (TCAS), as well as the ASA

Manager, to make it robust. However, the automation doesn’t make the best use of its back-up

human aid when it shows the problem to the operator with little time for the human to act.

Operations can be more flexible than the automation allows them to be if the ASA Manager is

able to reintroduce some this flexibility.

Priority: High
93

D.5.3 Attention, Situation Awareness, and Memory

1. Are there any excessive demands on attention?

This is a question for ongoing, detailed HITL research. Observation of different situations may

be the only way to answer this question.

In discussions of the ASA concept, Controllers predicted they would have no problem with

resolving one or two situations at a time. Controllers reported that a complex conflict involving

six aircraft would consume all of their attention, but this may be a reasonable thing (not

excessive) given the complexity of the situation.

Priority: Medium

2. What is the time effort to gain situational awareness to perform a normal task?

For normal tasks, the ASA tools will prevent LoS, and the ASA Manager will have five minutes

of guaranteed conflict-free time to return any aircraft to its 4D trajectory. However, a controller

who takes minutes to identify a normal situation is likely to take too long to recognize nonnormal situations that require far quicker (and more complex) responses. Although this should

be researched, it is likely that future operators will need to gain situation awareness as quickly as

current day Controllers – taking only a few seconds to recognize a situation if they have been onposition for a while.

Priority: Medium to High

3. If the human operator is passively monitoring and suddenly there is a need to get back into the

loop, how long does it take to do this? Is there enough time to do this for the given concept? What

is the workload involved? Is it a feasible amount workload?

With ASA automatically conducting most of the separation assurance, there may be periods

when the ASA Manager has little to do. During such extended periods of low workload, an ASA

Manager may be required to deal with the activation of TSAFE or some other situation. Several

aspects of such circumstances will need study: type of information the ASA Manager will need

to deal with the situation, such as gaining appropriate situational awareness; how long the ASA

Manager will have to deal with the situation; how the ASA Manager will interact with any

DSTs; and other factors for the ASA Manager to successfully deal with the situation.

Priority: Medium to High

4. Can automation complement or replace human monitoring function by calling attention to salient

and critical events?

Automation can call attention to some events such as conflict aircraft pairs and aircraft that are

off-trajectory. In the current development, TSAFE indicates the most recent resolution that it has

sent to the aircraft and there is a table of data link messages that are awaiting action. So, the

current situation is clearly displayed. However, other safety-critical events, such as aircraft

equipage failure or delay in flight deck execution of clearances may be harder to call attention in

a timely manner.

Priority: High (safety critical)
94

5. Are there recognizable indicators of emergency and critical safety events? Are there situations

where alertness of human operator to new information and indicators may be a problem?

Depending on the sector, there can be areas where aircraft are more likely to conflict, such as

where a departure flow is climbing into the en route sector.

One normal trigger event is a predicted LoS. Emergency trigger events vary. Attention should

be given to having as much standardization as possible in indicating and responding to

emergences under different circumstance and modes.

Priority: Medium to High

6. Are the modes of operations always clear to the human operators in both nominal and offnominal situations?

When the ASA Manager is dealing solely with equipped aircraft, this will not likely be a

problem. Nor will it be a problem for Controllers only dealing with unequipped aircraft flying

under traditional ATC control. However, if an ASA Manager is handling both equipped and

unequipped aircraft in mixed equipage there will need to be a clear distinction for the two types

of aircraft if they are flying under different rules.

When there is a problem of equipment failure, some indication for the ASA Manager and

Controller would provide a reminder and prevent confusion. The ASA system should be

transparent to pilots in that they do not need to know who they are talking to or even what mode

they are operating under as this would create the opportunity for mode errors. (If the aircraft are

operating under self-separation, then pilots do need to know the mode they are in.)

Priority: Medium

7. Are there situations where fixation and tunneling may be a problem?

This is a question for ongoing, detailed HITL research.

Controllers reported that, in today’s NAS, they can become focused on particularly complex

problems. In the current day, they would rely on their Supervisor and D-side to watch for other

developing events if they have a particularly hard problem to deal with. In the future, ASA

Managers may have to rely on the automation to fulfill the same function. Definition of wider

team roles, like Supervisor and the remit of other ANSP in the team could be designed to

mitigate situations where one ANSP is heavily engaged with a problem.

Priority: Medium

8. What strategies are used to allocate attention during task execution? How does the human

operator know when to attend to the automation?

When TSAFE becomes active, it highlights the aircraft involved in the conflict and this

highlighting is the cue to the Controller to attend to this problem, at least to develop an

awareness of it if not to intervene. When aircraft have other problems that move them off their

trajectory, AAC highlights the data blocks for these aircraft, which, again, is the cue for the

controller.

Priority: Medium
95

9. How do the ANSPs keep situation awareness of the traffic when airspace and routes are less

structured in NextGen operations? Do they need a way to “anchor” their mental models of the

air traffic situations?

This is an issue for research. If the concept assumes fewer route structures, the ANSPs cannot

rely as much on past route knowledge and may not be able to maintain situation awareness of the

traffic. With less structures, the ANSP will likely need to rely more on the automation for

conflict detection and other monitoring tasks.

Priority: Medium

10. Are there situations where prospective memory is particularly important?

In general, the ASA tools assist the ASA Manager with prospective memory (PM) tasks by

highlighting aircraft that need attention. Specific instances where prospective memory is

required could be revealed through HITL (or other types of) studies.

Priority: Medium

11. Are there situations that will tax working memory capacity?

Situations that tax working memory are more likely than those that tax PM if ASA tools remain

close to their current design. While the ASA Manager can display aircraft trajectories and data

blocks, they will have to perform the complex spatial reasoning to solve a multi-aircraft problem

in working memory. The ASA Manager will look at a wide traffic flow picture when working

on a potential LoS between two aircraft to ensure any resolution does not create other conflicts

“down the line”. ASA tools currently consider only pair-wise conflicts and do not highlight any

other aircraft that could potentially become involved in the problem. Thus, other aircraft that

could become involved in the problem have to be remembered by the operator.

Priority: Medium

D.5.4 Communication and Coordination

1. Is the coordination overly complex?

An ASA Manager will need to coordinate with pilots, Controllers under mixed equipage,

supervisors, Traffic Flow Managers, and perhaps other ASA Managers. When all these people

will need to coordinate, and procedures for such coordination, will need to be developed.

However, such coordination should not be a major concern as long as those involved have

enough time to contact each other. A question is when to use voice communication and when to

use data communication. Voice is typically used when communication has to be quick.

Priority: Medium

2. What impact do time constraints have on flexibility for coordination? How much time is allowed

for negotiation, if any? How many modification attempts are acceptable?

Time pressure limits the decision context. TSAFE alerts the ASA Manager at three minutes to

LoS, so the Controller has approximately two minutes for negotiation before an action has to be

taken. Any number of modifications to the resolution in that time is acceptable. Although there

are no limits to the modifications per se, the time it takes to send a message to the flight deck and
96

to receive a response will limit the number of changes that can be sent and received in the two

minute time-frame.

Priority: Medium to High

3. Who or what has final authority? How will an end to negotiations be determined?

As noted in the question above, there are two minutes in which negotiation can take place. An

end to negotiation will be determined as time runs out. Final authority rests with the flight crew

who will enact whichever maneuver they consider is the most recent or appropriate but this

would only be true for the aircraft safety and not for the separation. The procedures for

negotiation and the decision authority need to be built into the concept.

Priority: Medium to High

4. How are trajectory changes negotiated via data link? Do they need accompanying voice

communication to express the intent? If both modalities are needed, how are the procedures of

that interaction managed?

AAC, TSAFE and/or the human operator (Controller, ASA Manager) can send data link

messages to the flight deck (and each other). In studies of the ASA concept, this lack of

restriction was not problematic for the tools or the operators. Controllers used voice if time was

short to convey urgency in addition to the basic content of the message but preferred to use data

link when they had more time. Currently, both modalities are needed but there were no

procedures that dictated which should be used and when. Again, this was not problematic in the

recent studies of the ASA, however, the use of communication modalities should be formalized

in procedures

Priority: Medium

5. Are communications via data link, voice, or both? If data link is called for, can the concept cope

with the expected delay and the difficulty of sharing one’s intent via data link? Are there

circumstances where only voice or data link communication should be used?

This is to be decided. As noted in the question above, in subjective observations Controllers

tended to choose data link for communications unless they wanted to convey urgency in their

message. Again, the use of communication modalities should be formalized in procedures.

Priority: Low

6. Are the underlying technological assumptions for communications and information sharing

adequate for the concept?

Yes. Aircraft are assumed to have the capability to fly 4D trajectories and to be able to receive

and transmit data communications, both capabilities that are available today. Ground-side tools

use Data Comm and satellite communications to locate aircraft, again technology available

today.

Priority: Low
97

7. Is the information that is shared among multiple operators appropriate regarding content, detail,

and type of format for each particular operator?

This is to be decided. The data link message table provides a list of messages that still require

action, however, message histories are not displayed as a default on the screen. Controllers find

histories useful at times when they are familiarizing themselves with aircraft that require

intervention, so the ability to call up a history would be useful to the ASA operators.

Priority: Medium

8. How important is participating in “party line” communications (i.e., a voice frequency shared

among several persons) as is used in current operations for situation awareness under NextGen?

Should there be analogous mechanism in data link communication?

This topic needs research. Despite early reports suggesting the party line is very important.

Controllers in the subjective study reported below liked data link for its “clean-ness” and did not

mention the loss of a party line as a problem. Flight deck crews may have a different experience

and this should be investigated.

Priority: Low to Medium

9. How will visual attention demands of data link communication affect other tasks that require

visual attention?

This requires research.

Studies by Wickens and others that have focused on Multiple Resource Theory (Wickens, 1992)

suggest that a person can pay attention to more than one item in the visual channel (or any

sensory channel) up to a certain attention-demand limit. If there are no demands on other

channels, this limit is higher and, of course, a person’s capacity can be affected by stress and

other distractors. Thus, data link communication does not necessarily affect other visual tasks

but under time and work pressure, this is highly possible.

Priority: Medium

10. How are the emergency responses coordinated between human operators?

As noted in question D.5.4.1 above, coordination procedures are not yet well specified in the

ASA concept. ASA Managers coordinate with aircraft pilots more often via voice when there is

time pressure or unusual circumstances (including emergencies). However, they agreed that as

the speed of data link improves they will prefer to use this ‘cleaner’ method of communication.

Priority: Medium to High

D.5.5 Organization, Selection, Job Qualification, and Certification Factors

1. What will be organizational structure, interactions between organizational units, and delegation

of authority to units (e.g., under TFM organizational relationships among national flow

management, regional traffic management units, and sector controllers)?

This is unclear. As NextGen concepts divide broadly by function rather than NAS structure

(sector), it is unclear whether multiple ANSP will be responsible for different activities within

one sector or whether one ANSP will take on multiple functions using multiple DSTs.

Determining the organizational structure for ASA will entail defining its relationship with other
98

NextGen concepts. The concept is workable assuming a similar organizational structure to

today, however, it is also possible to see where efficiencies could be gained by changing the

relationships between personnel. For example, an ANSP concerned only with tactical activities

may be able to control, with the help of automation, much larger sectors than they can today.

Priority: Medium

2. What skills, abilities, and traits are needed by personnel (e.g., ability to use automation, and to

work more as a manager of operations that are handled mostly by automation with occasional

involvement by the operator)?

ASA Managers will benefit from being comfortable with using a computer. They will work

primarily as a manager of operations that are handled mostly by automation with their occasional

involvement rather than as a Controller. They will need to be able to be vigilant with low levels

of activity but at the same time be able to develop an awareness of a developing situation

rapidly.

Priority: Medium

3. How difficult will it be for human operators to develop an understanding of the capabilities and

limitations of the computer-based models they will employ?

Understanding the algorithms behind ASA will be crucial for ASA Managers, as this will give

them the cues they need to know when to react and how to prioritize aircraft they have to

manage. Currently the behaviors of the ASA algorithms are straightforward, and their

capabilities and limitations are clear. As the algorithms are developed in more detail, and

become more complex to encompass more situations, this may no longer be the case. Training

programs to instruct new users how the algorithms work and where their limitations lie will be

key to users’ understanding and their decisions to act when on-position.

Priority: Low

4. What additional training is needed for the human operators to perform the necessary operations

for the given concept?

Future operators will need to be comfortable using displays and working with a computer. The

automation will complete many tasks on its own without the knowledge of the human operator;

new controllers will have to be able to accept this and work with the automation. However, the

same level of situation awareness and decision-making skill that current day operators have is

still required at least while the automation is being developed.

Priority: Medium

5. How will the human operators train for the more diverse and dynamic airspace and route

structures that will exist in NextGen operations?

This is a question for NextGen integration research rather than for the ASA concept in particular.

As an aside, the airspace structures are also expected to be more generic, at least in the high level

en route airspace in which ASA is currently being researched. For this airspace, training should

enable the Controller to work a number of different sectors. Training standards for other

airspaces will have to be defined as the ASA concept is scoped for these airspaces.

Priority: Low
99

6. Should human operators train to maintain skills that are only needed when the automation fails

(e.g. monitoring for conflicts)? How do they train for random failures and anomalies?

Yes. Other skills are important too and these should be defined within the concept so that

training programs can be built. Currently, pilots train for rare and off-nominal events through

simulator exercises. A similar template could be used for Air Traffic Controllers by giving them

an opportunity to keep their skills sharp with recurrent simulator training.

Priority: Medium to High

D.6 Summary of Key Human Performance Issues for ASA

Section D.4 addressed humans performing tasks by examining nine human factors (e.g., attention

allocation, memory, workload, etc.). Section D.5 took the perspective of what the tasks ask of

humans by examining human performance issues in the ASA operational context. We feel that this

level of detail is necessary for the design of the ASA Research Focus Area to produce a concept in

which human operators can perform efficiently and safely, because there will be much more

automation and different operating modes than those of today. This section summarizes the key

human performance issues and our recommendations/countermeasures for them. The

recommendations are mainly in the context of humans performing their tasks under ASA rather than

fundamental HF research in general.

The summary of recommended research is presented under the same five categories as used in

Section D.5. The main drivers from ASA for these recommendations are the increased use of

automaton and Decision Support Tools, the ASA Manager only being involved under certain

circumstances, and mixed equipage.

D.6.1 Use of Automation and Decision Support Tools for ASA

For the ASA concept, there were many key human performance issues related to the use of

automation and DSTs. These issues arise from the increased dependence on automation for

separation assurance and a significant change in the roles and responsibilities in time and safety

critical situations. All of the questions in this section were rated as High priority. We will briefly

describe the issues and our recommendation for potential countermeasures.

Distribution of roles and responsibilities between human and automation

Issue: Roles and responsibilities are not clearly delineated in the ASA concept. If the automation is

responsible for nominal separation and the ASA Manager handles the off-nominal pilot requests and

conflict situations, there needs to be a clear transition from when the automation is handling the

traffic and when the human takes over. If two automation systems handle the conflict situation (i.e.,

strategic and short-term tactical conflict resolvers), there also needs to be a clear transition from one

algorithm to another. Similarly, if there are two human operators handling different traffic in mixed

equipage, clear delineation of roles is necessary.

Potential Countermeasure/Next Step: A systematic analysis of all combinations of roles

should be cataloged and a select set of viable combinations should be explored further by

concept refinement, cognitive walkthroughs, or HITL simulation.

Human as “Automation Manager”

Issue: If the automation handles nominal separation responsibilities and the ASA Manager handles

the off-nominal situations, the human is out-of-the-loop in the traffic situation until s/he is called in
100

to help. There needs to be sufficient time for the human to get “up to speed” on the problem and

smart DSTs to provide necessary information in a timely and reliable fashion for the human to

quickly assess the situation. In this type of time and safety critical situation, the human needs to trust

the information that the automation provides and the solutions that it generates. Finally, once the

human is brought into the problem to work complimentarily with the automation to solve the

problem, there needs to be a clear decision making authority during short-term conflict or other

critical situations when multiple humans and automation can take action to solve same traffic

situation.

Potential Countermeasure/Next Step: Each of the issues mentioned above – i.e. human outof-the-loop, necessary time to gain traffic awareness, smart DSTs to provide information to

the human, trust in automation, and decision-making authority – are highly critical to the

success of the concept. An examination of the issues requires an initial design of the concept

instantiations, procedures, and tool prototypes. For each of the possible iterations of the

concept, focused walkthroughs or HITL simulations should be conducted to evaluate the

feasibility of the design.

Recovery from errors and failures

Issue: Since automation handles safety-critical responsibility in this concept, its potential failure

needs to be sufficiently mitigated. The current design has two independent levels of automation

(e.g., AAC and TSAFE), one of which is relatively fail-safe. However, all potential failure modes

that can be identified should be examined to see if there can be adequate recovery. In the current

design, the failure of strategic conflict resolver (AAC) is recovered by the presence of an

independent short-term resolver (TSAFE) but no procedures or tools (except TCAS) exist when

TSAFE fails. If the human is expected to intervene, there needs to be a set of indicators that can act

as a diagnostics for the automation failure. Similarly when emergency or off-nominal traffic

situation occur (e.g., data link failure), the automation needs to be “aware” of the off-nominal

situation and act accordingly. Procedures also need to be built to recover from the situation.

Potential Countermeasure/Next Step: The issues in this category are difficult concept-related

questions. The concept needs to be refined as a part of human-centered design in which the

automation and DST requirements are considered in the context of its impact on human

during off-nominal situations. A list of off-nominal, safety-critical situations should be

generated and each of the situations should be followed through in a cognitive walkthrough

or functional analysis to explore potential concept refinements that can mitigate the issue.

Once a set of mitigating designs are developed, procedures, tools, and traffic situations

should be developed and tested to evaluate the efficacy of the solutions.

D.6.2 Human Decision-making and Workload for ASA

Decision-making in time and safety critical situations

Issue: If the automation is responsible for separation assurance and pilot requests under nominal

situations but the ASA Manager handles off-nominal situations related to requests and recovery

from short-term TSAFE maneuvers, it is important to design the role of the ASA Manager to have

ample time to perform his/her tasks. Time is inversely linked to ease of decision-making. The

concept design is such that the ASA Manager may need to face many situations in which s/he will

have a short time to assess the traffic situation and react.
101

Potential Countermeasure/Next Step: Since the nature of the concept presents the likely

possibility of human decision-making in short timeframe under safety critical situations, any

changes or refinement of the concept design should double check if they pose potential

adverse impact on the human operator’s ability to make time critical decisions. The concept

changes should be assessed via walkthroughs.

Workload

Issue: Peak workload can be an issue for the ASA Manager if s/he is given a large airspace with an

assumption that automation takes care of most of the tasks during nominal operations but during offnominal traffic and/or weather events, many aircraft require the ASA Manager’s attention at the

same time. Peak workload could also be a problem for the Controller during mixed operations if s/he

needs to maneuver unequipped aircraft around ASA-equipped aircraft during off-nominal traffic

and/or weather situations.

Potential Countermeasure/Next Step: The concept needs to be refined with detailed tools and

procedures that can potentially mitigate the situation. The instantiated concept should be

evaluated with HITL simulation to see the impact of peak workload under the operational

condition.

Mixed equipage

Issue: If both ASA-equipped and unequipped aircraft can fly in the same airspace, the specific

interaction between ASA Managers and Controllers needs to be worked out. The details of the

interactions are needed to examine the human performance issues – e.g., the size of the airspace that

ASA Manager handles, the “rules of the road” for the equipped and unequipped aircraft, operational

procedures during mixed equipage conflicts, tools available to help the ASA Managers and the

Controllers in keeping track of the equipped and unequipped aircraft, etc.

Potential Countermeasure/Next Step: A significant effort is needed to provide the details of

the operational assumptions related to the mixed equipage airspace. Once the details of the

concept are identified, the concept should be evaluated for the impact on the human

operators using prototype tools, procedures, and scenarios in HITL study.

Brittleness/resiliency of the concept

Issue: Given that the concept requires a high level of dependence on automation in a safety-critical

environment, error in the automation can lead to the brittleness of the concept unless sufficient

layers of human and automation redundancy are built in. The two independent conflict resolution

systems are one layer of redundancy and the presence of the ASA Manager to handle situation that

the automation cannot is another. However, the ASA Manager might not be able to act as a useful

back-up if s/he has little time to act. In addition, more layers of redundancy are needed to handle the

plethora of off-nominal situations. It seems that the ASA Manager will require someone to monitor

his/her workload, stress, etc., in ways that Front Line Managers monitor controllers today, but it is

unclear what that role will be.

Potential Countermeasure/Next Step: A human-system analysis is needed to chart out the

chain of humans and automation that should be developed to build in the redundancy in the

new concept. A series of walkthroughs should be conducted to step through various offnominal situations to see if the redundant functions of humans and automation will be

sufficient to recover from the situation.
102

D.6.3 Attention, Situational Awareness, and Memory for ASA

Gaining situation awareness in time-critical situations

Issue: The issues for attention, situational awareness, and memory are mainly driven by low

workloads for the ASA Manager while ASA automation is providing Separation Assurance but

needing to become involved when ASA automation cannot handle a pilot request or other situation.

Design aspects in this area include:

• time for operator to get back in loop if the operator is passively monitoring and suddenly there

is a need to get back into the loop;

• means and time for ASA Manager to gain situation awareness under such situations;

• ASA Manager remembering appropriate, with perhaps recommendations being provided by

automation, to respond when s/he needs to become involved; and,

• triggers for events that indicate an emergency or anomaly

Potential Countermeasure/Next Step: Any changes and/or refinement to the concept should

lead to double-checking of the assumption that the human will have enough time and proper

indicators for gathering necessary information to perform his/her tasks. The level of detail

needed to address this question is likely to involve tool and display development. Prototype

tools should be developed and evaluated with focused part-task studies.

D.6.4 Communication and Coordination for ASA

Complexity of coordination in time-critical situations

Issue: Time pressure limits the decision context. Once the ASA Manager is alerted of the offnominal conflict, request, or emergency situations, s/he needs to assess with whom to coordinate

before taking action. Coordination with multiple pilots might be necessary and they may not have

full awareness of the situation as well. Both the pilots and the ASA Manager must determine if the

coordination needs to involve automation or if the decision can be made solely among humans. The

situation is further complicated in mixed equipage situations which may involve pilots, ASA

Manager, Controller(s), and various separation assurance automations. It may also be unclear who

would have the final decision authority. All decisions must occur with clear and efficient

coordination, as many decisions are likely to be both time and safety critical.

Potential Countermeasure/Next Step: A significant effort is needed to understand how best to

coordinate/negotiate among multiple people during time-critical situations. Research in

collaborative decision-making as well as a catalog of observations on how the current ATM

system achieves minimal and efficient coordination in similar situations are both useful in

designing a sensible coordination mechanism in the ASA concept. Full team configuration

that includes all humans and automations with potential decision-making authority need to

be accounted for and coordination procedures and supporting DSTs need to be developed

for the possible combinations of people/automation that require coordination. Once

developed, the procedures and tools can be evaluated using walkthroughs and/or HITL

simulations.
103

D.6.5 Organization, Selection, Job Qualification, and Certification Factors for ASA

Maintaining skills that are seldom used

Issue: If the ASA Manager handles the off-nominal conflict, request, or emergency situations but the

nominal situations are handled by the automation, the human may not have enough exposure to the

traffic situation to effectively recognize and resolve the situation in a time-critical manner. Finding a

training procedure to keep their monitoring and decision skills sharp when the occurrence is rare will

be a challenge.

Potential Countermeasure/Next Step: A systematic analysis should be done to determine

which air traffic skills that exist today that will likely become degraded for the future ANSP.

In particular, the skills for the ASA Manager that are critical to the job but may not be often

used should be compiled. Once the skills are cataloged, recurrent training exercises for rare

and off-nominal events for the ASA Manager should be developed.

D.7 References

Kopardekar, P., et al. (2009). Feasibility of mixed equipage operations in the same airspace. Eighth

USA/Europe Air Traffic Management Research & Development Seminar ATM, June, Napa,

California, USA.

Prevot, T., Homola, J., Mercer, J., Mainini, M., & Cabrall, C. (2009). Initial evaluation of NextGen

Air/Ground Operations with ground-based Automated Separation Assurance. Eighth

USA/Europe Air Traffic Management Research & Development Seminar ATM, June, Napa,

California, USA.

Wickens, C. (1992). Engineering Psychology and Human Performance 2nd ed. New York: Harper

Collins.
104
105

Appendix E: Cognitive Walkthrough of Automated Separation

Assurance

Four Research Focus Areas (RFAs) in the NextGen-Airspace Project—namely Separation

Assurance (SA), Airspace Super Density Operations (ASDO), Traffic Flow Management (TFM),

and Dynamic Airspace Configuration (DAC)—were examined for human performance issues and

reported in Appendix D, F, G, and H (SA, ASDO, TFM, and DAC, respectively). At some point in

concept designs, the human performance issues identified will need to be addressed in detail. As

discussed in prior chapters, this can be covered using Human in the Loop (HITL) simulations, such

as those run to explore ASA (Prevot, et al., 2009). However, there are other simpler quicker

techniques that can be used to explore the issues at a high level which will help researchers focus

their inquiries in more detailed HITL simulations thus ensuring time and effort is spent on the areas

that need research. One such method is the cognitive walkthrough (Polson, Lewis, Rieman, and

Wharton, 1992; Polson & Smith, 1999). As will be explained later in this appendix, one concept

from SA, namely ground-based Automated Separation Assurance (ASA), was used as the subject of

a walkthrough to illustrated this approach. This walkthrough of ASA is reported in this Appendix E.

The cognitive walkthrough described below is an illustration of how a knowledge elicitation method

can be used to begin to find answers to questions that will ultimately influence concept design. A

number of the issues raised in the sections above were included in the walkthrough, but not all could

be included because there was not enough time and resources to run a longer study. Some examples

are presented below to illustrate how a design issue (such as those listed in Appendix D.6) can be

operationalized as research questions.

From the four RFAs, we chose one concept instantiation from SA, namely ground-based Automated

Separation Assurance, for the cognitive walkthrough. The walkthrough explored the potential

human performance issues that were identified in Appendix D and looked for further insights and/or

answers to the identified issues.

The walkthrough was focused on one human operator in ASA, namely the ASA Manager, who was

described in Appendix D. In the cognitive walkthrough, participants viewed sample traffic

situations and were probed with various HF questions. The questions were devised from the HF

categories covered for SA in Appendix D. The results of the study are summarized below.

E.1 Purpose of Cognitive Walkthrough

Our aims in the cognitive walkthrough of ASA were twofold:

1. Describe and test a method (i.e., cognitive walkthrough) that could be used by concept

research teams to take a “first-cut” investigation of potential HF issues

2. Use this method to identify and explore the impact, from a user’s perspective, of HF issues in

advance of “locked-in” tool design

A final goal is to propose how such findings could be used to refine operational requirements and/or

the design of future testing and research.
106

A general ground-based separation assurance concept was created for this analysis, modeled closely

after the AAC concept (Erzberger, 2001) in the Separation Assurance RFA (described in Appendix

D), in which the ground-based automation assumes separation responsibilities and the ASA Manager

plays a supervisory role to the automation. The ASA scenario used the in the walkthrough was more

narrowly focused than the ASA Manger roles described in Appendix D.3, in that ASA Manager, in

conjunction with short-term conflict resolution automation, acts as a back-up safety network to the

overall system for off-nominal situations that create unresolved short-term conflicts. In the

walkthrough the ASA Manager was not presented with additional complexities, like user preferences

or pop-up weather.

Based on a review of separation assurance (Appendix D) and NextGen HF issues (Appendices B and

C), a number of issues were expected to be salient to ASA Managers. From these issues, a set of

example hypotheses were generated. We hypothesized that:

• ASA Manager workload could be high when traffic levels are high

• ASA Manager could have limited or low trust in the conflict detection and resolution (CD&R)

tools because they act without ASA Manager review

• as the ASA Manager is out of the loop due to the CD&R tools acting without human review,

the ASA Manager may have trouble forming an understanding of a situation they are required

to step into

• as the ASA Manager is out of the loop, s/he may require longer times to formulate resolutions

and take action when s/he is asked to step in to monitor short-term conflict situations

• as the automation dictates the timeframe in which the ASA Manager sees a problem, the ASA

Manager may object that their decision time is constrained

• now that the ASA Manager is part of a human-automation team, the division of responsibilities

within their sector is unclear

• ASA Manager will favor using voice for communication with aircraft over data link because

voice is familiar

• ASA Manager will be uncomfortable working with an “unpredictable” automated tool because

s/he has no knowledge of its programming.

These eight hypotheses refer to the five categories of human performance issues discussed in

Appendix D.6 for SA to identify gaps in HF design and research. They also cover six of our nine

areas of HF issues (see Appendix B): workload, decision making, responsibilities, interaction with

automation, communication, and attention. There were not expected to be any issues arising with

the areas of memory and the concept was not considered to be developed enough to be able to

answer questions of certification and training. The ninth area, potential errors and their recovery,

were treated as a special case and as independent variables for our study.

E.2 Cognitive Walkthrough Approach

The cognitive walkthrough (CW) technique is widely used in usability studies to test proposed

computer interfaces (see Jeffries, Miller, Wharton & Uyeda, 1991, for a review as CW was being

introduced). It was developed by Polson, et al. (1992) based on a theory of exploratory learning

which posits that an interface that is easy to learn to use is also easy to use (i.e., if a user who has

never seen an interface can successfully navigate through and complete basic tasks, the interface is

user friendly).
107

Briefly, the technique entails showing a potential user a mock up of a new interface and gaining their

feedback through directed questioning as they indicate how they would perform various tasks that

the interface is designed to facilitate. Questions focus on participants’ reasoning about the

interface/automation—what they think it is doing now, what they expect to happen when they push a

button, what they think they have to do next, and so on.

To meet the conditions of the walkthrough, a researcher is required to put detailed thought into the

behaviors of the prototype interface and into describing the action sequence for each task selected.

Although the process of using the prototype has to be thought out in detail, the way the tool is

presented does not have to be an advanced mock-up but can be a very simple presentation, as simple

as a drawing of how the interface could look. Time should be spent scoping user characteristics to

identify what their goals and knowledge will be when using the proposed tool.

Advantages of the method are that the degree of interface development can be minimal, as little as a

series of hand-drawn storyboard frames that show the proposed state of the interface at each step of

a task. Additionally, the technique demands few resources, so it can be performed cheaply and

quickly. A researcher can draw up a storyboard and present it to a participant in an ordinary office

setting without any specialized equipment.

These advantages make the cognitive walkthrough a candidate technique for exploring other issues

of concern (and not just usability) of a tool as it is being designed and prototyped. The cognitive

walkthrough in this Appendix (E) tested whether a modified walkthrough method was feasible and

gave useful findings for a candidate concept that was set up to be representative of a concept in an

early prototype phase. Two major changes were made to the original Polson, et al. (1992) method.

Questions about human factors issues generated from the five HF issues we assessed in our review

of SA (see Appendix D) were swapped for the original four Polson, et al. (1992) questions. This

was done to address the issues that we had flagged as important for SA in Appendix D. Secondly,

we did not use drawn storyboards to convey events and interfaces. Instead, we had video recordings

from a previous experiment that were adapted to our walkthrough. The recordings were obtained

from the Airspace Operations Laboratory (AOL, NASA Ames Research Center) HITL simulation of

a prototype Separation Assurance concept (Prevot, et al., 2009; Homoloa, 2008). The video

recordings showed a controller’s display in the AOL HITL simulation and, therefore, the actions that

the participant controller had taken. Our ASA Manager subjects observed these recordings as the

situation story (or stimulus) for their comments. By doing this, all of our ASA Manager participants

watched exactly the same events unfolding in the same way. Thus, our cognitive walkthrough

focused on a specific set of events, and the questions that were asked were phrased specifically to fit

this set of events.

E.3 Description of Separation Assurance Concept Studied in Walkthrough

Our ASA Manager participants were instructed that the environment they were evaluating had two

automation tools operating for separation assurance. The first tool functioned by looking ahead 3 to

20 minutes for conflicting trajectories, identifying a conflict, and then solving it by sending a

trajectory change to one of the aircraft. These avoidance maneuver solutions also included a

trajectory to return the aircraft to its original route. The second tool, acting as a safety back-up to

the first tool, only looked ahead from 0 to 3 minutes and functioned in the same way as the first tool

except that it only vectored aircraft away from the conflict without including a trajectory path that
108

would return the aircraft to its original route. In our activity we called these tools a Mid-term

Separation Assurance tool (MSAT) and a Short-term Separation Assurance tool (SSAT)

respectively. Both of these tools operated by only looking at two aircraft at a time and assigned

either a vertical or lateral solution to one of the aircraft via an uplink. The aircraft rerouted with the

SSAT were flagged to the ASA Manager, who had to return them to their original 4D trajectories

when possible. The ASA Manager role was to monitor primarily the SSAT, but to study both tools,

and to maneuver SSAT vectored traffic back onto its original route. In this study-designed concept,

ASA Managers were also expected to assist in emergency and off-nominal situations.

E.4 Study Method

E.4.1 Walkthrough Generation

Based on the discussion of human factors issues in NextGen (covered in Appendices B and C) and

specifically in the Separation Assurance RFA (covered in Appendix D), the five human performance

issues addressed for SA in Appendix D.6 were broken out into sub-issues, which we call human

factors themes, for specific attention. These human factors themes were selected as being

potentially affected by introducing NextGen automation for the Separation Assurance function.

They are listed in Table E.1.

Table E.1. HF Subthemes and Their Relationship to the Five HF

Design and Research Areas for NextGen

Five HF Question Groups (Appendix D.6) HF Themes Addressed in Walkthrough

Attention & Memory Attention

Monitoring

Situation awareness

Memory

Decision making & Workload Decision making

Time pressure

Workload

Task management

Communication & Coordination Communication

Coordination

Roles

Responsibilities

Interaction with Automation Interaction with automation

Trust

Selection/ job qualification / certification,

organization

Training

Procedures
109

A walkthrough was constructed by comparing questions relating to these themes. Four off-nominal

events were selected from two HITL simulation runs from the AOL test data. For the study,

participants observed events in the video recordings described below, and then were asked and

responded to our HF questions.

Each event1was chosen because it represented a different off-nominal problem, but some parameters

were similar. All the events took place in one high altitude en-route sector (above Indiana) in the

ZID (Indianapolis) Center. This sector (ZID91) is a busy area, taking arrivals and departures from

St Louis to the west and Louisville to the east. There was more traffic than in the current

day—usually two times as many aircraft, which equates to approximately 30 aircraft at any one

time.

Event 1: Southwest (SWA) 339 begins a northwesterly climb that will take it through the

trajectory of Acey (ASQ) 1360, which is in level flight traveling southeast. The

conflict is first flagged by SSAT at three minutes. The first resolution (for ASQ1360 to

turn left) is not executed and has to be resent.

Event 2: SWA1864 enters the sector on a northwesterly climb that will take it through the

trajectory of American (AAL) 140, which is in level flight traveling northeast. The

conflict is first flagged by SSAT at one minute. ATC asks the American to turn left,

and when this is possibly not enough for separation, asks the Southwest to also turn

left.

Event 3: Selected from a different experimental run, this event takes place in a lower traffic level

of approximately 1.5 times current day traffic. Jetlink (BTA) 39 and AAL711 enter the

sector, in close proximity, Jetlink is flying southwest and American northwest. The

American is climbing through the trajectory of the Jetlink. SSAT sends a resolution to

the Jetlink to turn left, while at the same time the controller issues a temporary altitude

hold to the American.

Event 4: A scripted failure of the datalink capability on an aircraft (Flagship/ Pinnacle (FLG)

144) flying approximately west to east across the lower portion of the sector brings it

into level conflict with a Northwest (NWA) 612 flying approximately south-west to

north-east. An additional layer of workload is added because as FLG loses its datalink

while a multi-aircraft conflict is developing in the northeast of the sector. The conflict

is tracked by MSAT and SSAT but is resolved by the controller because the clearance

SSAT sends (repeatedly) to NWA612 goes unanswered.

Each event was analyzed to identify its progression. A step was marked if there was an action by the

controller, an aircraft or the automation and then drawn out as a storyboard tile. For example, when

an aircraft was sent a clearance, this was counted as a step. This example is shown in Figure E.1 for

Event 2, when the controller sends a message to the American aircraft to turn left. On average, the

events were broken down into twelve steps, although Event 11 had an additional seven steps where

the data link malfunction was reported.

 1

 All the events and aircraft created for this study (and in the video recordings from the AOL study)

are fictional and are not based on any real world event.
110

A relevant question that addressed one of the 16 cognitive study themes was identified for each step

in the event breakdown. For some of the steps, where it seemed appropriate, an organizational

question was also created. For example, for this step in Event 2 the cognitive theme identified was

time pressure and the question was “Is data link fast enough for this resolution?”and the

organizational question was “As you take the conflict, how should control shift from automation to

human?”. Over the four events, 136 human factors issue questions were generated. Questions were

not unique—if the research group thought a question was relevant or important in more than one

event or step it was asked again.

Figure E.1. Example sheet from the task breakdown for Event 2 in the walkthrough.

In addition, where the event progression showed that the problem may have advanced the

controller’s thinking or understanding (e.g., s/he received more information), a cognitive

walkthrough-style question was constructed that focused on one of three points of view that relate to

the controller’s interaction with the automation. These three questions were:

1. Did you expect that to happen?

2. What do you expect to happen next?

3. What would you do?

In the example in Figure E.1, the cognitive walkthrough style question to the participant was

“Would you have done this?”

E.4.2 Participants

Six retired controllers took part in our study; all had more than 25 years of experience with the

current air traffic control system. However, they also had a reasonable amount of experience using
111

the AOL’s prototype tools, and were familiar with the initial operation of some of the NextGen

concepts.

E.4.3 Equipment and Stimulus Events

The events evaluated by our participants included traffic levels representative of both 2 times and 3

times current day traffic levels (30–45 aircraft in the sector at one time). These events were drawn

from recorded video files from the previous HITL simulation in the AOL. Video files were

originally recorded and played back through the Camtasia Studio 6 screen recorder software. The

videos were displayed to the participants on a Barco 28” LCD monitor (ISIS model) with a

resolution of 2000 x 2000 pixels. Participants’ answers were recorded by an Olympus digital voice

recorder (WS-311M model).

E.4.4 Procedure

Participants received a short orientation, which explained our purpose and introduced the sector that

was the focus for the day. The automation notation on the display was briefly described but, as all

participants had worked with these tools in the AOL before, it was assumed that they understood the

icons and display elements. Beyond a list of tasks that they would be required to perform (e.g., put

aircraft separated by SSAT back onto their original routes, deal with off-nominal or emergency

circumstances), ASA Manager procedures, operating rules and jurisdiction were not set out for our

participants.

Each of the four events was shown three times to each participant. First, the event was played in

real time. Participants were free to ask questions or make comments as they watched. Second, the

event was stepped through. The video was forwarded to each breakdown step and then paused at a

point just as the action for that step had just happened. Figure E.2 shows an example of the way the

problem area on the display for Event 2 looked at as it was paused at the step described in the

storyboard in Figure E.1. (The aircraft of interest are in the bottom-right corner of this picture. The

slightly wider shot gives a better illustration of the level of traffic and other tasks that the controller

was working on in conjunction with the problem.) One member of the team briefly described the

occurrence in the step and a question with a cognitive theme was asked, plus a cognitive

walkthrough style question if one had been selected. In the third playback, a number of the steps

from the task breakdown were selected and an organization question was asked for each one of

these.

Participants’ answers were recorded on a small hand-held digital recorder and later transcribed.

Qualitative analyses were planned. The analyses for this report selected 56 key questions that

addressed the 16 themes. Once these questions had been transcribed and summarized, they were

cross-referenced with the specific SA human factors issues (Appendix D.6) and the general human

factors issues (Appendix B). As noted above, our goal was not primarily to gain answers to our

questions. We wanted to ascertain whether and how the issues we predicted would be described by

participants and to determine whether a cognitive walkthrough of events would be a useful method

for collecting data about potential human factors issues of prototype systems.
112

Figure E.2. Snapshot from Event 2 in the walkthrough as a resolution is sent

to the American Airlines 140.

E.5 Results and Discussion

The questions and answers exchanged during our walkthrough are too numerous to list in this

chapter. However, to illustrate the method, a small sample of five questions was selected and the

issues they address are discussed below. These are the two questions asked in our example above

(see Figures E.1 and E.2) that concerned time pressure and shift of control and a further three

questions concerning responsibilities, trust in the automation and controllers’ expertise. The

questions were phrased to refer to a particular step in an event. Participant responses to more (but

not all) of the HF issues identified in Appendices D and B above can be found in the addendum to

this appendix.

E.5.1 Time Pressure

One of the questions asked at the step shown in Figures E.1 and E.2 was “Is data link fast enough [to

issue] this resolution?” and is one in the set of five questions constructed to ask participants about

the pressures of time in decision making. Time pressure was highlighted as an issue in our general

HF issues analysis (Appendix D.6.2). For this question, participants replied that as time to put a

resolution in place gets shorter, they favor using voice over data link to issue clearances. They

thought voice is faster, although, most of the participants conceded that data link may become faster
113

in the future and, if it became as fast as voice is currently, it would be a better medium because it is

“cleaner” (no static, no step-ons, no mis-hears).

From other questions in the set, it was found that participants were able to estimate whether they had

enough time to solve a conflict, when their ability to do that would become marginal and when they

needed to act quickly for any solution to be put in place. “It’s getting to the point of no return,”

commented P5 as he watched one of the events. Participants were also able to identify which types

of solution maneuvers could still be executed in the remaining time and which were no longer

possible. When challenged with a situation of heightened severity, most participants indicated that

less than three minutes was too short a time horizon to reasonably be expected to take action if their

workload was anything above minimal.

E.5.2 Shift of Control

In Event 2, one of six questions about the shift of control from the automation to the controller was

asked: “As you take a conflict, how should control shift from automation to human?”. These

questions were considered part of the larger set of questions asking about interaction with

automation (Appendix D.6.1). Participants indicated that as soon as they took an action on an

aircraft in their sector, control would shift to them so that aircraft and the problem would become

their responsibility. One participant noted that, because of this, it is unwise to take actions on

aircraft with limited data blocks, which means you have to presume/trust that the automation is

successfully separating these aircraft. Shift of control in the other direction, from human to

automation, seemed to have a more clear delineation for participants. They agreed that once an

aircraft could have its data block dropped to a limited tag, indicating there were no more tasks to

complete for it and its 4D route had been accepted and recognized by the automation (and was not

predicted to be in conflict for the remainder of the sector and beyond), participants assumed that

they had given control back to the automation and they were once again no longer responsible for

that aircraft.

E.5.3 Responsibility

Shift of control is a part of defining controllers’ responsibilities. This was an area where participants

had trouble giving answers because responsibilities under a future SA environment were unclear to

them. An example of a responsibility question is from the step just prior to those shown in Figures

E.1 and E.2, when it is clear that the automation identified the conflict with very little time until LoS

(a minute): “The SSAT failed to detect the potential conflict until the last minute. Who is

responsible?” This question addresses directly the HF issue of being at the “sharp end” of the

separation assurance process. Participants answered that if the automation fails to detect the conflict

until the last minute, they still thought they would be responsible for maintaining separation,

although participants did note that a one-minute warning is far too little for this to be a fair

responsibility. This issue seems to be a key problem—can a controller be held responsible for a LoS

if s/he had no knowledge of it until a minute before it happened (and it is too late to prevent the

LoS)?

E.5.4 Trust in the Automation

The problem that makes responsibility an issue is that the prototype automation did not reliably flag

a conflict while there was a reasonable time to act. We predicted that functionality like this would

manifest itself as an issue of trust in the automation. To explore this, four questions about trust were
114

asked, for example “Are you happy to leave the automation to solve the conflict?” Contrary to our

predictions, some participants said that they would trust the automation, although others said they

would not. One reason participants gave for trusting the automation was that if they did not trust it,

they would then need to check every clearance that the automation was giving. They could not do

this, so they had no choice but to trust the automation to a certain level for their partnership with the

automation to work. Their reason for not trusting the automation was a lack of time. As discussed

above, if a LoS were imminent, there was not enough time to wait to see if the automation could

generate a good solution. Participants would act by giving a clearance they knew would work (to

achieve a degree of certainty). From their responses, it seemed that for any one clearance/resolution

participants would trust the automation initially, but if there were a problem with the clearance, they

would not risk trusting the automation a second time for that same problem.

E.5.5 Controllers’ Expertise

Under NextGen, it is likely that controllers’ required skill sets will change. Some of the questions

asked, for example those about memory and training, highlighted the extent of controllers’ skills

currently. Four participants highlighted the importance of localized knowledge (i.e. sector flows,

pilot alertness, etc.) in shaping what they would expect of aircraft under their control. Other answers

illustrated the value of retroactive (long term) memory. Because current air carrier schedules are

regular and aircraft generally fly the same routes every day, controllers become familiar with the

paths of the aircraft that fly through a sector during on-the-job training and working it regularly.

Participants said that if they were trained on a sector they would be able to remember the route to

put an aircraft back onto if it had been vectored off its route for separation.

E.5.6 Summary and Recommendations of Content Results

One of our two aims for this walkthrough was to take a “first-cut” investigation of potential HF

issues from a user’s perspective. The findings of the Appendix D review of human performance

issues for SA were used to generate possible hypotheses of human factors issues that might arise for

an example ground-based ASA concept and tool. Using the process described above, participant

responses were reviewed and catalogued to address as many of the ASA and human factors issues

identified as possible. They were also used to support or oppose our eight hypotheses.

Five of our eight hypotheses were not supported by participants’ answers to our questions:

1. ASA Manager workload could be high when traffic levels are high. Workload was not

necessarily predicted to be high (by participants in our study) when traffic levels were high.

Based on the assumption that the automation would be separating all equipped (4D) aircraft,

participants judged that they could handle ten to twelve aircraft that needed a controller to

perform some task for them, like a reroute, or two to three pair-wise conflicts at a time (with a

three or more minute warning).

2. As the ASA Manager is out of the loop, due to the tools acting without human review, the ASA

Manager may have trouble forming an understanding of a situation they are required to step

into. Even without training on the sector they were viewing, participants were able to follow

the traffic and often predicted future conflicts before the automation flagged them, suggesting

they had an awareness of the presented situations. However, our participants were controllers

trained in the current day manual methods. When training is updated to encompass NextGen

tools, this kind of pattern recognition may not be a skill-set that is developed by incoming

ASA Manager.
115

3. As the ASA Manager is out of the loop, they may require longer times to formulate resolutions

and take action. Participants did not seem to require a longer time to take action. During their

first viewing of an event, a participant would often comment on what their strategy would

have been. From this, we observed that participants developed these action plans in time to

issue clearances had they been actually working the sector.

4. ASA Manager did not think they would be uncomfortable working with an automated tool. Our

participants were split in their opinions, with three participants indicating that they would try

to review and alter clearances that the SSAT sent, or issue different clearances altogether from

those that the automation had decided were best. Three participants thought they would easily

accept the automation sending clearances without their review.

5. ASA Manager did not favor using voice for communication with aircraft. Under normal

circumstances, participants preferred data link, as it is “cleaner” (no static, no step-ons, no

miss-hears). At the moment, they favor using voice over data link to issue critical clearances

because it is faster. But, as data link becomes faster in the future, it will become a better

medium.

Our remaining three hypotheses were modified by the answers received:

6. ASA Manager could have limited or low trust in the automation tools because the automation

acts without ASA Manager review. As discussed above, participants described that they had to

trust the automation for the traffic to be a workable problem but that when time was short they

could not afford to wait to see whether the automation could work its way round to a viable

solution.

7. As the automation dictates the timeframe in which the ASA Manager sees a problem, ASA

Manager may object that their decision time is constrained. This prediction was supported. If

the automation flagged a conflict at or before three minutes, participants estimated that they

would have enough time to solve a conflict (although three minutes is cutting it close).

However, in three of the off-nominal events that were presented, the automation did not flag

the conflict until there were two minutes or less to LoS. In these cases, participants felt that

they had not been given enough time to create an elegant solution to the problem, and perhaps

not enough time to solve the LoS at all.

8. Now the ASA Manager is part of a human-automation team, the division of responsibilities

within their sector is unclear. This prediction was supported. In contrast to today’s

responsibilities, which at their most basic level are very clear cut and straightforward (if an

aircraft is in your sector it’s your responsibility), it was not clear when aircraft in our NextGen

examples were the ASA Manager’s responsibility and when this fell to the automation.

Participants, in the absence of other guidance, had to assume today’s rules still applied. That

these rules would be unworkable and unacceptable under NextGen became very clear when

imminent LoS problems popped up in our examples. Also unclear was when (or if)

responsibility shifted as an ASA Manager began to work on a problem. Our participants

usually assumed that as soon as they took an action they were now responsible for separating

the aircraft, but there was no notation on the display to indicate that responsibility had been

transferred.

From this review, it would seem concerns that usually spring to mind when designing automation,

such as workload, communication and situation awareness are being addressed by functional studies

with ASA automation prototypes (such as the work undertaken in the AOL). However, a raft of
116

issues that ties time constraints with responsibilities, procedures, and training need to be addressed if

the prototype tools are to be developed further. If recommendations were highlighted from this

walkthrough, they would focus on these issues. The following are four example recommendations

to illustrate this:

1. Procedures that define ASA Manager responsibilities need to be scoped in more detail in

future studies. This is likely to change how participants act, as they are assuming different

levels of responsibility from current day rules. One area that needs to be defined is exactly

when an ASA Manager becomes responsible for a LoS and when control shifts back to the

automation.

2. Flagging conflicts needs to become more reliable at the three to four minute point. At three

minutes, all conflicts have to be flagged to give ASA Manager a reasonable window of time

in which to take an action.

3. Consistency and predictability of a tool is paramount. Even though the prototype automation

made poor decisions and resolutions, this did not seem to bother our participants because they

quickly learned the resolution was going to be problematic and they could predict that they

would need to act. Training the rules a tool operates by and the implications that follow are

therefore important aspects of the human-automation relationship that study participants will

be able to use in their decision making.

4. Some indication of who/what is responsible for an aircraft or the shift of control should be

displayed for a participant. While it is likely that a fully trained operator of the deployed

NextGen ASA tools will not need this indication, for the time that prototype systems are

under development such indicators may be valuable for both participants and researchers.

E.6 Summary Discussion of the Walkthrough as a Method

Our second aim was to describe and test the walkthrough as a method, to estimate whether concept

research teams could gain insight from the process, and to examine what levels of effort are required

to complete it.

In our estimation, the walkthrough did enable us to look at human factors issues in a greater level of

detail and with more focus on the exact problems than the type of expert analysis presented in

Appendix D. However, the expert analysis was an invaluable starting point for the walkthrough in

that it identified HF areas on which the walkthrough questions needed to focus. Asking the

questions gained information from a user’s perspective and often reframed the problems at a more

operational level. Although participants were observers themselves, and did not work the traffic,

their responses were consistent (where they could be matched) with participants from the earlier

AOL studies and it is considered that these findings are meaningful. Other advantages of this

method are that it is indeed inexpensive and can be put together in a relatively short time. Both of

these advantages can be capitalized on further if the expert analysis (which is useful in and of itself)

has already been completed. But even including the effort for this expert analysis in the process,

walkthroughs require far less effort and resources than a human-in-the-loop simulation.

A concern we initially had for our specific walkthrough was that the many questions asked during

the walkthrough would not combine to provide answers to the broader hypotheses and concept

questions regarding SA as described in Appendix D. Being able to relate our themes to our

hypotheses to give relevant answers indicated that we had addressed the essence of the issues in

which we were interested in the CW questions. Addressing study hypotheses could also provide
117

useful information for concept developers. Developers could use the themes to identify areas of

their concept that require more clarification or scoping.

In our particular approach to the walkthrough, we used video of a working prototype as our stimulus.

The video was a particularly rich source of stimuli to evoke responses from the test subjects about

the ASA scenario. Such a video is not likely to be available in future walkthroughs, but other types

of videos are possible, such as recording during shakedown runs prior to a HITL simulation.

Alternatively, graphic images of the display, and hypothetically what procedures and problems could

happen, could be built as a video, independently of any tool, before working prototypes are

available. In our walkthrough, participants were not able to interact with the display at all because

they were shown a video, not even a reconstruction of the event on a workstation.

Two aspects of the images presented to participants were important: the whole display was presented

and that the images were dynamic—actions could be seen in real time—even though we paused the

video to ask questions. The value of presenting the whole display was that participants gained an

awareness of how much traffic “2x” looked like and what kinds of events could be occurring

concurrently (i.e., the size or scope of the task). The value of seeing actions in real time was that

participants could estimate their workload and task/time pressures. These two factors may be

invaluable only in this instance because the particular ASA Manager position and function we

looked at raised issues and concerns about these specific HF topics. But, more generally, time,

workload, and task scope are likely to be HF topics of interest for most NextGen concepts.

While in the SA scenario we examined with a walkthrough was helped by the use of a video,

walkthroughs without videos, using storyboards or other forms of human-machine interfaces and

event presentations are still fruitful techniques to gain more understanding of human factor issues in

an inexpensive and time efficient manner. It is suggested that cognitive walkthroughs would be a

good complement to HITL simulations that aim to explore concepts. A walkthrough would offer the

most value if it were used as a precursor to a HITL study, because its findings could inform the

procedures, rules and functionality that are to be tested in the simulation.

E.7 References

Endsley, M. (1996). Automation & situation awareness. In R. Parasuraman & Mouloua, M.,

Automation and Human Performance, pp163-182. New Jersey: LEA.

Erzberger, H. (2001). The Automated Airspace Concept, Proceedings of the 4th USA/Europe Air

Traffic Management R&D Seminar, Santa Fe, New Mexico.

Homola, J. (2008). Analysis of human and automated separation assurance at varying traffic levels.

Masters Thesis, San Jose State University, San Jose, California.

Joint Planning and Development Office (2007). Concept of Operations for the Next Generation Air

Transportation System, Version 2.0, Washington, DC.

Jeffries, R., Miller, J.R., Wharton, C. & Uyeda, K. (1991). User interface evaluation in the real

world: a comparison of four techniques. In Proceedings of the SIGCHI conference on Human

factors in computing systems: Reaching through technology. New Orleans, Louisiana, pp: 119

– 124.

Polson, P.G. & Smith, N. (1999). The cockpit cognitive walkthrough. In the Proceedings of the

Tenth International Symposium of Aviation Psychology, OSU, Columbus, Ohio, pp: 427-432.
118

Polson, P.G., Lewis, C., Rieman, J., & Wharton, C. (1992). Cognitive walkthroughs: a method for

theory-based evaluation of user interfaces, International Journal of Man-Machine Studies, 36

(5) 741-773, May.

Prevot, T., Homola, J., Mercer, J., Mainini, M., & Cabrall, C. (2009). Initial evaluation of NextGen

Air/Ground Operations with ground-based Automated Separation Assurance. Eighth

USA/Europe Air Traffic Management Research & Development Seminar ATM 2009, June,

Napa, California, USA.
119

Addendum to Appendix E. Detailed Analysis of Questions

Asked of Human Subjects in Walkthrough

This addendum to Appendix E discusses the answers to questions asked of the subjects of the

walkthrough in detail.

To illustrate how the questions were developed for the human performance issues, Table E.2 shows

examples of questions asked for several human performance issues related to the use of Automation

and Decision Support Tools (DSTs) for ASA. It is noted that the cognitive walkthrough questions

were written to address a specific event in the video used in the walkthrough, so the questions in

Table E.2 may seem out of context.

Table E.3. Sample Questions Asked for Use of Automation and Decision

Support Tools for ASA

HF Design and Performance issue Specific Cognitive Walkthrough Question

Trust of the ASA Manager in the ASA

automation and DSTs

Having noticed that the automation hasn’t

seen this as a conflict, would this lead a

controller to believe there’s some info he

doesn’t have that the automation does which

makes this not a conflict?

Consistency of the interactions between the

ASA automation, ASA Manager DSTs, (and

Controller DSTs under mixed equipage)

How many times (or how long in time) should

the automation attempt working with a noncompliant aircraft before handing it off to the

controller?

Needed interactions between the ASA

automation, ASA Manager DSTs, (and

Controller DSTs under mixed equipage)

What would you do if the aircraft did not turn

quickly enough or did not turn at all?

The design of human interactions with

automation, DSTs, and displays

Is it too much work to impose on a controller

to enter the information into the system after

he has issued a maneuver to an IFR aircraft;

is there an easy/conceivable way to facilitate

this?

Detecting and recovering from human errors or

system failures

Is this a sufficient flag to let the ATC know that

the conflict is going to happen because the

resolution maneuver hasn’t occurred yet?

Presentation of information to the ASA

Manager when s/he needs to step in and act

For this situation, would you prefer a canned

solution or the freedom to pick and choose a

solution to put SWA back on route?

For the cognitive walkthrough, at least one question was generated for each step or event that the

participants observed in the video. The questions were developed for the human factor themes

listed in Table E.1 in Section E.4.1. Table E.3 shows the breakdown of how many questions
120

addressed each theme from the total of 136 questions that were asked during the walkthrough.

The right column shows how many of these 136 questions provided particularly useful

information that was analyzed in Appendix E. The total number of questions that yielded

particularly useful answers was 56.

Table E.3. Number of Question by Human Factors Themes

Human Performance Factors in

NextGen (see Appendix B)

Number of questions asked Key questions for analysis

Attention 6 2

Monitoring 8 3

Situation awareness 5 3

Decision making 10 7

Time pressure 5 3

Workload 7 4

Task management 12 5

Communication 11 5

Memory 4 4

Roles 5 3

Responsibilities 12 4

Coordination 6 4

Interaction with Automation 8 3

Trust 4 2

Training 2 1

Procedures 10 3

Miscellaneous questions 21 0

The following subsections of the Addendum to Appendix E discuss the answers of the human

subjects in the walkthrough to the 56 questions with particularly useful answers. The 56 questions

are grouped into the five categories of human factor questions used in Section D.5 of Appendix D.

The five categories are presented in the order in which they are presented in Section D.5.

Addendum E.1 Answers Related to Use of Automation and DSTs

Under ASA, the ASA Manager will interact with automation far more than s/he does currently. This

automation may operate at one or more levels, with the human-automation function allocation

determined by this. Automation should be designed by taking a number of human usability criteria

into account. Three of these usability criteria are not producing false expectations, natural

interaction, and trust.

• Level of automation: Participants seemed fairly happy with the level of automation presented.

However, when asked about the automation taking on more tasks, participants did not always

think that this would provide useful assistance. They rejected a suggestion that the automation
121

provide a list of possible solutions for them to choose from. They did not want to enter into

any kind of negotiation with the automation and stressed, on more than one occasion, their aim

is to solve a problem and move on, especially under time critical off-nominal events. In this

vein, they wanted the automation to pick a solution and execute it: “they [automation] have

their [decision] and you agree with it, if not, you go to yours.”

• Allocation of function: When given the option to have automation sense controller initiated

clearances or to have to type these in themselves, some participants said it would be good to

have the automation do this as long as it was accurate. Others preferred to enter clearances

themselves for two reasons, one is that you should complete a procedure you started and see it

through to the end, and the second is that it maintains your situation awareness.

• Not producing false expectations: When a participant identified a conflict that the automation

had not flagged, they did not think that the automation had information that they did not, rather

participants thought they were more aware than the computer and had anticipated (and often

solved) the conflict before the computer flagged it.

• Automation transparency: The automation often did not implement the clearance that our

participant controllers would have chosen. Often this was due to the participant looking at the

wider circumstances where the automation focused on the pair of aircraft in conflict.

However, as the automation will be predictably less optimal, it is likely that ASA Manager will

learn during training when they are likely to disagree with the automated solutions and why.

• Natural interaction: Participants liked the functions available through the automation. They

liked the  data-block icon that allowed them to pull up the aircraft’s route and they thought

the click-and-drag reroute tools were both easy to use and very useful.

• Trust with automation: One reason participants gave for not trusting the automation was a lack

of time. If LoS were imminent, there was no longer enough time to wait to see if the

automation could generate a good resolution. Participants would act by giving a clearance

they knew would work (to achieve a degree of certainty). Participants were aware that a

downside of not trusting the automation was thinking that they needed to check every

clearance that the automation was giving. They could not do this, so they needed to trust the

automation to a certain level for their partnership with the automation to work. From their

responses, it seemed that for any one clearance/resolution the participants would trust the

automation initially, but if there was a problem with the clearance, they would not risk trusting

the automation a second time for that same problem.

• Comfort with automation: In regards to comfort with automation’s clearances in general, the

participants were divided in their views. Half the participants indicated that they would have a

predisposition to review, alter, or issue different clearances from the automation’s decisions

and half said they could easily accept the automation sending clearances that the controller

would not need to be made aware of or be able to review.

Several factors were brought up by the subjects related to the roles of human operators and

automaton and to information provided by automation.

• Being at the “sharp end”: Participants were clear that they assumed they were responsible for

the separation of all of the aircraft in their sector. Even when shown an example where the

automation failed to detect the conflict until the last minute before LoS, participants still

thought the controller was responsible for maintaining separation; although they did note that

a one minute warning is far too little for this to be a fair responsibility, and herein lies the
122

problem. Could a controller really be held responsible for a LoS if s/he had no knowledge of it

until a minute before it happened? This would suggest that future SA concepts will need to

change the allocation of responsibility in a sector controlled by humans and automation.

• Final separation authority: The way the ASA concept was presented in our walkthrough

indicated that the automation was responsible for separation under normal conditions, but as

soon as conditions became off-nominal, including whenever the SSAT was invoked, the

controller was required to become involved and therefore responsibility shifted to them.

Participants agreed that they were responsible for separation from the time they were required

to step in until the aircraft was back on its original route, but again, the issue of being given

“fair warning” was raised.

• Clear roles: Participants clearly see their role currently as a provider of separation assurance

and a facilitator for aircraft in their sector. While the addition of automation makes some roles

and responsibilities less clear, there were one or two areas where participants had definite

opinions. If an aircraft contacts you, even if you are the wrong person to talk to, you

automatically become responsible for that aircraft. So, your role changes to that of responsible

controller. Participants did not ever want to be forced into the role of observer by the

automation denying them the option to act, they always wanted to be able to step in if they saw

a need.

• Readiness for adverse events: Controllers want to know as much as possible about any event

where they might have to act. Because of this, participants indicated they would want as much

information as possible about a conflict where the data-blocks were brought up to their fourline version. Participants did not feel there was a contradiction, or grey area, if they were told

about a problem they did not later need to solve, they just wanted to be prepared for an

occasion where they did have to step in.

Addendum E.2 Answers Related to Decision-making and Workload

For separation assurance, the ASA Manager has to determine when to intervene in a situation that is

arising and what to do if an aircraft deviates from the planned route.

• When to intervene: Participants always opted to step in to assist in the off-nominal events

presented. Usually controllers want to solve a conflict as soon as they can to keep their

workload manageable so that tasks do not pile up. All participants said they would step in at or

before three minutes to a LoS whenever they are able. In one event, the controller (from the

AOL study) did not act prior to three minutes. Participants gave various reasons for him/her

not acting as soon as the conflict was identified by the automation, ranging from negligence to

a confusing display.

• Actions if aircraft deviate: Participants differed from the automation in their opinion of

whether an aircraft was deviating from the last mutually-agreed upon clearance. In short, they

considered that until a pilot physically accepted a clearance, the previous clearance/flightpath

held. In contrast, the prototype automation system considered the most recently issued

clearance to be the valid one. If aircraft did not comply with necessary clearances, participants

described a variety of ways they could try to get in contact, including enlisting the help of the

air carrier’s AOC or the military.
123

In general, controllers make decisions based on their mental models of the problems they see. Most

often they will apply a rule-based decision making process, to find the first good solution in the time

they have, including both predicted probability and consequences as key factors.

• Mental models: Participants were able to predict when they saw the maneuver that the ASA

tool issued whether it would resolve the problem or not – from approximately four minutes

before LoS. This implies controllers were referring to a schema where they could run the

problem forward (level 3 SA; Endsley, 1996) to see if the solution would solve the problem.

In one example, participants predicted that the automation’s actions were LoS prevention but

not a resolution, and that it would only buy time until a second set of (more complex)

resolution clearances could be issued. Participants were remarkably consistent in their

suggested solution strategies, especially as aircraft were moving closer to a loss of separation,

again suggesting that they were referring to a finely tuned mental model for their analyses of

events.

• Key factors: Participants consider factors from the wider situation in their decision-making.

For imminent events, they consider the density of surrounding traffic and whether the aircraft

has any other issues (like a communication problem) in their choice of resolution. When there

is more time, participants consider which aircraft have been in the air longer, who is due to

make a maneuver or begin a descent, and the flight paths of the surrounding traffic.

• Rule-based decision making: The consistency between participants’ solutions indicated that

they worked within a set of parameters to make their choices about resolutions. They always

had a range of “plan B” options, although they described how these options would become

more and more limited as the time to loss of separation grew nearer. For example, for closer-in

conflicts, participants indicated they would choose vertical (altitude change) solutions rather

than the automation’s lateral solutions, which meant they almost always disagreed with the

automation in these off-nominal events.

• Satisficing: Participants, with no reference points aside from the aircraft movements

themselves, were very accurate judges of whether a resolution would separate two aircraft.

When issuing solutions, they described “rounding up,” or adding a few degrees to, turns they

issued just to add some space for maneuvering. While at first this seems inefficient, talking

through the events we presented showed that this strategy allowed for events to unfold less

smoothly than hoped and the resolution remained valid. For example, the flight deck could

take longer than anticipated to execute the maneuver but still achieve the required 5nm of

separation. The automation, however, calculated solutions to exact tolerances. These were

more efficient solutions but they were only good if the target aircraft complied as soon as the

clearance was issued. If the aircraft did not comply quickly, the automation needed to recalculate and re-send resolutions, usually increasing the severity of the turn and creating a

degree of confusion for the observer.

• Time pressure: Participants were able to estimate whether they had enough time to solve a

conflict, when their ability to do that would become marginal, and when they needed to act

quickly for any solution to be put in place (e.g., “It’s getting to the point of no return”). They

were also able to identify which types of solution maneuvers could still be executed in the

remaining time and which were no longer possible. Participants also said they change the way

they execute a clearance as time grows shorter, using voice to issue their clearances. Their

reasoning was that voice is faster, although most of the participants conceded that data link

may become faster in the future and, if it became as fast as voice, it would be a better medium

because it is cleaner (e.g., no static, no step-ons, no miss-hears).
124

• Prioritization of tasks: An indication of a clear mental model is the ability to appropriately

organize tasks. Participants demonstrated this, as they agreed about the order in which a series

of control activities had to be completed to resolve problems. For one event, they described

prioritizing as putting a free-track aircraft back on its route before climbing another to altitude.

Three workload human factors issues were identified for ASA that focused on an ASA Manager’s

“busyness” and their attention.

• Underload: Participants agreed it is good to have something to do when they are on-position.

They did not consider that administrative tasks, such as entering routine information into the

computer, would add adversely to workload. One participant commented that “housekeeping”

tasks are good because they keep you focused on the scope and keep your attention on the

aircraft traffic.

When their sector is not busy, participants described that they would scan their space, but also

use the time to check weather, to look up information about different aircraft, and to gather

information that they do not have time to review when they are solving conflicts. They might

also “reach out” to help other controllers around them if they are busy.

• Prompting: Participants welcomed the possibility that the automation would prompt them

when they needed to act, but the prototype system we used is not ready to do this because, in

all of the off-nominal cases in the walkthrough, the participant identified problems before the

automation. For most participants, a full data-block was enough initial information to begin to

work on a problem.

• Off-nominal and attention: The problems that could not be solved by the automation were

complex and time-critical by the time they were flagged to the controller. Participants

admitted that under some of these situations, they would be focused on one area of their scope

and paying very little attention to other areas—relying on the automation to detect and solve

these more peripheral problems while they were absorbed by the off-nominal event.

General mental workload issues, that can affect performance, include the transition from low to high

workload and the type of workload that is incurred.

• Workload transition: Although the load on an en-route ASA Manager can rapidly change from

low to high workload as an off-nominal situation pops up, often our participants identified an

upcoming problem before the automation flagged it and commented about actions s/he would

be taking prior to the situation developing to defray its impact.

• Workload types: All three types of workload are present for an ASA Manager: busyness,

emotional stress and cognitive difficulty. Participants described emotional stress arising when

they anticipate a loss of separation between aircraft, describing controllers as “distracted.”

Situation complexity was noted by all participants as a determinant of workload, but also that

its limits vary from person to person. Participants did not consider that they would be too busy

to manage most of the situations presented to them. They were reluctant to say they would

ever be at maximum workload, instead describing their attention management strategies as

narrowing to focus solely on the main problem occurring in their sector.
125

Addendum E.3 Answers Related to Attention, Situation Awareness, and Memory

Attention allocation is the cornerstone of the central controller activity of monitoring to maintain

situation awareness (SA). Controllers can voluntarily focus attention on key events or may have

their attention “grabbed” by unexpected cues.

• Primary focus of attention: Participants described developing scan patterns to suit the sector

they are working. In all but the most complex off-nominal cases, they would maintain a

periodic scan of their sector while they were dealing with separating specific aircraft. Once

they put a solution in place, participants described moving back to a general scan but finishing

every sweep by checking the progress of the resolution.

• Conscious allocation of attention: Controllers are pattern recognition experts and part of their

skill lies in an intimate familiarity of the sectors they work. Because of this, controllers know

which areas of their sectors to watch more closely, and aircraft actions that do not conform to

normal patterns are very salient cues. Often sectors have “hotspots,” where corridors cross or a

departure flow is climbing to altitude, and participants did not like aircraft popping up here in

conflict with aircraft in cruise (level flight).

• Involuntary attention “grab”: For participants, a handful of cues concerning aircraft speed,

angle of closure and proximity, are significant and thus attention “grabbing” because they

signal a rapid narrowing of solution options.

Three monitoring issues were identified for ASA: monitoring a conflict probe for potential

separation violations, monitoring expected routes/traffic flow, and monitoring of resolution advisory

compliance.

• Automated conflict detection: All participants indicated that prompts toward potential conflicts

would be a controller’s top priority. Namely, the presence of other conflicts would redirect

their attention and essentially negate their monitoring of other situations/tasks.

• Traffic flow: In terms of monitoring expected traffic flows or routine routes of aircraft,

participants strongly emphasized their attentiveness to such patterns. Again however, active

involvement with aircraft deviating from these routes was expected to be dropped if potential

conflict situations arose. Participants indicated that the observation of a clearance issued to

regain an intended or expected flight path rank relatively low(er) in importance.

• Resolution advisory compliance: While one controller answered that he would continue to

monitor a clearance issued as a resolution until he confirmed visual vertical separation, all of

the other controllers indicated a need for active monitoring until visual lateral separation was

achieved on the scope. A J-ring-like visual cue is important for depicting accurate distances.

In sum, the monitoring of aircraft for conformance after a clearance is issued is necessarily

dependent on the reasons for the clearance and the activity level of other potential conflicts. (A

general flat rule, for controllers or automation, as to how long an aircraft requires monitoring after a

clearance would be impractical and contrary to a controller’s intuitions.)

A controller has to monitor situations of both volition and demand.

• Voluntary: Here controllers’ behaviors are expected to show the most variance in terms of what

specific tasks they choose to engage themselves in. Factors observed from participants’

answers included comfort with perceived distances, familiarity with traffic patterns in the

sector, and amount of reliance on memory aids.
126

• Demanded: Here controllers’ answers showed great consistency, indicating a shared

expectation both for events that would grab or otherwise force their attention and for what they

would do in these situations.

Controllers have to have a broad understanding of activity in their sector and a “big picture” of what

is happening. For example, a controller has to be generally aware of traffic flows in his/her sector

and route non-conformances.

• General awareness: To maintain a general awareness of their sector, participants felt they

should have as much information as is available at all times (e.g., “more information for the

controller is better”) so, for example, all conflicts should be flagged. Controllers may also

search for more information before they take an action. Automation functions to dig deeper

into data should be available. Having the full data-block brought up for a pair of aircraft in

conflict is desirable because even if the controller does not intend to solve the problem

immediately, the data-blocks serve as a reminder that s/he has to come back to this pair. One

SA aspect that not all participants supported was whether to be told as much as possible about

what the automation is doing. If participants felt information would make them responsible for

checking the automation, they felt they would become overloaded; if the automation provided

information just for awareness, this was acceptable.

• Awareness of non-conformance: The automation indicates aircraft non-conformance by

showing the airplane icon as a free-track (enlarged and blue). However, the underlying

message is often, but not always, that the aircraft remains in conflict with another, because the

maneuver was issued to solve a potential conflict. Participants thought that an indication that a

conflict was solved before it actually was is poor information (e.g., “A conflict should not go

away until an airplane has acknowledged the clearance”). Participants did not feel there would

be confusion because ASA Manager training would make such peccadilloes clear. ASA

Managers would learn, and therefore be aware of, what these indications mean, but training to

compensate for a bad interface is not a good approach to tool design.

• Providing awareness to others: As they have an overview of their sector, controllers are able to

pass much information onto flight crews. However, participants indicated that they would not

give lengthy explanations. Instead they favored quick and efficient interaction. In some cases,

certain phrases have implications that controllers do not want to invoke unless it is appropriate.

“Calling traffic” is one of these phrases. Participants said that they would not “call traffic” too

early, before a situation became time critical, because the received meaning (by flight crews) is

that a radical maneuver is required, not just the standard turn (or climb).

In ASA, ASA Managers have to use their retroactive and their prospective memories to control

aircraft. In general, under NextGen, they will also be relying on the automation to assist with cues

and reminders.

• Prospective memory: If the controller has too many tasks to complete, s/he will have to

prioritize tasks and remember to come back to those that have a lower priority. Participants

described that they would set themselves up a visual cue as a reminder. Usually, the controller

will leave the data block expanded to its four-line readout or, as an extra reminder that the

aircraft needs a new clearance, will extend the distance of the data block from the aircraft icon.

• Retroactive (long term) memory: Because air carrier schedules are regular and aircraft fly

basically the same routes every day, controllers become familiar with the paths of the aircraft
127

that fly through a sector during on-the-job training and working it regularly. If an aircraft has

been vectored off its route for separation, participants said if they were trained on a sector

(familiar with it) they would be able to remember the route to put an aircraft back onto.

• Human-plus-machine memory: Participants indicated that they would have certain

expectations about the information that the automation is showing them and would use that

information for their decision making. Because of this, it is important that the automation

actually shows the controller the information they are expecting to see. For example, when

asked whether they want to see the route that the automation has proposed for the aircraft or

the route that the aircraft is currently flying, participants chose the latter. The automation

should not assume that an aircraft will comply and change the route it is showing for an aircraft

until it has confirmation that the aircraft has changed its trajectory onto the new route.

Addendum E.4 Answers Related to Communication and Coordination

Three communication issues were identified for ASA: communicating under non-urgent situations,

communicating under urgent situations, and communicating with IFR aircraft.

• Non-urgent communication: Interactions with other controllers was not anticipated to be

different from today (for both routine communications and urgent information). All

participants talked about face-to-face discussions with controllers of surrounding sectors and

verbal updates through teleconferences to keep the controller network informed. Participants

were happy to use data-com as their primary means of communication with flightdecks.

• Urgent communication: Participants described that they would reserve verbal communication

for occasions when they regarded it necessary for separation. Their choice about when to

switch from data-com (DC and other automated messages) to voice seemed to be based on

their confidence (trust) in the system, but all would opt to use voice when resolving “close-in”

situations—where there is three minutes or less to a loss of separation.

• Communication with IFR aircraft: Participants were comfortable using the radio for

communication with aircraft under their sole control (not being co-handled by the automation),

and were aware of what this entails because they factored this into their workload

considerations (see below).

An ASA Manager is likely to communicate with three entities: other ASA Managers and

supervisors, pilots, and the tools they are using. Communication reduces uncertainty but can be

compromised by noise, equivocation, and/or redundancy.

• Mode of communication: As described above, participants envisioned using data-com

primarily to talk to flightdecks and verbal communication to talk to other ASA Managers.

Participants wanted as full and complete information as possible to be displayed by the

automation. In one of our events, the automation initially only showed one aircraft in a

conflict pair, participants cited this as an example where they needed more information and the

automation obviously had information that it was not displaying. They understood that for the

automation to function correctly they would have to input any clearances they gave verbally,

and did not feel this was an undue burden; it is just part of their job.

• Properties of information transfer: Participants were copacetic with having very little

information about aircraft in surrounding sectors and aircraft within their own sector being

separated by the ASA tools. However, as soon as an aircraft required their intervention,

participants wanted as much information as possible to become available about the aircraft
128

concerned (e.g., “you don’t keep secrets”). In these cases, they saw no problem with receiving

information from another controller that was also displayed by the automation.

Addendum E.5 Answers Related to Organization, Selection, Job Qualification, and

Certification Factors

In general, roles and responsibilities should be clear, one person should have final authority under

both normal and abnormal conditions, and these should be defined through procedures.

• Procedures: In the current day system, procedures for interacting with others are clearly

defined, e.g., controllers coordinate if they solve a problem close to a boundary, issuing

“traffic” is an indication that a LoS is imminent, etc. In a future system, procedures also need

to be defined with respect to the controller-automation interaction. As one example, a

participant noted that he would want a function that would let him see what kind of solution

the automation is working toward issuing, and if it is trying to issue a solution. If this

information were available, controllers might be able to judge whether they need to step in

earlier in an off-nominal situation.

• Off nominal procedures: One aspect of defining procedures for off-nominal situations is when

the automation should inform the controller that their assistance is required. Most participants

indicated they wanted a warning three minutes before the LoS with as much information as

possible. In other situations, where a resolution has been issued and the aircraft is not

receiving it, participants indicated that the automation should flag a non-response to them after

a minute.

• Relative responsibilities: Participants agreed that they could not oversee all the aircraft in their

sector in the examples we presented to them. In this case, they thought the automation should

be responsible for all the dimmed-out aircraft that it was separating (e.g., “In this future world,

there’s a complex relationship between man and machine!”). Given that working in a sharedresponsibility environment is a very different mind-set from today, this will be a key retraining

issue and will require very clear delineation of responsibilities by concept developers.

• Clarity of presentation of procedures: Two participants spoke directly towards the need for a

clear-cut “black-and-white” mandatory intervention requirement, and a third related indirect

mutual agreement by explaining how he would abide by a steadfast formula for action

(although he self-defined his own specific parameters).

• Off-nominal situations: When challenged with a situation of heightened severity, most

participants indicated that less than three minutes was too close a time horizon to reasonably be

expected to take action if their workload was anything above minimum.

A general change in job expectations requires consideration of the type of person selected and

trained for the position of a controller in NextGen.

• How a “controller” is born: Participants were equally mixed in reporting whether their

reliance on knowledge of a standard turn rate (e.g., 3 degrees per second) and other such

expectations on aircraft compliance times came from specific training or from their general onthe-job experiences. Four participants highlighted the prevalent importance of localized

knowledge (e.g., sector flows, pilot arousal, etc.) in shaping what they would expect of aircraft

under their control.
129

Appendix F: Airspace Super Density Operations

Airspace Density Operations (ASDO) enables high-efficiency trajectory-based operations, that are

robust to weather and other disturbances, to meet NextGen demands in super-dense and

regional/metroplex airspace while minimizing environmental impact. This includes the

requirements to: (1) develop technologies to simultaneously sequence and de-conflict aircraft

leading to trajectory management for aircraft in terminal and extended terminal (as necessary)

airspace; (2) develop precision spacing and merging capabilities to reduce workload and spacing

variance between aircraft in terminal and extended terminal airspace; and (3) develop methods for

optimizing ASDO resource utilization among interconnected airportals.

F.1 Overview Description of ASDO

The following are basic elements of ASDO that are assumed for this identification of human tasks

and human factor issues. It is noted that this identification and characterization of human factor

issues is derived from addressing ASDO only for arrivals, not departures.

• Required Navigation Performance (RNP). RNP is a statement of the navigation performance

possessed by an aircraft necessary for it to operate within a defined airspace. RNP equipped

aircraft can more precisely fly defined trajectories and can safely operate routes with less

separation than previously required.

• Continuous Descent Approach (CDA). CDA is a method by which aircraft approach airports by

flying a constant degree angle of descent (e.g., 3 degrees) until near the runway (e.g., meeting

the Instrument Landing System). This allows aircraft to descend at their optimal vertical

profile with reduced fuel burn and noise.

• Flight Deck Merging and Spacing (FDMS). FDMS delivers accurate, low-variance spacing for

merging and arriving aircraft and reduces the need for air traffic control (ATC) interventions.

During CDA operations, FDMS helps maintain capacity by optimizing the overall spacing

among a stream of aircraft. Note that the pilot is responsible for maintaining spacing with the

aircraft in front, but this is different from separation responsibility.

• Flexible Arrival Routes. Dynamic entry points for terminal airspace and flexible arrival routes

will help maintain throughput when weather patterns pass through the terminal airspace. The

flexible routes require dynamic fixes and corridors for arrivals and departures that can be

relocated in response to weather.

• Closely Spaced Parallel Runways (CSPR). CSPR increases airport capacity through the use of

paired operations on very closely spaced parallel runways.

• Planning Interactions with Surface. Planning interactions between terminal arrival paths and

surface operations coordinate terminal airspace capacity with surface capacity.

• Metroplex Operations. Planning trajectories for multiple airports in a metroplex will deal with

interdependencies of, and provide for, operations at multiple airports in a metropolitan area.

• Better Wake Vortex Handing. Improved knowledge of wake vortex will allow trajectories to be

planned that allow less separation due to wake vortex.

For the purposes of this paper, the number of ASDO elements to be analyzed for human

performance issues identification has been down-selected to operations that involve RNP routes on
130

CDAs, some of which are capable of FDMS. These components are highlighted by the JPDO as

achievable in the near- to mid-term timeframe.

In these types of operations, aircraft are assigned a 4D arrival profile at the top of descent. A local

scheduler with pre-defined time window determines when an aircraft can enter super density

airspace. The 4D arrival profile follows a CDA or other optimized profile descent to maximize fuel

efficiency and minimize noise. During the descent, the aircraft equipped with ADS-B and flight

deck automation/displays can merge and space behind another equipped aircraft and reduce the

inter-arrival spacing buffer by actively managing the spacing interval at a tight tolerance.

F.2 Summary of Required Technology for ASDO

Compared to other RFAs, ASDO makes relatively low technology assumptions for nearer-term

implementations. For example, CDA, and RNP routes are possible with advanced Flight

Management System (FMS), RNAV, and Automatic Dependent Surveillance-Broadcast (ADS-B)

for precision navigation and surveillance – and all are available today. FDMS can be achieved with

ADS-B and advanced cockpit displays and automation. CSPR operations with better wake vortex

handling require RNAV/RNP capability and wake sensing and prediction technology. In addition,

advanced decision support tools, net-centric flight information exchange, and probabilistic weather

forecast products are likely to be needed for far-term applications.

Data link is assumed for high altitude airspace where the initial descent occurs for arrivals. Data link

allows for flexible routes that can be constructed and uplinked to the flight crew without prior

published routes. Data link should also be useful in the terminal airspace to dynamically construct

4D trajectories but it might not be available in the mid-term timeframe.

F.3 Summary of Human Roles for ASDO

Before defining the human roles in ASDO, some additional assumptions about the concepts need to

be added. First, ASDO airspace is not expected to be fully automated, inferring that controllers will

be aided by DSTs to manage the traffic. Some of the functions of the ground-based DSTs are the

ability to monitor conformance of aircraft to their assigned 4D trajectory and provide alerts during

trajectory deviation and provide potential resolutions to get the aircraft back in conformance.

Terminal airspace boundaries extend farther away from the airport than today, creating airspace with

reduced separation standards of terminal airspace (i.e., 3 miles lateral separation) farther away from

airport compared to en route airspace separation standards (i.e., 5 miles lateral separation).

Although there are a number of human operators involved in ASDO operations, three human roles

are selected and described below.

1. Traffic Manager: manages the arrival and departure routes. Currently these routes extend

beyond terminal airspace but may fall exclusively in the terminal area if its boundaries are

extended. The “Traffic Manager” described here may have different names and may span

multiple positions depending on the situational context and the facility. For example, there

may be a separate Arrival Flow Manager and a Departure Flow Manager who focus solely on

the arrival or the departure stream. For the purpose of this section, we will use a generic

Traffic Manager term for this function.
131

2. Controller: manages arrival and departure flows in and near the terminal airspace. Their main

tasks include separation management, maintaining throughput, and monitoring conformance

to 4D trajectories along RNP requirements and CDAs. They may also support ground-based

separation assurance and FDMS, as well as dealing with aborted approaches, dealing with

anomalies, assisting aircraft in changing trajectories in response to weather and other factors,

working with pilot and Traffic Manager to return an aircraft that deviated for some reason to

the arrival stream, etc.

3. Pilot: reviews and executes 4D trajectories that come via data link. S/he manages spacing

and/or separations during FDMS and performs CSPR operations when appropriate.

Before listing operator roles and responsibilities for ASDO below, it is noted that the

functionalities listed in Section F.1 for ASDO are a collection of capabilities that could be

implemented independently. One or more could be deleted from the list, as well as some not

listed could be added.

The roles and responsibilities hypothesized for these three operators are listed in the follow

sub-sections.

F.3.1 Designated Traffic Manager Roles and Responsibilities for ASDO

In this concept, the Traffic Manager is hypothesized to assess the forecasted demand and capacity to

an airport or a metroplex to set dynamic fixes, corner posts, arrival/departure routes and

corresponding airspace. This position (or positions) is critical with dynamic flexible routes

envisioned in the far-term ASDO concepts but plays a role in the mid-term as well.

The following are tasks hypothesized to be performed by the Traffic Manager:

1. Use Decision Support Tools (DSTs) to set airspace characteristics.

With the help of DSTs, the Traffic Manager can evaluate which set of airspace characteristics

will provide the best efficiency and throughput while minimizing the traffic complexity. S/he

can set the aircraft equipage levels on various routes (e.g., segregate aircraft streams based on

equipage levels or run mixed streams), determine fixes and corner posts, accommodate

different airspace configuration based on weather/wind information, and enable/disable CSPR

operations depending on the traffic and weather situation.

2. Use Decision Support Tools (DST) to plan 4D trajectories for arrivals and departures.

The Traffic Manager can plan arrival and departure routes based on various factors such as

forecasted traffic, airport capacity, weather, etc. With the help of DSTs, Traffic Manager can

evaluate and plan the best arrival/departure trajectories for a given look-ahead window (e.g.,

up to 2 hours).

In the mid-term timeframe, the trajectories are likely to be a pre-determined set of flexible

routes (FAA, 2007). An alternative to the pre-determined routes may be routes planned by the

DSTs that are automatically sent to aircraft via data link. Such implementation is being

explored in oceanic and domestic airspace. The routes may be planned individually or as a

part of a flow. If the DST assists in planning the routes, the Traffic Manager should have the

option of reviewing the trajectories planned by the DST and modify them if necessary.
132

3. CSPR operations.

The Traffic Manager may initiate and end CSPR operations as conditions warrant. S/he may

also use the CSPR DST (e.g., runway sequence with estimated arrival time) to pair the

aircraft. Terminal trajectories for the CSPR are expected to be planned via automation and

sent to the aircraft via data link. Tasks for CSPR operations are similar to those listed in item

2 above, “Use decision support tools (DST) to plan 4D trajectories for arrivals and

departures”.

4. Conduct any re-planning needed following anomalies.

The terminal controller will deal initially with any anomalies and will interact with Traffic

Manager for any re-routing needed following an anomaly.

F.3.2 Controller Roles and Responsibilities for ASDO

The Controller may be performing the functions below for more than one airport, if a metroplex is

involved. Also, there may be more than one Controller for an airport or metroplex, depending on the

level of traffic and workload.

The following are tasks hypothesized to be performed by the Controller in the ASDO operations:

1. Awareness of various ASDO operations at the beginning and during the shift.

Depending on what ASDO operations, routes, equipage and airspace have been instantiated

during a given time frame, the Controller needs to be aware of the current and future status

and characteristics of the airspace that they are managing. If the operations change during

middle of a shift (e.g., FDMS and CSPR are activated), the Controller needs to be conscious

of changes in responsibilities.

2. Maintain separation and perform other nominal Controller tasks.

The Controller maintains separation with the help of ground-based separation assurance DST.

During FDMS or CSPR operations, the separation tasks may be delegated to the flight deck.

Ground-based DSTs are expected to assist with conflict detection and resolution, as well as

finding speed/route solutions to deliver aircraft on schedule near the airport. As the aircraft

approaches the airport, some aircraft equipped with FDMS capabilities may switch to selfspacing to provide better spacing accuracy.

3. Monitor and maintain conformance to the 4D trajectories.

Unlike current day operations in which aircraft deviate often from their flight plans, aircraft

are expected to fly along 4D trajectories with pre-determined RNP precision. Keeping aircraft

along the trajectories will likely reduce Controller workload in keeping aircraft safely

separated while being off-trajectory but there is likely to be an increase in workload for

monitoring the conformance of the aircraft with their assigned 4D trajectories. Ground-based

DSTs are expected to be available to assist Controllers to monitor the conformance and in

returning a non-conforming aircraft to its 4D trajectory.

4. Perform CSPR operations.

The terminal Controller identifies aircraft that can be paired for CSPR operations with the

help of DSTs that predict the arrival schedule and the potential pairs of aircraft. Once the

aircraft are in a proper location where they can be paired, the Controller engages the CSPR

operations by assigning the lead and the following aircraft with proper spacing. As the pilots

are flying the CSPR approaches, the Controller monitors the situation.
133

F.3.3 Pilot Roles and Responsibilities for ASDO

In general, pilots are expected to fly aircraft along 4D trajectories. Whenever they receive a new

planned trajectory via data link, they review and execute the route. In a flexible routing environment,

the new trajectories may involve different corner posts, meter fix positions, flight corridors, etc.,

than a previous flight into the same airport or differ from those expected at the start of a flight.

If an aircraft is engaged in FDMS, the pilots maintain spacing with the aircraft in front. Whenever

the aircraft cannot continue the FDMS operations, pilots alert the Controller who then needs to take

over the spacing operation.

For CSPR operations, the pilots accept or reject participating in the CSPR operation when notified.

If they accept to fly the CSPR, they identify the lead aircraft and fly the CSPR operations with

proper spacing with the help of the flight deck DSTs that provides feedback on both current and safe

spacing from the lead aircraft.

F.4 Human Factors for ASDO

We have previously discussed that we take two perspectives to identify and characterize human

performance issues for the individual Research Focus Areas: one is the perspective of the human

performing the required tasks and the other the perspective what the tasks imply for the human

operators. Section F.4 addresses the perspective of the human performing the tasks by examining

the 9 factors of human performance presented in Chapter 3 (e.g., attention allocation, memory,

workload, etc.). Section F.5 will take the perspective of what the tasks ask of humans using the

human performance-related questions from Appendix C (e.g., who is responsible between

automation and human for a particular task? If automation DST fails, will human operators

recognize it & recover? Can controllers perform all the functions they need to? etc.).

The human tasks in Section F.3 were examined for the ability of operators to perform the tasks, both

from a safety consideration and with respect to performing the tasks in a manner that achieves the

benefits of ASDO. The human performance issues identified are described below for the Traffic

Manager, controller, and pilot.

F.4.1 Attention Allocation

Under ASDO, as under many of the new concepts that will be part of NextGen, there will be a

greater variety of tasks to be performed by all human operators, resulting in more Attention

Allocation, Situation Awareness, and Monitoring issues. Attention strategies must be designed

based on the relative importance and expected frequencies of various operational events. A model

of such events in a typically busy ASDO period could serve as the basis for developing attention

strategies and training programs for traffic managers, controllers, and pilots.

The Traffic Manager must continually monitor weather with a concern for the need to divert aircraft

or reconfigure the terminal airspace, and the general flow of aircraft into and out of the terminal

airspace must also be monitored. The traffic Manager must also monitor operational elements for

which s/he needs to consider before setting the criteria, including arrival fixes, needs for metroplex

diversions, and CSPR or CDA arrangements.
134

The extent of attention allocation, situation awareness, and monitoring performance issues for the

Traffic Manager will depend on several factors: level of traffic, which will influence the level of

general activities, during ASDO; the extent of functionalities that are part of ASDO, such as CDA,

FDMS, CSPR operations, Better Wake Vortex Handing, and Metroplex Operations; and whether the

Traffic Manager has any duties other than dealing with ASDO operations. Thus, whether Attention

Allocation, Situation Awareness, and Monitoring issues need to be considered for the Traffic

Manager should be considered during the design and evaluation of the ASDO overall function.

The primary monitoring task of the Controller is always to maintain separation between aircraft,

unless the separation task is delegated to the pilots. Even in those situations, the Controller should

provide backup safety by being ready to help whenever necessary.

In the ASDO operations in which aircraft are expected to stay on 4D trajectories, the Controller

should monitor the conformance to the 4D trajectories and look for aircraft deviation or procedural

violations—whether meeting designated fixes, properly merging, CDAs, or spacing requirements.

DSTs are expected to provide alerts, but the Controller should nevertheless attend to the whole

traffic pattern to maintain situation awareness. This type of supervisory functioning will perhaps be

the greatest change of NextGen from current operations and will require much HITL simulation to

fully understand how interaction with the DSTs for Attention Allocation, Situation Awareness, and

Monitoring is best achieved.

In addition to the usual tasks of flying the aircraft, NextGen will require the pilot to attend to the

data link display for any messages (instructions regarding weather, airport reconfiguration, fixes,

directed trajectory modifications, deviations from intended trajectory, CSPR or CDA plans, etc.)

During descent the pilot must continuously monitor spacing from the leading aircraft, especially

after being assigned self-separation responsibility near the runway. In CSPR operations, the pilot

will need to give particular attention to maintaining the aircraft’s trajectory and to awareness of the

position of parallel aircraft.

Because of the variety of variables to be attended to, the pilot will have to develop an attention

sampling strategy, but still be prepared and not surprised to be interrupted by data link messages or

displayed (visual or auditory) alerts or voice messages.

F.4.2 Decision-making and Mental Modeling

It must be established which decisions are normally provided by automation, where either ANSP or

pilots must concur, and which decisions must normally be made independently by ANSP or pilots.

ANSP may, as necessary, countermand the automation, and pilots may, as necessary, countermand

either automation or ANSP. Either of the latter may at times decide not to trust the guidance of the

automation.

Traffic Manager’s decisions include: planning airport reconfiguration, planning arrival fixes,

planning CDAs, planning CSPR landings, and setting criteria for metroplex diversions. Presumably

there will be “playbook” DSTs to establish the parameters for all such planning. Policy guidance for

when to distrust DST recommendations should be provided.

Through training and posted data link notices, Controllers must be aware of the decision criteria

used by Traffic Managers for airport reconfiguration, arrival fixes, CDAs, CSPR landings, and
135

metroplex diversions. Within the constraints set by Traffic Managers, Controllers must perform

their roles in initiating each of those operations, particularly when CDAs, CSPR landings, and

metroplex diversions are initiated for individual aircraft. They also issue instructions via data link

for (clearances to) self-separation in final descent and to land. Assignments to runways, taxiways,

and gates are computer generated and automatically communicated to the pilot, but the Controller

must be generally aware of what they are.

The major responsibility of the Controller is to intervene when DSTs alert to a separation violation

or a significant deviation of an aircraft from its assigned trajectory, and then provide advice on how

to recover. The latter could be by getting back into the originally assigned trajectory or by diverting

to a new trajectory that may necessitate modifying trajectories of trailing aircraft.

The pilot must review and decide to accept any instruction given by the Controller. If he requests an

alternative action then the Controller must agree for that alternative to be accepted, except in

emergency situations where the pilot justifiably acts on his own initiative. If the Controller does not

agree, the pilot must be willing to defend any further action taken. Policies for such negotiation

need to be established by anticipating situations where pilot and Controller might disagree.

F.4.3 Communication

It must be decided which communications are normally by data link and which by voice for airground communications. Data link communications have the advantages that: 1) the information is

clearly specified on the receiver’s visual display and may be left there until executed; 2) the sender

may check what is typed; and 3) there is no waiting for a shared (party line) frequency to become

available so that a message can be sent. The disadvantages are; 1) that the message can nevertheless

be mistyped and/or misread; and 2) that the typing and reading cycle takes more time than the

speaking listening cycle. Voice communication has the advantage that a back and forth series of

messages may be communicated quickly and naturally. However, with voice: 1) either party may

have to wait for a shared frequency to become available; 2) messages may be poorly spoken or

easily misunderstood; and 3) messages may be forgotten because they are immediately gone from

availability.

There is also a decision as to whether to have some sort of computer-based communication or voice

between the Traffic Manager and Controllers, as well as some combination.

Having advanced traffic display tools available should be a major enhancement of communication

between and among persons on the ground and in the air. The system-wide information network

(SWIM) is expected to make other information available to traffic managers, controllers and pilots.

There needs to be further definition of what information, e.g., regarding weather, availability of

facilities, traffic flow changes, emergencies, etc. needs to be automatically sent (“pushed”) and to

which persons, or just made available upon request (“pulled”), or restricted from and therefore not

available to certain persons on the network.

Communication protocols should be highly standardized and common for the various functions so

that at least the procedure is not felt by the communicators to be a source of confusion and delay.

The communication challenge for the Traffic Manager is to keep the appropriate Controllers abreast

of decisions regarding changes in both airspace configuration and traffic flow in and out of airports,

and accordingly what operational constraints need to be imposed. If an airport must be reconfigured
136

the adjoining sector boundaries may need to be altered and the Controllers reassigned in certain

ways, mostly according to a playbook based on winds and weather.

Because Traffic Managers’ communication with Controllers is less time-stressed than that between

Controllers and pilots, the choice of communication mode depends on which is most convenient.

Routine controller-pilot communications are planned to be via data link, but either pilot or

Controller may choose to resort to voice, e.g., if a time-critical negotiation must take place regarding

a trajectory change or for some other reason. Advanced traffic displays will embody and

automatically communicate separation conflicts and trajectory deviations, as well as anticipation of

weather.

Messages regarding separation, trajectory deviation, requests from ground to modify trajectory,

confirmation that the FMS accepts the pilot’s instruction, and all clearances will normally be

communicated to aircraft via data link, possibly with a simple auditory signal to alert the pilot to the

presence of a new message. Time-critical alerts such as TCAS would of course have special

auditory alarms for the pilot, possibly combined with computer-generated speech much as the

ground-proximity warning. The cockpit display of traffic information (CDTI) will be a major means

of communication to aid in separation and traffic situation awareness.

Probably the pilot is the one most affected by a switch to data link communications. This is partly

due to loss of the party line that enhances the pilot’s situation awareness with regard to what other

aircraft are being asked to do.

F.4.4 Memory

The memory lapses that now occur in terminal air traffic management such as readback-hearback

errors, pilots forgetting instructions or whether they had proper clearances, etc., are expected to be

mostly rectified by use of data link communications.

Prospective remembering to perform or complete certain operations when distracted by other events

may be the more serious problem. In particular for ASDO, the aircraft density will be high and

precision in acting when expected to act will be important; there will also be less time to remember

how to react to any anomalies. Situations requiring prospective memory will have to be studied in

the context of HITL simulations.

Traffic Managers will run computer based models to extrapolate weather and traffic demand to

analyze constraints on flow and airport capacity. They must remember the model assumptions to

correlate with the model results. Design of Decision Support Tools should provide memory aids.

A current memory problem for Controllers is forgetting what instructions were given to one aircraft

when giving instructions to a second aircraft that may interact with the first, e.g., for clearances,

acceptance of requests for change in trajectory, etc. DSTs could be designed to record instructions

and mitigate problems that arise from such forgetting.

The pilot may be the one most affected by memory problems, particularly of the prospective sort,

since multiple tasks must be done more or less simultaneously, and certain procedural steps may

easily be forgotten, especially when distractions intervene.
137

F.4.5 Workload

A major human performance issue is workload, both for the Controller and for the pilot. As was

mentioned under Memory in Section F.4.4, the greater density of aircraft and needed precision of

operations under ASDO will also potentially impact Controller and pilot workload. For the

Controller, provision of advanced automation and decision support tools are intended to decrease

workload while enabling any one Controller to handle more traffic. For the pilot, the addition of

data link is intended to decrease workload by making certain information readily available and

relieving memory concerns. For both persons, DSTs are intended to make operations smoother and

more reliable. It remains to be determined whether data link communication and additional DSTs

themselves add to cognitive workload and make overall performance better.

Probably the Traffic Manager will be least burdened by any additional workload caused by using

data link and the new DSTs. However the additional traffic, greater variety of operational modes

(e.g., CDAs, changing arrival fixes, etc.), and the additional complexity of larger and more

integrated metroplex terminals may add back some workload, especially when there are unexpected

events that are not in the TMU plan.

It has been shown experimentally that a single terminal Controller can handle safely only a limited

number of aircraft, perhaps 12–16, in the current mode of continual manual vectoring. Moving to

ADS-B and 4D trajectories, where the Controller becomes a supervisor of the automation, is

intended to enable more aircraft to be accommodated per Controller in the terminal sector.

However, as it is well known that humans are not good monitors, the nature of cognitive workload in

the terminal Controller position has yet to be fully understood. All that is known is that the feel of

the task will change with the interaction with automation. Automation will relieve workload for the

normal flow of events but may exacerbate workload for unexpected contingencies.

The pilot confronted automation years ago with the glass cockpit and the Flight Management

System, and therefore the nature of pilot tasks should not change so much with ASDO. We have a

somewhat better appreciation of what the cognitive workload will be for the pilot as compared to the

controller.

F.4.6 Interaction with Automation, Decision Support Tools (DSTs), and Displays

From a human factors perspective, perhaps the greatest change in NextGen tasks is the extensive

dependence on decision support tools and automation. Most likely some Controllers will adapt just

fine, while others will feel alienated, since they no longer find themselves directly and totally

responsible for air traffic management.

The Traffic Manager will have available much more sophisticated computer-based models than now

for predicting weather, traffic flows, and the resultant constraints in terminal flow and airport

capacity. Effective use of these tools will require displays and human interaction designs that are

well human-engineered and that enable scaling to be readily adjusted in both space and time.

Controllers of RNP aircraft will have intelligent automation to recommend proper trajectories to

merge and space, initiate and control continuous descents, pair for closely spaced parallel runways,

assign runways and taxiways, and provide clearances. How many of these tasks are done by the
138

Controller/Controller automation and how many by the Traffic Manager /Traffic Manager

automation needs to be designed with attention to human performance. Whether the Controller’s

displays should duplicate all the functions provided to the pilot is an open question. The

Controller’s automation should track all of these operations so that the Controller may be assured of

proper execution. The DSTs should alert for imminent separation violations (including

consideration of intent knowledge for both aircraft) and warn for wake turbulence.

Pilots of RNP aircraft will have intelligent automation to accept from the ground and activate in the

FMS the recommended trajectories to merge and space, perform continuous descents, pair for

closely spaced parallel runways, assign runways and taxiways, and provide clearances. The pilot

should have appropriate displays to tackle all of these functions. The pilot’s automation should also

alert for imminent separation violations (including intent knowledge for both aircraft), and warn for

wake turbulence.

F.4.7 Organizational Factors

A single TRACON will serve a larger area than today. Presumably it will include at least the

metroplex for the principal airport. More than likely it will extend to a larger airspace than now,

consistent with the Big Airspace concept (FAA, 2007). It may also involve some functions

previously handled by the ARTCC, or at least will involve closer coordination with them than now

exists. Organizational questions will be part of the TRACON size decision.

One organizational question for the Traffic Manager is how lines of authority between Traffic

Managers and Controllers will be divided. Another question is how will the Traffic Manager’s role

compare to the controllers’ shift supervisor’s roles, and will these be the same person?

There are several organizational issues related to the Controller under ASDO: 1) will the changed

nature of the Controller task, namely from continuous vectoring to supervisor of automation, require

rethinking of Controller shift arrangements; 2) will the current R-side, D-side task allocation remain

similar to what is now used, or will the addition of automation require rethinking this allocation; 3)

will the ratio of TRACON Controllers to aircraft remain as today or decrease; and 4) what ultimate

responsibilities will Controllers have when they act upon the recommendations of automation?

Issues for the pilot are: 1) how much flexibility can be granted to the pilot/AOC to request trajectory

changes (without disturbing the computed optimization and planning by the TMU and making the

coordination task of the controller too difficult); 2) how will the use of so much more automation

affect the contribution of the AOC and its interaction with the pilot; and 3) what ANSP DST and

SWIM information will be shared with the AOC?

F.4.8 Potential Errors and Recovery from Human Errors or System Failures

Because of the complexity of ASDO operations, many types of human error and system failure must

be anticipated and recovery strategies considered. Sometimes there are troublesome symptoms that

are manifest prior to the time when it is understood what has failed. As were established in the

nuclear power industry following the Three Mile Island accident, “symptom-based procedures”

should be developed to avert major disasters caused by time delays that occur while humans are

communicating to decide what has failed and how to resolve the traffic conflicts. Symptom-based

procedures are transitional means to “buy time” safely, until a more considered resolution can be put
139

into effect. Such an approach is also important since there less time to respond to human error and

system failure with the greater traffic density and likely less separation between aircraft under

ASDO. Some human error or system failure situations for traffic manager, controller, and pilot are

discussed below.

Computer models, their advice, and subsequent instructions to Controllers will prove to be incorrect

or inappropriate some fraction of the time. Traffic Managers should be taught to recognize such

situations when they first become evident and take proper measures to cancel the initial advice and

use back-up procedures so that transition to safe recovery can be accomplished.

There are numerous problem situations the Controller may face. There can be minor and frequently

occurring failures, such as an aircraft having to deviate from its programmed continuous descent or

an aircraft deviating from assigned trajectories. A somewhat more serious but nevertheless

troublesome issue is aircraft having to “go around” and somehow having either to get back into an

already crowded queue or be diverted to an alternate airport. Finally, there are unexpected events

such as computer or facility failures or pilot blunders which cause major disturbances to a large

segment of traffic that must be dispersed safely and brought back into the airport while the upstream

traffic is put into a holding mode. An extensive simulation program of training/retraining should be

developed that includes failures at all such levels.

Pilots should be put through the same HITL simulations as described above. There is probably no

need for motion-based full-fidelity simulators for such exercises, as the issues are more ones of

recognizing problems and developing cognitive coping strategies.

F.4.9 Potential Selection, Certification, and Training

Interaction with automation, use of negotiated 4D trajectories, data link communications, and other

RNP-enabled operations such as optimized continuous curved descents will require a great deal of

training. Some combination of lecture and discussion, individualized computer-based concepts

instruction and testing, group cognitive walkthroughs, part-task simulations, and multi-person HITL

simulations (where some “persons” may be computer models) will be required.

Certification of some NextGen equipment should possibly be combined with tests to determine how

effective training will be, the point being that if certain aspects are difficult to train (use) the

equipment may need to be modified.

Traffic Manager’s performance will be affected by training that focuses on responsibilities and use

of computer modeling tools to project ahead and anticipate flow and capacity constraints. Training

in the nature of new responsibilities and interactions with Controllers will be required.

Controller training should include training in normal operations as well as those that occur rarely or

are not anticipatable (see item immediately below). Training in the many new uses of automation

will be extensive.

The FAA should cooperate with airlines to help develop scenarios for NextGen training and

retraining for continuous descents; paired approaches and landings; and especially in the use of data

link for communication, the nature of pilot interaction with Controllers and ground-based

automation in combination, and the acceptance or renegotiation of recommended trajectory changes.
140

F.5 Human Operations Issues, Design Issues, and Countermeasures for ASDO

Section F.4 discussed Human Factor issues for ASDO. This subsection takes the knowledge gained

from assembling these human factor issues and looks at the list of questions presented in Appendix

C. As discussed in Section 3.3 and Appendix C, these questions are addressed in the context of the

ASDO concept. Not all of these questions will be design issues for the concept. However, each

question will be covered and comments will be made regarding how the question relates to ASDO.

Again, as discussed in Section 3.3 and Appendix C, a priority rating of high, medium, or low was

assigned to each question. These sections also discussed the criteria behind assigning a priority

rating.

F.5.1 Use of Automation and DSTs

1. What is the distribution of roles and responsibilities across different human operators and

automation? Is the responsibility clear for each task? Does the concept clearly delineate the

functions by the different agents (human or automation) either by time or by procedures? If not

(meaning two agents can potentially act on a same aircraft at the same time), does the concept

clearly identify an expedient coordination/negotiation mechanism to clarify the intent of multiple

agents in the time constraint given for the problem? How is intent communicated? Will the

human and automation “trade” control and responsibility of the tasks (i.e. hand-off the tasks

between them)? What are the mechanisms for switching tasks between them? Do automation and

human evaluate the operations in a similar manner?

At a simple level Traffic Managers will be concerned with planning and decision making and

Controllers will be concerned with execution and maintaining safety. Beyond these high level

distinctions, it is less clear how roles and responsibilities will divide. One organizational question

for the Traffic Manager is how lines of authority between Traffic Managers and Controllers will

be divided. Another question is how the Traffic Manager’s role will compare to the controllers’

shift supervisor’s roles, and whether these will be the same person. A question that arises for all

NextGen concepts is the ultimate responsibilities that Controllers will have when they act upon

the recommendations of automation.

Since automation under ASDO mostly operates as a DST, the human operators are assumed to be

in control. An exception is if an aircraft originates from an airspace with ground-based

automated separation assurance until the aircraft transfers to self-spacing or self-separation. In

this case, roles and responsibilities for human operations and automation need to be clearly

defined.

The distribution of roles and responsibilities between Controllers and pilots need to be clearly

delineated when the tasks such as self-spacing shift from Controllers to pilots. In FDMS

operations, an explicit delegation of the spacing task to an equipped flight deck provides clear

delineation of the functions initially. Past simulations have shown that pilots managing spacing

while Controllers monitor separation works well when the aircraft spacing is greater than

separation minimum but becomes a problem near the runway when self-spacing can encroach

upon the separation minimum while the Controllers are still responsible for the loss of separation

(Callantine, et al., 2006).

Priority: High
141

2. Who has decision making authority when there are conflicting decisions or advice given by some

combination of automation and other operators? Is there a clear trigger event for any one

operator to decide and act? Is there enough time to solve the problem under such circumstances?

Are there procedures to do this? Does the operator get to choose how long to wait if they have to

act or does the automation determine this for them?

Although it is clear that the pilot has the final decision when time is short and action has to be

taken, who has decision authority before this when routes are being negotiated is less clear.

Negotiation and decision making under normal circumstances need to be decided. While it seems

simple to divide the Traffic Manager and the Controller roles as “planning” and “action”, these

two positions will have to work closely together to ensure that the Traffic Manager makes plans

that can be executed (the Controller can control). At a tactical level, the Controller will have the

authority for decisions, while the Traffic Manager will have planning authority. Automation adds

another level of complexity and the circumstances in which the operator is able to override

automated decisions needs to be clearly specified.

Priority: High

3. Is automation allowed to execute tasks without the human operator in-the-loop or is automation

providing DST functions with operator in-the-loop? Can the automation tell whether the human

operator is in the loop or not? If the operator is not in the loop, can they jump in when they need

to? If the automation is responsible will the operator lose motivation to pay sufficient attention?

I.e. what is the time constraint that the operator has to solve the problem? How critical is the

solution (e.g. safety critical)? Is the trigger event salient? Is the operator (e.g. controller, pilot)

being out-of-the-loop feasible for a given concept?

ASDO airspace is not expected to be fully automated, inferring that controllers will be aided by

DSTs to manage traffic and thus will always be in-the-loop to some extent. Some of the functions

of the ground-based DSTs are the ability to monitor conformance of aircraft to their assigned 4D

trajectory, provide alerts during trajectory deviation, and provide potential resolutions to get the

aircraft back into conformance. Thus, automation will monitor the traffic and the operator will be

expected to jump in if they need to. Given the precision and short time headways for action,

automation will have to cue the Controller to trigger events.

Priority: Low

4. Are there any cases where the human operator will need to become involved, such as when there

is an exception to nominal situations? If so, how and when does the human operator know there

is an exception and s/he needs to act? What information does the automation need to present to

the human in the exception cases? How much advanced notice does the human need?

The major responsibility of the Controller is to intervene when DSTs alert to a separation

violation or a significant deviation of an aircraft from its assigned trajectory, and then provide

advice on how to recover. Controllers will rely on DSTs to keep them aware of aircraft deviating

from their 4D trajectories when they are in flight formation such as a CSPA. Deviations that

controllers are less likely to see, until alerted will be aircraft slowly drifting off their precise

trajectory rather than sudden departures.

Priority: Low
142

5. What is the proper degree of trust by humans in the automation? For example, too much may

lead to complacency and too little to disuse or misuse. How much reliance on the automation is

necessary for the concept to work?

All operators will have to trust the automation they are using, since there will be little time for

human operators to examine DST recommendations and change them. Traffic Managers will

have to be confident that the trajectories calculated by DSTs are accurate and safe. Controllers

will have to be confident that the aircraft positioning information the DSTs are displaying is

correct, and pilots will have to be confident that the trajectories they are sent are conflict free. All

operators of ASDO will rely heavily on DSTs, thus, the automation needs to be designed such

that its recommendations will be acceptable to human operators a high percentage of the time.

Priority: Low

6. Do human operators have the appropriate amount of information on their displays to do their

jobs successfully within the time constraints that they have?

Effective use of these tools will require well-designed displays and human interaction procedures

that enable scaling to be readily adjusted in both space and time. A concern is displaying all the

relevant information if one Controller or Traffic Manager is working with two or three different

concepts, and potentially different DSTs at the same time. If operators have all the information

they need presented on one display, there could be an issue of clutter, however, hiding critical

information will prevent operators from making sound decisions. Display design will be a critical

aspect of the feasibility of the ASDO concepts.

Priority: High

7. If automation/DSTs fails, will human operators realize it and recover? Will the automation

properly indicate failure?

Because of the complexity of ASDO operations, many types of human error and system failure

must be anticipated and recovery strategies considered. It is likely that the human operators will

be working so closely with the automation that they will see a failure should it occur. However,

if a system is degraded, using inaccurate data, for example, the system may look like it is still

working but be giving poor suggestions. In these cases automation needs to clearly indicate that

its calculations have been compromised. Traffic Managers should be taught to recognize such

situations when they first become evident and take proper measures to cancel the initial advice

and use back-up procedures to make a transition to a safe recovery.

Priority: Medium

8. Will the DSTs recognize a problem and help in emergency situations? Are the DSTs smart enough

to indicate when an unusual situation is beyond their ability to be useful and should not be relied

upon?

Both of these properties would be useful in ASDO tools. In general, problematic and emergency

situations are when human operators can most use automated help, and often this is when DSTs

are least useful. So, just from a general perspective these properties would be valuable. More

specifically, Controllers will be relying on DSTs to monitor the traffic flows and to alert them

when aircraft are moving out of tolerances that are too small for the Controller to detect.

Priority: Medium
143

F.5.2 Human Decision-making and Workload

1. Does a single human operator handle multiple operations? Or will different operators handle

different functions in the same airspace?

Yes. Traffic Managers will have to plan CDSs, CSPA, and FDMS as well as other maneuvering

through metroplexes. Controllers will have to monitor and actively follow these procedures and

pilots will have to fly them.

Priority: Low

2. Is there complex mixed equipage airspace? Can the combined human operators and automation

handle the complex mixed equipage airspace?

The answer is both yes and no. At peak times, it is likely that aircraft will be segregated by their

level of equipage and directed to different airports to maximize throughput efficiency. However,

at times of low demand, RNP requirements may be relaxed and mixed equipage allowed into all

airports. Currently ATC handle mixed equipage, so there is precedent that they will be able to

do so in the future.

A complication to this division is that although most discussions of ASDO state that only

properly equipped aircraft will fly arrival trajectories to metroplexes during peak times, aircraft

with very different flight characteristics (e.g., Very Light Jet vs. B747) could meet the same

RNP criteria and will be flying along a same path. Mixed equipage in terms of aircraft

performance, not avionics, is likely to create difficulties in managing separation.

Priority: High

3. Is there a sufficient timeframe for decision-making for critical situations, especially off-normal

situations?

For ASDO, the aircraft density will be high and precision in acting when expected to act will be

important; there will also be less time to remember how to react to anomalies. Whether the time

is sufficient to allow the operator to make critical decisions will have to be determined through

HITL testing.

The Traffic Manager will likely be able to make decisions in advance of an aircraft’s arrival at

the TRACON and not be under critical time pressure. The Controller, however, will need to

make decisions quickly. The role of DSTs and automation in Controller tasks needs to be

defined and the time and workload for Controllers to make decisions needs to be investigated. At

the same time, the pilot tasks, under the busiest combination of ASDO operation modes, need to

be examined for the timeframe required for the flight deck to make required decisions.

Priority: High

4. Can the human operator attend to multiple alternatives in support of optimization? How many

options should be presented?

A human operator can attend to multiple alternatives although, as time reduces, the number of

options that the operator has time to consider will shrink. From this it would suggest that the

ideal number of options is related to the time that the operator has available to make their

decision.

Priority: Low
144

5. Are there peak workload situations that should be moderated? What metrics are appropriate?

It has been shown experimentally that a single terminal Controller can only handle safely a

limited number of aircraft, perhaps 12-16, in the current mode of continual manual vectoring.

ADS-B and 4D trajectories, where the Controller becomes a supervisor of the automation, is

intended to enable Controllers to accommodate more aircraft in the terminal sector. However, as

humans are not good monitors, the nature of cognitive workload in the terminal Controller

position has yet to be fully understood. Pilots already encounter periods of high workload during

approach. It is likely that although pilot tasks may change, that this will remain a period of high

workload and, even with the assistance of automation, that pilots will need to be taught/ develop

expertise to manage this.

Priority: High

6. Can human operators perform all the functions assigned to them? If not, can the function be

distributed across multiple operators or can they be offloaded to the automation?

This is a key question for research. The ASDO concepts split the planning and execution

functions between two operator positions, however, there are many ASDO concepts, e.g.,

FDMS, CSPA, CDA, etc. Whether two operators are able to plan and oversee the execution of

multiple concepts has not been determined.

Priority: High

7. What is the workload impact when there is transitioning between airspace configurations or

different modes of operations (e.g., from CDA to FDMS)?

This will have to be determined through research. For the Traffic Manager, the greater variety of

operational modes in itself is likely to add some workload. Transitioning between airspace

configurations that will permit some ASDO concepts and not others will be an additional

complexity to the transitions today that are incurred just when there is a switch of runway.

These complex, concept- (not environment-) driven transitions will add a layer of workload, and

possibly increase the frequency of these higher workload periods.

Priority: High

8. If there are mixed surveillance sources with different precisions, can the human operators utilize

the more precise information available for some of the aircraft and the less precise surveillance

information on other aircraft?

Yes. Controllers have demonstrated in previous HITL studies that they are able to control traffic

that has differing levels of trajectory prediction and conflict avoidance. These results indicate

that Controllers are likely to also be able to handle aircraft with mixed data precision.

Priority: Low

9. Are there appropriate layers of human operator redundancies built into the concept so that some

operators monitor others to anticipate if help is needed?

The wider organizational structure of the ASDO controller positions has yet to be determined.

However, the concept already has some redundancy built into the concept because there will be a

Traffic Manager and a Controller working the same space. An organizational question is how

lines of authority between Traffic Managers and Controllers will be divided. Another question is
145

how will the Traffic Manager’s role compare to the controllers’ shift supervisor’s roles, and will

these be the same?

Priority: Low (as key issues are raised in other questions but could turn to MEDIUM)

10. In responding to emergencies or recovering from error, are there standard responses generated

across different modes of operation or does it need to be different for each mode? If different is

the operator likely to be confused about which mode is appropriate?

Because of the complexity of ASDO operations, many types of human error and system failure

must be anticipated and recovery strategies considered. While standard responses would be

preferable, it is likely that each ASDO concept will give rise to different types of problems and

therefore will have to be handled on a case-by-case basis. An extensive simulation program of

training/retraining should be developed that includes failures at all levels.

Priority: Medium to High

11. How brittle is the concept? How flexibly can the operations adapt to off-normal situations? How

easy is it for the concept to fall apart under off nominal situations, and can any such situations

be anticipated? How are fail-soft modes and resilience built into the system?

The answer to this question will be determined by the robustness and flexibility of the

automation. If automation can tolerate and correct for aircraft deviating from their routes to an

extent, then there will be fewer break out situations where aircraft will have to be managed back

into the flow. Having to leave slots for errant aircraft to be re-incorporated into a flow will

significantly reduce the effectiveness of a concept. If the automation cannot correct for slight

deviations, then it needs to be able to flag these to the Controller who can then correct a

trajectory before an aircraft moves outside of its permitted envelope.

Priority: Low

F.5.3 Attention, Situation Awareness, and Memory

1. Are there any excessive demands on attention?

Research is required to properly investigate this question. With the right DST, it is possible that

players will be able to attend to all the procedures they are involved with without overload.

However, given that all ASDO concepts require precision aircraft maneuvering, it seems likely

that attention management will be an issue.

Priority: Medium to High

2. What is the time required to gain situational awareness to perform a normal task?

The Controller will take on a more supervisory role under ASDO, which will perhaps be the

greatest change of NextGen from current operations. It will require much HITL simulation to

fully understand how interaction with the DSTs should work to obtain and maintain a high level

of Situation Awareness (as well as attention) in the areas where it is most required. I.e., how will

a controller focus their attention/ situation awareness?

Priority: High
146

3. If the human operator is passively monitoring and suddenly there is a need to get back into the

loop, how long does it take to do this? Is there enough time to do this for the given concept? What

is the workload involved? Is it a feasible amount workload?

Under ASDO, Traffic Managers and Controllers are continually involved. They use DSTs, but

there is not automation that conducts a whole process. Thus, there are unlikely to be periods

where the Traffic Managers and Controllers are passively monitoring and suddenly there is a

need to get back into the loop.

However, if the operator does not have situation awareness, establishing this is affected by many

factors. It will depend on the complexity of the situation, the number of other tasks the operator

is having to concurrently handle, and the operator’s level of expertise, both in general and with

the specific situation.

Priority: Low

4. Can automation complement or replace human monitoring function by calling attention to salient

and critical events?

Automation will be necessary to complement, but not replace, the human monitoring function in

ASDO. Controllers will need assistance monitoring the CSPR process, FDMS and the

conditions for any changes that will interrupt these processes. Traffic Managers must

continually monitor weather and the general flow of aircraft into and out of the terminal airspace.

Automation can assist in both these areas. It will also be able to monitor the operational

elements a Traffic Manager needs to consider before setting the criteria for ASDO procedures

(e.g., metroplex diversions, and CSPR or CDA arrangements).

Priority: Low

5. Are there recognizable indicators of emergency and critical safety events? Are there situations

where alertness of human operator to new information and indicators may be a problem?

The Controller should monitor conformance to the 4D trajectories and look for aircraft deviation

or procedural violations—whether meeting designated fixes, properly merging, CDAs, or

spacing requirements. DSTs are expected to provide alerts, but the Controller should

nevertheless attend to the whole traffic pattern to maintain situation awareness. Each concept

within ASDO will have its own indicators for critical events. To take an example of CSPA, if a

following aircraft moves out of its safe envelope behind the lead aircraft, either by dropping

behind, moving to the side or speeding up, it could get caught in the wake of the lead aircraft,

which at such close quarters is a critical event.

Priority: High

6. Are the modes of operations (e.g., automated, self, or controller separation assurance) or

airspace configuration always clear to the human operators (i.e., pilot and/or controller) in both

nominal and off-nominal situations?

Pilots, who will be expected to merge and space on some descents, pair up and follow or lead on

other approaches, or just adhere to a CDA, may find the modes of operations confusing unless

they are clearly displayed in the cockpit.

Controllers and Traffic Managers who may be working with three or more different concepts and

DSTs within their airspace could experience mode confusion if their displays are not well
147

designed. For example, at a very basic level they will need to know what ASDO capabilities are

in use at any one time.

Priority: High

7. Are there situations where mental fixation and “cognitive tunneling” may be a problem?

Possibly. All the ASDO concepts require precise aircraft maneuvering in tight spaces. Pilots

and Controllers will be expected to closely monitor the progress of a procedure, such as FDMS,

to ensure they are within specified tolerances that have tangible safety implications if they are

broken. These kinds of high cost situations seem to have potential for cognitive tunneling.

However, both pilots (and Controllers) will have other tasks that they need to complete

simultaneously, e.g., pilots need to be preparing the aircraft for landing also.

Priority: Low (not necessarily low but there are other things that are more important to

research)

8. What strategies are used to allocate attention during task execution? How does the human

operator know when to attend to the automation?

Because of the variety of variables to be attended to, the pilot will have to develop an attention

sampling strategy that will be assisted if the cockpit displays are distinctly different based on the

operation at hand. The Controller, who will be monitoring multiple procedures possibly based

on different concepts, will require the automation to flag aircraft that may require assistance or

intervention if they are deviating from their prescribed 4D trajectory in some way.

Priority: Low

9. How do the human operators maintain situation awareness of the traffic when airspace and

routes are less structured in NextGen operations? Do they need a way to anchor their mental

models of the air traffic situations?

Under ASDO, there will be a greater variety of tasks to be performed by all human operators,

resulting in more Attention Allocation, Situation Awareness, and Monitoring issues. Attention

strategies must be designed based on the relative importance and expected frequencies of various

operational events. A model of such events in a typically busy ASDO period could serve as the

basis for developing attention strategies and training programs for traffic managers, controllers,

and pilots.

Priority: Low (only because the next question will cover similar ground when it comes to

research)

10. Are there situations where prospective memory is particularly important?

Pilots, in particular, will need to be prepared for interruptions by data link messages or displayed

alerts or voice messages. In these cases, they will have to remember what they were doing and

the point at which they were interrupted.

Prospective remembering to perform or complete certain operations when distracted by other

events may be the more serious problem. In particular for ASDO, the aircraft density will be

high and precision in acting when expected to act will be important; there will also be less time

to remember how to react to any anomalies. Situations requiring prospective memory will have

to be studied in the context of HITL simulations.
148

The pilot may be the one most affected by memory problems, particularly of the prospective

sort, since multiple tasks must be completed more or less simultaneously, and certain procedural

steps may easily be forgotten, especially when distractions intervene.

Priority: High

F.5.4 Communication and Coordination

1. Is the coordination overly complex?

This is to be decided. Given that aircraft will be maneuvering close together, it will be important

that coordination is simple and direct and that the procedures are also straightforward, clear and

do not create problems for the human operators.

Priority: Medium (this wouldn’t be a show stopper but it’s pretty important that they get this

right)

2. What impact do time constraints have on flexibility for coordination? How much time is allowed

for negotiation, if any? How many modification attempts are acceptable?

If procedures are negotiated and set up well in advance, there will not be any time constraints on

coordination. However, if a procedure needs to be re-negotiated time may be tight and there

may not be a chance for negotiation. An example would be if a following aircraft in CSPA

begins to deviate in busy airspace and needs to be given a trajectory to recover. While this is not

(yet) an emergency situation, it is also not one where there is time for negotiation.

Priority: Low

3. Who or what has final authority? How will an end to negotiations be determined?

Pilots will have final authority over their own aircraft. The decision whether or not to execute a

maneuver rests with them. Negotiations are likely to have a time limit since procedures need to

be put into place (trajectories loaded into FMS) by a certain point in an aircraft’s descent for the

aircraft to be able to capture these routes.

Priority: Low

4. How are the trajectory changes negotiated via data link? Do they need accompanying voice

communication to express the intent? If both modalities are needed, how are the procedures of

that interaction managed?

There is likely to be a mix of both voice and data link. It is likely that most trajectory changes

can be negotiated over data link but if the maneuver is time-critical or safety related, Controllers

and pilots may opt to use voice to convey urgency.

Priority: Low

5. Are communications via data link, voice, or both? If data link is called for, can the concept cope

with the expected delay and the difficulty of sharing one’s intent via data link? Are there

circumstances where only voice or data link communication should be used?

There is likely to be a mix of both voice and data link. It must be decided which communications

are normally by data link and which by voice. This decision must be made not only for airground communications but for ground-ground communications also.

Priority: Low
149

6. Are the underlying technological assumptions for communications and information sharing

adequate for the concept?

A current disadvantage of data link is that it is slower than voice, however, with predicted

advances in both data link technology and voice recognition, it is likely that data link will

become a quicker media.

The system-wide information network (SWIM) is expected to make much required information

available to traffic managers, controllers and pilots. There needs to be further definition of what

information has to be automatically sent and what is made available upon request. The other

side to this process is that the safety critical information and instructions that pass between

Controller, pilot and Traffic Manager need to be protected from interference (by chance or

intent) and (for this reason) have restricted accessibility.

Priority: Low

7. Is the information that is shared among multiple operators appropriate regarding content, detail,

and type of format for each particular operator?

Within the triad team of the ASDO Traffic Manager, Controller and pilot, the information they

share will be appropriate in content and detail and format.

Priority: Low

8. How important is participating in “party line” communications (i.e., a voice frequency shared

among several persons) as is used in current operations for situation awareness under NextGen?

Should there be analogous mechanism in data link communication?

Communication protocols should be highly standardized and common for the various functions

so that the procedure is not felt by the communicators to be a source of confusion and delay.

Although there has been concern that the party line should be retained somehow, controllers

interviewed in a Cognitive Walkthrough (see Appendix E) were not concerned about its loss. It

must be noted that this was their subjective opinion when they were not working traffic, so they

did not experience the lack of a party line. It is also likely that pilots gain more from party line

communication than controllers and that they will feel its loss more acutely.

Priority: Low

9. How will visual attention demands of data link communication affect other tasks that require

visual attention?

This is a question for HITL research. Previous work did not find data link a distraction for

pilots, however electronic messaging was used only for low priority messages. More recent

studies that have used data link (but not studied it) do not report marked effects of data link use

on other tasks. However, if data link is to become a means of communicating safety-critical

information, this question needs to be studied.

Priority: Medium to High

10. How are the emergency responses coordinated between human operators?

This is to be decided. In emergency situations, Controllers will intervene when DSTs alert to a

separation violation or a significant deviation of an aircraft from its assigned trajectory, and then
150

provide advice on how to recover. If the situation is not resolved, the pilot justifiably acts on his

own initiative. If the Controller does not agree, the pilot must be willing to defend any further

action taken. Policies for such negotiation need to be established. These will include how the

response is coordinated both between the key players and within the wider team that includes

Traffic Managers and supervisors.

Priority: High

F.5.5 Organization, Selection, Job Qualification, and Certification Factors

1. What will be the organizational structure, interactions between organizational units, and

delegation of authority to units?

Under ASDO, the current TRACON positions will be split into two functions: Traffic Manager

and Controller. The Traffic Manager will manage the arrival and departure routes while the

Controller will manage the execution of these routes. The Traffic Manager may span multiple

positions depending on the situational context and the facility, and the organizational structure

has yet to be determined. One organizational question for the Traffic Manager is how lines of

authority between Traffic Managers and Controllers will be divided. Another question is how

will the Traffic Manager’s role compare to the controllers’ shift supervisor’s roles, and will these

be the same person?

Priority: High

2. What skills, abilities, and traits are needed by personnel (e.g., ability to use automation, and to

work more as a manager of operations that are handled mostly by automation with occasional

involvement by the operator)?

NextGen ASDO personnel will still need the skills they have currently – keen spatial awareness,

local and system knowledge – but will also need to be comfortable with using and interacting

with automation. Controllers will no longer find themselves directly and totally responsible for

air traffic management, being in a more managerial role overseeing the automation, so

supervisory skills will be an advantage.

Priority: Low

3. How difficult will it be for human operators to develop an understanding of the capabilities and

limitations of the computer-based models they will employ?

The completeness and informativeness of training and automation description will determine

operators’ difficulty in developing an understanding of the computer-based models they use. In

addition, well-engineered displays and automation-human interaction procedures will assist

operators’ understanding.

ASDO tools may be complex but extensive testing should allow their capabilities and limitations

to be listed.

Priority: Medium
151

4. What additional training is needed for the human operators to perform the necessary operations

for the given concept? How do they train to understand the automation systems, including failure

detection?

As with all the Research Focus Areas covered in this paper, future human operators will need to

be comfortable using displays and working with computers. There will be many operational

modes of ASDO and with these different modes being on and off at different times, operators

will need to be comfortable working under different situations during a shift.

Traffic Managers will require training that focuses on their new responsibilities and interactions

with Controllers, also in the use of computer modeling tools to project ahead and anticipate flow

and capacity constraints.

Controller training should include training in normal operations as well as those that occur rarely

or are not anticipatable. Training in the many new uses of automation will need to be extensive.

Priority: Medium to High

5. How will the human operators train for the more diverse and dynamic airspace and route

structures that will exist in NextGen operations?

Most successful training is based on teaching trainees basic skills, then moving on to normal

action sequences and then, after they have developed some proficiency, moving to action

sequences that have anomalies or perturbations. ASDO operator training could follow this

pattern, training skills in a stable airspace initially and then adding diverse and dynamic elements

gradually. From this, a potential issue is the length of time that operators spend in training.

Priority: Medium to High

6. Should human operators train to maintain skills that are only needed when the automation fails

(e.g. monitoring for conflicts)? How do they train for random failures and anomalies?

Controller training should include training in normal operations as well as those that occur rarely

or are not anticipatable. Flight crews already train for rare and unusual events as part of their

recurrent LOFT. For Controllers, some combination of lecture and discussion, individualized

computer-based concepts instruction and testing, group cognitive walk-throughs, part-task

simulations, and multi-person HITL simulations will be required. In these latter training

simulations full teams, including the Traffic Manager and ATM supervisors would be able to

take part to gain experience.

Priority: Low

F.6 Summary of Key Human Performance Issues for ASDO

Section F.4 addressed the perspective of the human performing the tasks by examining nine human

factors (e.g., attention allocation, memory, workload, etc.). Section F.5 took the perspective of what

the tasks ask of humans by examining human performance issues in the ASDO operational context.

We feel that this level of detail is necessary for the design of the ASDO Research Focus Area to

produce a concept in which human operators can perform efficiently and safely, what with the use of

much more automation and different operating modes than today. This section summarizes the key

human performance issues and our recommendations/countermeasures for them. The

recommendations are mainly in the context of humans performing their tasks under ASDO rather

than fundamental HF research in general.
152

The summary of recommended research is presented under the same five categories as used in

Section F.5. The main drivers from ASDO for these recommendations are the increased use of

automaton and Decision Support Tools, the high number of operational modes that will operate at

once and/or be turned on and off, and the reduced separation criteria coupled with high traffic levels

and resulting requirement for precision in operations.

These ASDO properties have focused the key questions currently around the first three categories of

Human Factors questions: use of automation; human decision making and workload; and attention,

situation awareness, and memory. Communication, coordination, training, selection, and

organization are all important issues but can only be addressed when basic concept feasibility (as

covered by the other categories) has been established.

F.6.1 Use of Automation and Decision Support Tools

Distribution of roles and responsibilities between human and automation

Issue: The distribution of roles and responsibilities across the Traffic Manager, Supervisor,

Controller, pilot, and ground/flight-deck automation is considered to be highly important and needs

to be delineated. In particular, an aircraft in ground-based separation that transitions to self-spacing

or self-separation may shift responsibilities between Controller, pilot, and their respective DSTs

multiple times within the same flight. The decision authority needs to be defined along with roles

and responsibilities. Decision authority may become problematic during self-spacing operations if

the pilots create separation errors during self-spacing and the separation responsibility still falls on

the Controller. In addition, either the pilots or the Controllers may be relying heavily on performing

the separation tasks and cannot perform the tasks without it, blurring the actual decision authority

between humans and automation.

Potential Countermeasure/Next Step: Functional analysis should be done to outline the roles

and responsibilities of humans and automations in the system. Human factors research

should be conducted to test potential problem areas. Some basic human factors research

related to human-automation interactions and collaborative decision-making can examine

how the tasks and responsibilities might be best distributed in a complex human-machine

system. Applied research that tests specific concept instantiations via HITL simulation—

especially focused part-task studies that test feasibility of a specific task-sharing team

configuration—would also be useful.

Design of ASDO decision support tools

Issue: In the ASDO concept, it is possible that a pilot, a Controller, or a Traffic Manager will need to

manage three or more concepts (e.g., ground or airborne separation, either in performance-based

TBO or regular operations) that are in operation in a single space. Therefore, it is possible that their

displays will require more information than can reasonably be displayed in one place and the result

will be screen clutter. ASDO will have numerous automation tools to help human operators with the

many operational modes (e.g., CDAs, FDMS, etc.). Attention needs to be given to the design of

human interactions with automation, DSTs and displays; and to the recovery from human errors or

system failures.

Potential Countermeasure/Next Step: Displays and DSTs should be prototyped and tested to

see if they can help the human operators to manage ASDO operational environment. Design
153

aspects to address are: distribution of roles and responsibilities across automation and

different human operators, such as between pilot performing self-spacing and controller

performing separation assurance; trust in automation by humans since humans rejecting

automation recommendations very often will slow arrivals; whether operators have enough

information on their displays to do their jobs successfully given the great traffic density and

the many operational modes of ASDO; and automation and/or DST failures as to how human

operators will recognize failure and recover and as to the brittleness of ASDO with precision

required to handle super-density operations.

F.6.2 Human Decision-making and Workload

There are numerous issues around human decision-making and mental models, memory, and

workload that need attention. Some of the key issues involve how a single human operator handles

multiple operations required for the traffic density and many modes of ASDO. Another issue is the

design of appropriate layers of human operator redundancies built into the concept so that some

operators monitor others to anticipate if help is needed, such as a supervisor who monitors the

workload of Traffic Managers planning the routes for high traffic levels.

Decision-making in mixed operations airspace

Issue: Depending on the design of ASDO airspace, it is possible to have mixed operations in the

same airspace. Aircraft with different RNP and flight deck equipage can fly under different rules.

Segregating airspace by RNP and other equipage levels will make its management easier but needs

to encompass enough aircraft to effectively use the space and the airports it is feeding. In mixed

operations, a key question is whether one or more human operators (e.g., pilots, Controllers) can

perform different functions for the different modes of operations (e.g., FDMS, CSPA).

Potential Countermeasure/Next Step: To determine how much mixed operations to allow,

fast-time simulation or other benefit analyses should establish the benefits of mixed

operations compared to segregated airspace and/or route structures. The concept should be

scoped to determine on paper whether human operators have more functions than they can

effectively manage. Once the concept is sufficiently vetted, the feasibility of the human roles

should be examined in HITL studies.

Workload

Issue: In mixed operations, monitoring and reacting to peak workload is a key issue since current

use of aircraft count to measure potential overload may no longer apply in complex mixed airspace

that may also include transitioning from one operational mode to another (e.g., switching from

CSPA to normal operations; switching between FDMS and non-FDMS operations).

Potential Countermeasure/Next Step: Peak workload situations, workload impact of

transitioning between operational modes, and workload under off-nominal situations should

be evaluated in HITL studies. Prior to this, research is needed to determine traffic

complexity and associated workload impact in a mixed operations environment as described

in ASDO. Aircraft count, delays, airport slots, and other metrics used today should be

replaced or augmented to take account of different performance-based equipage levels,

different route structures, and mixed operations rules.
154

Responses to emergency or off-nominal situations

Issue: A recurring theme in ASDO is whether operators will have enough time to make decisions

in critical situations, both normal and off-nominal. Because of the complexity of ASDO

operations, many types of human error and system failure must be anticipated and recovery

strategies considered. While standard responses would be preferable, it is likely that each ASDO

concept will give rise to different types of problems and therefore will have to be handled on a

case-by-case basis.

Potential Countermeasure/Next Step: Evaluation of decision-making time should probably

occur through HITL simulation in addition to other methods. To address the emergency and

off-nominal events in mixed operations, a series of scenario-based walkthroughs should be

generated to capture the off-nominal situations that can be imagined and look for gaps in the

design by stepping through the scenarios. Once the failure modes are captured and the

concept is refined to mitigate them, an extensive simulation program of training/retraining

should be developed that includes failures at all levels.

F.6.3 Attention, Situational Awareness, and Memory

To make timely decisions, operators will need to have an awareness of the situation and enough

resources to attend to critical information. Thus, these issues are of key importance as they feed into

the process discussed above. Excessive demands on attention will only be established through HITL

research where people working realistic displays can be observed.

Design aspects in this area include: the operator always knowing the mode in which they are

operating under normal operations with the many different possible operational mode combinations

(e.g., what modes are on and off, and what the runway configuration is, which may change more

frequently than today), memory of actions in each particular operating mode, need for emergency

trigger events what with all the operational modes, and aids to help the human operators respond to

emergencies given the different operating modes in which the human operators may be working.

Gaining situation awareness in time and safety critical situations

Issue: In mixed operations, there may be excessive demands on attention to monitor and keep track

of different operational rules in airspace with tight spacing and formation. Potential lack of resources

can mean safety critical events are not attended within the short window when they can be diffused.

One factor that helps with the management and prevention of emergency and critical events is if

there are indications that such an event is imminent or developing. If operators can be alerted to a

situation beginning to develop before it has reached a critical condition, the operator can act to

prevent, or at least reduce, the effects of the situation.

Potential Countermeasure/Next Step: Concept, operational procedures, roles, and DSTs

should be adequately designed to mitigate the potential attention overload problem. Once the

concept is sufficiently designed, it should be tested in HITL studies (focused part-task studies

would help) that evaluate the efficacy of the design. For safety critical situations, research to

identify off-nominal and emergency event indicators can have an impact on both the safety of

the concept and also human factors in terms of the ability of an operator to manage offnominal events.
155

Potential modal errors in attention and memory

Issue: In mixed operations, Controllers will need to develop the ability to be aware of multiple

flows operating under different concepts within their area of jurisdiction, so the operational criteria

and their tolerances may differ between aircraft that are physically close together. Controllers will

need to be able to keep these conditions “straight” mentally and act appropriately to each of the

different events they are responsible for. There is great potential for modal confusion in such

environment, in which an aircraft is assumed to be operating in one separation mode (e.g., tight

spacing with FDMS) when it is in fact in another (e.g., low equipage aircraft that requires larger

spacing buffer), creating a safety critical situation.

Having to monitor multiple aircraft under different conditions, there is likely to be a high load on

operators’ working memory. With so many simultaneous activities, operators will have to be

prepared to be interrupted often while they are completing tasks. Interruptions tend to place a high

load on prospective memory (you have to remember to go back to the activity you were working on)

for all ASDO operators.

Potential Countermeasure/Next Step: Further human factors research is needed in the

context of ASDO mixed operations to determine what types of modal errors can occur in

both attention and memory. Past human factors research in this area should be leveraged to

understand where the potential problems can occur. Further research should be done to

explore potential solutions that can mitigate the errors. The concept should be refined to

mitigate potential modal errors and then evaluated to see its efficacy.

F.6.4 Communication and Coordination

Visual attention demands of data link communication

Issue: If data link is to be used in safety-critical messages such as clearances to resolve conflicts, the

visual attention demands of data link in an environment that potentially has high visual clutter

should be carefully examined.

Potential Countermeasure/Next Step: There is a wealth of human factors research looking

into visual attention demands, which could be drawn on to inform this area of study. The

need to undertake operational visual attention demand research is to study electronic

communication when it is the primary means of sharing and transferring information. If

data link is to become a means of communicating safety-critical information, it is important

to study its impacts on other operator tasks in general.

Coordination of emergency responses

Issue: In emergency situations, Controllers will intervene when DSTs alert to a separation violation

or a significant deviation of an aircraft from its assigned trajectory, and then provide advice on how

to recover. If the situation is not resolved, the pilot justifiably acts on his own initiative. If the

Controller does not agree, the pilot must be willing to defend any further action taken. Policies for

such negotiation need to be established. These will include how the response is coordinated both

between the key players and within the wider team that includes Traffic Managers and supervisors.

Potential Countermeasure/Next Step: Coordination of emergency responses should be an

extension of general coordination procedures, which have to be scoped in detail for the

concept. Coordination needs to “add value”, in that a combined response from a number of

operators should provide a more effective solution than the efforts of a controller acting
156

alone. Coordination should not add workload for an operator and only serve the purpose to

keep others informed. DSTs should be used for the purpose of information sharing to assist

operators by removing administrative tasks.

F.6.5 Organization, Selection, Job Qualification, and Certification Factors

Issue: In ASDO, there are numerous operational modes and a need for precision in timing for

actions. Thus, training is needed for performance under these aspects, and there is a need for human

operators to have abilities to handle a variety of operational modes. In addition, the organizational

structure under ASDO is unclear and will need to be clarified for the concept to develop. There are

likely to be people (or a person) who are responsible for determining the route structure and

operational modes (we call these people “Traffic Managers”) and others who monitor the

conformance of the traffic and manage normal control functions. However, how the roles are

distributed is less clear, both functionally and organizationally.

Potential Countermeasure/Next Step: Designing training schedules/programs for ASDO

operations should not be an issue but training is an often overlooked component in concept

design. Given that human operators in ASDO might work often in mixed operations, how

such operations can be trained should be given some early thought. If issues arise that

indicate that the number of operations in ASDO is beyond human ability even after proper

training, redesign of operational procedures and the airspace should occur early in the

research. The current methods of training should be cataloged and steps should be taken to

see how they can be modified to handle ASDO operations. Feedback from Subject Matter

Experts should be solicited to assess if it would be possible for Controllers and pilots to

learn and operate in this environment.

F.7 References

Callantine, T., Lee, P., Mercer, J., Prevôt, T., and Palmer, E. (2006). Air and ground simulation of

terminal-area FMS arrivals with airborne spacing and merging. Air Traffic Control Quarterly,

14(1), 47-67.

Federal Aviation Administration (2007). Integrated arrival/departure control service (Big Airspace)

concept validation. Washington, DC: FAA Office of Air Traffic Organization Operations

Planning.
157

Appendix G: Traffic Flow Management

Traffic Flow Management (TFM) identifies and resolves imbalances in the demand and supply of

NAS resources, such as airspace and runways. This includes the requirement to: 1) assimilate

weather into TFM by exploring the impact of weather (convective and non-convective) on the

capacity of NAS resources; 2) explore the roles and responsibilities of airspace users on TFM

decision making; 3) develop and explore the utility of both aggregate (i.e., flow based) and aircraft

level flow control models; and 4) develop heuristic and optimization approaches for developing

optimal flow control strategies (e.g., rerouting, departure control, airborne flight delays).

Under NextGen TFM, more precision is expected in the planned and actual trajectories. Higher

fidelity in trajectory prediction, along with the increased precision in weather prediction will allow

more a optimal schedule which in turn will increase the airspace capacity. Automation for TFM will

be primarily in the form of decision support tools (DST) to support the Traffic Flow Managers in

planning aircraft trajectories.

At a more detailed level, the drivers of future TFM are:

• increased air traffic: Air traffic increases of 2 to 3 times today’s traffic will require more

extensive route planning and organization.

• varied equipage: There will be a greater range of equipage on aircraft. For example, some will

be equipped for 4D trajectories and some not. There will likely be some airspace restricted to

aircraft with certain levels of equipage. This will complicate the job of TFM.

• user preferences: More user preferences is a goal in the future, and this will add to the

complexity of TFM.

• different airspace: Airspace in the future will be different than today. There may not be a need

to fly waypoint-to-waypoint, and there may be larger airspace units than today’s sectors

enabled by ground-based automated separation assurance. These will drive changes in TFM

functions.

• Dynamic Airspace Configuration: Airspace configuration can be changed in the future under

DAC.

• more information and better prediction: There will be better predictions for winds and weather.

• flight plans more precise and aircraft more precisely meet flight plans: Flight plans will be for

4D trajectories (or at least 3D with constraints to be met during flight). Aircraft will meet their

4D flight plans more precisely. This has implication for more precise traffic flow management

planning since aircraft can be expected to meet precise planned routes.

Within TFM, National Traffic Flow Management will work to accommodate user preferred gate-togate trajectories. Then, within this, Regional Traffic Flow Management will provide a tactical

control loop to adjust strategic control solutions at a more local level.
158

G.1 Overview Description of TFM

At a high level, the function of TFM is simple to state: plan aircraft trajectories to maximize

capacity. There are, of course, more specific functions that provide this overall capability:

• aircraft will transmit and receive precise digital data that includes routes and times for the

aircraft to cross key points in the airspace

• point in space metering: Properly equipped aircraft will be given times to be at boundaries

between airspaces, and will not be required to fly waypoint to waypoint.

• timeframe for TFM: Prior to TFM handling a flight, a scheduler starts far in advance using

historical data to make predictions and schedules. As the planned day comes nearer, historical

data is gradually replaced by current data & the schedule is updated. TFM kicks in at a time

horizon of 1 to 8 hours for national flow management and 20 minutes to 2 hours for regional

flow management.

• TFM determines an aircraft’s planned flight path from pushback to gate arrival, and this

planned trajectory becomes a gate-to-gate 4D “contract”

• TFM addresses daily capacity and planning problems, such as weather

• user preferences are considered. (User preferences and consequent negations for these user

preferences between TFM and Airline Operations Centers [AOC] will be the subject of

research under the TFM RFA.)

• fairly allocate flights to capacity constrained NAS resources

• deal with off-nominal scenarios

• deal with weather impact

• coordinate with Dynamic Airspace Configuration and mode of Separation Assurance concepts

G.2 Summary of Required Technology for TFM

The general types of technology assumed for TFM are 4D trajectory-based operations, Performance

Based Services, and ground-based automated separation assurance. It is further assumed that TFM

will operate along with Dynamic Airspace Configuration and Airspace Super Density Operations

(ASDO), as planning trajectories is part of these concepts.

Specific technologies assumed include data link, ADS-B to provide accurate position data, and

System-Wide Information Management (SWIM). En Route Automation Modernization (ERAM)

and Traffic Management Advisor (TMA) are assumed for use by Air Traffic Control.

Aircraft equipage is assumed for Cockpit Display of Traffic Information (CDTI), Area Navigation

(RNAV), and Required Navigation Performance (RNP).

G.3 Summary of Human Roles for TFM

TFM functions are assumed to be performed at two levels: one national TFM facility and multiple

regional TFM facilities. The national TFM facility is likely to be part of today’s National Flow

Control at Command Center. Regional TFM facilities are likely to be part of today’s regional

Traffic Management Units (TMUs). For the purposes of this document, those in the National Flow
159

Control facility dealing with TFM will be called the National Traffic Flow Manager, and the persons

with TFM duties in the Regional TFM facility will be called the Regional Traffic Flow Manager.

Roles of Controllers, pilots, and Airline Operations Centers (AOC) are also covered. The latter is

covered for TFM since there will likely be coordination between TFM and AOCs about user

preferred routes.

The human roles analyzed for TFM are:

1. TFM functions at two levels:

a. National Traffic Flow Managers: at one national TFM facility with that will perform

strategic flight trajectory management by setting a NAS-wide schedule based on:

• advanced traffic and weather prediction tools

• user preferred routes from AOCs

b. Regional Traffic Flow Managers: at regional TFM facilities that provide tactical TFM to

adjust the strategic solutions at a local level due to more local or late occurring weather

and traffic demand changes. They may also negotiate with AOCs for updated user

preferred routes.

2. AOC: coordinates with the National and/or Regional Traffic Flow Managers to negotiate their

preferences on the routes and the schedule. Assuming that smart DSTs will be available, the

coordination may be done through the scheduling tool by having the AOCs input the preferred

scheduled slots and the routes directly into the tool.

3. Controller: concerned with monitoring and managing an assigned airspace. For TFM, s/he

executes any Traffic Management Initiatives (TMIs) and monitors aircraft for its conformance

to the schedule. The Controller will have different duties depending on the Separation

Assurance (SA) mode in the airspace, for example in classic airspace the term “Controller”

may be used and in automated ground-based SA, the term “ASA Manager.” We use only the

term “Controller” for this basic function throughout the TFM concept.

4. Pilot: coordinates with AOC to request and fly user-preferred routes that are most optimal to

the airline. S/he conforms to any TMIs that are being executed by the Controller.

G.3.1 National and Regional Traffic Flow Manager Roles and Responsibilities for TFM

Both the National and Regional Traffic Flow Managers are expected to be heavy users of

automation aids. Much information on conditions, including current status and predictions, on such

events as weather, traffic loads, airport conditions, etc. will be displayed and searchable. There will

be continual updates on departure times, both expected and actual, of aircraft. DSTs for planning

routes and for “what if” analysis will be used. There will be negotiations with aircraft and AOC. In

addition, the National and Regional Traffic Flow Managers will be expected to exchange

information with each other.

It is noted for determining human performance issues for the National and Regional Traffic Flow

Managers that the Traffic Flow Managers should not be under critical time pressure for most of their

work. There may be some times when weather, such as snow storms in the mid-west or northeast,

result in a heavier workload, but the Traffic Flow Managers’ work would likely not be time critical

in a safety sense as a sector Controller’s might be.
160

The National and Regional Traffic Flow Manager tasks are covered in one list below even

though the tasks are performed in different facilities. This is to avoid redundancy in stating the

tasks. The tasks for both National and Regional Traffic Flow Managers are assumed to begin on

the day a flight is taken. Traffic Flow Management in advance of the current operational day is

not covered in this paper.

1. At beginning of a shift, interpret strategic flight trajectory information previously entered into

TFM automation by the long-term scheduling function.

2. Conduct initial planning for an aircraft trajectory from origin to destination.

The National Traffic Flow Manager will do this with historic information, updated information

on the pushback time of the aircraft, and the latest information on conditions for the flight (e.g.,

weather, airspace restrictions, etc.). The National Traffic Flow Manager will coordinate with the

AOC about user preferences that can be accommodated by TFM.

3. National Traffic Flow Manager transmits, and Regional Traffic Flow Manager receives, strategic

flight trajectory information from National TFM facility.

The information from the National TFM facility will likely be transmitted by data

communication and entered automatically into the Regional TFM automation. There will also be

voice communication between the Regional and National TFM faculties. The Regional Traffic

Flow Manager interprets this information and plans activities involved in flight trajectory

planning.

4. Monitor conditions in airspace that will affect trajectories using TFM automation.

Automation may provide alerts when conditions meet certain criteria, such as for Weather,

Special Use Airspace, traffic demand levels, airspace configuration, and any planned changes in

airspace configuration, and user requests for preferences.

For the National Traffic Flow Manager, there may be conditions when it is appropriate to notify

appropriate Regional Traffic Management Units of the need to change trajectories for a set of

aircraft.

5. Change parameters in DSTs and/or other appropriate TFM automation when conditions

monitored in Item 2 indicate changes are needed.

Some parameters may be set automatically, but there are likely to be parameters requiring

interpretation, such as allowable airspace unit traffic levels or interpretation of weather

conditions loading.

6. Monitor or receive notification from automation when there is a need to plan trajectories for

particular aircraft.

The Regional Traffic Flow Manager decides whether trajectory for a particular aircraft needs to

be modified from that provided by the National TFM facility. Automation will likely provide

such an indication. Obtain and interpret any user preferences.

7. Coordinate with AOCs about user requests for preferred trajectories.

The National and Regional Traffic Flow Managers coordinate with the AOC about user

preferred reroutes during flights that can be accommodated. This coordination will happen in

particular when changes in routes are needed due to such conditions as weather, Special Use

Airspace restrictions, and congested airspace. It needs to be determined if this coordination will

be by voice, data communication, or both.
161

8. Plan trajectories for aircraft in your responsibility using DSTs.

For the trajectory planning, use information on airspace conditions and constraints, such as those

listed in Item 2 above, and on equipage and performance characteristics of aircraft in or

approaching the airspace. Collaborate as necessary with other Traffic Flow Managers (i.e.,

National TFM with Regional TFM, and Regional TFM with National TFM). Interact with DSTs

to plan trajectories; evaluate trajectories recommended by DSTs; accept DST recommendations,

run DST again, or make manual modifications.

9. Communicate trajectories to aircraft.

As appropriate for circumstances, communicate trajectories to aircraft via datalink, to

appropriate Controller for communicating to aircraft, or from National Traffic Flow Manager to

appropriate Regional Traffic Flow Manager.

 G.3.2 AOC Roles and Responsibilities for TFM

The AOC will be involved in the TFM concept in coordination with the National and Regional

Traffic Flow Manager regarding user preferred trajectories. The AOC may initiate a request for a

preferred route or may respond to the initiation of a route change by a Traffic Flow Manager.

1. Coordinate with the National Traffic Flow Manger while an aircraft is at a departure gate at the

beginning of a flight about the flight route.

Convey any user preferences regarding the route. Respond and negotiate with the National

Traffic Flow Manager about the route as the National Traffic Flow Manager is planning the

route. The mode for this communication (e.g., voice, data communication, or both) needs to be

determined.

2. Collaborate with National and Regional Traffic Flow Managers about route changes when

contact is initiated by the Traffic Flow Managers.

Convey any changes to user preferences regarding the route. Respond and negotiate with the

National Traffic Flow Manager about the route as the National or Regional Traffic Flow

Manager is planning the route.

3. Initiate collaboration with National and Regional Traffic Flow Managers about route changes

when felt needed by the AOC

Convey any user preferences regarding the route. Respond and negotiate with the National

Traffic Flow Manager about the route as the National or Regional Traffic Flow Manager is

planning the route.

G.3.3 Controller Roles and Responsibilities for TFM

The person who manages traffic in whatever airspace (e.g., sector, super-sector, tubes or corridors,

automated ground-based SA, classic SA, self-separation, etc.) will be called a “Controller” in the

TFM section. It is noted that these “Controllers” follow different procedures and have different

qualifications depending on the airspace unit and its SA mode. Controllers will have DSTs and

automated information provided.

1. Notify Regional TFM of any need for change.

Some examples are asking for change in the number of aircraft in an airspace as affected by

Controller workload, and communicate about the need for trajectory changes (e.g., weather.

sector loading).
162

2. Collaborate with Regional TFM about trajectory changes as needed .

3. Receive trajectory changes from regional TFM supervisors.

Trajectory changes from Regional TFM are likely to be sent via automation, but some may be

via voice in certain circumstances.

4. Review and conduct maneuvers as communicated by Regional TFM.

Use DSTs to review and plan trajectories as appropriate.

5. Communicate with aircraft to provide the trajectory changes.

Send trajectory changes via data link for equipped aircraft, otherwise via voice.

6. Resolve conflicting pilot requests which may be of a nearer-term timeframe than those handled by

Regional TFM facility.

7. Conduct any changes in aircraft trajectories in response to pop-up weather that occur in a

shorter timeframe that Regional TFM can deal with.

8. Resolve conflicting pilot requests that may be of a nearer-term timeframe than those handled by

regional TFM facility

9. Perform other Controller tasks.

For any SA mode assigned to the airspace for which the Controller is responsible, the Controller

needs to perform the tasks for that SA mode.

Perform non-TFM tasks, such as handoffs as appropriate, monitor trajectory execution by

aircraft, respond to emergencies and anomalous situations.

G.3.4 Pilot Roles and Responsibilities in TFM

1. Perform SA tasks appropriate to airspace in which flying (e.g., self-separation, ground-based SA,

classic airspace, etc.).

2. Respond to any maneuver requests by data link or voice from Controller.

3. Provide user requests to Regional TFMs or Controllers.

Participate in any collaborative negotiations as needed. Conduct route re-planning in way

appropriate for the airspace in which flying; use on-board DSTs for this

4. Perform non-TFM tasks appropriate to the airspace in which you are flying.

Perform any tasks related to handoffs appropriate for the airspace in which you are flying.

Monitor movement of convective weather based on out-of-the-window view, onboard radar

system, and net-centric weather information. Handle emergencies and other abnormal situations;

work with Controller as needed.

G.4 Human Factors for TFM

In Appendices D.4 (Separation Assurance), F.4 (ASDO), and H.4 (DAC), human factors were listed

for each RFA. Traffic Flow Management as envisioned in NextGen does not appear to pose many

new human factors issues that are not already covered in the other three RFAs. Thus, the human

performance issues for TFM are presented without delineating them by the nine categories.

Traffic Flow Managers are not normally as much constrained with respect to time. The main jobs of

TFM at national, regional, and terminal levels are to evaluate weather, capacity and flow constraints
163

with a strategic (one hour to one day) horizon using sophisticated new decision support tools; decide

on re-routings and changes in time-based metering; and grant “best equipped, best served”

navigation service as feasible.

We do not anticipate that the workload transients and issues of attention allocation and memory will

be critical for normal operations, though in critical situations (large-scale weather problems, national

emergencies) quick decisions will have to be made under duress. Regional Traffic Flow Mangers

will coordinate with Controllers, AOCs, and TFM personnel in other TMUs, and the National

Traffic Flow Managers.

TFM personnel must have a good sense of flow control and be familiar with the performance of

differently equipped aircraft, information that presumably will be available to them on their DSTs.

They also must have an understanding of the capabilities and limitations of the computer-based

models they employ. Erroneous decisions in traffic flow orders or in airspace reconfiguration can

cause serious consequences on a large scale. These requirements call for special training beyond

that currently required.

The coordination between AOCs and National and Regional Traffic Flow Managers raises some

interesting issues regarding the use of decision support systems (DSTs). What information will the

AOC have? Will it be the same as the Traffic Flow Managers? Will the AOC have similar DST

capabilities as the Traffic Flow Managers, at least for how the AOC will assess possible routes

relating to route negotiation? How will the AOC and Traffic Flow Managers exchange preferences

and possible solution routes (e.g., via voice, data communications, or both) and what will each have

on their screens? There will need to be procedures for the coordination on routes, including how

long the negotiation can continue and how to end the negotiation.

The design of DSTs will require care. Weather, and even future traffic levels, are probabilistic. It

will likely be difficult for Traffic Flow Managers and AOCs to make decisions with probabilistic

estimates of the likelihood of different types of weather occurring. The DSTs will likely need to

make recommendations based on interpretation of these probabilities to give routes that are highly

likely to work. The Traffic Flow Managers and AOCs will need to learn to deal with uncertainty to

make decisions that are highly likely to provide routes that work. For decisions that do not work

out, hopefully these will be rare, the DSTs need to be able to present backup routes.

Under NextGen, flight environments are likely to be more dynamic than today, what with Dynamic

Airspace Configuration, different modes of separation, and different modes of operation under

Airspace Super Density Operations. Thus, traffic flow management planning will likely be complex

on occasion. Workload for Traffic Flow Managers and for AOCs will be a consideration under such

circumstances. These circumstances and the related workload—along with the design requirements

for DSTs and for coordination processes—will need attention.

The “best equipped-best served” policy poses an interesting new issue for human factors, as the

policy is likely to deny some aircraft their traditional requests and expectations while favoring

others. TFM personnel will have to be well trained in diplomacy and how to implement such a

policy.
164

G.5 Human Operations Issues, Design Issues, and Countermeasures for TFM

This section takes the knowledge gained from assembling these human factors observations and

looks at the list of questions presented in Appendix C. Questions are answered in the context of

TFM and a priority rating high, medium, and low has been assigned to each question.

G.5.1 Questions for Use of Automation and DSTs

1. Who is responsible between automation and human for a particular task?

For most of the functions in the TFM concept, automation is mainly behaving as DSTs and

providing recommendations to the Traffic Flow Managers and AOCs, with the Traffic Flow

Manager having responsibility for deciding upon and executing an action.

Controllers get information from the Traffic Flow Manager.

Priority: Low

2. Who has decision making authority when there are conflicting decisions or advice given by some

combination of automation and other operators?

Under TFM, the roles and responsibilities among the Traffic Flow Manager, AOC, Controller,

and pilot are distinct enough that conflicts over authority are unlikely. There may be cases,

however, where there are conflicts in decisions on routes between the Traffic Flow Manager and

AOCs. The Traffic Flow Manager is assumed to have final authority for a routing decision. It is

assumed that there will be some hierarchy between Regional and National Traffic Flow

Managers regarding decision-making authority.

Priority: Low

3. Is automation allowed to execute tasks without the human operator in-the-loop or is automation

providing DST functions with operator in-the-loop?

Automation under TFM mostly operates as a DST and is not expected to operate without the

human operator being in the loop.

Priority: Low

4. Are there any cases where the human operator will need to become involved, such as when there

is an exception to nominal situations?

Route planning are primary duties of Traffic Flow Managers and AOCs. This is not expected to

be an issue for TFM.

Priority: Low
165

5. What is the proper degree of trust by humans in the automation? How much reliance on the

automation is necessary for the concept to work?

TFM will break down as an efficiency enhancing concept if the Traffic Flow Manager feels s/he

cannot rely on the automation’s recommendations, but must take time to come up with his/her

own understanding. Knowledge is needed on how the Traffic Flow Manager will judge and

interpret route recommendations, particularly given the dynamic nature of the airspace

environment under NextGen. Trust in the automation is not expected to be an issue for the

Controller and pilot.

Priority: High (For Traffic Flow Manager and AOC); Low (For Controller and pilot)

6. Do human operators have the appropriate amount of information on their displays to do their

jobs successfully within the time constraints that they have?

Some information that TFM DSTs will deal with are probabilistic (e.g., weather, and even future

traffic levels). While DSTs will likely need to make recommendations based on interpretation of

these probabilities to give routes that are highly likely to work, design of information to present

to Traffic Flow Managers and AOCs for route decision-making does need careful research. Also

under NextGen, flight environments will likely be more dynamic than today (e.g., Dynamic

Airspace Configuration, different modes of separation, and different modes of operation under

Airspace Super Density Operations), and design of information displays under these different

circumstances also need research.

Priority: High

7. If automation/DSTs fails, will human operators realize it and recover?

As with any new concept, the range of failures needs to be defined, from power outages to

automation not calculating the algorithms. Failures are likely to be recognizable, except if the

DST gives erroneous recommendations, which is a problem in most any DST. This would

particularly be a problem if automation provides erroneous information about route

recommendations.

If automation is unable to determine a satisfactory recommendation for routing aircraft,

procedures for the Traffic Flow Manager to make decisions should be designed.

Priority: Medium (Assuming there is time to deal with this situation. Certain types of

failures could be critical; in such cases reassess the question based on type of

failures following concept design.

8. Will the DSTs recognize a problem and help in emergency situations?

The need for automation to indicate emergency or off-nominal events that affect aircraft route

planning needs to be studied, particularly in congested airspace or near weather. There may be

the need for an emergency indicator if dynamic changes cause some aircraft trajectories

previously planned to need rerouting while rerouting of other aircraft is being planned.

Priority: Medium to High (Critical to safety.)
166

G.5.2 Questions for Human Decision-making and Workload

1. Does a single human operator handle multiple operations?

Within the TFM concept, there appear to be few instances of a human operator handling multiple

operations. The Controller will need to perform separation assurance and other normal flight

monitoring and managing functions while communicating with Traffic Flow Managers about

implementing route changes. This may not be a major problem, but there will likely be more

reroutes than today under NextGen.

Priority: Low (need is Low for the case as defined; this could be a more difficult issue if

there are frequent reroutes)

2. Is there complex mixed equipage airspace?

There may be mixed equipage flying in airspace units where the Traffic Flow Mangers are

planning trajectories. The DSTs for TFM will need to have capabilities to handle aircraft with

different types of equipage. Traffic Flow Managers will also need to plan trajectories for aircraft

with different types of equipage. If there is a complex mix of equipage on aircraft (e.g., the

airspace includes very light jets, business jets, commuter aircraft, and air carrier aircraft), in one

airspace unit, then planning trajectories will be more complex. The Traffic Flow Manager will

need knowledge of aircraft equipage and performance along with route structures to change the

airspace., More knowledge is needed about the concept regarding mixed equipage to obtain an

understanding of HF issues for this situation. The AOC will likely be handling a smaller range

of aircraft equipage types that are flown by a particular airline.

Other impacts of mixed equipage would be part of other concepts, such as mode of separation

assurance.

Priority: Medium to High (depending on mix of equipage on aircraft)

3. Is there a sufficient timeframe for decision-making for critical situations, especially off-normal

situations?

The timeframe for planning an airspace configuration change should not be an issue for the

Traffic Flow Manager and AOC unless there are many reroutes to plan in a short period, perhaps

driven by a sudden change in weather.

Priority: Medium

4. Can the human operator attend to multiple alternatives in support of optimization?

The Traffic Flow Manager and AOC will likely work with the DST to examine various

alternatives for routing an aircraft. There is not likely to be excessive pressure on the Traffic

Flow Manager and AOC to attend to multiple alternatives for the rerouting unless there is a short

period to plan reroutes for numerous aircraft, such as might happen with a sudden change in

weather. The Controller will be conducting handoffs, conflict resolutions, and reroutes as

needed, and is not likely to be dealing with multiple alternatives. This question is not an issue

for pilots.

Priority: Medium (for Traffic Flow Managers and AOC; could be higher if there are

numerous flights to reroute in a short time period)
167

5. Are there peak workload situations that should be moderated?

The Traffic Flow Mangers and AOC could have periods of high workload if there are a high

number of reroutes to plan in a short period, such as to deal with a sudden change in weather.

The higher number of aircraft flying in the future under NextGen will add to possible peak

workloads. Handling these reroutes along with handoffs, conflict resolutions, and other tasks

may increase workload unreasonably. It is recommended that the TFM design process address

how the Traffic Flow Manager and AOC can, with DST, handle such a situation, as well as

procedures for the controller. Peak workload for the pilot under TFM should not be an issue.

Priority: Medium to High (in short periods with a high number of reroutes)

6.Can human operators perform all the functions assigned to them?

The human operators in TFM should be able to perform functions assigned to them since the

functions are not particularly complex. The only issue is Traffic Flow Managers, AOC, and

Controllers being able to deal with needed reroutes during a short time period. This is covered in

Question G.5.2.5 immediately above.

Priority: Low

7. What is the workload impact when there is transitioning between airspace configurations or

different modes of operations?

An Airspace Manager within a TMU will plan for airspace reconfigurations and differences in

modes of operations (such as different modes of separation assurance), this is more likely to be a

bigger issue for controllers under other RFA concepts. However, impacts on Traffic Flow

Managers and AOC should be investigated if there are frequent changes in airspace or changes

in mode of separation assurance within an airspace unit.

Priority: Medium

8. If there are mixed surveillance sources with different precisions, can the human operators utilize

the more precise information available for some of the aircraft and the less precise surveillance

information on other aircraft?

This question is not likely to apply to the TFM concept. If separation standards were to be

reduced at some time in the future for some aircraft due to increased precision of surveillance,

TFM DSTs and procedures would need to be revised and this question re-examined.

Priority: Low (unless separation standards were to be reduced at some time in the future)

9. Are there appropriate layers of human operator redundancies built into the concept so that some

operators monitor others to anticipate if help is needed?

The Controllers are expected to operate similarly to today and there will be an Area Supervisor

to oversee controllers and their workload. For the Traffic Flow Manager there will usually be

enough time for them to conduct rerouting work, but the concept design should address oversight

of them, particularly for situations when there are many aircraft to reroute in a short period.

Oversight of AOCs is an airline issue. If the AOC cannot keep up with coordination with the

Traffic Flow Managers, the Traffic Flow Managers would likely go ahead with traffic flow

decisions.

Priority: Medium
168

10. In responding to emergencies or recovering from error, are there standard responses generated

across different modes of operation or does it need to be different for each mode?

Different modes of operation will occur under other concepts, but not likely to be driven by the

TFM concept. This question would need to be investigated under other concepts, such as

Dynamic Airspace Configuration and Airspace Super Density Operations.

Priority: Low

11. How brittle is the concept?

As long as there is not an extremely large number of reroutes to handle in a short time period or

frequent changes in airspace configurations and modes of operation, brittleness should not be a

problem for TFM. But in cases where the above issues apply, ways to mitigate brittleness should

be investigated.

Priority: Medium to High (high under situations with large number of reroutes to handle in a

short time period or frequent changes in airspace configurations and modes of

operation)

G.5.3 Questions for Attention, Situation Awareness, and Memory

1. Are there any excessive demands on attention?

A high demand for attention by Traffic Flow Managers is only likely to occur when there is an

extremely large number of reroutes to handle in a short time period or when frequent changes in

airspace configurations and/or modes of operation require gaining new situational awareness

and/or adjustments in trajectories.

Controllers may need intense attention concentration if rerouting numerous aircraft in a short

time period while monitoring for and resolving conflicts and conducting handoffs.

Priority: Medium to High (high if a large number of aircraft to reroute in a short time period)

2.What is the time required to gain situational awareness to perform a normal task?

The Traffic Flow Mangers may need to gain situational awareness in a short time period if there

is a change in airspace configuration and/or modes of operation. Gaining situational awareness

in short period under TFM is not likely to be a problem for AOC, Controllers, and pilots.

Priority: Medium

3. If the human operator is passively monitoring and suddenly there is a need to get back into the

loop, how long does it take to do this?

The human operators under TFM are not likely to have periods of passive monitoring, followed

by sudden needs to get back into the loop. Their tasks will tend to be continuous.

Priority: Low
169

4. Can automation complement or replace human monitoring function by calling attention to salient

and critical events?

It should be possible to design alerts for notifying Traffic Flow Managers of the need to plan for

situations that require attention and for other critical events under TFM.

Priority: Medium

5. Are there recognizable indicators of emergency and critical safety events?

The need for triggers to indicate emergency or off-nominal events during Traffic Flow

Management should be studied since this is a safety issue. A Traffic Flow Manager could

become so involved in planning a large number of reroutes that s/he does not notice an

emergency without an alert. Also the airspace and trajectories in the future under NextGen may

be complex so that automated alerts of emergencies are needed.

Priority: Medium to High (because this is critical to safety)

6. Are the modes of operations (e.g., automated, self, or controller separation assurance) or

airspace configuration always clear to the human operators (i.e., pilot and/or controller) in both

nominal and off-nominal situations?

The current airspace configuration and modes of separation assurance need to be clear to the

Traffic Flow Managers, AOCs, and Controller. Making these clear to human operators has been

discussed under the ASA, ASDO, and DAC concepts. How to make these clear to human

operators under TFM also needs study.

Priority: Low to Medium (assuming the class of airspace does not change in a

reconfiguration; for example, a sector keeps the same type of separation assurance [e.g.,

keeps as ground-based separation assurance] and same type of operation [e.g., stays as a flow

corridor airspace])

7. Are there situations where mental fixation and “cognitive tunneling” may be a problem?

This is potentially a problem under TFM concept when the Traffic Flow Managers overly fixate

on re-planning complex reroutes of numerous aircraft. The Traffic Flow Manager may focus too

much on a solution to a problem and not take in the larger context of the problem to be able to

re-assess the potential solutions from different perspectives.

Priority: Medium to High

8. What strategies are used to allocate attention during task execution?

This question needs to be addressed for Traffic Flow Managers, AOCs, and Controllers handling

TFM tasks during periods of reroute numerous aircraft, particularly in a short time period.

Priority: Medium to High
170

9. How do the human operators maintain situation awareness of the traffic when airspace and

routes are less structured in NextGen operations?

TFM under NextGen is about planning aircraft trajectories in the less structured and changing

airspace and routes under NextGen operations. This question is somewhat covered in Questions

G.5.3.6, 7, and 8 immediately above. However, it does need attention for TFM since TFM is

about taking advantage of the increased flexibility under NextGen to enhance efficiency.

Priority: Medium to High

10. Are there situations where prospective memory is particularly important?

This is not likely to be a concern for TFM operations. There is some relationship of this

question to Question G.5.3.6 as to the human operators remembering what to do the in the

current airspace configuration and/or mode of separation assurance both for normal operations

and for an anomalous event.

Priority: Medium

G.5.4 Questions for Communication and Coordination

1. Is the coordination overly complex?

The Traffic Flow Manager is the primary decision-maker for changing routes. The Traffic Flow

Manager and AOCs will coordinate on the changes with the AOC requesting preferences. This

should not be a complex coordination, but how the Traffic Flow Mangers and AOC will

exchange information on possible routes and the impacts of these routes does need attention.

The Regional and National Traffic Flow Managers will also coordinate with each other.

Priority: High (because the means of and procedures for coordination between Traffic Flow

managers is important to success of TFM)

2. What impact do time constraints have on flexibility for coordination?

Time constraints are likely to impact the effectiveness of coordination between the Traffic Flow

Managers and AOCs when a large number of reroutes need be planned in a short time period,

particularly when the need for reroutes is sudden, such as a sudden change in weather.

Priority: Medium to High

3. Who or what has final authority?

In current operations, the final authority can be a complex situation in which the authority

changes from National TFM, to regional TFM, then controllers, and pilots depending on the

scale of the traffic problem and the safety criticality of the traffic situation. In the future TFM,

the issue is further complicated by the fact that each human operator relys much more on

automation and DSTs to provide suggestions that s/he may not fully understand. The authority

structure and the necessary coordination should leverage the suggestions existing in the current

system but the details need to be revisited given the new tools, procedures, and team

configuration.

Priority: Medium to High
171

4. How are the trajectory changes negotiated via data link?

The means (e.g., voice, data communications, or both) for Traffic Flow Managers and AOCs to

negotiation reroutes needs to be studied. Effective and timely negotiations are critical to

achieving efficiency benefits and increased granting of user preferences under TFM that are

desired by NextGen.

Priority: High

5. Is communications via data link, voice, or both?

See Question G.5.4.4 immediately above.

Priority: High

6. Are the underlying technological assumptions for communications and information sharing

adequate?

Technical issues, such as whether information can be sent and received with the needed speed,

appropriate encryption, and via the assumed network, are not likely to be a problem in the design

of TFM. It is more the clarity and understandability of possible reroutes between Traffic Flow

Managers and AOCs and between Regional and National Traffic Flow Managers that is the

communication of information issue for TFM.

Priority: Medium

7. Is the information that is shared among multiple operators appropriate regarding content, detail,

and type of format for each particular operator?

As has been stated previously, clear and understandable information is critical for effective and

timely negotiations between Traffic Flow Managers and AOCs and that between Regional and

National Traffic Flow Managers is critical to achieving efficiency benefits and increased

granting of user preferences under TFM and that are desired by NextGen.

Priority: High

8. How important is participating in “party line” communications (i.e., a voice frequency shared

among several persons) as is used in current operations for situation awareness under NextGen?

 “Party line” communications so that multiple human operators beyond a Traffic Flow Manager

and an AOC participate in the same coordination and/or negotiation is not likely to be necessary

for TFM.

Priority: Low

9. How will visual attention demands of data link communication affect other tasks that require

visual attention?

Communication of visual representation of reroutes and their impact is likely to be important for

TFM. However, the attention demands of understanding visuals should not be an impact on

other tasks since the planning of reroute using visuals is central to TFM.

Priority: Low
172

10. How are the emergency responses coordinated between human operators?

Circumstances for emergency responses need to be studied for TFM. They do not seem as

evident for ASA, ASDO, and DAC. However, since this is a safety issue, this does need study.

Priority: Medium to High (because this is a safety issue)

G.5.5 Questions for Organization, Selection, Job Qualification, and Certification Factors

1. What will be organizational structure, interactions between organizational units, and delegation

of authority to units?

The roles between National and Regional Traffic Flow Management Units will need to be

defined. This should not be a difficult issue.

Priority: Medium

2. What skills, abilities, and traits are needed by personnel?

As with all the Research Focus Areas covered in this paper, future human operators will need to

be comfortable using displays and working with DSTs. With airspace configuration and modes

of operation changing dynamically under NextGen, future human operators will need to be

comfortable working under different situations during a shift and from one day to the next.

Traffic Flow Managers will need to be able to assess traffic situations that are more complicated

than today. An issue is whether controllers will have the needed skills to move to a Traffic Flow

Manager position as they move to such positions today.

Priority: High

3. How difficult will it be for human operators to develop an understanding of the capabilities and

limitations of the computer-based models they will employ?

Traffic Flow Managers and AOCs will use DSTs more than today; however, the TFM DSTs

should be straightforward to learn given the personnel have the skills and ability to use DSTs as

is addressed in Question G.5.5.2 immediately above. The computer-based tools for Controllers

will likely not be much different from today; they will be used for more intense periods of

reroutes.

Priority: Medium (for Traffic Flow Managers; Low for Controllers and pilots)

4. What additional training is needed for the human operators to perform the necessary operations

for the given concept?

Training for Traffic Flow Managers in using the new TFM DSTs will be needed along with how

to use them for “what if” and other possible forms of generating and analyzing alternative

trajectories. Training for Controllers will mainly be for the more intensive period of activity.

Both will need training in how to respond to emergencies and remember, perhaps using

indicators from automation, the current airspace configuration or mode of separation assurance.

Priority: Medium
173

5. How will the human operators train for the more diverse and dynamic airspace and route

structures that will exist in NextGen operations?

This is issue is central for Traffic Flow Managers and AOCs to enable the benefits of TFM.

Priority: High

6. Should human operators train to maintain skills that are only needed when the automation fails?

If TFM DSTs fail, and the Traffic Flow Managers know the DSTs have failed, then there will be

less rerouting as is the case today. Safety will not likely be impacted, but capacity will be.

Priority: Medium

G.6 Summary of Key Human Performance Issues for TFM

Section G.4 addressed the perspective of humans performing the tasks, Section G.5 identified gaps

in, and potential countermeasures related to, human performance issues for TFM concepts by

answering questions that take the perspective of concept design.

This Section G.6 summarizes our priority recommendations for TFM HF research by discussing the

questions that were rated High in Section G.5. It is observed that the main drivers from TFM for

these recommendations are the over-reliance on the automation and DSTs, coordination complexity,

and potential issues with situation awareness and workload due to possible short time periods for

planning reroutes (e.g., due to sudden weather).

G.6.1 Use of Automation and DSTs

The issues that arise in using automation and DSTs are trust and reliance on the recommendations of

the DSTs, having sufficient information on displays, and dealing with problems and emergency

situations. The priorities of three questions listed below are rated Medium to High.

Trust and reliance in automation and DSTs

Issue: The National and the Regional Traffic Managers will need to trust and rely on the

automation’s (i.e., DST’s) recommendations and parameter information for TFM to provide

benefits. A capability for the Traffic Managers to examine and independently verify the automation

solutions may be difficult if there are a large number of aircraft and complex flight patterns, or

rapidly changing weather.

Potential Countermeasure/Next Step: Human Factors research is needed to understand how

humans process complex information such as the TFM functions in contrast to the

automation solution. An analysis is needed on how the Traffic Flow Manger will judge and

interpret trajectory and schedule solutions recommended by the DSTs based on the traffic

prediction/scheduling optimization algorithms.

Appropriate information on displays

Issue: Information display for Traffic Flow Managers under TFM will likely be complex since

DSTs will deal with probabilistic situations (e.g., weather, and even future traffic levels) and since

flight environments will likely be more dynamic than today (e.g., Dynamic Airspace Configuration,

different modes of separation, and different modes of operation under Airspace Super Density

Operations). Design of information displays under these different circumstances needs research.
174

Potential Countermeasure/Next Step: Situations for which Traffic Flow Managers will need

to make decisions, types of decisions they will need to make, and types of information they

will need for the decisions should be identified. Potential information displays can then be

prototyped. The displays can be refined via walkthrough or HITL simulation. How to

present probabilistic information (e.g., weather) and consequent recommend trajectories

needs research.

DSTs recognize a problem and help in emergency situations

Issue: Automation will likely need to indicate emergency or off-nominal events that affect aircraft

route planning needs, particularly in congested airspace, near weather, or with changing airspace

configurations under DAC. There may be the need for an emergency indicator if dynamic changes

cause some aircraft trajectories previously planned to need rerouting while rerouting of other aircraft

is being planned. It is recommended that having the automation also make recommendations for

resolving the emergency be explored.

Potential Countermeasure/Next Step: A systematic analysis of all cases where alerts and

recommendations are needed for the Traffic Flow Manager should be conducted. Ways to

provide alerts and recommendations to deal with the emergencies should be formulated and

refined in HITL simulations.

G.6.2 Human Decision-making and Workload

There are issues around human decision-making, mental models, and workload that need attention.

The priorities of following questions were rated Medium to High.

Complex mixed equipage airspace

Issue: Mixed equipage complicates the TFM as different aircraft will have different equipage and

difference performance characteristics that affect trajectory planning. It will be more complex for

the Traffic Flow Manager to plan routes. The Traffic Flow Manager will need knowledge of aircraft

equipage and performance along with route structures to change the airspace.

Potential Countermeasure/Next Step: More knowledge is needed about the concept

regarding mixed equipage to obtaining an understanding of HF issues for this situation.

Once the types of mixed equipage situations and possible rectifying procedures are known,

walkthroughs and HITL simulations should be conducted to understand the limits for human

operators and DSTs to handle mixed equipage and to refine procedures.

Workload impact when transitioning between airspace configurations

Issue: The Traffic Flow Mangers could have periods of high workload if there are a high number of

reroutes to plan in a short period, such as to deal with a sudden change in weather. The higher

number of aircraft flying in the future under NextGen will add to possible peak workloads. The

workload impact for the Controllers could also be a problem for a possible excessive number of

reroutes along with handoffs, conflict resolutions, and other tasks.

Potential Countermeasure/Next Step: Situations where a large number of reroutes may and

where they are accompanied by changing weather and airspace configurations should be

identified. For Controllers, also identify such situations accompanied by a large number of

conflicts and handoffs. The workload can be examined through HITL simulations.
175

Procedures for the Traffic Flow Mangers and Controllers work in such situations can also

be examined in HITL simulations.

Brittleness of concept

Issue: A large number of reroutes to plan in a short time, and perhaps accompanied by change in

weather and airspace configurations, could cause the TFM function to breakdown if there is not

adequate planning for operation in such situations.

Potential Countermeasure/Next Step: Situations where there are a large number of reroutes

in a short time, and with rapidly changing weather and changing airspace configurations,

need to be defined. Then the capabilities of the automation designed. The automation

designs should be tested in simulations to examine the robustness of the automaton for

supporting Traffic Flow Managers and Controllers in a timely manner in all the situations

they are expected to encounter.

G.6.3 Attention, Situation Awareness, and Memory

The issues that arise in using automation and DSTs are related to demand for attention and

situational awareness and for dealing with problems and emergency situations. The priorities of

questions listed below are rated Medium to High.

Demand for attention and situational awareness

Issue: The potential for high attention demands and for mental fixation while planning numerous

reroutes should be investigated. Related to this is the need for strategies to allocate attention during

task execution and to maintain situation awareness when airspace and routes are less structured in

NextGen operations. These are issues for Traffic Flow Mangers, but could also be issues for

Controllers if they reroute numerous aircraft in a short time period while monitoring for and

resolving conflicts and conducting handoffs. It is important for the Traffic Flow Manager to be able

to take in the larger context of the problem to be able to re-assess the potential solutions from

different perspectives.

Potential Countermeasure/Next Step: Situations that may drive a high demand on the Traffic

Flow Manager and Controller as discussed above need to be defined. The ability for a

Traffic Flow Manager and a Controller to handle these situations, and possible strategies

for these operators to handle the demand can be investigated with HITL simulations.

Indicators of emergency and critical safety events

Issue: The need for any indictors of emergency or off-nominal events during Traffic Flow

Management activities needs to be studied since this is a safety issue. A high number of aircraft to

reroute, complex trajectories, and changing airspace configurations could lead to the Traffic Flow

Manager not noticing emergency situations without automated alerts.

Potential Countermeasure/Next Step: Emergency or off-nominal events for which the human

operators under TFM need to be aware should be identified. Responsibilities for automation

and for human operators need to be allocated for monitoring and resolving routes. The

means to alert the human operators can be designed. The potential designs for the human to

be alerted about and to deal with emergences can be validated and refined using HITL

simulations. Those emergencies allocated to automation should also be studied to see if they

can be identified and solved by automation.
176

G.6.4 Communication and Coordination

The issues that arise for communication and coordination are related to the complexity of

coordination and to handling emergency responses under time pressure. The priorities of the

questions below are rated High.

Communication and Coordination for Negotiations

Issue: The issues surrounding communications and coordination to enable negotiations for TFM are

quite complex. Designing DSTs, procedures and human roles for this complex coordination is

critical to achieving efficiency benefits and increased granting of user preferences that are desired by

NextGen. Six questions relating to this topic in Section G.5.4 are rated High or Medium to High.

Coordination linkages include AOC with National and Regional TFM, National TFM and regional

TFM, pilots and AOC, pilots and Controllers, Controllers and TFM, and if DAC exists TFM and

DAC. How these various parties will exchange information on possible routes and the impacts of

these routes need study. Time constraints are likely to impact the effectiveness of coordination

when a large number of reroutes need be planned in a short time period, particularly when the need

for reroutes is sudden, such as due to a sudden change in weather. Airspace reconfigurations under

DAC will complicate coordination.

The means (e.g., voice, data communications, or both) for coordination among the different

operators needs to be studied. Content, detail, and type of format for each particular operator also

needs study to assure clear and understandable information for effective and timely negotiations

among all parties.

Who has final authority after all this coordination is an issue since authority changes from National

TFM, regional TFM, controllers, and pilots depending on the scale of the traffic problem and the

safety criticality of the traffic situation.

Potential Countermeasure/Next Step: Situations that will drive a complex coordination and

negotiation among the different operators need to be defined. The ability for Traffic Flow

Managers, AOCs, Controllers and pilots to handle these situations and possible strategies

for these operators to handle the situations can be investigated with HITL simulations.

These HITL simulations may be complex as all the issues listed immediately above need to be

addressed.

Coordination of emergency responses between human operators

Issue: Depending on the type of emergency, a response could involve coordination among Regional

Traffic Flow Mangers, National Traffic Flow Mangers, Controllers, and pilots. With TFM dealing

with different functions and situations than today, coordination for emergency response needs to be

studied for TFM.

Potential Countermeasure/Next Step: Emergency situations need to be defined along with

potential ways to deal with these situations, who will need to be involved in handling the

emergency, and then what coordination is needed. Coordination and responses in these

situations would then be studied and refined in HITL simulations.
177

G.6.5 Organization, Selection, Job Qualification, and Certification Factors

Different types of tasks and ways of conducting tasks will be part of NextGen. Characteristics of

future human operators and training for the operators drive HF questions of concern. The priorities

of questions listed below are rated High.

Skills, abilities, and traits needed by personnel

Issue: Future human operators will need to be comfortable using displays and working with DSTs

and will need to be comfortable working under different situations during a shift and from one day

to the next. Traffic Flow Managers will need to be able to assess traffic situations that are more

complicated than today. An issue is whether controllers will have the needed skills to move to a

Traffic Flow Manager position as they move to such positions today.

Potential Countermeasure/Next Step: The types of tasks human operators will conduct and

the skills, abilities, and traits needed for these tasks should be identified. Then types of

personnel needed should then be defined. Some HITL simulations may help define what is

needed and how to identify these needs in humans. Career paths should be examined to

identify possible career progressions for people with particular characteristics.

Training for the more diverse and dynamic airspace and route structures

Issue. The human operators will work with in a variety of different situations, such as different

airspace configurations during a shift as well as from one day to the next, different modes of

separation assurance, and more frequent trajectory changes. How to train for working under

changing conditions and how to conduct on-the-job training when conditions change are issues.

Potential Countermeasure/Next Step: Once the situations under which human operators will

work and the characteristics of these workers are defined, then how to train needs to be

studied. Similar work situations in other industries should be examined to learn how these

organizations conduct training. Training experts will need to be involved. It may help to

conduct prototype training sessions to learn about and refine training methods.
178
179

Appendix H: Dynamic Airspace Configuration

Dynamic Airspace Configuration (DAC) increases capacity through strategic airspace organization

and dynamic allocation of airspace structures and controller resources. This includes the

requirement to: 1) develop overall classifications of airspace; 2) identify new classes of airspace for

NextGen operations; 3) develop adaptable airspace with algorithms and technologies for

dynamically changing airspace to accommodate demand; and 4) develop generic airspace

characteristics to increase interchangeability of facilities and controllers.

Under DAC, airspace configurations can be dynamically changed to balance projected airspace

complexity and/or Controller workload. For example, air traffic demand, weather, and other events

may increase the number of aircraft that will need to fly through a particular sector or sectors that

exceeds the number that a Controller can safely handle. Instead of delaying the aircraft in the

upstream sectors to reduce the traffic, DAC reconfigures the sectors to balance the traffic complexity

and/or volume to a level that can be handled by the Controllers, thereby effectively reducing delays

and increasing capacity. Increased capacity could possibly be accomplished without a change in the

number of sectors but the number of sectors could increase if needed. After the demand has

subsided, the sector boundaries could be adjusted to their original positions, assuming that there is a

“normal” set of sector boundaries.

H.1 Overview Description of DAC

The DAC concept addresses changing sector boundaries in real-time to adjust new air traffic demand

to the workload constraints of the Controllers. DAC algorithms formulate optimal sector boundaries

for a particular traffic demand and the results are presented to a human operator (we will call the

person “Airspace Manager”). The Airspace Manager is likely to be a part of the Traffic

Management Unit (TMU) who uses the algorithms and DSTs to plan the new airspace configuration

and decides when to change the configuration.

The following are basic elements of DAC assumed for this identification of human tasks and human

factor issues:

• boundaries of an airspace unit can be changed dynamically

• airspace can be designated for only aircraft with specified levels of equipage

• a variety of airspace types are available (e.g., sectors, super-sectors, corridors or tubes, etc.)

• DAC is likely performed in the TMU along with Traffic Flow Management

• it is assumed that the airspace class will not change (e.g., stay as classic airspace, flow corridor,

etc.)

• DAC can be in both terminal and en route airspace

H.2 Summary of Required Technology for DAC

The ground technology assumed for DAC includes automated Decision Support Tools (DSTs) to aid

the Airspace Manager in selecting airspace configurations and to support implementing a change in

airspace configuration. Controllers will use DSTs in managing the airspace for which they are

responsible. These DSTs will have capabilities to preview the upcoming airspace boundary changes

on their displays. It is assumed that the boundary changes can be propagated digitally to all human
180

operators who need it, including Airspace Manager, Controllers, Front Line Managers, Traffic Flow

Manager, etc.

Aircraft equipage depends on the equipage requirements for a particular airspace. However, data

link is considered to be a prerequisite for DAC in both the air and on the ground. Aircraft need to

have data link to receive 4D trajectories, FMS to process 4D trajectories, and also RNP in certain

airspace types. ADS-B would provide more location precision of aircraft and hence add benefits to

DAC, but it is not assumed as necessary. Aircraft flying in self-separation airspace will be equipped

with automation for use when flying under self-separation.

H.3 Summary of Human Roles for DAC

Three human roles are described in this section on DAC. Two are service provider roles and one is

the pilot:

1. Airspace Manager: makes decisions in real-time on when and how to change the airspace

configuration. The Airspace Manager is assumed to be part of the Traffic Management Unit;

it is expected that duties of the Airspace Manager will include both Traffic Flow Management

and Dynamic Airspace Configuration management unless the DAC duties prove to be large

enough to preclude this.

2. Controller: monitors and manages an assigned airspace. During sector reconfiguration, s/he

needs to transfer ownership of aircraft to their proper sectors after the boundary change by

coordinating with the neighboring sector Controllers.

3. Pilot: executes clearances issued by automation or a human operator and participates in

handoffs. When necessary, the pilot coordinates the maneuvers with a Controller. The pilot

can also request reroutes.

The roles and responsibilities hypothesized for these three operators are described further in the

following sub-sections.

H.3.1 Airspace Manager Roles and Responsibilities for DAC

The following are tasks hypnotized to be performed by the Airspace Manager working in the TMU:

1. Monitor conditions and decide when to change airspace configuration.

With the help of DSTs, the Airspace Manager monitors and evaluates conditions and assesses

when to change airspace configurations. The Airspace Manager will interact with others in

the TMU on need to change airspace configurations

2. Decide on new airspace configuration.

In the mid-term timeframe, the airspace configuration change will likely be selected from

predefined airspace alternatives or by following predefined procedure to change the airspace.

Predefined guidelines will be followed in these decisions, and DSTs will be used. The aircraft

equipage to be required for the reconfigured airspace will be specified.

3. Plan the airspace configuration change.

Airspace reconfigurations to be made will be integrated with surrounding airspace so that all

appropriate airspace will operate cohesively. The Airspace Manager will work with other
181

TMU personnel on any needed aircraft route changes to take advantage of the new airspace

configurations.

The Airspace Manager will notify and work with Controller(s) in the airspace units affected

by the airspace configuration change about the upcoming airspace change. Controllers are

expected to have an advisory role in both the revised configuration of the airspace as well as

the timing for the change. If the number of airspace units is to change as part of the

reconfiguration, the Airspace Manager will work with the supervisor(s) of the Controllers so

that a re-assignment schedule for individual Controllers to the new airspace units is

developed. This is needed so that all airspace units are covered by a Controller qualified for

the particular new airspace. The Airspace Manager or the Controller supervisor(s) notifies

Controllers of their new airspace assignments for the new airspace configuration.

4. Initiate action to change airspace configuration.

The Controller(s) are notified to implement their roles for airspace configuration change. The

Airspace Manager works with Controllers to implement new airspace configurations as

needed.

The Airspace Manager assures that all relevant parties are aware of airspace reconfiguration

(e.g., Controllers, TMU personnel, other Airspace Managers, etc.).

5. Deal with any emergencies or anomalies.

The Airspace Manager will act as needed to respond to emergencies or anomalies, such as the

need to operate an airspace in a different manner than intended, e.g., airborne equipage

failures causing several aircraft to revert to classic separation assurance in an automated

ground-based SA airspace. The Controller is likely to initiate emergency responses but may

need to involve the Airspace Manager.

H.3.2 Controller Roles and Responsibilities for DAC

The following are tasks hypothesized to be performed by the Controller (the generic term

“Controller” is used regardless of airspace unit being worked). The airspace unit is initially assumed

to be TBO airspace but it may also be a corridor or tube, an airspace unit with automated groundbased separation assurance, or a self-separation airspace unit. It is noted that the Controllers within

these different airspace units will follow different procedures and may have different qualifications.

However, the tasks below are general enough that they apply to any Controller operating in any of

these airspace units.

1. At beginning of a shift, the new Controller interacts with the departing Controller to receive

information about current airspace assignments and any expected change in airspace

configuration of which the departing Controller is aware.

This task is assumed since airspace configurations a particular Controller will work may be

different from one shift to another.

2. Prepare to implement a change in airspace configuration.

The Controller will receive notification about an imminent change in airspace and will work

with the Airspace Manager on the change. For example, the Controller could provide advice

on both the revised configuration of the airspace as well as the timing for it to change.
182

For more complex airspace reconfigurations where the number of sectors changes or the mode

of Separation Assurance changes, a controller supervisor will likely be involved to assign new

controllers to the added sectors or decide which controllers will no longer work a sector if the

number of sectors decreases. Under such reconfigurations, a Controller will likely receive

notification of new airspace assignment under the new airspace configuration from the

Controller supervisor. The Controller will review flights to be reassigned in the configuration

change, will review roles and responsibilities in the new airspace (with help from a DST), and

will conduct final planning for changing the airspace configuration. If applicable, the

Controller interacts with Controller(s) who managed aircraft in previous airspace

configuration that are now managed her/him in the new airspace configuration. This

interaction will review any issues or planned maneuvers that are outstanding. Is it noted that

such an interaction goes both ways, for aircraft previously managed and aircraft to be

managed.

3. Implement change to new airspace assignment and begin control of new airspace.

Perform handoffs, as applicable depending on level of automation for handoffs, to transfer

aircraft that are now assigned to another Controller and receive aircraft that are now their

responsibility.

4. Resolve any conflicts that occur during configuration change.

5. At the end of shifts, interact with replacement Controller to inform her/him of current airspace

configuration and any expected change in airspace configuration of which the departing

Controller is aware.

6. Perform routine tasks for the assigned airspace.

These tasks include performing separation assurance in the appropriate mode for their

assigned airspace unit, handoffs, and assisting pilots in handling emergencies and other

abnormal situations.

H.3.3 Pilot Roles and Responsibilities for DAC

At this time, it is not known whether there will be tasks for the pilot involved in the aircraft flying

during a transition to a new airspace configuration. It may be that the pilot only performs needed

tasks for a handoff (e.g., communication frequency change), executes a trajectory changes pertinent

to the new airspace configuration, or has no involvement in the configuration change. A pilot might

be consulted by the Airspace Manager in the TMU or by a Controller during the planning for an

airspace configuration change on such matters as weather or ride comfort.

H.4 Human Factors for DAC

The human tasks in Section H.3 were examined for the ability of humans to perform the tasks, both

from a safety consideration and from performing the tasks in a manner that the benefits of DAC

would be achieved. The human performance issues identified are described below for the Airspace

Manager, Controller, and pilot. The nine human factors categories defined in Appendix B were used

for this assessment.
183

H.4.1 Attention Allocation

Under DAC, as under many of the new concepts that will be part of NextGen, there will be a greater

variety of tasks to be performed by all human operators involved in operations of the National

Airspace System (NAS). With a greater variety of duties to be performed by any one person (i.e.,

multitasking, sometimes under a high workload), there will be more Attention Allocation, Situation

Awareness, and Monitoring issues.

One example of Attention Allocation, Situation Awareness, and Monitoring issues is that the

Airspace Manager(s) in a TMU who will be responsible for DAC will have other duties besides

those for DAC. The Airspace Manager must monitor the traffic flow decision support tool (DST) at

a sufficient rate to observe and be aware of changing flow parameters, including weather, winds,

need for traffic diversions, and abnormal traffic in multiple sectors. Since the person charged with

Airspace Manager duties will not likely be working at these duties continually and will have other

work in the TMU, appropriate scan frequencies and patterns need to be established by experiment.

Since the need to change airspace configuration under DAC is not likely to be extremely frequent or

urgent, vigilance during a long period with little change in flow parameters could be a concern.

Thus, there are human performance issues regarding scan frequencies, and perhaps alerts, indicating

the changing parameters require observing that an airspace reconfiguration may be appropriate.

The above paragraph implies the question of how much the Airspace Manager concerned with DAC

can perform other TMU duties? More knowledge on the likely frequency of airspace

reconfiguration and on the workload for performing airspace reconfiguration will be needed to

answer this question.

En route sector Controllers would normally also be scanning displays of traffic in their own and

adjoining sectors. The major HF concern with attention allocation is that the scanning be

sufficiently often, given other workload demands, and that the displays be designed to reliably reveal

the kinds of traffic parameters of concern, such as sector boundaries, traffic levels, winds and

weather, special use airspace, and differentiation between equipped and unequipped aircraft. This

concern of information display regarding awareness is important since the airspace unit for which

the Controller is responsible will be changing under DAC, and therefore parameters that were

normally constant, such as sector boundaries and special use airspace, may change. Where the latter

variables can be measured quantitatively, visual or auditory alerts of significant changes can be

employed.

When a Controller first begins working a particular airspace, either at the beginning of their shift on

a particular day or when assigned to a newly configured airspace during their workday, the

Controller will need to gain awareness of the airspace to which assigned. This is because airspace

units will be changing. Some of this awareness might be provided by DSTs and some by verbal

briefing from the Controller working the airspace unit immediately before.

Pilots would not normally be attending to the need for DAC.

H.4.2 Decision-making and Mental Modeling

The decision to reconfigure airspace would be taken by the Airspace Manager after appropriate

notice of intention and feedback from sector Controllers. An Airspace Manager should have a
184

written “playbook” of pre-defined airspace configurations (e.g., based on wind directions, weather,

and major en route flows for time of day) with a code designation for each “play.” Occasionally

circumstances may require a non-playbook action, and in this case reconfiguration decisions should

be clearly identified in communications. The reconfiguration should then involve a negotiation as to

which specific aircraft would be the last ones accommodated under the old configuration and which

one would be the first under the new configuration. Complexity to make a decision is an issue,

particularly for non-playbook airspace configuration.

There are likely to be limits as to the amount and type of criteria that can be processed for DAC

decision-making; on the other hand the criteria should be complete and detailed enough that

beneficial airspace reconfiguration decision are made. Another performance issue is how much does

a DST recommend and how much does it allow the Airspace Manager to assess the situation. This

has implications for an Airspace Manager’s confidence in the DST recommendation as well as in

timely performance.

The sector Controllers presumably would influence the decision only with regard to their

compliance, their request for an alternate reconfiguration plan, or their suggestions or request for

alternate timing of the change. They may make input on their workload as it affects the need for an

airspace reconfiguration.

Pilots would not normally be part of DAC decision-making, other than providing off-the-nose

information on weather and wind conditions. The Airspace Manager may have exchanges with

Airline Operations Centers (AOCs).

H.4.3 Communication

The Airspace Manager will have conversations with other workers in the TMU, which is likely to be

face-to-face communication. When there is seen to be a need for airspace reconfiguration, there

should be the option for the Airspace Manager to type data-link messages to or to use voice

communication with one or multiple (simultaneous broadcast) sector Controllers to suggest a need to

reconfigure. This should be done in plenty of time to effect the reconfiguration. Some

reconfigurations may be standard for time of day. The Airspace Manager may communicate with

AOCs in deciding on re-configurations.

Appropriate communication modes, use of the modes, and content to communicate are human

performance issues. Also, will there be situations where it is important that the participants (e.g.,

Airspace Manager, other TMU workers, Controllers, Pilots, AOCs) have a common understanding

of the situation and reconfiguration action to be taken.

When informed of a potential airspace reconfiguration, sector Controllers may then reply with their

concerns (e.g., indicating that reconfiguration should not occur until some rerouting is completed).

Here the use of a “party line” should be evaluated for multi-party discussion of if, when, and how to

reconfigure. For a sufficiently long time window and expected “playbook” reconfigurations, datalink communication would probably suffice. There should be standardized terminology and format

for DAC communications.
185

If the reconfiguration calls for reassignment of sector Controllers, that may require voice

communication to managers at the appropriate centers, with confirmation that the new assignments

are clearly understood and will be staffed on time.

There will likely need to be face-to-face communications between a Controller exiting from serving

a particular airspace and the Controller beginning to serve that airspace. This communication will

bring the new Controller up-to-date on conditions in that airspace. This could occur at the beginning

of a Controller's shift or when a new Controller is assigned to an airspace unit as part of a

reconfiguration. Such communication would be needed, since with DAC there could be changes in

the airspace of which the new Controller needs awareness.

Whether the Controller and pilots will communicate about implementing airspace configuration

changes will depend on whether pilots need information about reconfiguration or whether the

airspaces through which they fly are transparent to them. This issue should be addressed during the

design of the airspaces that will be re-configured.

H.4.4 Memory

Retrospective memory issues might involve feedback on reasons why a particular reconfiguration

geometry or schedule would be inappropriate (communicated verbally by a sector Controller or by

several sector Controllers) that might be forgotten by the Airspace Manager in the TMU in the

confusion of reconfiguration implementation.

Prospective memory problems might occur where a sector Controller becomes so distracted by a

reconfiguration exercise that an impending separation violation is overlooked. When computerbased information is available regarding reconfiguration timing, auditory or visual prompts may be

employed to overcome such a problem.

If the response to an event by a Controller in the airspace unit being worked is sufficiently different

from responses in other airspace units previously worked (e.g., there will be more airspace

configuration types under DAC, and will these different types be worked during one shift or in

consecutive days?), then there may be memory issues for the Controller remembering which

airspace they are working and how to respond to an event. Perhaps reminders provided by DST will

be useful.

An issue is whether the additional tasks to conduct under DAC cause memory issues of what to do in

any particular task in a particular airspace unit being worked.

Memory issues for pilots will depend on whether a pilot would respond differently to some event

occurring, whether urgent or not, depending on the airspace configuration in which the pilot is

flying. If the pilot is expected to respond differently in different airspace configurations, then

memory of which airspace configuration type the aircraft is operating in and memory of how to

respond to the occurrence in that airspace configuration would be human performance issues.

H.4.5 Workload

The workload issues for the Airspace Manager, Controller, and pilot are discussed together in this

section, since the extent of overlapping tasks affects the workload of each.
186

Mental workload is anticipated to be a major HF concern in NextGen for all the human actors in

various phases of flight. When a particular airspace configuration change is a routine change (i.e.,

according to the playbook) and when there is sufficient anticipation by all parties, no one’s workload

should be excessive. But there can be excessive mental workload when a reconfiguration is imposed

unexpectedly or with a tight time deadline, or when a sector Controller and/or a pilot are just in

process of communicating about, and getting set on, a modification in routing around weather or are

in the process of recovering from a TSAFE or TCAS maneuver to reestablish separation. Workload

issues would be affected by the extent of criteria for DAC and the complexity of the decision

process, including amount of communications with others. There may be a need for strategy for

shedding workload if necessary.

H.4.6 Interaction with Automation, Decision Support Tools, and Displays

Presumably both the TMU and the sector Controller will have decision support tools that include:

1. Predictive models of airport and regional capacity and flow

2. Capability to examine weather and traffic flow anywhere in the NAS on which particular

latitude/longitude regions or altitudes can be selected, as well as traffic areas and equipage of

aircraft (much of this exists today)

3. Advice on how to modify airspace configuration with “what would happen if...” capability

4. Provision to communicate with sector Controllers and others as necessary

This Decision Support Tool (DST) capability might be replicated at the National Command Center

as well.

The DST suites should include graphical depictions of current and (selected) planned airspace

configurations. There is a question whether the pilot will have, or even should have, such a

depiction; probably not.

There are multiple variables concerning human-automation: 1) generic usability; 2) human factors

assessment of the display design; 3) user’s knowledge of what the automation is designed to do and

capable of (and with what confidence); 4) user’s knowledge of what a DST is not designed to do and

what should never be expected of it; and 5) user’s knowledge and skill in using it. These

considerations apply to all the above-mentioned DSTs.

It was mentioned under previous categories of human performance issues that it may be helpful for

DSTs to remind the users of the type of airspace unit in which they are operating and what actions

appropriate to particular events that might occur are.

H.4.7 Organizational Factors

A major organizational factor is the authority of the TMU or someone else to dictate an en route

airspace reconfiguration. There may also be new roles for multi-sector planners who play a major

role in arranging upstream flow to be compatible with downstream flow in the reconfigured

airspace.
187

It is likely that the National Command Center will have authority to overrule a regional TMU.

Airline operation centers may or may not be involved. We foresee that sector Controllers will have

an advisory role in both the revised configuration of the airspace as well as the timing for it to

change (i.e., which arriving and which departing aircraft will be last for the old and first for the

new). All these actors must have an appropriate mental model of the command and control structure

and how it is meant to work.

With changes in the size of airspace units being part of DAC, there may be changes in the number of

Controllers needed during a standard duty shift period at an en route center. An issue is how the

centers will be staffed with Controllers so that this change in the number of Controllers due to DAC

can be accommodated. Questions are: 1) Where will the Controllers come from when more

Controllers are needed due to an airspace configuration change to small, more numerous airspace

unit; and, 2) what will happen to Controllers not needed when airspace units are increased in size

and require fewer Controllers? There is a line of authority issue concerning the role of the

Controller manager to assign Controllers to newly reconfigured airspace units. This Controller

manager may need to be consulted by the Airspace Manager on the feasibility of a particular

airspace reconfiguration concerning adequacy of staff for the new configuration.

H.4.8 Potential Errors and Recovery from Human Errors or System Failures

Examples of human errors that might occur in airspace reconfiguration are:

1. An inappropriate reconfiguration is imposed that makes traffic flow worse and adds delays

2. The timing for the reconfiguration is ill advised so that some aircraft are stranded in the old

configuration and may have to be dealt with in an awkward manner to maintain the planned

flows

3. Some sector Controllers do not undertake new assignments as instructed, or are confused

about them (e.g., which plan in the playbook is to be used)

4. Some sector Controllers object to the reconfiguration as planned, there is back-and-forth

negotiation, and there is additional delay or lack of clarity in how and when a reconfiguration

will occur

In each such case there must be anticipated recovery procedures. These should include additional

communication to clarify and check for understanding by all parties involved.

As was motioned under Section H.4.4 Memory, if operating actions are sufficiently different from

one airspace configuration to another, there is a possibility of erroneous actions by not remembering

the airspace type being worked or flown in and not remembering actions appropriate to that airspace

type. Further, errors are possible in responding to erroneous actions due to memory issues over

which airspace unit type is being used and how to respond to an erroneous event in that airspace

type. Perhaps reminders, of both airspace configuration unit being worked and appropriate actions

and error responses for that airspace unit, provided by DST will be useful.

H.4.9 Potential Selection, Certification, and Training

Mathematical models used in DSTs for predicting and evaluating capacity and flow and for initiating

airspace reconfiguration decisions, will have to be tested, validated, and verified. Training in their

use will be needed. Certainly in the TMU, there will be more use of DSTs, not only for DAC but
188

DSTs for other purposes also. A requirement for some “computer smarts” is likely to be part of

selection criteria for those working in the TMU.

Certification by the FAA has traditionally been of hardware and software relatively independently of

the ability of the various personnel to use it. Certification of Controllers in various positions has

been relatively independent of the hardware/software design. It may be that in the future

certification of DST hardware/software systems and of the human users will have to be done jointly,

at least to some degree.

A requirement for some “computer smarts” is likely to change the Controller selection criteria.

Also, Controllers will need to have the ability to adapt to working different airspace units in a shift

and from one day to the next.

As with the TMU above, it may be that in the future certification of DST hardware/software systems

and of the human users will have to be done jointly, at least to some degree.

Any change in pilot characteristics is likely to be influenced by the extent of changes in the use of

on-board DSTs under DAC, which is not expected to be significant.

H.5 Human Operations Issues, Design Issues, and Countermeasures for DAC

Section H.4 discussed Human Factor issues for DAC. This subsection takes the knowledge gained

from assembling these human factor issues and looks at the list of questions presented in Appendix

C. As discussed in Section 3.3 and Appendix C, these questions will be addressed in the context of

the DAC concept. Not all of these questions will be design issues for DAC. However, each

question will be covered and comments will be made regarding how the question relates to DAC.

Again as discussed in Section 3.3 and Appendix C, a priority rating high, medium, and low has been

assigned to each question. These sections also discussed the criteria behind assigning a priority

rating.

H.5.1 Questions for Use of Automation and DSTs

1. Who is responsible between automation and human for a particular task?

For most of the functions in the DAC concept, automation is mainly behaving as DSTs and

providing recommendations to the Airspace Manager, and thus the Airspace Manager has

responsibility for deciding upon and executing an action.

Controllers get information from the Airspace Manager. This question does not affect pilots

under DAC.

Priority: Low
189

2. Who has decision making authority when there are conflicting decisions or advice given by some

combination of automation and other operators?

Under DAC, the roles and responsibilities among the Airspace Manager, Controller, and pilot are

distinct enough that conflicts over authority are unlikely. There may be cases, however, within

the TMU where there are conflicts in decisions on airspace reconfigurations among the Airspace

Manager and other employees in the TMU. It is assumed that will be some hierarchy within the

TMU regarding decision-making authority.

Priority: Low

3. Is automation allowed to execute tasks without the human operator in-the-loop or is automation

providing DST functions with operator in-the-loop?

Automation under DAC mostly operates as a DST and is not expected without the human

operator being in the loop.

Priority: Low

4. Are there any cases where the human operator will need to become involved, such as when there

is an exception to nominal situations?

The Airspace Manager may have settled on a particular airspace configuration when there is a

notice by the DAC automation that conditions indicate the need for a different airspace

reconfiguration. However, this is expected far enough in advance of the implementation of an

airspace reconfiguration that this question will not be an issue. The Controller is also expected

to have a sufficient notice of an airspace reconfiguration from the Airspace Manager that the

question is also not expected to be an issue for the Controller. It is not expected to be an issue

for the pilot.

Priority: Low

5. What is the proper degree of trust by humans in the automation? How much reliance on the

automation is necessary for the concept to work?

DAC will break down as an efficiency enhancing concept if the Airspace Manager feels s/he

cannot rely on the automation’s recommendations but must take time to come up with their own

understanding of the future traffic situation. Research is needed on how the Airspace Manager

will use the DSTs/automation to judge and interpret boundary change impacts. Trust in the

automation is not expected to be an issue for the Controller and pilot.

Priority: Medium to High (for Airspace Manager; Low for Controller and pilot)

6. Do human operators have the appropriate amount of information on their displays to do their

jobs successfully within the time constraints that they have?

It should be possible to design DAC automation to provide human operators with sufficient

information for the concept to operate efficiently. Care should be taken in the presentation of

predicted information (e.g., traffic flows, weather, etc.) so that the Airspace Manager will

understand the likelihood of future conditions and be able to make airspace reconfiguration

decisions. This is a HF issue to address when close to implementation.

Priority: Low to Medium
190

7. If automation/DSTs fails, will human operators realize it and recover?

Failures are likely to be recognizable, except if the DST gives erroneous recommendations,

which is a problem in most any DST. This may be a significant problem for the Airspace

Manager if automation provides erroneous predictions about the future traffic and therefore

recommends inappropriate airspace reconfiguration. Wrong configurations may result in sector

overload but the Controllers and the Area Supervisors should alert the Airspace Manager of such

situations.

If automation is unable to determine a satisfactory recommendation for airspace reconfiguration,

procedures for the Airspace Manager to make decisions should be designed.

Priority: Medium

8. Will the DSTs recognize a problem and help in emergency situations?

How automation indicates emergency or off-nominal events during airspace configuration

changes needs to be studied, particularly when aircraft are near the boundary of the new or old

airspace configuration. There may be the need for an emergency indicator if not all aircraft

trajectories have been checked for compatibility with the configuration change or if some aircraft

is deviating from its planned path during an airspace configuration change.

Priority: Medium to High (critical to safety)

H.5.2 Questions for Human Decision-making and Workload

1. Does a single human operator handle multiple operations?

Within the DAC concept, there appear to be few instances of a human operator handling multiple

operations. The Controller will need to perform separation assurance functions while changing

airspace unit assignment. Procedures for dealing with separation assurance in this circumstance

will need to be developed. The Airspace Manager will likely be performing regular TFM duties

between instances of airspace reconfiguration, and there will need to be procedures to relieve the

Airspace Manager of TFM duties when DAC duties need attention.

Priority: Low (need is Low for the case as defined; this could be a more difficult issue if

there are complex or frequent airspace reconfigurations)

2. Is there complex mixed equipage airspace?

There may be mixed equipage flying in airspace units under going reconfiguration. Procedures

will need to be developed for transferring control of variously equipped aircraft to the new

airspace units during the dynamic airspace reconfiguration. If there is a complex mix of

equipage on aircraft (e.g., the airspace includes very light jets, business jets, commuter aircraft,

and air carrier aircraft), then developing procedures for controllers to handle switching control of

aircraft may be more complex for this to be done in the short time required. There may also be

more work to address for the Airspace Manager to plan the new airspace configuration and the

transition. The Airspace Manager will need knowledge of aircraft equipage and performance

along with route structures to change the airspace. More knowledge is needed about the concept

regarding mixed equipage to obtain an understanding of the HF issues.

Other impacts of mixed equipage would be part of other concepts, such as mode of separation

assurance.

Priority: Medium to High (depending on mix of equipage on aircraft)
191

3. Is there a sufficient timeframe for decision-making for critical situations, especially off-normal

situations?

The timeframe for planning an airspace configuration change should not be an issue for the

Airspace Manager. When a change in airspace configuration is planned, setting the time when

the change will occur should be planned so that the timing of the change will not cause duress

for any of the human operators. There may be a timeframe issue for the Controller adjusting to

the new airspace and gaining situation awareness in the new airspace when the Controller has to

quickly deal with a potential conflict. This has a high degree of importance for Controllers and is

low for the Airspace Manager and pilot.

Priority: Medium (for Controller; Low for Airspace Manager and pilot)

4. Can the human operator attend to multiple alternatives in support of optimization?

The Airspace Manager is likely to work with the DST to examine various alternatives for an

airspace reconfiguration. Since the Airspace Manager will be acting in advance of the

implementation of the reconfiguration, there is not likely to be excessive pressure on the

Airspace Manager to attend to multiple alternatives for the reconfiguration. The Controller will

be conducting handoffs, and conflict resolutions as needed, and is not likely to be dealing with

multiple alternatives. This question is not an issue for pilots.

Priority: Low

5. Are there peak workload situations that should be moderated?

Since the Airspace Manager will be acting in advance of the implementation of the

reconfiguration, there should be time for the Airspace Manger to plan an airspace

reconfiguration and there will not be situations where peak workload needs to be moderated.

The Controller could have a peak workload in handling the handoffs, and conflict resolutions if

needed, to implement an airspace reconfiguration. It is recommended that the DAC design

process address how the Airspace Manager can, with DST, anticipate such a situation for the

controller and formulate the airspace reconfiguration process to deal with this situation. A high

level of conflict resolutions occurring during the airspace configuration could be a problem.

This circumstance also should be addressed in the DAC concept design. Peak workload for the

pilot under DAC should not be an issue.

Priority: Medium (for Controller, degree depends on the need to handle conflict resolutions

during the reconfiguration; Low for Airspace Manager and pilot)

6. Can human operators perform all the functions assigned to them?

The human operators in DAC should be able to perform functions assigned to them since the

functions are not particularly complex. The only issue is Controllers being able to deal with any

needed conflict resolutions that occur during the sector reconfiguration in addition to handoffs.

This is covered in Question H.5.2.5 immediately above.

Priority: Low
192

7. What is the workload impact when there is transitioning between airspace configurations or

different modes of operations?

This question for DAC also relates to Questions H.5.2.5 and 6 immediately above. The

workload impact of concern is for a controller dealing with handoffs and any needed conflict

resolutions that occur during the sector reconfiguration.

Priority: Medium to High (for Controller, degree depends on the need to handle conflict

resolutions during the reconfiguration; Low for Airspace Manager and pilot)

8. If there are mixed surveillance sources with different precisions, can the human operators utilize

the more precise information available for some of the aircraft and the less precise surveillance

information on other aircraft?

This question is not likely to apply to the DAC concept.

Priority: Low

9. Are there appropriate layers of human operator redundancies built into the concept so that some

operators monitor others to anticipate if help is needed?

The Controllers are expected to operate similarly to today and there will be an Area Supervisor

to oversee controllers and their workload. For the Airspace Manager there will usually be

enough time for them to conduct their work on airspace re-configuration, but the concept design

should address their oversight.

Priority: Medium

10. In responding to emergencies or recovering from error, are there standard responses generated

across different modes of operation or does it need to be different for each mode?

Under the DAC concept there are not different modes of operations (e.g., changing modes of

separation assurance) but there are different airspace configuration in which the controller will

operate. The controller will need to know the current airspace boundaries in which s/he is

working. However, the response to an emergency can be standard across the airspace

configurations. For the Airspace Manager, this is not such an issue, but s/he will need to know

the current airspace configuration.

Priority: Medium (for controller; Low for Airspace Manger and pilot)

11. How brittle is the concept?

To avoid brittleness, controllers will need an automation tool during airspace reconfiguration to

support dealing with safety critical events so they can maintain needed situation awareness (e.g.,

conflict probe to know if aircraft are separated properly). If controllers do not have such an

automation tool, then there will be a need to limit the type of configuration changes (e.g., limit to

splitting and combining sectors and not adding or deleting sectors) so that it is easy for

controllers to maintain needed situation awareness.

For the Airspace Manger, the airspace configuration in operation could continue to be used if the

Airspace Manager is not able to plan a new airspace configuration. There would be a negative

impact on efficiency, but safety would not likely be affected. Handling problems that might

occur during a transition to a new airspace configuration will need to be covered during detailed

concept design.

Priority: High (for Controller; Low for Airspace Manager and pilot)
193

H.5.3 Questions for Attention, Situation Awareness, and Memory

1. Are there any excessive demands on attention?

Controllers may require significant attention in monitoring handoffs and conflicts of new and

unfamiliar aircraft during a change in airspace configuration. The potential for high attention

and situation awareness demands should be investigated and mitigated.

Priority: Medium to High (for Controllers; Low for Airspace Managers and pilots)

2. What is the time required to gain situational awareness to perform a normal task?

The time for Controllers to gain situational awareness to performance separation assurance as

well as handoffs during the implementation of a new airspace configuration needs study. This

should not be an issue for Airspace Mangers unless there are frequent or complex

reconfigurations. The pilot will not be affected by this question.

Priority: Medium to High (for Controllers; Medium for Airspace Managers; Low for pilots)

3. If the human operator is passively monitoring and suddenly there is a need to get back into the

loop, how long does it take to do this?

The Airspace Manager(s) in a TMU must monitor the DAC decision support tool at a sufficient

rate to be aware of the need for an airspace reconfiguration by observing changing flow

parameters, including weather, winds, need for traffic diversions, and abnormal traffic in

multiple sectors. Since this person will likely have other duties in the TMU, appropriate scan

frequencies and possible alerts indicating the need for airspace configuration change will need

attention. Controllers will be consulted on the timing for the airspace change and should have

time to prepare for it.

Priority: Low

4. Can automation complement or replace human monitoring function by calling attention to salient

and critical events?

There will likely be automated alerts to indicate to the Airspace Manger to examine whether an

airspace reconfiguration is needed. There will likely be alerts to the controller about possible

conflicts of aircraft during an airspace reconfiguration.

Priority: Medium (for Controllers and Airspace Managers; Low for pilots)

5. Are there recognizable indicators of emergency and critical safety events?

Suitable triggers to indicate emergency or off-nominal events during airspace configuration

change needs to be studied, particularly for aircraft near the boundary of the new or old airspace

configuration. There may be the need for emergency triggers if not all aircraft trajectories have

been checked for compatibility with the configuration change or if some aircraft is deviating

from its planned path during an airspace configuration change.

Priority: Medium to High (critical to safety)
194

6. Are the modes of operations (e.g., automated, self, or controller separation assurance) or

airspace configuration always clear to the human operators (i.e., pilot and/or controller) in both

nominal and off-nominal situations?

The Airspace Manager and Controller will need to know the current airspace configuration under

which they are operating. The extent of this question is influenced by the complexity of airspace

changes and on the frequency of changes. Airspace configuration will likely not be changed

very frequently. However, it is likely that there should be a reminder to these human operators

about the current airspace configuration for which they are responsible. There may also need to

be reminder on how to respond to emergencies and anomalous situations depending on the

airspace configuration.

Whether the pilot needs a reminder on the current airspace configuration needs further study.

Priority: Low to Medium (assuming the class of airspace does not change in a

reconfiguration. For example, a sector keeps the same type of separation assurance

[e.g., keeps as ground-based separation assurance] and same type of operation [e.g.,

stays as a flow corridor airspace].)

7. Are there situations where mental fixation and “cognitive tunneling” may be a problem?

This is not likely to be a problem under DAC concept unless the Controller overly fixates on

handoffs to the point of delaying conflict resolutions during the implementation of an airspace

reconfiguration or overly fixates on conflicts to the point of delaying handoffs.

Priority: Medium (for Controller; Low for Airspace Supervisor and pilot)

8. What strategies are used to allocate attention during task execution?

This question needs to be addressed for controllers handling separation assurance and handoffs

during the implementation of a reconfiguration. Airspace Managers will likely have sufficient

time to perform airspace reconfiguration planning. This should not be an issue for pilots under

DAC.

Priority: Medium (for Controller; Low for Airspace Supervisor and pilot)

9. How do the human operators maintain situation awareness of the traffic when airspace and

routes are less structured in NextGen operations?

Under DAC this question is subsumed under Questions H.5.3.6, 7, and 8 immediately above. It

is a general question that needs attention in NextGen.

Priority: Medium

10. Are there situations where prospective memory is particularly important?

This is not likely to be a concern for DAC operations. There is some relationship of this

question to Question H.5.3.6 as to the human operators remembering what to do the in the

particular airspace configuration they will be operating both for normal operations and for an

anomalous event.

Priority: Low (assuming this question will be addressed in Question 6 above)
195

H.5.4 Questions for Communication and Coordination

1. Is the coordination overly complex?

The Airspace Manager is the primary decision-maker for changing an airspace configuration.

The Airspace Manager and Controllers will coordinate on the change, and the Controller will

give advice and may suggest a change. This should not be a complex coordination, but attention

should be given to the Controller not being unduly distracted from separation assurance and

other functions while coordinating an airspace configuration change. The Airspace Manager

will also coordinate with other Traffic Flow Managers in the TMU and with other Airspace

Managers; this should not be complex.

Priority: Medium

2. What impact do time constraints have on flexibility for coordination?

As long as configuration changes are not so frequent to overly disrupt controllers from their

normal work, which is not likely, this should not be a problem. If airspace reconfigurations were

to involve particularly complex airspace units or changes in separation assurance, then time

constraints might be a problem for controllers when Airspace Managers coordinate with them.

Priority: Medium (Unless there are complex airspace units involved)

3. Who or what has final authority?

This is rather straightforward for DAC. The Airspace Manger has final authority in planning

airspace reconfigurations but with the Controllers being able to express an objection that has

final authority as to agreeing with the change and timing for the change. The procedure for the

Controllers to object needs to be developed.

Priority: Medium

4. How are the trajectory changes negotiated via data link?

This should not an issue for DAC unless there will be numerous trajectory changes as part of an

airspace reconfiguration.

Priority: Low (unless numerous trajectories changes become part of an airspace

reconfiguration)

5. Is communications via data link, voice, or both?

Means of communication for negotiation and for controllers to communicate with pilots for

conflict resolutions during a reconfiguration needs to be decided for DAC, but this should not be

an overly difficult subject for assessment. Once decided, the means of communication are not

likely to add to the complexity of the DAC concept.

Priority: Low

6. Are the underlying technological assumptions for communications and information sharing

adequate?

Technical issues, such as “can information be sent and received with the needed speed,

appropriate encryption, and via the assumed network?” are not likely to be a problem in the

design of DAC. However, the technology for Controllers to communicate conflict resolutions to

pilots during the implementation of a reconfiguration does need attention.

Priority: Low
196

7. Is the information that is shared among multiple operators appropriate regarding content, detail,

and type of format for each particular operator?

It should not be difficult to design appropriate information in terms of its content, detail, and

type of format for communications in the DAC concept.

Priority: Low

8. How important is participating in “party line” communications (i.e., a voice frequency shared

among several persons) as is used in current operations for situation awareness under NextGen?

Simultaneous communications for coordination among the Airspace Manager and multiple

Controllers would likely be useful in coordinating on the appropriateness of a reconfiguration. It

should be relative straightforward to design such a communication capability.

Priority: Low

9. How will visual attention demands of data link communication affect other tasks that require

visual attention?

If the controller is to view graphically possible airspace configurations when coordinating with

the Airspace Manager and while performing separation assurance and other duties, this could be

an issue. The means of presenting an airspace reconfiguration to Controllers will need concept

design attention.

Priority: Medium

10. How are the emergency responses coordinated between human operators?

This does need attention in the design of the DAC concept. If a Controller is faced with multiple

conflict resolutions, off-nominal events, or emergency situations during a configuration change,

s/he may have a difficult time understanding with whom s/he needs to coordinate in order to

handle the situation safely. The decision-making is likely to revert from skill-based to rule-based

in an unfamiliar airspace and the additional time required for rule-based decisions may not be

adequate if the decisions are time-critical.

Priority: High

H.5.5 Questions for Organization, Selection, Job Qualification, and Certification Factors

1. What will be organizational structure, interactions between organizational units, and delegation

of authority to units?

The roles between a national flow control unit and regional Traffic Management Units will need

to be defined. This should not be a difficult issue.

Priority: Medium

2. What skills, abilities, and traits are needed by personnel?

As with all the Research Focus Areas covered in this paper, future human operators will need to

be comfortable using displays and working with DSTs. With airspace configurations changing

dynamically under DAC, future human operators will need to be comfortable working under

different situations during a shift and from one day to the next.
197

Airspace Mangers will need to be able to assess traffic situations in a TMU that are more

complicated than today. An issue is whether Controllers will have the needed skills to move to

an Airspace Manager position in a TMU as they move to TMU positions today.

Priority: High

3. How difficult will it be for human operators to develop an understanding of the capabilities and

limitations of the computer-based models they will employ?

Airspace Managers will be using DSTs different than today; however, however the DAC DSTs

should be straightforward to learn if the personnel have the skills and ability to use DSTs as is

addressed in Question H.5.5.2 immediately above. The computer-based tools for Controllers

will likely not be much different from today; they will be used for more intense periods of

handoffs, accompanied by potential conflict resolutions, during the implementation of a

reconfiguration.

Priority: Medium (for Airspace Mangers; Low for Controllers and pilots)

4. What additional training is needed for the human operators to perform the necessary operations

for the given concept?

Training for Airspace Manager in using the new DAC DSTs will be needed along with how to

use them for “what if” and other possible forms of generating and analyzing alternative airspace

configurations. Training for Controllers will mainly be for the intensive period of handoffs and

possible conflict resolutions during a reconfiguration implementation. Both will need training in

how to respond to emergencies and remember, perhaps using indicators from automation, what

the current airspace configuration is.

Priority: Medium

5. How will the human operators train for the more diverse and dynamic airspace and route

structures that will exist in NextGen operations?

This is an issue for both Controllers and Airspace Managers what with changing airspace

configurations. The Controller will work in different airspace configurations during a shift as

well as from one day to the next. Controllers have not worked in such a situation before.

Priority: High

6. Should human operators train to maintain skills that are only needed when the automation fails?

If DAC DSTs fail, and the Airspace Manager knows they have failed, then airspace may remain

constant as today. Safety will not be impacted, but capacity will be. For the Controller,

procedures, and training in these procedures, will be needed for the case where automated aids

fail during the intensive handoff, and possibly for conflict resolutions, during implementation of

reconfigurations.

Priority: Medium

H.6 Summary of Key Human Performance Issues for DAC

This Section H.6 summarizes our priority recommendations for DAC HF research by discussing the

questions that were rated High or Medium to High in Section H.5. The main drivers from DAC for
198

HF issues are reliance on the automation and DSTs, complexity of situations in which DAC may

operate, and working in changing airspace configurations.

H.6.1 Use of Automation and DSTs

The issues that arise in using automation and DSTs are for trusting and reliance on the

recommendations of the DSTs and dealing with problems and emergency situations. The priorities

of two questions listed below are rated Medium to High.

Reliance in automation and DSTs

Issue: The Airspace Manager will need to trust and rely on the automation’s (i.e., DST’s)

recommendations and parameter information for DAC to provide benefits since there is extensive

information to process to make a decision. A capability for the Airspace Manager to examine

various impacts may be difficult to design if there are several sectors or airspace units involved and

if there are a large number of aircraft and complex flight patterns.

Potential Countermeasure/Next Step: An analysis is needed on how the Airspace Manager

will judge and interpret boundary change impacts of airspace reconfigurations

recommended by the DST. Cognitive walkthroughs and HITL simulation with discussion

with the test subjects should be conducted.

DSTs recognize a problem and help in emergency situations

Issue: Under DAC the automation functions as a DST with human operators having decision

authority. Failures are likely to be recognizable, except if the DST gives erroneous

recommendations, which is a problem in most any DST. A problem may occur for the Airspace

Manager if automation provides erroneous predictions about the future traffic and therefore

recommends inappropriate airspace reconfiguration. Wrong configurations may result in sector

overload but the Controllers and the Area Supervisors should alert the Airspace Manager of such

situations. With the ASA Manager having responsibility for responding in emergencies, there

should be indicators from automation providing emergency alerts to aid the humans since the

situation with changing airspace configurations is more complex than today. It is recommended that

having the automation also make recommendations for resolving the emergency be explored.

Potential Countermeasure/Next Step: In the case of automation being unable to determine a

satisfactory recommendation for airspace reconfiguration, procedures for the Airspace

Manager to make decisions should be designed. For emergency alerts, a systematic analysis

of all cases where alerts and recommendations are needed for the ASA Manager should be

conducted. Ways to provide alerts and recommendations should be formulated and refined

in HITL simulations.

H.6.2 Human Decision-making and Workload

There are issues around human decision-making, mental models and workload that need attention.

The priorities of following questions were rated Medium to High.

Complex mixed equipage airspace

Issue: Mixed equipage flying in airspace units under going reconfiguration complicates the situation

as different aircraft will have different equipage and different performance characteristics. It will be
199

more complex for the Airspace Manger to plan an airspace reconfiguration. It may also be more

complex for Controllers to handle the switching control of aircraft in the short time required.

Potential Countermeasure/Next Step: More knowledge is needed about the concept

regarding mixed equipage to obtaining an understanding of HF issues for this situation.

Once the types of mixed equipage situations and possible procedures to handle them are

known, walkthroughs and HITL simulations should be conducted to understand the limits for

human operators and DSTs to handle mixed equipage and to refine procedures.

Workload impact when transitioning between airspace configurations

Issue: The workload impact for the Controllers could be a problem if there are an excessive number

of conflicts and handoffs for the controller to deal with during an airspace configuration transition.

Potential Countermeasure/Next Step: Situations with a large number of conflicts and

handoffs for the controller to deal with during an airspace configuration transition should be

identified. The workload can be examined through HITL simulations.

Brittleness of concept

Issue: Controllers will need an automation tool to support performing in safety critical events

during a configuration change. Without automation, the type of configuration changes will need to

be limited to exclude more complex reconfigurations, such as splitting and combining sectors. The

degree of automation available will impact the extent of efficiency benefits possible.

Potential Countermeasure/Next Step: The type of airspace reconfigurations to be made

under DAC need to be defined, and then the capabilities of the automation designed. The

automation designed could be tested with HITL simulations to examine the robustness of the

automaton DST for supporting Airspace Mangers in a timely manner in all the situations

they are expected to encounter.

H.6.3 Attention, Situation Awareness, and Memory

The issues that arise in using automation and DSTs are related to demand for attention and situation

awareness and for dealing with problems and emergency situations. The priorities of questions

listed below are rated Medium to High.

Demand for attention and situational awareness

Issue: The potential for high attention demands and time for Controllers to gain situation awareness

should be investigated, and when Controllers will be monitoring for and resolving conflicts as well

as conducting handoffs to place aircraft in new sectors. Controllers handling separation assurance

and handoffs during the implementation of a reconfiguration may need strategies to allocate

attention.

Potential Countermeasure/Next Step: Situations that may drive a high demand on the

Controller as discussed above need to be defined. The ability for Controller to handle these

situations and possible strategies for the controller to handle the demand can be investigated

with HITL simulations.
200

Indicators of emergency and critical safety events

Issue: The need for alerts to indicate emergency or off-nominal events during airspace configuration

changes needs to be studied, particularly for aircraft near the boundary of the new or old airspace

configuration. There may be the need for emergency indicators if not all aircraft trajectories have

been checked for compatibility with the configuration change or if some aircraft is deviating from its

planned path during an airspace configuration change.

Potential Countermeasure/Next Step: Emergency or off-nominal events for which the human

operators under DAC need to aware should be identified. Responsibilities for automation

and for human operators need to be allocated for monitoring and resolving these needs to be

delineated. Then means to alert the human operators can be designed. The potential designs

that will alert operators about and give information to deal with emergences can be

validated and refined using HITL simulations. Those emergencies allocated to automation

should also be studied to see if they can be identified and solved by automation.

H.6.4 Communication and Coordination

The issues that arise for communication and coordination are related to handling emergency

responses under time pressure. The priority of question below is rated High.

Coordination of emergency responses between human operators

Issue: If a Controller is faced with multiple conflict resolutions, off-nominal events, or emergency

situations during a configuration change, s/he may have a difficult time understanding with whom

s/he needs to coordinate in order to handle the situation safely. The decision-making is likely to

revert from skill-based to rule-based in an unfamiliar airspace and the additional time required for

rule-based decisions may not be adequate if the decisions are time-critical.

Potential Countermeasure/Next Step: Situations for the Controller involving multiple

conflict resolutions, off-nominal events, or emergency situations during a configuration

change need to be defined. Then potential ways for the Controller to deal with these

situations, including coordination needs, should be formulated. Controller coordination and

responses in these situations would then be studied and refined in HITL simulations.

H.6.5 Organization, Selection, Job Qualification, and Certification Factors

Different types of tasks and ways of conducting tasks will be part of NextGen. Characteristics of

future human operators and training for the operators drive HF questions of concern. The priorities

of questions listed below are rated High.

Skills, abilities, and traits needed by personnel

Issue: Future human operators will need to be comfortable using displays and working with DSTs

and will need to be comfortable working under different situations during a shift and from one day

to the next. Airspace Mangers will need to be able to assess traffic situations in a TMU that are

more complicated than today. An issue is whether Controllers will have the needed skills to move to

an Airspace Manager position in a TMU position as they move to TMU positions today.

Potential Countermeasure/Next Step: The types of tasks human operators will conduct and

the skills, abilities, and traits needed for these tasks should be identified. Then types of

personnel needed should be defined. Some HITL simulations may help define what is needed
201

and how to identify these needs in humans. Career paths should be examined for possible

career progressions for people with particular characteristics.

Training for the more diverse and dynamic airspace and route structures

Issue: The Controller will work in different airspace configurations during a shift as well as from

one day to the next. Controllers have not worked in such a situation before. Airspace Managers will

also be working with changing airspace configurations, as will Traffic Flow Mangers in the TMU.

How to train for work in changing conditions and how to conduct on-the-job training when

conditions change are issues.

Potential Countermeasure/Next Step: Once the situations under which human operators will

work and the characteristics of these workers are defined, then how to train needs to be

studied. Similar work situations in other industries should be examined to learn how these

organizations conduct training. Training experts will need to be involved. It may help to

conduct prototype training sessions to learn about and refine training methods.
202
203

Appendix I: Integrated DAC-SA

This report focuses on identifying key human performance issues for four individual Research Focus

Areas (RFAs). Concepts from more than one RFA are very likely to operate simultaneously in an

integrated manner in the future NextGen NAS. There will then likely be additional human

performance issues for such integrated operation of RFAs. This Appendix presents a representative

assessment of human performance issues for one integrated pair of RFAs, namely Automation

Separated Airspace (ASA) and Dynamic Airspace Configuration (DAC).

I.1 Overview Description of Integrated ASA and DAC

For this examination of integrated ASA and DAC concepts, a specific instantiation of ASA and

DAC is assumed. One part of our assumed scenario for this integrated analysis is a High Altitude

Airspace (HAA) that is envisioned to service fully data linked aircraft performing Trajectory-Based

Operations (TBO) in a relatively simple traffic environment (e.g., mostly level flight with little local

knowledge needed). Controllers are assumed to provide flight management in the High Altitude

Airspace. In this High Altitude Airspace environment, it may be possible to take advantage of

flexible and dynamic airspace without significantly negative impacts on controller performance.

This High Altitude Airspace is felt to be a likely airspace in which DAC will operate, and thus is

taken as the integrated analysis presented in this chapter.

Above High Altitude Airspace, we assume a ground-based automated separation-based airspace to

handle a high volume of aircraft without active separation management by controllers. For our

analysis, we will use the Automated Separation Assurance (ASA) concept covered in Chapter 4 for

this airspace above High Altitude Airspace.

The simplest form of DAC is probably to change the lateral airspace boundaries in response to

changing traffic demand or weather. In this case, the airspace units being changed are of the same

type, (i.e., sectors reconfigured in the High Altitude Airspace do not intrude into the ASA airspace

above it). A more complex form of DAC, but likely needed to meet a range of traffic and weather

situations, is to change the boundaries between the different classes of airspace above and below

each other. In this case, High Altitude Airspace sectors would be allowed to expand vertically into

ASA airspace. Examples of conditions that might cause such changes in vertical airspace

boundaries include traffic demand and mixes, weather tops, and winds/ride condition.

In summary, for this analysis of human performance issues under integrated concepts, we will

examine sectors in High Altitude Airspace being dynamically expanded into and out of Automation

Separated Airspace above it. Lateral changes in airspace without interacting with a difference

concept were covered in Chapter 7 and are not covered here.

The descriptions of and assumed technology for ASA and DAC were described in Chapters 4 and 7,

respectively, and will not be repeated here.

I.2 Summary of Human Roles and Responsibilities for Integrated ASA and DAC

In Chapters 4 and 7 human roles for ASA and DAC, respectively, were delineated individually for

ASA Manager, Airspace Manager (for DAC), Controller, and pilot. In this, Appendix I, about the

integrated operation of ASA and DAC, human tasks for the human operators will be delineated
204

together for the scenario described in Section I.1. Roles for the individual operators will not be

listed separately since they were listed in Chapters 4 and 7.

The Airspace Manger is assumed to work in a TMU with other Traffic Flow Managers. They are

assumed to work traffic flows and the airspace changes needed in both High Altitude Airspace and

ASA airspace.

It is also assumed that the ASA Airspace has no mixed equipage in that all aircraft in the ASA

airspace are equipped for and are flying under ASA. When High Altitude Airspace is extended into

ASA Airspace, we assume that ASA aircraft transition to fly at a higher altitude and remain in

exclusive ASA airspace. We do assume that during a transition period both ASA-equipped and nonASA-equipped aircraft may fly in the new High Altitude Airspace being expanded into ASA

airspace while ASA-equipped aircraft transition up to higher ASA airspace and non-ASA-equipped

aircraft transition up to the expanded High Altitude Airspace.

During this transition, operations are similar to those described in the Appendix section D.3 when

there is mixed equipage with the ASA Manager attending to ASA-equipped aircraft and High

Altitude Airspace Controller attending to non-ASA equipped aircraft. Besides handling the

unequipped aircraft in the expanded part of the High Altitude Airspace, the High Altitude Airspace

Controller would also have to handle other High Altitude Airspace aircraft that would be flying in

original High Altitude Airspace. Handling unequipped aircraft in both the mixed equipage and nonmixed equipage parts of the new High Altitude Airspace.

Tasks hypothesized for the Airspace Manager (for DAC), ASA Manager, HAA Controller (we will

use the term HAA Controller to remind that the Controller is managing aircraft in the High Altitude

Airspace), and pilot are listed below.

1. At the beginning of the joint ASA/DAC scenario, the ASA Manager and pilot are

performing the tasks in ASA airspace as delineated in Section D.3; the Controller, and

pilot are performing the tasks in High Altitude Airspace as described in Section D.3.2; and

the Airspace Manager is performing the tasks for DAC as delineated in the Appendix

section H.3.

2. The Airspace Manager monitors conditions to judge whether and when they warrant changing

airspace configuration. The Airspace Manager uses Decision Support Tools (DST) to assess

when to change airspace configuration and interacts with others in the TMU on the need to

change airspace configuration.

3. The Airspace Manager decides that the tops of weather cells will reach the top of the High

Altitude Airspace and restrict passage to some airports. Upon examining traffic in the ASA

above the High Altitude Airspace the Airspace Manager concludes that there is excess

capacity in the ASA such that if the High Altitude Airspace sector boundaries were

dynamically changed vertically into the ASA, the High Altitude Airspace traffic can

efficiently maneuver around the weather. The change may also be initiated by the National

Traffic Management Unit.
205

4. Negotiations are conducted between the Airspace Manager, other Traffic Flow Managers,

Front Line Managers, and AOCs about extending the High Altitude Airspace into the

ASA Airspace.

Airspace Manager uses DSTs to assess aircraft trajectories in ASA Airspace that will need

to be changed to provide room for the High Altitude Airspace to be raised vertically. The

Airspace Manager then recommends a way to extend High Altitude Airspace into ASA

Airspace and communicates this change to the ASA Manager. The ASA Manager uses

DSTs to gain situation awareness of aircraft positions and intent in the ASA Airspace and

assesses implications for the proposed airspace change. ASA Manager agrees with or

suggests modification to the Airspace Manager airspace change request. ASA Manager

and Airspace Manager negotiate and agree on an airspace change. The ASA Manager also

negotiates with the AOCs about the impacts of the new trajectories and reconfigured

airspace on airline operations and efficiency considerations.

The Airspace Manager also coordinates with the Front Line Manager of the affected HAA

Controllers to get their agreement to the proposed sectors being reconfigured.

5. Airspace Manager and ASA Manager set up the airspace change for extending High Altitude

Airspace into ASA airspace.

The Airspace Manager coordinates trajectory changes with the ASA Manager to change

trajectories for ASA-equipped aircraft that will change trajectories to vacate the new HAA

sector expansion within a 3 to 20 minute look-ahead time, and the ASA Manager sets

parameters for the ASA automation make these trajectory changes. The Airspace Manager

plans trajectory changes for ASA-aircraft beyond the 20-minute look-ahead time and for

the High Altitude Airspace aircraft. The Airspace Manager coordinates with the HAA

Controllers and ASA Manager for planning these trajectory changes, including for

workload considerations of the HAA Controllers and ASA Manager. Thus, the trajectory

changes for the reconfiguration and the time to begin the reconfiguration are planned.

In lateral boundary changes of sectors in High Altitude Airspace, there may be fewer or

more resulting sectors, as was covered in Chapter 7. A boundary change with an

accompanying change in number of High Altitude Airspace Controllers as well as

extending the airspace into ASA will make for a complex situation. If the number of

controllers changes, the Airspace Manager will work with the supervisor(s) of the

Controllers so a re-assignment of individual Controllers to the new airspaces is developed.

The Airspace Manager or the Controller supervisor(s) notifies Controllers of their new

airspace assignments for the new airspace configuration.

6. The Airspace Manager initiates the change to the reconfigured airspace.

The HAA Controller communicates the maneuvers to pilots. Pilots execute the

maneuvers. The HAA Controller monitors that the maneuvers are executed as planned.

The Controller needs to perform separation assurance, handoffs, and other routine tasks

during this period. It is noted that in Chapter 7 there were few trajectory changes in

reconfiguring airspace and the primary work for controllers in implementing a
206

reconfiguration was to initiate and accept handoffs of aircraft being in new sectors;

however, under the scenario of HAA moving into ASA airspace, there will be trajectory

changes for the HAA Controllers to handle.

The ASA Manager monitors that the ASA automation is correctly maneuvering ASAaircraft into their new trajectories.

The Airspace Manager monitors both HAA and ASA airspaces to see that the aircraft are

moving into their new airspaces as planned. The Airspace Manager also is monitoring the

adequacy of the airspace change to provide the needed capacity, that the reduced ASA

airspace is providing sufficient capacity for ASA-aircraft, and that the weather is evolving

as expected.

7. During the reconfiguration until all ASA-aircraft are moved from the expanded High Altitude

Airspace into the remaining ASA airspace, the ASA Managers and HAA Controllers handle

the mixed equipage aircraft as discussed in Sections D.3.1 and D.3.2.

8. When the weather cell is no longer a capacity limitation to the original High Altitude

Airspace, the process in the above steps is conducted in a modified manner to return the

airspaces to their original configurations.

9. If the weather cell should evolve in a different way than expected, the Airspace Manager may

possible need to make further modifications in the airspace configuration and/or may need to

further re-route traffic.

The ASA Manager and HAA Controller also need to monitor for changing weather that

may mean the reconfiguration airspace cannot be continued to be used as planned.

I.3 Human Factors for Integrated ASA and DAC

I.3.1 Attention Allocation

The use of several types of airspace will result in more complexity for Attention Allocation,

Situation Awareness, and Monitoring for the Airspace Manager(s) in a TMU. Not only is there

monitoring for conditions to change airspace configuration within one type of airspace, such as

lateral sector boundary changes in High Altitude Airspace, but also monitoring for extending one

airspace type into another. The extension of an airspace unit from one type of airspace into another

type, will be more complex than a change within a single type of airspace, so the Airspace Manager

will need more time to plan such an airspace configuration change. Such a change in airspace

configuration involving two types of airspace will not occur often, so the work assignments for the

Airspace Manager in the TMU need to be flexible for the Airspace Manager to leave other duties

and take enough to time to attend to a more complex multiple airspace reconfiguration when needed.

When airspace is to be extended from one airspace into another airspace, there is gaining situation

awareness and allocating attention to a greater variety of aircraft performance characteristics and

airspace configuration factors. There will be trajectory changes for the controllers to handle that

would not be present in airspace reconfigurations entirely with one type of airspace. They need to

be ready to handle a conflict during the transition period, either by a transitioning aircraft or an

aircraft that is not transitioning. The ASA Manager will likely need to monitor the performance of
207

the ASA automation to handle the ASA-aircraft trajectory change since there may be many changes

within a short time period and there may be anomalies that the ASA automation has difficulty

handling in a short timeframe.

When a Controller and ASA Manager come on duty with the reconfigured airspace, either at the

beginning of their shift on a particular day or when assigned to a newly configured airspace

during their workday, they will need to gain awareness of the airspace to which assigned. Some

of this awareness might be provided by DSTs and some by verbal briefing from the Controller or

ASA Manager working the airspace unit immediately before. If there is a change in the number

of Controllers due to a change in the number of High Altitude Airspace sectors during the

airspace reconfiguration, the Controllers brought in to work new sectors will, of course, need to

gain situation awareness.

There are also Attention Allocation, Situation Awareness, and Monitoring issues for the HAA

Controller and ASA Manager to attend to separation assurance, handoffs, and other routine tasks

with mixed equipage present during the transition to the new airspace.

Pilots will need to execute maneuvers into the new airspace, but the impact on each pilot should not

be a concern as long as the mode of their individual separation assurance does not change.

I.3.2 Decision-making and Mental Modeling

Complexity in making a decision for extending one airspace into another is an issue. It might be

helpful for an Airspace Manager to have a written “playbook” of pre-defined airspace configurations

(e.g., based on wind directions, weather, and major en route flows for time of day) and guidelines on

how to adapt a pre-defined airspace configuration to specific conditions. A procedure is also needed

for planning trajectory changes for moving aircraft in both types of airspace to implement the new

airspace configuration.

There are likely to be limits as to the amount and type of criteria that can be processed for this

type of DAC decision-making; on the other hand the criteria should be complete and detailed to

address extending one airspace into another. Another performance issue is how much does a

DST recommend and how much does it allow the Airspace Manager to assess the situation?

This has implications for Airspace Manager confidence in the DST recommendation as well as in

timely performance.

The HAA Controllers and ASA Managers presumably would influence the decision through

negotiations with regard to their suggestions for an alternate reconfiguration plan or request for

alternate timing of the change. They may make input on their workload as it affects the need for an

airspace reconfiguration.

Pilots would not normally be part of DAC decision-making, other than providing off-the-nose

information on weather and wind conditions. The Airspace Manager may have exchanges with

Airline Operations Centers. The pilot must decide whether to accept the maneuver routing to a new

airspace. Care should be taken by the pilot not to reject too many transition maneuvers since the

HAA Controllers and the ASA Managers will have numerous aircraft to deal with in the transition.
208

I.3.3 Communication

When the Airspace Manager sees the need for extending one airspace type into another, s/he will

likely negotiate with the ASA Manager, HAA Controllers, and AOCs about the potential change.

The best means for this communication needs study, such as data link messages, voice

communication, or both. This communication may involve more than one Controller, ASA

Manager, and/or AOC. The communication should be done in plenty of time to effect the

reconfiguration. It is important for a complex change of one airspace type extending into another for

all participants (e.g., Airspace Manager, other TMU workers, Controllers, ASA Managers, pilots,

AOCs) to have a common understanding of the situation and reconfiguration action to be taken.

If the reconfiguration calls for reassignment of HAA Controllers to different sectors, that may require

voice communication between ASA Manager and controller supervisors at the appropriate centers,

with confirmation that the new assignments are clearly understood and will be staffed on time.

There will likely need to be face-to-face communications between an HAA Controller exiting from

serving a particular airspace and the HAA Controller beginning to serve that airspace, if there is a

change in HAA Controllers. This communication will bring the new HAA Controller up-to-date on

conditions in that airspace. This could occur at the beginning of a HAA Controller's shift or when a

new HAA Controller is assigned to an airspace unit as part of a reconfiguration. Such

communication would likely be needed since there could be changes in the airspace of which the

new HAA Controller needs awareness.

Whether the HAA Controller and ASA Manager need to communicate with pilots about impending

airspace configuration changes will depend on whether pilots need information about

reconfiguration or whether the airspaces through which they fly are transparent to them. This issue

should be addressed during the design of the airspaces that will be reconfigured.

I.3.4 Memory

With the complexity of one airspace type being extended into an other, retrospective memory issues

are a concern for feedback on reasons why a particular reconfiguration geometry or schedule would

be inappropriate (communicated verbally by HAA Controllers or ASA Managers) that might be

forgotten by the Airspace Manager during the confusion of reconfiguration implementation.

If the response to an event by a HAA Controller in the sector being worked is sufficiently different

from responses in other sectors previously worked (e.g., will there will be more airspace

configuration types under DAC; will these different types be worked during one shift or in

consecutive days), then there may be memory issues for the HAA Controller remembering the

boundaries of airspace they are working and how to respond to an event, particularly an emergency

or anomalous event. Perhaps reminders provided by DST will be useful.

With the more complex situation of one airspace type extending into another type during an

airspace reconfiguration, whether there are more tasks to conduct in such a configuration change as

opposed to a configuration change where an airspace does not extent into another type of airspace

may be a memory issue. The human operators will need to remember what to do in both types of

reconfigurations, both to handle the change, to handle conflicts that may occur, and to handle offnominal events.
209

ASA Managers dealing with ASA-equipped aircraft have a longer time horizon than the today’s en

route controller for planning trajectories. Thus, there may be a need for them to remember and make

adjustments for situations previously planned before an airspace transition is undertaken.

I.3.5 Workload

Mental workload is anticipated to be a major HF concern in NextGen for all the human operators in

various phases of flight. In Chapter 7 on DAC, it was stated that when a particular airspace

configuration change is a routine change (i.e., according to a playbook) and when there is sufficient

anticipation by all parties, no one human operator’s workload should be excessive. But with

airspace configuration changes involving more than one type of airspace, the situation is likely to be

more complex, and workload may be an issue. Particularly when the HAA Controller and ASA

Manager involved will have to still conduct their duties for situation assurance, hand offs, and other

regular situations. Remember that the HAA Controller needs to deal with conflicts and the ASA

Manager needs to respond to the activation of a TSAFE-type capability. When it is necessary to

deal with conflicts during such a change in airspace configuration, there may be a need for a

workload shedding strategy. An issue for the integrated concept is how to handle pop-up weather,

other than the weather cells driving the airspace configuration change, that may occur during the

planning and implementation period.

I.3.6 Interaction with Automation, Decision Support Tools, and Displays

The Automation, DST, and display issues presented in Chapters 4 and 7 apply to the integration of

ASA and DAC, perhaps with even greater concern. Automation to help the National Traffic Flow

Manager, Airspace Manger, ASA Manager, HAA Controller, and AOC gain the needed situation

awareness and to help with planning for the airspace reconfiguration will be needed. This

automation will need to present clearly and in a timely manner a wide range of conditions, possible

alternative airspace reconfigurations, and reconfiguration transition plans to the human operators.

Automation may help negotiations and coordination among the human operators.

It was mentioned under previous categories of human performance issues that it may be helpful for

DSTs to remind human operators of the type of airspace unit in which they are operating and what

actions are appropriate to particular events that might occur for that airspace unit.

I.3.7 Organizational Factors

A major organizational factor is the authority of the TMU or someone else to dictate an en route

airspace reconfiguration with different human operators involved in different airspace types.

It is likely that National Traffic Flow Management will have authority to overrule a regional TMU.

AOCs may or may not be involved. We foresee that HAA Controllers and ASA Managers will have

an advisory role in both the reconfiguration of an airspace as well as the timing for it to change (i.e.,

which arriving and which departing aircraft will be last for the old and first for the new). All these

actors must have an appropriate mental model of the command and control structure and how it is

meant to work.

If changes in the size of airspace units are part of an airspace reconfiguration, then the organizational

issues discussed in Appendix H.4.7 apply.
210

I.3.8 Potential Errors and Recovery from Errors or System Failures

Similar human errors or system failures to those discussed in Sections D.4.8 and H.4.8 apply to

Chapter I. The range of possible errors and failures must be anticipated and recovery procedures

developed. These should include additional communication to clarify and check for understanding

by all parties involved.

As was motioned under Section I.3.4 (Memory), there is a possibility of erroneous actions by not

remembering the airspace type being worked and by not remembering actions appropriate to that

airspace type if operating actions are sufficiently different from one airspace configuration to

another. In particular, this may be a problem in responding to off-nominal and emergency events.

Perhaps reminders, of both airspace units being worked and appropriate actions and error responses

for that airspace unit, provided by DST will be useful.

I.3.9 Personnel Selection, Certification, and Training

A requirement for “computer smarts” and for being able to work with very dynamically changing

operations are likely to be part of selection criteria for those working in the types of airspace

reconfigurations discussed in this chapter.

I.4 Human Operations Issues, Design Issues, and Countermeasures for

Integrated ASA and DAC

The 46 questions that were addressed for ASA and DAC (in Sections D.5 and H.5., respectively)

apply to the integration of ASA and DAC. The 46 questions will not be discussed here as there

would be considerable redundancy with Sections D.5 and H.5. Instead, we will focus on priority

human factor issues and questions for integrated ASA and DAC.

I.5 Key Human Performance Issues for Integrated ASA and DAC

For this selection of high priority items, we rely on the questions we rated as high priority in

Sections D.6 and H.6.

I.5.1 Use of Automation and DSTs

The issues that arise in using automation and DSTs are about trusting the recommendations of the

DSTs and dealing with problems and emergency situations. The questions we consider high priority

are listed below.

Distribution of roles and responsibilities between humans and automation

Issue: Roles and responsibilities need to be clearly delineated between human operators and

automation for handling the ASA-equipped aircraft during implementation of the reconfiguration to

expand High Altitude Airspace into the ASA airspace. How much of the trajectory changes for

ASA-equipped aircraft are handled by the ASA Manager and how much by the ASA automation?

For the trajectory changes handled by the ASA automation, either the ASA Manager, Traffic

Manager, or some automated transfer of data from the DAC planning DST will need to enter the

trajectory changes for ASA-aircraft into the ASA automation. Roles with HAA Controllers and any

DSTs they might use under Trajectory Based Operations and data link need to be clear as well as
211

roles between Airspace Manager and the HAA Controllers in finalizing new trajectories and

communicating them to HAA aircraft.

Potential Countermeasure/Next Step: A systematic analysis of the steps to handle transition

of ASA-equipped aircraft and HAA aircraft to their new airspaces should be conducted and

promising procedures to handle this should be explored. Further refinement can be

performed through user ratings and observer ratings that can be stimulated through

walkthroughs or simulations, or focus group appraisal.

Trust in automation and DSTs

Issue: The Airspace Manager and the Airspace Manager will need to trust and rely on the DAC

automation’s (i.e., DST’s) recommendations and on the ASA managing of aircraft (as determined

per the issue immediately above), since there is extremely extensive information to process to make

a decision and many aircraft to reroute. A capability for the Airspace Manager to examine various

impacts may be difficult to design if there are several sectors or airspace units involved and if there

are a large number of aircraft and complex flight patterns. The HAA Controller will also need to

have confidence in the recommendations of any DSTs s/he might be using.

Potential Countermeasure/Next Step: An analysis is needed on how the Airspace Manager

will judge and interpret boundary change impacts of airspace reconfigurations

recommended by the DST. An analysis will also be needed the tasks of the ASA Manager

and HAA Controller. Observers in HITL simulations and analysis of observed errors can be

used to evaluate trust.

DSTs recognize a problem and help in emergency situations

Issue: Under DAC the automation functions as DSTs with human operators having decision

authority. Failures are likely to be recognizable, except if the DST gives erroneous

recommendations, which is a problem in most any DST. A problem may occur for the Airspace

Manager if automation provides erroneous predictions about the future traffic and therefore

recommends inappropriate airspace reconfiguration. Wrong configurations may result in sector

overload but the ASA Managers, Controllers, and Area Supervisors should alert the Airspace

Manager of such situations. There should be indicators from automation providing emergency alerts

to aid the human operators since the situation with changing airspace configurations is more

complex than today. It is recommended that having the automation also make recommendations for

resolving the emergency be explored.

Potential Countermeasure/Next Step: In the case of automation being unable to determine a

satisfactory recommendation for airspace reconfiguration, procedures for the Traffic Flow

Manager to make decisions should be designed. For DSTs providing emergency alerts and

response recommendations, a systematic analysis of all cases where alerts and

recommendations are needed for the Airspace Flow Manager, ASA Manager, and HAA

Controller should be conducted. Ways to provide alerts and recommendations should be

formulated and refined in HITL simulations.

I.5.2 Human Decision-making and Workload

There are issues around human decision-making and mental models and workload that need

attention.
212

Complex mixed equipage airspace

Issue: Mixed equipage flying in what will be the new HAA airspace being expanded into ASA

airspace complicates the situation as different aircraft will have different equipage and different

performance characteristics. It will be more complex for the Airspace Manger to plan an airspace

reconfiguration. It may also be more complex for Controllers to handle handoffs associated with the

reconfiguration in the short time required. Any interactions between ASA Managers and Controllers

need to be worked out.

Potential Countermeasure/Next Step: More knowledge is needed about the concept

regarding mixed equipage to obtaining an understanding of HF issues for this situation.

Once the types of mixed equipage situations and possible procedures to handle them are

known, walkthroughs and HITL simulations should be conducted to understand the limits for

human operators to handle mixed equipage and to refine procedures and DSTs to handle

mixed equipage.

Workload impact when transitioning between airspace configurations

Issue: The workload impact for the Airspace Manager, ASA Manager, and Controllers could be a

problem for a possible excessive number of aircraft to reroute and for an excessive number of

conflicts (and handoffs if applicable) to resolve during an airspace configuration transition.

Potential Countermeasure/Next Step: Situations with a large number of aircraft to reroute

and large number of conflicts (and handoffs) to deal with during an airspace configuration

transition should be identified. The workload can be examined through subjective rating

scales, secondary tasks, communication, or physiological indices gathered through HITL

simulations.

Decision-making in time and safety critical situations

Issue: With the complex situation under reconfigurations High Altitude Airspace into ASA airspace,

it is important to design the role of all human operators to have ample time to perform their tasks.

Time is inversely linked to ease of decision-making. The concept design is such that the human

operators may need to face many situations in which they will have a short time to assess the traffic

situation and react.

Potential Countermeasure/Next Step: Since the nature of the concept presents the likely

possibility of human decision-making in a short timeframe under safety critical situations,

any changes or refinement of the concept design should double check if they pose potential

adverse impact on the human operator’s ability to make time critical decisions. The concept

changes should be assessed via walkthroughs and/or HITL simulations.

Brittleness of concept

Issue: All human operators will need an automation tool to support performing in safety critical

events during the configuration change. The degree of automation available will impact the extent

of efficiency benefits possible. How to have someone monitor the human operators’ workload,

stress, etc., in ways that Front Line Managers monitor controllers today is unclear.

Potential Countermeasure/Next Step: The type of airspace reconfigurations to be made need

to be defined and then the capabilities of the automation designed. The automation design

could be tested with cognitive walkthroughs and HITL simulations to examine the robustness
213

of the automaton DST for supporting human operators in a timely manner in all the

situations they are expected to encounter.

I.5.3 Attention, Situation Awareness, and Memory

The issues that arise in using automation and DSTs are related to demand for attention and situation

awareness and for dealing with problems and emergency situations.

Demand for attention and situation awareness

Issue: The potential for high attention demands and time for all human operator positions to gain

situational awareness should be investigated. Under DAC discussed in Chapter 7, Controllers were

assumed not to change aircraft trajectories during an airspace change, only conduct handoffs for

aircraft changing sectors. But in High Altitude Airspace moving into ASA airspace, Controller will

need to maintain situation awareness necessary to change routes of aircraft moving into the new

High Altitude Airspace. Controllers will also be monitoring for and resolving conflicts (and perhaps

conducting handoffs) while rerouting aircraft in expanded High Altitude Airspace sectors, ASA

Managers will be overseeing ASA-equipped aircraft moving from the new High Altitude Airspace

and dealing with ASA duties (See Section 4.4), and the Airspace Manager will be planning a

complex airspace reconfiguration. All operators may need strategies to allocate attention.

There will likely also be situations of low workload for the human operators when there is no

airspace reconfiguration and if they need to become involved in a complex reconfiguration workload

will quickly increase. Design aspects in this area include:

• time for operators to get back in loop if they are passively monitoring and suddenly

there is a need to get back into the loop

• means and time to gain situation awareness under such situations;

• remembering appropriate responses, with perhaps recommendations being provided by

automation, to respond when they need to become involved

• triggers for events that indicate an emergency or anomaly

Potential Countermeasure/Next Step: Situations that may drive high demands on human

operators as discussed above need to be defined as well as situations where there maybe high

demands following a period of low demand. The ability for the operators to handle these

situations and possible strategies to handle the demand can be investigated through cognitive

walkthroughs and HITL simulations – where errors of omission and commission can be

recorded.

Indicators of emergency and critical safety events

Issue: The need for any alerts to indicate emergency or off-nominal events during airspace

configuration change needs to be studied, particularly for aircraft near the boundary of the new or

old airspace configurations. There may be the need for emergency indicators if not all aircraft

trajectories have been checked for compatibility with the configuration change or if some aircraft is

deviating from its planned path during an airspace configuration change.

Potential Countermeasure/Next Step: Emergency or off-nominal events for which the human

operators need to aware should be identified. Responsibilities for automation and for human

operators need to be allocated for monitoring and resolving need to be delineated. The
214

means to alert the human operators can then be designed. The potential designs for human

to be alerted about and to deal with emergences can be validated and refined using cognitive

walkthroughs and HITL simulations. Those emergencies allocated to automation should

also be studied to see if they can be identified and solved by automation.

I.5.4 Communication and Coordination

The issues that arise for communication and coordination are related to handling emergency

responses under time pressure.

Coordination of emergency responses between human operators

Issue: Coordination for responding to emergency situations is complicated in that there is mixed

equipage that may involve coordination among pilots, ASA Managers, and Controller(s). It may

also need to be determined if the coordination needs to involve automation or if the decision can be

made solely among humans. These parties may have confusion as to who needs to coordinate in

different situations. It may also be unclear who would have the final decision authority. All

decisions must occur with clear and efficient coordination, as many decisions are likely to be both

time and safety critical.

Potential Countermeasure/Next Step: A significant effort is needed to understand how best

to coordinate/negotiate among multiple people during time-critical situations. Research in

collaborative decision-making as well as a catalog of observations on how the current ATM

system achieves minimal and efficient coordination in similar situations is useful in

designing a sensible coordination mechanism under ASA and DAC mixed operations. Full

team configuration that includes all humans and automations with potential decision-making

authority need to be accounted for and coordination procedures and supporting DSTs need

to be developed for the possible combinations of people/automation that require

coordination. Once developed, the procedures and tools can be evaluated using

walkthroughs and/or HITL simulations.

I.5.5 Organization, Selection, Job Qualification, and Certification Factors

Different types of tasks and ways of conducting tasks will be part of NextGen. Characteristics of

future human operators and training for the operators drive HF questions of concern.

Skills, abilities, and traits needed by personnel

Issue: Future human operators will need to be comfortable using displays and working with DSTs

and will need to be comfortable working under different situations during a shift and from one day

to the next. Work situations will be more complex with high altitude airspace extending into ASA

airspace. Human operators will need to be comfortable more coordination and more complex

rerouting of aircraft.

Potential Countermeasure/Next Step: The types of tasks human operators will conduct and

the skills, abilities, and traits needed for these tasks should be identified. Types of personnel

needed should then be defined. Some HITL simulations may help define what skills are

needed and how to identify these in humans.
215

Training for the more diverse and dynamic airspace and route structures

Issue: The Controllers and ASA Managers will work in different airspace configurations during a

shift as well as from one day to the next. These operators have not worked in such a situation

before. Airspace Managers will also be working with changing airspace configurations, as will

Traffic Flow Mangers in the TMU. How to train for work in changing conditions, and how to

conduct on-the-job training when conditions change, are issues.

Potential Countermeasure/Next Step: Once the situations under which human operators will

work and the characteristics of these workers are defined, then how to train needs to be

studied. Similar work situations in other industries should be examined to learn how these

organizations conduct training. Training experts will need to be involved. It may help to

conduct prototype training sessions to learn about and refine training methods.
Report Documentation Page Form Approved

OMB No. 0704-0188

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources,

gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this

collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and

Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person

shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YY)

05-01-10

2. REPORT TYPE

Contractor Report

3. DATES COVERED (From – To)

01/21/08–12/31/09

4. TITLE AND SUBTITLE

Identification and Characterization of Key Human Performance

Issues and Research in the Next Generation Air Transportation

System (NextGen)

5a. CONTRACT NUMBER

5b. GRANT NUMBER

NNX08AE66A

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S)

Paul U. Lee, Tom Sheridan, James L. Poage, Lynne Martin,

Kimberly Jobe, and Chris Cabrall

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESSES(ES)

San Jose State University Research Foundation

201 N. Fourth Street, Fourth Floor

San Jose, California 95112

8. PERFORMING ORGANIZATION

REPORT NUMBER

TH-081

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration

Washington, DC 20546-0001

10. SPONSORING/MONITOR’S ACRONYM(S)

NASA

11. SPONSORING/MONITORING

REPORT NUMBER

NASA/CR 2010-216390

12. DISTRIBUTION/AVAILABILITY STATEMENT

Unclassified—Unlimited

Subject Category: 03 Availability: NASA CASI (301) 621-0390 Distribution: Nonstandard

13. SUPPLEMENTARY NOTES

Point of Contact: Paul U. Lee, Ames Research Center, MS 262-6, Moffett Field, CA 94035;

650-604-2011

14. ABSTRACT

This report identifies key human-performance-related issues associated with Next Generation Air Transportation

System (NextGen) research in the NASA NextGen-Airspace Project. Four Research Focus Areas (RFAs) in the

NextGen-Airspace Project – namely Separation Assurance (SA), Airspace Super Density Operations (ASDO), Traffic

Flow Management (TFM), and Dynamic Airspace Configuration (DAC) – were examined closely. In the course of the

research, it was determined that the identified human performance issues needed to be analyzed in the context of

NextGen operations rather than through basic human factors research. The main gaps in human factors research in

NextGen were found in the need for accurate identification of key human-systems related issues within the context of

specific NextGen concepts and better design of the operational requirements for those concepts. By focusing on humansystem related issues for individual concepts, key human performance issues for the four RFAs were identified and

described in this report. In addition, mixed equipage airspace with components of two RFAs were characterized to

illustrate potential human performance issues that arise from the integration of multiple concepts.

15. SUBJECT TERMS

air transportation; human performance; decision-making; workload; training; attention allocation; memory.

16. SECURITY CLASSIFICATION OF: 19a. NAME OF RESPONSIBLE PERSON

STI Help Desk at email: help@sti.nasa.gov

a. REPORT

U

b. ABSTRACT

U

c. THIS PAGE

U

17. LIMITATION

OF

ABSTRACT

UU

18. NUMBER

OF

PAGES

230

19b. TELEPHONE NUMBER (Include area code)

STI Help Desk at: (301) 621-0390

Standard Form 298 (Rev. 8-98)

Prescribed by ANSI Std. Z-39-18