A Controller-in-the-Loop Simulationof Ground-Based AutomatedSeparation Assurance in a NextGen
Environment
Jeffrey Homola, Thomas Prevot, Joey Mercer,
Connie Brasil, Lynne Martin, & Christopher Cabrall
San Jose State University | NASA Ames Research Center

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Acknowledgments
NASA Airspace Program
Federal Aviation Administration (FAA)
NASA Langley Research Center’s Air Traffic
Operations Laboratory (ATOL) team

ENRI International Workshop on ATM/CNSTokyo, Japan: November 10-12, 2010

The Problem•NextGen is expected toaccommodate a threefold(3X) increase in air trafficdemand compared to today’slevels•Cognitive resources of air
traffic controllers are limited•Conventional clearancebased separation assurance(SA) is not possible in theenvisioned high density
environment

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

The Approach•“ground-based
automated separation
assurance”•the ground-based
automation manages theseparation•the operators manage theautomation, provide
additional services andmake decisions

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Airspace Operations Laboratory
Beginning in 2007, a series of HITL simulations on groundbased automated SA have been conducted in the AOL

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

SA1 (2007)•Tested SA at threeprogressive levels of traffic
density:1X, 2X, 3X•Varied levels of automatedSA support across trafficlevels: Manual, Interactive,
Fully Automated

J. Homola, “Analysis of Human and Automated Conflict Resolution Capabilities at Varying Levels of Traffic Density”. (Master’s Thesis).
San Jose State University, San Jose, California, 2008.
T. Prevot, J. Homola, and J. Mercer, “Human-in-the-Loop Evaluation of Ground-Based Automated Separation Assurance for NextGen”.
ICAS 2008-11.4.5, and AIAA-ATIO-2008-8885, Anchorage, Alaska, 2008.

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

•Automation provided
significant benefits in termsof safety and efficiency
particularly at 2X and 3X.
Significant reduction inworkload. Resolutions
provided by automation
generally acceptable.

SA2 (2008)•Tested ground-based automatedSA at 2X and 3X with tacticalconflict and off-nominal
situations•Varied levels of TSAFE support
across traffic levels•Automation handled strategic
conflicts. Participants handled
conflicts deferred by automation,
tactical conflicts, pilot requests,and emergencies.

T. Prevot, J. Homola, J. Mercer, M. Mainini, and C. Cabrall, “Initial Evaluation of NextGen Air/Ground Operations with Ground-Based
Automated Separation Assurance”. ATM2009, Napa, California, 2009.
J. Homola, T. Prevot, J. Mercer, M. Mainini, and C. Cabrall, “Human/Automation Response Strategies in Tactical Conflict Situations”.
DASC 2009, Orlando, Florida, 2009.

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010•98% of strategic and 75% oftactical conflicts resolved byautomation, 95% of resolutions
acceptable to flight crew
participants, workload generallylow.

SA3 (2010) Background• NASA’s FY2010 ARMD Annual Performance
Goal: “Conduct simulations of automatedseparation assurance with sequencing,spacing, and scheduling constraints.”• JPDO concerns regarding the “lack of clarity”
surrounding the functional allocation of new
functions and responsibilities between theground-based ATC and flight deck-based
systems.
ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

SA3 (2010)
Two separate but collaborative studies on automated SA conductedfrom both the air- and ground-side perspectives in the ATOL and AOL

Poster created at NASA Langley for NASA Langley Center Team Award Ceremony on June 3rd 2010
D. Wing, T. Prevot, J. Murdoch, et al. “Comparison of Airborne and Ground-Based Functional Allocation Concepts for NextGen Using Human-In-The-Loop Simulations”. AIAA, 10th ATIO, 2010.

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Overall Experiment Design
Short duration runs

Medium duration runs

Exploratory Long duration runs

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Experiment Design
Short duration runs

Medium duration runs

Exploratory long duration runs

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Participants• Six FAA front line managers staffed radar and area
supervisor test positions. They were fromdifferent en route centers and current on radar• Four recently retired confederate controllers
staffed remaining radar test sector positions• Four retired confederate “ghost” controllers
controlled traffic outside of test area
• Ten general aviation pilots served as pseudopilots
for aircraft in the test scenarios
ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Airspace•4 en route test sectors
from ZKC and ZID centers•Surrounding airspacecontrolled by confederates•Flight Level 290 and above•Mixture of overflight andtransitioning aircraft to andfrom area airports•All aircraft fully datacomm and ADS-B equipped
ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Air Traffic
NextGen Level B

60504030
20100051015202530

Average Number ofAircraft in Sector

Average Number ofAircraft in Sector

NextGen Level A
6050403020100051015Time (min)ZID 80Traffic level

ZID 81ZKC 90

Forecast foryear

2025Time (min)ZKC 98

ZID 80

ZID 81ZKC 90ZKC 98

Test airspace capacity

Aircraft in ZKC-90

Number of % of 2010
aircraft value

Number % of 2010
mean/peak value

2010

108 100%

15 / 18 100%

NextGen A

2025

162 150%

35 / 42 233%

NextGen B

2030+

216 200%

46 / 54 300%

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 201030

SA Functional Allocation
Automation

Controller

Detect Separation Conflicts
Resolve trajectory-based conflicts (if withintolerances)
Resolve all time-critical traffic conflicts
Alert controller to urgent problems

Supervise the automation
Resolve trajectory conflicts flagged by theautomation
Monitor and maintain schedule compliance
Place aircraft back on trajectory following automatedtactical maneuvers

Provide trajectory planning assistanceUse data comm to communicate

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Apparatus•MACS simulation platform•Advanced controller displays•71 cm Barco displays•DSR keyboards and trackballs•Voice Switching and Comm.
System (VSCS) emulation•Wall projections of current
and predicted traffic situations
ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Procedure• Two week study— Three days training— Five days data collection• 30-minute runs
• Participants divided into two teams• Runs conducted simultaneously in twoparallel “worlds”• ZKC and ZID sectors divided within eachworld to ensure inter-facility coordination• FAA test participants rotated throughsupervisor and radar positions fordifferent perspectives
ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Results— Airspace and traffic—Workload—Conflict detections and resolutions—Losses of separation—Subjective participant feedback

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Airspace and Traffic
Average Number ofAircraft in Sector

NextGen Level A
6050403020100051015202530Time (min)ZID 80

ZID 81ZKC 90ZKC 98Average Number ofAircraft in Sector

NextGen Level B
60504030201000510152025Time (min)ZID 80

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

ZID 81ZKC 90ZKC 9830

Workload
Workload by condition

Workload by sector• Workload significantly higher at Traffic Level B than Level A• Scheduling constraints did not have an effect on workload• ZKC90 had significantly higher workload than ZID80 and ZID81 but not ZKC98
ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Conflicts: Detections• Traffic Level B had significantly more conflicts predicted to lose separation in thetest airspace than Level A• STA scheduling condition had significantly fewer conflicts than the Baseline
condition without scheduling constraints
ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Conflicts: Resolutions
Resolutions by condition

Resolutions by sector• Majority of conflicts resolved by automation• ATC involved resolutions increased with traffic levels• TSAFE events increased with traffic levels• ZKC98 required the greatest number of conflict resolutions issued• ZKC98 required greater ATC involved resolutions and TSAFE clearances
ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Losses of Separation
Events by condition

Events by sector• Traffic Level B resulted in greater numbers of separation events• Baseline and STA conditions resulted in equal numbers of operational errors• Baseline had overall greater number of separation events• ZKC98 had the greatest number of separation events followed by ZKC90

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Subjective Feedback:
Level A

Level B

Mean Rating on Scale

76
5
432

1
DemandSupplyUnderstanding

SART scale

Situation Awareness Rating Scale

SART

Situation
Awareness• Spread of responses both
between traffic levels andsectors• “A little” attention demand• “Average” supply ofattention• “Very good understanding”
of situation• “Reasonable situation
awareness”

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Subjective Feedback: Acceptability• Questions on acceptability of operationsaligned with Controller Acceptance Rating
Scale (CARS)• Acceptability of safety in Traffic Level A rated
at 90.6% and Traffic Level B at 67.5%• Volume of traffic not a concern but thegreater complexity of the traffic and fewer
resolution options wereENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Subjective Feedback: Impressions• “…it seemed as if controller and automation
fought against each other at times to resolve
conflicts.”• “it seems fairly natural, why not do it?”• “You’re on the right track.”• “It’s inevitable, I think the concept is strong,
it needs work and testing, I think it’s the way
we’re going to go.”ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Summary• Increase from Traffic Level A to B provided themost noteworthy results• Mean workload, conflicts detected, and losses ofseparation counts were all higher in Level B• “Reasonable situation awareness” wasmaintained at both traffic levels but Level B wasrated as less safe (CARS) and more attentiondemanding (SART) than A• At the sector level, local complexity more of anissue than simple aircraft count (e.g., ZKC98)
ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Conclusion• The functional allocation of separation
assurance between controller and groundbased automation presented was wellreceived and held promising results• An important component to being able toaccommodate the envisioned future demandis the appropriate identification and handlingof local complexities
ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010

Questions?

ENRI International Workshop on ATM/CNSTokyo, Japan November 10-12, 2010