National Aeronautics and Space Administration

www.nasa.gov

AOSP Newsletter

Airspace Operations and Safety Program (AOSP)

APR-JUN 2021 | Quarter 3

NASA ATD-2 Phase 3 Stormy 

2021 Commences Formal Data 

Collection

3

ATD-2 Reaches More Than One 

Million Gallons of Fuel Saved at 

Charlotte Intl.

11
2 AOSP NEWSLETTER // APR-JUN 2021

AOSP IN THE NEWS

NASA Articulates Vision 

for Regional Air Mobility

Aviation Week (4/23) reports: “to 

succeed, urban air mobility (UAM) 

requires multiple technological 

and operational transformations to 

come together. But through targeted 

technology investments, regional 

air mobility can offer early benefits 

for communities, industry and 

investors, a new NASA white paper 

has concluded.”

NASA Envisions Revived 

U.S. Regional Air 

Transportation Network

FlightGlobal (4/26) reports: “the 

USA’s many small, underused 

airports, combined with autonomous 

and electric-propulsion technologies, 

could enable a vast new US regional 

air transportation system. That is 

according to new NASA-published 

report that lays out a vision called 

‘Regional Air Mobility’ – the idea 

that technology can make air travel 

between small US airports viable.”

‘It’s Not Quite Like 

Riding A Bike’: Pilots 

Prepare to Fly Again

New York Times (5/07) reports: 

“one returning pilot lost control 

of an aircraft during landing and 

skidded off the runway into a 

ditch. Another just returning from 

furlough forgot to activate a critical 

anti-icing system designed to prevent 

hazards in cold weather. Several 

others flew at the wrong altitudes, 

which they attributed to distractions 

and lapses in communication. In 

all of these incidents, which were 

recorded on NASA’s Aviation Safety 

Reporting System, a database of 

commercial aviation mistakes that 

are anonymously reported by pilots 

and other airline crew, the pilots 

involved blamed the same thing for 

their mistakes: a lack of practice 

flying during the pandemic.”

Springfield Will Be Part 

of NASA’s ‘Advanced Air 

Mobility’ Research

Dayton (OH) Daily News (5/17, 

Gnau) reports: “NASA has chosen 

a Springfield-based flight research 

center to help shape the future of 

passenger travel and cargo delivery, 

only the strengthening the site’s 

continue role in such research. 

FlyOhio, based at the Ohio/Indiana 

UAS Center and Test Complex 

in Springfield, received the award, 

meant to push to reality ‘multiple 

use cases for personal travel and 

delivery of goods across the state of 

Ohio,’ the FlyOhio organization said 

in a release.”

NASA Readies For eVTOL Test 

Flights with Joby Prototype

Aerospace Daily (5/18) reports: 

“NASA has agreed to work with five 

U.S. state and local governments on 

how to integrate passenger and cargo 

advanced air mobility into their 

transportation plans.”

Drones Flying Near 

Airports, Infrastructure 

Prompt U.S. Action

Wall Street Journal (5/20) reports: 

“the Federal Aviation Administration 

and the National Aeronautics and 

Space Administration are developing 

a joint national air-traffic-control 

system for low-flying drones.”

Watch NASA Prepare for 

Our Flying-Car Future

Mashable (5/26) reports: “instead 

of taking a bus or train to work, 

what about a flying car? A Nova TV 

special, The Great Electric Airplane 

Race, airing Wednesday night on 

PBS, explores the possibility of 

using autonomous, battery-powered 

flight for commuting or taxi trips. 

As part of his research into electric 

flight, reporter Miles O’Brien visited 

NASA’s Ames Research Center and 

hopped into its 10-story vertical 

motion simulator to see what it 

would be like to fly around San 

Francisco in one of these vehicles.”
AOSP NEWSLETTER // APR-JUN 2021 3

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

NASA ATD-2 Phase 3 

Stormy 2021 Commences 

Formal Data Collection 

POC: GREG JURO AND ERIC CHEVALLEY

NASA’s Airspace Technology 

Demonstration 2 activity entered 

the Phase 3 formal data collection 

period entitled “Stormy 21” on 

April 1. The data collection is 

scheduled to last four months – until 

the end of July. Stormy 21 includes 

real-time operational participation 

by seven field facilities including 

American, Envoy, and Southwest 

airlines, Dallas/Fort Worth Air 

Traffic Control (ATC) Tower, Dallas 

Love Field ATC Tower, Dallas/Fort 

Worth Terminal Radar Approach 

Control, and Fort Worth Center. 

The spring and early summer seasons 

were selected for the formal testing 

period due to frequent convective 

weather resulting in surface delay 

and reroute opportunities at the 

North Texas airports. Stormy 21 

data collection culminates months of 

shadow testing, extensive field user 

training, and thorough assessment 

of actual Trajectory Options Set 

(TOS) reroute submissions by flight 

operators and reroute approvals 

by ATC. Biweekly reviews of TOS 

activities and system use with the 

field demonstration partners during 

the past six months led to software 

enhancements that were deployed 

to the operational facilities on 

April 5. The latest software version 

(6.0) includes agile solutions 

developed in collaboration with 

field users to mitigate the impact of 

ATC staffing constraints caused by 

COVID-19. Online tutorials and 

ongoing remote training sessions 

for field users helped identify 

operational needs and prepare 

the users to act upon reroute 

opportunities. Stormy 21 provides 

the opportunity to evaluate the 

system in various weather and traffic 

Example of an ATD-2 Phase 3 TOS Reroute
4 AOSP NEWSLETTER // APR-JUN 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

conditions, and learn how and when 

the system is utilized and what 

benefits could be obtained by using 

TOS to reroute delayed flights – 

both from an individual flight and 

from a system-wide perspective. 

Lessons learned will be gleaned 

from data analyses as well as from 

field users’ feedback. These lessons 

will provide insights into future 

systems requirements for NASA, 

the FAA, and the airline industry. 

ATM-X Discussion with FAA 

Technical Center to Identify 

Collaboration Opportunities 

POC: KURT SWIERINGA

On Jan. 14, the Airspace 

Technology Demonstration 2 

(ATD-2) team launched the daily 

dissemination of Trajectory Option 

Set (TOS) activity reports for the 

FAA and field demo partner airlines, 

including American Airlines 

and Envoy Air at Dallas Fort 

Worth International Airport and 

Southwest Airlines at Dallas Love 

Field Airport. The content for the 

reports has been in development 

for several months, incorporating 

feedback from field demonstration 

partners through biweekly meetings, 

tabletop exercises, and reviews 

of initial prototypes. The reports 

contain airline-specific details of 

the previous day’s TOS activity 

for impacted flights based on data 

computed by the ATD-2 system on 

departure flights in Dallas’s airspace. 

Additionally, the reports summarize 

cost-saving alternate routes 

suggested by ATD-2 to meet the 

demand-capacity imbalance in the 

D10 TRACON boundary arising 

from restrictions such as departure 

fix closures and Miles-in-Trail. The 

reports also provide estimated 

benefits at carrier, fleet, airport, and 

system-wide levels associated with 

each TOS route. The field demo 

partners are eager to use the TOS 

Activity Reports to further their 

understanding of TOS benefits 

and continue to offer feedback 

to improve the ATD-2 system. 

Digital Information 

Platform Meeting with 

American Airlines 

POC: MIRNA JOHNSON

The Air Traffic Management – 

eXploration (ATM-X) project’s 

Digital Information Platform (DIP) 

team is reaching out to interested 

parties to establish relationships that 

will help shape the development of 

the system concept, and eventually 

to a successful technology adoption. 

On April 5, the DIP team met with 

American Airlines representatives, 

including from Data Science Analytics, Operations Engineering, and 

Flight Department, to hold a freeform discussion to better understand 

stakeholder needs and expectations 

for a data-driven platform. This was 

a similar format to the meeting held 

with Delta Airlines on March 25. 

After attending System Wide Information Management (SWIM) 

Industry FAA Team (SWIFT) #13, 

where the DIP team first presented 

the concept, the team followed up 

for feedback on the concept and 

captured the measurable benefits it 

could bring. This is expected to be 

one of many sessions to dive deeper 

into the data and service needs of the 

stakeholder community. 

ATM-X Attends Kickoff 

Meeting for ASTM 

Working Group 

POC: KURT SWIERINGA

Becky Hooey, program director of 

NASA’s Aviation Safety Reporting 

Systems (ASRS), participated in the 

Unmanned Aircraft Safety Team 

(UAST) plenary meeting on Oct. 

15. The UAST is a government/

industry group chartered in 2016 

to develop consensus-based and 

data-driven safety enhancements for 

unmanned aircraft systems (UAS) 

and operations. The team consists 

of more than 75 leaders representing 

organizations that span the UAS 

community from the FAA, NASA, 

and industry. Hooey described 

her team’s work to enhance 

ASRS to enable better reporting 

of UAS incidents, including the 

development of a UAS-specific 

report form and augmentation of 

the existing ASRS taxonomy. The 

UAS-specific reporting capability is 

scheduled to launch in early 2021 
AOSP NEWSLETTER // APR-JUN 2021 

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

and is expected to yield a rich data 

source that can be used by the FAA, 

NASA, industry, and academia to 

identify safety-relevant trends and 

incident precursors. This effort 

is extending an existing safety 

capability to include UAS and is 

represented jointly by Hooey and 

Steve Young, who is also a member 

of the UAST. 

ATM-X, UAM, and NC-DT 

Complete Final Report 

on X3 Simulations

POC: SAVVY VERMA AND JOHN SPRAGUE

Between May and December of 

2020, the Air Traffic Management 

– eXploration (ATM-X) project’s 

Urban Air Mobility (UAM) 

subproject designed and conducted 

X3 simulations with the National 

Campaign’s Developmental Test 

(NC-DT) airspace partners in 

collaboration with the NC’s Airspace 

Test Infrastructure team, who 

provided data collection capabilities. 

Out of the 11 NC-DT airspace 

partners, nine participated in at least 

one data collection activity. The data 

analyses for the X3 simulations were 

completed and individual data was 

provided to participants (industry 

partners) in February 2021. On 

April 7, a final report co-authored 

by the UAM and NC subprojects, 

describing the tests, scenarios, 

airspace, and results, along with 

insights gained regarding UAM 

operations, was completed as a NASA 

Technical Memorandum. This was 

distributed to all the partners who 

participated in the X3 simulations, 

marking the completion of the 

milestone outlined in the agreement 

with the industry partners.

V&V Vision 2045 NASA 

Research Announcement 

Awardee Selection 

Advances Assurance of 

Autonomy Development 

POC: MISTY DAVIES

AOSP’s System-Wide Safety project 

and the Transformative Aeronautics 

Concepts Program put out a joint 

call for external stakeholders to 

define a research roadmap for 

the verification and validation 

(V&V) and assurance of autonomy 

through the year 2045. The V&V 

Vision 2045 NASA Research 

Announcement solicited a number 

of excellent submissions, and NASA 

has selected an awardee. Boeing 

Research and Technology, together 

with General Electric Aviation, 

Collins Aerospace, the University of 

Michigan, Massachusetts Institute 

of Technology, and the University of 

Texas at Austin, will collaborate over 

a one-year period of performance. 

This roadmap will consider both 

near-term needs together with 

industry and standards consensus 

bodies, as well as long-term needs 

for achieving the advanced Urban 

Air Mobility maturity levels. 

Additionally, the roadmap will 

be used to guide NASA, as well 

as other national stakeholders, 

in strategic research investment 

decisions. The kickoff for the 

NASA Research Announcement is 

anticipated to be in May 2021. 

SWS Demonstrates Extension 

of the Dashboard Using 

Relevant Case Study (Taxinet) 

POC: GUILLAUME BRAT

Researchers from the System-Wide 

Safety project have completed work 

demonstrating the extension of the 

dashboard using a relevant case study 

(Taxinet – an autonomy testbed 

platform supplied by Boeing) by 

performing a live demonstration 

to project management and other 

subproject managers. The Safety 

Management System (SMS) 

concept is a proactive approach to 

system safety, intended to provide 

a structured means of safety-related 

decision making. The goal is to 

identify and manage safety risks 

well before system failures and 

loss events occur, while increasing 

confidence that risk controls in place 

are performing as intended. SMS 

is central to the FAA’s approach to 

achieving acceptable levels of safety 

risk for both aircraft systems and 

airspace/air traffic management 

systems. Two central pillars of the 

FAA’s SMS approach are Safety Risk 

Management and Safety Assurance. 

Safety Risk Management is meant to 

identify and assess risk and develop 

5
6 AOSP NEWSLETTER // APR-JUN 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

risk mitigation controls; Safety 

Assurance involves monitoring and 

tracking the performance of the risk 

mitigation controls defined during 

Safety Research Management. The 

team demonstrated a preliminary 

integration of risk-based design, 

development, and assurance in 

the Assurance Case Automation 

Toolset. Specifically, the team is 

allowing trade trees that record 

decision alternatives such as design 

decisions to be presented – outlining 

the impact of each alternative on 

baseline risk levels and existing 

assurance artifacts, decisions about 

those alternatives to be made and 

recorded, and justifications of those 

decisions to be provided as assurance 

arguments. Thus, integrating 

decision-making, assurance impact 

analysis, and safety assurance case 

development within a common 

framework is achieved. The team 

also illustrated the approach on the 

Taxinet system and demonstrated 

that upon a risk materializing, they 

can enumerate design alternatives 

showing how risk-informed decision 

making is used to select a feasible 

design alternative while also updating 

the associated assurance case.

SWS Demonstrates Expansion 

& Application of Uncertainty 

Representation Framework 

POC: STEVE YOUNG AND WENDY OKOLO

Researchers from the SystemWide Safety (SWS) project 

have completed work to expand 

the uncertainty management 

framework described in NASA 

TM-220440 (January 2020) which 

was associated with an SWS annual 

performance indicator. Evaluation 

of the expanded framework was 

completed for safety-related 

services, functions, and capabilities 

(SFCs) under development within 

the project. This framework 

for uncertainty representation, 

handling, and management was 

designed to accommodate different 

sources of uncertainty (associated 

with inputs, models, measurements, 

and computing methods) in the 

computation of the SFC outputs 

and metrics. By identifying the 

sources of uncertainty and how 

they propagate or manifest during 

flight operations, safety assurance 

assumptions can be validated and 

appropriate margins can be defined. 

The report also gives an overview 

of the groundwork laid to compute 

SFC outputs/metrics for which 

the uncertainty representation 

framework was tested and evaluated. 

Work on expanding and applying 

the framework was conducted at 

both NASA’s Ames Research Center 

in California and NASA’s Langley 

Research Center in Virginia. At 

NASA Ames, the focus was on 

wind uncertainty, positioning 

system errors, and uncertainties 

associated with unmanned aerial 

vehicle electric powertrains. At 

NASA Langley, the focus was on 

uncertainties associated with nonparticipant casualty risk assessment. 

Space Act Agreement 

Annex Executed Between 

SWS Project and Boeing 

POC: MISTY DAVIES AND LANCE PRINZEL

On April 19, a Space Act 

Agreement annex was executed 

between NASA’s System-Wide 

Safety (SWS) project and Boeing 

for the “integration, demonstration, 

and evaluation of autonomous 

systems and tools.” The agreement 

signed is Annex 16 under the 

Space Act Agreement umbrella 

SAA1-23831. The annex will enable 

the application and integration 

of formal methods based analysis, 

verification, testing, and monitoring 

tools in the design-time and 

operation-time assurance processes 

of autonomous systems. The 

collaborative effort has the objective 

to demonstrate the capability 

of these tools to determine the 

quality and safety improvement 

in the development life cycle 

of autonomous systems. The 

agreement supports development 

of solutions and helps inform 

NASA participation in artificial 

intelligence (AI) standardization 

(e.g. SAE G-34/EUROCAE 

WG-114) in aviation standards, to 

identify safety challenges and gaps, 

and to establish standards in the 

use of tools developed with this 

collaboration. The partnership with 

Boeing continues the exploration 

of autonomy and AI requirements, 

assurance approaches, evaluation 

methods, and means of compliance 
AOSP NEWSLETTER // APR-JUN 2021 

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

that support AI standardization and 

explore paths to certification. 

SWS Participates in 2021 

Boeing IRAD Review 

POC: NATASHA NEOGI

The Boeing Independent Research 

and Development (IRAD) review 

took place for three hours per week 

between March 9 and April 22. 

Many subject matter experts (SMEs) 

external to Boeing were invited to 

review the work of cutting-edge 

Boeing researchers across many 

technical topics. Researcher Natasha 

Neogi of NASA’s Langley Research 

Center in Virginia was invited to 

participate as an SME reviewer for 

the autonomous systems subtopic, 

and there was considerable discussion related to work being done on 

verification, validation, and certification in the context of unmanned 

aerial vehicles that were directly 

related to NASA’s System-Wide 

Safety project interests. Neogi also 

served as a reviewer for research on 

guidance, navigation, and control 

as well as work on precision, navigation, and timing. She was able 

to exchange opinions with other 

expert reviewers from government, 

including the Air Force Research 

Laboratory, Department of Energy, 

and academia including MIT and 

Georgia Tech, and helped synthesize 

broad consensus reviewer-response 

documentation. This was the first 

ever virtual instantiation of the 

event held online via WebEx. 

SWS HC2S Research 

Presented to Boeing 

Training and Flight Deck 

Technical Groups 

POC: CHAD STEPHENS

On April 22 and 28, Chad Stephens 

of the System-Wide Safety (SWS) 

project presented invited talks to 

more than 100 attendees on joint 

NASA-Boeing research in Human 

Contributions to Safety (HC2S) 

to the Boeing Global Services 

(BGS) Standards and Training 

group and the Boeing Commercial 

Airplanes (BCA) Future Flight 

Deck technical group, and on a 

recent human-in-the-loop flight 

simulation study conducted 

in collaboration with Boeing 

researchers under an existing nonreimbursable Space Act Agreement 

at the Boeing Commercial Flight 

Training Campus in Miami. The 

focus of the research study was 

collecting and analyzing data on 

how commercial aviation flight 

crews handle routine disturbances, 

both expected and unexpected, in 

commercial flight operations to 

explore HC2S. The presentations 

described the collaborative study 

involving three commercial aviation 

flight crews performing four area 

navigation standard terminal arrival 

route (RNAV STAR) arrivals into 

Charlotte Douglas International 

Airport in the Boeing 737 NextGen 

full flight simulator. The objective 

of the study was to understand 

the factors involved in productive 

safety demonstrated by flight 

crews using safe and necessary 

non-adherence to RNAV STAR 

arrivals as the operational example. 

The BGS Standards and Training 

group’s interests in the SWS HC2S 

research are focused on identifying 

and enabling resilient performance 

as evidenced and supported 

by competency-based training 

methods. The BCA Future Flight 

Deck technical group’s interests 

in the SWS HC2S research are 

focused on critical considerations 

for designers and developers of 

advanced commercial transport 

systems. The recent human-inthe-loop simulation study was 

conducted in collaboration with 

researchers from Boeing’s Research 

and Technology group (Beth Biddle 

and Jack Dwyer) and from Boeing’s 

Standards and Training group (Jim 

Wilkerson). Follow-up actions from 

the meetings included discussing 

collaborations as future internal 

projects are planned, as well as 

providing publications describing 

SWS HC2S research.

ATM-X/UAM Subproject 

Presentation at MIT Lincoln 

Laboratory ATC Workshop 

POC: IAN LEVITT

On April 27 and 28, Ian Levitt, 

principal engineer for the Air 

Traffic Management – eXploration 

(ATM-X) Urban Air Mobility 

(UAM) subproject, presented at 

the MIT Lincoln Laboratory Air 

7
8 AOSP NEWSLETTER // APR-JUN 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

Traffic Control Workshop. Levitt 

kicked off the session on Advanced 

Air Mobility (AAM) and UAM 

with a briefing covering the UAM 

airspace concept and the subproject’s 

plans for researching UAM’s 

airspace architecture with the X4 

simulation. Joby Aviation followed, 

providing an industry perspective 

on UAM, and MIT Lincoln 

Laboratory presented on operational 

simulation capabilities that they 

are developing. The panelists then 

transitioned to a lively Q&A session, 

touching on topics such as weather, 

demand modeling, noise impacts, 

applications of artificial intelligence 

and machine learning, and regulatory 

paths to operationalization. The 

exchange was a good opportunity 

to showcase the breadth of NASA’s 

research portfolio in the area of AAM 

and their approach to collaboration 

with industry. 

SWS Presents to the 

Autonomous Defense 

Systems T&E Working Group 

POC: GUILLAUME BRAT

System-Wide Safety (SWS) 

researcher Guillaume Brat presented 

the research being conducted by 

the Robust Software Engineering 

(RSE) group on software and 

system assurance for autonomy to 

the Autonomous Defense Systems 

(ADS) Test & Evaluation (T&E) 

working group on April 28. For the 

past 10 years, Brat and his group 

have been developing tools and 

techniques for software assurance 

in aviation systems under the SWS 

project. The tools cover all phases 

of the development process, from 

requirements to coding and then 

testing. RSE researchers have 

worked closely with industry to 

develop open source tools that 

can make a difference on real 

systems. The research first targeted 

traditional aviation systems such 

as transport aircraft. For the past 

two years, however, the focus has 

shifted to increasingly autonomous 

aviation systems that will be used 

for the Advanced Air Mobility 

market (e.g. autonomous package 

delivery or autonomous air taxis) 

and for disaster response (e.g. 

unmanned aerial vehicles for 

wildfire management). As part of 

his presentation, Brat presented 

the strategy adopted to build 

these tools and what implications 

it has on a potentially new 

certification process for machinelearning enabled systems. The 

ADS T&E working group aims 

at developing a guidebook for the 

T&E of autonomous systems. The 

guidebook is being developed by 

the Scientific Test and Analysis 

Techniques Center of Excellence 

as guidance for test planners and 

test engineers for ADS. The group 

consists of representatives of the 

various armed forces including 

the Coast Guard and has drawn 

interest from the Earth science 

community, especially for disaster 

response and wildfire management. 

The systems being targeted rely on 

artificial intelligence, in particular, 

machine learning. Brat was asked 

by the working group leadership 

to present an overview of the tools 

(FRET–requirement elicitation 

and analysis, CoCoSim–verifying 

Simulink models, IKOS–static code 

analysis, AdaStress–adaptive stress 

testing, MARGInS–identifying safe 

operational boundaries through 

statistical testing, R2U2–runtime 

monitoring, and Prophecy–

analyzing neural networks) and 

what impact they will have on 

certification of increasingly 

autonomous systems. 

SWS Delivers Capability that 

Extends NASA’s Precursor 

Analysis Scope 

POC: NIKUNJ OZA

The System-Wide Safety (SWS) 

project has recently completed 

work delivering an initial capability 

that extends NASA’s previous 

precursor analysis to risk over 

multiple adverse events. SWS’s 

Technical Challenge 1: Integrated 

Terminal Area Risk (milestone 

TC1-02-03) recently completed 

work developing a new algorithm 

that accepts, as input, a dataset of 

flights that have various adverse 

events within them and identifies 

precursors to these adverse events, 

including the features that describe 

these precursors and the times at 

which they occur. This work was 

completed on May 1. 
AOSP NEWSLETTER // APR-JUN 2021 9

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

NASA Aeronautics 

Cybersecurity Workshop 

POC: PAUL NELSON

On May 4, an ARMD Cybersecurity 

Workshop was virtually conducted by 

System-Wide Safety researcher Paul 

Nelson of NASA’s Glenn Research 

Center in Cleveland. This workshop 

was the first in a series of workshops 

presenting important cybersecurity 

topics of critical interest to ARMD 

research. The series began with an 

overview of ARMD’s cybersecurity 

efforts, the ARMD cybersecurity 

team capabilities, and the direction 

of aviation cybersecurity. The value 

of incorporating cybersecurity 

engineering early in a project’s 

lifecycle was discussed. A question 

and discussion period was held 

following the presentation. A repeat 

of this workshop was held on May 13. 

NASA/FAA Quarterly Review

POC: MIKE MADSON AND ROXANA CORZO

On May 10, the Air Traffic 

Management – eXploration 

(ATM-X) project met with 

stakeholders to exchange 

information at the NASA/FAA 

quarterly review. FAA participants 

included Diana Liang, Enterprise 

Portfolio manager; Nabil Sandhu, 

Advanced Air Mobility Below Visual 

Line of Sight National Airspace 

System Evaluation (BNE) lead; 

Sherri Magyarits, Upper Class 

E Traffic Management (ETM) 

Concept of Operations (ConOps) 

lead and Research Transition Team 

co-lead; and Nouri Ghazavi, Urban 

Air Mobility ConOps lead. The FAA 

presented updates on their National 

Airspace System 2035 vision, BNE 

research, and UAM ConOps. 

ATM-X presented research status 

and schedule updates for the “Sky 

for All” concept development, 

Pathfinding for Airspace with 

Autonomous Vehicles (PAAV), 

Extensible Traffic Management 

(xTM), ETM, and UAM Airspace 

Management. Sky for All was 

presented by Shawn Engelland, who 

provided status updates on future 

schedules, as well as an update on 

the final version of the Sky for All 

video displayed at the NASA virtual 

booth at the Air Traffic Control 

Association Technical Symposium 

held from May 17-21. PAAV was 

presented by Robert Fong and Tod 

Lewis who updated the group on 

PAAV’s status and the LMI market 

study findings, as well as future 

collaboration opportunities with the 

FAA. Jaewoo Jung presented ETM 

research transition team updates 

and followed up with a NASA/

FAA open discussion regarding 

collaboration on the Digital Mesh 

Technology Application (DMTA) 

RTT xTM – ATC working 

group. UAM subproject manager 

Kevin Witzberger presented the 

partnership status on Joby Aviation 

(formerly Uber Elevate) as well 

as the overview and timeline for 

the National Campaign (NC-1) 

Simulation Effort X4.

Preliminary UAM Airspace 

Roadmap Shared with Key FAA 

Personnel 

POC: KEVIN WITZBERGER

The UAM subproject management 

team met with representatives 

from the NASA Aeronautics 

Research Institute (NARI) and 

representatives from the FAA’s 

AJV (Mission Support), ANG 

(NextGen), and ATO (Air 

Traffic) united to discuss NASA’s 

preliminary version of the Urban 

Air Mobility (UAM) Airspace 

Roadmap. This roadmap is tailored 

for the FAA to facilitate an efficient 

review and comment cycle and 

is the first of three near-term 

UAM Airspace Roadmap artifacts 

that are currently in progress. 

The UAM Airspace Roadmap 

summarizes how NASA’s UAM 

airspace research is expanding on 

the existing UAM Maturity Level 

(UML) framework to establish 

a system engineering artifact 

that will document assumptions, 

requirements, constraints, and 

system architectures during the 

lifecycle of NASA’s UAM airspace 

research and development. The 

UML scale is a useful framework for 

understanding and evaluating the 

evolution of the National Airspace 

System as it pertains to UAM, 

where the UMLs themselves are 

the periods of change that build up 

to significant “step-functions” in 

operational capabilities. While the 

existing UML definitions provide 
10 AOSP NEWSLETTER // APR-JUN 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

an extensive and well-defined 

treatment of the progression of 

UAM from a vehicle operations 

perspective, they are limited in 

their treatment of the airspace 

component. The UAM Airspace 

Roadmap is expected to start 

identifying the airspace progression 

necessary to evolve the airspace 

towards UML-4. After updating the 

roadmap from feedback received 

from key FAA representatives, 

NASA intends to share it with 

Advanced Air Mobility’s Mission 

Integration Office and executive 

board working group later in May. 

xTM Subproject Joins FAA for 

UPP Phase 2 Briefing Effort to 

CTIA 

POC: JAEWOO JUNG

On May 12, the Air Traffic 

Management – eXploration 

(ATM-X) project’s Extensible 

Traffic Management (xTM) 

subproject manager, Jaewoo Jung, 

joined the FAA for a briefing on the 

status of the Unmanned Aircraft 

Systems (UAS) Traffic Management 

(UTM) Pilot Program (UPP) 

Phase 2 effort to CTIA (formerly 

Cellular Telecommunications 

and Internet Association), a 

trade association representing the 

wireless communications industry 

in the United States. UPP is an 

important component of the FAA’s 

UTM implementation process. 

NASA, FAA, and industry partners 

successfully completed UPP Phase 

1 in Summer 2019, demonstrating 

an information-sharing network 

for UAS operating below 400 

feet. UPP Phase 2 flight activities 

were completed in Autumn 2020 

and included a demonstration 

of the FAA Flight Information 

Management System prototype and 

infrastructure, new technologies 

and data to validate the latest 

ASTM (formerly American Society 

for Testing and Materials) standards 

for remote identification and UAS 

service supplier interoperability, 

and UAS volume reservations 

to notify drone operators of 

emergencies and to ensure other 

UTM capabilities are integrated 

appropriately. UPP Phase 2 also 

evaluated secure information 

exchanges between the FAA, 

industry, and authorized users for 

ensuring data integrity. The CTIA 

audience appreciated the briefing, 

commending the dedication and 

creativity of the UPP Phase 2 

team for safely overcoming the 

COVID-19 challenges while 

preparing and conducting field 

tests. Answers to questions about 

test details, such as cybersecurity 

arrangement and operational 

density, were provided by the FAA 

and NASA briefers. The UPP effort 

was initially supported by NASA’s 

UTM project, and transitioned to 

the ATM-X project with the closing 

of the UTM project. The ATM-X 

project is continuing cooperation 

with the FAA and industry for a 

potential follow-on UPP event. 

Diagnostics & Prognostics 

Group Releases New 

Prognostics Research Python 

Packages 

POC: WENDY OKOLO AND CHRIS TEUBERT

System-Wide Safety project 

researchers from the Diagnostics 

and Prognostics Group and the 

Prognostics Center of Excellence 

(PCoE) at NASA’s Ames Research 

Center in California have released a 

pair of Python packages to assist in 

prognostics research. These packages 

provide modular, extendable 

tools for creating and simulating 

degradation (prognostics) models, 

performing systems healthstate estimation and prediction, 

creating new systems healthstate estimation and prediction 

algorithms, benchmarking 

prognostics performance, and 

visualizing results. These tools are 

intended to help researchers in 

industry, academia, and government 

build new models, algorithms, 

and software architectures for 

prognostics. Functionality is similar 

to the previously released MATLAB 

Prognostics Model, Algorithm, and 

Metrics Libraries, but these tools 

are written in Python, responding 

to a trend towards Python in 

the Prognostics and Health 

Management community, and apply 

the lessons learned since the creation 

of the MATLAB libraries. There 

are two parts to the PCoE’s Python 

prognostics packages: prognostics 

models and prognostics algorithms. 
AOSP NEWSLETTER // APR-JUN 2021 11

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

The team members are Matteo 

Corbetta, Chetan Kulkarni, and 

Chris Teubert.

Focus Meeting for the Data 

Analysis and Prediction RTT 

Effort Fosters Interest 

POC: NIKUNJ OZA

On May 13, a focus meeting for 

the Data Analysis and Prediction 

Research Transition Team (RTT) 

effort was led by Nikunj Oza. The 

meeting fostered interest between 

NASA and FAA personnel on 

connecting with Testbed and 

discovering where machine learning 

is relevant. At the previous quarterly 

meeting, a breakout session led by 

Kee Palopo discussed the Air Traffic 

Management – eXploration Testbed 

and facilitated a desire for more 

focused discussion that was provided 

at this monthly focus meeting. 

ATD-2 Reaches More Than One 

Million Gallons of Fuel Saved 

at Charlotte Intl.

POC: DIVYA BHADORIA

As of May 16, NASA’s Airspace 

Technology Demonstration 2 

(ATD-2) subproject, in collaboration 

with American Airlines and FAA 

Air Traffic Control facilities, has 

saved flight operators more than 

one million gallons of jet fuel since 

September 2017 during the Phase 

1 and 2 field demonstrations at 

Charlotte Douglas International 

Airport. These savings resulted 

from ATD-2 Integrated Arrival, 

Departure, and Surface (IADS) 

system-recommended actions, 

such as holding departures at gates 

with engines off, surface metering 

during peak periods, pre-scheduling 

the departure approval request 

(APREQ) flights at gates, and 

renegotiating the release times 

of APREQ flights while taxiing 

through electronic coordination 

between ATD-2’s IADS system 

and the FAA’s Time Based Flow 

Management/Integrated Departure 

Arrival Capability, which eliminates 

verbal communications between 

Charlotte Tower and Air Route 

Traffic Control Centers. In addition 

to the fuel savings, the ATD 2 

IADS system has reduced CO2 

emissions by 21.4 million pounds, 

which is equivalent to planting 

more than 159,000 urban trees 

based on formulae developed by 

the US Environmental Protection 

Agency (EPA). Flight operators at 

Charlotte also saved 5,527 hours 

of engine run time, and more 

than one million dollars in crew 

cost. The detailed calculations and 

additional operational benefits are 

described in the figure below.

NASA/AOSP Play Active Role 

at ATCA Technical Symposium 

POC: BARRY SULLIVAN

Several NASA speakers participated 

at the Air Traffic Control 

Association (ATCA) Technical 

Symposium from May 17-21. 

Each year ATCA, alongside 

technical co-chairs NASA and the 

FAA, holds its premier technical 

conference in which hundreds of 

people come together to focus on 

Summary of ATD-2 Benefits up to May 16, 2021
12 AOSP NEWSLETTER // APR-JUN 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

the latest ideas, research, products, 

and services in the air traffic 

control arena. During the technical 

symposium, attendees learn 

more about federal involvement, 

including exciting research and 

development conducted at the 

FAA William J. Hughes Technical 

Center and at NASA. Additionally, 

attendees hear industry and 

government leaders speak on 

new entrants and non-traditional 

airspace users, artificial intelligence/

augmented reality, machine learning, 

automation, aviation safety, and 

much more; they also see the latest 

in air traffic control and air traffic 

management research, products, 

and services. Attendees include 

representatives from government, 

industry, academia, and other 

aviation stakeholders. This year 

NASA speakers included Bob 

Pearce, associate administrator 

of ARMD, who participated in a 

discussion with Pamela Whitley, 

associate administrator for the 

FAA’s NextGen, on a fireside 

chat “Charting Aviation’s Future.” 

Akbar Sultan, director of AOSP, 

participated in the opening remarks 

giving an overview of ARMD’s 

research portfolio and relevant 

research capabilities/facilities. 

Cheryl Quinn, deputy director of 

AOSP, participated as a speaker 

on “The Digital Transformation 

of Our ATM System” panel. Misty 

Davies, System-Wide Safety project 

manager, was a speaker on the 

“Intelligent Airspace with AI: From 

Bots to Safety Critical Aviation 

Operations” panel. Al Capps of the 

Airspace Technology Demonstration 

2 subproject was a speaker on the 

”R&D: Paving the Way to a More 

Sustainable, Resilient NAS” panel. 

Parimal Kopardekar of the NASA 

Aeronautics Research Institute 

participated in a roundtable 

discussion. Barry Sullivan 

integration manager of AOSP, was 

a speaker on the “ATCA Technical 

Symposium Buzz Session.” 

In addition, AOSP held virtual 

booths highlighting the program’s 

research portfolio and technical 

achievements. This year’s event 

had more than 400 attendees who 

participated in the week-long 

virtual event. For more information 

see the following website: https://

www.atca.org/techsymposium

ATM-X PAAV Meets with FAA 

ANG-E 

POC: ROB FONG

On May 25, the Pathfinding 

for Airspace with Autonomous 

Vehicles (PAAV) subproject met 

with the FAA William J. Hughes 

Technical Center (FAATC) Office 

team (ANG-E) to explore potential 

collaborations between NASA’s Air 

Traffic Management – eXploration 

(ATM-X) project and the FAATC to 

accelerate research and development 

to address critical barriers to enable 

routine unmanned aircraft cargo 

operations. During the meeting, 

PAAV discussed its approach, which 

included development of a concept 

of operations for unmanned aircraft 

cargo operations, planned research, 

partnership strategy, and flight 

demonstration plans. A follow-on 

meeting is planned for June 8 to 

continue discussions of potential 

collaborative efforts between the 

FAATC and ATM-X.

SWS Project Manager Leads 

AAM Ecosystem Cross-Cutting 

Working Group Assurance of 

Autonomy Panel 

POC: MISTY DAVIES

System-Wide Safety project 

manager Misty Davies led a panel 

for the Advanced Air Mobility 

project’s Ecosystem Cross Cutting 

working group Assurance of 

Autonomy virtual session on May 

25. Approximately 135 attendees 

participated. Panelists for that 

session included George Romanski, 

FAA chief scientific and technical 

advisor for aircraft computer 

software; Jim Paunicka, a technical 

fellow for autonomy with Boeing 

Research and Technology; Eric 

Wright, the head of certification 

and test at Archer; and Suresh 

Kannan, an autonomy expert at 

Nodein. Primary topics included 

the European Union Aviation 

Safety Agency’s new guidance for 

the assurance of machine learning/

artificial intelligence, drivers for 

the use of autonomy, and a few 

highlighted barriers and challenges. 
AOSP NEWSLETTER // APR-JUN 2021 13

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

NASA Presents MuchAppreciated ASRS Capability 

to the UAST

POC: BECKY HOOEY AND STEVE YOUNG

Based on recommendations by the 

industry-based Unmanned Aircraft 

Safety Team (UAST), System-Wide 

Safety has been working with the 

FAA to enhance NASA’s Aviation 

Safety Reporting System (ASRS) 

to enable better reporting of 

Unmanned Aircraft System (UAS) 

incidents as well as better analysis 

of reports regarding safety-relevant 

trends and precursors in this 

domain. In April, two important 

milestones were achieved toward 

this goal. First, on April 2, a revised 

FAA Advisory Circular (AC 00-

46) on ASRS was published. This 

new advisory circular provides, 

among other things, protections 

from punitive action to those 

in the UAS domain who submit 

reports to the system regarding 

safety-related events or occurrences. 

This new and extensive group of 

individuals goes well beyond those 

who traditionally have reported 

events to ASRS (e.g. airline pilots 

and air traffic controllers). Secondly, 

on April 14, a UAS-specific form 

was published as part of the ASRS. 

This allows anyone who submits a 

report to use a tailored form that 

utilizes vernacular more commonly 

known to individuals involved in 

flights in the UAS domain. This 

form was developed in conjunction 

with UAST members (both 

industry and government) and 

Becky Hooey’s ASRS team. With 

this capability now on-line, and the 

associated FAA policy in place, the 

resulting reports should provide 

a wealth of new insights, analysis, 

and risk mitigation research to be 

undertaken regarding real-world 

UAS hazards, incidents, accidents, 

and contributing factors. At a 

follow-on meeting on May 26, 

Becky Hooey presented the new 

ASRS capability. NASA and Hooey 

received kudos and appreciation 

from the chairs during the meeting. 

The UAST sees this new ASRS 

capability as much needed and is 

looking forward to learning from 

the reports that will be generated 

because of the new capability. 

DIP Receives Request For 

Information Responses 

POC: MIRNA JOHNSON

The Air Traffic Management – 

eXploration (ATM-X) project’s 

Digital Information Platform (DIP) 

subproject released a request for 

information (RFI) on March 14 

to obtain information defining 

collaboration strategy and identify 

community needs and goals. As 

a follow-on activity, DIP hosted 

an online information session 

on April 14 that provided the 

stakeholder community with 

background on DIP, example use 

cases, collaboration approach, and 

areas of potential contributions 

both from NASA and interested 

parties. There were approximately 

120 external participants who 

engaged in a Q&A session after the 

presentation. The purpose of this 

information session was to help 

the community understand the 

scope of DIP and thus respond to 

the RFI in a meaningful way. In 

late May, the RFI received more 

than 40 responses from various 

communities which included 

six flight operators, 18 service 

providers, 12 data integrators, 

and six cloud and infrastructure 

providers. Although DIP has a 

near-term objective to prioritize 

traditional operations, DIP must 

be positioned to support and 

scale to emerging operations as 

well. By receiving at least eight 

respondents with direct interest in 

emerging operations, DIP will be 

engaging with a diverse and broad 

community to shape the concept 

and demonstrations. 

NASA Langley Hosts 13th 

Annual NASA Formal Methods 

Symposium 

POC: AARON DUTLE

The 13th Annual NASA Formal 

Methods Symposium (NFM 2021) 

took place virtually from May 

24–28. More than 150 people from 

industry, academia, and government 

participated in the event. Formal 

Methods researchers and practitioners 

from institutions such as the FAA, 

ONERA (French Aeronautics, Space, 

and Defense Research Lab), NASA, 
14 AOSP NEWSLETTER // APR-JUN 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

and universities around the planet 

contributed to the program. More 

information can be found at the 

NFM 2021 website, https://shemesh.

larc.nasa.gov/nfm2021/, including 

links to recordings of all the talks 

and keynote addresses. The program 

included 24 research papers, five 

internationally renowned keynote 

speakers, and an affiliated workshop 

(F IDE 2021). The NASA Formal 

Methods team, including many 

members from the National Institute 

of Aerospace, organized the event, 

which was sponsored and funded by 

the System-Wide Safety project, with 

technical support provided by the 

NASA Aeronautics Research Institute.

SWS Project Team Introduces 

New Framework for 

Accommodating Uncertainty 

POC: WENDY OKOLO AND STEVE YOUNG

The System-Wide Safety project 

team has created and demonstrated 

a framework for uncertainty 

representation, handling, and 

management (TC2-03-08). This 

framework, explained in a paper 

published March 29, 2021, “An 

Approach For Uncertainty 

Quantification And Management 

of Unmanned Aerial Vehicle 

Health,” and authored by Matteo 

Corbetta, Chetan Kilkarni, Portia 

Banerjee, and Elinirina Robinson is 

designed to accommodate different 

sources of uncertainty (such as 

the inputs, models, measurements, 

and computing methods) in the 

computation of In-Time Aviation 

Safety Management System (IASMS) 

service outputs and metrics. This 

research proves uncertainty can 

be reasonably quantified in order 

to lessen threats pertaining to or 

originating from autonomous 

systems in the National Airspace 

System. The framework is still 

evolving as more scenarios are 

discovered and will be completely 

implemented in the future to 

support the IASMS.

NASA Renews Space Act 

Agreement with GE for Safety 

Assurance of Flight Critical 

Systems 

POC: GUILLAUME BRAT

NASA has just renewed its Space 

Act Agreement (SAA) with General 

Electric (GE) to collaborate on 

software assurance for increasingly 

autonomous systems. The umbrella 

agreement is for five years, lasting 

through May 2026. The first 

three-year annex focuses on safety 

assurance research to broaden 

knowledge of operational risk 

assessment, evidence curation, and 

adaptive stress testing. The previous 

SAA between NASA and GE 

Global Research expired on May 

24. This agreement has increased 

the collaboration between NASA 

and GE. Some of the activities 

and accomplishments under this 

SAA included: (1) regular monthly 

meetings to review research needs 

and possible solutions; (2) successful 

technology transfer of a NASAdeveloped adaptive stress testing 

tool (AdaStress) to GE Aviation; (3) 

technology transfer of AdvoCATE, 

a NASA-developed tool to create 

and review safety cases; and (4) 

review of the Formal Requirements 

Elicitation Tool (FRET), a NASAdeveloped tool to elicit and 

formalize requirements for critical 

systems. The new agreement with 

GE enables NASA to work with all 

GE business units and builds on 

the past collaboration by applying, 

for example, the AdaStress tool to 

a flight management system and 

helping GE incorporate it into their 

production testing environment. 

SWS and Agility Prime 

Discuss Firefighting Use Case 

Collaboration 

POC: MISTY DAVIES AND NATASHA NEOGI

On June 2, the System-Wide Safety 

(SWS) project held a technical 

integration meeting with Agility 

Prime’s leadership to discuss possible 

collaboration points for Agility 

Prime’s Firefighting Use Case and 

SWS’s Safety Demonstrator Series. 

Joey Mercer presented the impact of 

NASA’s Scalable Traffic Management 

for Emergency Response Operations 

(STEReO) activity under the 

Transformative Aeronautics 

Concepts Program’s Convergent 

Aeronautics Solutions project, and 

Natasha Neogi presented the current 

plans for SWS and how the Safety 

Demonstrator Series will leverage 
AOSP NEWSLETTER // APR-JUN 2021 15

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

STEReO. Previous technical 

interchange meetings focused on 

Agility Prime’s plans and SWS’s InTime Aviation Safety Management 

System. These meetings are in 

preparation for the upcoming Space 

Act Agreement between NASA 

and Agility Prime being led by the 

Advanced Air Mobility Mission 

Integration Office.

SWS Presents on Safety 

Issues of Integrating CargoCarrying UAS into the NAS 

POC: NATASHA NEOGI

On June 4, Natasha Neogi 

presented a briefing at the Air 

Traffic Management – eXploration 

(ATM-X) project’s Pathfinding 

for Airspace with Autonomous 

Vehicles Concept of Operations 

Workshop on a safety perspective 

on integrating cargo-carrying 

Unmanned Aircraft Systems (UAS) 

into the National Airspace System. 

Entitled “Towards Automitigation: 

A Safety Perspective,” Neogi’s 

presentation highlighted certification 

challenges for UAS as well as issues 

related to operational approvals 

that these systems would need to 

receive in order to gain access to the 

airspace system. Non-traditional 

methodologies for tackling hazard 

analyses of novel concept of 

operations for new vehicle systems 

were discussed, specifically the 

system theoretic process analysis. 

Additionally, an algorithm 

for automatically performing 

contingency management maneuvers 

was outlined.

NASA Research Announcement Kickoff Meeting Held for 

SWS Work with Notre Dame 

and Iowa State

POC: LILLY SPIRKOVSKA

On June 7, a kickoff meeting was 

held to share and review plans for a 

new cooperative research agreement 

between the System-Wide Safety 

(SWS) project and a team from 

the University of Notre Dame 

and Iowa State University. The 

three-year agreement is a result of 

a NASA Research Announcement 

solicitation. The number of small 

Unmanned Aircraft Systems 

(sUAS) have risen dramatically in 

recent years, resulting in numerous 

reported incidents involving near 

collisions in controlled airspace or 

out of control sUAS in populated 

areas. This creates the risk of serious 

accidents. The problem is exacerbated 

by the diverse quality and robustness 

of sUAS and also by the varied skills 

of their operators. This research 

activity focuses on: (1) developing a 

diverse dataset of annotated sUAS 

flight logs collected from internal 

flights and crowd-sourced from 

other FAA part 107 certificated 

remote pilots, as well as hobbyist 

pilots; (2) developing techniques 

for detecting and interpreting 

anomalous flight patterns; (3) finding 

runaway emergent behavior due 

to interactions between hardware, 

software, and environmental 

conditions; and (4) analyzing 

underlying causes of flight 

anomalies to identify appropriate 

mitigations. The planned work 

directly benefits ongoing research 

in the SWS project. The team 

anticipates mutual leveraging of 

cooperative efforts, utilizing their 

analytics and insights as part of 

NASA’s monitor-assess-mitigate 

strategy and in deploying SWS’s 

services, functions and capabilities 

on their sUAS. 

NASA-Joby Urban Air Mobility 

Use Case Discussion with 

the FAA 

POC: SAVVY VERMA

On June 10, the Air Traffic 

Management – Exploration 

(ATM-X) project’s Urban Air 

Mobility (UAM) subproject 

conducted their first of four planned 

meetings with the FAA and industry 

partner Joby Aviation to create a set 

of use cases in the Dallas area that 

were presented and discussed with 

FAA stakeholders. This series of 

meetings will help to refine airspace 

and procedure concepts for UAM 

operations in the near term. FAA 

participants included representatives 

from the FAA’s NextGen, Air Traffic 

Organization, and Air Traffic Services 

organizations. This activity is seeking 

direct input from airspace experts 

in the form of group discussions 

focused on a series of UAM scenarios 

or use cases that range from 
16 AOSP NEWSLETTER // APR-JUN 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

operations flying in Class E or G 

airspace that require no or minimal 

interaction with air traffic controllers 

to operations inside controlled 

airspace. The usage of current day 

tools like letters of agreements to 

achieve UAM operations were also 

explored. The use cases/scenarios 

will cover a range of operational 

challenges and potential solutions. 

The results and lessons learned from 

these sessions will be used to identify 

research questions and upcoming 

simulation studies planned for the 

UAM subproject. 

ATM-X xTM/ETM Continue 

Presenting Upper Class 

E Research Plans to 

Stakeholders 

POC: WILLIAM CHAN AND JAEWOO JUNG

This past quarter, the Extensible 

Traffic Management (xTM) 

subproject of the Air Traffic 

Management – eXploration 

(ATM-X) project provided several 

detailed briefing’s of its Upper 

Class E Traffic Management 

(ETM) research to their various 

stakeholders. On April 7, the 

ETM work was presented to 

the Association for Unmanned 

Vehicle Systems International. 

The presentation was a result of 

a previous presentation to the 

Aerospace Industries Association 

on March 17. William Chan 

provided an overview of the ATM-X 

project to set the context, while 

Jaewoo Jung presented the ETM 

information specific to the xTM 

subproject. Jung discussed the 

challenges of operating in Upper 

Class E airspace, reviewed the work 

that has been done in previous FAA 

and industry tabletop meetings, and 

xTM subproject work to define a 

service-based cooperative approach 

to ETM. 

On April 30, the team met with the 

assistant director of the Montana 

Space Grant Consortium (MTSGC), 

an affiliate of NASA’s National 

Space Grant College and Fellowship 

project. In 2017, MTSGC led 

a nationwide eclipse ballooning 

campaign during the total solar 

eclipse (website: https://eclipse.

montana.edu/). In this campaign, 

students conducted balloon flights 

in the Upper Class E airspace from 

more than 25 locations across 

the eclipse path from Oregon to 

South Carolina, and live videos and 

images from the near-spaceflights 

were submitted for display on the 

NASA website. Proposals are being 

submitted to conduct a similar 

campaign during the 2023 annular 

and 2024 total solar eclipses. The 

ETM team and MTSGC discussed 

potential opportunities for 

collaboration and agreed to meet 

again to discuss details of more 

specific areas for collaboration and 

appropriate agreement processes 

that can be leveraged. 

On June 10, the ETM team 

met with representatives from 

industry and the FAA. In this 

meeting, a Department of Defense 

representative briefed on how the 

U.S. Navy currently coordinates 

high-altitude operations for 

munitions testing with the FAA, and 

MTSGC representatives presented 

stratosphere balloon operations 

for the 2017 solar eclipse and their 

coordination with FAA centers. 

NASA and the FAA provided a 

review of the Unmanned Aircraft 

Systems Traffic Management system 

and discussed how its requirements, 

architecture, and data exchange 

process could be leveraged for the 

ETM system development. The 

meeting concluded with an update 

on NASA’s ETM simulation work 

and the attendees were invited to 

the upcoming ETM workshop, 

scheduled for July 21-22, to discuss 

and further develop ETM concepts.

DIP Meets with America for 

Airlines and Delta to Discuss 

RFI Evaluation Status 

POC: MIRNA JOHNSON

On June 10, NASA’s Digital 

Information Platform (DIP) team 

met with Andy Cebula, vice 

president of America for Airlines, 

and Robert Goldman, senior 

manager of air traffic management 

at Delta Airlines, to discuss next 

steps following the request for 

information (RFI) evaluation. 

NASA is looking to engage with 
AOSP NEWSLETTER // APR-JUN 2021 17

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

the airline community to brief 

takeaways from the responses 

and follow up on priorities 

and gain further insight. Delta 

expressed willingness to share the 

necessary non-proprietary data 

for a successful collaboration 

with industry service providers. 

NASA initiated discussions to 

understand what is needed from 

the community for a successful 

transition plan. These meetings are 

in preparation for upcoming Space 

Act Agreements between NASA and 

interested airlines. 

NASA SWS Participation in 

the DARPA ARCOS Principal 

Investigator Meeting 

POC: MALLORY GRAYDON

Researchers Mallory Graydon, 

Paul Miner, and Natasha Neogi 

participated in the principal 

investigator’s meeting of the Defense 

Advanced Research Projects Agency 

(DARPA)’s Automated Rapid 

Certification of Software (ARCOS) 

project on June 10, representing 

the System-Wide Safety project. 

Graydon and Sushil Birla of the 

Nuclear Regulatory Commission 

participated on a government 

evaluator panel, answering questions 

from program participants. ARCOS 

participants briefed DARPA and 

the government reviewers on their 

progress, including the creation of 

data stores for certification evidence, 

tools for producing test and analysis 

evidence on demand, and tools 

for assembling safety arguments 

to present and explain safety and 

security concepts and evidence to 

certification agents. While ARCOS 

aims to reduce the cost of certifying 

software-intensive military systems, 

the tools and techniques produced 

may have applications to software 

used in aviation and spaceflight 

applications. The web-site for 

DARPA’s ARCOS project is https://

www.darpa.mil/program/automatedrapid-certification-of-software. 

PAAV Holds Tabletop 

Walkthroughs of Unmanned 

Aircraft Cargo Scenarios 

POC: ROB FONG

The Air Traffic Management – 

eXploration (ATM-X) project’s 

Pathfinding for Airspace with 

Autonomous Vehicles (PAAV) 

subproject held a week-long 

tabletop walk-through of unmanned 

aircraft cargo scenarios with 

subject matter experts, including 

remote pilots, controllers, and 

dispatchers, from May 3-7. The goal 

of the tabletop walkthrough was to 

highlight shortfalls and gaps that 

prevent routine unmanned aircraft 

cargo operations in the National 

Airspace System within the current 

Air Traffic Control operating 

paradigm. A second concept of 

operations tabletop exercise was 

held virtually from June 14-18. This 

exercise was a continuation of the 

first tabletop held in May 2021, 

and new topics were discussed such 

as clearances, surface operations, 

launch and recovery, and increased 

automation considerations. The 

identified shortfalls from both 

tabletop exercises will be used to 

inform trade studies of possible 

near-term solutions as well as 

to inform possible automation 

solutions in the future. 

NASA/Northrop Grumman 

Autonomous Systems Meeting 

POC: ROB FONG

On June 24, the Air Traffic 

Management – eXploration 

(ATM-X) project’s Pathfinding for 

Airspace with Autonomous Vehicles 

(PAAV) subproject met with 

Northrup Grumman – a potential 

flight demonstration partner. In 

the meeting, NASA discussed 

opportunities for collaboration 

and in-kind contributions for a 

potential partnership. Northrop 

Grumman provided a tour of 

their automation laboratory and 

their proposed flight asset and its 

capabilities during the visit. The 

information NASA obtained from 

the meeting will be used to scope 

a non-reimbursable partnership to 

support demonstrations to verify 

PAAV procedures and technologies 

to enable routine unmanned 

autonomous cargo operations in 

the National Airspace System in a 

relevant flight environment. 
18 AOSP NEWSLETTER // APR-JUN 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

ATM-X Attends AAM Executive 

Board Airspace Operations 

Working Group Meetings

POC: ARWA AWEISS AND IAN LEVITT

This past quarter the Air Traffic 

Management – eXploration 

(ATM-X) Urban Air Mobility 

(UAM) subproject team attended 

the Advanced Air Mobility (AAM) 

executive board airspace operations 

working group meetings. At the 

May 21 meeting, the UAM team 

presented early-term use cases. 

Savvy Verma, technical lead for 

airspace procedures and design 

at NASA’s Ames Research Center 

in California, walked the group 

through procedures for entry into 

a corridor at the boundary between 

Class E and Class B airspace in 

the Dallas/Fort Worth operational 

setting. NASA participants 

included UAM personnel, the AAM 

Mission Integration Office, and 

other members of the executive 

board. FAA participants included 

representation from the NextGen 

Office and multiple safety and 

operations organizations within 

the FAA’s Air Traffic Organization. 

Ian Levitt, NASA co-lead for the 

working group and principal 

engineer of UAM, helped facilitate 

the conversation. A rich discussion 

unfolded, highlighting the need 

for the working group to validate 

key UAM concept of operations 

principles (e.g. around corridors and 

provider of services for UAMs) as 

they pertain to the near- and earlyterm integration of UAM operations 

into the National Airspace System. 

At the follow-on meeting on June 

25, the focus was on early UAM 

Maturity Level use cases (UML-1/

UML-2). Verma presented again 

on some work that was conducted 

in collaboration with Joby Aviation 

(formerly Uber Elevate) and 

presented “Early UAM Operations 

in the Dallas Airspace.” There was 

constructive dialogue among the 57 

participants in attendance.

SWS Develops Shielded 

Learning Approach Enabling 

the Enforcement of Safety 

Properties 

POC: NATASHA NEOGI AND MISTY DAVIES

In June, System-Wide Safety 

researcher Natasha Neogi, along with 

Ufuk Topcu and Suda Bharadwaj 

of the University of Texas at Austin, 

successfully collaborated in designing 

a framework which enables the 

enforcement of safety properties for 

agents that may possess learningenabled components. The team 

formalized safety-oriented constraints 

derived from FAA regulations by 

encoding them as temporal logic 

formulae. However, the wellregulated safety-critical nature of 

aviation led to multiple competing 

safety constraints that can be traded 

off based on the operational context. 

They then developed, formalized, 

and demonstrated via simulation 

a method for facilitating an urban 

air mobility operation that is both 

scalable and minimally violates 

the temporal logic constraints. 

Numerical results show that the run 

time for the proposed algorithm is 

suitable for very large problems and 

is backed by theoretical guarantees 

of correctness with respect to given 

temporal logic constraints. This work 

has led to a paper at the 2021 NASA 

Formal Methods Workshop, two 

accepted journal articles, and forms 

a fundamental part of Bharadwaj’s 

Ph.D. thesis, which has been 

nominated for the Best Thesis award 

at the University of Texas at Austin 

College of Engineering. 
AOSP NEWSLETTER // APR-JUN 2021 19

RECOGNITION

Integrated Communications 

Navigation and Surveillance 

(ICNS) Conference

POC: RAFAEL APAZA

The ICNS Conference was 

conducted virtually from April 

20-22. The program offered three 

plenary sessions that focused 

on “Global Harmonization, New 

Opportunities and Unforeseen 

Issues,” “The Future Trajectory 

for ATC,” and “Evolving ICNS 

Technologies for Drones, AAM, 

and Beyond.” A technical program 

presented global communications, 

navigation, and surveillance 

research and technology advances. 

Rafael Apaza of NASA’s Glenn 

Research Center in Cleveland 

was the chair of the Special 

Topics and New Entrants track 

and served in organizing and 

moderating the Paper Presentation 

track. Additionally, a paper titled 

“Investigation and Evaluation of 

Advanced Spectrum Management 

Concepts for Aeronautical 

Communications” was included 

as part of the conference 

proceedings. The paper was 

co-authored by Eric Knoblock 

and Rafael Apaza, Hongxiang Li, 

Zhe Wang, Ruixuan Han, and 

Nathan Schimpf of the University 

of Louisville and Nathaniel Rose 

of HX5. Apaza is also a member 

of the ICNS executive committee 

and currently serves as the 

conference communications chair. 

SWS Hosts Virtual Booth 

at AUVSI Conference 

POC: KAITLYN FOX

NASA’s System-Wide Safety project 

participated in the Association 

for Unmanned Vehicle Systems 

International (AUVSI) Xponential 

Virtual Conference held from 

May 4-6. System-Wide Safety 

volunteers hosted a virtual booth 

for event attendees. Unlike a 

traditional booth setting, this 

event focused on developing 

connections with other participants 

with an intended outcome 

of connecting with relatable 

organizations to explore potential 

collaboration opportunities. 

“NASA UTM: A Giant Leap 

for Air Transportation” 

Video Premiers 

POC: JAEWOO JUNG

On May 6, the “NASA UTM: A 

Giant Leap for Air Transportation” 

video premiered through live 

streaming on YouTube and 

Facebook with more than 6,000 

total combined views thus far. 

This 24-minute video, produced 

by the NASA X team, explains 

what Unmanned Aircraft Systems 

Traffic Management (UTM) is, 

why it is needed, and how it is 

designed to work. The video also 

features the dedicated NASA team 

at various field centers together 

with its government and industry 

partners, showing the viewers a 

better understand of the work 

accomplished and the testing that 

is done in real-world settings. The 

video can be viewed on YouTube 

at the following website: https://

youtu.be/vepo3q87Grc. 

System-Wide Safety 

Participates in Advanced 

Air Mobility Career Day 

POC: KYLE ELLIS AND MISTY DAVIES

On May 12, Kyle Ellis of NASA’s 

System-Wide Safety project 

participated in the Advanced 

Air Mobility (AAM) career day 

student panel to engage middle 

and high school students from 

across the country, sharing with 

them on the future of aviation 

and encouraging the pursuit of 

STEM education. The all-virtual 

event included a session titled 

“How the Digital Generation Will 

Influence Aviation,” consisting 

of a panel of three students and 

Ellis. Matthew Harding, a physics 

teacher at Iowa City West High 

School, who was also Ellis’s former 

high school teacher, participated 

as the moderator. More than 500 

viewers, some representing entire 

classrooms, were actively tuned 

into the discussion focusing on 

areas of excitement and workforce 

skills needed for NASA to assist 

the aviation industry to overcome 

the challenges necessary to realize 

the vision of AAM. This STEM 

engagement activity and others 

like it serve as essential connection 
20 AOSP NEWSLETTER // APR-JUN 2021

RECOGNITION

opportunities with students who 

represent tomorrow’s workforce, 

seeking to foster the talent 

pipeline necessary for NASA to 

accomplish its future missions. 

IEEE Robotics and Control 

Engineering Seminar – UAM 

Control-Centric Approach 

POC: IRENE GREGORY

On May 14, Irene Gregory of the Air 

Traffic Management – eXploration 

(ATM-X) project’s Urban Air 

Mobility (UAM) subproject gave an 

invited seminar to the Institute of 

Electrical and Electronics Engineer 

(IEEE) Control Systems Society 

entitled “UAM: Control-Centric 

Approach Addressing Technical 

Challenges.” The talk highlighted 

major challenges to enable UAM, the 

most challenging sector of Advanced 

Air Mobility, and then highlighted 

research efforts at NASA that address 

these challenges. The work has been 

funded by three ARMD projects and 

is an example of cross-disciplinary 

collaboration and leveraging of 

resources. The presentation outlined 

some of these challenges and current 

approaches to addressing them. For 

example, in order to achieve full 

market potential and access to UAM, 

vehicle autonomous flight is required. 

A key barrier to autonomous 

flight in a large multi-agent system 

is the handling of off-nominal 

situations and contingencies in a 

safe and predictable manner. An 

approach to intelligent contingency 

management was presented along 

with recent results and open 

problems. Additionally, she discussed 

another major barrier to ubiquitous 

UAM – the noise signature produced 

by vehicles with multiple rotors. 

She presented NASA’s approach to 

minimizing such noise within the 

framework of the acoustically aware 

vehicle. The recording can be found 

at http://ieeecss.org/videos-webinars. 

SWS Participates in 

Women in Aerospace for 

Michigan Students 

POC: MISTY DAVIES

On May 15, Misty Davies, project 

manager of System-Wide Safety, 

moderated a breakout session panel 

at the virtual Women in Aerospace 

conference. The conference was 

organized for Michigan students, 

K-12 and academia, and included 

professional women, college students, 

and high school robotics students 

in engineering. Approximately 

60 students were in attendance. 

Panel topics included engineering 

pathways, preparation for STEM 

careers, and the importance of 

resilience to overall life success. 

SWS Researchers Present 

a Paper on IASMS at ISAP 

POC: LANCE PRINZEL AND MISTY DAVIES

On May 18, Lance Prinzel and Paul 

Krois presented a paper entitled 

“Examining the Changing Roles 

and Responsibilities of Humans 

in Envisioned Future In-Time 

Aviation Safety Management 

Systems (IASMS)” at the 

International Symposium on 

Aviation Psychology (ISAP). The 

paper addressed how today’s safety 

levels would be maintained through 

an IASMS concept of operations as 

Advanced Air Mobility transforms 

aviation. The IASMS represents 

a system-of-systems perspective 

on interconnected services, 

functions, and capabilities needed 

to proactively reduce risks in 

increasingly complex operational 

environments and where the 

balance between humans and 

automation changes in relation to 

such envisioned systems, which 

may lead to novel human-machine 

interaction paradigms and humanautonomy teaming for informed 

contingency management. There 

were several additional NASA 

presentations at ISAP and the 

virtual conference was attended 

by more than 130 human factors 

scientists and academics. The 

proceedings can be accessed 

at https://aviation-psychology.

org/. Of special note, former 

NASA employee Amy Pritchett 

was made an ISAP fellow for her 

distinguished contributions to 

aviation psychology over the years, 

and in memoriams were included 

for Key Dismukes of NASA’s Ames 

Research Center in California 
AOSP NEWSLETTER // APR-JUN 2021 21

RECOGNITION

and Tom McCloy of the FAA 

Flight Deck Human Factors. The 

next ISAP will be held in 2023. 

SWS Researcher 

Presents Tutorial at 

2021 IEEE International 

Conference on PHM 

POC: WENDY OKOLO AND STEVE YOUNG

On June 7, System-Wide Safety 

researcher Portia Banerjee 

presented a virtual tutorial at 

the 2021 Institute of Electrical 

and Electronics Engineers 

(IEEE) International Conference 

on Prognostics and Health 

Management (PHM). Banerjee’s 

presentation, entitled “In-Time 

Safety Assessment and Risk 

Prediction for Unmanned Aerial 

Systems,” provided a summary 

of System-Wide Safety project 

work as well as a description of 

diagnostic and prognostic tool 

implementation to predict risks 

for Unmanned Aircraft Systems 

(UAS). While the presentation 

focused specifically on UAS, 

the applications of the tutorial 

techniques encompass various 

domains. Different prognostics 

and health management techniques 

(including Kalman filtering, 

uncertainty propagation, frequency 

spectrum analysis, Bayesian Belief 

network, quantifying uncertainty, 

etc) were demonstrated and 

applied to in-time risk prediction. 

Banerjee also extrapolated 

the process of monitoring, 

assessing, and mitigating risks 

to more general applications. 

SWS Team Introduced 

SDSP Consolidation 

Dashboard for Testing 

POC: WENDY OKOLO AND STEVE YOUNG

In June, a Supplemental Data Service 

Provider (SDSP) consolidation 

dashboard was developed to 

display System-Wide Safety (SWS) 

services for Unmanned Aircraft 

Systems (UAS). Two functional 

SDSPs monitoring battery levels 

and proximity to static obstacles 

developed by the SWS team at 

NASA’s Ames Research Center in 

California have been integrated into 

the consolidation dashboard. The 

dashboard visualization of these 

SDSPs is being tested via simulation 

at NASA Ames with flight testing 

scheduled for July at NASA’s 

Langley Research Center in Virginia. 

During these flight tests, live UAS 

at NASA Langley and simulated 

UAS at Ames will communicate 

concurrently to demonstrate 

integration and test the consolidation 

dashboard functionality. 

SWS Submits Article in 

American Airlines Publication 

POC: JON HOLBROOK AND MISTY DAVIES

In June, the System-Wide Safety 

(SWS) project submitted an 

additional article to American 

Airlines’ Safety Preflight publication. 

Authored by Kevin B. Gregory of the 

Fatigue Countermeasures Laboratory 

at NASA’s Ames Research Center 

in California, the article titled 

“NASA’s Fatigue Countermeasures 

Laboratory and American Airlines 

(AA) Fatigue Risk Management 

System (FRMS)–A Working 

Relationship” describes American 

Airlines and NASA’s long-standing 

partnership conducting human 

factors fatigue research in airline 

operations. Thus far in 2021, SWS 

has authored an article in every issue 

of Safety Preflight, highlighting 

the collaborative work between 

NASA and American Airlines. 

These include the following:

• May issue: “Pilots Causing 

Safety: NASA and American 

Airlines Working Together” 

by John Holbrook. The 

article highlights the need to 

understand not only the errors 

that reduce safety, but the desired 

behaviors that produce safety 

and the ongoing collaboration 

between NASA and American 

Airlines. 

• March/April issue: “Improving 

Aviation Together: NASA and 

American Airlines Collaboration” 

featuring the rich history of SWS 

and American Airlines working 

together 

• March/April issue: “From 

Jumpseat to Earthquake: 10 

Unexpected Days in Chile” by 

Lance Prinzel
22 AOSP NEWSLETTER // APR-JUN 2021

RECOGNITION

• January/February issue: “The 

Impact of COVID-19 on 

American Airlines Operations: A 

New Perspective on Some Old 

Problems” by Steve Casner, Chad 

Stephens, and Lance Prinzel

• January/February issue: “Is the 

Last Inflight Rest Break Really 

the Best?” by Erin Flynn-Evans 

and Kevin Gregory

Safety Preflight issues are hosted internally on American Airlines servers. 

Copies of the NASA articles can be 

provided upon request by emailing 

kaitlyn.d.fox@nasa.gov. 
National Aeronautics and Space Administration

Headquarters

300 E. Street, SW

Washington, DC 20024

www.nasa.gov/aeroresearch

www.nasa.gov