National Aeronautics and Space Administration

www.nasa.gov

AOSP Newsletter

Airspace Operations and Safety Program (AOSP)

OCT-DEC 2021 | Quarter 1

Proof-of-Concept of Airline 

Operations Center Dashboard

4

NASA-Developed Algorithms 

Identify Previously Unknown 

Safety Issues

9
2 AOSP NEWSLETTER // OCT-DEC 2021

AOSP IN THE NEWS

With NASA Partnership, 

Orlando Begins Planning 

for Air Taxis, Flying Cars

Orlando (FL) Sentinel (10/7) reports 

“With NASA partnership, Orlando 

is preparing for when flying cars are 

an option for those who want to soar 

over congested highways or between 

nearby cities. And they may arrive 

far sooner than 2062, as The Jetsons 

predicted.”

.
AOSP NEWSLETTER // OCT-DEC 2021 3

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

NASA Kicks Off Autonomy 

V&V 2045 Vision NRA 

POC: GUILLAUME BRAT

On Oct. 1, the NASA Autonomy 

Verification and Validation (V&V) 

2045 Vision NASA Research 

Announcement (NRA) was kicked 

off between NASA’s SystemWide Safety project and a Boeingled contractor team. The main 

goal of this NRA is to develop a 

comprehensive landscape of the 

V&V needs for autonomy through 

2045. Moreover, this NRA team 

will also develop an assurance (V&V 

techniques) technology roadmap 

to describe what services would 

be enabled if these technologies 

are realized, and identify the 

technologies needed to fill the gaps 

in the current regulatory framework 

for autonomy. The results of this 

NRA will provide guidance for 

future autonomy V&V technology 

development and will enable 

NASA to prioritize autonomy V&V 

funding over the next few years. 

For this NRA, NASA selected a 

team led by Huafeng Yu (Boeing), 

including industry and academia 

experts. Major aviation companies 

were represented by Mike Durling 

of General Electric Global 

Research, Darren Cofer of Collins 

Aerospace, as well as Boeing and its 

affiliates (e.g., Aurora). Academic 

experts include Ella Atkins of the 

University of Michigan, Chuchu 

Fan of the Massachusetts Institute of 

Technology, and Ufuk Topcu of the 

University of Texas at Austin. This 

core team will be responsible for 

developing the documents and will 

be advised by representation from 

government agencies, including the 

FAA, the Air Force Research Lab, 

the Naval Research Lab, and the 

Defense Advanced Research Projects 

Agency, which all have an important 

stake in this technology. The report’s 

first draft will be available in Jan. 

2022, while the final report will be 

developed by the end of Sep. 2022. 

The kickoff presentations mostly 

focused on short introductions 

by the team, the project’s goals 

and scope, the process that will be 

followed to develop the roadmap, 

and the schedule. This kickoff was 

well attended by NASA staff and 

researchers as well as representatives 

from other government agencies. 

AOSP Director Akbar Sultan 

and Transformative Aeronautics 

Concepts Program Director John 

Cavolowsky were in attendance and 

both emphasized the importance 

of the work in shaping the 

Aeronautics Research Mission 

Directorate’s future research. 

NC and UAM Conduct 

Follow-on Meeting 

with Wisk Aero 

POC: KEVIN WITZBERGER

On Oct. 1, representatives from the 

National Campaign (NC) and the 

Air Traffic Management-eXploration 

project’s Urban Air Mobility (UAM) 

subproject held a follow-on meeting 

with Wisk Aero. The meeting’s 

purpose was to discuss potential 

airspace, routes, and procedures, 

following initial discussions held 

back in Sept. Wisk Aero is a key 

vehicle developer and participant in 

the NC-1 flight tests due to their 

desire to lean into partnering with 

multiple X4 airspace partners that 

are currently collaborating with 

NASA to develop their company’s 

Provider of Services for UAM that 

will be tested in the X4 simulation 

effort. Following the simulation 

activity, Wisk Aero and multiple 

airspace providers will conduct a 

flight demonstration with an electric 

vertical takeoff and landing vehicle 

to assess the functional allocation 

and data exchanges needed for 

scalable UAM operations. 

ATM-X xTM ETM Team Meets 

with FAA and Industry 

POC: JAEWOO JUNG

On Oct. 5, the Air Traffic Management-eXploration (ATM-X) project’s Extensible Traffic Management 

(xTM) subproject Upper Class E 

Traffic Management (ETM) team 

met with representatives from the 

FAA and industry. In this meeting, 

Airbus and Sceye shared information regarding their recent flight 

tests, and a representative from the 

Aerospace Industries Association presented a proposal for the Cooperative Operations in Higher Airspace 

Concept. This proposal is in a draft 

stage and the group discussed the 

proposed concept. The meeting con-
4 AOSP NEWSLETTER // OCT-DEC 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

cluded with an update on NASA’s 

ETM simulation work. The group is 

planning to meet once more in 2021 

to further discuss and develop ETM 

concepts. 

Proof-of-Concept of Airline 

Operations Center Dashboard

POC: NIKUNJ OZA, CHAD STEPHENS

 AND LANCE PRINZEL

On Oct. 5, Booz Allen Hamilton 

demonstrated to the System-Wide 

Safety project team a dashboard 

displaying the results of running 

machine learning (long short-term 

memory) time series methods (autoregressive integrated moving average), 

and survival analysis (Kaplan-Meier 

estimator) on FAA System-Wide 

Information Management data. The 

dashboard displayed the risk of 

unstable approaches for each flight 

for each point along its track. By 

displaying this for multiple tracks, 

an indication of risk for regions of 

the airspace can also be visualized. 

This demonstration was a proofof-concept that is a stepping-stone 

toward a dashboard demonstration 

that will constitute the final 

deliverable of Technical Challenges-1 

(Integrated Terminal Area Risk). 

The final deliverable dashboard will 

take, as input, data that is currently 

used by the American Airlines 

International Operations Center and 

will return one or more measures 

of risk, including anomalies and 

precursors to safety issues. 

ATM-X DIP and TACP Data 

and Reasoning Fabric 

Teams Conduct TIM 

POC: MIRNA JOHNSON

On Oct. 6, the Air Traffic 

Management-eXploration (ATM-X) 

project’s Digital Information 

Platform (DIP) subproject and the 

Data and Reasoning Fabric team 

from the Transformative Aeronautics 

Concepts Program’s (TACP’s) 

Convergent Aeronautics Solutions 

project gathered for the first in a 

series of virtual Technical Interchange 

Meetings (TIMs) to share approaches 

of common capabilities identified 

in both projects and to identify 

potential areas for collaboration 

or integration. Each project 

offers a platform or framework 

to connect services and there are 

some common basic features with 

similar requirements. However, 

due to different community 

needs and objectives, there are 

also differentiation features. The 

first TIM covered approaches to 

common functionalities in terms 

of deployment and cloud usage. 

Follow-on TIMs will cover additional 

common functionalities, such as 

registry and catalog of services, as 

well as security for authentication 

and authorization. 

SWS Project Meets with 

UAS Service Providers

POC: MICHAEL VINCENT

On Oct. 6, System-Wide Safety 

(SWS) project representatives 

Michael Vincent, associate project 

manager, and Ersin Ancel attended 

the Association for Unmanned 

Vehicle Systems International 

Hampton Roads Autonomous 

Systems Demonstration Day 

event to network with companies 

developing Unmanned Aircraft 

Systems (UAS) and UAS-related 

services to explore potential 

collaborations with the SWS project. 

They met with CEO Marco Sterk 

of Longbow Group to discuss the 

possibility of collaborating on their 

upcoming effort to get approval 

to fly UAS between Fort Monroe 

in Hampton, Virginia, and the 

City Environment for Range 

Testing of Autonomous Integrated 

Navigation (CERTAIN) facility at 

NASA’s Langley Research Center 

in Virginia. In addition, they 

met with CEO Bill Fredricks of 

Advanced Aircraft Company to 

discuss the applicability of their 

long endurance, hybrid-powertrain 

UAS to monitor methane emissions 

from landfills. This is a current 

UAS operation of interest for a 

Small Business Innovative Research 

(SBIR) contractor. The SWS 

representatives also met with Vice 
AOSP NEWSLETTER // OCT-DEC 2021 

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

President James Eanes of Daniel 

H. Wagner Associates to discuss 

the applicability of two of their 

SBIR contracts involving urban 

microclimate monitoring and 

autonomy assurance. Follow up 

meetings with all three contacts are 

currently being planned. 

ATM-X DIP Team Conducts 

Technical Interchange 

Meeting with FAA 

POC: MIRNA JOHNSON

As an outcome of the recent NASA/

FAA Quarterly Review held in late 

Aug., the Air Traffic ManagementeXploration (ATM-X) project’s 

Digital Information Platform 

(DIP) subproject held a Technical 

Interchange Meeting (TIM) with 

the FAA from Oct. 13-14. The 

TIM’s focus was related to the FAA’s 

Flight and Flow Information for 

Collaborative Environment. The 

meeting provided an opportunity 

for NASA to hear and ask 

questions firsthand from the FAA 

on requirements, constraints, and 

concerns. The flight and flow data 

will be part of the extensible and 

dynamic data sharing for decisionmaking and negotiation. The DIP 

team is looking to learn more 

about the object data structures 

to inform the digital information 

sharing architecture for the DIP 

platform. The information layer is 

important to support microservices 

in an information-centric National 

Airspace System architecture. 

Standardizing information exchange 

will enable the platform to be 

more flexible and more scalable 

than previous systems. Follow-on 

activities going forward will be 

coordinated under the Digital 

Mesh Technology and Applications 

Research Transition team. 

NASA Participation at DARPA 

Assured Autonomy Principal 

Investigator’s Meeting 

POC: NATASHA NEOGI

From Oct. 13-15, NASA SystemWide Safety project researcher 

Natasha Neogi participated in the 

principal investigator’s meeting of 

the Defense Advanced Research 

Projects Agency’s (DARPA’s) 

Assured Autonomy project. As 

a subject matter expert, Neogi 

attended several breakout sessions 

related to scalability issues as well as 

domain specific robustness concerns. 

Assured Autonomy participants 

briefed DARPA and government 

subject matter experts on their 

progress, including efforts targeted 

toward the formal assurance of 

neural networks with rectified linear 

units, runtime monitoring and 

verification approaches for safety 

critical applications, and techniques 

for assessing the safety of Learning 

Enabled Cyber Physical Systems 

(LE-CPS) under uncertainty 

and disturbances. The Assured 

Autonomy project aims to enable 

continual assurance of the learning 

enabled system. Participants seek 

to create guarantees of safety and 

functional correctness for the 

system, provided provisionally at 

design time, and then to continually 

monitor, update, and evaluate the 

system at operation time as the 

system and its environment evolves. 

While the Assured Autonomy 

project research objectives prioritize 

challenge problems in the militarily 

relevant autonomous vehicle space, 

it is anticipated that the tools, 

toolchains, and algorithms created 

will be relevant to other LE-CPS, 

specifically in civil aviation and 

space applications. The website for 

the meeting is: 

https://www.darpa.mil/program/

assured-autonomy 

System-Wide Safety 

Project Kicks Off New 

Research Awards

POC: ALWYN GOODLOE AND MISTY DAVIES

This past quarter the System-Wide 

Safety project held two virtual 

kickoff meetings for new research 

awards. The first one occurred 

from Oct. 4-5 with Geometric 

Data Analytics, Inc. (GDA). 

Approximately 20 people were in 

5
6 AOSP NEWSLETTER // OCT-DEC 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

attendance from GDA, NASA’s 

Langley Research Center in Virginia, 

NASA’s Ames Research Center in 

California, the University of Virginia, 

Collins Aerospace, University of 

Minnesota, Raytheon, the National 

Science Foundation, Air Force 

Office of Scientific Research, and 

the Office of the Secretary of 

Defense. The focus of the award is 

to conduct a study on the potential 

for topological data analysis to 

yield actionable specifications 

from high-dimensional data for 

assuring safety-critical machine 

learning-enabled applications such 

as autonomous aircraft. The talks 

showed how topological tools such as 

persistence diagrams and distributed 

persistence could illuminate the 

shape of high-dimensional data 

used to train and test deep learning 

systems. This analysis can yield 

topological invariants that can 

serve as actionable specifications 

for verifying machine learningenabled systems. Also, techniques 

from topology can help answer 

questions such as why deep neural 

networks produce such good results. 

The GDA team also presented the 

concept of Optimal Uncertainty 

Quantification, which is a method 

for assuring systems as reliable, 

safe, or unsafe in the presence of 

uncertainty and novel input data. 

In addition, several use cases were 

presented. The GDA presentations 

were given by lead investigator Paul 

Bendich, as well as James Polly, Abe 

Smith, and Gabrielle Angeloro. 

Matthew Dwyer of the University 

of Virginia presented his research on 

tools for assuring machine learningenabled systems. A lively discussion 

ensued as the work being carried 

out on this award could greatly 

improve the tools being built in 

Dwyer’s group. Additionally, there 

was lively interaction with Collins 

Aerospace, Raytheon, and University 

of Minnesota researchers, who are 

interested in applying the techniques 

to generate tests of machine learningenabled systems.

The second kickoff meeting 

occurred on Oct. 15 with Galois, 

Inc. The award is for a new runtime 

verification framework tool that 

specifies properties that are to be 

verified at runtime. The framework 

generates C code monitors that are 

integrated with the system under 

observation. The focus of the award 

is to conduct research on applying 

automated tools to produce a 

mathematical proof that the C code 

generated is indeed equivalent to 

its specification. The concept is to 

generate a denotation of both the 

Copilot specification and the C 

code in the mathematical language 

used by Satisfiability Modulo 

Theories (SMT) solvers and use 

SMT-based proof engines to prove 

a bi-simulation argument. The 

general methodology is applicable to 

verifying code generated from many 

embedded domain specific languages. 

System-Wide Safety Project 

Presents to AAL Safety 

Leaders and Executives 

on SMS Improvements 

POC: MISTY DAVIES AND KYLE ELLIS

On Oct. 20 and 22, Kyle Ellis 

and Misty Davies presented 

Safety Management System (SMS) 

improvements to the American 

Airlines (AAL) Operations Data 

Analysis Group (ODAG) and 

Operations Standards Board 

(OSB), respectively. The ODAG is 

the monthly meeting where lead 

safety professionals at the airline 

meet to discuss outcomes and 

identify necessary risk controls 

based on continuous SMS data 

analyses. ODAG’s findings are 

then briefed to airline top safety 

and operations executives across 

multiple departments at the OSB, 

where risk mitigation actions 

to be taken are reviewed. One 

objective of the briefings included 

providing awareness and an 

overview to new AAL leadership 

of the recent developments toward 

a predictive and prognostic safety 

risk dashboard, enabled by machine 

learning-enabled algorithms, 

which are being demonstrated 
AOSP NEWSLETTER // OCT-DEC 2021 

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

on airline Flight Operations 

Quality Assurance and Integrated 

Operations Center operational data.

ATM-X Meets with Aurora 

Flight Sciences to Discuss 

Potential Collaboration 

Opportunities 

POC: KURT SWIERINGA

On Oct. 22, the Air Traffic 

Management-eXploration (ATM-X) 

project met with Aurora Flight 

Sciences, a subsidiary of Boeing, 

to discuss potential collaboration 

opportunities with the Pathfinding 

for Airspace with Autonomous 

Vehicles subproject. The discussions 

included overviews of NASA’s and 

Aurora’s work toward enabling 

the integration of increasingly 

autonomous aircraft into airspace 

shared with conventional aircraft 

to identify potential touch points. 

Future discussions will be used to 

flush out potential collaboration 

responsibilities for a Space Act 

Agreement. 

SWS Project Holds Technical 

Interchange Meeting 

with Longbow Group 

POC: KYLE ELLIS

On Oct. 26, several System-Wide 

Safety (SWS) project researchers 

held a Technical Interchange 

Meeting with Marco Sterk and 

Ken Dudley of Longbow Group. 

Longbow Group is leading the 

ecosystem development of the 

Fort Monroe Unmanned Aircraft 

Systems (UAS) test site located 

in Hampton, Virginia, and is in 

discussions with several regional and 

national industry partners to outline 

several UAS operational use cases. 

The SWS project is in the process 

of identifying key collaboration 

opportunities for joint flight tests 

with Longbow Group and its 

ecosystem partners. An additional 

Space Act Agreement annex 

beyond the initial high density 

Vertiports project annex could be 

needed to address specific research 

goals for the SWS project and its 

Technical Challenges-2 (In-Flight 

Safety Predictions for Emerging 

Operations) and future Safety 

Demonstrator Series flight tests. 

ATM-X PAAV Subproject Briefs 

the NASA/FAA AAM Executive 

Board’s ConOps Subgroup 

POC: KURT SWIERINGA

On Oct. 28, the Air Traffic 

Management-eXploration 

(ATM-X) Pathfinding for Airspace 

with Autonomous Vehicles 

(PAAV) subproject povided an 

overview of their planned work 

at the NASA/FAA Advanced Air 

Mobility executive board’s concept 

of operations subgroup. The 

presentation included discussion 

of the industry’s need, barriers that 

prevent advancement to higher levels 

of autonomy, and the work the 

PAAV subproject is doing to address 

those gaps. 

ATM-X DIP Subproject Attends 

first MR-TBO TIM at the FAA 

Testbed in Daytona Beach 

POC: MIRNA JOHNSON

From Nov. 2-4, the Air Traffic 

Management-eXploration (ATM-X) 

project’s Digital Information 

Platform (DIP) subproject team 

attended the FAA’s first MultiRegional Trajectory-Based 

Operations (MR-TBO) Technical 

Interchange Meeting (TIM) at the 

FAA testbed in Daytona Beach, 

Florida. A large portion of the 

meeting was dedicated to reviewing 

the scenarios needed to begin the 

integration and testing phase of 

the FAA’s 2022 MR-TBO spring 

demonstration. The demonstration’s 

goal is to inform requirements in 

a closed-loop system to identify 

challenges and gaps. The scenarios 

reviewed covered protocols 

and procedures for requesting, 

approving, and acknowledging trial 

plan requests and flight trajectory 

updates between multiple air traffic 

management service providers. The 

operations span pre- and postdeparture scenarios, with multiple 

7
8 AOSP NEWSLETTER // OCT-DEC 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

flights as the subject in each of 

the scenarios. The organizations 

collaborating in MR-TBO are 

looking to understand how to 

update and communicate a 

trajectory plan that everyone is 

working toward. The FAA and 

their partners will continue to work 

on the concept and risk reduction 

activities through FY24. The 

ATM-X project is collaborating 

with the FAA to identify valuable 

services and scenarios to further 

validate MR-TBO scope and 

procedures. By attending the 

TIM, the DIP team was able to 

gain insight into the benefits of 

MR-TBO and the challenges that 

remain to be addressed. Feedback 

from the TIM is being used as input 

to formulate concepts and select 

features for a flight deck service that 

would be valuable to demonstrate 

in FY25 for DIP’s Sustainable 

Aviation-2 demonstration. 

ATM-X PAAV Attends the 

Multi-Vehicle Working 

Group Meeting 

POC: ARWA AWEISS

On Nov. 3, the Air Traffic 

Management-eXploration (ATM-X) 

project’s Pathfinding for Airspace 

with Autonomous Vehicles (PAAV) 

subproject attended the MultiVehicle (m:N) Working Group 

meeting. The Working Group 

brings together stakeholders from 

government, industry, and academia 

to identify and reduce barriers to 

m:N operations, an operational 

configuration that envisions a 

ratio of multiple operators (m) 

controlling multiple vehicles 

(N) between them. The barriers 

addressed by the Working Group 

range from technical, regulatory, 

safety assurance, community 

acceptance, and others. Identified 

barriers are considered across a 

variety of multi-vehicle control 

contexts (e.g., Urban/Advanced 

Air Mobility, drone delivery, 

infrastructure inspection, disaster 

response and recovery, and highaltitude pseudo-satellite operations) 

and form the bases for future 

research to confront operational, 

technical, and regulatory gaps. 

The Working Group ultimately 

plans to work with regulators to 

identify existing regulations and 

determine what needs to be done 

to accommodate m:N operations in 

the future. The meeting commenced 

with a Working Group report 

followed by breakout sessions to 

discuss m:N high-priority issues 

and Detect and Avoid use cases for 

the Radio Technical Commission 

for Aeronautics. 

ATM-X DIP Workshop on 

Architecture and Data 

Integration Services 

POC: MIRNA JOHNSON

On Nov. 17, the Air Traffic 

Management-eXploration (ATM-X) 

project’s Digital Information 

Platform (DIP) subproject held 

their first in a series of workshops 

intended to provide a deeper dive 

into the technical concepts and 

core features of the DIP concept. 

This session focused on the 

platform’s functional architecture 

and the data integration services 

that are being developed and 

will be demonstrated in DIP’s 

first collaborative demonstration, 

scheduled for mid-July 2022. The 

objective was to inform potential 

collaborators of how registered 

services would search for, and 

discover, data integration and 

aviation services on the platform. 

Participants were walked through 

NASA’s reference implementation 

of a data integration service that 

provides fused data available for 

service providers to consume. The 

virtual workshop hosted more than 

150 attendees, spanning federal and 

state government agencies, airport 

authorities, and industry solution 

providers, enabling traditional and 

emerging operations. The attendees 

participated in live dialogues 

with the panel, providing insight 

and feedback on how their use 

cases, needs, and challenges can 

be addressed by DIP. This was a 

beneficial forum to engage with 

stakeholders to further inform them 

about the DIP architecture and 

raise interest to collaborate in the 

partner services demonstrations. 
AOSP NEWSLETTER // OCT-DEC 2021 9

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

GPS Prediction Application 

Video Production 

POC: EVAN DILL AND MISTY DAVIES

The System-Wide Safety (SWS) 

project communications team 

produced a two-minute video 

showcasing the work of SWS 

researchers on a Global Positioning 

System (GPS) prediction application. 

The video features Natasha Neogi, 

Evan Dill, and Julian Gutierrez 

explaining how their research 

will positively impact the lives 

of the general public in many 

ways by predicting the quality of 

GPS anytime, anywhere. This is 

particularly important for the future 

safety of autonomous vehicles. 

GPS is a critical component in 

banking systems, our power grid, 

telecommunications, and even traffic 

lights. Determining and predicting 

the GPS quality of entire cities can be 

critical in life threatening situations 

and can increase safety and save on 

resources in everyday life. The link for 

the YouTube video, uploaded on Nov. 

22, is: https://www.youtube.com/

watch?v=HxLD302nQHg

NASA-Developed Algorithms 

Identify Previously 

Unknown Safety Issues 

POC: NIKUNJ OZA

The System-Wide Safety project 

supports research and development 

of machine learning methods for 

vulnerability discovery, which is 

the process of finding safety issues 

that have not been codified in the 

exceedances (sets of rules) that 

represent known safety issues. These 

research and development efforts have 

led to the development of several new 

algorithms, including the Multiple 

Kernel Anomaly Detection (MKAD) 

algorithm. These algorithms have 

found several previously unknown 

safety issues. They have also identified 

anomalies that represent allowable 

operations that have not been codified 

into standard operating procedures. 

One example of an MKAD-identified 

anomaly includes unusual landing 

trajectories at John F. Kennedy 

International Airport in New York 

that were deemed acceptable because 

they represented cargo flights landing 

at night with no other traffic around 

and had a need to land closer to their 

cargo facilities. A second example was 

a drop in airspeed during takeoff that 

seems to happen more often in longer 

aircraft where throttles are rapidly 

increased during takeoff to reduce the 

probability of tail strike, but then the 

throttle is reduced to return to the 

appropriate range of airspeed. These 

are important discoveries for the 

airline industry that are made by the 

algorithms developed at NASA. 

SWS Presents IASMS 

Concept at USRTC Meeting 

POC: KYLE ELLIS

On Dec. 1, at the Unmanned 

Systems Research and Technology 

Center (USRTC) at Fort Monroe 

in Hampton, Virginia, the SystemWide Safety (SWS) project 

presented its In-time Aviation Safety 

Management System (IASMS) 

concept research and development 

efforts toward technology solutions 

to enable safe integration of new 

entrants into the National Airspace 

System. Attending the meeting 

were Steve Bradford, Diana Liang, 

Biruk Abraham, Nourredin Ghazavi, 

and T.J. Tejasen from the FAA’s 

NextGen Office; Marco Sterk and 

key members of the Longbow 

Group; Hampton, Virginia Mayor 

Donnie Tuck; leaders of Hampton 

University; as well as Lou Glaab 

from NASA’s Advanced Air 

Mobility project’s High Density 

Vertiplex subproject. This high-level 

engagement represents a highpotential collaboration ecosystem 

for NASA to engage in key research 

opportunities to mature its 

operational services and capabilities. 

SWS Project Researchers 

Participate in SAE 

Standards Meetings 

POC: MISTY DAVIES

This past quarter, System-Wide 

Safety (SWS) project researchers 

engaged with the SAE G-34 

Standards Committee at several 

meetings. From Oct. 28-29, 

researchers met with the full SAE 

G-34 committee, which is tasked 

to create standards that will enable 
10 AOSP NEWSLETTER // OCT-DEC 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

the use of machine learning and 

artificial intelligence in aviation. The 

meeting was to discuss key concerns 

that need further consensus. These 

included the creation of criticality 

and complexity levels, how to 

address human-machine teaming 

within the standards, and tool 

qualification. There is significant 

overlap in issues being considered 

by the G-34 industry standards 

committee and applied research and 

evidence that is being generated 

under the SWS project umbrella. At 

a follow-on meeting on Dec. 2, SWS 

researchers again participated in the 

SAE G-34 committee on Artificial 

Intelligence in Aviation meeting. 

SWS researchers are actively 

contributing broadly across the G-34 

subcommittees, serving as subject 

matter experts for the concerns 

inherent to the use of machine 

learning and artificial intelligence 

in aviation. SWS project researchers 

are also introducing the committee 

to recently developed frameworks 

and techniques for assuring these 

capabilities and providing expertise 

on these techniques’ strengths 

and weaknesses. 

ATM-X DIP Technical Reviews

POC: MIRNA JOHNSON

On Dec. 2, the Air Traffic Management-eXploration (ATM-X) project’s 

Digital Information Platform (DIP) 

subproject held two internal technical reviews. These included the 

platform Research Concept Review 

(RCR) and the Sustainable Aviation 

(SA)-1 Preliminary Experiment/

Demo Review (PER). The DIP 

subproject is focused on designing, 

developing, and demonstrating a 

platform that supports an ecosystem 

of digital services to enable improved 

flight operator decision-making. The 

platform RCR’s purpose is to ensure 

alignment with the ATM-X project’s 

research objectives. The SA-1 demonstration will validate the platform 

by demonstrating the ability of traditional flight operators to access and 

use real-time data-driven services 

for Collaborative Digital Departure 

Rerouting (CDDR). The purpose 

of the SA-1 PER is to ensure the 

technical quality of the SA-1 demonstration, which aims to deliver 

a reduction in emissions and fuel 

associated with aviation operations. 

Data collection is scheduled to begin 

in April 2022. Requirements for 

the reviews were consistent with the 

entrance and exit criteria adapted 

from NPR 7123.1B, Appendix G. 

Reviewers have since submitted feedback and the DIP subproject team 

will disposition the action items and 

develop a timeline to resolve them 

by the next set of reviews. The PER 

of Platform in Partner Services Build 

Up Demo 1 is currently scheduled 

in March 2022 and the SA-1 CDDR 

Demonstration for Final Experiment 

Review is scheduled in April 2022. 

NASA and FAA Conduct 

Quarterly Review at NASA’s 

Ames Research Center in 

California 

POC: MIKE MADSON

The Air Traffic ManagementeXploration (ATM-X) project held 

a quarterly meeting with the FAA 

at NASA’s Ames Research Center 

in California from Dec. 6-8. Topics of discussion were selected from 

the ATM-X research areas of common interest. NASA and the FAA 

shared their plans and activities 

in the areas of Extensible Traffic 

Management, increasing the use of 

unmanned aircraft in the National 

Airspace System (NAS), collaborative demonstration of airspace technologies with commercial airlines 

in the DIP subproject, and shared 

the web portal to solicit industry 

feedback and describe a vision for a 

future NAS under the ATM-X Sky 

for All activity. Dec. 8 focused on 

Unmanned Aircraft Systems (UAS) 

Traffic Management (UTM). NASA 

and the FAA discussed approaches 

to advance the state of the art of 

UTM, including NASA/FAA UTM 

Research Transition Team activities. 

The FAA provided updates about 

information exchange for UAS 

remote identification, UAS service 
AOSP NEWSLETTER // OCT-DEC 2021 11

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

supplier standards, and Beyond 

Visual Line of Sight Aviation 

Rulemaking Committee activities, as well as objectives for a 2022 

FAA-led UTM field demonstration. 

NASA briefed the FAA about its 

Federated Airspace Management 

development capability, also known 

as Freddie, and the FAA talked 

about their experience with using 

the Freddie capability for the UTM 

evaluation work. Both sides reaffirmed their commitment to the 

UTM work and agreed to continue 

the conversation about potential 

joint activities. The FAA group 

ended the day with a tour of the 

Airspace Operations Lab. There 

were positive actions from the discussions to conduct several followon Technical Interchange Meetings 

based on these topics prior to the 

next quarterly meeting, currently 

scheduled at the end of March 

2022. This will enable deeper technical discussions among the various 

teams to strengthen collaboration. 

ATM-X DIP Meets with Airlines 

for America 

POC: MIRNA JOHNSON

On Dec. 16, the Air Traffic Management-eXploration (ATM-X) project’s 

Digital Information Platform (DIP) 

subproject presented a deep dive of 

technical content to the America 

for Airlines Air Traffic Management 

Council. The discussion included 

content previously shared with 

external stakeholders on Nov. 17, as 

part of the DIP Architecture and 

Data Integration Services Workshop 

#1. The intent of the meeting was 

to gather direct input from flight 

operators on the functional architecture of DIP and the reference 

implementation of data integration 

services (DIS) to be demonstrated in 

DIP’s first collaborative demonstration in FY22. Key questions such as 

the pros and cons of a unique DIS 

or competitive market of DIS were 

also discussed to help inform the 

architecture of DIP moving forward. 

ATM-X Project X4 Partners 

Conduct Connectivity Tests

POC: KEVIN WITZBERGER

This past quarter, the Air Traffic 

Management-eXploration (ATM-X) 

project’s Urban Air Mobility (UAM) 

subproject began conducting tests 

with its X4 simulation partners. On 

Oct. 1, two ATM-X project X4 

airspace partners, OneSky and Unmanned Experts, conducted initial 

connectivity tests of their provider of 

services for UAM (PSUs) to a NASA 

prototype of an FAA-Industry Data 

Exchange Protocol (FIDXP) system 

developed for X4 simulations. The 

PSUs connected successfully to the 

FIDXP and sent requests for tokens 

with their encrypted signatures. After 

FIDXP authenticated their signatures, 

authorization tokens were returned 

to their PSUs. With these tokens, 

their PSUs can then submit requests 

to get airspace data (e.g., corridors, 

waypoints, tracks, etc.) from the 

Airspace Structure Definition Service 

(ASDS). The remaining partners will 

schedule connectivity tests in the 

upcoming weeks. This was a critical 

first step toward achieving the X4 

simulation objective of demonstrating successful information transfer 

between UAM airspace PSUs for 

strategic conflict management. 

On Oct. 28, the UAM team successfully completed additional connectivity tests with all seven Strategic 

Conflict Management Simulation X4 

airspace partners. The seven partners and NASA conducted initial 

connectivity tests of their PSUs to a 

prototype FIDXP system developed 

for the X4 simulation. They were all 

able to send and receive airspace data 

to and from the ASDS. This was the 

first sprint test in a series of X4 tests. 

The series of sprints within the scenarios were used to validate features 

that will be applied in a collaborative scenario simulation with the X4 

partners that will run from March to 

June 2022. 

Additional tests occurred on Nov. 

19 and Dec. 17, in which the team 

again successfully completed further 
12 AOSP NEWSLETTER // OCT-DEC 2021

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

steps toward National Campaign-1 

(NC-1). Six of the seven partners 

(ANRA, Avision, Metron/Airbus, 

OneSky, SkyGrid, and Unmanned 

Experts) and NASA successfully 

conducted a Nominal Flight Test 

Simulation. Participants were able to 

conduct vehicle simulation, telemetry updates, conformance monitoring, and operation close-out during 

the test. These were the second and 

third Sprint tests in a series of X4 

simulation tests aimed at preparing 

the airspace partners and the NASA 

PSU for inclusion in NC-1. Sprint 

#4 will exercise a FIDXP Airspace 

Authorization scenario with the window opening for testing on Jan. 17, 

2022, and an expected end date of 

Feb. 4, 2022. Based on the outcome 

of Sprint #4, an assessment will be 

made on whether the participants 

have demonstrated sufficient capabilities to continue with NC-1. The 

series of sprints within the scenarios 

are used to validate features that will 

be used in a collaborative scenario 

simulation with the partners that will 

run between March and June 2022. 

SWS Partner Completes Large 

Set of Tests on Small UAS 

Batteries and Motors 

POC: STEVE YOUNG

In Dec. 2021, the Lone Star Unmanned Aircraft System (UAS) 

Center of Excellence and Innovation, a partner of the System-Wide 

Safety (SWS) project, located in 

Texas, completed a large set of tests 

for SWS on small UAS batteries and 

motors. Over the course of seven 

weeks, testing included 168 charging 

cycles, 146 use cycles, approximately 

90 hours of battery use time, and 

over 310 hours of charging time on 

12 separate batteries. Motor performance was recorded during the use 

cycles and included both indoor 

bench tests and outdoor flights (in 

a caged/netted area). The collected 

data is helping with SWS modeling 

work and our understanding on how 

these types of batteries and motors 

can fail. 

SWS Collaborates on Two 

Advanced Air Mobility RF 

Spectrum Scanning Activities 

Executive Summary 

POC: STEVE YOUNG

The System-Wide Safety (SWS) project has recently begun collaborating 

with the Advanced Air Mobility project’s High Density Vertiplex (HDV) 

subproject on two radio frequency 

(RF) spectrum scanning activities. 

The first activity is collecting nearcontinuous scans over an extended 

period (Nov. 19, 2021-March 1, 

2022) to baseline the RF environment across the City Environment 

for Range Testing of Autonomous 

Integrated Navigation (CERTAIN) 

test range for selected frequencies of 

interest. Results will be analyzed and 

used to help the HDV subproject 

obtain approvals for new types of operations. The second activity will establish a real-time feed of data from 

SWS spectrum monitoring stations 

to operators and users who will be 

using the Remote Operations for Autonomous Missions Ground Control 

Center that is being setup for HDV 

demos at NASA’s Langley Research 

Center in Virginia. Although just 

underway, both efforts are going well 

and should provide mutual benefits. 

SWS Completes Work on Run 

Time Assurance for eVTOL 

Aircraft 

POC: AARON DUTLE

In Dec. 2021, the System-Wide 

Safety (SWS) project completed an 

analysis and evaluation of run-time 

assurance for assuring the safety of 

aircraft functions and systems for 

a novel electric vertical takeoff and 

landing (eVTOL) aircraft architecture. The analysis showed that the 

inclusion of a run-time architecture 

aided in the demonstration of safety 

assurance for aviation systems that 

contain advanced control techniques, 

such as adaptive control, machine 

learning, artificial intelligence, and 

search-based algorithms. The results 

of this research will be shared during 

a meeting of the SWS FAA/NASA 
AOSP NEWSLETTER // OCT-DEC 2021 13

TECHNICAL AND PROGRAMMATIC HIGHLIGHTS

Research Transition Team meeting 

on Jan. 20, 2022. 

NASA’s SWS Completes Major 

Milestone for Increasingly 

Autonomous Systems 

POC: MISTY DAVIES

In Dec. 2021, the System-Wide 

Safety (SWS) project completed a 

major milestone in which NASA 

evaluated the usefulness and efficacy of assurance approaches for 

increasingly autonomous systems. To 

make sure that the approaches were 

generalizable across different levels 

of autonomy, use cases included 

human-machine teaming, an automated planning application, and 

runtime assurance for electric vertical 

takeoff and landing aircraft. Several 

industry and academic partners, 

including Collins Aerospace, Penn 

State, Raytheon, and Pratt and Whitney, helped NASA gather evidence 

for the assurance approaches. Overall, 

the results were very promising and 

demonstrated there are approaches 

that can give a higher level of assurance and enable levels of automation 

and autonomy beyond what is flown 

today. As part of the research, NASA 

and its industry partners also identified remaining gaps and next steps. 
14 AOSP NEWSLETTER // OCT-DEC 2021

RECOGNITION

SWS Participates at 

AIAA/IEEE DASC 2021

POC: TERRY MORRIS

From Oct. 3-7, numerous SWS 

project managers, leaders, and 

personnel participated in the 

40th AIAA/IEEE Digital Avionics 

Systems Conference (DASC) held 

in San Antonio, Texas. This was 

the first ever hybrid DASC where 

presenters and attendees could 

participate in person or virtually. 

SWS personnel participated at 

every level of the conference, from 

organizing the technical program, to 

selecting the keynote speakers, to 

serving as session chairs, authors, 

and presenters. The impact of SWS 

personnel was even seen in the 

acknowledgements where SWS 

leaders were thanked for their 

contributions to various author 

papers. From the Program, Track 

and Session organization side 

of the conference, Terry Morris 

served as the technical program 

chair. Steve Young served on the 

Keynote and Panel Committee 

and was instrumental in inviting 

NASA’s Aeronautics Research 

Mission Directorate Associate 

Administrator Bob Pearce as one 

of the keynote speakers. Evan 

Dill, Natasha Neogi, and Mahyar 

Malekpour did an incredible job 

as session chairs. Various SWS 

leaders and personnel contributed 

greatly as authors and presenters. At 

least eight papers were submitted 

regarding SWS-funded efforts, 

and at least three of those papers 

received “Best of Session” awards. 

SWS Researcher Presents 

Continuous Learning at SPS 

National Learning Session 

POC: JON HOLBROOK AND MISTY DAVIES

From Oct. 20-21, Jon Holbrook 

was invited to virtually give the 

keynote address at the Solutions for 

Patient Safety (SPS) Fall National 

Learning Session. SPS represents 

a national network of more than 

145 children’s hospitals, and more 

than 1,200 safety leaders from 

across the network attended the Fall 

Learning Session. The objectives 

of the meeting were to examine 

best practices for promoting safety 

and safety culture for hospital 

patients and employees, and to 

identify new and emerging safety 

efforts. Holbrook was invited 

to present based on his work on 

human contributions to safety in the 

System-Wide Safety (SWS) project. 

His presentation, “Cultivating a 

Culture of Continuous Learning: 

Insights and Examples from the 

Aerospace Industry,” described 

insights and examples from 

emerging safety efforts in the 

aerospace industry focused on 

learning not only from infrequent 

failures, but also from everyday 

events that are usually successful. 

SWS Researchers Present at 

Aviation Safety InfoShare on 

Expanding Our Safety Mindset 

POC: JON HOLBROOK AND CASSIE HILDITCH

From Nov. 2-4, System-Wide 

Safety (SWS) project researcher Jon 

Holbrook provided a presentation 

at the Aviation Safety InfoShare 

Conference in Pittsburgh, 

Pennsylvania, and described a safety 

mindset that expands the current 

understanding of what constitutes a 

safety-relevant event and improves 

the ability to learn from everyday 

work. The presentation offered 

practical suggestions, specific 

examples, and lessons learned based 

on a recent paper coordinated 

by the Flight Safety Foundation 

that included contributions from 

organizations across the aviation 

industry. Holbrook’s contributions 

to this paper and presentation 

builds on his work on human 

contributions to safety in the SWS 

project. Aviation Safety InfoShare 

is attended by operational safety 

personnel from around the world for 

the purpose of advancing aviation 

safety learning through sharing safety 

issues and mitigation strategies. In 

addition, Cassie Hilditch presented 

findings from a survey conducted on 

assessing the impact of COVID-19 

on pilot sleep and fatigue. This was 

an important opportunity to share 

results with the aviation industry and 

to garner information on developing 
AOSP NEWSLETTER // OCT-DEC 2021 15

RECOGNITION

fatigue challenges so that the SWS 

project can best support airline 

safety as the industry returns to 

full service. Hilditch’s presentation 

was titled “How is the Pandemic 

Affecting Pilot Fatigue? A Survey.” 

SWS Associate Project 

Manager Presented at 

2021 BAM Forum 

POC: WENDY OKOLO

On Nov. 3, System-Wide Safety 

(SWS) project Associate Project 

Manager Wendy Okolo presented 

“UAM in the NAS: Safety Above All” 

at the 2021 Business of Automated 

Mobility (BAM) Forum: Flight 

Path to Urban Air Mobility (UAM)/

Advanced Air Mobility (AAM). The 

virtual presentation, watched by 

630 forum attendees, provided an 

overview of SWS research and its 

overall mission. Okolo explained 

the ongoing work to develop 

the In-time System-wide Safety 

Assurance services, functions, and 

capabilities that form the building 

blocks of an envisioned In-time 

Aviation Safety Monitoring System. 

The presentation concluded with 

a look to potential future safety 

implications as the use of the 

National Airspace System expands 

with UAM and AAM. The 2021 

BAM Forum was a two-day, onlineonly event that featured topics of 

interest to industry and government 

leaders, including unmanned 

aircraft system traffic management, 

safety and security, and remote 

pilot technology and autonomy. 

SWS Participates in 

Prognostics and Health 

Management Conference 

POC: CHETAN KULKARNI, CHRIS TEUBERT

AND WENDY OKOLO

From Nov. 29-Dec. 2, System-Wide 

Safety (SWS) project researchers 

and management participated 

in the 13th Annual Conference 

of the Prognostics and Health 

Management Society. This event 

was fully virtual and offered speaker 

and panel sessions, keynote and 

luminary speakers, tutorials, and 

additional events to present the 

state-of-the-art and other advances 

in aerospace, transportation, 

human health and performance, 

and artificial intelligence. SWS 

project researcher Chetan Kulkarni 

served as the Technical Program 

Committee Chair and other 

researchers presented papers, 

led sessions, and gave tutorials 

on prognostics algorithms, 

recent developments, and tools. 

Diagnostics and Prognostics group 

lead Chris Teubert presented a 

technical tutorial, “A Guide to 

the NASA Python Prognostics 

Package,” which gave a hands-on 

overview of the recently released 

Prognostics Python Packages. The 

tutorial familiarized participants 

with a collection of research tools 

for developing prognostic models, 

simulating degradation of systems, 

performing prognostics, analyzing 

results, and developing new 

prognostic algorithms. Teubert also 

provided examples of how the tools 

are used at NASA for SWS and 

other projects and can be extended 

to other applications in the future. 

SWS Attends Invited 

Talk: Innovation in LowAltitude Navigation 

POC: ANDREW MOORE AND EVAN DILL

System-Wide Safety (SWS) project 

researchers were invited to present 

NASA’s innovation in low-altitude 

navigation, the first public 

presentation of the research, to a 

group of 75 aviators from large U.S. 

electric companies at the Electric 

Power Research Institute’s Annual 

Unmanned Aerial Vehicle (UAV) 

Workshop on Nov. 30. Before this 

effort, Global Positioning System 

(GPS) navigation loss due to foliage 

attenuation, a common concern 

for this audience, had never been 

quantified. This work demonstrates 

the severity of GPS loss in 

forested corridors. In addition to 

infrastructure inspection flights, 

foliage-degraded navigation impacts 

UAV storm recovery, property 

survey, and search and rescue 

operations. UAV inspection of 

infrastructure, such as pipelines 
16 AOSP NEWSLETTER // OCT-DEC 2021

RECOGNITION

and transmission lines, frequently 

requires flights within “arboreal 

canyons” (i.e., swaths of cleared 

ground surrounded by trees and 

brush). While foliage attenuation 

of conventional radio signals has 

been well studied, this research is 

the first to quantify degradation of 

GPS signals by foliage. Attenuation 

of radio signals varies by foliage 

species and, if a flight path courses 

through a species that has not 

been characterized, a survey is 

required to accurately estimate the 

severity of GPS degradation. NASA 

researchers taught the aviators about 

the hardware and analysis needed 

to survey their flight corridors and 

were directed to review the SWS 

project-supported AIAA paper, 

“Volume Raycasting of Global 

Navigation Satellite System (GNSS) 

Signals Through Ground Structure 

Lidar for UAV Navigational 

Guidance and Safety Estimation,” 

by Andrew Moore, Matthew 

Schubert, Nick Rymer, Daniel 

Villalobos, J. Sloan Glover, Derin 

Ozturk, and Evan T. Dill. The full 

paper, which can be accessed at 

https://doi.org/10.2514/6.2022-

2218, describes the survey method 

details. A steam pipeline at a 

NASA facility was studied for the 

flight path, and 60 percent of the 

GPS signal was lost for 10 meters 

of foliage thickness between the 

receiver and the satellite. The 

signal was completely knocked 

out (>90 percent loss) when the 

thickness was 30 meters. The 

full study was first published 

at AIAA SciTech 2022. 

SWS Presents IASMS 

Concept to the CAAS 

POC: KYLE ELLIS

On Dec. 7, the Civil Aviation 

Authority of Singapore (CAAS) 

invited Kyle Ellis to present NASA’s 

visionary concept and detailed 

implementation plans for the InTime Aviation Safety Management 

System (IASMS). The international 

community views the technological 

developments and conceptual 

framework being delivered by 

NASA’s System-Wide Safety (SWS) 

project as critical to enabling 

existing and advanced air mobility 

aviation sectors to safely integrate in 

a transformed global airspace. The 

presentation outlined a key roadmap 

of technical challenges and solutions 

to achieve a safely transformed 

airspace system that includes novel 

markets and modes of transportation. 

Partnering in the presentation was 

Andrew Carter of ResilienX, a key 

industry collaborator building out 

industry developed implementations 

of the IASMS services and 

capabilities at key sites such as the 

NUAST, Fly Ohio, and Longbow 

Group Unmanned Systems Research 

and Technology Center in Hampton, 

Virginia, among others. This work 

supports the NASA Aeronautics 

Research Mission Directorate’s 

Strategic Thrust 5 research and 

development activities performed 

within AOSP. 

Advanced Air Mobility 

Academy Series for 

STEM Engagement 

POC: KURT SWIERINGA

On Dec. 8, NASA’s Air Traffic 

Management-eXploration (ATM-X) 

project supported an educational 

event, hosted by the Advanced Air 

Mobility Academy Series for Science, 

Technology, Engineering, and 

Mathematics (STEM) Engagement. 

The purpose of the event is to 

inspire the next generation of air 

traffic management researchers. The 

event also celebrated Hour of Code, 

which took place during Computer 

Science Education Week, from 

Dec. 6-12, which is an annual event 

meant to encourage K-12 students 

to explore coding. The event began 

with a presentation that provided 

middle and high school students 

with an overview of Advanced 

Air Mobility and the research 

that ATM-X is doing to enable 

the future air traffic management 

system. The presentation was 

followed by a coding exercise where 

students created geofences around 

high-risk areas to enable a small 

unmanned aircraft to operate safely. 
National Aeronautics and Space Administration

Headquarters

300 E. Street, SW

Washington, DC 20024

www.nasa.gov/aeroresearch

www.nasa.gov