Flight Demonstration of Unmanned

Aircraft System (UAS) Traffic Management

(UTM) at Technical Capability Level 4

Joseph Rios , Arwa Aweiss , Jaewoo Jung , Jeffrey Homola , Marcus Johnson , Ronald Johnson 1 2 3 4 5 6

NASA Ames Research Center, Moffett Field, CA, 94035, USA

The NASA Unmanned Aircraft Systems (UAS) Traffic Management (UTM) Project

executed the fourth and final UTM Technical Capability Level demonstration between May

and August 2019. Two Federal Aviation Administration (FAA)-designated UAS test sites

managed the range, partners, and operations to meet the requirements set forth by the UTM

Project. All stakeholders supported the execution of the flight testing through close

collaboration. Results of the demonstration indicate the viability of the UTM concept to

manage large scale operations and contingencies in an urban environment. The

demonstration also provided insight into key technological gaps that must be addressed

before such operations are routine, safe, and efficient. Standardization efforts related to

UTM and the industry participants of those efforts can leverage the results and experiences

of this flight activity to accelerate and more firmly ground forthcoming standards. The FAA

and other regulators will be able to leverage results to inform future rule-making and

identify additional gaps that require further analysis.

I. Introduction

UTM is a novel approach to managing the airspace for small UAS (sUAS) operations. In the United States,

UTM is envisioned as “the manner in which the FAA will support operations for sUAS operating in low altitude

airspace [1].” UTM is novel in the sense that it does not rely on a centralized system to provide management

services, as they have been traditionally provided for aviation. “UTM utilizes industry’s ability to supply services

under the FAA’s regulatory authority where these services do not currently exist. It is a community-based traffic

management system, where the Operators and entities providing operation support services are responsible for the

coordination, execution, and management of operations, with rules of the road established by FAA [1].”

UTM is both a federated set of services and a framework for managing low-altitude sUAS operations. It is

part of the National Airspace System and interoperable where necessary with traditional Air Traffic Management

(ATM) and Air Traffic Control (ATC). UTM was designed to meet the needs of several stakeholders including the

operators for sUAS and the FAA. The UTM ecosystem can offer services for flight planning, communications,

separation, and weather, among others. This novel approach to managing air traffic leverages UAS Service Suppliers

(USS) to interoperate and support operators in sharing their intents, strategically deconflicting, providing appropriate

airspace updates, and other key functions. For more details on the UTM concept of operations, see Ref. [1] and Ref.

[2].

1 Research Aerospace Engineer, Aviation Systems Division, Mail Stop 210-10, Senior Member. 2 Aerospace Engineer, Aviation Systems Division, Mail Stop 210-10, Senior Member. 3 Aerospace Technologist, Aviation Systems Division, Mail Stop 210-6, Senior Member. 4 Aerospace Technologist, Human Systems Integration Division, Mail Stop 262-4. 5 Research Aerospace Engineer, Aviation Systems Division, Mail Stop 210-10, Member. 6 Aerospace Engineer, Aeronautics Research Directorate, Mail Stop 210-3, Senior Member

Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

 AIAA AVIATION 2020 FORUM 

 June 15-19, 2020, VIRTUAL EVENT 

 10.2514/6.2020-2851 

 This material is declared a work of the U.S. Government and is not subject to copyright protection in the United States. 

 

 

 

 

 

 

 

 

 

 AIAA AVIATION Forum 
 

 

 

 

 

 

 

 

Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

A series of four Technical Capability Levels (TCLs) were defined by NASA to develop and test the increasing

capabilities of UTM needed to manage more complex UAS operations. NASA successfully progressed through

several phases of flight testing, with TCL4 being the final such demonstration. The major events are summarized in

Figure 1. The tests were performed at different test locations, including the FAA-designated UAS test sites.

Figure 1. Summary of major NASA UTM flight testing activities.

The final TCL, TCL4, demonstrated sUAS operations with the following features as defined by the NASA

UTM concept of operations [2], supported by the UTM System:

1. Beyond-visual-line-of-sight operations

2. Urban environments, higher densities

3. Autonomous vehicle-to-vehicle (V2V) capabilities, Internet-connected systems

4. Large-scale contingency mitigation

The TCL4 environment and capabilities were planned to support use cases for news gathering, deliveries, and

personal use. These capabilities and use cases were additive from the previous TCLs and thus encompass

interactions with manned aviation, individual operation contingency management, vehicle tracking, and public

safety operations, amongst others. A goal of TCL4 was to investigate system responses to real-world events related

to these use cases. For example, UAS-to-UAS and UAS-to-manned aircraft encounters, weather events, and

emergency priority operations.

II. Approach

To engage the FAA-designated test sites in the TCL4 testing, six of the FAA-designated UAS test sites were

provided a Statement of Work (SOW) to which they provided proposals. The SOW was written with detailed

scenarios and technical requirements. A lesson learned by the UTM Project and the test sites from previous UTM

flight tests (specifically TCL1, National Campaign I [3], TCL2 [4, 5], National Campaign II [6], and TCL3 [7, 8])

was that too much flexibility in the scenario requirements led to inadequate coverage of Project goals and

discrepancies in trying to compare results across test sites. Thus for TCL4, five specific scenarios with key

characteristics and test events, including a mix of live and simulated operations, were provided as part of the SOW.

NASA evaluated the SOW responses and selected the Nevada Institute for Autonomous Systems (NIAS) and the

Lone Star UAS Center of Excellence & Innovation (LSUASC) located in Nevada and Texas, respectively, to

execute the TCL4 Demonstration. The test sites were responsible for forming their team of industry, government,
 

 

 

 

 

 

 

and community partners, developing test plans per the SOW, and ultimately executing the flight tests to provide the

required data to NASA. In total there were 27 unique industry and city partners across the two sites.

Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

A. Scenarios

The primary research goals of TCL4 were to collect data to understand the requirements regarding safely

enabling large-scale beyond visual-line-of-sight (BVLOS) sUAS operations in urban environments; evaluate the

small UAS Detect and Avoid, Communications, Navigation, and Remote ID technologies available at the time of the

demonstration; evaluate UTM services being developed to address technical and operational challenges of urban

operations; and identify gaps in the capabilities of current technology which would be needed to enable urban

operations. The TCL4 demonstration was scenario-driven and integrated different research objectives into each

scenario. The five scenarios were created and designed by NASA to represent a “day in the life of UAS operations”

in a geographic region and focused on the research challenges associated within various UTM focus areas.

Scenario 1

In an urban environment, nominal high density UTM operations are taking place with multiple mission types

and use cases being carried out. A simulated weather event is forecast to impact the area, which results in a warning

region being established in the form of a UAS Volume Reservation (UVR). Operators with current or planned

operations within the UVR geometry respond by returning to the launch point, re-planning to utilize an identified

safe landing location that is de-conflicted from other operations, or avoiding takeoff from within the UVR.

Scenario 2

A pop-up concert takes place at a local venue that results in a number of UTM operations in the area

supporting a variety of use cases reaching medium to high density traffic levels. An incident, like a fire or a medical

emergency, occurs at the concert resulting in a UVR being established to allow for a Public Safety UTM response

and clearance of non-essential operations from the area. Those operations clearing the UVR must replan and

de-conflict to ensure safe exit. UTM-enabled Remote ID is used to identify and contact operators that have not

cleared the area. Special access to the UVR is granted for news coverage in addition to the public safety vehicles.

Scenario 3

UTM operations are being conducted in the vicinity of an active airport with a medium density of sUAS

operations. Reported events take place that require Remote ID of specific operations as well as security responses to

monitor situations near the airport. Piloted general aviation aircraft are conducting flights in the area and one

aircraft’s path results in a conflict with a UTM operation’s operation volume with a subsequent response from the

UAS operator. Another operation experiences a loss of communication that results in the transition to a Rogue state,

communicated to nearby operations with resulting maneuvers to avoid.

Scenario 4

High density UTM operations are taking place in an urban environment. A simulated low battery situation

forces a vehicle to land quickly, which requires nearby operations to re-plan and avoid the landing vehicle. Later,

large-scale loss of communication and navigation events are experienced that require contingency management

procedures in response. Remote ID is requested to identify vehicles that are reported to be congregating in a

particular area during the events.
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

Scenario 5

Multiple events are taking place in a suburban area, which draws a gradually increasing number of sUAS

performing a variety of supporting missions. The increase in density is accommodated through strategic

deconfliction that eventually requires UAS Service Supplier (USS) negotiations for re-planning or agreement to

allow overlapping Operation Volumes. Operational density in one area increases with subsequent negotiations that

result in shared airspace for multiple vehicles and a transition to cooperative separation. One USS supporting

multiple operations experiences a critical failure, which results in contingency management procedures for affected

flights and a switch to an alternate USS where able.

B. Test characteristics

Test Characteristics (CH) were used to help define the flight environment in which operations were to be

conducted, and the types of flight profiles needed to fulfil a given scenario. The general categories of CHs are

described in Table 1.

Table 1. Summary of Test Characteristics for the TCL4 Flight Demonstration.

Category Description Scenarios

Density of Operations Number of UAS airborne within an identified area 1, 2, 3, 4, 5

Tempo of Operations Number of takeoffs and landings within an identified area 1, 4, 5

Operation Volume

Types

Area Based Operation Volumes (ABOV) or Transit Based Operation

Volumes (TBOV) identified in an operation

1, 2, 3, 4, 5

Airspace Type Operation in controlled or uncontrolled airspace 1, 2, 3

Ground Obstructions Operation proximity to structures / buildings 1, 4

Obstruction Types Operation proximity to Dynamic or Static obstructions 4, 5

Automated

Launch/Land

UAS launch or land location relative to operator location (co-located or

remote)

1, 3, 4, 5

Flight Profiles General classes of flight profiles such as linear inspection, and area

inspection

1, 2, 3, 4, 5

Launch/Landing Profiles UAS launch or land at combinations of ground and rooftop locations 1, 4

USS Negotiations USS requests for modifications of operations managed by other USS 1, 2, 3, 5

Priority Operations Operations constrained by navigational capability (in-flight emergency)

or responding to priority missions (public safety)

2, 3, 4

Remote UAS ID UAS identification through or not through the USS network, by a

terrestrial or aerial entity

2, 3, 4
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

USS Participation Number of USS supporting operations and sharing data per USS

specifications

1, 2, 3, 4, 5

C. Test Events

Test Events (TE) were used to induce changes to the environment or operation with the intent of exercising

different technologies and procedures. The general categories of TEs are described in Table 2.

Table 2. Summary of Test Events for the TCL4 Flight Demonstration.

Category Description Scenarios

UAS Volume

Reservations

An airspace restriction is enacted which limits the availability of the

airspace to certain vehicles

1, 2

Conflicts A UAS comes within a defined range of either manned aircraft,

obstacle, or another UAS

1, 2, 3, 4,

5

Safe Landings Landings within/outside of operation volumes, with

operational/non-operational safe landing capability

1, 2, 3, 4

USS Impact A USS is lost (stops providing service), or an Operation changes to a

different USS

5

Unexpected Unmanned

Aircraft (UA) Behavior

UA has an inflight accident or is flying in a reckless manner 4

Loss of C2 Small scale and large scale loss of C2 link based on the number of

operations airborne

2, 3, 4, 5

Loss of Navigation Small scale and large scale loss of navigation based on the number of

operations airborne

2, 3, 4, 5

Remote UAS ID Query UAS identification through or not through the USS network 2, 3, 4

III. Measures of Performance

Each UTM high-level requirement, originally presented in the NASA UTM ConOps document [2], has an

associated Measure of Effectiveness (MOE). Ref. [9] states that an MOE is a “measure by which a stakeholder's

expectations will be judged in assessing satisfaction with products or systems produced and delivered in accordance

with the associated technical effort.” In addition, “[a]n MOE is typically qualitative in nature or not able to be used

directly as a ‘design-to’ requirement.” Each MOE is supported by multiple MOPs, which are quantitative in nature.

This approach and these terms are detailed in the NASA Systems Engineering Handbook [10], and a depiction is

shown in Figure 2.

NASA identified twenty key Measures of Performance (MOPs) for TCL4, categorized into one or more of

four categories as shown in Figure 3: data and architecture; sense and avoid; communications and navigation; and

concepts. These categories align with the FAA-NASA Research Transition Team on UTM subgroups [11].
 

 

 

 

 

 

 

Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

Figure 2. Relationship of high-level requirements, MOEs, and MOPs.

Figure 3. Summary of Measures of Performance, organized by research focus.

The Concept MOPs focus on assessing pilot perspectives on the UTM System, and validating the ability of the

system to handle high-density, BVLOS operations. The Architecture MOPs investigate strategic deconfliction of

operations, replanning in the presence of dynamic restrictions, security of USS data exchanges, and latency of

communications between USSs. The Communications and Navigation MOPs measure the rate of safe landings and

various aspects of communications, control, and navigation losses. The Sense and Avoid MOPs focus on the

containment of flights within operational volumes, vehicle to vehicle reception, operator alerting of conflicts, the

duration of conflicts within UTM, and the frequency of loss of well-clear. There are also MOPs that cross the

concept and architecture groups involving contingency responses, priority operations, and remote identification.

The results of each MOP will be published in detail in various venues as they are developed. Some initial

results are summarized in Section V.

IV. Test Planning and Execution

The TCL4 flight test built up through several key steps. NASA researchers developed detailed scenarios, test

events, and characteristics relevant to NASA’s research goals. In this section the overall preparation and execution

process is provided, followed by the plans by the NV and TX test sites to deliver on NASA's goals.

A. Planning

The SOW described in Section II was provided to the Test Sites in early Fall 2018. The schedule included

therein had several key milestones. A detailed test plan was required as part of the SOW response. The selected

Test Sites were then required to iterate on that plan with NASA to produce a second version that would ultimately

guide the execution of the tests.
 

 

 

 

Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

The finalized test plans were then used as the basis for two key activities with all test stakeholders: a tabletop

walkthrough of all scenarios and a collaborative simulation. The tabletop exercise clarified the team’s

understanding of the flow of the flight test and helped identify gaps in terms of data collection, logistics, timelines,

etc. The collaborative simulation involved all partners with several vehicles included as hardware-in-the-loop

connected to their respective USSs as the test sites and NASA simulating the operations and communications,

emulating the day-of-operations activities as well as possible.

The test sites were then required to execute shakedown flights in order to exercise their field deployment plans

in their respective urban environments. In parallel with all of these activities, the USSs that were partnered with the

test sites underwent software checkout processes [12] to ensure their interoperability and readiness to support the

field activities.

B. Nevada

NIAS proposed a test range across two sites in Reno, Nevada and completed Scenarios 1, 2, and 4. Scenarios 1

and 4 occurred in the downtown area (see Figure 4) and Scenario 2 occurred in and around Idlewild Park. Scenarios

3 and 5 were not performed by NIAS, in order to focus efforts on NASA’s higher priority requirements and to take

advantage of fewer range reconfigurations.

NIAS had 25 partners. Of these partners, 18 participated directly in flight testing as industry and city partners.

Five partners were UAS Service Providers (USS) [Uber Elevate, Airmap, AiRXOS (GE Venture), ANRA, and

Avision Robotics] and six UAS operators [Uber Elevate, AviSight, Drone America, Switch, Utah State University,

and AiRXOS].

Shakedown (practice) testing at Nevada took place May 15-23, 2019 and the official flight Demonstration

occurred June 17-28. During the demonstration, NIAS had 144 live flights with a total active operation time of about

18 hours. During a given scenario and run, up to five live UAS flew concurrently. Additionally, more than 500

simulated flights were flown over the course of the demonstration concurrent with the live flights to create higher

density UAS operations and to safely execute certain flight interactions.

Figure 4. Downtown Reno test range.
 

 

 

 

 

 

Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

C. Texas

LSUASC proposed a test range across four sites in Corpus Christi, Texas and completed all five scenarios.

Scenario 1, 3, 4, 5 occurred in the downtown (see Figure 5) and waterfront sites, Scenario 2 in the Arts and

Entertainment district, Scenario 3 also involved operations at Corpus Christi Port and Corpus Christi airport.

LSUASC had 21 partners, 14 of which were industry and city partners. Three UAS operators [Near Earth

Autonomy, LSUAC, and the Corpus Christi Fire Department] flew eight live UAS vehicles supported by seven

USSs [five of those that participated in Nevada with the addition of One Sky (AGI) and Collins Aerospace].

LSUASC performed a tabletop exercise with NASA in May 2019 to step through the processes and operations.

Then, shakedown testing was executed July 25-31, 2019 and the official flight Demonstration took place August

12-23, 2019. During the demonstration, LSUASC had 208 live flights with a total active operation time of about 37

hours. During a given scenario and run, up to seven live UAS flew concurrently. Additionally, more than 400

simulated flights were flown over the course of the demonstration concurrent with the live flights to create higher

density UAS operations and to safely execute certain flight interactions.

Figure 5. Downtown Corpus Christi test range.

D. TCL4 Fleet and Equipment

Across the two test sites, there were 8 unique multicopters used (DJI M600, M200, M210, and Mavic; Air

Robot AR200; Drone America Nav X; 3DR Solo; and Tarot X6) in the flight demonstration. Also, there were a

variety of on-board and ground equipment and infrastructure on the UASs and at the sites such as radars, detect and

avoid and obstacle avoidance technologies, communication and navigation technologies, parachute technology, and

weather stations, that were used to support testing. UAS Command and Control (C2) communications equipment

was in two variants, one that used the Industrial, Scientific, and Medical (ISM) frequency spectrum and another

using Long Term Evolution (LTE) spectrum. GPS was the main navigation technology for participating sUAS.

E. Operational Incidents

While the vast majority of operations and vehicles performed as expected, there were a few occasions in which

unexpected issues were observed. There were wiring issues with one of the UAS models resulting in the damage to

two vehicles during hard landings. This necessitated the grounding of that model for some time during the testing
Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

 

 

 

 

 

 

 

 

window to address the issue. Radio frequency environmental issues caused damage to two vehicles while operating

in the urban environment. The likely cause for the first was GPS multipathing7, while the second exhibited

command and control interference. These two incidents occurred on the same range on different days. Another

model vehicle experienced multiple instances of control interference, but without an unplanned landing or vehicle

damage. One vehicle was also moderately damaged due to an unexpected parachute activation, landing hard, but

clear of people or property on the ground.

While the root causes of these incidents, and their eventual solutions, are distinct (wiring issues, RF

environment problems, payload malfunction), there are general lessons to learn that are likely more valuable than

solving the individual problems. Some of these are touched upon in the Lessons Learned subsection below. At a

high level, it will be important to share information about off-nominal situations, including crashes, in an operational

environment such that the entire UTM community can benefit from what is experienced by individual operations,

operators, and manufacturers.

Manned incursions by medical helicopters into the test range occured in both Nevada and Texas. These

incursions interrupted UAS operations and were handled procedurally by range safety personnel as such events were

considered in the safety case for both ranges. UAS involved in TCL4 were always able to stay clear of the manned

operations. These manned incursions happened in spite of test coordination with the helicopter dispatch. This

highlights a known need within future UTM operational environments for appropriate integration with traditional

aviation. That potential integration can take many forms, but is not discussed further in this document.

E. NASA Data Collection

Data collection and analysis were critical to the success of TCL4. USSs submitted data collected from the

operations that they supported as well as data from the test sites as required by a Data Management Plan (DMP) [13,

14].

V. Findings

A. MOPs

In this section, some of the key results, and an example of the result, are summarized. Each MOP that is

intended for external reporting is detailed in other publications. The following subsections group the MOPs by the

publication in which they are more fully reported; for in-depth analysis and visualization of the results, please see

the referenced publication.

USS Network Performance

The communication performance of the USSs is summarized by three MOPs fully reported in a NASA Technical

Memorandum [13]. For example, network latency between USSs was the focus of UTM-MOP-13. The results for

this MOP showed that latency is not a likely impediment to nominal operation of a future, operational UTM system,

with the 95th percentile latency across all USS-USS messages being 532ms (see Figure 6). Results did indicate that

performance requirements need to be developed on a per endpoint basis, as the processing required for the various

USS endpoints can vary significantly. UTM-MOP-16 successfully demonstrated high density operations (at least 10

operations within 0.2nmi2) with no observed negative network effects. To maintain this level of quality, USSs would

have to horizontally scale their resources based on the geographical extents under their management, which is a

reasonable expectation for modern web services.

7 GPS multipath occurs when the radio signals from the GPS satellite(s) are received after traveling a variety of

distances due to reflecting off of nearby objects, like buildings. This introduces uncertainty in the GPS calculations

determining position.
Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

 

 

 

 

 

Figure 6. UTM-MOP-13 results for latency of USS-USS data exchanges.

Strategic Deconfliction

The performance related to the strategic deconfliction of UTM operations was measured in three MOPs and

are fully reported in a NASA Technical Memorandum [15]. UTM-MOP-01 measured the overall strategic

deconfliction rate and showed that USSs were able to use UTM protocols to strategically deconflict operations by

meeting the overall minimum success criterion of more than 95% of nominal operations strategically deconflicted.

Figure 7. Strategic deconfliction results over 327 operations.

Communications and Navigation

The performance related to the communications and navigation of operations within TCL4 was measured in

four MOPs reported in a NASA Technical Memorandum [16]; the results are briefly summarized here. Two of these

MOPs rely on a TCL4 definition for a conflict. UA-to-UA conflicts were defined as being within 700ft horizontally

and 200ft vertically of each other. Note that this does not imply or state any standard or accepted definition of

well-clear for sUAS, but provides a reasonable set of values for testing purposes. Also note that operations may be

strategically deconflicted and still be in “conflict” via this definition.
 

 

 

 

Rate of safe landing was measured in UTM-MOP-05. Of the 219 landings used in this analysis, 117 (~53%)

occurred within 3m of a planned landing location or within an identified polygon, such as a contingency landing

polygon, 26 (~12%) were outside of 3m but within 5m of a planned landing location, and 35 (~16%) were outside of

5m but within 10m of a planned landing location. The remaining operations also landed safely, but did so

inconsistently with their operations plans, indicating a need to ensure intent is appropriately updated and shared in

Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

UTM. Figure 8 illustrates these values.

Figure 8. Rate of safe landing (UTM-MOP-05) in TCL4.

Rate of loss of navigation during conflict was measured in UTM-MOP-06. For this MOP, both operations in

conflict needed to have navigation data in order to be considered. A loss of navigation was determined by either a

provided Navigation Accuracy Category Position (NACp) or a calculated position dilution of precision (PDOP)

relative to an applicable threshold. The majority of operations in conflict which provided either NACp or PDOP

indicated a loss of navigation capability resulting in 14205 seconds where at least one operation had a navigation

loss out of a total conflict duration of 15523 seconds, a rate of about 92%. This was poorer than targeted for the

flight test, but illustrates the difficulty of GPS reliance in an urban environment, especially during conflict situations.

For UTM-MOP-19, most operations with reported data maintained command and control links throughout

their operation (128 out of 140). The 12 operations that experienced a loss of C2 accounted for 1807 seconds of lost

airborne C2 time, which was about 1.8% of the airborne time over 140 operations. For UTM-MOP-09 both

operations in conflict needed to have C2 data in order to be considered. Of the operations identified in MOP-19

which experienced a loss of C2, only 1 was in conflict, resulting in 28 seconds of C2 loss out of a total conflict

duration of 12553 seconds, which translates to a less than 1% rate. UTM-MOP-09 was defined with a minimum

success criterion of < 4%, thus the operations met expectations for this MOP.

Human Factors

UTM-MOP-15 was entitled “Crew Assessment of UTM Information Properties” and was focused on gathering

data related to user interactions with UTM components. A full report on this MOP is provided in two separate

NASA Technical Memoranda [17][18]. Crew interactions with the UTM System are mediated through the displays

provided by USSs, so impressions can vary. UTM information was reported by flight crews to be useful for building

their awareness of the flight situations, and data collected from flight crews indicated that UTM provided

information that contributed to users’ ability to operate safely and effectively within the UTM environment. UTM

system-to-user communications were possibly better than users were aware of, since data exchanged between

system components of UTM are not always displayed to the user, depending on the USS user interface

implementations. Although crews had good situation awareness of their ownship, they requested additional UTM

information when the information they had access to was not sufficiently usable, salient, intuitive or complete on
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

their current displays. Information management on UTM, especially of alerts, was reported to be a distracting

activity to some crews and may benefit from filtering capabilities. As expected during flight tests in a complex

environment, observers noted a fair amount of crew cross checking of UTM information with alternate information

sources, suggesting that crews’ confidence and trust in UTM information were still being established.

B. Related NASA UTM Project Research

The TCL4 flight activities spawned a great deal of research beyond the core MOPs. Work supported by the

UTM Project and related to TCL4 has been published on navigation performance characterization[19], which

provides data related to sUAS operation in urban environments. In addition, the reporting of off-nominal operations

[20][21] has been further developed and tested in multiple flight tests, with recommendations for incorporation into

an operational environment. Vehicle-to-vehicle system design [22] and path planning with obstacle avoidance [23]

were also studied with point-design solutions to ensure alignment with the UTM concept and lay the groundwork for

future requirements development. NASA UTM research related and contemporaneous to TCL4 includes the

development of mission task elements for sUAS [24], ground risk assessment [25][26], sense-and-avoid

characterization [27], and sUAS vehicle cybersecurity [28].

C. Lessons Learned

There were many lessons learned from this demonstration. NASA has cataloged several lessons for various

purposes and audiences. These and other lessons have been transferred in various ways to industry (via standards

development meetings, workshops, etc.), to the FAA (via the NASA-FAA Research Transition Team activities), and

internally at NASA (to other Projects and to NASA Centers).

TCL4 had several high-level objectives which could be summarized into three major categories. Objective 1

(O1) was to identify technological gaps to enable UTM. Objective 2 (O2) was to demonstrate and illustrate the

viability of the UTM concept through live flight tests in an urban environment. Finally, Objective 3 (O3) was to

inform standardization of UTM that will occur through future rulemaking and standards developing organizations.

Three columns in Table 3 indicate how the lessons learned supported the high-level objectives (O1-O3)

Table 3. Summary of key lessons learned in TCL4.

Lesson Title Description O1 O2 O3

Altitude

reference

The lack of standardization in referencing altitudes across components in the

sUAS domain is a significant hurdle that NASA and others have noted

previously. For USS-USS data exchanges, WGS84 measured in feet has been

used by NASA for a few years. However, USSs and other stakeholders often

make translations of altitude measurements from other sources with other

reference frames, units, or approaches that are not always fully understood by

all the users of those data. This potentially causes false or missed conflicts

between operations when USSs attempt to compare altitudes with each other

for deconfliction or conformance purposes. This is a significant issue that

industry must address for safe integration and operationalization of UTM.

✓ ✓

Discovery

worked

When the architecture of UTM evolved between TCL1 and TCL3 to a fully

federated system, NASA recognized the need for a system for the key

federated components (USSs) to discover each other for communications.

NASA engaged with industry partners via technical outreach activities to

force the discussion on a discovery system. NASA’s initial approach to prove

✓ ✓
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

the concept was improved upon by industry and successfully tested in TCL3

and TCL4. This transfer from government research to industry innovation on

a component of the UTM ecosystem is a key example of how the system

should continue to evolve, with industry taking a larger and larger role in the

specification of the operational system based on insights gained through

collaborative testing.

Off-nominal A critical path in off-nominal situations handling was identified (e.g. a human

situation director of flight operations coordinating UAS and USS in response to

handling off-nominal situation) and automation is suggested to remove that critical

path [20]. An approach involving each USS providing a semi-standardized

interface to collect off-nominal situations reports was identified as the best

approach. Continued collection of off-nominal reports and sending them to

the Aviation Safety Reporting System was suggested so that evaluation and

analysis can continue in order to understand off-nominal situations and reduce

their occurrence [19].

✓ ✓

Degraded C2 Given the obstacles in the urban environment, reliance on GPS for navigation

and navigation and the known characteristics of communications between the pilot and

performance in

urban

ground infrastructure, it was expected that there would be degraded

performance of UA navigation and C2 links. It was valuable to obtain ✓ ✓

environment measurements of dozens of flights using multiple UAS platforms to document

these performance issues.

Gathering Early and often partner engagement pays off. This was planned as a feature

consensus of the UTM Project from the beginning and it bore fruit through collaborative

testing. The efforts of the industry partners provided them key insights into

the UTM Concept that could not be easily gained in any other way. Those

partners, in turn, have taken the concept into the standardization process and

have become global experts and ambassadors of the UTM concept as

understood by NASA and the FAA.

✓

Value of flying Early in the development of UTM at NASA, the Project had a choice of flight

in the NAS testing in restricted airspace or in the National Airspace System (NAS). The

value of flying in the NAS was assumed to outweigh the increased effort to

execute the flight tests. This assumption was realized throughout all flight

tests, including (and especially) in TCL4. It was vital to gather public

acceptance and understand community concerns. It was valuable to go

through the approval processes with the FAA to learn the limitations and

opportunities to improve that process for NASA and the FAA. It was

beneficial to work through safety cases that took into consideration pop-up

operations that were not involved in the testing, experience incursions by

non-participating operations, and to then consider how that might affect or

drive the concept.

✓ ✓
 

 

 

 

 

 

 

 

Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

USS

negotiation

development

The most complex protocol tested in TCL4 was USS-USS Negotiation.

Given project schedule limitations, the protocol was not defined to a level that

allowed for USSs to properly implement for full testing in TCL4. However,

even with a limited implementation, the value of negotiations was evident.

Further refinement of negotiations will be important in all phases of UTM

development and operationalization.

✓ ✓

VI. Conclusion

TCL4 testing accomplished NASA’s general objectives and several auxiliary objectives. We accelerated

partner development of the seven participating USSs, tested in an urban environment, tested the state of the art of

many existing commercial technologies, and established the need for improvements or new technologies. The test

sites, partners, and NASA gained a better understanding of the FAA process necessary for authorizations, waivers,

and vehicle safety approvals as well as city approvals required for UAS operation in urban environments and were

able to provide that experience back to the FAA.

As a capstone activity for the NASA UTM Project, TCL4 highlighted how far the concept and implementation

of UTM has come since its inception at NASA Ames over six years ago. Industry and other government agencies

are poised to take the lessons and experience gained through collaborative flight testing and move toward

operationalization of UTM. This is generally the primary goal of all NASA aeronautics research, thus indicating a

measure of success for the overall UTM effort at NASA.

References

[1] Federal Aviation Administration, “Unmanned Aircraft System (UAS) Traffic Management (UTM) Concept of Operations

v2.0,” published March 2020.

[2] Kopardekar, P., Rios, J., Prevot, T., Johnson, M., Jung, J., and Robinson III, J.E., “Unmanned Aircraft System Traffic

Management (UTM) Concept of Operations,” 16th AIAA Aviation Technology, Integration, and Operations Conference

(ATIO), Washington, D.C., June 2016. https://doi.org/10.2514/6.2016-3292.

[3] Rios, J., Mulfinger, D., Homola, J., Venkatesan, P., “NASA UAS Traffic Management National Campaign: Operations

Across Six UAS Test Sites,” 35th Digital Avionics Systems Conference (DASC), Sacramento, CA, 2016.

https://doi.org/10.1109/DASC.2016.7778080.

[4] Johnson, M., Jung, J., Rios, J., Mercer, J., Homola, Prevot ,T., Mulfinger, D., and Kopardekar, P., “Flight Test Evaluation

of an Unmanned Aircraft System Traffic Management (UTM) Concept for Multiple Beyond-Visual-Line-of-Sight

Operations,” 12th USA/Europe Air Traffic Management Research and Development Seminar (ATM2017), Seattle, WA, June

26–30, 2017.

[5] Homola, J., Mohlenbrink, M., Dao, Q., Claudatos, L., Martin, L., and Mercer, J., “UAS Technical Capability Level 2

Unmanned Aircraft System Traffic Management (UTM) Flight Demonstration: Description and Analysis,” IEEE-DASC, St.

Petersburg, FL., September 17–21, 2017. https://doi.org/10.1109/DASC.2017.8101994.

[6] Aweiss, A., Owens, B., Rios, J., Homola, J., and Mohlenbrink, C., “Unmanned Aircraft Systems (UAS) Traffic Management

(UTM) National Campaign II,” AIAA SciTech Forum, Kissimmee, FL., January 8–12, 2018.

https://doi.org/10.2514/6.2018-1727.

[7] Homola, J., Martin, L., Cencetti, M., and Aweiss, A., “UAS Traffic Management (UTM) Technical Capability Level 3

(TCL3) Flight Demonstration: Concept Tests and Results,” 38tth DASC 2019, San Diego, CA, September 8-12, 2019.

https://doi.org/10.1109/DASC43569.2019.9081617.

[8] Aweiss, A., Homola, J., Rios, J., Jung, J., Johnson, M., Mercer, J., Modi, H., Torres, E., and Ishihara, A., “Flight

Demonstration of Unmanned Aircraft System (UAS) Traffic Management UTM at Technical Capability Level 3”, 38tth

DASC 2019, San Diego, CA, September 8-12, 2019. https://doi.org/10.1109/DASC43569.2019.9081718.

[9] NASA, Procedural Requirements, NPR 7123.1B, April 2013.
Downloaded by Joseph Rios on June 23, 2020 | http://arc.aiaa.org | DOI: 10.2514/6.2020-2851 

[10] NASA, Systems Engineering Handbook, NASA SP-2016-6105 Rev2, 2016.

[11] NASA and FAA, UAS Traffic Management (UTM) Research Transition Team (RTT) Plan, January 2017,

https://utm.arc.nasa.gov/docs/2017-FAA_NASA_UTM_RTT_Plan.pdf.,

[12] Smith, I., Rios, J., Mulfinger, D., Baskaran, V., Verma, P., “UAS Service Supplier Checkout: How UTM Confirmed

Readiness of Flight Tests with UAS Service Suppliers,” NASA TM–2019–220456, December 2019.

[13] Modi, H., et al., “UTM TCL4 Data Management Plan As-Built Approach,” NASA Technical Memorandum, submitted for

publication.

[14] Rios, J., et al., “UAS Service Supplier Network Performance: Results and Analysis from Flight Testing Multiple USS

Providers in NASA’s TCL4 Demonstration,” NASA TM-2020-220462, January 2020.

[15] Rios, J., Homola, J., Craven, N., Verma, P., Baskaran, V., “Strategic Deconfliction Performance: Results and Analysis from

NASA’s Technical Capability Level 4 Demonstration,” NASA Technical Memorandum, submitted for publication.

[16] Jung, J., Craven, N., “Small Unmanned Aircraft System Communications and Navigation Performance: Results and

Analysis from NASA’s Unmanned Aircraft System Traffic Management Technical Capability Level 4 Demonstration,”

NASA Technical Memorandum, submitted for publication.

[17] Martin, L., Wolter, C., Jobe, K., Goodyear, M., Manzano, M., Cencetti, M., Mercer, J., Homola, J., “TCL4 UTM (UAS

Traffic Management) Texas 2019 Flight Tests, Airspace Operations Laboratory (AOL) Report,” NASA TM-2020-220516,

March 2020.

[18] Martin, L., Wolter, C., Jobe, K., Manzano, M., Blandin, S., Cencetti, M., Claudatos, L., Mercer, J. & Homola, J., “TCL4

UTM (UAS Traffic Management) Nevada 2019 flight tests, Airspace Operations Laboratory (AOL) report,” NASA

Technical Memorandum, submitted for publication.

[19] Logan, M., Bird, E. ,Hernandez L., Menard, M., “Operational Considerations of Small UAS in Urban Canyons,” AIAA

SciTech 2020 Forum, Orlando, FL, 6-10 January 2020, https://doi.org/10.2514/6.2020-1483.

[20] Jung, J., Rios, J., Drew, C., Modi, H., Jobe, K.,“Small Unmanned Aircraft System Off-Nominal Operations Reporting

System: Unmanned Aircraft System Traffic Management Technical Capability Level 4 Implementation, Data Collection and

Analysis,” NASA TM-2019–220302, February 2020.

[21] Jung, J., Nag, S., “Automated Management of Small Unmanned Aircraft System Communications and Navigation

Contingency, ” AIAA SciTech 2020 Forum, Orlando, FL, 6-10 January 2020, https://doi.org/10.2514/6.2020-2195.

[22] Chakrabarty, A., Ippolito C., Baculi J., Krishnakumar, K., Hening, S., “Vehicle to Vehicle (V2V) communication for

Collision avoidance for Multi-copters flying in UTM -TCL4,” AIAA SciTech 2019 Forum, San Diego, CA, 7-11 January

2019, https://doi.org/10.2514/6.2019-0690.

[23] Chakrabarty, A., Stepanyan, V., Krishnakumar, K., Ippolito C., “Real-Time Path Planning for Multi-copters flying in

UTM-TCL4,” AIAA SciTech 2019 Forum, San Diego, CA, 7-11 January 2019, https://doi.org/10.2514/6.2019-0958.

[24] Lampton, A.K., Klyde, D.H., Prince, T., Swaney, T., Belcastro C. M., “Toward Developing MTEs for Multirotor sUAS in

Controlled Wind Conditions,” AIAA SciTech 2020 Forum, Orlando, FL, 6-10 January 2020,

https://doi.org/10.2514/6.2020-1507.

[25] Ancel, E., Capristan, F.M., Foster, J.V., Condotta, R.C., “In-Time Non-Participant Casualty Risk Assessment to Support

Onboard Decision Making for Autonomous Unmanned Aircraft,” AiAA Aviation 2019 Forum, Dallas TX, 17-21 June 2019,

https://doi.org/10.2514/6.2019-3053.

[26] Ippolito, C., “Dynamic Ground Risk Mitigating Flight Control for Autonomous Small UAS in Urban Environments,” AIAA

SciTech 2019 Forum, San Diego, CA, 7-11 January 2019, https://doi.org/10.2514/6.2019-0961.

[27] Consiglio, M., Duffy, B., Balachandran, S., Glaab, L., Muñoz, C., “Sense and Avoid Characterization of the Independent

Configurable Architecture for Reliable Operations of Unmanned Systems,” Thirteenth USA/Europe Air Traffic Management

Research and Development Seminar (ATM2019), Vienna, Austria, 17-21 June 2019.

[28] Ippolito C., Krishnakumar, K., “An Interface-Based Cybersecurity Subsystem Analysis on a Small Unmanned Aerial

Systems,” AIAA SciTech 2019 Forum, San Diego, CA, 7-11 January 2019, https://doi.org/10.2514/6.2019-1459.