NASA/TM-20205007032 

Small Unmanned Aircraft System

Communications and Navigation

Performance 

Results and Analysis from NASA’s Unmanned Aircraft 

Systems Traffic Management Technical Capability

Level 4 Demonstration 

Jaewoo Jung 

NASA Ames Research Center, Moffett Field, California 

Nicholas Craven 

Millennium Engineering and Integration Company, Moffett Field, California 

AUGUST 2020 

1 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

NASA STI Program ... in Profile 

Since its founding, NASA has been dedicated 

to the advancement of aeronautics and space 

science. The NASA scientific and technical 

information (STI) program plays a key part in 

helping NASA maintain this important role. 

The NASA STI program operates under the 

auspices of the Agency Chief Information Officer. 

It collects, organizes, provides for archiving, and 

disseminates NASA’s STI. The NASA STI 

program provides access to the NTRS Registered 

and its public interface, the NASA Technical 

Reports Server, thus providing one of the largest 

collections of aeronautical and space science STI 

in the world. Results are published in both nonNASA channels and by NASA in the NASA STI 

Report Series, which includes the following report 

types: 

� TECHNICAL PUBLICATION. Reports of 

completed research or a major significant 

phase of research that present the results of 

NASA Programs and include extensive data 

or theoretical analysis. Includes compilations of significant scientific and technical 

data and information deemed to be of 

continuing reference value. NASA counterpart of peer-reviewed formal professional 

papers but has less stringent limitations on 

manuscript length and extent of graphic 

presentations. 

� TECHNICAL MEMORANDUM. 

Scientific and technical findings that are 

preliminary or of specialized interest, 

e.g., quick release reports, working 

papers, and bibliographies that contain 

minimal annotation. Does not contain 

extensive analysis. 

� CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees. 

� CONFERENCE PUBLICATION. 

Collected papers from scientific and 

technical conferences, symposia, seminars, 

or other meetings sponsored or 

co-sponsored by NASA. 

� SPECIAL PUBLICATION. Scientific, 

technical, or historical information from 

NASA programs, projects, and missions, 

often concerned with subjects having 

substantial public interest. 

� TECHNICAL TRANSLATION. 

English-language translations of foreign 

scientific and technical material pertinent to 

NASA’s mission. 

Specialized services also include organizing 

and publishing research results, distributing 

specialized research announcements and 

feeds, providing information desk and personal 

search support, and enabling data exchange 

services. 

For more information about the NASA STI 

program, see the following: 

� Access the NASA STI program home page 

at http://www.sti.nasa.gov 

� E-mail your question to help@sti.nasa.gov 

� Phone the NASA STI Information Desk at 

757-864-9658 

� Write to: 

NASA STI Information Desk 

Mail Stop 148 

NASA Langley Research Center 

Hampton, VA 23681-2199 

2 
 

 

 

 

 

 

 

NASA/TM-20205007032 

Small Unmanned Aircraft System

Communications and Navigation

Performance 

Results and Analysis from NASA’s Unmanned Aircraft 

Systems Traffic Management Technical Capability

Level 4 Demonstration 

Jaewoo Jung 

NASA Ames Research Center, Moffett Field, California 

Nicholas Craven 

Millennium Engineering and Integration Company, Moffett Field, California 

August 2020 

3 
 

Acknowledgments 

The authors appreciate Unmanned Aircraft Systems Traffic Management team members 

and Technical Capability Level 4 demonstration participants for their dedication and hard 

work. 

This report is available in electronic form at 

https://www.sti.nasa.gov

4 
 

 

 

 

 

 

 

 

 

 

 

 

 

1. Abstract 

NASA performed research and development of technologies and requirements for traffic 

management of small Unmanned Aircraft Systems (UAS). In this effort, four measures of 

performance (MOPs) were developed to understand the performance of small UAS 

communications and navigation systems in urban operations. This Technical Memorandum (TM) 

describes UAS Traffic Management (UTM) operational architecture, UTM Technical Capability 

Level 4 (TCL4) flight tests that took place in two different urban settings, the four MOPs, and the 

TCL4 MOP results. 

2. UTM Operational Architecture 

In UTM, a UAS Service Supplier (USS) provides appropriate operational data to the operator and 

other USSs. These data exchanges are defined by a set of Application Programming Interfaces 

(API) and UAS Service Supplier Specification [Rios 2019]. Figure 1 shows a high-level data 

exchange in UTM using the APIs. UTM is complementary to the FAA’s Air Traffic Management 

(ATM) systems, and the interaction between the National Airspace System (NAS) and UTM takes 

place through Flight Information Management System (FIMS). 

Figure 1. UTM Data Exchange Architecture 

5 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

3. TCL4 Flight Test 

Test Description 

NASA performed UTM research and development in four Technical Capability Level (TCLs). In 

TCL1 Visual Line-of-Sight (VLOS) operations such as agriculture, firefighting, and infrastructure 

monitoring were addressed with a focus on geofencing and operations scheduling [Rios 2016]. 

Technologies and requirements for Beyond Visual Line-of-Sight (BVLOS) operations in sparsely 

populated areas were examined in TCL2 and for operations over moderately populated areas in 

TCL3 [Johnson 2017], [Homola 2017], [Aweiss 2019]. TCL4 built on the earlier TCLs and focused 

on BVLOS operations in urban areas for tasks such as newsgathering and package delivery, and 

for managing large-scale contingencies. TCL4 is briefly described here and for more information 

see [Rios 2020]. 

The TCL4 demonstration in 2019 was a set of scenario-driven activities that integrated the UTM 

research and development objectives. Several scenarios were created focusing on different 

challenges associated with operations in a potential operational TCL4 environment, such as 

persistent, mixed tempo, and high-density BVLOS operations over an urban area. Two sets of 

operational tests were conducted at the Reno, NV test range and the Corpus Christi, TX, test 

range. A total of 144 live operations were flown from June 18-28, 2020, in the Reno flight test 

range for some of the scenarios, with nine live Unmanned Aircraft (UA) produced by three different 

manufacturers. These included one vehicle from AirRobot GmbH & Co. KG, five from Drone 

America, and three from SZ DJI Technology Co., Ltd. Five USSs, developed by AirMap Inc., 

AiRXOS (a GE venture), ANRA Technologies, Avision Inc., and Uber Technologies Inc., 

supported the operations. A total of 208 operations were flown from August 12-16, and August 

19-23, 2020, in the Corpus Christi, Texas range for all scenarios, with nine UA from two 

manufacturers, including two from 3DR and seven from SZ DJI Technology Co., Ltd. Seven USSs 

developed by AirMap Inc., AiRXOS, ANRA Technologies, Avision Inc., Collins Aerospace (a unit 

of United Technologies Corp.), OneSky (Analytical Graphics, Inc.), and Uber Technologies Inc. 

supported the operations. 

For all flight operations, arrangements were made to allow only participants in the defined range. 

Extended VLOS operations were used to emulate BVLOS operations. That is, the UA was in sight 

of designated visual observers when the operator was unable to see it due to obstruction (e.g., 

UA behind a building) or distance, and the observers and the operators maintained a line of 

communication among them to keep track of the operation. 

Since TCL4 focused on operations in urban areas, it is important to understand UAS 

communications and navigation performance in an urban setting that would enable safe and 

efficient operations. For this purpose, data were collected from UA, the Ground Control System 

(GCS) and the USS, and the collected data were processed to calculate MOPs for the 

performance assessment. 

Data Collection 

The TCL4 data collection architecture evolved with the lessons learned from previous TCLs where 

the NASA UTM team worked with internal and external stakeholders to execute flight tests and 

collect data. In the TCL4 test, there were two flows of data into NASA from the stakeholders. First 

was the data exchanged as part of the operational concept [FAA 2018], [NASA 2016], such as 

operation plans, UA positions, and other operational UTM messages. All of these data were 

collected during testing with a designated USS Data Collector (UDC) that interacted with the other 

USSs as if it were a USS. The other flow of data came via the system set up by NASA’s Data 

6 
 

 

 

 

 

 

 

 

Management Plan (DMP) team. The DMP data were for research purposes and not part of the 

operational concept. These data were collected during and after testing. 

Both data flows ended up in a Universal Data Store (UDS), a single source of truth for all TCL4 

data. The UDS within this architecture ensured consistencies across numerous data visualization 

and analysis efforts. Figure 2 summarizes the TCL4 data collection architecture [NASA 2020]. 

Figure 2. TCL4 Data Collection Architecture 

4. Communications and Navigation Measures of Performance 

The NASA UTM team defined four MOPs to assess the performance of small UAS 

communications and navigation systems in the TCL4 settings. Table 1 shows a summary of the 

MOPs addressed in this TM. The following two subsections describe data preparation and 

reduction steps for the MOP calculations. 

7 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 1. Summary of Communications and Navigation MOP 

MOP Description Minimum 

Success 

Target 

Success 

Rate of loss of 

Command and 

Control (C2) 

This MOP calculates the percentage of time across 

all operations that an operation was without 

complete C2. 

< 4% < 1% 

Rate of C2 loss 

during a Conflict 

This MOP calculates the percentage of operations 

in a conflict that experienced a loss of C2 during the 

conflict. 

< 4% 0% 

Rate of Navigation 

Degradation during a 

Conflict 

This MOP measures the rate of navigation 

performance degradation per airborne time during 

conflicts. 

≤ 15% of 

the time 

of conflict 

0% of the 

time of 

conflict 

Rate of Safe Landing This MOP measures the percentage of a safe 

landing. 

≥ 25% ≥ 75% 

Data Preparation 

The data needed to perform the analysis in this TM was sourced from data exchanges within the 

USS Network and data collected by the Test Site and provided directly to NASA via the DMP 

system. From the USS Network Data, operation plans were used to identify the airspace that the 

operator intends to use for the operation, described as a series of 4D volumes. UTM messages 

were used to keep track of state changes to the operation, including any priority or contingency 

status changes. 

From the DMP data, the following models1 were used: 

1. ConPreRunOp provided planned landing locations, such as safe landing locations, 

rooftop landing locations, visual line of sight and beyond visual line of sight locations. 

2. ConActualLanding provided the actual landing location(s) and time(s). 

3. AuxiliaryUASOperation provided another source for actual landing location(s) and 

time(s). 

4. FlightEssentialTelemetryAndUasStates provided data taken from the vehicle’s logs 

which included location, communication status, and navigation status data. 

1 All TCL4 DMP data models are available at https://github.com/nasa/utmdocs/tree/master/TCL4%20Data%20Management 

8 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Data Reduction 

Operations for Analysis 

All data were collected during each run of each scenario. However, not all data that end up in the 

TCL4 datastore were intended or appropriate for the calculation of each MOP. To determine valid 

operations for analysis, the metadata of each model is filtered as follows: 

1. Only models identified by the submitter as valid via a metadata flag 

2. Only valid demonstration scenarios, with corresponding valid run numbers. Some 

scenarios are for testing or shakeout of the operations or various subsystems and are not 

intended for data analysis. 

3. Only operations that changed altitude by 75ft, which is based on the standard GPS error 

model of ~2 sigma 

4. Only live flights (i.e. not simulated) 

All operations not intended for data analysis are excluded from the results. 

Upon completion of these initial reductions, and review of the quantity and quality of the other 

DMP data models submitted for those operations, it was determined that only one of the two test 

sites provided enough of the data sufficient for analysis. As a result, only the operations from the 

Texas test site were considered. 

Determining Landing Locations 

For this analysis, the valid operations identified in the ‘Operations for Analysis’ section were used 

as the preliminary filter. This list is then further reduced to only operations with landing location(s) 

and time(s). These values were received via the DMP, the ConActualLanding and 

AuxiliaryUASOperation models, as discussed in the ‘Data Preparation for MOP Calculations’ 

section. 

Using this method, 184 distinct operations were identified. From these operations, there were a 

total of 219 distinct landings for analysis. Distinct landings were defined as landings with a unique 

location (i.e. latitude and longitude) and time. Since this data was taken from two sources, it is 

possible that a single ‘actual landing’ is represented by two or more landings in the analysis. If, 

for example, the same ‘actual landing’ was submitted in the ConActualLanding model, and the 

AuxiliaryUASOperation model with a slightly different timestamp or location, they would be 

considered two different landings in this analysis. 

9 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Determining Conflict Occurrences 

For this analysis, the valid operations identified in the ‘Operations for Analysis’ section were used 

as the preliminary filter. This list of operations was then further reduced to only operations with 

FlightEssentialTelemetryAndUasStates location data (i.e. Latitude, Longitude, and Altitude). 

To identify the operations ‘in conflict’, the FlightEssentialTelemetryAndUasStates location data 

for each operation were compared against the data for all other relevant operations (i.e. 

operations in the same scenario and run). If at a given time, two operations were within 700ft 

laterally and 200ft vertically, the two operations were considered ‘in conflict’. From these ‘in 

conflict’ occurrences, the duration of a given conflict could be determined. An operation pair could 

possibly have multiple conflict occurrences (i.e. the operations go within the conflict distance 

bounds, move further apart than the bounds, and return to within the bounds). To account for 

such conditions, if the operations were ‘out of conflict’ for longer than 20 seconds, the next time 

they were ‘in conflict’ would count as the start of a new conflict occurrence. 

Using this method, 136 distinct operations were identified. Between those operations, a total of 

172 conflicts occurred. 

Determining Loss of Command and Control (C2) Occurrences 

For this analysis, the valid operations identified in the ‘Operations for Analysis’ section were used 

as the preliminary filter. This list of operations was then further reduced to only operations with 

FlightEssentialTelemetryAndUasStates communication data. For communication data, NASA 

asked the test site and partners to provide the metrics that they would use to identify a loss of C2. 

These metrics would then be added as ‘sensors’ in the FlightEssentialTelemetryAndUasStates 

model. The metric provided was identified as ‘CONNECTED_TO_RC_DJI’. 

For the MOP calculation, any occurrence of that sensor with a value of ‘0’ for longer than 5 

seconds is identified as a loss of communication. 

Using this method, a total of 140 operations were identified that included C2 status data. Of those 

140 operations, 12 experienced a loss of C2. 

Determining Degraded Navigation Occurrences 

For this analysis, the valid operations identified in the ‘Operations for Analysis’ section were used 

as the preliminary filter. This list of operations was then further reduced to only operations with 

data that could be used to determine a degraded navigation condition, which came from the 

FlightEssentialTelemetryAndUasStates DMP model. 

Similar to the communication data, NASA asked the test site and partners to provide the metrics 

that they would use to identify degraded navigation. These metrics would then be added as 

10 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

‘sensors’ in the FlightEssentialTelemetryAndUasStates model. The metric provided was identified 

as ‘NAVIGATION_ACCURACY_CATEGORY_POSITION’. Any occurrence of 

‘NAVIGATION_ACCURACY_CATEGORY_POSITION’ with a value less than 10 for longer than 

5 seconds indicated degraded navigation. 

Using this method, a total of 140 operations were identified that included navigation status data. 

Of those 140 operations, 96 experienced a degraded navigation condition. The following four 

subsections describe the four MOPs in detail. 

Rate of Loss of C2 

Background 

The small UAS operators are expected to perform missions within constraints and directives that 

can change over time (e.g., a Temporary Flight Restriction). Therefore, operators must know 

whether they can communicate or not with their UA to adhere to dynamic constraints and 

directions [Jung 2020a]. For the TCL4 environment, the UTM team targeted <1% of the 

accumulated operational time to experience Loss of C2, with <4% as a minimum success. During 

the TCL4 demonstration, the operators used commercial off-the-shelf communications systems 

using Industrial Scientific, and Medical (ISM) frequency bands and also Long-Term Evolution 

(LTE) devices for communications. 

Results 

Of the 140 operations with the data necessary for this MOP calculation, 12 experienced a loss of 

C2. Those 12 operations had a total of 1807 seconds in a loss of C2 condition. The total duration 

of the 140 operations was 103,067 seconds, resulting in a C2 loss occurring in about 2% of the 

total duration. This meets the minimum success criteria. 

Discussion 

Of the 12 operations that experienced a loss of C2, 11 experienced loss of C2 near the beginning 

or the end of the operation; 7 experienced the loss within one minute of the operation starting or 

ending and 4 experienced the loss within 10 meters of the starting or ending position. Such a 

trend implies that there was a high level of Radio Frequency Interference (RFI) near the takeoff 

and landing locations in these 11 cases. In the 12th case, loss of C2 occurred between 199 meters 

and 232 meters from the GCS location, as shown in Figure 3 where green indicates where the 

operation flew with C2 capability, yellow indicates where C2 was lost, and the star icon indicates 

the GCS location. In this case, RFI from an unknown source or the loss of radio line of sight could 

have disrupted the C2 link. These findings enforce an earlier recommendation to the operator for 

monitoring the RF characteristic of the intended operation area and examine the radio noise floor 

during operations [Jung 2019]. The UA in Figure 3 continued flying the programmed route with 

the loss of C2. This implies a few possibilities. One is that the conditions used to identify loss of 

C2 for the MOP calculation did not trigger the UA’s loss of C2 handling procedure such as return 

to its launch location. Another is that the UA recovered C2 before activating the UA’s loss of C2 

handling procedure. For future studies, it is recommended that the actual parameters used by 

UAs to identify loss of C2 condition to be logged both at UA and the GCS, along with the 

11 
 

 

 

 

 

 

 

 

 

 

associated thresholds and conditions. This would help researchers and operators gain further 

insights into understanding the loss of C2. 

Figure 32. Top-Down View of an Operation with Loss of C2 

Rate of C2 Loss during a Conflict 

Background 

For operations in the TCL4 environment, it is expected that UA to UA conflict is resolved with UA 

onboard, and/or ground-based, Detect and Avoid (DAA) functionality. C2, in particular UA 

telemetry, is necessary for the operator to be informed of a conflict and its resolution (e.g., UA 

heading away from each other). The UTM Project established a minimum success criterion of 

less than 4% rate of C2 loss during a conflict, and a target success of 0% for this MOP. 

Results 

Results identified in the ‘Determining Conflict Occurrences’ and ‘Determining Loss of C2 

Occurrences’ sections in the Appendix are combined to determine the number of operations which 

had a loss of C2 occurrence during a conflict. Only conflicts with C2 data for both operations are 

considered. 

2 Underlying map data is ã OpenStreetMap contributors. https://www.openstreetmap.org/copyright 

12 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Of the 97 conflicts where both operations had C2 status data, only 1 of the 97 conflicts had an 

operation that experienced a loss of C2 during the conflict. The 97 conflicts resulted in a total 

conflict duration of 12,553 seconds, and only 28 seconds where at least 1 of the operations 

experienced a loss of C2. This resulted in a less than 1% rate, which met the minimum success 

criteria. 

Discussion 

At the time of the TCL4 demonstration, DAA solutions were not available and for safety, all 

operations were procedurally separated. For example, one UA was programmed to fly at higher 

altitudes than another UA in the same area. For analysis, a conflict was designated after the 

demonstration with the following threshold; two UA within 700 ft laterally and 200 ft vertically. 

Since conflicts are designated post-flight, they were passively resolved. That is, the two UA in the 

conflict did not maneuver to resolve the conflict but flew the routes as programmed. 

Figure 4 shows the tracks of an example conflict. Note that the track figure is in 2D, whereas the 

conflict is determined in 3D. In the figure, the green lines indicate the full flight path, the yellow 

lines indicate the portion of the flight path where the operations were in conflict, and the red lines 

indicate the portion of the flight path where there was a loss of C2, which may have been caused 

by RFI or the loss of radio line of sight. The UA in Operation 1 continued flying the programmed 

route with the loss of C2. As previously mentioned, this implies a few possibilities. One is that the 

conditions used to identify loss of C2 for the MOP calculation did not trigger the UA’s loss of C2 

handling procedure, such as return to its launch location. Another is that the UA recovered C2 

before activating the UA’s loss of C2 handling procedure. 

Figure 43. Top-Down view of Conflicting Operations 

3 Underlying map data is ã OpenStreetMap contributors. https://www.openstreetmap.org/copyright 

13 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Rate of Navigation Degradation during a Conflict 

Background 

To safely resolve a conflict, a properly functioning navigation system is necessary. The UTM 

Project established a minimum success criterion of less than 15% rate of navigation degradation 

during a conflict, and a target success of 0% for this MOP. 

Results 

Results identified in the ‘Determining Conflict Occurrences’ and ‘Determining Loss of Navigation 

Occurrences’ sections in the Appendix are combined to determine the number of operations which 

had a loss of navigation occurrence during a conflict. Only conflicts with navigation data for both 

operations are considered. 

Of the 97 conflicts where both operations had navigation status data, in 86 of the conflicts (about 

89%) at least one of the operations experienced degraded navigation. Of the 86 cases, 82 cases 

involved one or both operations experiencing degraded navigation performance for the entire 

duration of the conflict. Table 2 shows details of this MOP result. 

The 86 conflicts resulted in a total duration of 11,252 seconds where at least 1 of the operations 

experienced navigation performance degradation, resulting in about 90% rate, far exceeding the 

success criteria of less than 15%. 

Table 2. UA Navigation Performance during Conflicts 

Navigation Performance Status Number of 

Conflicts 

(97 total) 

Degraded Nav

Duration 

(sec) 

Conflict 

Duration 

(sec) 

Both UA degraded for the entire conflict duration 67 9605 9605 

1 UA degraded for the entire conflict duration, 

1 UA degraded for a portion of the conflict 

duration 

1 141 141 

1 UA degraded for the entire conflict duration, 

1 UA Nominal 

14 1391 1391 

1 UA degraded for a portion of the conflict 

duration, 

1 UA Nominal 

4 115 249 

Both UA Nominal 11 0 1167 

14 
 

 

 

 

 

 

 

 

 

 

 

 

 

Discussion 

To calculate this MOP, the Horizontal Accuracy Category, NACp, of 10 is used as the minimum 

navigation performance. This category is associated with 10 meters or better horizontal accuracy 

bound (e.g., 9 meters) [RTCA 2011]. Figures 5, 6, and 7 show examples of nominal and degraded 

navigation performance during conflicts. In each figure, there are two sub-figures; one which 

shows the tracks of the two operations, and another which shows the navigation values with an 

indication of when the operations are in conflict (represented by the shaded region of the plots). 

Note that the track figure is in 2D, whereas the conflict is determined in 3D. In the track subfigures, the green lines indicate the full flight path, the yellow lines indicate the portion of the flight 

path where the operations were in conflict, and the red lines indicate the portion of the flight path 

where there was degraded navigation. 

In Figure 5, two operations enter conflict but do not experience degraded navigation. As the figure 

shows, the operations move within the ‘conflict’ distance several times during the operation, and 

the NACp values are always greater than the minimum (10) value. Figure 6 depicts a pair of 

operations where only one operation experiences a loss of navigation during the conflict. Figure 

7 shows a pair of operations where both operations experience degraded navigation. 

Figure 54. Example: two operations which enter conflict but do not experience degraded navigation 

4 Underlying map data is ã OpenStreetMap contributors. https://www.openstreetmap.org/copyright 

15 
 

 

Figure 65. Example: two operations which enter conflict and one experiences degraded navigation 

5 Underlying map data is ã OpenStreetMap contributors. https://www.openstreetmap.org/copyright 

16 
 

 

 

 

Figure 76. Example: two operations which enter conflict, and both experience degraded navigation 

For the TCL4 Demonstration, the operators used UAS built-in functions to monitor navigation 

performance, and no significant issues were reported throughout the operations [Jung 2020b]. 

This contrasts with the high rate of navigation degradation results, as represented by lower than 

6 Underlying map data is ã OpenStreetMap contributors. https://www.openstreetmap.org/copyright 

17 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

10 NACp value. The discrepancy between the lack of actual navigation performance issues 

reported by the operators, and the MOP result, implies that whereas the 10-meter horizontal 

position accuracy was difficult to achieve during the TCL4 demonstration, the UTM was still able 

to maintain conformance. That is, UA operated without leaving their operation volumes. 

Rate of Safe Landing 

Background 

For TCL4, the safe landing of UA was defined as the landing of UA at a planned location, with an 

assumption that this location was kept clear of people and properties not related to landing. The 

UTM Project established a minimum success criterion of greater than 25% rate of the UA 

achieving safe landing, with a target success of greater than 75% for this MOP. 

Results 

For the actual landing locations, the ones identified in the ‘Determining Landing Locations’ section 

in the Appendix were used. For the planned landing locations, a designated landing position 

included in the operation plan was used. If an operation announced that a contingency was 

initiated, the contingency landing polygon was used as well. Note that the operation plan uses a 

position (i.e. point) and the contingency plan uses a polygon for indicating landing location. A 

‘safe’ landing was determined when UA landed within 3 m, 5 m, or 10 m of a designated landing 

position, or when landed inside of the polygon defined in the contingency plan. The three 

distances reflect potential position error that can come from UA navigation system. 

Of the 219 landings, 117 occurred within 3m of a planned landing location or within an identified 

polygon (53%), 26 (12%) were outside of 3m but within 5m of a planned landing location, and 35 

(16%) were outside of 5m but within 10m of a planned landing location. As shown in Figure 8, 

81% of landings were considered safe with the 10 m threshold, meeting the target success criteria 

for this MOP. 

Figure 8. Rate of safe landing 

18 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Discussion 

Of the 219 landings, 41 (or 19%) landed more than 10 m away from the planned location. Whereas 

many of these operations were manually conducted by pilots, the data did not distinguish between 

the number of manual and automated landings. For the TCL4 demonstration, which used an 

Extended VLOS setting to emulate BVLOS operations, these landings did not have any actual 

safety implication as visual observers and the operator always had the UA in sight and made 

certain that there were no persons in the UA’s landing path. For actual BVLOS operations, the 

UA should always land at the planned location that is properly prepared and maintained (e.g., a 

fenced platform that only allows operators to enter) and have the capability to avoid causing 

detrimental impact to people and property when it must land away from the planned location. 

Summary 

Data collected from the TCL4 demonstration were used to assess small UAS communications 

and navigation performance in urban settings. Four MOPs, rate of loss of C2, rate of C2 loss 

during the conflict, rate of navigation degradation during the conflict, and rate of the safe landing, 

were calculated for this assessment. At the time of the TCL4 demonstration, the DAA capability 

was not implemented and this led to the procedural separation of concurrent operations 

preventing UA to UA conflict situations. Since two of the MOPs are associated with conflicts, 

reinvestigation of these MOPs with DAA solutions is recommended. For future studies on 

communication performance, it is recommended that the actual parameters used by UAs to 

identify loss of C2 condition be logged both at UA and the GCS, along with the associated 

thresholds and conditions. This would help researchers and operators gain further insights into 

understanding the loss of C2. 

References 

[Aweiss 2019] Aweiss, A., et al., “Flight Demonstration of UTM at Technical Capability Level 3”, 

IEEE/AIAA 38th Digital Avionics Systems Conference (DASC), September 2019. 

[FAA 2018] FAA NextGen Office, "Unmanned Aircraft Systems (UAS) Traffic Management 

(UTM) Concept of Operations", v1.0, May 2018. 

[Johnson 2017] Johnson, M., et al., “Flight Test Evaluation of UTM Concept for Multiple BeyondVisual-Line-of-Sight Operations”, 12th UAS/Europe ATM R&D Seminar, June 2017. 

[Homola 2017] Homola, J., et al., “Technical Capability Level 2 UAS Traffic Management Flight 

Demonstration: Description and Analysis”, DASC 2017, September 2017. 

[Jung 2019] Jung, J., Nag. S., and Modi H., “Effectiveness of Redundant Communications 

Systems in Maintaining Operational Control of Small Unmanned Aircraft”, DASC 2019, 

September 2019. 

[Jung 2020a] Jung, J., and Nag, S., “Automated Management of Small Unmanned Aircraft 

System Communications and Navigation Contingency”, AIAA SciTech Forum 2020, January 

2020. 

19 
 

 

 

 

 

 

 

 

 

 

[Jung 2020b] Jung, J., Rios, J., Drew, C., Modi, H., and Jobe, K., “Small Unmanned Aircraft 

Systems Off-Nominal Operations Reporting System”, NASA Technical Memorandum, 

NASA/TM-2019-220302. 

[NASA 2016] Kopardekar, P., Rios, J., Prevot, T., Johnson, M., Jung, J., Robinson III, J., 

"Unmanned Aircraft System Traffic Management (UTM) Concept of Operations", AIAA Aviation 

Forum 2016, June 2016. 

[NASA 2020] Rios, R., et al., “UAS Service Supplier Network Performance, Results and 

Analysis from Flight Testing Multiple USS Provider in NASA’s TCL4 Demonstration”, NASA 

Technical Memorandum, NASA/TM-2020-220462. 

[Rios 2016] Rios, J., et al., “NASA UTM National Campaign”, DASC 2016, September 2016. 

[Rios 2019] Rios, J., et al., “UAS Service Supplier Specification”, NASA Technical 

Memorandum, NASA/TM-2019-220376, October 2019. 

[Rios 2020] Rios, J., et al., “Flight demonstration of UTM at Technical Capability Level 4”, AIAA 

Aviation Forum 2020, June 2020. 

[RTCA 2011] “Minimum Operational Performance Standard (MOPS) for 1090 MHz Extended 

Squitter Automatic Dependent Surveillance-Broadcast (ADS-B) and Traffic Information 

Services-Broadcast (TIS-B)”, RTCA DO-260B, December 2011. 

20