THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Small UAS Detect and Avoid Requirements Necessary for
Limited Beyond Visual Line of Sight (BVLOS) Operations:Final Report

October 25, 2022

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

NOTICE
This document is disseminated under the sponsorship of the U.S. Department of Transportation in
the interest of information exchange. The U.S. Government assumes no liability for the contents
or use thereof. The U.S. Government does not endorse products or manufacturers. Trade or
manufacturers’ names appear herein solely because they are considered essential to the objectiveof this report. The findings and conclusions in this report are those of the author(s) and do notnecessarily represent the views of the funding agency. This document does not constitute FAA
policy. Consult the FAA sponsoring organization listed on the Technical Documentation page as
to its use.

ii

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

LEGAL DISCLAIMER
The information provided herein may include content supplied by third parties. Although the dataand information contained herein has been produced or processed from sources believed to bereliable, the Federal Aviation Administration makes no warranty, expressed or implied, regarding
the accuracy, adequacy, completeness, legality, reliability or usefulness of any information,
conclusions or recommendations provided herein. Distribution of the information contained
herein does not constitute an endorsement or warranty of the data or information provided hereinby the Federal Aviation Administration or the U.S. Department of Transportation. Neither theFederal Aviation Administration nor the U.S. Department of Transportation shall be held liable
for any improper or incorrect use of the information contained herein and assumes noresponsibility for anyone’s use of the information. The Federal Aviation Administration and U.S.
Department of Transportation shall not be liable for any claim for any loss, harm, or other damages
arising from access to or use of data or information, including without limitation any direct,
indirect, incidental, exemplary, special or consequential damages, even if advised of the possibilityof such damages. The Federal Aviation Administration shall not be liable to anyone for any
decision made or action taken, or not taken, in reliance on the information contained herein.

iii

THIRD PARTY RESEARCH. PENDING FAA REVIEW.
Technical Report Documentation Page
1. Report No.

2. Government Accession No.

3. Recipient's Catalog No.

A18_ A11L.UAS.22
4. Title and Subtitle

5. Report Date

October 24, 2022
Small UAS Detect and Avoid Requirements Necessary for Limited Beyond Visual
Line of Sight (BVLOS) Operations: Final Report

6. Performing Organization Code

7. Author(s)8. Performing Organization Report No.

Mark Askelson: https://orcid.org/0000-0002-8521-7158
Henry Cathey: https://orcid.org/0000-0003-4496-1076
Timothy Bruner: https://orcid.org/0000-0002-7591-8823
9. Performing Organization Name and Address

University of North Dakota (4201 James Ray Dr. Stop 8367, Grand Forks, ND 58202)
New Mexico State University (1780 E Univ. Ave., Las Cruces, NM 88003)
Kansas State University (2310 Centennial Road, Salina, KS 67401)
12. Sponsoring Agency Name and Address

10. Work Unit No.
11. Contract or Grant No.

15-C-UAS
13. Type of Report and Period Covered

U.S. Department of Transportation
Federal Aviation Administration
Washington, DC 20591

Final Report
14. Sponsoring Agency Code5401
15. Supplementary Notes

Conducted in cooperation with the U.S. Department of Transportation, Federal Highway Administration, ASSURE UAS Center of
Excellence.
16. Abstract

The demand for Beyond Visual Line Of Sight (BVLOS) operations using small Uncrewed Aircraft Systems (sUASs) is high. A
major impediment to realization of these operations is the Detect And Avoid (DAA) function. This effort explored many DAA
challenges through execution of numerous tasks, results of which are described subsequently.
The Operational Framework effort resulted in a slight expansion of a previously-developed use case taxonomy and identificationof technologies that are now being operationally utilized. Minimal additional information regarding expansion of the operationalframework based on Radio Line Of Sight (RLOS) coverage was identified.
A significant simulation effort to evaluate DAA system and Uncrewed Aircraft System (UAS) characteristics that impact
maintenance of well clear was conducted. DAA characteristics that had the greatest impact are range and Field of View (FoV).
Update rate and latency impacts were not as dramatic, while horizontal and vertical resolution were the least impactful. UAS
characteristics that strongly impact maintenance of well clear are response time (i.e., the time required to initiate a maneuver) and
UAS speed.
An overarching test plan was developed. This test plan describes test locations/performers, dates of testing, DAA systems used in
tests, overarching test objectives, individual test plan structure, methods for maintenance of safety during testing, data collection
approaches, the structure of test reports, and test metrics/artifacts.
Seven rounds of flight tests were completed. Important outcomes from these tests include development of a systematic approachfor evaluating DAA systems, identification of test metrics/artifacts, test data collection methods/best practices, methods for enabling
flight test safety, methods for executing both horizontal and climb- and descend-into encounters, evaluation of DAA systems(especially the detection component), and utilization of results in American Society for Testing Materials (ASTM) committees tosupport standards development.
This effort involved a broad set of tasks designed to inform Federal Aviation Administration (FAA) regulations and industry
standards regarding sUAS DAA systems. Through execution of these tasks and application of the numerous methods required to
do so, the team has significantly advanced sUAS DAA, which will enable more rapid integration of sUAS into the National Airspace
System (NAS)—especially for BVLOS operations.
18. Distribution Statement
17. Key Words
This document is available to the U.S. public through the
Detect and Avoid
National Technical Information Service (NTIS), Springfield,
Test Plan
Virginia 22161. This document is also available from the Federal
Flight Test
Aviation Administration William J. Hughes Technical Center at
Standards
actlibrary.tc.faa.gov.
19. Security Classif. (of this report)

Unclassified
Form DOT F 1700.7 (8-72)

20. Security Classif. (of this page)

Unclassified

21. No. of Pages

117

Reproduction of completed page authorizediv

22. Price

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

TABLE OF CONTENTSTECHNICAL REPORT DOCUMENTATION PAGEIV

TABLE OF CONTENTS

V

TABLE OF FIGURESIXTABLE OF TABLESXIITABLE OF ACRONYMS

XIII

EXECUTIVE SUMMARYXV

1INTRODUCTION

12NON FLIGHT-TEST REPORTS

2

2.1

TASK 1: OPERATIONAL FRAMEWORK.......................................................................................... 2

2.1.1

Objectives ............................................................................................................................................... 2

2.1.2

Methods .................................................................................................................................................. 2

2.1.3

Summary of Results ................................................................................................................................ 2

2.1.4

Lessons Learned ..................................................................................................................................... 4

2.2

TASK 2: UPDATE OF SUAS DAA SOLUTIONS INVENTORY.......................................................... 5

2.2.1

Objectives ............................................................................................................................................... 5

2.2.2

Methods .................................................................................................................................................. 5

2.2.3

Summary of Results ................................................................................................................................ 5

2.2.4

Lessons Learned ..................................................................................................................................... 6

2.3

TASK 3: COORDINATION WITH STANDARDS AGENCY TO ESTABLISH FRAMEWORK .................. 6

2.3.1

Objectives ............................................................................................................................................... 6

2.3.2

Methods .................................................................................................................................................. 6

2.3.3

Summary of Results ................................................................................................................................ 6

2.3.4

Lessons Learned ..................................................................................................................................... 7

2.3.4.1

Metrics7

2.3.4.2

Simulation and Flight Testing

7

2.4

TASK 4: SEPARATION FRAMEWORK ............................................................................................ 7

2.4.1

Objectives ............................................................................................................................................... 7

2.4.2

Methods .................................................................................................................................................. 8

2.4.3

Summary of Results ................................................................................................................................ 8

2.4.4

Lessons Learned ................................................................................................................................... 10

2.5

TASK 5: TEST PLAN ................................................................................................................... 10

2.5.1

Objectives ............................................................................................................................................. 10

2.5.2

Methods ................................................................................................................................................ 10v

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3

2.5.3

Summary of Results .............................................................................................................................. 10

2.5.4

Lessons Learned ................................................................................................................................... 11

FLIGHT-TEST REPORTS
3.1

11

UAF MARCH 2019 ..................................................................................................................... 11

3.1.1

Objectives ............................................................................................................................................. 11

3.1.2

Dates/Schedule ..................................................................................................................................... 11

3.1.3

Location................................................................................................................................................ 12

3.1.4

System Tested ....................................................................................................................................... 12

3.1.5

Test Plan Overview .............................................................................................................................. 13

3.1.6

Sample Test Cards ................................................................................................................................ 13

3.1.7

Summary of Results .............................................................................................................................. 14

3.1.8

Lessons Learned ................................................................................................................................... 14

3.2

NMSU JULY 2019...................................................................................................................... 15

3.2.1

Objectives ............................................................................................................................................. 15

3.2.2

Dates/Schedule ..................................................................................................................................... 15

3.2.3

Location................................................................................................................................................ 15

3.2.4

System Tested ....................................................................................................................................... 17

3.2.5

Test Plan Overview .............................................................................................................................. 21

3.2.5.1

CTLS and Multi-Rotor Tests

21

3.2.5.2

CTLS and Fixed-Wing UAS

23

3.2.5.3

Ultralight Spyder and both Multi-Rotor and Fixed wing UAS’s

23

3.2.5.4

Pop-up Encounters

23

3.2.6

Sample Test Cards ................................................................................................................................ 23

3.2.7

Summary of Results .............................................................................................................................. 27

3.2.8

Lessons Learned ................................................................................................................................... 36

3.3

UAF AUGUST 2019 .................................................................................................................... 38

3.3.1

Objectives ............................................................................................................................................. 38

3.3.2

Dates/Schedule ..................................................................................................................................... 38

3.3.3

Location................................................................................................................................................ 38

3.3.4

System Tested ....................................................................................................................................... 39

3.3.5

Test Plan Overview .............................................................................................................................. 39

3.3.6

Sample Test Cards ................................................................................................................................ 39

3.3.7

Summary of Results .............................................................................................................................. 40

3.3.8

Lessons Learned ................................................................................................................................... 42

3.4
3.4.1

UND SEPTEMBER 2020.............................................................................................................. 43
Objectives ............................................................................................................................................. 43vi

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3.4.2

Dates/Schedule ..................................................................................................................................... 43

3.4.3

Location................................................................................................................................................ 44

3.4.4

System Tested ....................................................................................................................................... 47

3.4.5

Test Plan Overview .............................................................................................................................. 50

3.4.6

Sample Test Cards ................................................................................................................................ 53

3.4.7

Summary of Results .............................................................................................................................. 57

3.4.8

Lessons Learned ................................................................................................................................... 59

3.5
3.5.1

Objectives ............................................................................................................................................. 59

3.5.2

Dates/Schedule ..................................................................................................................................... 59

3.5.3

Location................................................................................................................................................ 59

3.5.4

System Tested ....................................................................................................................................... 60

3.5.5

Test Plan Overview .............................................................................................................................. 60

3.5.6

Sample Test Cards ................................................................................................................................ 63

3.5.7

Summary of Results .............................................................................................................................. 66

3.5.8

Lessons Learned ................................................................................................................................... 68

3.6

UND JUNE 2021......................................................................................................................... 69

3.6.1

Objectives ............................................................................................................................................. 69

3.6.2

Dates/Schedule ..................................................................................................................................... 69

3.6.3

Location................................................................................................................................................ 69

3.6.4

System Tested ....................................................................................................................................... 69

3.6.5

Test Plan Overview .............................................................................................................................. 70

3.6.6

Sample Test Cards ................................................................................................................................ 73

3.6.7

Summary of Results .............................................................................................................................. 77

3.6.8

Lessons Learned ................................................................................................................................... 77

3.7

4

UAF JUNE 2021 ......................................................................................................................... 59

NMSU FALL 2021 ..................................................................................................................... 77

3.7.1

Objectives ............................................................................................................................................. 78

3.7.2

Dates/Schedule ..................................................................................................................................... 79

3.7.3

Location................................................................................................................................................ 79

3.7.4

System Tested ....................................................................................................................................... 80

3.7.5

Test Plan Overview .............................................................................................................................. 82

3.7.6

Sample Test Cards ................................................................................................................................ 84

3.7.7

Summary of Results .............................................................................................................................. 88

3.7.8

Lessons Learned ................................................................................................................................... 95

OTHER TASKS/ADDITIONAL MATERIAL
4.1

96

OSU RADAR SYSTEM DEVELOPMENT AND TESTING ................................................................ 96vii

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

5CONCLUSIONS

98

BIBLIOGRAPHY

101

viii

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

TABLE OF FIGURES
Figure 1. Example separation framework results for a radar-based DAA system.

9

Figure 2. Overhead view of the Poker Flat Research Range.

12

Figure 3. Example test card used during the March 2019 UAF ACUASI campaign.

14

Figure 4. Test location, just North of Las Cruces, NM, for the July 2019 NMSU campaign.

16

Figure 5. Dirt runways on the Jornada Experimental Range.

16

Figure 6. Trailer setup.

17

Figure 7. The Casia system.

18

Figure 8. NMSU’s CTLS light sport aircraft.

19

Figure 9. NMSU’s Spyder ultralight aircraft.

19

Figure 10. Iris’s RMRC Anaconda (fixed wing).

20

Figure 11. Iris’s SUI Endurance (multi-copter) and NMSU’s 3D Robotics Solo.

20

Figure 12. Example test metadata card for the NMSU July 2019 campaign.

24

Figure 13. Example test script/plan for the NMSU July 2019 campaign.

25

Figure 14. Encounter geometry with overlays of the planned flight location.

26

Figure 15. Overall encounter geometry mapped onto the flight test location.

27

Figure 16. Example multi-copter UAS flight path.

27

Figure 17. DAA testing flights on day 1—full flights.

28

Figure 18. DAA testing flights on day 1—near testing location.

29

Figure 19. Day 1 flight altitudes.

30

Figure 20. Crossing encounter DAA test.

31

Figure 21. DAA test encounter after turn had been initiated.

32

Figure 22. Mapping of Iris DAA detections based on encounter angle and distance.

33

Figure 23. Example scenario (closing head-on scenario) utilized during the August 2019
UAF ACUASI campaign.

40

Figure 24. Summary of viable encounters versus detections.

42

Figure 25. Planned daily schedule for the September 2020 UND campaign.

43

Figure 26. Locations of testing elements during the September 2020 UND campaign.

45

Figure 27. Sectional centered on the September 2020 flight test area.

46

Figure 28. Images from the Lovas farm.

47

Figure 29. The C Speed Lightwave Radar utilized during the September 2020 UND
campaign.

49

ixTHIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 30. Illustration of encounters associated with the CA flying northwest to southeast.

51

Figure 31. As in Figure 7 but for the CA flying southeast to northwest.

52

Figure 32. Example FTD test card for the September 2020 UND campaign.

56

Figure 33. Illustration of the 154437-154653 UTC 23 September 2020 encounter.

59

Figure 34. Helio Courier 295 CA used as an intruder during the June 2021 UAF campaign.

61

Figure 35. Cessna 206 CA used as an intruder during the June 2021 UAF campaign.

62

Figure 36. Trinity F90 VTOL UA used as an intruder during the June 2021 UAF campaign.

62

Figure 37. Bradley Ranch encounter profile with intruder in the traffic pattern.

63

Figure 38. Example test card (profile 2) from the June 2021 UAF campaign.

66

Figure 39. Ground motion induced noise experienced with the Echoflight radar.

67

Figure 40. Variability of Echoguard initial detection range.

68

Figure 41. Planned test schedule for June 2021 UND campaign.

69

Figure 42. Example of the RangeVue™ Pro display.

70

Figure 43. Illustration of descend-into encounters used during the June 2021 UND
campaign.

72

Figure 44. Example FTD test card for the June 2021 UND campaign.

76

Figure 45. Plan view of the test area for the Fall 2021 NMSU campaign.

80

Figure 46. The Iris CasiaX system.

81

Figure 47. CasiaX cameras mounted on the left wing of ownship.

81

Figure 48. CasiaX cameras mounted on the right wing of ownship.

82

Figure 49. CasiaX camera mounted on the front of ownship.

82

Figure 50. Configuration of horizontal encounters executed during the Fall 2021 NMSU
campaign.

83

Figure 51. Configuration of climb- and descend-into encounters executed during the Fall
2021 NMSU campaign.

84

Figure 52. Compass rose test card used during the Fall 2021 NMSU campaign.

86

Figure 53. Intruder descend behind test card used during the Fall 2021 NMSU campaign.

88

Figure 54. 16 November 2021 flight profiles and detections.

90

Figure 55. As in Figure 54, but with revised scales.

91

Figure 56. As in Figure 54 but for 17 November 2021 flight profiles and detections.

92

Figure 57. As in Figure 56, but with revised scales.

93

Figure 58. As in Figure 56, but with a change of scales (zoomed-in).

94

Figure 59. Image of the OSU passive radar system.

97x

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 60. Results obtained with the OSU passive radar system.

xi

98THIRD PARTY RESEARCH. PENDING FAA REVIEW.

TABLE OF TABLES
Table 1. Planned daily schedule for the March 2019 UAF ACUASI campaign.

12

Table 2. Summary of DAA test encounter parameters for the July 2019 NMSU campaign.

35

Table 3. Iris detection results for July 2019 NMSU campaign.

36

Table 4. VO data summary.

41

Table 4. Information regarding the L3Harris™ DAA system.

48

Table 5. Information regarding the UA used during the September 2020 UND campaign.

53

Table 6. Information regarding the intruder aircraft used during the September 2020 UND
campaign.
53
Table 7. Information regarding the UA used during the June 2021 UND campaign.

73

Table 8. Information regarding the intruder aircraft used during the June 2021 UND campaign.
73
Table 9. Planned number of encounters for encounter scenarios for the Fall 2021 NMSU
campaign.
78
Table 10. DAA system used in the Fall 2021 NMSU campaign.

xii

80

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Table of Acronyms
Acronym
A&G
AC
ACUASI
ADS-B
AGL
ASTM
BVLOS
CA
CPA
DAA
DCAPS
DJI
DTEM
EFP
EO
FAA
FoR
FoV
FTD
GBDAA
GCS
HEFP
IMU
IR
KSU
LRE
MESA
MP
MSL
NAS
NMAC
NMSU
NPRM
OSU
PIC
RLOS
RMRC
RTCA
SARP
SBSS
SME
SMS
sUAS
SUI
SWaP
UA
UAS
UAF
UASTS
UND
UTM

Meaning
Alerting and Guidance
Air Conditioner
Alaska Center for Unmanned Aircraft Systems Integration
Automatic Dependent Surveillance-Broadcast
Above Ground Level
American Society for Testing Materials
Beyond Visual Line Of Sight
Crewed Aircraft
Closest Point of Approach
Detect And Avoid
Data Collection and Processing System
Da-Jiang Innovations
Detect, Track, Evaluate, and Maneuver
Encounter Focal Point
Electro-Optical/Electronic Observer
Federal Aviation Administration
Field of Regard
Field of View
Flight Test Director
Ground-Based Detect and Avoid
Ground Control Station
Horizontal Encounter Focal Point
Inertial Measurement Unit
InfraRed
Kansas State University
Launch and Recovery Element
Metamaterial Electronically Scanning Array
MegaPixel
Mean Sea Level
National Airspace System
Near Mid-Air Collision
New Mexico State University
Notice of Proposed RuleMaking
Ohio State University
Pilot In Charge
Radio Line Of Sight
Ready Made RC
Radio Technical Commission for Aeronautics
Science And Research Panel
Surveillance and Broadcast Services Subsystem
Subject Matter Expertise
Safety Management Systemsmall Uncrewed Aircraft System
Straight Up Imagery
Size, Weight, and Power
Uncrewed Aircraft
Uncrewed Aircraft System
University of Alaska Fairbanks
UAS Test Site
University of North Dakota
Unmanned Traffic Managementxiii

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

VAS
VO
VTOL

Value Added Service
Visual Observer
Vertical Take-Off and Landingxiv

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

EXECUTIVE SUMMARY
The demand for Beyond Visual Line Of Sight (BVLOS) operations using small Uncrewed Aircraft
Systems (sUASs) is high. A major impediment to realization of these operations is the Detect And
Avoid (DAA) function. Several challenges exist for sUAS DAA. This effort explored many ofthese challenges through execution of numerous tasks, results of which are described subsequently.
Given the broad set of tasks, multiple methods were applied to execute them. These include review
of previous efforts, analysis and synthesis, simulation, and testing and validation.
The Operational Framework effort resulted in a slight expansion of the previously-developed use
case taxonomy (Cathey and Hottman 2017) and identification of technologies that are now being
operationally utilized. Minimal additional information regarding expansion of the operationalframework based on Radio Line Of Sight (RLOS) coverage was identified.
Maintenance of a DAA systems inventory indicated that the characterization provided by Askelson
et al. (2017) still applies. This includes strengths and limitations of on/off board systems identified
by Askelson et al. (2017). Some of the most promising systems identified in this task were utilized
during evaluation of DAA test methods.
A significant simulation effort to evaluate DAA system and Uncrewed Aircraft System (UAS)characteristics that impact maintenance of well clear was conducted. DAA characteristics that had
the greatest impact are range and Field of View (FoV). Update rate and latency impacts were notas dramatic, while horizontal and vertical resolution were the least impactful. UAS characteristics
that strongly impact maintenance of well clear are response time (i.e., the time required to initiate
a maneuver) and UAS speed.
An overarching test plan was developed. This test plan describes test locations/performers, datesof testing, DAA systems used in tests, overarching test objectives, individual test plan structure,methods for maintenance of safety during testing, data collection approaches, the structure of test
reports, and test metrics/artifacts.
Seven rounds of flight tests were completed. Important outcomes from these tests include:• A systematic approach for evaluating DAA systems• Identification of test metrics/artifacts• Test data collection methods/best practices• Methods for enabling flight test safety• Methods for executing both horizontal and climb- and descend-into encounters• Evaluation of DAA systems (especially the detection component of DAA systems)• Utilization of results in American Society for Testing Materials (ASTM) committees tosupport standards development
This effort involved a broad set of tasks designed to inform Federal Aviation Administration(FAA) regulations and industry standards regarding sUAS DAA systems. Through execution ofthese tasks and application of the numerous methods required to do so, the team has significantly
advanced sUAS DAA, which will enable more rapid integration of sUAS into the National
Airspace System (NAS)—especially for BVLOS operations.

xv

1

INTRODUCTION

The demand for Beyond Visual Line Of Sight (BVLOS) operations using small Uncrewed Aircraft
Systems (sUASs) is high. A major impediment to realization of these operations is the Detect And
Avoid (DAA) function. Several challenges exist for sUAS DAA. This effort explored many ofthese challenges.
The purpose of this project, “A18_A11L.UAS.22 – Small UAS Detect and Avoid Requirements
Necessary for Limited Beyond Visual Line of Sight (BVLOS) Operations: Separation
Requirements and Testing” (A18), was to inform Federal Aviation Administration (FAA)
regulations and industry standards regarding sUAS DAA systems. The A18 scope included thefollowing questions:• What are the use cases requiring DAA for BVLOS operations?
• What DAA systems are available, what are their capabilities and limitations, and are theymature enough to support BVLOS operations?
• What characteristics of DAA systems and Uncrewed Aircraft Systems (UASs) must be
considered to ensure maintenance of well clear status?
• How should sUAS DAA systems be evaluated to ensure they provide safe separation
services in the National Airspace System (NAS)?
• What is the recommended test method(s) to evaluate different DAA systems?
This project was conducted with the following tasks:
1. Development of an Operational Framework for sUAS BVLOS Operations—New Use
Cases, Industry Focus, and Framework Expansion
a. sUAS Use Case Data Collection/Analysis
b. Exploration of Framework Expansion
c. Revision of sUAS Radio Line Of Sight (RLOS) Boundary Recommendation with
Collected Use Case Data and for Expanded Operational Frameworks
2. Update of sUAS DAA Solutions Inventory
3. Coordination with Standards Agency to Establish Framework
4. Development of Separation Frameworka. Impacts of Characteristics of DAA System on Maintenance of Well Clear
b. Impact of Uncrewed Aircraft (UA) Characteristics on Maintenance of Well Clear
5. Development of a Testing Plan
6. Testing of a) the recommended DAA testing plan and b) candidate DAA systems
a. Conduct Flight Tests
b. Testing of Well Clear Definitionc. Testing of Separation Frameworkd. Testing of DAA Test Plane. Revision of Safety Management System (SMS)/Safety Risk Management Output
7. Final Report
This report is the product of Task 7: Final Report. To manage the length of this report, tasks forwhich separate reports were produced are summarized herein. For greater detail, the interested
reader may review those reports.

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

2

NON FLIGHT-TEST REPORTS

2.1 Task 1: Operational Framework
The Task 1 effort was to prepare a report on the Development of an Operational Framework for
sUAS BVLOS Operations. A report titled “Development of an Operational Framework for SmallUAS Beyond Visual Line of Sight (BVLOS) Operations—New Use Cases, Industry Focus, andFramework Expansion” was completed submitted to the FAA on 9 October 2019, and later revisedon 8 November 2019 (Cathey et al. 2019).
2.1.1 Objectives
The Cathey et al. (2019) report provided detail regarding the three distinct Task 1 subtasks:• Subtask 1a: sUAS Use Case Data Collection/Analysis• Subtask 1b: Exploration of Framework Expansion• Subtask 1c: Revision of sUAS RLOS Boundary Recommendation with Collected Use Case
Data and for Expanded Operational Frameworks
The sUAS Use Case Data Collection/Analysis effort provided an extended assessment of use cases
following a previous analysis (Cathey and Hottman 2017). The objective was to provide an updateto this previous work in the three areas noted above.
2.1.2 Methods
Potential sUAS use cases were gathered to assess prospective BVLOS applications that may use
DAA approaches. The uses cases gathered were focused on advancements or operations within
the last approximately two years. The goal was to identify any new use cases and to further fleshout some of the categories and applications. This period has been a growth and expansion stage
for the industry. Most use cases in the past few years have adjusted and abide by the FAA Part
107 rules that limit operations to altitudes of ≤ 400 ft. The previously-developed Cathey and
Hottman (2017) use case taxonomy was used as an initial basis for categorization of the different
types of flights/missions.
2.1.3 Summary of Results
The previously developed taxonomy was slightly revised to include twelve distinct categories:• Aerial Data Collection• Aerial Photography/Videography• Aerial Surveying/Mapping• Agriculture• Emergency Services• Flight Training/Education• Inspection• Marketing• Research• Search/Rescue• Surveillance/Monitoring, etc.
• Other
Forty seven different subcategories capture different aspects of these top-level uses. As noted,
low-altitude sUAS and Crewed Aircraft (CA) operations use cases and expressed industry needs
are evolving. New applications and reports appear almost daily. To refine the scope of this effort,
2

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

the approach was to explore use cases from when the Cathey and Hottman (2017) report was issued
until the release of the Cathey et al. (2019) report. The team further drilled down on key use cases
where there is broad industry need or application to further flesh out these front-line emerging
areas.
Specific examples of all of the various use case areas are included. There was no attempt made to
document every published application; the goal was to present representative applications withineach area. The examples were almost limitless. The use cases found and detailed once again
demonstrate the statement that UAS applications are only limited by one’s imagination. It wasalso clear from assessing the user operations that many applications do not fall cleanly within any
one set of particular categorization lines. For example, aerial data collection may also include
photography and mapping, and may be used for agricultural applications or inspections. The
applications and use cases often cover multiple areas during one mission. The key applications
using UAS include survey/mapping, imaging, environmental monitoring, patrol/security, disaster
response, precision agriculture, and reconnaissance/surveillance/intelligence. Almost all use
multiple elements and many are being fueled by better detector/sensor systems, improved data
handling, and artificial intelligence.
Also gathered from the use cases was the common adoption of some of these technologies so they
are not now new and cutting edge, but are common tools that are part of the normal work process.
This has been observed in many engineering assessment applications, marketing, public safety,
and environmental monitoring applications. The technology and uses are maturing. Publishedarticles tend to highlight specific new unique applications or first-time or unique events. A new
category, “other”, is a catch-all category that includes unique operations that do not fit well in anyof the other categories and are not vibrant enough to stand uniquely alone. One potential new use
case area is “Outdoor and Recreation”. This area will likely expand over time.
Minimal additional information was available concerning expansion of the operational framework
based on RLOS coverage. Again, with many users now operating under Part 107, the user basehas trended toward working within these set FAA structures. To that end, the use cases collected
and detailed, for the most part, did not provide any useful or actionable information related to
RLOS boundaries or issues. The literature search turned up no real references to limitations, lost
link, or communication issues. Articles and information focused on capabilities and not
limitations. Producers did not provide detailed data or information that provided additional insight.
Even with the available data, the testing performed up to the time of the Cathey et al. (2019) report
was predominantly executed within Line Of Sight and well within RLOS. None of the testing on
this task has pushed the limits to provide new or revised information regarding the limits. Without
specific targeted testing, expanding RLOS understanding is not possible. There is norecommendation at this point for the revision or expansion of the operational framework based on
the previous sUAS RLOS Boundary Recommendations based on the collected new and reviseduse cases or use case data.
An operational framework that defines the environment and conditions under which therecommended requirements will enable sUAS operations BVLOS is presented. Considerations
for BVLOS operations involve a number of interrelated elements that are needed for safe flight.
These elements result in potential constraints on the systems and operations. The three elementsof significant interest are:3

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

1) The conditions or locations in which one flies must be conducive to safe flight operations;2) The operator must operate in a safe fashion; and
3) The aircraft must be capable of reliable and safe BVLOS operations.
A series of assumptions and limitations that can facilitate BVLOS operations are provided. The
Exploration of Framework Expansion effort takes three things into consideration. First, the use
cases were collected. Second, potential changes due to newly gathered information related to
RLOS operations were gathered. Third, other external drivers such as new FAA regulations orindustry/community recommendations from the ASTM, the Science And Research Panel (SARP),
etc., were reviewed. Considering Part 107, the FAA’s direction toward allowing night operations
and flights over people, the SARP’s “Well Clear Hockey Puck”, and the like, minimal changes arerecommended from the original framework proposed. The trade space has not evolved
significantly since the initial Cathey and Hottman (2017) report. The framework may not beprescriptive nor does it include an exhaustive set of actions; the framework includes strategies thatcan build upon FAA and industry actions that should result in an increase in BVLOS flights in thenear term.
2.1.4 Lessons Learned
Below is a summary of all of the primary strategies and recommendations to help facilitate sUAS
BVLOS operations in the NAS [based upon Cathey et al. (2019)]:
1. Require a minimal set of limitations for BVLOS operations
a. Operating timei. Daytime
ii. Nighttime – following recent FAA 2019 Notice of Proposed RuleMaking(NPRM) for Night Operations
b. Meteorological conditions: Visual Meteorological Conditions
c. Altitude: ~400 feet Above Ground Level (AGL) (altitude modified to follow Part
107)
d. Overflighti. No densely populated areas
ii. Operations over people – following recent FAA 2019 NPRM for Operations
over People
e. Airport proximity limitations: greater than or equal to 5 miles
f. Critical operating limitations: greater than or equal to 3 miles of criticalinfrastructureg. Operational control: RLOS will determine distance; no daisy chaining of control
stationsh. Aircraft visibility: optimize color, lighting, and design for conspicuity
i. Ensure safe separations as defined by the SARP “Well Clear Hockey Puck” of
2,000 ft horizontal separation and 250 ft vertical separation
2. Develop a consensus-based and research-based design strategy
3. Utilize common phases of flight to facilitate recommendations and potential regulatory
input to the FAA
4. Develop a taxonomy and use cases that result in a manageable set of recommendations forregulatory and recommendation purposes

4

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

5. ASTM could lead the development of design and other data for BVLOS operations based
upon current and proposed research (ASTM is doing a portion of this in the ASTM DAA
Performance Requirements Task Group)
6. A DAA system—either airborne or ground-based—must be operational with the system
7. sUAS BVLOS operations in the NAS can take place without extensive and very expensive
infrastructure
8. International operations and requirements should be considered in formulating the BVLOS
requirements for the United States
9. Develop a more robust/realistic RLOS model for BVLOS that more accurately predicts theenvironment, links, and limitations
10. Utilize SMS to assess risk as BVLOS evolves
11. Utilizing candidate DAA and other enabling BVLOS technologies, develop, verify, andvalidate test methodologies for these current systems and apply this to future systems
12. Anticipate that the near future will demand autonomous BVLOS without a human pilot
due to the large number of anticipated flights within an airspace
2.2 Task 2: Update of sUAS DAA Solutions Inventory
2.2.1 Objectives
The objective of this task was to track developments associated with DAA systems designed towork with sUAS.
2.2.2 Methods
Data regarding sUAS DAA systems were gathered through extraction of information provided
publicly (e.g., at presentations or on web sites) and through interactions with industry. The annual
Association for Uncrewed Vehicle Systems International XPONENTIAL conference was a usefulvenue for collecting information regarding evolving DAA systems and systems that could be used
as components of DAA systems.
2.2.3 Summary of Results
Results of this task indicated that the characterization of DAA systems provided by Askelson etal. (2017) for the project “A2_A11L.UAS.22 – Small UAS Detect and Avoid Requirements
Necessary for Limited Beyond Visual Line of Sight (BVLOS) Operations” (A2) still applies.
Askelson et al. (2017) identified the major steps/components of DAA as Detect, Track, Evaluate,
and Maneuver (DTEM). The characteristics of DAA systems relate to DTEM and can beorganized using the categories (Askelson et al. 2017):• Modality: Cooperative vs. Noncooperative (D and T)• Sensor Type: Active vs. passive, non-cooperative sensing method (D)• Location: On/off board (DTEM)• Degree of Autonomy: Human In The Loop, Human Over The Loop, etc. (DTEM)
It is noted that non-cooperative sensing methods that have been identified for sUAS DAA systems
include:• Radar• Light Detection And Ranging• Electro-Optical/InfraRed (EO/IR)• Acoustic

5

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

The strengths and limitations of on/off board systems identified by Askelson et al. (2017) continue
to apply. Off board systems provide required performance but are costlier and commonly require
more personnel to operate. On board systems are much less expensive and resource intensive, but
struggle to provide the required performance.
Because the set of DAA systems and supporting technologies is constantly evolving, a list is notprovided herein. Some of the most promising systems were utilized during evaluation of DAA
test methods (Section 3). Types of DAA systems utilized during tests include those that leveraged
cooperative [Automatic Dependent Surveillance-Broadcast (ADS-B)] and non-cooperative (EO
and radar) sensors. Characteristics of DAA systems utilized during testing are provided in Section
3 of this report.
2.2.4 Lessons Learned
Two key take-aways for this task are:
1. The technologies that can be applied to sUAS DAA are evolving rapidly.
2. Some entities operate in a “silent” mode in which they reveal very little regarding theirtechnologies, which is likely driven by a desire for competitive advantage.
2.3 Task 3: Coordination with Standards Agency to Establish Framework
2.3.1 Objectives
The objective of this task was to establish a standards agency framework to address sUAS DAA
and to translate lessons learned from this research into industry standards. ASTM was identified
as the proper organization for this task.
2.3.2 Methods
The A18 team coordinated with others within the industry to establish the required framework. In
fact, others within the industry had already developed a foundation for this. Thus, the A18 teamsupported the efforts that were already underway and provided Subject Matter Expertise (SME).
The primary A18 team members who supported this task were Kansas State University (KSU) andthe University of North Dakota (UND).
2.3.3 Summary of Results
Within ASTM, two working groups were established:
1. WK62668: Detect and Avoid Performance Requirements Task Group
2. WK62669: DAA Test Methods Task GroupThe DAA performance group produced a standard that was published in July 2020 (ASTM 2020).
KSU provided significant support to development of ASTM (2020), including participating inworking group meetings and completing writing tasks. During development of ASTM (2020), key
elements that were identified include:
1. Identification and significance of test artifacts.
2. The roles of flight test and simulation in gathering DAA system performance data.
3. The linkage between flight test and simulation test methods.
While A18 team engagement was important to development of ASTM (2020), its engagement iseven more significant with the DAA test methods group. In addition to SME support for thisgroup, one of the A18 Principal Investigators (Askelson) also serves as a co-chair for this group.
Although the DAA test methods group has not yet reached the ballot phase for its draft standard,
it has made significant progress. Many of the findings from A18 have supported the DAA test
6

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

methods group, including test procedures and metrics. These are described in more detail in
Section 3.
2.3.4 Lessons Learned
The ASTM DAA working groups have driven significant advancements. These include
exploration of metrics for evaluation of DAA systems, an enhanced understanding of the DAA
steps/timeline, and requirements of entities demonstrating compliance. Some of the greatest
challenges, which likely relate to most standards, have been:• Identification of proper metrics for required performance• Identification of metrics used to demonstrate performance• Determining the “mix” of simulation and testing (both bench and field testing)• Delineating requirements that enable applicant flexibility and broad application given thevariability in NAS traffic characteristics (types of intruders, density of intruders, etc.)
Additional information is provided in the following subsections.
2.3.4.1 Metrics
The significance and type of data were an initial stumbling block within the working groups.
While performance standards are generally agnostic to any specific technology, the types of datacollected by a given sensor may be more specific – e.g., data collected with an optical sensor may
differ from that collected with an airborne radar. Initial discussions focused on ensuring that test
methods, test metrics, and other sources of data remained agnostic of any particular technology.
2.3.4.2 Simulation and Flight Testing
Early in the development of the standard, it was noted that there was benefit to utilizing simulation
for modeling encounters, as simulations enable relatively inexpensive execution of a very large
number of simulations and evaluation of performance for encounters that would be hard to conductsafely through flight tests. This is especially true, as the repository of airspace encounter models
developed by MIT Lincoln Labs (Weinert 2021) provides a starting point for defining simulated
encounters for use in determining DAA system performance. However, it was also noted thatflight testing can be used to validate assumptions used for encounters that may be devised toevaluate unique characteristics of given DAA systems and sensors. Discussions centered upon
how and when simulation and flight testing should be employed and at what point each is mostpractical for a system manufacturer and/or integrator.
Similar to the identification of the roles of flight test and simulation, the test methods group
discussed how to link test data gathered from flight test and simulation. Discussions regarding thelinkage of flight test and simulation test methods are still ongoing.
At the time of this report, the working group is adjudicating proposed test methods at test artifacts
internally. This process is intended to identify test methods that correlate to individual test
requirements, resolve conflicts that may exist regarding the wording of test requirements, andultimately prepare the standard for initial ballot within the ASTM F38 committee.
2.4 Task 4: Separation Framework
2.4.1 Objectives
The objective of this task was to determine the characteristics of DAA systems and of UAS thatmust be considered to ensure maintenance of well clear status. Answering this question provides

7

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

insight into questions regarding the ability of existing DAA systems to enable maintenanceof well clear and regarding evaluation/test methods for small UAS (sUAS) DAA systems.
2.4.2 Methods
An automated approach to simulating UA encounters with CA was developed based upon previous
simulation work described by Askelson et al. (2017). Encounter simulations involved a simulated
CA intruder (a Cessna 182T turbo) and either a fixed-wing or multi-rotor sUAS. The baselinecharacteristics of the simulated sUAS were designed to align with typical performance
characteristics of the respective types of sUAS. Encounters were divided into three sets: coaltitude, climbing/descending, and stress. Stress encounters involved the intruder flying in two
different directions relative to ownship while maintaining a minimum safe horizontal separation
distance that would not trigger a DAA alert before turning (left- or right-) into the ownship’s pathat the last minute. All encounter geometries were conservatively designed to be collisiongeometries (with no action taken, the aircraft would collide).
Impacts of DAA characteristics and sUAS characteristics were evaluated. For DAA
characteristics, four sensors classes—control sensor (theoretically ‘perfect’ sensor), EO/IR, radar,and acoustic—were considered. For these sensor classes, seven characteristics were evaluated:
latency, update rate, range, horizontal and vertical FoV, and horizontal and vertical resolution.
Impacts of variations in these characteristics were evaluated by ‘sweeping’ through values for one
characteristic while holding the values of the other characteristics constant.
Dependence upon sUAS characteristics was explored by examining variations with maximum andminimum indicated airspeed, maximum climb and descent rate, maximum yaw rate/bank angle,
and pilot response time. As with DAA system characteristics, impacts were explored by varying
the values of one characteristic while holding the values of the other characteristics constant.
The risk ratio, which is the probability of an event with a DAA system divided by the probabilityof an event without a DAA system, is the metric used herein to evaluate performance and risk.
The risk ratio has been adapted by the ASTM WK62668 Detect and Avoid Performance
Requirements Task Group as the fundamental metric for evaluating DAA system performance.
Due to the nature of the encounter sets used [all encounters result in a Near Mid-Air Collison(NMAC) when run unmitigated], herein the risk ratio is the number of encounters for which wellclear or NMAC distances using the currently loaded aircraft or sensor configuration could not be
maintained vs. all encounters in a simulation sweep. Thus, encounters are not weighted according
to likelihood of occurrence. It is further noted that because risk ratios are commonly determinedusing a larger set of simulations than executed in this task, the results from this task are commonlyreferred to as ‘estimated risk ratios’.
2.4.3 Summary of Results
An example of results is provided in Figure 1. As shown in Figure 1, estimated risk ratio drops
with increasing detection range, as expected. The lowest value for estimated risk ratio is driven
by the limited FoV of the assumed DAA sensor [cf. Kaabouch et al. (2020) for details]. Distances
to the well clear volume, for encounters in which well clear is retained, increase and then stabilize
with increasing detection range. For the assumed conditions and these simulations, maintenanceof well clear starts to be possible for detection ranges in the 7000-8000 ft range.

8

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 1. Example separation framework results for a radar-based DAA
system [cf. Figure 19 of Kaabouch et al. (2020)]. Lefthand y-axis is distanceto well clear volume (minimum, mean, and maximum) for encounters inwhich well clear was maintained and righthand y-axis is estimated risk ratio.
X-axis is radar detection range in 104 ft. Results are for co-altitudeencounters, an assumed multi-rotor aircraft, and a radar-based DAA systemwith limited FoV in both the horizontal and vertical directions. For further
details, see Kaabouch et al. (2020).

The DAA characteristics that had the greatest impact on estimated risk ratio were range and FoV.
Update rate and latency impacts were not as dramatic, while horizontal and vertical resolution
were the least impactful.
EO/IR had very low success in maintaining well clear, to the point that baseline performance wasless sufficient as a tool for well clear maintenance than as a tool for avoiding NMACs. In the
EO/IR range test, regular success for well clear did not start occurring until roughly 7,000-8,000ft detections. This is large and double the average range of sensors on the market at the time ofthis evaluation (4,000 ft baseline). The acoustic sensor class required detections at greater ranges(10,000 ft and greater) to maintain well clear status owing to the slower speed of sound. The radar
sensor class enabled maintenance of well clear at similar detection ranges as the EO/IR sensor
class.
Radar appears to be the best sensor class of the three under consideration; however, it is alsoarguably the most challenging for sUAS, especially copters given the Size, Weight, and Power(SWaP) characteristics of the sensor. It may be a reasonable candidate, especially if commercial
systems improve, for midsized and larger fixed-wing vehicles.
Given no sensor limitations, the greatest defense a UAS has to maintain well clear is a vigilant andfast-responding pilot. A reasonable delay/response time for a pilot of average skill is 5 s.
9

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Reduction of maneuver delays, even by a couple of seconds, can have an impact that is larger thanmost characteristics evaluated in these simulations. Moreover, operating a faster UAS enables
significant risk reduction. Such risk reductions occur for relatively low UAS performance gains,which is encouraging for average aircraft in the UAS commercial fleet since normal performanceis enough to garner the best gains in a DAA context.
2.4.4 Lessons Learned
The separation framework effort produced results that:• Elucidate the most important DAA system characteristics for maintaining well clear—detection range and FoV• Illustrate the most important UA characteristics for maintaining well clear—(pilot)
response time and aircraft speed• Underscore the significant challenges associated with onboard DAA systems—providing
the required performance given SWaP limitations• Indicate that many current onboard DAA systems may be better considered to be NMAC
and/or collision-avoidance systems given their performance (e.g., detection range and
FoV)
2.5 Task 5: Test Plan
2.5.1 Objectives
The high-level objective for this task was to provide an overarching test plan for flight tests events
conducted during A18.
2.5.2 Methods
Previous research and experience from previous test campaigns were used to develop an
overarching test plan.
2.5.3 Summary of Results
Test locations/performers are described. For these, dates of testing and high-level test
characteristics are provided. Seven rounds of flight tests were completed. Tests included
evaluation of individual components (e.g., the Detect step) and evaluation of total DAA system
performance, including metrics identified in the existing sUAS DAA performance standard(ASTM 2020).
DAA systems used in testing are described. Four different DAA systems were used, including
off-board (ground-based) radars, on-board radars, on-board electro-optical sensors, andcooperative (ADS-B) sensors.
A description of overarching test objectives is provided. This includes how these objectives relate
to evaluation of DAA element performance, to the separation framework (validation of simulations
performed under Task 4 of A18), and to ASTM standards regarding sUAS DAA.
The structure of a test plan is delineated, including descriptions of test plan elements. Testing isgrounded in a geometric approach, which leverages varying encounter geometries (horizontal andvertical) to evaluate DAA system performance. Other variables to be considered when testing
DAA systems include trajectory type (straight vs. curved, collision vs. non-collision, etc.), intruder
speed, sUAS speed, and clutter severity.

10

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Maintenance of safety during testing is paramount. Approaches for ensuring safety during bothhorizontal and vertical encounters were developed and tested. In addition, example tests scripts
were developed. For some tests, different scripts were used for different roles for each encounter,
which can be helpful with eliminating confusion and enhancing efficiency in the field.
Data collection was enabled through either data collection sheets or digital collection tools. Twotypes of data collection sheets were used: static for collection of metadata that are constant for
multiple encounters and an encounter sheet that is used for an individual encounter.
The structure of test reports is delineated. This includes types of information that should be
provided. Information types can be divided between test structure/design and test results. Metrics
were described, including metrics utilized by ASTM (2020) (e.g., risk ratio), metrics that provide
an overview of overall encounter performance [e.g., Closest Point of Approach (CPA)], andmetrics that provide information regarding performance during the DTEM components of DAA.
2.5.4 Lessons Learned
While the team learned a great deal as the overarching test plan and flight tests co-evolved (e.g.,
metrics to be used, methods for maintaining safety, etc.), perhaps the most important lesson was
the overall approach to testing. DAA system testing could be conducted by emulating conditions
experienced with real-world use cases (e.g., pipeline inspection) or by systematically varying
factors that affect DAA system performance (e.g., encounter angle, intruder speed, etc.). Because
use of real-world use cases produces results that likely could not be easily transferred to otherenvironments/use cases, the latter approach is recommended and was developed in the overarching
test plan. However, testing using the conditions associated with specific use cases could enableacceptance of certain DAA systems for those use cases (e.g., shielded operations).

3

FLIGHT-TEST REPORTS

3.1 UAF March 2019
3.1.1 Objectives
The objectives of this flight test were:• Evaluate when alerts are provided from DAA systems versus human observers.
• Evaluate DAA systems in the context of real-world use cases (e.g., emulated pipeline
inspection and survey).
3.1.2 Dates/Schedule
Tests were conducted during the week of 25-29 March 2019. The planned daily schedule isprovided in Table 1.

11

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Table 1. Planned daily schedule for the March 2019 UAF ACUASI campaign.
Local Time [Alaska
Daylight Time]
8:00-9:00
9:00-9:45
9:45-10:00
10:00-10:30
10:30-2:25
2:25-2:45
2:45-3:30
3:30-4:00

Activity
Mission overview
Travel to range
Range and safety overview
Ground checks (communication, pre-flight)
Test flights
Pack up
Drive to UAF
Debrief

3.1.3 Location
Tests
were
conducted
at
the
Poker
Flat
Research
Range(https://www.pfrr.alaska.edu/content/welcome-poker-flat) and were conducted by the Universityof Alaska Fairbanks (UAF) Alaska Center for Unmanned Aircraft Systems Integration (ACUASI).
The Poker Flat Research Range is illustrated in Figure 2.

Figure 2. Overhead view of the Poker Flat Research Range.

3.1.4 System Tested
Systems tested during this campaign were Echodyne ground-based radars, Iris EO system, and
ADS-B.
12

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3.1.5 Test Plan Overview
The test plan was comprised of:• Testing of detection component of DAA systems• Encounters that were scenario-driven (use-case based—pipeline inspection and survey)• 5 test profiles• Ownship UA• Varied intruders (fixed- and rotary-wing CA and UA)
3.1.6 Sample Test Cards
A sample test card used during this campaign is shown in Figure 3. It is noted that test cards
evolved during A18 as the team learned from previous test campaigns.

13

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 3. Example test card used during the March 2019 UAF ACUASI campaign.

3.1.7 Summary of Results
Results indicated that the DAA systems commonly detected intruders before ground observers,
though not always. If data were available, ADS-B outperformed ground observers, as expected.
3.1.8 Lessons Learned
The primary lessons from this initial flight test campaign were:

14

THIRD PARTY RESEARCH. PENDING FAA REVIEW.
••

Evaluating DAA systems by systematically varying factors that affect their performance isgenerally favored relative to evaluating them by emulating conditions experienced withreal-world use cases (e.g., pipeline inspection).
Greater distances are needed to ensure safe separation of aircraft during testing.

3.2 NMSU July 2019
The New Mexico State University (NMSU) conducted a number of DAA flight tests addressing
part of Task 6, Subtask 6a, conducting flight tests of a candidate DAA system. Multiple different
flight tests of various DAA systems were part of the testing conducted. A report titled, “A18 –Small UAS Detect and Avoid Requirements Necessary for Limited Beyond Visual Line of Sight(BVLOS) Operations: Task 6 NMSU July 2019 Testing of Candidate DAA System Test Report”
was submitted to the FAA on 5 December 2019, and after multiple exchanges with the FAA, arevised report was submitted on 16 April 2020 (Cathey et al. 2020). This report documented theplanning, testing, and results of flight testing at the NM UAS Flight Test Site in July 2019 at the
Jornada Experimental Range, North of Las Cruces, NM. The following sections provide asummary of this flight testing.
3.2.1 Objectives
The approach was to set up various encounters of a UAS equipped with an onboard DAA systemwith different crewed and uncrewed intruders. A focus of this testing was to assess the sensor
system for sensor performance, rather than a complete DAA system that included maneuvers. The
testing approach was to present various intruder vehicles in different geometries. The testing
included two different CA, a light sport and an ultralight, as well as a small UAS. The DAA
system was placed on both a multi-copter as well as a fixed wing aircraft.
3.2.2 Dates/Schedule
The overall arc of this round of testing took place over a four-day period from Monday, July 15,
to Thursday, July 18, 2019. The four days were set up with the system checks to start on day one,and then different encounter scenarios on the follow days. A short summary of the events is asfollows:• Day 1 (15 July 2019) – Review of mission plan and check flights of vehicles• Day 2 (16 July 2019) – Multi-copter with Iris system installed, flights at different altitudes,
and light sport intruder encounters at various angles and crossing patterns• Day 3 (17 July 2019) – Fixed wing with Iris system installed, flights at different altitudes,
and light sport intruder encounters at various angles and cross patterns• Day 4 (18 July 2019) – Multi-copter with Iris system installed, flights at different altitudes,
ultralight intruder encounters at various angles and cross patterns, and pop ups with small
UAS
3.2.3 Location
The testing took place at the Jornada Experimental Range near Las Cruces, NM. This location is
on gated access United States Department of Agriculture property where the NMSU Flight Test
Site regularly performs operations. This is a remote area with only a limited number of personnel
in the area. The location is shown in Figure 4.

15

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 4. Test location, just North of Las Cruces, NM, for the July 2019 NMSU
campaign.

This location has been used by the NMSU UAS Flight Test Site team as a primary test location.
It is just to the West of the White Sands restricted airspace. There are limited overhead flights ofmanned aircraft. The area has a dirt airstrip that was used for flights, provided a visual geometricmarker for the intruder pilots to align encounter runs, and as an emergency landing location if
needed for the manned aircraft (Figure 5).

Figure 5. Dirt runways on the Jornada Experimental Range.

16

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

NMSU has a trailer on site for flight control. Because of the large number of participants andobservers, the team added a second “personnel trailer” with air conditioning, power, tables, chairs,
coffee, microwave, mini-fridge, and a Wi-Fi hot spot. Sunshades, generators, and designatedsafety areas for personnel were provided. Images of the test setup location are provided in Figure
6.

Figure 6. Trailer setup.

3.2.4 System Tested
An airborne visual system from Iris Automation called Casia was used on the UAS’s for thistesting. The Casia DAA hardware is shown in Figure 7. Previous testing of this system wascompleted by the University of Alaska Fairbanks team. The planning to test this specific system
was coordinated with the FAA in advance. The system consists of the Casia Module (processingunit) and a camera. The system is integrated into an aircraft and provides situational awarenessvia DAA technology (https://www.irisonboard.com/). The system tested was their originallyreleased “standard design”, and has a 1.6 MegaPixel (MP) camera. Iris advertised that the system
had a 65° x 50° field of view, 416 m (1,365 ft) average detection range, and a system mass of 73
g.

17

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 7. The Casia system.

It is important to note the advertised horizontal FoV of 65°. The specific test planning was
designed around testing within, to the edges of, and beyond this FoV. This is clear in thegeometries of the testing shown in the testing approach section below. This is noted here because
Iris has recently released an improved system with a wider FoV and greatly increased detection
range. It has a much-improved camera, updated software, and a few other improvements. For allof these tests, the following system was used:• System Description – Original Casia System• Software – Software Version 20.1
• Camera – 1.6 MP, Standard Camera
A number of different flight assets were used as part of these tests. Iris Automation brought
primary and back up aircraft for both fixed wing and multi-copter platforms. The two different
platforms used by Iris Automation that were equipped with the Iris Casia DAA were:• Ready Made RC (RMRC) Anaconda (fixed wing)• Straight Up Imagery (SUI) Endurance (multi-copter)
NMSU flew three different intruders against the Casia DAA system. The first was a manned lightsport aircraft and the second was an ultralight aircraft. Both of these vehicles were flown with apilot and co-pilot that served as an onboard Visual Observer (VO). The last day of testing, the
NMSU team used a small multi-copter to simulate popup encounters. The three different platforms
used by NMSU were:• CTLS Light Sport aircraft• Spyder Ultralight aircraft• 3D Robotics Solo
Images of all of these vehicles are provided in Figures 8-11.

18

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 8. NMSU’s CTLS light sport aircraft.

Figure 9. NMSU’s Spyder ultralight aircraft.

19

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 10. Iris’s RMRC Anaconda (fixed wing).

Figure 11. Iris’s SUI Endurance (multi-copter) and NMSU’s 3D Robotics Solo.

20

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3.2.5 Test Plan Overview
The plans for the initial round of testing were provided to the FAA in two complimentary formats
for review. The first was a text description of the “DAA System Encounter Scenarios” and thesecond was a PowerPoint presentation titled “A18 Encounters rev1”. The “DAA System
Encounter Scenarios” had the goal of mapping detection range under various encounter scenariosthat included Head On, Overtaking, Cross FoV, and Pop-Up Encounters.
The overall approach utilized the following:• CA at 500 ft elevation• Lateral separation of (generally) ~100 ft• UA with Iris Onboard will be at 100 ft, 200 ft, and then 400 ft elevation• UA with Iris Onboard flown along a straight track (direction determined on flight day –cardinal direction, along dirt runway, or in/out of the wind) nominally 100 ft to 500 ft long(aircraft dependent) with a minimal loop at the end of the track
3.2.5.1 CTLS and Multi-Rotor Tests
The conditions for these tests were:
1) Circle Pattern for system check (start of each flight day)a. CA
i. CTLS
ii. ~100 kts
iii. 500 ft altitude
iv. Several circles of varied diameter to ensure system checkb. UA
i. Multi-copterii. 5 m/sec along encounter lineiii. 100 ft altitude
2) Head-On Encounters
a. CA
i. CTLS
ii. ~100 knotsiii. 500 ft altitude
iv. Flight along one side of runway (ex. only on the North or West sides) as
defined by encounter direction based on conditions
b. UA – Multi-copteri. 5 m s-1 along encounter lineii. 100 ft altitudeiii. Flight along one side of runway (ex. only on the South or East sides)c. Profilei. View head on and trigger
ii. Stop, rotate vehicle 180°, and procced again at 5 m s-1
iii. Attempt to watch vehicle exit FoV
3) Possible repeat the above with approach from opposite/other direction
4) Repeat with different oblique angles
a. Encounters at 10°, 20°, 30°, and 40° off direct head on5) Repeat steps above with UAS at 200 ft altitude (possible that only a portion of these steps
will be repeated at this altitude)
21

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

6) Repeat steps above with UAS at 400 ft altitude
7) Possibly repeat selected steps above with multi-rotor in fixed position (i.e., no 5 m s-1
forward speed)8) Overtaking Encounters
a. CA
i. CTLS
ii. ~100 knotsiii. 500 ft altitude
iv. Flight along one side of runway (ex. only on the North or West sides) as
defined by encounter direction based on conditions
b. UA
i. 5 m s-1 along encounter lineii. 100 ft altitudeiii. Flight along one side of runway (ex. only on the South or East sides)c. Profilei. View 180° out from head on
ii. Trigger and attempt to watch vehicle exit FoV9) Repeat with different oblique angles
a. Encounters at 10°, 20°, 30°, and 40° off direct head on10) Repeat steps above with UAS at 200 ft altitude (possible that only a portion of these steps
will be repeated at this altitude)11) Repeat steps above with UAS at 400 ft altitude
12) Possibly repeat selected steps above with multi-rotor in fixed position (i.e., no 5 m s-1
forward speed)13) Cross FoV
a. CA
i. CTLS
ii. ~100 knotsiii. 500 ft altitude
iv. Flight at various distances beyond UAS track line
1. 500 ft
2. 1,000 ft
3. Others possible
b. UA—Multi-copteri. 5 m s-1 along encounter lineii. 100 ft altitudeiii. Flight along one side of runway (ex. only on the South or East sides)c. Profilei. View head on and trigger
ii. Halt vehicle and with no rotation, attempt to watch vehicle exit FoV14) Repeat steps above with UAS at 200 ft altitude
15) Repeat steps above with UAS at 400 ft altitude
16) Possibly repeat selected steps above with multi-rotor in fixed position (i.e., no 5 m s-1
forward speed)
Additional retests can be performed as desired.

22

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3.2.5.2 CTLS and Fixed-Wing UAS
The conditions for these tests were:
17) Circle pattern check (see 1 above)18) Repeat all or selected steps 2 to 16 using CTLS as CA and small fixed-wing UAS with Iris
onboard3.2.5.3 Ultralight Spyder and both Multi-Rotor and Fixed wing UAS’s
The conditions for these tests were:
19) Circle pattern check (see 1 above)20) Repeat selected steps 2 to 16 using Ultralight Spyder as CA and multi-copter and small
fixed-wing UAS with Iris onboard
3.2.5.4 Pop-up Encounters
The conditions for these tests were:• Pop-up sUAS will be used at the end of each day or during periods where there are issues
with the CA• Various locations within the detection system FoV will be employedo UAS altitudes of 100 ft, 200 ft, and 400 ft
o Pop-up ranges from 500 to 1000 ft (or more) at various positions within thedetection wedge
3.2.6 Sample Test Cards
As noted above, the planning was provided to the FAA in two documents, the “DAA System
Encounter Scenarios” and the PowerPoint presentation “A18 Encounters rev1”. Initial test card
formats provided to the FAA are shown in Figures 12 and 13.

23

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 12. Example test “metadata” card for the NMSU July 2019
campaign.

24

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 13. Example test script/plan for the
NMSU July 2019 campaign.

The test card format evolved over time. The ones above are not in the ‘Test Card’ format which
was developed later for this project. Similar information was included for the test planning. This
25

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

round of testing was used as part of the basis for the later development of the standard test card
format developed by the team for later test campaigns.
The encounter geometries were based on the Iris Casia’s systems 65° FoV. The desire was to fly
the UAS along a set line and then have the intruder aircraft enter the field of view at different
angles relative to this flight path. The encounters were set to vary in 10° increments across theFoV from opposite directions. As the intruder entered the FoV, the UAS was to turn and watch
the intruder exit the FoV to obtain an additional data point. A schematic of the encounter geometry
with overlays of the planned flight location is shown in Figure 14.

Figure 14. Encounter geometry withoverlays of the planned flight location.

Taking into account the Iris system’s detection range, the desire was to have the intruder alignedon the desired encounter geometry outside of the systems detection range. Intruder runs were set
to be on course at ~2 mi out, well beyond the detection range (noted by the manufacturer as 416
m or 1,365 ft average detection range) of the system. Maintaining the exact geometries was notparamount since the desire was to prepare a system detection map. Focus was on detection andnot avoidance maneuvers. The overall encounter geometry mapped onto the flight test location isprovided in Figure 15, while Figure 16 shows an example multi-copter UAS flight path.

26

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 15. Overall encounter geometry mapped onto the flight test location.

Figure 16. Example multi-copter UAS flight path. The fixed wing UAS pathdoubles this length and loops at the ends.

3.2.7 Summary of Results
Detailed flight logs and plots for each flight day are included in Cathey et al. (2020). To provide
a sample of the types of information presented, a few representative examples of the flight-testing
information gathered are provided. A 1 and 2 mi square box was set up as reference for this testingwith the 2 mi range as a point to ensure that the intruder was aligned on the intended encounter
angle outside the detector range, and the 1 mi box defining the points where the encounters started.
Plots of flights conducted on the first day of testing are provided in Figures 17 and 18.

27

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 17. DAA testing flights on day 1—full flights.

28

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 18. DAA testing flights on day 1—near testing location.

An example of the flight altitude profiles is provided in Figure 19, with both the encounter periods
and DAA triggers highlighted. Two images of an encounter are provided in Figures 20 and 21.

29

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 19. Day 1 flight altitudes. Encounter crossings and detections are highlighted.

30

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 20. Crossing encounter DAA test.

31

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 21. DAA test encounter after turn had been initiated.

Multiple pages of encounters were captured in a spreadsheet format that included the following
information:• Date• DAA System• DAA System Vehicle• Intruder• Time Intruder enters 1 nm box• Time Intruder exits 1 nm box• Encounter Type• Encounter Geometry• Flight Direction• DAA Trigger Detection Time
• DAA Orientation 0=along flight path, CW angles• Encounter Angle• DAA Vehicle Lat at detect time
• DAA Vehicle Lon at detect time
• DAA Vehicle altitude at detect time
• Aprox. DAA Vehicle alt above ground• Closest Intruder Time Stamp• Intruder Vehicle Lat at detect time
• Intruder Vehicle Lon at detect time
• Intruder Vehicle altitude at detect time
• Detection Range (km)• Detection Range (ft)• Notes32

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

These files are not provided within this report because of the size of the files.
A mapping of the detections by day is provided in Figure 22. These have all been nominalized to
distances and encounter angles as if the DAA system was stationary and represents the origin point.
The distances at which the DAA system detected the intruder are shown. An overlay on this showsa function of the encounter angle. Detections along the flight track, represented by the 0° angle,
represent most of the detections. The other encounter angles show clear detections mostly alongthe 10°, 20°, and 30° radials on either side of this center line. A few detections occurred outsidethese ranges as well. Essentially, Figure 22 is a map of all of the detections shown in summary
tables. The crossing patterns, circular detection patterns, and pop-up flights are included but are
difficult to identify with this full mapping.

Figure 22. Mapping of Iris DAA detections based on encounter angle and distance.

An added complication was that the UAS was indicating AGL altitudes and the CA provided Mean
Sea Level (MSL) altitudes. A post flight review of the vertical separation was completed to assess
the vertical separation at CPA. a vertical separation distance of 250 ft was set for the testing (nomargins were used—just 250 ft). It was concluded that the flights did violate this verticalseparation distance. A summary of vertical separations at CPA each day for the is:33

THIRD PARTY RESEARCH. PENDING FAA REVIEW.
Day 1 – CTLS versus multi-copter
32 total passes (13 passes were < 250 ft)
435.92 ft Max
291.66 ft Average
163.72 ft Min
Day 2 – CTLS versus fixed wing
24 total passes (5 passes were < 250 ft)
467.29 ft Max
324.26 ft Average
189.84 ft Min
Day 3 – Spyder Ultralight versus multi-copter
4 total passes (3 passes were < 250 ft)
253.23 ft Max
231.88 ft Average
211.19 ft Min
Summary of all Crossings
60 total passes (21 passes were < 250 ft)
467.29 ft Max
300.72 ft Average
163.72 ft Min
The less than 250 vertical feet violations are a lesson learned for future testing. Most of theseclosest points of approach had significant lateral separation (> 1,000 to 1,500 ft) as well, but were
within 2,000 ft lateral separation.
This testing design provided a large number of encounter geometries and configurations to test theIris DAA system. The different flight assets provided a number of different presentations for
detection. It was noted that some false triggers occurred when the system was first turned on,
during the flights at lower altitudes, and on other occasions. The flights at higher altitude had
fewer false detections. A number of times the aircraft was not oriented along the desired track,and this resulted in fewer detections than could have been obtained with the testing. This also wasa lesson learned.
The various encounter scenarios are different and it is difficult to compare results for different
aircraft in a head-on approach, a cross-FoV approach, and a pop up encounter. Each represent
different aircraft, geometries, and conditions. As noted, the flight altitude of the UAS carrying the
DAA system has an impact as well. The following are a few summary points for detection
distances:• Multi-copter with CTLS in a counter clockwise circle around the flight area – Maximum
distance of ~2,118 ft and minimum distance of ~1,104 ft, with an average detection
distance of ~1,766 ft.
• Multi-copter with CTLS in head on runs – Maximum distance of ~3,169 ft and minimum
detection distance of ~362 ft, with an average detection distance of ~1,603 ft.
• Multi-copter with CTLS in across the field of view runs – Minimum distance of ~869 ft
and a maximum of 2,556 ft, with an average of 1,844 ft detection distance.
• Fixed Wing with CTLS in head on runs – Detections were in the ~492 ft to over ~1,732 ft
range. Most were in the ~750 to ~1,000 ft range.

34

THIRD PARTY RESEARCH. PENDING FAA REVIEW.
•

Fixed Wing with CTLS across the field of view runs – Detections between ~1,430 and over
~2,870 ft. Most were in the > 2,000 ft range.
• The Iris system had no issues identifying the manned Spyder ultralight at ranges beyond
2,500 ft from the DAA system location.
• Multi-copter with Spyder in in a clockwise circle around the flight area – Detections ranged
from ~2,160 ft to the East, ~2,030 ft to the West, ~2,325 ft to the North, and ~1,560 ft to
the South. The furthest distance in the “fishhook” to the south was ~2,620 ft.
• Multi-copter with Spyder in in a counter clockwise circle around the flight area –Detections ranged from ~1,450 ft to the East, ~1,910 ft to the West, ~1,815 ft to the North,and ~1,080 ft to the South.
• Multi-copter with Spyder – Detection distances were between ~1,080 ft and up to ~2,620ft.
• Multi-copter with Spyder in across the field of view runs – The Spyder was detected at
ranges between ~485 ft to ~1,477 ft, with an average distance of 939 ft.
• All of the sUAS pop up flights were detected at ranges from ~90 ft to ~415 ft, with anaverage of ~261 ft, and generally at altitudes comparable to the Iris aircraft.
In tabular form, a summary of the DAA system encounter parameters is provided in Table 2.
Table 2. Summary of DAA test encounter parameters for the July 2019 NMSU campaign.

As noted, this flight test report was provided to the FAA for review. Comments and feedback
were provided to the authors. After completion of this report, two additional supplemental reports
were provided to the FAA. The supplemental reports include “Casia Detect and Avoid System
Stress Test Results Summary” and “July 2019 Iris DAA System Testing Results Comparison”(dated 18 February 2020 – revised 16 April 2020) for further performance assessment. These are
not fully repeated here, and can provide some additional information related to posttest assessment.
Key findings are provided in Table 3.

35

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Table 3. Iris detection results for July 2019 NMSU campaign.

The FAA reviewers desired additional insight into the internal sensor processing. While this was
not an intent of this effort, manufacturer-provided system information was supplied to the FAA
for review.
3.2.8 Lessons Learned
Numerous lessons learned were gained as part of this testing. These lessons were shared within
the A18 research team and with the FAA. Some of these lessons learned resulted in onsite changes
in real time to the testing process. Some were beneficial toward making changes in future testing.
Some of these changes were immediately implemented in the flight testing that followed in Alaska.
This bullet list captures the post testing list of lessons learned:• Better defined mission approach in advance:o Approach was developed in advance and reviewed between Iris and NMSUo Review by all stake holderso Revise as necessaryo Define optionso Ensure flexibility and fall back plans (Option B, C, etc.)• Mission statuso Highlight status and where in the sequence during the missiono Call out at start of various phases (e.g., noting where in script)• Ensure using consistent data references – e.g., MSL or barometric altitude• Ensure better transfer of mission status to observers
o External speaker to track commanding and detections
o Visual of command screens (TV or computer console)o Status board on which all can see progress

36

THIRD PARTY RESEARCH. PENDING FAA REVIEW.
•
•

•

•

•

•

Post brief as soon as possible after – standing in less than comfortable conditions (crammed
in a trailer, outside in the sun, or in a cold locations with no chairs) makes the inputs short,relevant, and succinct.
Data collectiono Provide sample data files in advance to ensure compatibility and that teams knowwhat to expecto Download data each day if possibleo Ensure download process and infrastructure can handle the quantity of data in atimely fashion
Flight altitudes and separationo Originally planned for UAS at 400 ft and Intruder at 500 ft and 100 ft lateralseparationo This was to address FAA’s desire to have encounters the same as with previous
flights in Alaskao Informed on site by the FAA that we needed to have 250 ft vertical separation to
be consistent with planned FAA regulations
o For UAS flights at 400 ft the Intruder altitude was changed to 650 ft
o Need to ensure clarity on approaches for crossing and distance
Site and personnel logisticso Mission support photo board
o Internet connection at site (used a Hot Spot)o Water, coffee, and snacks
o Provide food/lunch for the flight teams (so they can concentrate on the flight
missions)o Metal stair railings were hot – wrap in white tapeo There was an issue with the circuit breaker in the command trailer when the battery
charger was plugged in—charger was moved outside near generator under a tento Issue with Air Conditioner (AC) in command trailero Could not run both AC’s in the support trailero Let attendees know that the road to the test location is gravel and dirt and that they
may want to plan their rental vehicles accordinglyPart 107 interpretationo There was an interpretation difference between operations and the FAA in relation
to Part 107▪ Team operated on day 1 with the Pilot In Charge (PIC) inside the command
trailer▪ Day 2 the FAA stated that that procedure was not valid and that the PIC had
to have eyes directly on the UAS▪ It was noted that the preamble to Part 107 allows for the PIC to “have theability” and that many others operate this way▪ This is a point for future clarification▪ To not shut operations down over this point, the decision was made tochange the PIC to an external pilot for remaining missions
Intruder altitudes
o Intruder pilots were instructed to fly at a nominal altitude that was 250 ft above thesUAS altitude. This altitude did not include any margin for the 250 ft vertical
37

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

•

separation and did not account for other flight variables like wind, updrafts,
calibration errors, etc. Targeting a specific vertical separation during encountersneeds to include margins and based upon the data gathered, it is recommended thatfor future testing the nominal vertical separation be at least 350 ft.
Othero Appeared on the first day that the N to S trigger direction was different/better thanthe S to N directiono Appeared with a flight or two that the mission started late on the track – post flightdata to assesso Question arose regarding how flight is handed off between pilots

A few conclusions can be drawn from the comparisons of the NMSU and Iris data:• Not all detections in the NMSU processed data would result in an avoidance maneuver• When detections occurred, the ranges for the minimum, average, and maximum were notmuch different between the two sets of results• The average detection distance between the Iris and NMSU data processing was ~14%,with Iris’s predicted range being slightly less
• The average detection range was between 1,149.6 and 1,312.3 ft• The number of false positives with the older Casia software was much higher• The software revision improved detections and reduced false positives• The system did not perform as well at the lower altitudes (< 200 ft AGL)—this was notedby both NMSU and Iriso The system performed better at higher altitudes (300 to 400 ft AGL)
• The Iris system had no issues identifying the manned Spyder ultralight at ranges beyond
2,500 ft from the DAA system location (this was the first time the Iris system had tried todetect an ultralight)• During the sUAS versus sUAS flights, detections of the sUAS occurred every encountereven though the system was not tuned specifically for detecting sUAS
3.3 UAF August 2019
A detailed report for this campaign is provided by Remmert and Purdy (2020). The following
provides a high-level overview of this campaign.
3.3.1 Objectives
The objectives for this campaign were:• Further development of a systematic approach to evaluating DAA systems• Evaluate the efficacy of the Iris Casia EO system
3.3.2 Dates/Schedule
Testing occurred during 19-22 August 2019. The planned daily schedule was similar to that usedduring the March 2019 UAF ACUASI campaign.
3.3.3 Location
Testing occurred at the same location as used during the March 2019 UAF ACUASI campaign—
the Poker Flat Test Range. This testing location is illustrated in Figure 2.

38

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3.3.4 System Tested
The Iris Casia system was used during this campaign. This system had the following
characteristics:• System Software revision 0.19.0.64• Camera – 1.6 MP FLIR Blackfly S 65° horizontal and 50° vertical FoV• Camera Lens: Arecont Vision MPL4.0 wide angle
While the analysis utilized data collected with the Iris Casia system, data were also collected using
ground-based Echodyne radars.
3.3.5 Test Plan Overview
The test plan consisted of:• Testing of the detection component of the Iris Casia system• Encounters that were scenario-inspired but that involved numerous encounter geometries• Execution of a total of 10 test profiles• Use of 2 ownship UA
o The fixed-wing Rightwing Drako The Endurance multi-copter• Use of 4 intruders
o The ACUASI S1000 multi-copter [Da-Jiang Innovations (DJI) airframe with
Pixhawk II autopilot] UAo Fixed-wing Helio Courier 295 CA
o Fixed-wing Cessna 206 CAo Rotary-wing Robinson R44 CA
3.3.6 Sample Test Cards
Details regarding the 10 test scenarios are provided by Remmert and Purdy (2020). Test cards hadformatting similar to those used during the March 2019 UAF ACUASI campaign. An example
scenario is shown in Figure 23.

39

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 23. Example scenario (closing head-on scenario) utilized during the August 2019 UAF ACUASI
campaign. The solid yellow line indicates the UA flight path, the white figure and arrows indicate theintruder and its flight path, and the dashed yellow line indicates a sight-line for a ground-based Echodyne
radar.

3.3.7 Summary of Results
One interesting aspect of this effort was use of VOs to determine when an encounter occurred andthe DAA system should have detected an intruder. To assist the VOs, ADS-B data were provided
to them. Given this, rough Probability of Detection values could be derived (Table 4). These
values are very rough and could depend strongly on the VOs. However, given these results, a few
properties of the Casia sensor are apparent. First is the fairly narrow FoV of the sensor; tests
heavily dependent upon seeing the aircraft across a wide range of angles like the Hover
Perpendicular Crossing and Concentric Circles had poor detection rates. Second is the dwell time
before DAA notification during which the aircraft has to remain in the FoV of the sensor. Tests
like Timed Circles and Moving Head On show that when the sensor is given an extended periodof time facing the aircraft it can reliably detect. Both Moving 10-40 and Hover In Place show theadverse impact of the combination of the narrow FoV and limited dwell time on detection success;the best results for each profile come from the shallow intercept angles of 10° and 15°,
respectively. These angles give the sensor an extended period of time with the aircraft in view aswell as some relative motion for detection.

40

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Table 4. VO data summary.

ActualDetections
Profile 1: Concentric Circles

Viable Encounters

Percentage

5

10

5

29

Total

10

39

Profile 3: Timed Circles

2

2

1

1

Total

3

3

100%

Profile 4: Crossing

5

18

28%

Profile 5: Pipeline

12

18

2

7

12

23

Total

26

48

54%

Profile 7: Moving Head On

7

7

100%

Profile 8: Moving 10-40

14

15

3

8

9

18

3

12

Total

29

53

Profile 9: Hover In Place

7

12

4

7

5

7

9

12

Total

25

38

66%

Profile 10: DAA Hover T-Cross

0

12

0%

26%

55%

Of the encounters with detections, the average time with the intruder within the FoV before
triggering was 10.8 s. This value is unexpectedly high, and likely results from the data being
skewed by predominantly head-on approaches where the plane is in the FoV but out of range foran extended period of time. Additionally, the average detection distance was determined to be 656
41THIRD PARTY RESEARCH. PENDING FAA REVIEW.

m (2152 ft), which is slightly larger than the 2000 ft horizontal separation recommended formaintaining “well clear” between a manned aircraft and a small UAS (Weinert et al. 2018).
IRIS’s stated detection rate for the tested system was 70% at 390 m when allowed 3 s. This typeof encounter is difficult to construct because the of limited FoV of the sensor and the required
altitude offset between UAS and intruder trends encounters to distances > 800 m. Despite this, thevery limited data acquired does show that IRIS achieved a 67% detection rate for distances < 400m (Figure 24).

Figure 24. Summary of viable encounters versus detections.

In addition to these results, the following were determined:• No false positives occurred for the VO-identified expected DAA system detections.
• The system struggles to detect aircraft when facing the sun.
3.3.8 Lessons Learned
Lessons learned during this campaign include:• Use of a 3D plotting tool helps with visualization of encounters• Iris Casia performance aligned with vendor specifications
o Detection range limitations result in this system not generally enabling maintenanceof well clear• Development of a systematic means for testing DAA systems is critical

42

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3.4 UND September 2020
A detailed report for this campaign is provided by Askelson (2022). The following provides ahigh-level overview of this campaign.
3.4.1 Objectives
The objectives of this test campaign were:• Collection of enough data to estimate loss of well-clear risk ratioo Term is labelled LRch, with the subscript c indicating use of collision geometries inwhich aircraft would arrive at the same location if a maneuver was not applied (anda vertical safety offset was not used) and the subscript h indicating horizontal
encounters were used
• Determination if the test methodology allows for maintenance of well clear during
encounters• Determination if the test methodology supports proper testing of the DAA system
3.4.2 Dates/Schedule
Tests were conducted during the week of 20-26 September 2020. 21 September 2020 was a
shakedown day, with flights planned for the afternoon if possible. The primary flight tests days
were 22-25 September 2020. The schedule for a nominal test day is provided in Figure 25.

Figure 25. Planned daily schedule for the September 2020 UND campaign.

43

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3.4.3 Location
Tests were conducted near Hillsboro, ND. The following provides more details regarding test
location.
The Command Center Trailer, from which the tests were directed, was located at the Lovas Farm,at approximately (-97.082223, 47.329763). One UA Launch and Recovery Element (LRE) was at
approximately (-97.084370, 47.329733), which is at the Lovas Farm and very near the CommandCenter Trailer, and the other UA LRE was at approximately (-97.090454, 47.327013). The DAA
system, C Speed radar, and Electronic Observer Trailer (location with DAA display system andfrom which maneuver commands were issued) were located on the ramp of the Hillsboro, ND,
airport at (-97.061847, 47.357982). The Hillsboro airport is approximately 1.8 nm northeast of
the Lovas Farm. For reference, the Horizontal Encounter Focal Point (HEFP; horizontal location
at which aircraft would nominally arrive at the same time if no maneuver was executed) was at(-97.087696, 47.328505).1 The locations of the Command Center Trailer, one UA LRE, and theElectronic Observer Trailer are illustrated in Figure 26.

1

Tests were conducted with aircraft separated by a vertical offset to ensure safety.

44

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 26. Locations of testing elements during the September 2020
UND campaign.

45

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

The UA was flown to the southwest of the Hillsboro airport to avoid any issues with the airport.
The manned aircraft launched from the Grand Forks International Airport (KGFK) and refueled at
the Hillsboro Airport (3H4) as needed. Figure 27 provides a sectional for the area. The test area
is Class G airspace up to 700 ft AGL and Class E airspace above 700 ft AGL (up to Class A
airspace).

Figure 27. Sectional centered on the September 2020 flight test area.

Figure 28 provides images from the Lovas Farm, the location of the Command Center Trailer and
one UA LRE. That UA LRE was approximately in the location of the large snowbank near thebarn shown in the upper-right panel of Figure 28. Flights tests were conducted to the south andwest of the Lovas Farm.

46

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 28. Images from the Lovas farm, the location of the Command Center Trailer and one UAS LRE.
View is to the north (upper-left), to the east (upper right), to the south (lower left), and to the west (lower
right).

3.4.4 System Tested
Testing was conducted using the L3Harris™ Technologies DAA system, which enables multiple
approaches for the Evaluate component of DAA.2 Table 4 provides information regarding the
L3Harris™ DAA System and regarding the two options for the Evaluate step.

2

Askelson et al. (2017) identified the major steps in DAA as Detect, Track, Evaluate, and Maneuver (DTEM). These
are defined as Detect—sense the presence of something that must be avoided through some means; Track—estimate
the path of the intruder; Evaluate—determine whether identified intruders pose a threat, prioritize threats, and identify
maneuver; Maneuver—execute maneuver. These map to the functions in ASTM (2020) according to: Detect Function
DF (Detect and Track), Alert Function A1F (portion of Evaluate wherein hazards are identified and prioritized), andAvoid Function A2F (portion of Evaluate where the maneuver is identified and Maneuver).

47

THIRD PARTY RESEARCH. PENDING FAA REVIEW.
Table 4. Information regarding the L3Harris™ DAA system.
DAA Step

DAA Steps

Description

DetectC-Speed Lightwave
Radar and ADS-B
SBSSVAS and Xtend

Detect data for non-cooperative targets are provided usinga C Speed Lightwave Radar. Data for cooperative targets
are provided through the Surveillance and Broadcast
Services Subsystem (SBSS) Value Added Service (VAS)
and through Xtend ADS-B units.

Best-source selection

Data having the most accurate information regarding
intruder locations/tracks are used. Track data are provided
through the SBSS VAS or by the C-Speed Lightwave
Radar.

Track

EO (Electronic
Observer)
Evaluate-orAlerting and Guidance(A&G) supporting the
EO

ManeuverHuman Pilot

Either the EO performs all functions in this step or A&G
is utilized to help the EO decide upon the maneuver. The
display system is RangeVue™ and ownship data areingested into the system through a telemetry feed from the
UAS Ground Control Station (GCS).

The FX20 flight crew executed maneuvers once received
from the EO. This involved setting new waypoints for the
FX20.

The L3Harris™ Technologies DAA system obtains intruder detection data from several sources.
Non-cooperative data are provided using a C Speed Lightwave Radar (C Speed 2022; Figure 29).
This is a low-cost, flexible, “software-defined”, S-Band, 2D (two-dimensional) radar technology
platform that can serve a broad range of surveillance missions through reconfiguring of its runtime parameters. Cooperative data are obtained from the L3Harris™ Surveillance and Broadcast
Services Subsystem (SBSS) Value Added Service (VAS) and from local Xtend™ ADS-B units(L3Harris 2022) that act as gap-fillers to provide surveillance coverage in areas that may not beeffectively covered by the FAA’s system (provided through the SBSS VAS).

48

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 29. The C Speed Lightwave Radar utilized during the September 2020 UND
campaign.

Track data are provided by the L3Harris™ SBSS VAS (cooperative intruders) and the C Speed
Lightwave Radar system. The L3Harris™ system performs a best source selection. Thus, it selects
the surveillance source that provides the best information regarding an intruder and displays thosedata (locations and tracks). Generally, the source that provides the lowest uncertainty regarding
intruder location is considered to be the best source. Thus, these flight tests generally utilized acooperative DAA system. This, however, had no deleterious impact on test objectives.
As indicated in Table 4, the L3Harris™ system can be used two different ways for the Evaluate
step. For the September 2020 flight tests, the A&G capability was not utilized because guidance
was not generally provided. This presumably occurred because of the availability of ADS-B
surveillance data for the intruder. With those data, the A&G likely recognized the vertical offset
between the aircraft (nominally 350 ft).3 The A&G system was still being developed during thesetests, however, and thus the lack of guidance could have occurred for other reasons. Consequently,
the EO used the RangeVue™ display to identify conflicts and determine maneuvers. The EO useda Stonecast radio to communicate maneuvers to the UA PIC.
Maneuvers were executed by the UA PIC. Callbacks of commanded maneuvers were commonlyused for acknowledgement.

3

A 350 ft vertical offset was utilized to enhance safety.

49

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3.4.5 Test Plan Overview
Testing involved the entire of the DTEM chain and were designed by systematically varying
factors that affect DAA system performance. Only horizontal encounters were executed, withvariations in intruder speeds (nominally 80 and 100 kts) included. Horizontal encounter
geometries were separated by 45°, with the CA flying either northwest to southeast or southeast tonorthwest. The encounter geometries associated with UA origination points for the CA flying
northwest to southeast (Figure 30) are:
A. 0°
B. 315°
C. 270°D. 225°
E. 180°The encounter geometries associated with UA origination points for the CA flying southeast tonorthwest (Figure 31) are:B. 135°
C. 90°
D. 45°
A total of 16 test scenarios/profiles were executed (8 encounter angles x 2 intruder speeds).

50

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 30. Illustration of encounters associated with the CA flying northwest to southeast. The top image
illustrates aircraft paths and origination points. The bottom figure provides labels for UA orignation points.

51

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 31. As in Figure 30 but for the CA flying southeast to northwest.

The UA that was flown is SkySkopes’ FX20 UAS manufactured by Robot Aviation.
Specifications for the FX20 are provided in Table 5. SkySkopes was the UAS operator
The C150 intruder aircraft is owned by UND and was operated by UND during the September
2020 flight tests. Information regarding this aircraft is provided in Table 6.

52

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Table 5. Information regarding the UA used during the September 2020 UND campaign.
The FX20 is a high-performance medium-range UAS that can be
operated by a crew of two and transported in a small truck or van. Its
design results in extremely low energy consumption. As a result, the allelectric flying wing can stay in the air for multiple hours (approaching 4hrs). A portable launcher allows for runway-independent operation, withrecovery conducted using either a net or skids.

Wing Span

3.0 m

Cruise Speed

27 m s-1

Maximum Takeoff Weight

12 kg

UAS Operator

SkySkopes

Endurance

2 hr

GCS Type

Robot Aviation

Autopilot

Micropilot

Table 6. Information regarding the intruder aircraft used during the September 2020 UND campaign.
The Cessna 150 is a two-seat, tricycle-gear general aviation airplane thatwas designed for flight training, touring, and personal use.

Wing Span

33 ft 2 inCruise Speed

82 ktsMaximum Takeoff Weight

1,600 lb

Operator

UND

Fuel Capacity

22.5 US gal

3.4.6 Sample Test Cards
Test cards were developed for three roles: Flight Test Director (FTD), UA pilot, and CA pilot. A
total of 48 cards (8 encounter angles x 2 intruder speeds x 3 roles) supported this test campaign.
For brevity, the FTD test card for one scenario is provided here (Figure 32).

53

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

54

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

55

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 32. Example FTD test card for the September 2020 UND campaign.

56

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3.4.7 Summary of Results
Numerous metrics for evaluating DAA system performance were evaluated. These include sample
risk ratio from ASTM (2020) and sample risk ratio uncertainty, which provides insight into
whether the proposed approach (number of encounters) provides a viable basis for evaluation ofrisk ratio. In addition, the team presented encounter events (well clear/horizontal wellclear/vertical well clear violations), evaluated overall encounter metrics such as CPA, andproposed and presented metrics for the DTEM stages of DAA. The team also evaluatedmaintenance of well clear status during testing, which is a goal during testing.
For test methodology, the team initially used a vertical aircraft altitude safety offset of 350 ft during
these tests which generally ensured maintenance of well clear status. Three minor well clear
violations occurred, however. Based upon this, the researchers recommend a 400 ft verticalaircraft altitude safety offset for horizontal encounters. Moreover, the team recommends closer
monitoring of intruder altitudes during testing to ensure that the intruder is operating at the desired
altitude.
For the encounter set that was evaluated, the sample risk ratio uncertainty, evaluated using threedifferent methods, indicates that the set provides viable guidance regarding conformance with theperformance standard. An evaluator must place this performance in context with the broader setof encounters, the breadth of which can be evaluated through simulation. Moreover, the metrics
that are developed herein for the major stages of DAA, which were developed through the needfor information regarding how the system is performing and qualified by pragmatic considerations(e.g., timing challenges associated with data collection), provide useful information regardingthese major stages (and are being considered by the ASTM WK62669 DAA Test Methods Task
Group). Finally, methods developed for analyzing encounters, including visualization techniquesand summary metrics, enable understanding of encounter characteristics. This includes thesituation wherein aircraft closure rates increase for a period of time after maneuver initiation.
Example results for an encounter executed during this test campaign are provided in Figure 33.
Further details regarding encounters, metrics, estimation of uncertainties, etc., are provided byAskelson (2022).

57

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 33. Illustration of the 154437-154653 UTC 23 September 2020 encounter. Upper left provides aplan view, with the UA flight path in blue, CA flight path in red, the period of vertical well clear violation
indicated by brown boxes superimposed on the UA flight path (well clear and horizontal well clear are
indicated by red and orange boxes, respectively), CPA indicated with a black star symbol, the HEFP
indicated with a red plus symbol, and the inbound portions of flight indicated by blue (UA) and red (CA)dashes. Upper right provides a plot of AGL altitudes (relative to HEFP altitude), with the UA altitudes inblue, the CA altitudes in red, the CA altitudes needed to maintain vertical well clear in dark grey, the timesof vertical well clear violation indicated by dashed bown lines (well clear and horizontal well clear are
indicated by dashed red and orange lines, respectively), the UA inbound portion indicated by blue triangles,
the CA inbound portion indicated by red triangles, and the Objective Encounter Period [OEP; see Askelson(2022)] beginning and end indicated by rotated red triangles. Bottom is the separation timeline, with thehorizontal well clear boundary indicated by the orange dashed line (colored red if a well clear violation
occurs), the horizontal NMAC boundary indicated by the solid red line, unmitigated CPAh (CPA in
horrizontal direction if UA does not maneuver) indicated by the red plus symbol, the time when both aircraft
are inbound indicated with a rotated red triangle (left side of plot), the OEP end indicated with a rotated red

58

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

triangle (right side of plot), and labels indicating the following: Dfst=First Detection, Testblsh = Track
Establishment, Ecaut = Caution, Ewarn = Warning, Mid = Maneuver Identification, Minit = Maneuver
Initiation, and Mcomp = Maneuver Completion.

3.4.8 Lessons Learned
Lessons learned from this round of flight tests include:• Failures associated with testing occur, including ones experienced during this round offlight tests:o The wired connection from the radar to the Electronic Observer Trailer failed theweek before testing. This was repaired on the Friday prior to testing.
o The Long Term Evolution connection dropped at the Electronic Observer Trailer.
This interrupted communications and connectivity for the Data Collection And
Processing System [DCAPS; see Askelson (2022)].
▪ Resulted in one encounter being unacceptable.
o The Command Center Trailer generator ran out of gas.
▪ The Uninterruptable Power Supply kicked in.
▪ Communications were lost—Zoom™.
• Detection challenges occur
o ADS-B drop-outs did occur and seemed to be focused on a certain location.
o Primary tracks (radar) did not always arise for aircraft taking-off and landing at the
airport.
▪ The clutter filter is believed to be the cause of this.
• Display glitches can occur
o Occasionally the locations of UA and CA did not (seemingly) update on
RangeVue™
▪ This resulted in aircraft positions “jumping”
3.5 UAF June 2021
A detailed report for this campaign is provided in the draft report Remmert and Purdy (2021). The
following provides a high-level overview of this campaign.
3.5.1 Objectives
The objective of this flight campaign was to evaluate the performance of the Echodyne Echoflight(onboard system) and Echodyne Echoguard (ground-based system). Emphasis was on thedetection capabilities of these two systems.
3.5.2 Dates/Schedule
Testing was conducted 2-4 June 2021. The planned daily schedule was similar to previous UAF
campaigns.
3.5.3 Location
The first day of flight tests were conducted at the Poker Flat Research Range, which was utilized
in previous UAF flight campaigns and is illustrated in Figure 2. The second and third flight testdays were conducted at Bradley Sky Ranch Airport located in North Pole, AK, 15 mi southeast of
UAF. This airport was selected for this flight test campaign because of light activity, few
obstructions, and an unpopulated area to the southwest of the runway. The runway surface is acombination of gravel and natural soil with no surface treatment.

59

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3.5.4 System Tested
The airborne DAA candidate in this report is the Echoflight DAA radar manufactured by
Echodyne. Echoflight is an Electronically Scanning Array radar that employs a Metamaterial
Electronically Scanning Array (MESA) technology for beam steering. Since the MESA
architecture is solid state it does not require moving parts, nor does it require phase shifters typicalof phased array antennas. This reduced complexity makes imaging radar solutions accessible to abroad range of applications, including Cost-Size, Weight, and Power constrained applications.
At the time of this campaign a turn-key Echodyne radar-based sense and avoid DAA solution did
not exist. The Echoflight radar system is marketed as a sensing solution; it is not a comprehensive
or stand-alone DAA system. The radars must be integrated into a DAA or Unmanned Traffic
Management (UTM) system. The sensing solution is comprised of the radar hardware, Field
Programmable Gate Array logic embedded software and an onboard Inertial Measurement Unit
(IMU) for attitude determination. Radar data can be processed onboard the radar with minimal
track data transmitted, or raw data can also be output for processing offboard.
In this effort the airborne Echodyne radar was evaluated as a sensor component of a candidate
DAA system. Development of a decision algorithm and control system to interpret and react tonon-compliant threats was beyond the scope of this effort. As such only the viability of the radar
as a sensing platform is considered.
The Echodyne Echoflight radar was integrated onto a heavy-lift hex-copter (ACUASI-owned
modified version of the Tarot X6) for testing. Details regarding that integration are provided by
Remmert and Purdy (2021).
The second Echodyne system evaluated was a co-located sectoral array comprised of four
Echoguard radars. Unlike the Echoflight, which is intended for use in dynamic conditions, theEchoguard is intended to be permanently mounted to a stationary ground support. ACUASI
engineering designed and fabricated a portable four sector mount with a collapsible mast that canbe transported and rapidly deployed at a given test location. In this case the radars were oriented
in 90° quadrants for a full 360° surveillance of the local test area.
Echoguard Security and Surveillance Radar is designed to enable multiple missions such as
perimeter security or critical infrastructure applications, counter-UAS detection and tracking of
intruders, and Ground-Based Detect and Avoid (GBDAA). It can provide the same level of safe
operation for sUAS that are too small to carry their own radar system by providing localized
situational pilot awareness of both ownship position, along with cooperative and non-cooperative
intruder aircraft. Multiple Echoguard units may be combined into flexible local networks forincreased coverage. Echoguard provides 20-30% longer range than the EchoFlight by maximizing
transmitter power, waveform processing, and through design of an all-passive cooling system (noforced air cooling) optimized for fixed and portable ground-based use.
3.5.5 Test Plan Overview
The test plan consisted of:• Testing of the detection component of the two types of Echodyne systems• Encounters that were geometrically and scenario inspiredo Horizontal, descend-into (intruder), and climb-into (intruder) encounters• Execution of a total of 11 test profiles
60

THIRD PARTY RESEARCH. PENDING FAA REVIEW.
••

Use of a heavy-lift hex-copter ownship UA
Use of 3 intruders
o Fixed-wing Helio Courier 295 CA
o Fixed-wing Cessna 206 CA
o Trinity 90+ Vertical Take-Off and Landing (VTOL) UA
The intruders are illustrated in Figures 34-36.

Figure 34. Helio Courier 295 CA used as an intruder during the June
2021 UAF campaign.

61

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 35. Cessna 206 CA used as an intruder during the June 2021
UAF campaign.

Figure 36. Trinity F90 VTOL UA used as an intruder during the June
2021 UAF campaign.

Details regarding test profiles are provided by Remmert and Purdy (2021). An example image fora profile in which ownship encounters an intruder (Cessna 206) that is either on final approach(descending) or departing (ascending) is provided in Figure 37. For this profile, horizontal wellclear status is maintained through enforcement of a 610 m lateral aircraft offset. It is noted that
this campaign incorporated more ascending/descending encounters (in addition to the pop-up

62

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

encounters executed in the July 2019 NMSU campaign). Doing so for full encounters (i.e.,
including the TEM steps in DTEM) was accomplished in a later test campaign (June 2021 UND).

Figure 37. Bradley Ranch encounter profile with intruder in the traffic pattern.

3.5.6 Sample Test Cards
Test cards were developed for the 11 profiles. An example that corresponds to Figure 37 isprovided in Figure 38.

63

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

64

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Flight Card #ACUASI_AK_95Z_Test
Mission 2

Card

Date/TimePrimary:
Objective
Secondary:
MannedaircraftandUAS
encounters are conducted as
Description
displayed. A lateral offset of 500m
used to provide safety margin.
UAS Platform ACUASI X-6
UAS Altitude 300’ AGL
UAS Speed 6 m/s
Intruder Cessna 206
Intruder(Traffic Pattern Altitude)
AltitudeIntruder Speed (Normal Cruise and landing)
Location

64.757964, - 147.390461. Bradley
Sky Ranch (95Z)

GCS Mission Planner
DAA SystemID
DAA Sensors
Uavionix Ping ADSB, EchoGroundSupporting
Radar,
EchoGuard
radar,
Technology
GroundAware GA-9120
Intruder Pilot
RPIC
VOs
MC
FD
COA/Waiver(s)65THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Action1

UA executes a E/W oriented
flight plan parallel to therunway offset by 500ft fromtherunwaycenterline.
Crewed aircraft executes
standard
touch-and-go
landings in the FOV of theground-based radar.

Condition # (objective)
RemarksCall
Echodyne radar FOV
AZ ±60°EL ±40°Time

Crewed aircraft toreportstandard
aviation traffic pattern

EchoGuard radar isoriented 360° FOV

2
3
4
5
6
7
8
9
10
11
12

END

Figure 38. Example test card (profile 2) from the June 2021 UAF campaign.

3.5.7 Summary of Results
After initial flight testing of the Echoflight UA platform, it was apparent that the airborne system
was adversely affected by ground motion. While some effect was anticipated, the extent to whicha dynamic platform would impact radar performance was unknown. While the Echoflight radars
do have an integrated IMU for the purpose of tracking self-stabilization, this functionality
apparently does not compensate for all platform dynamics. Any relative motion of the groundwith respect to the airborne platform induces multiple false tracks. This occurs because groundbased objects appear to move within the Field of Regard (FoR) and stationary objects areinterpreted as moving targets. This phenomenon is most pronounced during yaw motion of the
UA platform (an example is provided in Figure 39). Unfortunately, for all encounters where theEchoflight platform was in motion, the radar data were rendered useless due to excessive noise.
The Echoflight and Echoguard are only capable of tracking 20 simultaneous targets at any given
time. Therefore, an excessive number of false tracks effectively saturates the radar. Due to thislimitation, only encounters where the Echoflight UA platform is in stationary hover are consideredin these test results.

66

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 39. Ground motion induced noise experienced with the Echoflight radar.

For encounters where the airborne UA platform was in a stationary hover, false tracks wereattenuated sufficiently to discern a few valid tracks from false positives. In a few encounters anincreased maximum detection range was noted as compared to the ground-based Echoguard units.
However, there was virtually no consistency; the range of first detection varied dramatically, evenbetween like encounters. It should be noted that this was an issue common to both the Echoflightand Echoguard units.
The tracks generated by the Echoflight radar consistently exhibited erratic variance in altitude ascompared to those from the Echoguard units. This altitude variance is a result of vibration andbuffeting of the airborne platform. The resulting low amplitude high frequency motion also
adversely affects the ability of the Echoflight unit to acquire new targets at extended range. Thisis a concern for a DAA sensor because it decreases the CPA for intruders.
For the duration of the test campaign, the Echoguard radars demonstrated a markedly improved
performance relative to the Echoflight radars. Given that the Echoflight and Echoguard units are
virtually identical, it can be inferred that performance is tightly coupled to the use case. Stationary
Echodyne units typically have better performance.
During the test campaign it was noted that the range of first detection varied. This characteristic
is illustrated in Figure 40, which conveys a significant spread in values. The source of thisvariability is as of now undetermined. However, given that the conditions of the tests, it can bereasonably inferred that the variability results from the tracking characteristics of the Echoguard.

67

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 40. Variability of Echoguard initial detection range.

The data clearly show there were instances during the test where the intruder was detected verylate in the encounter. This placed the moment of initial detection just outside the well clear
volume. It should also be noted that the Echoguard radars are ground-based DAA sensors and will
most likely be utilized as sensors for a UTM system. Therefore, the Echoguard units may notnecessarily be co-located with the UA. In order to achieve full coverage a UTM would likely
employ a network of Echoguard arrays. If the worst-case detection ranges presented in this report
are indicative of the minimum distances required between sensors, such a network may not bepracticable.
Another noteworthy attribute that was identified during this test was the effect of intruder aircraft
maneuvers on Echoguard tracking. During tight radius turns the radar would consistently lose
track of the intruder aircraft. The Echoguard radars employ a probabilistic predictive analysis aspart of their detection and tracking algorithm. During maneuvers the radar cross section is dynamic
as is the vector of the intruder. The resultant loss of confidence in the track position is likely why
the radar loses tracks during tight maneuvers.
3.5.8 Lessons Learned
Lessons learned during this campaign include:• The dynamic environment associated with the Echoflight system results in significant
ground clutter (when flying near the ground)• Vibration appears to affect altitude estimates of intruders generated using the Echoflightsystem• The Echoguard (and presumably Echoflight) system can lose tracks during tightmaneuvering of intruders• Variations in detection range for the Echoguard system can be significant

68

THIRD PARTY RESEARCH. PENDING FAA REVIEW.
•

Both the Echoflight and Echoguard systems show promise for DAA, although challenges
exist

3.6 UND June 2021
A detailed report for this campaign is provided by Askelson and Stephens (2022). The following
provides a high-level overview of this campaign.
3.6.1 Objectives
The objectives for this flight test campaign were:• Determination if the test methodology results in well-clear maintenance during encounters(especially with the increased vertical safety offset and for descend-into encounters)• Collection of additional samples at selected horizontal encounter geometries andevaluation of impacts on statistical parameters
3.6.2 Dates/Schedule
Tests were conducted during the week of 13-19 June 2021. 14 June 2021 was a planned
shakedown day. The team encountered challenges with provision of UA location data to the
L3Harris™ system. These issues were resolved on the afternoon of 15 June, resulting in three test
days: 16-18 June 2021. The planned schedule for that week is provided in Figure 41.
Start
600
630
700
730
800
830
900
930
1000
1030
1100
1130
1200
1230
1300
1330
1400
1430
1500
1530
1600
1630
1700
1730
1800

End
630
700
730
800
830
900
930
1000
1030
1100
1130
1200
1230
1300
1330
1400
1430
1500
1530
1600
1630
1700
1730
1800
1830

Day 1: 15 June 2021

Day 2: 16 June 2021

Day 3: 17 June 2021

Day 4: 18 June 2021

Meet at NPUASTS (4201 James Ray Dr.) Meet at NPUASTS (4201 James Ray Dr.) Meet at NPUASTS (4201 James Ray Dr.) Meet at NPUASTS (4201 James Ray Dr.)
Travel to Site
Travel to Site
Travel to Site
Travel to Site
Morning Briefing
Morning Briefing
Morning Briefing
Morning Briefing
Site Setup and Preparation

Site Setup and Preparation

Site Setup and Preparation

Site Setup and Preparation

Flight Testing

Flight Testing

Flight Testing

Flight Testing

Lunch and Aircraft Refueling

Lunch and Aircraft Refueling

Lunch and Aircraft Refueling

Lunch and Aircraft Refueling

Flight Testing

Flight Testing

Flight Testing

Flight Testing

Site and Equipment Tear Down

Site and Equipment Tear Down

Site and Equipment Tear Down

Site and Equipment Tear Down

Debrief and Schedule Review

Debrief and Schedule Review

Debrief and Schedule Review

Debrief and Schedule Review

Travel to NPUASTS

Travel to NPUASTS

Travel to NPUASTS

Travel to NPUASTS

Figure 41. Planned test schedule for June 2021 UND campaign.

3.6.3 Location
Test locations were the same as those used during the September 2020 UND campaign.
3.6.4 System Tested
The system tested was the same as that used during the September 2020 UND campaign except
the display system was upgraded to the RangeVue™ Pro display system, which did not provide
A&G. For the Evaluate step, the EO used the RangeVue™ Pro display to identify conflicts andidentify maneuvers, and a Stonecast radio to communicate maneuvers to the UA PIC. In the
69

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

absence of an A&G system, distance-based circles were drawn around the UA, as illustrated in
Figure 42. The radii of these (outer) circles are generally equal to the horizontal extent of thehazard zone as defined by Radio Technical Commission for Aeronautics (RTCA) (2017;§2.2.4.3.2), which vary with horizontal encounter geometry to account for changes in closure rates
and depends upon the alert threshold. For these tests, a late (60 s) alert threshold was used to limit
the distance between UA origin points and the Encounter Focal Point (EFP), which simplified Part
107 operations in that daisy-chained ground observers were not required and sped up the testingprocess by decreasing the amount of time required for set-up between encounters. For simplicity,
distances for 0° and 180° horizontal encounter angles were computed, and distances forintervening angles were linearly interpolated from those values. In addition, a buffer of 0.17 mi
was added for 180° encounters, which provides an additional 5 s. This was based upon
computations of the amount of time required for the UA to travel 2000 ft [the horizontal well-clear
distance used in ASTM (2020)] from the line adjoining the aircraft tracks for a 180° encounter,the estimated amount of time required for initiating a maneuver (~15 s), and the distance theintruder travels in that amount of time.

Figure 42. Example of the RangeVue™ Pro display. The UA is indicated by the triangle symbol locatednear the center of the display, with concentric rings drawn around it. The intruder is indicated by the triangle
symbol surrounded by a magenta-colored circle. The outer ring around the UA serves as the alert for the
EO to identify and communicate a maneuver to the UA PIC.

3.6.5 Test Plan Overview
Testing involved the entire of the DTEM chain and encounters were geometrically based. During
the prior set of flight tests conducted by the UND and NPUASTS in September 2020 (Askelson2022), horizontal encounters including the full range of possible encounter angles (0°-360°) andusing a 45° increment (0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°) were conducted. Analysis of70

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

those tests indicated an interesting pattern in which the horizontal component of CPA (CPAh)
appeared to depend upon the encounter angle, with smaller CPAh values associated withovertaking encounters (encounter geometries of 180° and 225°) and larger values associated withhead-on encounters such as 0° and 45° [cf. Figure 22 of Askelson (2022)]. To explore whether
the differences observed during the September 2020 flight tests consistently occur, the June 2021
tests utilized a selected subset of horizontal encounter angles (0°, 45°, 180°, 225°). In addition,
for the horizontal encounters and these encounter angles, 10 tests at each angle were incorporated
into the encounter test script (as opposed to 5 in the September 2020 test). Executing more tests
at each angle produces greater confidence in observed differences and also enables exploration of
dependence of statistical properties (e.g., variance) on the number of samples through comparison
with previous test results. These horizontal encounters followed collision trajectories, wherein theaircraft were flown towards an EFP using a vertical safety offset. This follows Askelson (2022),with the exception of a larger vertical safety offset of 400 ft [vs 350 ft with Askelson (2022)] used
for this round of flight tests. Collision trajectories were used because they result in the strictest
timing requirements for DAA for each encounter angle.
In addition to horizontal encounters, tests also included descend-into encounters, as shown in
Figure 43. For these, horizontal origination points were the same as in horizontal encounters forthe corresponding horizontal encounter angles, which were 0° and 135° for the June 2021 descendinto encounters. In these encounters, the intruder descended at a rate of 500 ft min -1 (a typicaldescent rate for the Piper Archer intruder aircraft used in this set of tests). Initial intruder altitude
was set such that the intruder and UAS would fly to the same location, horizontally and vertically,
without the use of the vertical safety offset (described subsequently). Thus, a collision-type
geometry was employed and applied to the point of enacting the vertical safety offset. Given thehorizontal 120 kt speed and 500 ft min-1 descent rate of the intruder, the vertical encounter angle
is +2°.
Because ground features that could be used to ensure horizontal aircraft safety offsets were either
not present or not aligned with horizontal encounter geometries, safety was maintained by having
the intruder halt descent by the time 800 ft AGL was reached. Given that the UAS was operated
at 390 ft AGL, this provided ~400 ft of vertical safety offset, which is 50 ft greater than that usedin the September 2020 tests. With this design, at 900 ft AGL the intruder is ~250 ft from thevertical well clear boundary of the UA and, at a descent rate of 500 ft min-1, ~30 s from the vertical
well clear boundary. This produced a trend prior to activating the vertical safety offset that
required a maneuver to maintain well clear in the vertical direction. In the horizontal direction,
the trend is constant (before and after activation of the vertical safety offset) and required amaneuver to maintain well clear in the horizontal direction. Thus, the descend-into encounter
design resulted in a trend to a well clear violation prior to activation of the vertical safety offset
that required a maneuver for maintenance of well clear.

71

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 43. Illustration of descend-into encounters used during the June 2021 UND campaign.

Speed variations (beyond those driven by environmental winds) were not included. A total of 6
scenarios were tested [(4 horizontal scenarios + 2 descend-into scenarios) x 1 intruder speed].
The UA that was flown is Isight Drone Services’ SuperVolo UAS. Specifications for theSuperVolo are provided in Table 7. Isight Drone Services operated the SuperVolo.
The Piper Archer intruder aircraft is owned by UND and was operated by UND during the June
2021 flight tests. Information regarding this aircraft is provided in Table 8.

72

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Table 7. Information regarding the UA used during the June 2021 UND campaign.
The SuperVolo is a long range, Vertical Take-Off and Landing (VTOL)UAS designed for simplified deployment/ease of use. It utilizes a hybrid
gas/electric power plant. The SuperVolo enables quick refueling forsuccessive flights and requires very little ground infrastructure for
operations. It also features a modular airframe that enables diverse
payload configurations and cost-effective maintenance.
3.0 m

Cruise Speed

18-34 m s-1

18.2 kg

UAS Operator

ISightServicesEndurance

8 hrs

GCS Type

ACER computerwith Swift GCS

Autopilot

CUAV PixHawk – Mavlink

Wing Span
Maximum Takeoff Weight

Drone

Table 8. Information regarding the intruder aircraft used during the June 2021 UND campaign.
The Piper Archer is a two-seat, tricycle-gear general aviation airplane thatis used heavily in UND’s aviation education and training programs.

Wing Span

35 ft 6 inCruise Speed

128 kts

Maximum Takeoff Weight

2,550 lb

Operator

UND

Fuel Capacity

50 US gal

3.6.6 Sample Test Cards
Test cards were developed for three roles: FTD, UA pilot, and CA pilot. A total of 18 cards [(4horizontal scenarios + 2 descend-into scenarios) x 1 intruder speed x 3 roles] supported this test
campaign. For brevity, the FTD test card for one scenario is provided here (Figure 44).

73

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

74

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

75

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 44. Example FTD test card for the June 2021 UND campaign.

76

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3.6.7 Summary of Results
The vertical aircraft safety offset was increased 50 ft relative to that used by Askelson (2022) to
400 ft. This helped ensure desired aircraft separation. The biggest enabler for ensuring
maintenance of vertical well clear during testing, however, was CA altitude monitoring in which
CA altitudes were adjusted if they were deemed to be too low. These resulted in no vertical wellclear violations occurring during the June 2021 test campaign. For descend-into encounters, CA
descent was halted to preserve a 400 ft vertical safety offset between aircraft. The pilots weresuccessful at halting their descent as planned, as the minimum vertical offset for the 10 descendinto encounters was 403 ft.
In addition to metrics developed by Askelson (2022), an encounter descriptor/metric labelled coast
period was added. This is needed because of flight of the UA under Part 107 rules, without theuse of daisy-chained observers, and the speed of the UA resulting in it reaching the visual range
limit associated with Part 107 prior to the time when aircraft separation was minimized. Whenthat limit was reached, the aircraft was directed to station-keep (fly circles around a defined
location). To obtain results consistent with a BVLOS type of operation in which such stationkeeping would not generally occur, the straight-line portion of the UA path associated with its
maneuver was extrapolated forward/coasted.
Execution of more encounters for given scenarios did not consistently reduce the standard
deviation of the aircraft separation metric analyzed herein. It did, however, reduce uncertainty inboth the mean and standard deviations of that metric.
No horizontal well clear violations occurred during this test campaign, resulting in a loss of wellclear risk ratio for this subset of encounter scenarios of 0.0. Uncertainty windows for this subset
of encounters ranged from 0.0-0.01 to 0.0-0.02. The exceptional performance of the SuperVolo
UA that was used in these tests—especially its cruising speed (up to ~66 kts), significantly enabled
maintenance of horizontal well clear.
3.6.8 Lessons Learned
Lessons learned from this round of flight tests include:• Failures associated with testing occur, including ones experienced during this round offlight tests:o Ingestion of UA telemetry into the DAA system can be a major challenge. It
resulted in a significant delay in data collection during the June 2021 tests.
• Detection challenges occur
o ADS-B drop-outs did occur and seemed to be focused on a certain location.
o Primary tracks (radar) did not always arise for aircraft taking-off and landing at the
airport.
• Display glitches can occur
o Occasionally the locations of UA and CA did not update on RangeVue™ Pro▪ This resulted in aircraft positions “jumping”
3.7 NMSU Fall 2021
The NMSU UASTS planned to execute an equivalent of 5 days of flight testing in the Fall of 2021.
Each flight set had multiple encounters profiles. Each profile was planned to be flown a minimumof 5 times with data collected during each event. Both ground based and airborne systems have77

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

been tested as part of the A18 research arc. Each has advantages and challenges. The A18 team
tested a number of different systems in UA to CA encounters, and UA to UA encounters that havebeen completed with safe separation distances. Areas identified as challenges are the climb-intoand descend-into encounters. These might be experienced with crop dusters or other CA flying
specific low-altitude missions/operations. This testing addressed these climb-into and descendinto types of encounters.
3.7.1 Objectives
This effort was designed to evaluate methods for testing DAA technologies by looking at types ofoperations, defining collected data parameters, and defining data processing required for validationof DAA technologies. The test methodology for DAA systems generally follows a phased
approach wherein the first phase is to understand what the system can do in simulation space before
advancing to flight test phases. For flight tests, the simplest encounters were executed using an
altitude offset. Subsequent testing was executed where encounters become co-altitude byincluding descend/climb-into encounters. Encounter profiles were planned to allow for the Iris
CasiaX system to detect an intruder aircraft from multiple angles and altitudes.
In previous DAA testing the A18 team tested a number of different systems UA to CA and UA to
UA encounters. For the planned climb-into and descend-into encounters, safe vertical offsets
could not be maintained with a UAS by the very nature of the test profiles. Once the vertical offset
is violated, the question of what lateral offset is required arises. To address this paradox and toensure safe flights, the DAA system was installed on a CA that served as a proxy UAS. This
allowed two CA with VOs onboard to safely and legally execute the planned encounters. Verticaland lateral offsets were incorporated into the test cards for safety. The planned number encounters
for the planned scenarios for this campaign are provided in Table 9.
Table 9. Planned number of encounters for encounter scenarios for the Fall 2021 NMSU
campaign.

Test Card

Test Card Description

Planned Number of encounters

1

Compass rose encounter(s)202Intruder descends behind UAS

5 x 2 = 10

3

Intruder climbs behind of UAS

5 x 2 = 10

4

Intruder descends in front of UAS

5 x 2 = 10

5Intruder climbs in front of UAS

5 x 2 = 106

UAS climbs in front of Intruder *5

7

UAS descends in front of Intruder *5

Total Encounters

70

78

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

3.7.2 Dates/Schedule
The flight testing was scheduled to occur during several periods to allow for testing of likeencounters on specific days to ensure safety. The team felt that it was safer to not mix different
types of encounters within a single day, but to do all encounters of the same type one day, and thenreset for the next configuration on subsequent days. This allowed the pilots and onboard VOs to
only concentrate on one type of flight profile for consistency and safety.
The Compass Rose encounters were competed on 21 October 2021 during two rounds of flights.
Initial data processing was completed after these flights to ensure system capture of theinformation and as a check on the flight planning and procedures. The Intruder Climb/Descend
Behind encounters were executed on 16 November 2021 during two rounds of flights. The Intruder
Climb/Descend In Front encounters were executed on 17 November 2021 during two rounds offlights. A short fueling and “leg stretching” break was inserted between the flights each day.
3.7.3 Location
Both aircraft launched from the Las Cruces International Airport (KLRU). Generally, most of theflight maneuvers took place away from the airport so as to minimize impact on general aviation
operations. The area is sparsely populated and all maneuvering was performed at or below 6500
ft MSL. To maintain safety, both aircraft involved in the operation had ADS-B in/out installed sothat pilots had a high level of situational awareness and to allow for smoother setup for encounters.
Figure 45 provides a plan view of the test area for this campaign.

79

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 45. Plan view of the test area for the Fall 2021 NMSU
campaign.

3.7.4 System Tested
Testing was conducted using the Iris Automation CasiaX DAA system, which is a multi-camera
vision-based DAA system. Differences from previous testing are that this system used 5 cameras
for a 360° view (versus one camera used in previous tests) and how the system as integrated. Table
10 outlines the technologies used.
Table 10. DAA system used in the Fall 2021 NMSU campaign.

Iris Automation,
CasiaX

Sensor: Vision
Based DAA

Casia X is the first commercially available 50° Verticaland 360° radial computer vision DAA system for UAS. It
also has a viewing range of 1.5 km. As an integrated
80

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

onboard hardware and software solution, Casia systems
are small, light and low power.

The Iris CasiaX system was designed pursuant to the emerging ASTM standards for performancerisk-ratios. All Casia systems combine ultra-lightweight and compact hardware with software that
integrates with most industrial drones (Figure 46). Figures 47-49 illustrate how the CasiaX
cameras were mounted for this test campaign.

Figure 46. The Iris CasiaX system.

Figure 47. CasiaX cameras mounted on the left wing ofownship.

81

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 48. CasiaX cameras mounted on the right wing ofownship.

Figure 49. CasiaX camera mounted on the front ofownship.

3.7.5 Test Plan Overview
The NMSU UASTS utilized a Flight Star E-LSA as a UAS surrogate. The CasiaX system was
integrated onto this aircraft. This was used for an increased level of safety during the climb-intoand descend-into flight encounters. The Flight Star E-LSA is a small 2 seat ultralight. The intruder
was a Flight Design CTLS owned and operated by NMSU.
The configuration for horizontal encounters is provided in Figure 50. The CasiaX DAA system
has a nominal range of 1.5 km and a FoV of 360⁰ horizontal x 50⁰ vertical. With this information
the encounters were designed around the compass rose. Encounters begin at a distance beyond therange of the vision system. The intruder either approaches from the east or the west with the UA
surrogate flying toward the center bullseye. The UA surrogate and the intruder will be separated
by 500 ft vertically during this phase of testing. Both the UA surrogate and intruder are CA withdedicated pilots and VOs to help ensure safety.
The configurations of climb- and descend-into encounters are provided in Figure 51. For theseencounters, the ascent/descent rate of 500 ft min-1 was used based on previous tests.

82

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 50. Configuration of horizontal encounters executed during the Fall 2021 NMSU campaign.

83

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 51. Configuration of climb- and descend-into encounters executed during the Fall 2021 NMSU
campaign.

3.7.6 Sample Test Cards
Encounters can be organized into two groups: the “compass rose” and the various climb-into anddescend-into encounters. Figure 52 provides the compass rose test card, while Figure 53 provides
a sample of the climb-into and descend-into encounters.

84

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Flight Card # NMSU_NM_DAA_Iris Test 1
The Intruder will fly both directions east to
west and then west to east. The intruder
Primary: This will test the FOV will maintain an altitude of 7000’ MSL atof the Iris DAA while an intruder all times during the encounters.
is crossing the path from right to
Objective left.
The UAS will start at WP 1 and fly the lines
Secondary:in sequential order crossing over thebullseye each time. This includes three
encounter angles plus head on and
Description Descending behind UAS
following.
The UAS will maintain
FlightStarSCII(UAS 6000’MSL at all times during the
UAS Platform
Surrogate)encounters.
Date/Time TBD

UAS Altitude 6000’ MSL descending 500 fpm
This this portion will be repeated 5 times(20 total encounters).

UAS Speed 60 mph (52 kts)
Intruder CTLS Light Sport
Intruder7000’ MSL
AltitudeIntruder Speed 80 mph (70 kts)
Location Las Cruces, 5 NM from airport

85

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

GCS N/ADAA System
CasiaX (360)ID
DAA Sensors 5 Cameras
Supporting(Vision Base)
Technology
Intruder PIC
Intruder VO
Surrogate
UAS PICSurrogate
UAS VO

Condition # (objective)
Action1RemarksCall

Get to Starting altitudes
UAS 6000’ MSL / Intruder
7000’MSL

2Line up with initial WP

3

Fly to bulls eye

4

Reset-for total of 5 encounters5
6
7

Figure 52. Compass rose test card used during the Fall 2021 NMSU campaign.

86

Time

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Flight Card # NMSU_NM_DAA_Iris Test 2
Date/Time TBD
Primary: This will test the FOVof the Iris DAA while an intruder The Intruder will start at 8000’ MSL and 2
is crossing the path from above km behind the UAS. Once the Intruder andObjective and to the rear of the UAS.
the UAS are at the designated waypoints,
the intruder will start a 500fpm decent until
Secondary:an altitude of 6000’ MSL The Profile willkeep the UAS and Intruder 1000’
horizontal separation.
Description Descending behind UAS
UAS Platform

FlightStar
Surrogate)

SCII

(UAS
The UAS will Maintain an altitude of7000’MSL and fly to bullseye.

UAS Altitude 7000’ MSLUAS Speed 60 mph (52 kts)
Intruder CTLS Light Sport

This Card will be repeated 5 times.

Intruder8000’ MSL descending 500 fpm
AltitudeIntruder Speed 80 mph (70 kts)

87

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Location Las Cruces, 5 NM from airport
GCS N/ADAA System
CasiaX (360)ID
DAA Sensors 5 Cameras
Supporting(Vision Base)
Technology
Intruder PIC
Intruder VO
Surrogate
UAS PICSurrogate
UAS VO

Condition # (objective)
Action1Remarks

Call

Time

Get to Starting altitudes
UAS 7000’ MSL / Intruder
8000’MSL

2Line up with initial WP

3

Once waypoint is met, intruder
will begin 500 fpm decent

4

Fly to bulls eye

5

Reset for total of 10 encounters6
7

Figure 53. Intruder descend behind test card used during the Fall 2021 NMSU campaign.

3.7.7 Summary of Results
Testing took place over two different time periods in the Fall of 2021. The “compass rose”
encounters were flown on 21 October 2021. The “rise into” and “descend into” encounters were
flown over two days on 16 and 17 November 2021. The full test report with details are included
in Cathey et al. (2022) which covers the test planning, setup, encounter geometries, test cards, andresults. It should be noted that the DAA system also had ADS-B as part of the detection filters. It
was able to identify aircraft via ADS-B as well as the visual system. During testing, the intruder
88

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

aircraft as well as commercial aircraft in the area (for example, American Airlines flights passing
over at > 20,000 ft) were detected. For these analyses, the ADS-B detections were filtered out andjust the visual system detections were analyzed to better simulate non-cooperative aircraft.
The compass rose encounters were also used as an initial test of the five-camera system to assess
the effectiveness of the integration of the system and to collect data for what was considered themore straight-forward test patterns. The surrogate UAS performed basically East-West passes andthe intruder crossed this path from different encounter angles. As noted, a built-in vertical aircraft
offset was used during these runs.
Challenges with the timing of encounters arose in which it was difficult to make both aircraft crossthe approximate center point of the planned testing geometry at/around the same time. This was
not as difficult with the compass rose encounters. Overall, the detections and tracking of aircraft
throughout the compass rose encounters was as expected. The five-camera system detected the
intruder aircraft at all encounter geometries. Detections were where the aircraft were actually
located and at ranges comparable to the published detection ranges. Simply put, there were no realsurprises or issues with the operation of the system. Post testing review of the data showed someslight tuning of the elevation and overlapping of the cameras was required. Overall this was very
minor.
The “climb into” and “descend into” runs were completed over two days. As noted previously,
timing challenges affected these encounters. These were exacerbated by the additional timing
element related to climbing or descending in front of or behind the other aircraft. These encounter
geometries were easy to design on paper and were much more difficult to implement in flight withthe added time dimension. It is easy to make an aircraft climb or descend along a desired path, butmore difficult to do this when it has to be perfectly timed with another aircraft also moving acrossthe test space. The aircraft are always in motion so they have pre-test-event timing that has toconsider the flight time and patterns to get both aircraft into the desired start positions when they
begin a run. This was a challenge.
The first day of these tests was 16 November 2021. The testing was divided into two sets ofsorties. This was mainly due to the flight time of the Spyder. This aircraft was flown for a periodof time, landed, refueled, and then flown again. The Spyder, with the DAA system installed, flewa basically level flight and the intruder CTLS performed the climb-into and descend-into
encounters. The challenge of test timing noted above is clearly apparent in Figure 54.

89

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

November 16, 2021
Climb Into and Descend Into Encounters
2400

2200

Altitude (m)

2000

1800
Nov16CTLS1

1600

Nov16CTLS2
Nov16Spyder1

1400

Nov16Spyder2
1200
13:30

Detections Final

14:30

15:30
16:30
17:30
Time
Figure 54. 16 November 2021 flight profiles and detections. Orange dots represent the altitude of the
Spyder (with DAA system), blue dots are CTLS intruder altitudes, and green dots are detections.

The first half of the first round of sorties showed zero detections. This was not a reflection of theperformance of the DAA system, but resulted from the challenge with setting up the timing of theencounters to put the aircraft into the right places at the right times with the ascent or descentprofiles. The first four passes did not result in any detections, and the fifth one had a single
detection at one point. The flight team realized they were not positioning the aircraft as desired
and adjusted the start timing and positioning for subsequent runs. The detections for the balanceof the runs were much more consistent. The same plot as above with revised scales is shown in
Figure 55.

90

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

November 16, 2021
Climb Into and Descend Into Encounters
2400

Nov16CTLS1
Nov16CTLS2

2300

Nov16Spyder1

Nov16Spyder2

Altitude (m)2200

Detections Final

2100
2000
1900
1800
1700
1600
13:45

14:15

14:45

15:15

15:45
16:15
Time

16:45

17:15

17:45

Figure 55. As in Figure 54, but with revised scales.

The test flights revealed a challenge with completing all of the planned test cards. The UAS climbinto and descend-into cards were problematic given the aircraft used for that role. The Spyder has
a hard time climbing within the test encounter area (timing and geometry challenge) and it is likelythat these encounters will have to be omitted in future testing. The Spyder Ultralight is really slow
at climbing, meaning that modifications to the profiles are needed to enable transitions from one
encounter the another.
The second day of testing was completed on 17 November 2021. Similar plots for the flight
altitudes and detections are provided in Figures 56 and 57.

91

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

November 17, 2021
Climb Into and Descend Into Encounters
2400

Nov17CTLS1
Nov17CTLS2

2200

Nov17Spyder1
Nov17Spyder2
Detection Final

Altitude (m)

2000

1800

1600

1400

1200
13:00

14:00

15:00

16:00

17:00

Time
Figure 56. As in Figure 54 but for 17 November 2021 flight profiles and detections.

92

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

November 17, 2021

Nov17CTLS1

Nov17CTLS2 Climb Into and Descend Into Encounters
Nov17Spyder1
2200

Nov17Spyder2
Detection Final

Altitude (m)

2100
2000
1900
1800
1700

1600
13:30

14:00

14:30

15:00

15:30
16:00
Time

16:30

17:00

17:30

18:00

Figure 57. As in Figure 56, but with revised scales.

A closer look at the first set of flights on 17 November 2021 is provided in Figure 58. This figure
best illustrates what was observed during the testing.

93

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Nov17CTLS1

November 17, 2021

Nov17CTLS2 Climb Into and Descend Into Encounters
Nov17Spyder1
2200

Nov17Spyder2
Detection Final

Altitude (m)

2100
2000
1900
1800
1700

1600
13:45:00

14:15:00

14:45:00
Time

15:15:00

15:45:00

Figure 58. As in Figure 56, but with a change of scales (zoomed-in).

A number of conclusions can be drawn from these data. Detections are apparent during ascendinto encounters when the intruder aircraft is ascending. The same is true for similar descend-into
encounters. Both are indicated in Figures 54-58 where the ‘green’ detections are overlaid with the
intruder altitude. Run 3 in Figure 58 is a clear example of a descend-into encounter. The last run
in Figure 58 is a clear example of an ascend-into encounter. For many of the encounters, theaircraft was detected visually for much of the ascent and descent trajectories when the aircraft werein visual range and the detections were at or above the simulated UAS’s altitude.
The major finding from these tests is that almost all of the detections occurred when the intruder
aircraft was at or above the simulated UAS flight altitude. This occurs for a number of potential
reasons. The system design and tuning are the first considerations. The DAA system was notdesigned/tuned to look below the horizon for aircraft so the ascent portion of the ascend-into
encounters (before reaching approximate co-altitude) had limited detections. The system is tuned
using thousands of images of aircraft. These are generally with sky as the background to theintruder aircraft.
Testing geometry is the second consideration and is related to the first consideration. The intruder
aircraft flying below the UAS altitude generally put the intruder aircraft into a position where it

94

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

had the ground and ground clutter as background. This is a much more challenging visual
detection, not only for the systems as configured, but also in reality with the background.
Detections were best when the intruder was above the plane of the simulated UAS. Detections
above the DAA system’s altitude were quite good. The detections where the system was looking
downward from horizontal were not as good. The detections of the intruder aircraft were goodwhen the aircraft was above the simulated UAS altitude, but were limited after the descent portionof the descend-into encounters. The aircraft were tracked for longer periods when above the DAA
system’s altitude.
The detection locations and altitudes are very close to the actual intruder location. The further
away the two aircraft are, the greater the uncertainty, but the estimated altitudes were generally
within 100-150 m.
3.7.8 Lessons Learned
A number of specific lessons learned that covered the initial system integration through testingand performance arose from this effort. These are noted below. The lessons learned from the testplanning and integration are as follows:• Close encounter testing of a UAS and a manned aircraft is difficult. Testing of these typesof systems proposed for UAS operations cannot be easily safely or legally tested wherevery close encounters with CA are required. Surrogate or proxy tests using two CA wasused to replicate this.
• Two aircraft with pilots onboard can be employed to safely replicate a CA intruder with
UAS encounters.
• This particular DAA system was generally designed to be used on a UAS at lower altitudes.
Planning for the encounters required an adjustment of the crossing altitudes needed toperform these tests safely.
Lessons learned related to the integration of this specific system into the aircraft include:• Time to integrate and test the Iris system was more time-consuming than anticipated.
Planning more time for this type of integration and flight testing prior to the research flightis suggested.
• Adapting a sUAS autopilot into the Spyder for data recording presented some challenges
with extensions and placement of the GPS to prevent GPS glitches.
• Initial flight testing was required after integration to adjust the camera elevation andoverlap. This was very minor, but a step to be included in planning.
Flight testing reveled some additional lessons learned:• Having 2 pilots in each aircraft provided an extra set of eyes looking for the
intruder/surrogate UAS as well as additional aircraft operating in the area.
• The Spyder Ultralight is really slow at climbing. Modifications to the profiles are needed
to enable effective transitions from one encounter another.
• UAS climb-into and descend-into encounters are problematic. The Spyder has a hard time
climbing within the test encounter area (timing and geometry challenge) and it is likely thatthese encounters will be omitted in future testing.
• The uAvionic telemetry system allowed for ideal connectivity during long-range
operations. Tracking and monitoring of both the Spyder (telemetry) and CTLS (ADS-B)

95

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

•

were effective. Constant contact with both aircraft allowed for enhanced situationalawareness.
During some of the descend behind the UAS encounters, relatively poor visibility down
and in front of the CTLS made it hard to maintain visibility of the Spyder. Use of ADS-B
and iPads allowed for maintenance of situational awareness.

Lessons learned related to post flight operations and the DAA system performance include:• The DAA system tested was designed for operational use and the encounter data were noteasy to extract for analysis. It is quite easy to assess an individual encounter or series ofencounters using the vendor’s post flight tools, but more difficult to actually use the raw
extracted encounter data. To be fair, the raw data formats from the vendor were not set up
for the type of post-flight assessment performed by the team.
• The DAA system produced very good estimates of intruder aircraft altitude.
• Detections above the DAA system’s altitude were quite good. The detections where thesystem was looking downward from horizontal were not as good. Again, it should be notedthat the system was not designed/tuned to look below the horizon for aircraft so the ascentportion of the ascend-into encounters (before reaching approximate co-altitude) had limited
detections. The detections of the intruder aircraft were good when the aircraft was above
the simulated UAS altitude, but were limited after the descent portion of the descend-into
encounters.

4

OTHER TASKS/ADDITIONAL MATERIAL

4.1 OSU Radar System Development and TestingIn addition to the flight testing described in Section 3, the Ohio State University (OSU) developed
and tested a passive radar system that could be utilized, with further development, for DAA. Thissystem leverages reflections from aircraft of existing terrestrial television broadcasts. During A18,
this system was matured from a prototype system with non-real-time data processing of detections
to a system with real-time detection and tracking having a 2 s update rate. The system leverages
a passive detection array that enables estimation of target location (Figure 59). One of theadvantages of such a system is it does not require a transmitter (since it is passive), which decreases
cost and size, weight, and power requirements.

96

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 59. Image of the OSU passive radar system.

The OSU team obtained promising results with their system, as illustrated in Figure 60. Given thecost and SWaP benefits of such a system, this approach is promising for GBDAA. Future work
would include further evaluation in a DAA context, further evaluation of system characteristics(e.g., track accuracy), clutter susceptibility, etc.

97

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Figure 60. Results obtained with the OSU passive radar system. The figure on the left shows a track
developed with the radar (yellow) and associated ADS-B detections (red diamonds) for that aircraft. The
figure on the right shows range-rate versus range for tracked objects with ADS-B detections for a tracked
aircraft (red diamonds).

5

CONCLUSIONSThis effort addressed critical questions associated with sUAS DAA, including:• What are the use cases requiring DAA for BVLOS operations?
• What DAA systems are available, what are their capabilities and limitations, and are theymature enough to support BVLOS operations?
• What characteristics of DAA systems and UASs must be considered to ensure maintenanceof well clear status?
• How should sUAS DAA systems be evaluated to ensure they provide safe separation
services in the NAS?
• What is the recommended test method(s) to evaluate different DAA systems?
These were addressed through execution of the following tasks:
1. Development of an Operational Framework for sUAS BVLOS Operations—New Use
Cases, Industry Focus, and Framework Expansion
2. Update of sUAS DAA Solutions Inventory
3. Coordination with Standards Agency to Establish Framework
4. Development of Separation Framework
5. Development of a Testing Plan
6. Testing of a) the recommended DAA testing plan and b) candidate DAA systems
7. Final Report98THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Given the broad set of tasks, multiple methods were applied to execute them. These include review
of previous efforts (Tasks 1, 3-4, and 5), analysis and synthesis (Tasks 1-3 and 5-6), simulation(Task 4), and testing and validation (Task 6).
Results for Tasks 1 and 4-6 are provided in separate reports. The interested reader is directed tothose for a detailed description of results (citations are provided throughout this report). A highlevel summary of results is provided herein.
The Operational Framework effort resulted in a slight expansion of the previously-developed use
case taxonomy (Cathey and Hottman 2017) and identification of technologies that are now being
operationally utilized. Minimal additional information regarding expansion of the operationalframework based on RLOS coverage was identified. In addition, environments, conditions,assumptions, and limitations that will enable BVLOS operations are discussed.
Results of Task 2 (DAA Solutions Inventory) indicated that the characterization of DAA systems
provided by Askelson et al. (2017) still applies. This includes strengths and limitations of on/off
board systems identified by Askelson et al. (2017). Some of the most promising systems identified
in this task were utilized during evaluation of DAA test methods.
Through Task 3 (Coordination with Standards Agency), the team worked with others in theindustry to stand up two ASTM groups focused on sUAS DAA. The team has contributed
significantly to both groups, with support of the test methods group arguably being the mostsignificant. Important contributions have been provided by the team in the areas of test methods,
test artifacts, and the interplay of testing and simulation for evaluating DAA system performance.
Task 4 (Separation Framework) involved a significant simulation effort to evaluate DAA system
and UAS characteristics that impact maintenance of well clear. The DAA characteristics that had
the greatest impact are range and FoV. Update rate and latency impacts were not as dramatic,
while horizontal and vertical resolution were the least impactful. UAS characteristics that strongly
impact maintenance of well clear are response time (i.e., the time required to initiate a maneuver)and UAS speed.
An overarching test plan (Task 5) was developed.
This test plan describes test
locations/performers, dates of testing, DAA systems used in tests, overarching test objectives,
individual test plan structure, methods for maintenance of safety during testing, data collection
approaches, the structure of test reports, and test metrics/artifacts.
Seven rounds of flight tests were completed. Important outcomes from these tests include:• A systematic approach for evaluating DAA systems• Identification of test metrics/artifacts• Test data collection methods/best practices• Methods for enabling flight test safety• Methods for executing both horizontal and climb- and descend-into encounters• Evaluation of DAA systems (especially the detection component of DAA systems)
o Evaluation of performance—especially detection range and FoV
o Identification of challenges (clutter, etc.)• Utilization of results in ASTM committees to support standards development

99

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

This effort involved a broad set of tasks designed to inform FAA regulations and industry standards
regarding sUAS DAA systems. Through execution of these tasks and application of the numerous
methods required to do so, the A18 team has significantly advanced sUAS DAA, which will enable
more rapid integration of sUAS into the NAS—especially for BVLOS operations.

100

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

Bibliography
Askelson, M., 2022: Small UAS Detect and Avoid Requirements Necessary for Limited Beyond
Visual Line of Sight (BVLOS) Operations: Task 6 UND September 2020 Flight Test
Report. USDOT FAA (ASSURE), 84 pp.
Askelson, M., and A. Stephens, 2022: Small UAS Detect and Avoid Requirements Necessary for
Limited Beyond Visual Line of Sight (BVLOS) Operations: Task 6 UND June 2021 Flight
Test Report. USDOT FAA (ASSURE), 75 pp.
Askelson, M., et al., 2017: Small UAS Detect and Avoid Requirements Necessary for Limited
Beyond Visual Line of Sight (BVLOS) Operations. USDOT FAA (ASSURE), 516 pp.
ASTM 2020: Standard Specification for Detect and Avoid System Performance Requirements.
F3442/F3442-20, 22 pp.
Cathey, H., and S. B. Hottman, 2017: Small UAS Beyond Visual Line of Sight Operational
Framework Technical Report and Assessment. USDOT FAA ASSURE, 26 pp.
Cathey, H., G. Lenzo, J. Millette, and K. Maynard, 2019: Development of an Operational
Framework for Small UAS Beyond Visual Line of Sight (BVLOS) Operations—New Use
Cases, Industry Focus, and Framework Expansion. USDOT FAA (ASSURE), 44 pp.
Cathey, H., J. Millette, T. Lower, and A. Denney, 2020: A18 – Small UAS Detect and Avoid
Requirements Necessary for Limited Beyond Visual Line of Sight (BVLOS) Operations:Task 6 NMSU July 2019 Testing of Candidate DAA System Test Report. USDOT FAA(ASSURE), 69 pp.
Cathey, H., et al., 2022: ASSURE A18 - Small UAS Detect and Avoid Requirements Necessary
for Limited Beyond Visual Line of Sight (BVLOS) Operations: NM UAS Flight Test Site
November 2021 DAA System Test Report. USDOT FAA ASSURE.
C Speed, 2022: Lightwave Radar. Accessed 11 October 2022, http://www.cspeed.com/radar.
Kaabouch, N., M. Mullins, K. Foerster, and M. Askelson, 2020: Small UAS Detect and Avoid
Requirements Necessary for Limited Beyond Visual Line of Sight (BVLOS) Operations:
Separation Framework Report. USDOT FAA (ASSURE), 58 pp.
L3Harris, 2022: ADS-B Xtend™ for Airports. Accessed
https://www.l3harris.com/all-capabilities/ads-b-xtend-airports.

11

October

2022,

Remmert, W., and L. Purdy, 2020: Small UAS Detect and Avoid Requirements Necessary for
Limited Beyond Visual Line of Sight (BVLOS) Operations: ACUASI August 2019 DAA
System Test Report. USDOT FAA (ASSURE), 20 pp.
Remmert, W., and L. Purdy, 2021: Small UAS Detect and Avoid Requirements Necessary for
Limited Beyond Visual Line of Sight (BVLOS) Operations: ACUASI June 2021 DAA
System Test Report. USDOT FAA (ASSURE), 68 pp.

101

THIRD PARTY RESEARCH. PENDING FAA REVIEW.

RTCA, 2017: Minimum Operational Performance Standards (MOPS) for Detect and Avoid (DAA)
Systems. RTCA, Inc., 838 pp.
Weinert,A.,
Airspace-Encounter-Models.
Accesses
13
https://github.com/Airspace-Encounter-Models/em-overview.

October

2022,

Weinert, A, S. Campbell, A. Vela, D. Schuldt, and J. Kurucar, 2018: Well-clear recommendation
for small unmanned aircraft systems based on unmitigated collision risk. J. Air Transp.,
26, 113-122.

102