NASA LogoNTRSNTRS - NASA Technical Reports ServerSearchmore_vertCollectionsAboutNewsHelpLogin Back to ResultsEvaluating Alerting and Guidance Performance of a UAS Detect-And-Avoid SystemA key challenge to the routine, safe operation of unmanned aircraft systems (UAS) is the development of detect-and-avoid (DAA) systems to aid the UAS pilot in remaining "well clear" of nearby aircraft. The goal of this study is to investigate the effect of alerting criteria and pilot response delay on the safety and performance of UAS DAA systems in the context of routine civil UAS operations in the National Airspace System (NAS). A NAS-wide fast-time simulation study was conducted to assess UAS DAA system performance with a large number of encounters and a broad set of DAA alerting and guidance system parameters. Three attributes of the DAA system were controlled as independent variables in the study to conduct trade-off analyses: UAS trajectory prediction method (dead-reckoning vs. intent-based), alerting time threshold (related to predicted time to LoWC), and alerting distance threshold (related to predicted Horizontal Miss Distance, or HMD). A set of metrics, such as the percentage of true positive, false positive, and missed alerts, based on signal detection theory and analysis methods utilizing the Receiver Operating Characteristic (ROC) curves were proposed to evaluate the safety and performance of DAA alerting and guidance systems and aid development of DAA system performance standards. The effect of pilot response delay on the performance of DAA systems was evaluated using a DAA alerting and guidance model and a pilot model developed to support this study. A total of 18 fast-time simulations were conducted with nine different DAA alerting threshold settings and two different trajectory prediction methods, using recorded radar traffic from current Visual Flight Rules (VFR) operations, and supplemented with DAA-equipped UAS traffic based on mission profiles modeling future UAS operations. Results indicate DAA alerting distance threshold has a greater effect on DAA system performance than DAA alerting time threshold or ownship trajectory prediction method. Further analysis on the alert lead time (time in advance of predicted loss of well clear at which a DAA alert is first issued) indicated a strong positive correlation between alert lead time and DAA system performance (i.e. the ability of the UAS pilot to maneuver the unmanned aircraft to remain well clear). While bigger distance thresholds had beneficial effects on alert lead time and missed alert rate, it also generated a higher rate of false alerts. In the design and development of DAA alerting and guidance systems, therefore, the positive and negative effects of false alerts and missed alerts should be carefully considered to achieve acceptable alerting system performance by balancing false and missed alerts. The results and methodology presented in this study are expected to help stakeholders, policymakers and standards committees define the appropriate setting of DAA system parameter thresholds for UAS that ensure safety while minimizing operational impacts to the NAS and equipage requirements for its users before DAA operational performance standards can be finalized.Document ID20160012543Acquisition SourceAmes Research CenterDocument TypeTechnical Memorandum (TM)AuthorsLee, Seung Man*(California Univ. Santa Cruz, CA, United States)*Park, Chunki*(California Univ. Santa Cruz, CA, United States)*Thipphavong, David P.*(NASA Ames Research Center Moffett Field, CA United States)*Isaacson, Douglas R.*(NASA Ames Research Center Moffett Field, CA United States)*Santiago, Confesor*(NASA Ames Research Center Moffett Field, CA United States)*Date AcquiredOctober 24, 2016Publication DateFebruary 1, 2016Subject CategoryAir Transportation And SafetyReport/Patent NumberARC-E-DAA-TN29444NASA/TM-2016-219067Funding Number(s)CONTRACT_GRANT: NAS2-03144WBS: WBS 357672.04.01.01.01Distribution LimitsPublicCopyrightPublic Use Permitted.KeywordsUAS Detect-and-Avoid (DAA) SystemAlerting and guidance PerformanceSystem Operating Characteristics (SOC) curvesAvailable DownloadsNameType20160012543.pdfSTIcloud_downloadcontent_copyvisibilityRelated RecordsThere are no records associated with this record.visibility_offNo Preview Available
 No Fear Act | Freedom of Information Act | Office of the Inspector General | Agency Financial ReportsNASA Web Privacy Policy and Important Notices | NASA Guidelines for Quality of InformationDisclaimers, Copyright, Terms of Use | Accessibility | NASA OCIO | USA.gov | NASA.govContact Us | Site MapSTI LogoNASA Official: JoAnne R. Calhoun Site Curator: STI Compliance and Distribution Services Last Modified: September 12, 2024