Skip to main content    

An official website of the United States government

Here’s how you know

**Official websites use .gov**              A **.gov** website belongs to an official government organization in the United States.            

**Secure .gov websites use HTTPS**              A **lock** (  LockA locked padlock) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.            

https://www.nist.gov/el/intelligent-systems-division-73500/standard-test-methods-response-robots/aerial-systems/aerial

            -  Topics  -  All Topics 
-  Advanced communications 
-  Artificial intelligence 
-  Bioscience 
-  Buildings and construction 
-  Chemistry 
-  Climate 
-  Cybersecurity 
-  Electronics 
-  Energy 
-  Environment 
-  Fire 
-  Forensic science 
-  Health 
-  Information technology 
-  Infrastructure 
-  Manufacturing 
-  Materials 
-  Mathematics and statistics 
-  Metrology 
-  Nanotechnology 
-  Neutron research 
-  Performance excellence 
-  Physics 
-  Public safety 
-  Resilience 
-  Standards 
-  Transportation 
 
            -  Publications 
            -  Labs & Major Programs  -  Laboratories    -  Communications Technology Laboratory 
  -  Engineering Laboratory 
  -  Information Technology Laboratory 
  -  Material Measurement Laboratory 
  -  Physical Measurement Laboratory 
-  User Facilities    -  NIST Center for Neutron Research 
  -  CNST NanoFab 
-  Research Test Beds 
-  Research Projects 
-  Tools & Instruments 
-  Major Programs    -  Baldrige Performance Excellence Program 
  -  CHIPS for America Initiative 
  -  Manufacturing Extension Partnership (MEP) 
  -  Office of Advanced Manufacturing 
  -  Special Programs Office 
  -  Technology Partnerships Office 
 
            -  Services & Resources  -  Standards and Measurements    -  Calibration Services 
  -  Laboratory Accreditation (NVLAP) 
  -  Quality System 
  -  Standard Reference Materials (SRMs) 
  -  Standards.gov 
  -  Time Services 
  -  Office of Weights and Measures 
-  Software 
-  Data    -  Chemistry WebBook 
  -  National Vulnerability Database 
  -  Physical Reference Data 
  -  Standard Reference Data (SRD) 
-  Storefront 
-  License & Patents 
-  Computer Security Resource Center (CSRC) 
-  NIST Research Library 
 
            -  News & Events  -  News 
-  Events 
-  Blogs 
-  Feature Stories 
-  Awards 
-  Video Gallery 
-  Image Gallery 
-  Media Contacts 
 
            -  About NIST  -  About Us    -  Leadership 
  -  Organization Structure 
  -  Budget & Planning 
-  Contact Us 
-  Visit 
-  Careers    -  Student programs 
-  Work with NIST 
-  History    -  NIST Digital Archives 
  -  NIST Museum 
  -  NIST and the Nobel 
-  Educational Resources 
 

Engineering Laboratory

## Intelligent Systems Division

-  Standard Test Methods for Response Robots  Expand or Collapse    -  NIST's Robotics Test Facility 
  -  Collaborating Facilities 
  -  Ground Systems 
  -  Aerial Systems  Expand or Collapse      -  Aerial Program Overview 
    -  Open Test Lanes and Scenarios 
    -  Obstructed Test Lane and Scenarios 
    -  Confined Test Lane and Scenarios 
    -  Sensor Test Lane 
    -  Simulation 
    -  Forms 
    -  Frequently Asked Questions 
  -  Aquatic Systems 
  -  Robot Competitions  Expand or Collapse      -  RoboCup Rescue 
    -  Aerial Competitions 
  -  Events and Meetings  Expand or Collapse      -  2023.01 ASTM E54.09 Meeting San Antonio, Texas 
    -  2022.06 ASTM E54.09 Virtual Meeting 
    -  2022.01 ASTM E54.09 Virtual Meeting 
    -  2021.10 UTAC Conference, Guardian Centers, Perry, GA 
    -  2021.06 ASTM E54.09 Virtual Meeting 
    -  2021.02 ASTM E54.09 Virtual Meeting 
    -  2018.06 ASTM E54.09 San Diego, CA 
    -  2018.02 ASTM E54.09, Gaithersburg, MD 
    -  2017.06 ASTM E54.09, Toronto, Canada 
    -  2017.01 ASTM E54.09/Virginia Beach Exercise, Virginia Beach, VA 
    -  Prior Events and Meetings 

# Aerial Program Overview

## Share

Facebook

Linkedin

Twitter

Email

                  -  Background 
                  -  Definition of Response Robot Includes sUAS 
                  -  Difference Between Standard Test Method and a Standard Robot 
                  -  Approach 
                  -  Impacts 
                  -  Initial Test Method Suite 
                  -  Frequently Asked Questions 

## **Background**

The National Institute of Standards and Technology, Engineering Laboratory, Intelligent Systems Division, is developing the measurement and standards infrastructure necessary to evaluate robotic capabilities for emergency responders and military organizations addressing critical national security challenges. This includes leading development of a comprehensive suite of *ASTM International Standard Test Methods for Response Robots*that includes more than 50 test methods for remotely operated ground, aerial, and maritime systems. Several different civilian and military sponsors have supported this effort. And the test methods have been replicated and used in dozens of locations worldwide to measure and evaluate response robot capabilities to meet the following objectives:
                  - Facilitate communication between user communities and commercial developers, or between development program managers and performers, by representing essential capabilities in the form of tangible test apparatuses, procedures, and performance metrics that produce quantifiable measures of success.
                  - Inspire innovation and guide developers toward implementing the combinations of capabilities necessary to perform essential mission tasks.
                  - Measure progress, highlight break-through capabilities, and encourage hardening of developmental systems through repeated testing and comparison of quantitative results.
                  - Inform purchasing and deployment decisions with statistically significant capabilities data. To date, the suite of ground robot test methods have been used to specify more than $60M worth of robot procurements for military and civilian organizations performing C-IED missions.
                  - Focus operator training and measure proficiency to track very perishable skills over time and enable comparison across squads, regions, or national averages. To date, the suite of ground robot test methods have been used by more than 250 civilian and military bomb technicians.

The suite includes 15 draft standard test methods for evaluating small initial emphasis on vertical take-off and landing systems and small hand launched fixed wing systems. For the VTOL systems, testing and practice starts within netted aviaries indoors and outdoors to avoid issues of flying in the national airspace. The test methods measure essential capabilities of robots and operator proficiency for hazardous missions defined by emergency responders and soldiers.

[Top]

## **Definition of Response Robot Includes sUAS**

The working definition of a “response robot” is a remotely deployed device intended to perform operational tasks at operational tempos. It should serve as an extension of the operator to improve remote situational awareness and provide means to project operator intent through the equipped capabilities. It should also improve effectiveness of the mission while reducing risk to the operator. Key features include:
                  - Rapidly deployed
                  - Remotely operated from an appropriate standoff
                  - Maneuverable in complex environments
                  - Sufficiently hardened against harsh environments
                  - Reliable and field serviceable
                  - Durable or cost effectively disposable
                  - Equipped with operational safeguards

[Top]

## **Difference Between a “Standard Test Method” and a “Standard Robot”?**

Standard test methods are simply agreed upon ways to objectively measure robot capabilities. They isolate certain robot requirements and enable repeatable testing. They are developed and validated through a consensus process with equal representation of users, developers, and test administrators. In the case of sUAS systems, one key standards committee is the ASTM International Standards Committee on Unmanned Aircraft Systems (F38). The resulting robot capabilities data captured within standard test methods can be directly compared even when the tests are performed at different sites at different times. They help establish confidence in a robot and remote operator’s ability to reliably perform particular tasks. Each standard test method includes the following elements:
                  - Apparatus (or prop):  A repeatable, reproducible, and inexpensive representation of tasks that the system is expected to perform. The apparatus should challenge the system with increasing difficulty or complexity and be easy to fabricate internationally to ensure all robots are measured similarly.
                  - Procedure:  A script for the system operator to follow (along with a Test Administrator when appropriate).  These tests are not intended to surprise anybody.  They should be practiced to refine designs and improve techniques.
                  - Metric:  A quantitative way to measure the capability. For example, completeness of 10 continuous repetitions of a task or distance traversed within an environmental condition, etc. Together with the elapsed time, a resulting rate in tasks/time or distance/time can be calculated.
                  - Fault Condition:  A failure of the robotic system preventing completion of 10 or more continuous repetitions. This could include a stuck or disabled vehicle requiring maintenance, or software issues at the remote operator control unit. All such failures are catalogued during testing to help identify recurring issues.

A “standard robot” is a completely different concept and should be carefully considered. A standard robot would presumably meet a well-defined equipment specification for size, shape, capabilities, interfaces, and/or other features. Such equipment standards are typically intended to improve compatibility, enable interoperability, increase production, lower costs, etc. This can be important for many reasons, but can also hinder innovation. In this effort, we **are not**proposing to develop a standard sUAS specification of any kind.  Rather, we are proposing to develop standard test methods that can be used to evaluate and compare entire classes of such systems in objective and quantifiable ways. The resulting suite of standard test methods could indeed be used to specify a standard sUAS at some point, similar to how they are used to specify purchases. But that is left to each user community to define for themselves based on their particular mission objectives. We use standard test methods to encourage implementation of new technologies to improve capabilities and measure progress along the way.

[Top]

## Approach

NIST is leading an international effort to develop the measurements and standards infrastructure necessary to evaluate and compare robotic capabilities for emergency responders and military organizations. The resulting standard test methods and performance metrics address critical national security challenges by enabling quantitative measurement and comparison of system capabilities and operator proficiency. They have been replicated by dozens of organizations worldwide to measure and evaluate response robot capabilities. They have helped inform procurement and deployment decisions with statistically-significant robot capabilities data for a variety of essential mission tasks. They have also helped guide robot manufacturers toward innovations that answer responder needs while encouraging hardening of developmental systems. To date, these standards have been used to specify more than $70M worth of ground response robot procurements for firefighters, bomb squads, and soldiers. These standards are now beginning to also focus operator training with newly developed measures of operator proficiency. 

There are roughly twenty sUAS test methods under development within the overall suites of test methods. The initial sUAS emphasis is on vertical take-off and landing systems and hand launched fixed wing systems. Ten of these test methods for Maneuvering and Payload Functionality are described in this document, having been adopted as measures of operator proficiency for Job Performance Requirements (JPR) within the NFPA 2400 

Standard for sUAS Used for Public Safety Operations. 
**Figure 1:**

Manufacturing

-  **‹** Aerial Systems 
-  Open Test Lanes and Scenarios **›** 

### HEADQUARTERS

301-975-2000

Webmaster | Contact Us | Our Other Offices

TwitterFacebookLinkedInInstagramYouTubeGiphyRSS FeedMailing List

Feedback

            -  Site Privacy 
            -  Accessibility 
            -  Privacy Program 
            -  Copyrights 
            -  Vulnerability Disclosure 
            -  No Fear Act Policy 
            -  FOIA 
            -  Environmental Policy 
            -  Scientific Integrity 
            -  Information Quality Standards 
            -  Commerce.gov 
            -  Science.gov 
            -  USA.gov 
            -  Vote.gov