Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Use Case Examples

Aerial Test Methods

Sub Committee Chair:

Adam Jacoff

Intelligent Systems Division

National Institute of Standards and Technology

U.S. Department of Commerce

Committee Chair:

Phil Mattson

Science and Technology Directorate 

U.S. Department of Homeland Security

Internet

RobotTestMethods.nist.gov

Adam	Jacoff

Intelligent	Systems	Division

NATIONAL	INSTITUTE	OF	STANDARDS	AND	TECHNOLOGY

U.S.	Department	of	Commerce

Email

RobotTestMethods@nist.gov

1

Version 2021A

WEBSITE POINTER: 

WATCH FABRICATION VIDEOS 

AND FLIGHT PATH ANIMATIONS

WEBSITE POINTER: 

DOWNLOAD STICKER FILES, FORMS AND 

PRACTICE SCORING VIDEOS Online Only Meeting

February 3, 2021

10:00am – 2:00pm EST
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

2

Aerial Test Methods

Use Case Examples

.

• Recent Validation Exercises 

• DHS/DOJ procurement evaluations

• Japan Manned Space Systems Corp.

• CA: Los Angeles Fire Dept. 

• CO: CoE for Advanced Technology Aerial Firefighting, 

• TX: Dept. of Public Safety and Reveille Peak Ranch Test Facility

• NE: Omaha Police Dept.

• OH: Woodlawn Fire Dept. 

• Canada: Canadian Emergency Response Robotics Assoc.

• APSA: ”Train the Trainer” class scheduled

• DroneResponders: Auditing for ASTM

POST

T-TOP

LEGS

POST

4x4x24in (10x10x60cm)

LEGS

2x4x20in (5x10x50cm)

with 45° opposing ends

T-TOP

2x4x12in (5x10x30cm)

BUCKETS

2 or 5 gal (7 or 19 liter)

Launch/Land

12in (30cm) Radius Center

Bucket Stand #1

(A,B,C,D)

10 ft (3 m) Spacings

FLIGHT LINE

SIGHT LINE

Launch/Land

12in (30cm) Radius Center

FLIGHT LINE

SIGHT LINE

Bucket Stand #1

(A,B,C,D)

Bucket Stand #2 

(E,F,G,H)

Bucket Stand #3

(I,J,K,L)

10ft (3m) Spacings

10 FT (3 M)

ALTITUDE LAUNCH LAND

Test Director 

Adam Jacoff

Intelligent Systems Division

Nat’l Institute of Standards and Technology 

U.S. Dept. of Commerce 

RobotTestMethods@nist.gov | 301-975-4235 

Sponsor 

Philip Mattson

Science and Technology Directorate 

U.S. Dept. of Homeland Security

These test methods are primarily intended for 

vertical takeoff and landing systems with an 

onboard camera and remote pilot display. Some 

are also applicable to fixed wing systems when the 

lane dimensions are extended to accommodate 

the orbit radius of forward flying systems. 

Bucket stands on a level surfaces ensure the top bucket is 

vertical and the angled buckets are 45 degrees. 

TRAVERSE TEST

Right and Left Ovals Around All Three Bucket Stands

ALIGNED

WHEN RING IS 

COMPLETELY 

VISIBLE

GROUND

AQUATIC

AERIAL
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

3

Validation Exercises

Use Case Examples

Disaster-Resilient Buildings, Infrastructure, 

and Communities Goal

Technical Expertise: Standards Committees

Slide37

Assess the portfolio of scientific expertise

Chair/Fellow, ASTM International Standards Committee on Homeland 

Security Applications: Response Robots (ASTM E54.09) 

This committee is standardizing 50 test methods for ground, aerial, and 

aquatic systems. It hosts two meetings annually to facilitate development 

and balloting of standards. Typically we conduct an associated test 

method validation exercise.

Member, NFPA Standards Committee on Small Unmanned Aircraft 

Systems Used for Public Safety Applications (NFPA 2400)

This committee has adopted our test methods to support Job 

Performance Requirements for remote pilot proficiency and will likely set 

thresholds of performance for firefighting and other public safety 

applications. 

Member, NATO SCI-ET-052 Research Task Group on EOD Robot TeleManipulation This committee is developing NATO’s roadmap for 

dexterous robot applied to C-IED/EOD applications.

2019 Host: Houston Fire Dept

2017 Host: Canadian CETA

2018 Host: San Diego Fire Dept

• 2020.08 DHS/DOJ sUAS Procurement Testing ($35M), Montgomery County Police Facility, MD (1 days)

• 2020.10 Air Force Large Ground Robot Procurement ($70M), Tyndall AFB, FL (Weeks)

• 2020.08 DHS/DOJ sUAS Procurement Testing ($35M), Montgomery County Police Facility, MD (5 days)

• 2020.09 Canadian Fire Training Facility Opening Exercise, Toronto Airport, Ontario, Canada (4 days)

• 2020.08 World Robot Summit Disaster Response Championship, Fukushima, Japan (4 days)

• 2020.06 RoboCupRescue International Championship, Bordeaux, France (5 days)

• 2020.05 AUVSI Exponential Conference (netted aviary), Boston, MA (3 days)

• 2020.04 Fire Dept. International Conference (FDIC) Hands-On Training, Indianapolis, IN (3 days)

• 2020.03 UTAC UAS Conference, Guardian Center, Perry, GA (4 days)

• 2020.03 Public Safety UAS Conference Validation Exercise, Crozet, VA (5 days)

Color Key: Ground Aerial Aquatic Multiple Standards
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

4

Validation Exercises

Use Case Examples

Disaster-Resilient Buildings, Infrastructure, 

and Communities Goal

Technical Expertise: Standards Committees

Slide37

Assess the portfolio of scientific expertise

Chair/Fellow, ASTM International Standards Committee on Homeland 

Security Applications: Response Robots (ASTM E54.09) 

This committee is standardizing 50 test methods for ground, aerial, and 

aquatic systems. It hosts two meetings annually to facilitate development 

and balloting of standards. Typically we conduct an associated test 

method validation exercise.

Member, NFPA Standards Committee on Small Unmanned Aircraft 

Systems Used for Public Safety Applications (NFPA 2400)

This committee has adopted our test methods to support Job 

Performance Requirements for remote pilot proficiency and will likely set 

thresholds of performance for firefighting and other public safety 

applications. 

Member, NATO SCI-ET-052 Research Task Group on EOD Robot TeleManipulation This committee is developing NATO’s roadmap for 

dexterous robot applied to C-IED/EOD applications.

2019 Host: Houston Fire Dept

2017 Host: Canadian CETA

2018 Host: San Diego Fire Dept

Color Key: Ground Aerial Aquatic Multiple Standards

• 2020.02 ASTM E54.09 Response Robots Meeting and Exercise, Atlanta, CO (3 days)

• 2020.01 Ohio Fire Training Facility Opening, Ohio (2 days)

• 2020.01 FDIC Fire/Rescue East, Daytona, FL (2 days)

• 2020.01 Los Angeles Fire Dept. Training, Los Angeles, CA (3 days)

• 2019.12 FAA Requirements Workshop for Fire Depts and Emergency Services, NIST (1 day)

• 2019.11 Atlantic Future Forum, UK HMS Queen Elizabeth, Annapolis, MD (2 days)

• 2019.11 DHS Familiarization Exercise, Army Camp Shelby, MS (5 days)

• 2019.10 World Robot Summit, Fukushima, Japan (5 days)

• 2019.09 NATO Aerial and Ground Exercise, Base Borden, Ontario, Canada (3 days)

• 2019.07 Aerial Validation Exercise at NIST (3 days)

• 2019.06 RoboCupRescue International Championship, Sydney, Australia (5 days)
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

5

Validation Exercises

Use Case Examples

Disaster-Resilient Buildings, Infrastructure, 

and Communities Goal

Technical Expertise: Standards Committees

Slide37

Assess the portfolio of scientific expertise

Chair/Fellow, ASTM International Standards Committee on Homeland 

Security Applications: Response Robots (ASTM E54.09) 

This committee is standardizing 50 test methods for ground, aerial, and 

aquatic systems. It hosts two meetings annually to facilitate development 

and balloting of standards. Typically we conduct an associated test 

method validation exercise.

Member, NFPA Standards Committee on Small Unmanned Aircraft 

Systems Used for Public Safety Applications (NFPA 2400)

This committee has adopted our test methods to support Job 

Performance Requirements for remote pilot proficiency and will likely set 

thresholds of performance for firefighting and other public safety 

applications. 

Member, NATO SCI-ET-052 Research Task Group on EOD Robot TeleManipulation This committee is developing NATO’s roadmap for 

dexterous robot applied to C-IED/EOD applications.

2019 Host: Houston Fire Dept

2017 Host: Canadian CETA

2018 Host: San Diego Fire Dept

Color Key: Ground Aerial Aquatic Multiple Standards

• 2019.06 ASTM E54.09 Response Robots Meeting and Exercise, Denver, CO (5 days)

• 2019.05 Western Regional Robot Rodeo, Sandia/Kirtland, Albuquerque, NM (5 days)

• 2019.05 Canadian Police College Training Exercise, London, ON Canada (7 days)

• 2019.04 Thermite RS2 firefighting robot capabilities evaluation (1 day)

• 2019.04 Army Tank Automotive Research and Development facility fabrication (remote)

• 2019.04 Fire Dept Training Conference (FDIC), Indianapolis, IN (3 days)

• 2019.04 Guardian Center Training, Perry, GA (2 days remote)

• 2019.04 Reveille Ranch Calibration, Texas Dept of Public Safety, Burnet, TX (2 days)

• 2019.04 InstantEye UAS capabilities evaluation, NIST (3 days)

• 2019.03 ASTM F38 standard balloted referencing 6 of our aerial test methods

• 2019.03 Navy Explosive Ordinance Disposal Tech Division facility fabrication (remote)

• 2019.03 Virginia UAS Summit on Public Safety, Crozet, VA (3 days)
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

California: Los Angeles CITY Fire Dept.

Use Case Examples
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Los Angeles CITY Fire Department

Unmanned Aerial System (UAS) Program

• UAS Program approved by LA City Council, 2017

• Established to fight fires, improve efficiency of training, 

respond to high risk incidents

• Developed UAS applications for:

• Brush Area Hazards inspections

• Brush Fire Mapping

• Thermal Hotspot Identification - assisting firefighters 

to identify and extinguish active fires

• HAZMAT, Urban Search and Rescue, and Swift Water 

Operations

• Situational Awareness - video-streaming to Emergency 

Operations Center for large scale events
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Public Safety UAS Standardization with 

NIST Aerial Test Methods

HOW LAFD uses NIST Aerial Test Methods:

• Basic Evaluation Standards to objectively certify LAFD Basic Remote Pilots

• Recognized by FAA and NFPA as recommended evaluation tool

• Scalable to evaluate advanced procedures and application

• Practical to evaluate equipment and perform Functional Check Flights (FCF)
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

1

ASTM E54.09

Status report on 

sUAV-STM evaluation 

excersize in Japan 

4th Feb. 2021

Japan: International Rescue System Institute

Use Case Examples
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

2

Purpose of this paper

To share the results of sUAV-STM evaluation exercose

in Japan (Nagaoka, Niigata) held on 25th-26th Jan. 

2021
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

3

Summary

Purpose: To identify the issues on the application of NIST 

sUAV-STM in Japan

Date: Jan. 25 - 26th 2021

Location: Nagaoka / Niigata, Japan

Participants: Pilot 11 , Proctor 6

Tasks: Basic / Position & Traverse (MAN only, 2-Lap each )

Lane: 4 lanes (simultaneous flight)

For the COVID-19 countermeasure;

Reduced number of the on-site 

Participants by, 

ü YouTube Live distribution

ü Independent 3-event repeated 

for 3 groups 

Day 1 Day2

9:30 - Preparation Group B (4 lanes) 

- Briefing

- Exercise (1 hr.)

- Flight

- Debriefing

13:30 –

-16:30

Group A (3 lanes)

- Briefing

- Exercise(1 hr.)

- Flight 

- Debriefing

Group C (4 lanes) 

- Briefing

- Exercise(1 hr.)

- Flight

- Debriefing

-19:00 Cleanup
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

4

Summary

n Open spaces for 4 lanes

Total 11 pilots flew the sUAV without GPS on 4 lanes in 2 days. 
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

5

Summary

n Safe Flight Booth

Left : Pilot with camera on his helmet for video 

distribution by YouTube.

Right : Visual Observer watching the large monitor to 

keep the Social Distance with the pilot
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

6

Summary

n Cameras and Video distribution system
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

7

Results and analysis

n Major Results 

75%

19%

6%

YES

Neutral

NO

Q Is the sUAV-STM easy to understand ?

63%

31%

6%

YES

Neutral

No

Q Is the sUAV-STM effective method for 

evaluating the PERFORMANCE OF sUAV?

Q Is the sUAV-STM effective method for 

evaluating the PERFORMANCE OF PILOTs?

69%

19%

12%

YES

Neutral

NO

Overall, the participantsʼ comments are positive for applying the NIST sUAVSTM into Japan.
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

8

Issues and analysis

n Results

Major comments

The targets shown on the Monitor are too small to judge the alignment.

Larger monitors have advantages for getting better scores, which is not fair.

expected actual

The size of monitor and/or the size of the targets shown on the monitor 

should be checked in advance. 
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

9

Issues and analysis

n Results

Major comments

It is important to record the detail of the sUAV specifications & pilots’ experiences, 

since those “conditions” drives the results of “man-machine integrated performance. 

Following information should be recorded on the evaluation form.

[Pilot info]

• Flight hours

• Type of sUAV business ( surveyor etc.) 

• BVLOS flight experiences 

• Major operation is Auto or Manual

• STM experiences

• normally 1 person flight (both of 

maneuvering and camera operations) or 2-

person operation (Camera operation is done 

by another person) 

[sUAV info]

• Visual sensor On / Off (P-mode flight 

or A-mode flight etc.)

• with or without Camera Zoom 

• Size of Monitor

• Size of Target view (Green Circle) 

shown on the monitor

• Velocity of Camera Tilt control

• Is the sUAV pilot’s familiar vehicle or 

the one flew for the first time today.
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

10

Issues and analysis

n Discussions 

One of the major debriefing discussion is the “allowable navigation by 

proctor for the pilot during flight.” 

During our event, we allowed the proctors to give some “words of guide” 

such as “ next, go to 2, forward ” “next, camera tilt down for 2A” to the pilot.

Some proctors gave the pilots a “ navigation for smooth flight and camera 

shooting” which may lead better score. On the other hand, some proctors 

might give some “waiting-time to the pilot” during recording on the form, 

which makes the pilotsʼ score worse. 

These navigations should be standardized and proctors should be trained . 
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

機体メーカー

機体モデル (例:Phantom )

機種 (例:4 Pro)

操縦者コード/監視者コード      /

施設

日時

開始時刻

判定根拠 (どちらかに〇)

0 スタート地点で時計を撮影し、上昇 Alighしたら〇 操縦者  撮影画像

1 スタート地点の高さ1Sでホバー 1A T B L T R B R T L

2 (上からみて)時計回りで機体を上の図の様に移動 1B R L B R T 飛行 SCORE

3 2B L R T R B L L Align合計

4 3B L B L B L B / 20

5 3C R T R B R T R E LIA B ILIT Y

6 3D B T R B L L T L (Align合計 / 20) x 100

7 2D T L B T L B R %

8 1D B B L T L T R R E F F IC IE N C Y

9 1A T B L T R B R T L Align合計/ 分

10 着陸地点の中央へ降下 中央/Perch1 T B L T R B R T L R A T E

反対周り

11 スタート地点の高さ1Sでホバー 1A T B L T R B R T L P A Y LO A D S C O R E

12 (上からみて)反時計回りで機体を上の図の様に移動 1D B B L T L T R R 正解したGap合計

13 2D T L B T L B R      / 100

14 3D B T R B L L T L 平均ACUITY

15 3C R T R B R T 正解したGap合計/Align合計

16 3B L B L B L B R IN G S

17 2B L R T R B L L E F F IC IE N C Y

18 1B R L B R T 正解したGap合計/分

19 1A T B L T R B R T L R A T E

20 着陸地点の中央へ降下 中央/Perch2 L R T R B L L

手順 | POSITION FO R M S A N S W ER K EY V ER S IO N 2020A

Circle Gap 正解なら〇

架台側を見て中央に着陸 ー 時計の撮影 ー このLapの終了

評価を中断した場合には記録全体に斜め線を引いて無効とし、右の理由に〇をつける。: 装置  落下  境界へ到達  安全上の理由

3m 5m 10m 

その他( m)

レーン間隔

平均 突風

m/s m/s

風速

5 10

分 分 分

時間制限

Standard Test Methods for Small Unmanned AircraftSystems ASTM International Standards Committee on Homeland SecurityApplications; Response Robots (E54.09) 

| Website:RobotTestMethods.nist.gov

太陽光 電灯 採光なし

1000+ Lx 300+ Lx < 1 Lx 

照明の状況

目視 / 目視外

部分的

操縦者の視覚

Interface

のみ

トライアル

番号

11

Issues and analysis

n Other topics

For the better 

understanding 

of NIST Score Form, 

Japanese-version was used 

for participants temporarily.

P osition 機体メーカー

M A N / P A Y 1 機体モデル (例:Phantom )

機種 (例:4 Pro)

操縦者コード/監視者コード      /

施設

日時

開始時刻

判定根拠 (どちらかに〇)

0 スタート地点で時計を撮影し、上昇 Alighしたら〇 操縦者  撮影画像

1 架台1の高さ1Sでホバー 1 T B L T R B R T L

2 下側の1をAlign/撮影、次に前方45°の2AをAlign/撮影 2A B T L T R B L B R 飛行 SCORE

3 機体を左回転 360° 1 T B L T R B R T L Align合計

4 下側の1をAlign/撮影、次に前方45°の2AをAlign/撮影 2A B T L T R B L B R / 20

5 機体を右回転 360° 1 T B L T R B R T L R E LIA B ILIT Y

6 下側の1をAlign/撮影、次に前方45°の2AをAlign/撮影 2A B T L T R B L B R (Align合計 / 20) x 100

7 架台1の高さ2Sへ上昇 1 T B L T R B R T L %

8 下側の1をAlign/撮影、次に前方45°の3AをAlign/撮影 3A B L T B L T L E F F IC IE N C Y

9 架台1の高さ1Sへ下降 1 T B L T R B R T L Align合計/ 分

10 下側の1をAlign/撮影、次に前方45°の2AをAlign/撮影 2A B T L T R B L B R R A T E

11 架台2へ移動 2 B L T B L T L

12 下側の2をAlign/撮影、次に前方45°の3AをAlign/撮影 3A B L T B L T R P A Y LO A D S C O R E

13 架台1へ後退 1 T B L T R B R T L 正解したGap合計

14 下側の1をAlign/撮影、次に前方45°の2AをAlign/撮影 2A B T L T R B L B R      / 100

15 架台2へ移動し、機体を右回転180° 2(逆) T R B T R B R 平均ACUITY

16 下側の2(逆)をAlign/撮影、次に前方45°の1CをAlign/撮影 1C B L B L B R 正解したGap合計/Align合計

17 着陸地点上空まで移動し、機体を左回転 180° 着陸地点 T R B R B R R IN G S

18 下側の着陸地点をAlign/撮影、次に前方45°の1AをAlign/撮影 1A T B L T R B R T L E F F IC IE N C Y

19 着陸地点の中央へ降下(2点) 中央/Perch1 T B L T R B R T L 正解したGap合計/分

20 前方のPerch 1、Perch 2を順に撮影 中央/Perch2 L R T R B L L R A T E

手順 | POSITION FO R M S A N S W ER K EY V ER S IO N 2020A

Circle Gap 正解なら〇

架台側を見て中央に着陸 ー 時計の撮影 ー このLapの終了

評価を中断した場合には記録全体に斜め線を引いて無効とし、右の理由に〇をつける。: 装置  落下  境界へ到達  安全上の理由

3m 5m 10m 

その他( m)

レーン間隔

太陽光 電灯 採光なし

1000+ Lx 300+ Lx < 1 Lx 

照明の状況

平均 突風

m/s m/s

風速

目視 / 目視外

部分的

操縦者の視覚

5 10

分 分 分

時間制限

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland SecurityApplications; Response Robots (E54.09) 

| Website:RobotTestMethods.nist.gov

Interface

のみ

トライアル

番号
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

12

Future Plan

n Further study and trial for sUAV-STM in Japan

We are going to further study the appropriate application of NIST 

sUAV-STM method in Japan including following discussions,

ü Proctor and visual observer training program 
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

22

World Robot Summit (2018-2020)

Use Case Examples

Standard Disaster Robotics Category, Fukushima Robot Test Field, Fukushima, Japan
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

23

RoboCupRescue Robot League (2000-present)

Use Case Examples

Dissemination: Robotics Research Competitions

Assess the effectiveness of disseminating project outputs

• We conduct annual international robotics research competitions, sometimes two a year.

• The RoboCupRescue Championships (shown below) use 20 ground robot tests set up in a large maze so 

they can be conducted individually as preliminaries then a comprehensive search mission for finals. 

• These competition focus on autonomous behaviors vs. remote teleoperation using all three sizes of

robot test lanes. Typically more than 30 teams participate.

• Most teams fabricate the test methods at their facilities to refine designs and practice.

p

ASTM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09)

This annual competition, chaired by Adam Jacoff since its inception in 2001, has evolved over the years into 

a rigorous validation exercise for the suite of standard test methods being used. It also provides multiple 

layers of outreach to different communities. The test methods inspire innovation in teams of robotics 

researchers needing reproducible measurements to evaluate and compare a wide variety of 

implementations. International test administrators learn how to conduct standardized testing in their own 

facilities. Program managers and companies use the quantitative results to identify best-in-class 

performers. Emergency responders and others learn how to use standard test methods to focus training

and measure operator proficiency prior to deploying into operational scenarios. In general, these 

competitions have become a very effective way to disseminate standard test methods internationally. As a 

result of this competition, facilities based on our test methods are being set up in Japan, Korea, and 

Thailand bringing the total to 15 worldwide.

Figure 1: The venue was an indoor sports arenaAll the test apparatuses were purchased and fabricated by 

Standard Test Methods for Response Robots

ASTM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09)

Figure 2: More than 35 ground and aerial test methods were arranged as individual testsrepeatable 

NAGOYA, JAPAN LARGE MAZE OF 20 GROUND ROBOT 

TEST LANES 120cm (48in) SCALE

Disaster-Resilient Buildings, Infrastructure, 

and Communities Goal

Dissemination: Robotics Research Competitions

Slide52

Assess the effectiveness of disseminating project outputs

• Typically more than 30 teams participate from all over the world, 

roughly 300 people each year.

• The annual Championships move internationally with teams typically 

qualified from several regional competitions.

• These competitions are recruiting tools for manufacturers and test 

facilities around the world. 

Standard Test Methods for Response Robots

ASTM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09)

RoboCupRescue

Championships

2020 Bordeux, France

2019 Sydney, Australia

2018 Montreal, Canada

2017 Nagoya, Japan

2016 Leipzig, Germany

2015 Hefei, China

2014 Joao Pessoa, Brazil

2013 Eindhoven, Netherlands

2012 Mexico City, Mexico

2011 Istanbul, Turkey

2010 Singapore, Singapore

2009 Graz, Austria

2008 Suzhou, China

2007 Atlanta, USA

2006 Bremen, Germany

2005 Osaka, Japan

2004 Lisbon, Portugal

2003 Padua, Italy

2002 Fukuoka, Japan

2001 Seattle, USA

2000 AAAI Conf, Austin, TX
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Nebraska: Omaha Police Dept.

Use Case Examples
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Texas Dept. of Public Safety and Reveille Peak Ranch Test Facility

Use Case Examples

Texas Dept. of Public Safety Stats:

Pilots: 100+

Aircraft: 100+

Annual Flights:10,000+

Main focus: Crash/crime scene reconstruction

News from Texas Legislature

- Training

- Software

- Adoption of NIST tests 

Reveille Peak Ranch Test Facility and Statewide Use 

Cases by Local Organizations (next pages)
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Texas Dept. of Public Safety and Reveille Peak Ranch Test Facility

Use Case Examples
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Texas Dept. of Public Safety and Reveille Peak Ranch Test Facility

Use Case Examples

• North Texas Public Safety Unmanned Response Team 

(PSURT) PSURT Dallas/ Ft. Worth

• Camp Mabry in Austin Texas Granite Defense & 

Technologies hosted four of the 5 BLUE DIU approved 

drones for the Texas Air National Guard.

• Harris County (Houston area) Fire Marshal's Office

• Reveille Peak Ranch test facility evaluation

• Texas Department of Emergency Management (TEDM) 

pursuant to Texas HB2340
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

28

Canadian CETA and CERRA Training/Credentialing

Use Case Examples

Lead Agencies;

CETA- Canadian Explosives Technicians Association

CERRA- Canadian Emergency Responders Robotics Association

Primary Locations:

Pearson International Airport ( Toronto Canada)

Grimsby Regional Training Centre ( Grimsby, Ontario , Canada)

CETA

CETA is the national association for police/military/government agencies 

tasked with response to explosives , chemical, biological, and radiological 

incidents in Canada. Current projects include EOD Standard training methods 

for both robots and bomb techs deployed in bomb suits.

CERRA

Spring 2020 established with focus on the public safety deployment of 

ground, air, water based robotics. Membership is open to any current or 

former public safety member or agency or any supporting government agency 

with an interest in response robots.
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Drone Responders Now Auditing for ASTM International Compliance

Use Case Examples
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

30

XXXXXXXXXXXXXXXXXXX

Safety | Capabilities | Proficiency

SLIDE TEMPLATE