Reproducible Tests for Maneuvering and Payload Functionality

Safety | Capabilities | Proficiency

20

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

9

30+ Aerial Drone Tests

Safety | Capabilities | Proficiency

CONFINED Test Lane 

and Related Scenarios

• Perch

• Wall

• Ground

• Alley

• Post

OBSTRUCTED Test Lane 

and Related Scenarios

• Perch

• Wall

• Ground

• Alley

• Post

OPEN Test Lane 

and Related Scenarios

• Position 

• Traverse

• Orbit

• Inspect

• ReconALTITUDE

SPACING

dard Test Methods for Small Unmanned Aircraft Systems

M International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

a plane crash site

ns

uck accident and fire

kedown

n

search

ack

e with delivery of personal floatation devices

ollapse with school bus and vehicle rescue

es 5 different flight paths to identify objects from safe altitudes in open 

e scalable for all sizes of aircraft to demonstrate positive control at all times 

can be performed outdoors or indoors to control lighting and weather. The 

ndoor basketball or tennis court for small drones and/or novice pilots to 

 in the national airspace.

Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Test Lanes for Various Sizes of Aircraft

Project Overview

3

 (PAY 1-5)

E SYSTEMS

OCEDURES

NEUVERING

PARE

1A

SCALABLE TEST LANES (ALTITUDE = SPACING) ALTITUDE

SPACING

OMNI BUCKETS

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

• Wide area search of a plane crash site

• Vehicle identifications

• Hazardous tanker truck accident and fire

• Suspicious vehicle takedown

• Bus hostage situation

• Collapsed structure search

• Subway terrorist attack

• Flooded house rescue with delivery of personal floatation devices

• Flash flood bridge collapse with school bus and vehicle rescue

OPEN TEST LANES

The Open Test Lanes evaluates 5 different flight paths to identify objects from safe altitudes in open 

environments. These tests are scalable for all sizes of aircraft to demonstrate positive control at all times 

with accurate perches. They can be performed outdoors or indoors to control lighting and weather. The 

smallest size lane fits on an indoor basketball or tennis court for small drones and/or novice pilots to 

practice safely without flying in the national airspace.

Figure 4) The standard tests use well defined altitudes, positions, orientations, and perches to evaluate both 

Maneuvering and/or Payload Functionality.

Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

48

Scalable Test Lanes for Various Sizes of Aircraft Project Overview

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

3

Introduction

Remotely operated aerial systems enable emergency 

responders to perform extremely hazardous tasks from 

safer stand-off distances. The U.S. National Institute of 

Standards and Technology is leading an international effort 

to develop standard test methods to help manufacturers, 

procurement professionals, and users objectively evaluate 

system capabilities and remote pilot proficiency to align 

with mission requirements. Thisimproves the safety and 

effectiveness of emergency responders as they save lives 

and protect property in our communities

The first step toward credentialing remote pilots is to get 

everybody onto the same measuring stick. That’s where 

standard test methods can play a key role. These test 

methods for Basic Maneuvering (MAN 1-5) and Payload 

Functionality (PAY 1-5) are being replicated across the 

country and internationally to focus training with 

quantitative measures of remote pilot proficiency. They are 

low cost and easy to replicate so everyone can measure 

their own progress over time and compare their proficiency 

to regional or national averages on similar 

systems. Concurrent test lanes can be set up to enable 

multiple systems and pilots to train or evaluate 

simultaneously. 

They are being standardized through the ASTM 

International Standards Committee on Homeland Security 

Applications; Response Robots (ASTM E54.09). They are also 

referenced as Job Performance Requirements in 

the National Fire Protection Association Standard for Small 

Unmanned Aircraft Systems Used For Public Safety 

Operations (NFPA 2400) and the ASTM Standard Guide for 

Training for Remote Pilot in Command of Unmanned 

Aircraft Systems Endorsement (ASTM F38.03 F3266-18). 

These suites of standard test methods provide common 

measures of capabilities with quantitative results. They can 

be conducted individually, in sequences, or embedded into 

operational training scenarios as repeatable and 

reproducible tasks with scores to augment qualitative 

assessments. Organizations using these tests set their own 

thresholds of acceptable system and pilot performance to 

align with their airspace, environment, and mission 

complexities. But those decisions are easier to make and 

trust when they’re based on quantitative performance data 

captured within standard test methods.

Basic Maneuvering (MAN 1-5) and Payload Functionality (PAY 1-5)

Safety | Capabilities | Proficiency

INSIDE EACH BUCKET

EVALUATE SENSORS

SMALL SYSTEMS LARGE SYSTEMS

INTERFACES PROCEDURES

SENSORS MANEUVERING

MEASURE & COMPARE

1A

SCALABLE TEST LANES (ALTITUDE = SPACING) ALTITUDE

SPACING

OMNI BUCKETS

CEDURES

EUVERING

ARE SCALABLE (ALTITUDE = SPACING)

ard Test Methods for Small Unmanned Aircraft Systems

 International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

 plane crash site

s

ck accident and fire

edown

earch

ck

 with delivery of personal floatation devices

apse with school bus and vehicle rescue

s 5 different flight paths to identify objects from safe altitudes in open 

 scalable for all sizes of aircraft to demonstrate positive control at all times 

an be performed outdoors or indoors to control lighting and weather. The 

door basketball or tennis court for small drones and/or novice pilots to 

n the national airspace.

se well defined altitudes, positions, orientations, and perches to evaluate both 

Functionality.

Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

48

est Lanes for Various Sizes of Aircraft

Project Overview

3

(PAY 1-5)

 SYSTEMS

CEDURES

UVERING

ARE

1A

SCALABLE TEST LANES (ALTITUDE = SPACING) ALTITUDE

SPACING

OMNI BUCKETS

E DRONES

5 FLIGHT PATHS

OPEN TEST LANE AND RELATED OPERATIONAL SCENARIOS

1

2

3

FLIGHT LINE

4

z

z

ALTITUDE S FLY STRAIGHT AND LEVEL OVER THE LANE CENTERLINE

START | END

ALIGN

NON-STOP

NON-STOP NON-STOP

NON-STOP

NON-STOP

NON-STOP

z

S

S S S S

5(S)

S

ALIGN CW

CCW

RECON

MAN/PAY 5

FLIGHT LINE

4

z

START | END 3

2

1

5(S)

S

S S S S

ALTITUDE 1⁄2( S) 2

3 4

5

1 2

5 4

3

CW

CCW

1

CCW

1 2

5 4

3

2

3 4

5

CW INSPECT

1

MAN/PAY 4

1

2

3

FLIGHT LINE

4

ALTITUDE S

START | END

5(S)

S

S S S S

1A

1B

2B

3B 3C

3D

2D

1D

TRAVERSE

MAN/PAY 2

FLIGHT LINE

4

z

z

z

ALTITUDE S

START | END 3

2

1

5(S)

ALTITUDE 2(S)

S

S S S S

POSITION

MAN/PAY 1

1

2

3

FLIGHT LINE

4 ALTITUDE 2(S)

START | END

S

S S S S

5(S

)

ALTITUDE S

#2 & 3A

3B 3C

3D

#1 & 3A

3B

3C

3D

ORBIT

MAN/PAY 3

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Figure 3) The Confined Test Lane evaluates drones flying indoors or in any confined spaces as appropriate f

Confined Test Lane and Related Operational Scenarios

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Confined Scenario: Structure Interior Rooms (South)

Safety | Capabilities | Proficiency

OPEN OBSTRUCTED CONFINED

MIDDLE INDOOR ALTITUDE

1 M (3 FT)

TOP INDOOR ALTITUDE

2 M (6 FT)

PERCH

0 M (0 FT)

MIDDLE INDOOR ALTITUDE

1 M (3 FT)

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

resulting trial scores across different locations can help determine best-in-class drone capabilities and remote 

pilots no matter where or when the testing occurs.

Figure 2) Other related standards address different mission complexities such as the Obstructed Test Lane which 

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

10

Measuring sUAS Performance: Close Proximity Inspection

Safety | Capabilities | Proficiency

VEHCLE INSPECTION

OBJECT INSPECTION

STRUCTURE INSPECTION STRUCTURE INSPECTION

DUAL BUCKET ALIGNMENTS GUIDE 

PILOTS INTO SAFE POSITIONS 

WITHIN PROXIMITY TO OBJECTS

Obstructed Test Lane and Related Operational Scenarios 

POST

1

2

4

3

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Montgomery County Police Drone Training & Evaluation Facility

Safety | Capabilities | Proficiency

DOJ/DHS National Unmanned Aircraft Systems Program Evaluation, August 2020

BUS EXTERIOR

AND INTERIOR

OBSTRUCTED TEST LANE

(2) OPEN TEST LANES

AND

SENSOR TEST LANE

STRUCTURE EXTERIOR

(WINDOWS)

STRUCTURE PERIMETER 

(GROUND)

INTERIOR ROOMS

(NORTH)

INTERIOR ROOMS

(SOUTH)

INTERIOR ROOMS

(NORTH)

INTERIOR ROOMS

(SOUTH)

STRUCTURE EXTERIOR

(WINDOWS)

STRUCTURE PERIMETER 

(GROUND)

BRIEFING

PAVILLION

RADIO COMMS RANGE

ATTENUATED IN APC

RAIN TOLERANCE

CONFINED TEST LANE

(IN CLASSROOM - NOT SHOWN)

NORTH SITE

SCALABLE 

ALTITUDES

CLOSE

PROXIMITY

GPS DENIED

AND DARK

LEVEL 1, 2, 3 LEVEL 4 LEVEL 5

Monday Jan. 23, 2023 4 pm – 8 pm

• Day and night familiarization flight demonstrations.

• Open to regional responders and the public.

• Everybody can watch and learn about the tests.

• FAA Part 107 license required to fly your own drone.

• Food served at sunset.

Hosts: 

Adam Jacoff, NIST and ASTM E54.09 Standards Committee.

Kai-Dee Chu, Science and Technology Directorate, DHS

Sgt. Daniel “Justin” Anders, SAPD sUAS Program Manager

Location:

San Antonio Police Department Training Facility

12200 SE I-410 Access Rd, San Antonio, TX 78221

RSVP to RobotTestMethods@nist.gov so we can facilitate 

parking and food service.

Test Methods for Evaluating Aerial Drones

Safety | Capabilities | Proficiency

RobotTestMethods.nist.gov

Fly Drones in Tests and Scorable Scenarios for 

Open, Obstructed, and Confined Environments

Test Methods for Evaluating Aerial Drone

Safety | Capabilities | Proficienc

RobotTestMethods.nist.gov

Standard Test Methods fo

ASTM International Standards C

Response Robots (E54.09) 

 

Figure 32) A floating omni bucket task was pulled t

to represent floating or swimming victims. It provid

 

Figure 33) The flashflood bridge collapse included 

comprehensive search for survivors.

Obstructed Scenario: Night Operations Hou

This scenario was actually conducted the week prio

scenarios above. This house surveillance at night u

apparatuses on all four sides of the house, guiding 

of very difficult obstacles such as the overhanging 

vertical test apparatuses with 10 positions and orie

100 points. A set of and 5 horizontal test apparatu

on or near objects of interest around the house fo

St

Structure exterior inspect

ground objects of interest

structure. The objective w

to perform a window/doo

Figure 26) Obstructed app

scenario.

Figure 27) The scenario fa

collapsed walls. Each Dua

high/low and left/right vie

Page 16 

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

 

Figure 19) LEFT: Other interior tasks using the small buckets in confined search spaces. CENT

apparatuses require vertical alignments. RIGHT: Three apparatuses require horizontal alignm

final perch position to maintain a view of an object of interest.

Obstructed Scenario: Vehicle Takedown 

Quad Screen Trial Video Example - Vehicle Takedown

Figure 20) LEFT: Vehicles of all sizes and conditions can be bucketized similarly for quantitativ

including panel vans and trucks. RIGHT: Dual bucket alignments guide the drone into position 

the front windshield from two different angles while using triangulation between the buckets 

 Figure 21) The bucket numbering leads the pilot around an orbit starting in front of the vehic

to the driver side, and so on. The large buckets targets are identified in the first orbit to asse

WHITE BUCKET ALIGNM

BLACK BUCKET ALIGNM

HORIZONTAL 

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

12

Obstructed Test Lane and Related Scenarios

Aerial Tests

- 5 -

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

resulting trial scores across different locations can help determine best-in-class drone capabilities and re

pilots no matter where or when the testing occurs.

Figure 2) Other related standards address different mission complexities such as the Obstructed Test Lane 

evaluates drones flying in close proximity to outdoor objects to inspect detailed features from 2-3 m (6-10 

These standard tests support drone researchers, manufacturers, and user organizations in different 

Researchers use these standard tests to understand mission requirements, encourage innovation, and demon

break-through capabilities. Manufacturers use these standard tests to evaluate design decisions, inte

emerging technologies, and harden systems. Various organizations use these tests to guide purchasing, 

deployment decisions, and focus training with measures of remote pilot proficiency for credentialing. 

standard tests support emergency response operations, critical infrastructure inspection, industria

commercial applications, and even recreational pilot credentialing (see Appendix A for a subset of tests th

support recreational pilot credentialing).

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

10

Measuring sUAS Performance: Close Proximity Inspection

Safety | Capabilities | Proficiency

VEHCLE INSPECTION

OBJECT INSPECTION

STRUCTURE INSPECTION STRUCTURE INSPECTION

DUAL BUCKET ALIGNMENTS GUIDE 

PILOTS INTO SAFE POSITIONS 

WITHIN PROXIMITY TO OBJECTS

Obstructed Test Lane and Related Operational Scenarios 

- 5 -

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

resulting trial scores across different locations can help determine best-in-class drone capabilities and remote 

pilots no matter where or when the testing occurs.

Figure 2) Other related standards address different mission complexities such as the Obstructed Test Lane which 

evaluates drones flying in close proximity to outdoor objects to inspect detailed features from 2-3 m (6-10 ft).

These standard tests support drone researchers, manufacturers, and user organizations in different ways. 

Researchers use these standard tests to understand mission requirements, encourage innovation, and demonstrate 

break-through capabilities. Manufacturers use these standard tests to evaluate design decisions, integrate 

emerging technologies, and harden systems. Various organizations use these tests to guide purchasing, make 

deployment decisions, and focus training with measures of remote pilot proficiency for credentialing. These 

standard tests support emergency response operations, critical infrastructure inspection, industrial and 

commercial applications, and even recreational pilot credentialing (see Appendix A for a subset of tests that can 

support recreational pilot credentialing).

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

10

Measuring sUAS Performance: Close Proximity Inspection

Safety | Capabilities | Proficiency

VEHCLE INSPECTION

OBJECT INSPECTION

STRUCTURE INSPECTION STRUCTURE INSPECTION

DUAL BUCKET ALIGNMENTS GUIDE 

PILOTS INTO SAFE POSITIONS 

WITHIN PROXIMITY TO OBJECTS

Obstructed Test Lane and Related Operational Scenarios 

POST

1

2

4

3

POST TEST WALL TEST

Obstructed Test Lanes and Scorable Inspe

Structure exterio

ground objects o

structure. The o

to perform a win

Figure 26) Obstru

scenario.

PERCH

Dual Bucket Alignments Guide Remote Pilots to Safe Positions 

VEHICLE INSPECTION ST

0 M (0 FT)

1 M (3 FT)

PERCH TEST

2 M (6 FT)

1 M (3 FT)

WALL TEST G

Confined Test Lane and Rela

Standard Test Methods for Small 

ASTM International Standards Committee o

Response Robots (E54.09) | Website

Confined Scenario: Structur

Safety | Capabilitie

OPEN OBSTRUC

SHOOT HOUSE SCENARIO FABRICATED ROOM TO ROOM S

Confined Test Lanes and Scorable Indo

LEVEL 4 | OBSTRUCTED LANES AND SCOR

LEVEL 5 | CONFINED LANES AND SCORA

INDOOR or

GPS DENIED
Users of aerial drones weighing less than 25 kg (55

lbs) at takeoff, also known as small unmanned

aircraft systems (sUAS) or remotely piloted aircraft

systems (RPAS), need ways to measure whether a

particular drone can perform specific missions in

unstructured, complex, and often hazardous

environments. These missions require various

combinations of elemental capabilities. Each

capability can be represented as a test method with

an associated apparatus and procedure enabling

repeatable and reproducible measures of

performance with objective results. These test

methods can be conducted individually or in

operationally relevant sequences and combinations

to evaluate drone capabilities and remote pilot

proficiency. The results measure the reliability of

the drone and remote pilot to perform the essential

mission tasks. A series of complementary test lanes

enable users to evaluate a wide spectrum of

intended missions. This event will include

familiarization flights with test methods and

scorable scenarios developed for Open, Obstructed,

and Confined environments. Bring a drone if you

want to fly, or just come watch. The event is open

to regional emergency responders and the public

Levels of Evaluation 

For Different MissionsT

OPEN Test Lane

Basic Proficiency/Safety Checkride

Evaluate positive aircraft control in quick 

trials for novices and recreational pilots 

using the first 2 of 5 flight paths.

• 10 minutes (5 minutes each)

• 40 Alignment Points

• 

• 

• 

• 

OPEN Test Lane

Maneuvering Trials

Evaluate open area maneuvering around 

ground objects using all 5 flight paths with 

no additional pilot workload.

• 25 minutes (5 minutes each)

• 100 Alignment Points

• 

• 

• 

• 

OPEN Test Lane

Payload Functionality Trials

Evaluate open area maneuvering while 

controlling zoom and exposure to identify 

ground objects using all 5 flight paths.

• 25 minutes (5 minutes each)

• 50 Alignment Points | 50 Acuity Points

• 

• 

• 

• 

OBSTRUCTED Test Lane

Payload Functionality Trials

Evaluate maneuvering in close proximity to 

objects while controlling zoom and 

exposure to inspect from 6-10 ft standoff.

• 25 minutes (5 minutes each)

• 50 Alignment Points | 50 Acuity Points

• 

• 

• 

CONFINED Test Lane

Payload Functionality Trials

Evaluate maneuvering while controlling 

zoom and exposure to search room to 

room and inspect from 3-4 ft standoff.

• 25 minutes (5 minutes each)

• 50 Alignment Points | 50 Acuity Points

• 

• 

• 

2

LEVEL

3

LEVEL

4

LEVEL

5

LEVEL

1

LEVEL

M

San Antonio Police Dept.

Training Facility

12200 SE I-410 Access Rd