Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Introduction

Aerial Test Methods

Sub Committee Chair:

Adam Jacoff

Intelligent Systems Division

National Institute of Standards and Technology

U.S. Department of Commerce

Committee Chair:

Phil Mattson

Science and Technology Directorate 

U.S. Department of Homeland Security

Internet

RobotTestMethods.nist.gov

Adam	Jacoff

Intelligent	Systems	Division

NATIONAL	INSTITUTE	OF	STANDARDS	AND	TECHNOLOGY

U.S.	Department	of	Commerce

Email

RobotTestMethods@nist.gov

1

Version 2021A

WEBSITE POINTER: 

WATCH FABRICATION VIDEOS 

AND FLIGHT PATH ANIMATIONS

WEBSITE POINTER: 

DOWNLOAD STICKER FILES, FORMS AND 

PRACTICE SCORING VIDEOS Online Only Meeting

February 3, 2021

10:00am – 2:00pm EST
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

2

• Reminder that electronic recording of ASTM meetings is prohibited.

• This meeting will run in accordance with the ASTM Antitrust Statement (see minutes).

Antitrust Statement

ASTM International is a not-for-profit organization and developer of voluntary consensus standards. ASTM's leadership in international 

standards development is driven by the contributions of its members: more than 30,000 technical experts and business professionals 

representing 135 countries.

The purpose of antitrust laws is to preserve economic competition in the marketplace by prohibiting, among other things, unreasonable 

restraints of trade. In ASTM activities, it is important to recognize that participants often represent competitive interests. Antitrust laws 

require that all competition be open and unrestricted.

It is ASTM's policy, and the policy of each of its committees and subcommittees, to conduct all business and activity in full compliance with 

international, federal and state antitrust and competition laws. The ASTM Board of Directors has adopted an antitrust policy which is found 

in Section 19 of ASTM Regulations Governing Technical Committees. All members need to be aware of and compliant with this policy. The 

Regulations are accessible on the ASTM website (http://www.astm.org/COMMIT/Regs.pdf) and copies of the antitrust policy are available 

at the registration desk. For a complete list of standards, see: http://www.astm.org/COMMIT/SUBCOMMIT/E5409.htm

Call To Order

Committee Meeting
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

3

Agenda

Committee Meeting

.

10:00 am EST Introduction to Open Test Lane

10:30 am EST Use Case Examples

12:00 pm EST Advanced Test Lanes and Embedded Scenarios

- Obstructed Test Lane

- Confined Test Lane

- Sensors and Radio Comms

- Others

2:00 pm EST Open Discussion 

POST

T-TOP

LEGS

POST

4x4x24in (10x10x60cm)

LEGS

2x4x20in (5x10x50cm)

with 45° opposing ends

T-TOP

2x4x12in (5x10x30cm)

BUCKETS

2 or 5 gal (7 or 19 liter)

Launch/Land

12in (30cm) Radius Center

Bucket Stand #1

(A,B,C,D)

10 ft (3 m) Spacings

FLIGHT LINE

SIGHT LINE

Launch/Land

12in (30cm) Radius Center

FLIGHT LINE

SIGHT LINE

Bucket Stand #1

(A,B,C,D)

Bucket Stand #2 

(E,F,G,H)

Bucket Stand #3

(I,J,K,L)

10ft (3m) Spacings

10 FT (3 M)

ALTITUDE LAUNCH LAND

Test Director 

Adam Jacoff

Intelligent Systems Division

Nat’l Institute of Standards and Technology 

U.S. Dept. of Commerce 

RobotTestMethods@nist.gov | 301-975-4235 

Sponsor 

Philip Mattson

Science and Technology Directorate 

U.S. Dept. of Homeland Security

These test methods are primarily intended for 

vertical takeoff and landing systems with an 

onboard camera and remote pilot display. Some 

are also applicable to fixed wing systems when the 

lane dimensions are extended to accommodate 

the orbit radius of forward flying systems. 

Bucket stands on a level surfaces ensure the top bucket is 

vertical and the angled buckets are 45 degrees. 

TRAVERSE TEST

Right and Left Ovals Around All Three Bucket Stands

ALIGNED

WHEN RING IS 

COMPLETELY 

VISIBLE

GROUND

AQUATIC

AERIAL
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

4

Collaborators

Tom Haus, Los Angeles Fire Dept. & CA-TF1, CA

Parry Boogard, Valley Regional Fire Authority & WA-TF1, WA

Clint Arnett, TEEX/Disaster City & TX-TF1, TX

George Hough, Fire Dept. of New York City & NY-TF1, NY

Jim Ingledue, Virginia Beach Fire Dept. & VA-TF2, VA

Mark Hundley, Virginia Beach Fire Dept. & VA-TF2, WA

Michael O’Shea, FAA UAS Integration Office (formerly U.S. DOJ)

Martin Hutchings, Sacramento Sheriff & IAB, CA

John Delaney, Arlington County Fire, Dept., & IAB, VA

Mike Marino, Prince George’s County Fire Dept. & IAB, MD

Coitt Kessler, Austin Fire Dept., TX

Chris Sadler, York County Fire Dept., VA

Andy Moore, Southwest Research Institute, San Antonio, TX

Al Frazier, Grand Forks County Sheriff’s Dept., ND

Ben Miller, CDPS COE for Aerial Technology Fire Fighting, CO

Mark Blanks, Virginia Tech University, VA

Daniele Nardi, Sapienza Universita di Roma, Italy

Max Delo, ESF-13, U.S. Marshals Service, DOJ

Bryan Gillespy, ESF-13, U.S. Marshals Service, DOJ

Gabriele Ferri, NATO CMRE, Italy

Howie Stockhowe, Virginia Beach Fire Dept, Virginia Beach, VA

Tony Galladora, Montgomery County Police, MD

Acknowledgements

Acknowledgements

This work was sponsored by the Department of Homeland Security, 

Science and Technology Directorate, through an interagency agreement 

with the National Institute of Standards and Technology (NIST). 

The NIST Team includes: 

Adam Jacoff, Raymond Sheh, Kamel Saidi, 

Kenny Kimble, and Ann Virts.

Dozens more people have contributed to the development and 

validation of these test methods. They include FEMA urban search and 

rescue task force teams, firefighters, law enforcement, collaborating 

test facilities, other civilian and military organizations, and commercial 

manufacturers. There are far too many to mention, but some of the 

ongoing (non-commercial) collaborators are listed below, roughly in 

order of their involvement:

Disclaimer

Commercial equipment shown in this document are for illustrative 

purposes only. This does not imply recommendation or endorsement by 

the National Institute of Standards and Technology, nor does it imply 

that the products identified are necessarily the best available for the 

purpose. 

Measurement Units

The International System of Units (a.k.a. SI Units) and U.S. Customary 

Units (a.k.a. Imperial Units) are used throughout this document. 

Approximate equivalents in each system of units enable use of readily 

available materials in different countries. This avoids excessive 

purchasing and fabrication costs. The differences between the stated 

unit dimensions are insignificant for comparison of test method results, 

so each set of units are considered standard for the purposes of these 

test methods. 

Satoshi Tadokoro, Tohoku University, Sendai, Japan

Tetsuya Kimura, Nagoaka Univ. of Technology, Nagoaka, Japan

Bob Gann, CDPS COE for Aerial Technology Fire Fighting, CO

Andy Olesen, Canadian Explosives Technicians Assoc., Canada

Tom Prentice, Reveille Peak Ranch, Burnet, TX

Michael Leo, Fire Department of New York City, NY

Luke Bergan, New South Whales Police Dept., Sydney, Australia

Katie Thielmeyer, Woodlawn Fire Dept. OH

Oliver Huke, RACE Test Facility, UKAEA, Oxfordshire, United Kingdom
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

5

Example Spectrum of sUAS (”Drones”)

Safety | Capabilities | Proficiency

Initial focus is VTOL, but some tests apply to forward flying aircraft when scaled up to the appropriate orbit radius.

Small Unmanned Aircraft Systems (sUAS)

Weigh Less Than 25 kg (55 lbs)
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Repeatable Maneuvering and Payload Functionality Tests 

Safety | Capabilities | Proficiency

vely evaluate 

ncy to align 

e safety and 

ey save lives 

ots is to get 

at’s where 

ese test 

nd Payload 

cross the 

with 

ency. They are 

n measure 

eir proficiency 

to enable 

te 

M 

land Security 

). They are also 

s in 

dard for Small 

Safety 

ard Guide for 

manned 

F3266-18). 

e common 

ults. They can 

mbedded into 

and 

alitative 

set their own 

rformance to 

ission 

o make and 

formance data 

INSIDE EACH BUCKET

1A

EVALUATE SENSORS

SMALL SYSTEMS LARGE SYSTEMS

INTERFACES PROCEDURES

SENSORS MANEUVERING

MEASURE & COMPARE SCALABLE TEST LANES (ALTITUDE = SPACING) ALTITUDE

SPACING

OMNI BUCKETS

9

Reproducible Tests for Maneuvering and Payload Functionality

Safety | Capabilities | Proficiency

6
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

11

Scalable for Indoor/Outdoor Venues

Open Test Lane

MEASURING TAPE CENTERLINE 

UP TO 90 M (300 FT)

1

2

3

FLIGHT LINE

4

5(S)

S

S

S

z S

ALTITUDE S

ALTITUDE 2(S)

S

ALTITUDE 5(S)

LAUNCH/LAND 

PLATFORM

OMNI BUCKET STANDS

(A-B-C-D)

SPACING = S ALTITUDES = S, 2(S), 5(S) LENGTH = 10(S) 

S = 3 m (10 ft) 3, 6, 15 m (10, 20, 50 ft) 30 m (100 ft) 

S = 6 m (20 ft) 6, 12, 30 m (20, 40, 100 ft) 60 m (200 ft) 

S = 9 m (30 ft) 9, 18, 45 m (30, 60, 150 ft) 90 m (300 ft)

Spacing (S) = Altitude

DATE/TIME

PILOT CODE

48IN SQUARE LANDING WITH

12IN CENTER RADIUS TARGET

100-300 FT MEASURING TAPE

(4) OMNI STANDS

24 INCH 0R 48 INCH 

LAUNCH / LAND

PLATFORM

ACUITY TARGET

FOR ALIGNMENT

DIGITAL CLOCK CAPTURES

DATE | TIME (SYNCED)

WHITEBOARD CAPTURES

MAKE | MODEL | CONFIG

(4) OMNI BUCKET STANDS

12 INCH RADIUS

CIRCLE FOR ACCURACY

CENTERLINE 

MEASURING TAPE

DATE/TIME

PILOT CODE

48IN SQUARE LANDING WITH

12IN CENTER RADIUS TARGET

100-300 FT MEASURING TAPE

(4) OMNI STANDS

1–4

TOP

FRONT 1A

LEFT 1B 1C BACK

1D RIGHT “A” BUCKET

ALWAYS POINTING 

TOWARD THE 

LAUNCH/LAND

PLATFORM

HANDLE

CENTERLINE

LAUNCH / LAND PLATFORM

DIGITAL CLOCK

DATE & TIME

WHITEBOARD INFO

PILOT & AIRCRAFT

(4) OMNI 

BUCKET STANDS

CENTERLINE 

MEASURING TAPE

MEASURING TAPE CENTERLINE 

UP TO 90 M (300 FT)

1

2

3

FLIGHT LINE

4

5(S)

S

S

S

z S

ALTITUDE S

ALTITUDE 2(S)

S

ALTITUDE 5(S)

LAUNCH/LAND 

PLATFORM

OMNI BUCKET STANDS(A-B-C-D)

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

11

Scalable for Indoor/Outdoor Venues

Open Test Lane

MEASURING TAPE CENTERLINE 

UP TO 90 M (300 FT)

1

2

3

FLIGHT LINE

4

5(S)

S

S

S

z S

ALTITUDE S

ALTITUDE 2(S)

S

ALTITUDE 5(S)

LAUNCH/LAND 

PLATFORM

OMNI BUCKET STANDS

(A-B-C-D)

SPACING = S ALTITUDES = S, 2(S), 5(S) LENGTH = 10(S) 

S = 3 m (10 ft) 3, 6, 15 m (10, 20, 50 ft) 30 m (100 ft) 

S = 6 m (20 ft) 6, 12, 30 m (20, 40, 100 ft) 60 m (200 ft) 

S = 9 m (30 ft) 9, 18, 45 m (30, 60, 150 ft) 90 m (300 ft)

Spacing (S) = Altitude

DATE/TIME

PILOT CODE

48IN SQUARE LANDING WITH

12IN CENTER RADIUS TARGET

100-300 FT MEASURING TAPE

(4) OMNI STANDS

24 INCH 0R 48 INCH 

LAUNCH / LAND

PLATFORM

ACUITY TARGET

FOR ALIGNMENT

DIGITAL CLOCK CAPTURES

DATE | TIME (SYNCED)

WHITEBOARD CAPTURES

MAKE | MODEL | CONFIG

(4) OMNI BUCKET STANDS

12 INCH RADIUS

CIRCLE FOR ACCURACY

CENTERLINE 

MEASURING TAPE

DATE/TIME

OT CODE

48IN SQUARE LANDING WITH

12IN CENTER RADIUS TARGET

100-300 FT MEASURING TAPE

(4) OMNI STANDS

K

D RIGHT 

LATFORM

DIGITAL CLOCK

DATE & TIME

ARD INFO

AIRCRAFT

(4) OMNI 

BUCKET STANDS

CENTERLINE 

MEASURING TAPE Spacing (S) = Altitude

7
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

10

POST

LEGS 

FLUSH WITH TOP

Easy Fabrication and Stowing

Open Test Lane

2 Gallon Buckets – Printed Stickers – Transportable 

PRINT TARGET STICKERS PERCH ACUITY TARGETS TRANSPORT IN GOLF BAGS

1A

2A

3A

AA

STOW LANES IN STACKS

EASY AND INEXPENSIVE TO FABRICATE

DATE/TIME

PILOT CODE

48IN SQUARE LANDING WITH

12IN CENTER RADIUS TARGET

100-300 FT MEASURING TAPE

(4) OMNI STANDS

1–4

TOP

FRONT 1A

LEFT 1B 1C BACK

1D RIGHT “A” BUCKET

ALWAYS POINTING 

TOWARD THE 

LAUNCH/LAND

PLATFORM

HANDLE

CENTERLINE

LAUNCH / LAND PLATFORM

DIGITAL CLOCK

DATE & TIME

WHITEBOARD INFO

PILOT & AIRCRAFT

(4) OMNI 

BUCKET STANDS

CENTERLINE 

MEASURING TAPE

8
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

BLACK BUCKETS & COLOR RINGS 

EMBEDDED INTO SCENARIOS

1

TOP

FRONT 1A

LEFT 1B 1C BACK

1D RIGHT

ALL “A” BUCKETS

POINT TOWARD THE 

LAUNCH/LAND

HANDLE

CENTERLINE

ONLY THE NUMBERS 

CHANGE FOR EACH 

OMNI BUCKET STAND

Omni Bucket Stands

Open Test Lane

1A

BROKEN INNER BLACK EDGE

IS THE JUDGEMENT POINT

13

POST

LEGS 

FLUSH WITH TOP

SHOWN NOT ALIGNED SHOWN ALIGNED

WHITE BUCKETS & GREEN RINGS 

IN STANDARD TEST LANES

9
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

10

Conduct Tests Two Ways

Open Test Lane

[4] 2x4x12in legs with 45deg taper both ends

[5] 2-gallon white buckets (8in diameter)

10] 3in screws to affix the legs – 2 per leg at top

[5] 1in screws to affix the buckets

[4] Slotted leg extensions

[8] 1⁄4-20 x 2in hanger bolts and wing nuts for the

leg extensions to slide on and tighten

[15] 8in diameter weatherproof stickers

(see website to download the sticker files)

[4] Big numbers 1-1-1-1 inside the top bucket

[4] Big letters A-B-C-D around the top bucket

[5] Acuity targets A-B-C-D inside bottom of all

[2] Perch acuity targets inside and bottom of A

.

Stickers inside each bucket have a GREEN INSCRIBED RING 

to guide alignment and a visual acuity target with 

TOP

( T )

( B )

BOTTOM 

( TR ) TOP RIGHT

( R ) RIGHT

( BR ) BOTTOM RIGHT

TOP LEFT ( TL )

LEFT ( L )

BOTTOM LEFT ( BL )

A

Gap size (mm) Gap size (inches) Equivalent Avg. Human vision at 6 m (20 !)

40 mm 1.58 in 4.5 %

16 mm 0.63 in 11 %

6.4 mm 0.25 in 28 %

2.6 mm 0.10 in 70 %

1.0 mm 0.04 in 180 %

0.41 mm 0.016 in 440 %

For accurate calibration, the outer diameter of the largest Landolt C optotype should be 200 mm (7.9 in). 

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

Nested Landolt C Vision Test Chart

#3

DATE/TIME

PILOT CODE

48IN SQUARE LANDING WITH

12IN CENTER RADIUS TARGET

100-300 FT MEASURING TAPE

A

D

C

B 1, 2, 3, 4

LAUNCH

AD

PERCH 

ACUITY

P1

P1

LEAVE CARRY HANDLE ON TOP BUCKET

1A

1A 1A ALIGNED (5 PTS)

PARTIAL (1 PT)

Align with each bucket long enough to capture a 

single alignment image (NO ZOOM) showing the 

green ring inside the bucket. Score 5 points for a 

continuous green ring or 1 point for a partial. Similar 

scoring for accurate or partial landings.

Align with each bucket long enough to capture a 

single alignment image (NO ZOOM) AND a single 

acuity image of each target (MAX ZOOM). Score 1 

point for each correct identification of the 5 

increasingly small Concentric C gap directions.

Basic Maneuvering (MAN)

ALIGN WITH BUCKETS AND LAND ACCURATELY

Payload Functionality (PAY) 

ALIGN AND IDENTIFY ACUITY TARGETS

20 ALIGNMENTS TOTAL UP TO 100 POINTS 20 TARGETS TOTAL UP TO 100 POINTS

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Montgomery County Police Drone Training & Evaluation F

Safety | Capabilities | Proficiency

DOJ/DHS National Unmanned Aircraft Systems Program Evaluation, Au

I

COOPEN TEST LANE

B

LENGTH: 30/60 M (100/200 FT)

E

OBSTRUCTED TEST LANE

DUAL BUCKET 

ALIGNMENTS

AT 2 M (6 FT) 

PROXIMITY TO 

OBSTACLES

LENGTH: 15 M (50 FT) L

ACCURATE (5 PTS)
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

11

Positive Aircraft Control (Part 107 Skills Test?)

Open Test Lane

Position

MAN/PAY 1

• Hover stably

• Basic maneuvers

• Land accurately

• 20 Buckets in 1 lap

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

14

Position 

(MAN 1 | PAY 1)

FLIGHT LINE

4

z

z

z

ALTITUDE S

START | END

3

2

1

5(S)

ALTITUDE 2(S)

S

S S S S

y

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

18

1

2

3

FLIGHT LINE

4

ALTITUDE S

START | END

5(S)

S

S S S S

Traverse 

(MAN 2 | PAY 2)

Traverse

MAN/PAY 2

• Fly sideways along a line

• Left and right directions

• Land accurately

• 20 Buckets in 2 laps
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

12

5 Different Tests – Simulated and Physical

Open Test Lane

1

2

3

FLIGHT LINE

4

z

z

ALTITUDE S FLY STRAIGHT AND LEVEL OVER THE LANE CENTERLINE

START | END

ALIGN

NON-STOP

NON-STOP NON-STOP

NON-STOP

NON-STOP

NON-STOP

z

S

S S S S

5(S)

S

ALIGN CW

CCW

Recon

MAN/PAY 5

Spiral 

MAN/PAY 4

Orbit 

MAN/PAY 3

Traverse

MAN/PAY 2

Position

MAN/PAY 1

1

2

3

4

z

z

FREE FLIGHT

z z

z

CW

CCW

CW

CCW

LAND

S

S S S S

FLIGHT LINE

1

2

3

FLIGHT LINE

4

START | END

S

S S S S

5(S)

ALTITUDE S

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

18

1

2

3

FLIGHT LINE

4

ALTITUDE S

START | END

5(S)

S

S S S S

Traverse 

(MAN 2 | PAY 2)

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

14

Position 

(MAN 1 | PAY 1)

FLIGHT LINE

4

z

z

z

ALTITUDE S

START | END 3

2

1

5(S)

ALTITUDE 2(S)

S

S S S S
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

13
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

14

Position Test (MAN/PAY 1)

Open Test Lane

Position 

(MAN 1 | PAY 1)

FLIGHT LINE

4

z

z

z

ALTITUDE S

START | END

3

2

1

5(S)

ALTITUDE 2(S)

S

S

S

S

S

• Hover stably in designated positions and orientations

• Align with two buckets simultaneously at each position

• Basic maneuvers between positions

• Altitude S and 2(S)

• Landing accuracy counts

• 20 alignments and targets in 1 lap

PAY 1-5 PAYLOAD FUNCTIONALITY 

IDENTIFY VISUAL ACUITY TARGETS 

Align with each bucket long enough to 

capture a single alignment image (NO 

ZOOM) AND a single acuity image of each 

target (MAX ZOOM). Score 1 point for each 

correct identification of the 5 increasingly 

small Concentric C gap directions.

MAN 1-5 BASIC MANEUVERING

ALIGN WITH BUCKETS 

Align with each bucket long enough to 

capture a single alignment image (NO 

ZOOM) showing the green ring inside the 

bucket. Score 5 points for a continuous 

green ring or 1 point for a partial. Similar 

scoring for accurate or partial landings

1A 1A NOT ALIGNED

ALIGNED

1A
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

15

1

2

3

FLIGHT LINE

4

ALTITUDE S

START | END

5(S)

S

S

S

S

S

Traverse 

(MAN 2 | PAY 2)

Traverse Test (MAN/PAY 2)

Open Test Lane

• Fly sideways along a line or object

• Leftward and rightward directions to align with angled buckets

• Altitude S throughout

• Landing accuracy counts

• 20 alignments and targets 2 laps

PAY 1-5 PAYLOAD FUNCTIONALITY 

IDENTIFY VISUAL ACUITY TARGETS 

Align with each bucket long enough to 

capture a single alignment image (NO 

ZOOM) AND a single acuity image of each 

target (MAX ZOOM). Score 1 point for each 

correct identification of the 5 increasingly 

small Concentric C gap directions.

MAN 1-5 BASIC MANEUVERING

ALIGN WITH BUCKETS 

Align with each bucket long enough to 

capture a single alignment image (NO 

ZOOM) showing the green ring inside the 

bucket. Score 5 points for a continuous 

green ring or 1 point for a partial. Similar 

scoring for accurate or partial landings

1A 1A NOT ALIGNED

ALIGNED

1A
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

16

1

2

3

FLIGHT LINE

4

START | END

S

S

S

S

S

5(S)

ALTITUDE S

Orbit Test (MAN/PAY 3)

MAN/PAY 3

• Orbit around a designated point

• Leftward and rightward directions to align with angled buckets

• Downward radius checks at start of each orbit

• Altitude S throughout

• 20 alignments and targets in 4 laps

PAY 1-5 PAYLOAD FUNCTIONALITY 

IDENTIFY VISUAL ACUITY TARGETS 

Align with each bucket long enough to 

capture a single alignment image (NO 

ZOOM) AND a single acuity image of each 

target (MAX ZOOM). Score 1 point for each 

correct identification of the 5 increasingly 

small Concentric C gap directions.

MAN 1-5 BASIC MANEUVERING

ALIGN WITH BUCKETS 

Align with each bucket long enough to 

capture a single alignment image (NO 

ZOOM) showing the green ring inside the 

bucket. Score 5 points for a continuous 

green ring or 1 point for a partial. Similar 

scoring for accurate or partial landings

1A 1A NOT ALIGNED

ALIGNED

1A
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

17

1

2

3

4

z

z

FREE FLIGHT

z

z

z

CW

CCW

CW

CCW

LAND

S

S

S

S

S

Spiral Test (MAN/PAY 4)

Open Test Lane

• Fly freely to align all around downward objects

• Rotate leftward and rightward directions alternately

• Any proximity is allowed 

• Any altitude is allowed

• 20 alignments and targets in 1 lap

PAY 1-5 PAYLOAD FUNCTIONALITY 

IDENTIFY VISUAL ACUITY TARGETS 

Align with each bucket long enough to 

capture a single alignment image (NO 

ZOOM) AND a single acuity image of each 

target (MAX ZOOM). Score 1 point for each 

correct identification of the 5 increasingly 

small Concentric C gap directions.

MAN 1-5 BASIC MANEUVERING

ALIGN WITH BUCKETS 

Align with each bucket long enough to 

capture a single alignment image (NO 

ZOOM) showing the green ring inside the 

bucket. Score 5 points for a continuous 

green ring or 1 point for a partial. Similar 

scoring for accurate or partial landings

1A 1A NOT ALIGNED

ALIGNED

1A
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

18

1

2

3

FLIGHT LINE

4

z

z

ALTITUDE X

FLY STRAIGHT AND LEVEL OVER THE LANE CENTERLINE

START | END

ALIGN

NON-STOP

NON-STOP NON-STOP

NON-STOP

NON-STOP

NON-STOP

z

S

S

S

S

S

5(S)

S

ALIGN

CW

CCW

Recon Test (MAN/PAY 5)

Open Test Lane

• Sustain speed over a line to establish stable hovers over objects

• Downward and angled alignments at each end

• S altitude throughout

• 8(S) distance each way for 10 lengths

• 80(S) total distance for trial 

• 20 alignments and targets in 5 laps

PAY 1-5 PAYLOAD FUNCTIONALITY 

IDENTIFY VISUAL ACUITY TARGETS 

Align with each bucket long enough to 

capture a single alignment image (NO 

ZOOM) AND a single acuity image of each 

target (MAX ZOOM). Score 1 point for each 

correct identification of the 5 increasingly 

small Concentric C gap directions.

MAN 1-5 BASIC MANEUVERING

ALIGN WITH BUCKETS 

Align with each bucket long enough to 

capture a single alignment image (NO 

ZOOM) showing the green ring inside the 

bucket. Score 5 points for a continuous 

green ring or 1 point for a partial. Similar 

scoring for accurate or partial landings

1A 1A NOT ALIGNED

ALIGNED

1A
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

1

2

3

FLIGHT LINE

4

z

z

ALTITUDE S FLY STRAIGHT AND LEVEL OVER THE LANE CENTERLINE

START | END

ALIGN

NON-STOP

NON-STOP NON-STOP

NON-STOP

NON-STOP

NON-STOP

z

S

S S S S

5(S)

S

ALIGN CW

CCW

Recon

MAN/PAY 5

• Sustain speed over a line

• Establish hovers over objects

• S altitude, 160(S) distance

• 20 alignments in 5 laps

MAN: Align only

5 minutes / 100 points

PAY: Align and Identify

10 minutes / 100 points

Spiral 

MAN/PAY 4

• Fly freely to inspect objects

• Any proximity (use zooms)

• Any altitude

• 20 alignments in 1 lap

MAN: Align only

5 minutes / 100 points

PAY: Align and Identify

10 minutes / 100 points

Orbit 

MAN/PAY 3

• Orbit to identify objects

• Left and right directions

• S altitude

• 20 alignments in 4 laps

MAN: Align only

5 minutes /100 points

PAY: Align and Identify

10 minutes / 100 points

Traverse

MAN/PAY 2

• Fly sideways along a line

• Left and right directions

• Landing accuracy

• 20 alignments in 2 laps

MAN: Align only

5 minutes / 100 points max

PAY: Align and Identify

10 minutes / 100 points

Position

MAN/PAY 1

• Hover stably

• Basic maneuvers

• Landing accuracy

• 20 alignments in 1 lap

MAN: Align only

5 minutes / 100 points

PAY: Align and Identify

10 minutes / 100 points

500 PTS

500 PTS

Circuit Training with Scores

Open Test Lane

*If your training aircraft has only a fixed camera, or limited range of motion, align with as many buckets as possible. Proficiency is compared using similar aircraft. 

IDENTIFY ACUITY TARGETS 

Align with each bucket long enough to capture a single 

alignment image (NO ZOOM) AND a single acuity 

image of each target (MAX ZOOM). Score 1 point for 

each correct identification of the 5 increasingly small 

Concentric C gap directions.

BASIC MANEUVERING PAYLOAD FUNCTIONALITY 

ALIGN WITH BUCKETS 

Align with each bucket long enough to capture a 

single alignment image (NO ZOOM) showing the 

green ring inside the bucket. Score 5 points for a 

continuous green ring or 1 point for a partial. Similar 

scoring for accurate or partial landings

1

2

3

4

z

z

FREE FLIGHT

z z

z

CW

CCW

CW

CCW

LAND

S

S S S S

FLIGHT LINE

1

2

3

FLIGHT LINE

4

START | END

S

S S S S

5(S)

ALTITUDE S

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

18

1

2

3

FLIGHT LINE

4

ALTITUDE S

START | END

5(S)

S

S S S S

Traverse 

(MAN 2 | PAY 2)

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

14

Position 

(MAN 1 | PAY 1)

FLIGHT LINE

4

z

z

z

ALTITUDE S

START | END 3

2

1

5(S)

ALTITUDE 2(S)

S

S S S S

ALIGNED

1A NOT ALIGNED

1A

1A
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

20

1

2

3

FLIGHT LINE

4

10FT 10FT 10FT 10FT

50FT

ALTITUDE 2(S)

START | END

ALTITUDE S

ALTITUDE 7(S)

ALTITUDE 5(S)

ALTITUDE 6(S)

7(S)

6(S)

5(S)

2(S)

Same Lane Supports Other Flight Paths

Open Test Lane

• 7(S) altitude

• 6(S) altitude 

• 5(S) altitude 

• 2(S) altitude 

• S altitude 

5 Different Orbits in Every Lane Spacing

(S) = 10ft, 20ft, 30ft, or other

12

1

2

3

4

z

z

FREE FLIGHT

z z

z

CW

CCW

CW

CCW

LAND

S

S S S S

FLIGHT LINE

1

2

3

FLIGHT LINE

4

START | END

S

S S S S

5(S)

ALTITUDE S

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

18

1

2

3

FLIGHT LINE

4

ALTITUDE S

START | END

5(S)

S

S S S S

Traverse 

(MAN 2 | PAY 2)

1

2

3

FLIGHT LINE

4

z

z

ALTITUDE X

FLY STRAIGHT AND LEVEL OVER THE LANE CENTERLINE

START | END

ALIGN

NON-STOP

NON-STOP NON-STOP

NON-STOP

NON-STOP

NON-STOP

z

S

S S S S

5(S)

S

ALIGN CW

CCW

Position 

(MAN 1 | PAY 1)

FLIGHT LINE

4

z

z

z

ALTITUDE S

START | END 3

2

1

5(S)

ALTITUDE 2(S)

S

S S S S

1

2

3

4

z

z

FREE FLIGHT

z z

z

CW

CCW

CW

CCW

LAND

S

S S S S

FLIGHT LINE

Recon

Spiral 

Orbit 

Traverse

Position

DATE/TIME

PILOT CODE

48IN SQUARE LANDING WITH

12IN CENTER RADIUS TARGET

100-300 FT MEASURING TAPE

(4) OMNI STANDS

1–4

TOP

FRONT 1A

LEFT 1B 1C BACK

1D RIGHT “A” BUCKET

ALWAYS POINTING 

TOWARD THE 

LAUNCH/LAND

PLATFORM

HANDLE

CENTERLINE

LAUNCH / LAND PLATFORM

DIGITAL CLOCK

DATE & TIME

WHITEBOARD INFO

PILOT & AIRCRAFT

(4) OMNI 

BUCKET STANDS

CENTERLINE 

MEASURING TAPE
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

21

Scoring Metrics for Test Trials

Open Test Lane

1. Completeness (Primary)

If you can’t finish a trial without faults, just keep track of how far into the trial 

sequence you get until you’re reliably finishing the trials. 

2. Score (Secondary)

For complete trials, you can start tracking your scores over time to find the average 

of the most recent five trials. That’s a good indication of your proficiency that can 

be compared to others using similar systems in similar test lanes.

3. Efficiency (Optional)

If two systems or pilots are consistently completing trials, and their average scores 

are perfect, then the average elapsed time of the last 5 trials can help identify the 

most efficient systems or pilots exhibiting the most effective techniques. 

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Track Scores Over Time

Open Test Lane

18

Download Forms Books Here

12

1

2

3

4

z

z

FREE FLIGHT

z z

z

CW

CCW

CW

CCW

LAND

S

S S S S

FLIGHT LINE

1

2

3

FLIGHT LINE

4

START | END

S

S S S S

5(S)

ALTITUDE S

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

18

1

2

3

FLIGHT LINE

4

ALTITUDE S

START | END

5(S)

S

S S S S

Traverse 

(MAN 2 | PAY 2)

1

2

3

FLIGHT LINE

4

z

z

ALTITUDE X

FLY STRAIGHT AND LEVEL OVER THE LANE CENTERLINE

START | END

ALIGN

NON-STOP

NON-STOP NON-STOP

NON-STOP

NON-STOP

NON-STOP

z

S

S S S S

5(S)

S

ALIGN CW

CCW

Position 

(MAN 1 | PAY 1)

FLIGHT LINE

4

z

z

z

ALTITUDE S

START | END 3

2

1

5(S)

ALTITUDE 2(S)

S

S S S S

1

2

3

4

z

z

FREE FLIGHT

z z

z

CW

CCW

CW

CCW

LAND

S

S S S S

FLIGHT LINE

Recon

Spiral 

Orbit 

Traverse

Position
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Track Scores Over Time

Open Test Lane

18

Download Forms Books Here

12

1

2

3

4

z

z

FREE FLIGHT

z z

z

CW

CCW

CW

CCW

LAND

S

S S S S

FLIGHT LINE

1

2

3

FLIGHT LINE

4

START | END

S

S S S S

5(S)

ALTITUDE S

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

18

1

2

3

FLIGHT LINE

4

ALTITUDE S

START | END

5(S)

S

S S S S

Traverse 

(MAN 2 | PAY 2)

1

2

3

FLIGHT LINE

4

z

z

ALTITUDE X

FLY STRAIGHT AND LEVEL OVER THE LANE CENTERLINE

START | END

ALIGN

NON-STOP

NON-STOP NON-STOP

NON-STOP

NON-STOP

NON-STOP

z

S

S S S S

5(S)

S

ALIGN CW

CCW

Position 

(MAN 1 | PAY 1)

FLIGHT LINE

4

z

z

z

ALTITUDE S

START | END 3

2

1

5(S)

ALTITUDE 2(S)

S

S S S S

1

2

3

4

z

z

FREE FLIGHT

z z

z

CW

CCW

CW

CCW

LAND

S

S S S S

FLIGHT LINE

Recon

Spiral 

Orbit 

Traverse

Position

22
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Standard Lanes and Repeatable Scenario Scores

Open Test Lane

VEHICLE INSPECTION DHS Responder Familiarization Exercise

ROOTOP 

FLUSH

OBSTACLE / SUNNY

RECESSED

OPEN / SUNNY

OPEN / SHADY

CONFINED

WIDE AREA SEARCH BUILDING EXTERIOR SEARCH

REMOTE PILOT TRAINING – CANADIAN POLICE COLLEGE, ONTARIO, CANADA

20 OMNI TARGETS = 100 PTS 20 EXTERIOR/INTEROR = 100 PTS 20 DOWNWARD 20 FORWARD

23
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

24

Football Field Layout

Open Test Lane

PILOT, VISUAL OBSERVER, 

AND FORM FILLER

PROCTOR TABLE / POWER / CHARGING / WATER

LANE 1 LANE 2 LANE 3 LANE 4 LANE 5

LANE 6

BOUNDARY

RADIUS: 10 m (30 ft)

BOUNDARY: ±15 m (45 ft) or +50% ERROR ALLOWED

BOUNDARY

BOUNDARYBOUNDARYBOUNDARYBOUNDARY

LANE SPACING 

3 m (10 ft)

BOUDARY (+50%)

±5 m (15 ft) 

LANE LENGTH

30 m (100 ft)

BOUNDARY

LANE SPACING

10 m (30 ft)

BOUNDARY (+50%)

± 15 m (45 ft) 

LANE LENGTH

100 m (300 ft)

ORBIT

TRAVERSE

BUFFER ZONEBUFFER ZONEBUFFER ZONEBUFFER ZONEBUFFER ZONEBUFFER ZONE

SENSOR PANELS

& FLIGHT LINES

LAUNCH/LAND

PLATFORM

OMNI BUCKET STANDS

#1, #2, #3, #4

ORBIT PATH AND

TRAVERSE PATH

12

1

2

3

4

z

z

FREE FLIGHT

z z

z

CW

CCW

CW

CCW

LAND

S

S S S S

FLIGHT LINE

1

2

3

FLIGHT LINE

4

START | END

S

S S S S

5(S)

ALTITUDE S

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

18

1

2

3

FLIGHT LINE

4

ALTITUDE S

START | END

5(S)

S

S S S S

Traverse 

(MAN 2 | PAY 2)

1

2

3

FLIGHT LINE

4

z

z

ALTITUDE X

FLY STRAIGHT AND LEVEL OVER THE LANE CENTERLINE

START | END

ALIGN

NON-STOP

NON-STOP NON-STOP

NON-STOP

NON-STOP

NON-STOP

z

S

S S S S

5(S)

S

ALIGN CW

CCW

Position 

(MAN 1 | PAY 1)

FLIGHT LINE

4

z

z

z

ALTITUDE S

START | END 3

2

1

5(S)

ALTITUDE 2(S)

S

S S S S

1

2

3

4

z

z

FREE FLIGHT

z z

z

CW

CCW

CW

CCW

LAND

S

S S S S

FLIGHT LINE

Recon

Spiral 

Orbit 

Traverse

Position
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Quantifying Practical Skills Requirements

Focus Training and Evaluate Proficiency for Credentialing

“ASTM F38 Standard Guide for Training for Remote Pilot in Command of UAS Endorsement”

Qualitative Task Performance Levels:

4) PROFICIENT 

• Can do the complete task quickly and accurately. 

• Can tell or show others how to do the task.

3) COMPETENT 

• Can do all parts of the task. 

• Needs only a spot check of completed work.

2) PARTIALLY PROFICIENT 

• Can do most parts of the task. 

• Needs only help on hardest parts. 

1) LIMITED

• Can do simple parts of task. 

• Needs to be told or shown how to do most of task. 

Quantitative Measures

Example Thresholds

80-100%

20-39%

40-59%

60-79%

Thresholds Set By Any Organization

Based on Airspace, Mission, Environment

Organization “A” Pass/Fail

Maybe populated, obstructed area, windy, night, etc.

Organization “C” Pass/Fail

Maybe rural, open area, calm, daytime, etc.

Organization “B” Pass/Fail

Maybe populated, open area, windy, daytime, etc.

25
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Set Your Minimum Thresholds for Pass/Fail

Focus Training and Evaluate Proficiency for Credentialing

• Organizations can set their own threshold for pass/fail in these tests based on their tolerance for reliability 

and/or efficiency. Complete trials are assumed.

• Measure everybody repeatedly over time and graph the results to help people understand their strengths and 

weaknesses. Then set minimum thresholds relative to the average or “expert” scores. Or adopt other 

organization’s thresholds as a central credentialing reference.

• At deployment time, each organization needs to consider their airspace restrictions, environmental variables, 

and mission complexity (night ops, BVLOS, etc.) to select a pilot and aircraft that’s likely to succeed.

Average Military 66%

Average Civilian 55%

Example proficiency data shown from bomb squads in ground robot tests 3626
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

27

Select Trial Settings for Different Flight Credentials

Focus Training and Evaluate Proficiency for Credentialing

CREDENTIALS Daylight/LOS BVLOS Night Ops

Standard Lane 

(Indoor or Outdoor)

Pilot’s Eyes On 

(Available)

Pilot’s Back Turned

(Interface Only) Lights Out, Buckets Lit

Embedded Scenario

(Indoor or Outdoor)

Pilot’s Eyes On 

(Available)

Pilot’s Back Turned

(Interface Only) Lights Out, Buckets Lit

ILLUMINATED BUCKETS PROVIDE 

POSITIONING AIDS LIKE STRUCTURE 

WINDOWS OR STREET LIGHTS

BVLOS

BACK TURNED

Line of Sight

FACING LANE

OR

THE PILOT’S BACK TURNED TO THE LANE FORCES 

RELIANCE ON THE INTERFACE 

(VISUAL OBSERVER REQUIRED)

1) Select the test lane and scenarios based 

on the intended environment and aircraft: 

– Open, Obstructed, or Indoor

2) Select the test procedure and time limit 

based on the intended mission:

– MAN (5 min. each) or PAY (10 min. each)

3) Select the minimum proficiency based on 

average or “expert” scores in the same trials:

– Example: 40%, 60%, 80% of “expert”

4) Select pilot view:

– Line of Sight or BVLOS

5) Select lighting (indoor or outdoor) for 

daylight or night credentials:

– Lighted/Daylight or Dark

DAYLIGHT / LOS ADDITIONAL
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

28

Montgomery County Police Drone Training & Evaluation Facility

Safety | Capabilities | Proficiency

DOJ/DHS National Unmanned Aircraft Systems Program Evaluation, August 2020

I

CONFINED TEST LANE

GPS DENIED

and

DARK (<1 LUX)

DUAL BUCKET 

ALIGNMENTS

AT 1 M (3 FT) 

PROXIMITY TO 

OBSTACLES

OPEN TEST LANE

B

LENGTH: 30/60 M (100/200 FT)

E

OBSTRUCTED TEST LANE

DUAL BUCKET 

ALIGNMENTS

AT 2 M (6 FT) 

PROXIMITY TO 

OBSTACLES

LENGTH: 15 M (50 FT) LENGTH: 7.5 M (25 FT)
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

30

Montgomery County Police Drone Training & Evaluation Facility

Safety | Capabilities | Proficiency

DOJ/DHS National Unmanned Aircraft Systems Program Evaluation, August 2020

BUS EXTERIOR

AND INTERIOR

OBSTRUCTED TEST LANE

(2) OPEN TEST LANES

AND

SENSOR TEST LANE

STRUCTURE EXTERIOR

(WINDOWS)

STRUCTURE PERIMETER 

(GROUND)

INTERIOR ROOMS

(NORTH)

INTERIOR ROOMS

(SOUTH)

INTERIOR ROOMS

(NORTH)

INTERIOR ROOMS

(SOUTH)

STRUCTURE EXTERIOR

(WINDOWS)

STRUCTURE PERIMETER 

(GROUND)

BRIEFING

PAVILLION

RADIO COMMS RANGE

ATTENUATED IN APC

RAIN TOLERANCE

CONFINED TEST LANE

(IN CLASSROOM - NOT SHOWN)

NORTH SITE
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

31

Montgomery County Police Drone Training & Evaluation Facility

Safety | Capabilities | Proficiency

WIDE AREA MAPPING EXTERIOR BUS EXTERIOR WINDOWS EXTERIOR GROUND INTERIOR ROOMS INTERIOR STAIRS INTERIOR BUS

DOJ/DHS National Unmanned Aircraft Systems Program Evaluation, August 2020

Operational Scenarios with Embedded Scoring 

(Each with 20 buckets/targets, 100 points maximum)

C x4 D H F G J K x2 L M

PERCH
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

High Hover Tests with Larger Buckets

High Hover Tests (100-300 ft)

32

(4) 20 GALLON

PERIMETER (1) 55 GALLON

IN CENTER
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Hover Stability Test

High Hover Test (100-300 ft)

33

Purpose: 

This test evaluates remotely piloted aircraft hover stability at different 

altitudes using dual bucket alignments at fixed distances from a 

designated hover position. 

This test can be used to evaluate free-flying or tethered aircraft system 

capabilities. This test can also be used as a repeatable training task to 

practice operation of the system interface and evaluate remote pilot 

proficiency. 

Summary of Test:

• Altitudes are 30 m (100 ft), 60 m (200 ft), 90 m (300 ft).

• Dual bucket alignments are performed at each altitude using 

buckets with an appropriate diameter to be seen clearly from the 

intended altitudes located straight down (90°) and along the 

centerline at an equal distance to the intended altitude (45°).

• At each altitude, separately timed alignment intervals are 

conducted alternating between the downward and angled 

alignment buckets (30 seconds per alignment, 60 seconds per complete 

repetition). 

30 m (100 ft)

60 m (200 ft)

90 m (300 ft)

60 m (200 ft)

30 m (100 ft)

90 m (300 

ft)

TETHER

LICENSE 

PLATES

OBJECTS OF 

INTEREST

THERMAL 

HAND WARMER

MOTION

DETECTION

HAZMAT 

LABELS

LICENSE 

PLATES 

OBJECTS 

INTERESTOF 

THERMAL 

ACUITY

VISUAL & COLOR ACUITY
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Purpose: 

This test evaluates remotely piloted aircraft camera pointing 

and zooming capabilities using sensor targets at various 

distances from a designated hover position and altitude. This 

test can be used to measure aircraft sensors including visual 

acuity, color acuity, thermal acuity, motion detection, hazmat 

label identification, etc. This test can also be used as a 

repeatable training task to practice the system interface and 

evaluate remote pilot proficiency for credentialing.

Features:

• Altitude checks defined by a center vertical bucket with 

angled buckets at equal distance to altitudes.

• Altitudes can be 30 m (100 ft), 60 m (200 ft), 90 m (300 ft)

Pan-Tilt-Zoom Cameras Test

High Hover Tests (100-300 ft)

34

30 m (100 ft)

60 m (200 ft)

90 m (300 ft)

60 m (200 ft)

30 m (100 ft)

90 m (300 

ft)

TETHER

LICENSE 

PLATES

OBJECTS OF 

INTEREST

THERMAL 

HAND WARMER

MOTION

DETECTION

HAZMAT 

LABELS

LICENSE 

PLATES 

OBJECTS 

INTERESTOF 

THERMAL 

ACUITY

VISUAL & COLOR ACUITY
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Purpose: 

This test evaluates remotely piloted aircraft camera pointing 

and zooming capabilities during high hover orbits to identify 

four different sides of an object from a designated altitude and 

radius.

This test can be used to measure aircraft sensor capabilities to 

identify objects of interest using targets for visual acuity, color 

acuity, thermal acuity, motion detection, hazmat label 

identification, etc. This test can also be used as a repeatable 

training task to practice the system interface and evaluate 

remote pilot proficiency for credentialing.

Features:

• Altitude and radius checks defined by a center omni with 

angled buckets and perimeter vertical buckets.

• Targets at the center with the omni bucket stand. 

Orbit Test

High Hover Tests (100-300 ft)

35

30 m (100 ft)

60 m (200 ft)

90 m (300 ft)

60 m (200 ft)

30 m (100 ft)

90 m (300 

ft)

LICENSE 

PLATES

OBJECTS OF 

INTEREST

THERMAL 

HAND WARMER

MOTION

DETECTION

HAZMAT 

LABELS

LICENSE 

PLATES 

OBJECTS 

INTERESTOF 

THERMAL 

ACUITY

VISUAL & COLOR ACUITY
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

North

(3) Washer head 

screws affix the 

bucket to the 

stand (under the 

foam ring)

(1) BUBBLE LEVEL -- 2D

(1) BUCKET BOTTOM REST

(4) ADJUSTABLE LEVELING 

LEG EXTENSIONS

Large Bucket Apparatuses

High Hover Tests (100-300 ft)

36

North

(9) 20-gallon white buckets with 18in diameter. 

https://www.uline.com/Product/Detail/H-1854W/Brute-Trash-Cans-and-Accessories/RubbermaidBrute-Trash-Can-20-Gallon-White

(9) Foam pipe insulation (2-3in diameter x 6ft long) cut to form 

the inscribed ring.

https://www.homedepot.com/p/Everbilt-3-4-in-x-6-ft-Foam-Semi-Slit-Polyethylene-Pipe-InsulationORP07812/204760801

(9) Center labels to identify each bucket (full page size or 8in)

(1) Omni directional stand with (5) 20-gallon buckets attached to 

it placed top center of the fuselage with straps and rope 

extensions to cinch tightly around the aircraft belly.

(4) Surrounding vertical bucket leveling stands if not on 

pavement (weight or stake the buckets to not move in wind)

(1) 300ft measuring tape and chalk paint and wand to mark 

measured locations.

(1) 45-degree angle level that spans the bucket top.
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Embedding into Larger Scenarios

High Hover Tests (100-300 ft)

37

North

60 m (200 ft)

30 m (100 ft)

90 m (300 ft)

ORBIT PATHS WITH OMNI CENTER AND DOWNWARD RADIUS CHECKS (9) 20-gallon white buckets with 18in diameter. 

https://www.uline.com/Product/Detail/H-1854W/Brute-Trash-Cans-and-Accessories/RubbermaidBrute-Trash-Can-20-Gallon-White

(9) Foam pipe insulation (2-3in diameter x 6ft long) cut to form 

the inscribed ring.

https://www.homedepot.com/p/Everbilt-3-4-in-x-6-ft-Foam-Semi-Slit-Polyethylene-Pipe-InsulationORP07812/204760801

(9) Center labels to identify each bucket (full page size or 8in)

(1) Omni directional stand with (5) 20-gallon buckets attached to 

it placed in the center of the scenario.

(4) Surrounding vertical bucket leveling stands if not on 

pavement (weight or stake the buckets to not move in wind)

(2) 300ft measuring tapes and chalk paint and wand to mark 

measured bucket locations.

(1) 45-degree angle level that spans the bucket top.
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Embedding into Larger Scenarios

High Hover Tests (100-300 ft)
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

39

Internet

RobotTestMethods.nist.gov

Adam	Jacoff

Intelligent	Systems	Division

NATIONAL	INSTITUTE	OF	STANDARDS	AND	TECHNOLOGY

U.S.	Department	of	Commerce

Email

RobotTestMethods@nist.gov

WEBSITE POINTER: 

WATCH FABRICATION VIDEOS 

AND FLIGHT PATH ANIMATIONS

WEBSITE POINTER: 

DOWNLOAD STICKER FILES, FORMS AND 

PRACTICE SCORING VIDEOS
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

40

Validation Exercises

Committee Business

Disaster-Resilient Buildings, Infrastructure, 

and Communities Goal

Technical Expertise: Standards Committees

Slide37

Assess the portfolio of scientific expertise

Chair/Fellow, ASTM International Standards Committee on Homeland 

Security Applications: Response Robots (ASTM E54.09) 

This committee is standardizing 50 test methods for ground, aerial, and 

aquatic systems. It hosts two meetings annually to facilitate development 

and balloting of standards. Typically we conduct an associated test 

method validation exercise.

Member, NFPA Standards Committee on Small Unmanned Aircraft 

Systems Used for Public Safety Applications (NFPA 2400)

This committee has adopted our test methods to support Job 

Performance Requirements for remote pilot proficiency and will likely set 

thresholds of performance for firefighting and other public safety 

applications. 

Member, NATO SCI-ET-052 Research Task Group on EOD Robot TeleManipulation This committee is developing NATO’s roadmap for 

dexterous robot applied to C-IED/EOD applications.

2019 Host: Houston Fire Dept

2017 Host: Canadian CETA

2018 Host: San Diego Fire Dept

• 2020.08 DHS/DOJ sUAS Procurement Testing ($35M), Montgomery County Police Facility, MD (1 days)

• 2020.10 Air Force Large Ground Robot Procurement ($70M), Tyndall AFB, FL (Weeks)

• 2020.08 DHS/DOJ sUAS Procurement Testing ($35M), Montgomery County Police Facility, MD (5 days)

• 2020.09 Canadian Fire Training Facility Opening Exercise, Toronto Airport, Ontario, Canada (4 days)

• 2020.08 World Robot Summit Disaster Response Championship, Fukushima, Japan (4 days)

• 2020.06 RoboCupRescue International Championship, Bordeaux, France (5 days)

• 2020.05 AUVSI Exponential Conference (netted aviary), Boston, MA (3 days)

• 2020.04 Fire Dept. International Conference (FDIC) Hands-On Training, Indianapolis, IN (3 days)

• 2020.03 UTAC UAS Conference, Guardian Center, Perry, GA (4 days)

• 2020.03 Public Safety UAS Conference Validation Exercise, Crozet, VA (5 days)

Color Key: Ground Aerial Aquatic Multiple Standards
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

41

Validation Exercises

Committee Business

Disaster-Resilient Buildings, Infrastructure, 

and Communities Goal

Technical Expertise: Standards Committees

Slide37

Assess the portfolio of scientific expertise

Chair/Fellow, ASTM International Standards Committee on Homeland 

Security Applications: Response Robots (ASTM E54.09) 

This committee is standardizing 50 test methods for ground, aerial, and 

aquatic systems. It hosts two meetings annually to facilitate development 

and balloting of standards. Typically we conduct an associated test 

method validation exercise.

Member, NFPA Standards Committee on Small Unmanned Aircraft 

Systems Used for Public Safety Applications (NFPA 2400)

This committee has adopted our test methods to support Job 

Performance Requirements for remote pilot proficiency and will likely set 

thresholds of performance for firefighting and other public safety 

applications. 

Member, NATO SCI-ET-052 Research Task Group on EOD Robot TeleManipulation This committee is developing NATO’s roadmap for 

dexterous robot applied to C-IED/EOD applications.

2019 Host: Houston Fire Dept

2017 Host: Canadian CETA

2018 Host: San Diego Fire Dept

Color Key: Ground Aerial Aquatic Multiple Standards

• 2020.02 ASTM E54.09 Response Robots Meeting and Exercise, Atlanta, CO (3 days)

• 2020.01 Ohio Fire Training Facility Opening, Ohio (2 days)

• 2020.01 FDIC Fire/Rescue East, Daytona, FL (2 days)

• 2020.01 Los Angeles Fire Dept. Training, Los Angeles, CA (3 days)

• 2019.12 FAA Requirements Workshop for Fire Depts and Emergency Services, NIST (1 day)

• 2019.11 Atlantic Future Forum, UK HMS Queen Elizabeth, Annapolis, MD (2 days)

• 2019.11 DHS Familiarization Exercise, Army Camp Shelby, MS (5 days)

• 2019.10 World Robot Summit, Fukushima, Japan (5 days)

• 2019.09 NATO Aerial and Ground Exercise, Base Borden, Ontario, Canada (3 days)

• 2019.07 Aerial Validation Exercise at NIST (3 days)

• 2019.06 RoboCupRescue International Championship, Sydney, Australia (5 days)
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

42

Validation Exercises

Committee Business

Disaster-Resilient Buildings, Infrastructure, 

and Communities Goal

Technical Expertise: Standards Committees

Slide37

Assess the portfolio of scientific expertise

Chair/Fellow, ASTM International Standards Committee on Homeland 

Security Applications: Response Robots (ASTM E54.09) 

This committee is standardizing 50 test methods for ground, aerial, and 

aquatic systems. It hosts two meetings annually to facilitate development 

and balloting of standards. Typically we conduct an associated test 

method validation exercise.

Member, NFPA Standards Committee on Small Unmanned Aircraft 

Systems Used for Public Safety Applications (NFPA 2400)

This committee has adopted our test methods to support Job 

Performance Requirements for remote pilot proficiency and will likely set 

thresholds of performance for firefighting and other public safety 

applications. 

Member, NATO SCI-ET-052 Research Task Group on EOD Robot TeleManipulation This committee is developing NATO’s roadmap for 

dexterous robot applied to C-IED/EOD applications.

2019 Host: Houston Fire Dept

2017 Host: Canadian CETA

2018 Host: San Diego Fire Dept

Color Key: Ground Aerial Aquatic Multiple Standards

• 2019.06 ASTM E54.09 Response Robots Meeting and Exercise, Denver, CO (5 days)

• 2019.05 Western Regional Robot Rodeo, Sandia/Kirtland, Albuquerque, NM (5 days)

• 2019.05 Canadian Police College Training Exercise, London, ON Canada (7 days)

• 2019.04 Thermite RS2 firefighting robot capabilities evaluation (1 day)

• 2019.04 Army Tank Automotive Research and Development facility fabrication (remote)

• 2019.04 Fire Dept Training Conference (FDIC), Indianapolis, IN (3 days)

• 2019.04 Guardian Center Training, Perry, GA (2 days remote)

• 2019.04 Reveille Ranch Calibration, Texas Dept of Public Safety, Burnet, TX (2 days)

• 2019.04 InstantEye UAS capabilities evaluation, NIST (3 days)

• 2019.03 ASTM F38 standard balloted referencing 6 of our aerial test methods

• 2019.03 Navy Explosive Ordinance Disposal Tech Division facility fabrication (remote)

• 2019.03 Virginia UAS Summit on Public Safety, Crozet, VA (3 days)
Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

43

XXXXXXXXXXXXXXXXXXX

Safety | Capabilities | Proficiency

SLIDE TEMPLATE