Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sensing & Radio Comms

Aerial Test Methods

Sub Committee Chair

Adam Jacoff

Intelligent Systems Division

National Institute of Standards and Technology

U.S. Department of Commerce

Committee Chair:

Phil Mattson

Science and Technology Directorate 

U.S. Department of Homeland Security

Internet

RobotTestMethods.nist.gov

Adam	Jacoff

Intelligent	Systems	Division

NATIONAL	INSTITUTE	OF	STANDARDS	AND	TECHNOLOGY

U.S.	Department	of	Commerce

Email

RobotTestMethods@nist.gov

1

Version 2021A

Online Only Meeting

February 3, 2021

WEBSITE POINTER: 10:00am – 2:00pm EST

WATCH FABRICATION VIDEOS 

AND FLIGHT PATH ANIMATIONS

WEBSITE POINTER: 

DOWNLOAD STICKER FILES, FORMS AND 

PRACTICE SCORING VIDEOS
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

2

DIFFERENT LEVELS OF ACUITY

Interface

LIVE

Streamed

BETTER

Recovered

BEST

Aerial Sensing: Visual Acuity Test

ASTM E2566-2017

VISUAL/THERMAL ACUITY TEST CHART CONCENTRIC LANDOLT RINGS – 30 REPETITIONS

A

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications

Response Robots (E54.09) Website: RobotTestMethods.nist.gov

3

4

5

B C E F Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Test ChartCombined Vision 

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01) A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long. Combined Vision Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Test ChartCombined Vision 

Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above should be 50 mm (1.97 in) long.

Combined Vision Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Combined Vision Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Test ChartCombined Vision 

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Test ChartCombined Vision Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

Methods for Response Robots

ards Committee on Homeland Security Applications;

nal Equipment; Robots (E54.08.01)

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. A 

2

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Test ChartCombined Vision 

ASTM International Standards Committee on Homeland Security Applications;

Standard Test Methods for Response Robots

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

should be 50 mm (1.97 in) long.

For proper calibration, the line above 

Combined Vision 

Test Chart

Standard Test Methods for Re

ASTM International Standards Committee on Homeland Secur

y 

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Test Chart

Combined Vision 

Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above should be 50 mm (1.97 in) long.

Combined Vision Test Chart Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above should be 50 mm (1.97 in) long.

Combined Vision Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. A For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart ASTM International Standards Committee on Homeland Security Applications;

Standard Test Methods for Response Robots

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

should be 50 mm (1.97 in) long.

For proper calibration, the line above 

Combined Vision 

Test Chart

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Test ChartCombined Vision 

1

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above should be 50 mm (1.97 in) long.

Combined Vision Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. A For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

for Response Robots

ee on Homeland Security Applications;

ent; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

Combined Vision 

Test Chart

ASTM International Standards Committee on Homeland Security Applications;

Standard Test Methods for Response Robots

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

Combined Vision 

Test Chart

for Response Robots

ee on Homeland Security Applications;

ent; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

ard Test Methods for Response Robots

ernational Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. 

sion 

Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01) A

Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

Combined Vision Test Chart

2A 2B 2C 2D 2E 2F

3A 3B 3C 3D 3E 3F

4A 4B 4C 4D 4E 4F

5A 5B 5C 5D 5E 5F

D

GAP SIZES:

RING 1: 38 mm (1.5 in)

RING 2: 13 mm (0.5 in)

RING 3: 5.0 mm (0.2 in)

RING 4: 2.5 mm (0.1 in)

RING 5: 1.3 mm (0.05 in) Visual Acuity Test Char

Landolt C Optotypes representing features from 

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; Response 

http://robottestmethods.nist.gov | robottestmethods@nist.go

100% 

Human 

6/6 (20/20)

Vision at 

3m (10ft)

100% Human 

6/6 (20/20)

Vision at 

6m (20ft)

0.6mm 0.6mm 0.6mm 0.6mm

0.8mm 0.8mm 0.8mm 0.8mm

0.9mm 0.9mm 0.9mm 0.9mm

1.1

mm 1.1mm 1.1mm 1.1mm

1.5

mm 1.5mm 1.5mm 1.5mm

1.8

mm 1.8mm 1.8mm 1.8mm

2.2

mm 2.2mm 2.2mm 2.2mm

1.3

mm 1.3mm 1.3mm 1.3mm

2.5

mm 2.5mm 2.5mm 2.5mm

3

mm 3mm 3mm 3mm

3.6

mm 3.6mm 3.6mm 3.6mm

4.3

mm 4.3mm 4.3mm 4.3mm

5

mm 5mm 5mm 5mm

6

mm 6mm 6mm 

7.27.27.2

2.2

mm 

2.5

mm 

3

mm 

3.6

mm 3.6mm 

4.3

mm 4.3mm 

5

mm 5mm 

6

mm 6mm 6m

7.2

mm 7.2mm 7.2mm 

100% Human 

6/6 (20/20)

Vision at 

6m (20ft)

SENSORS  AUTONOMY  PROFICIENCY  DURABILITY

 ENERGY  MOBILITY  DEXTERITY  COMMS  SAFETY 

RESPONDERS

MANUFACTURERS

ADMINISTRATORS

AERIAL  GROUND  AQUATIC

CHALLENGE  TRAIN  EVALUATE

STANDARDTEST METHODS

RESPONSE ROBOT CAPABILITIES

1A 1B 1C 1D 1E 1F

MORE CONTINUOUS TARGETS 5 DEEP CONCENTRIC C TARGETS
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

3

Aerial Sensing: Color Acuity Test

COLOR DOTS ASTM WK54755 ROTATING HAZMAT LABELS

This bar should be 150mm long.

1

A B C D E F

2

3

4

5

1A 1B 1C 1D 1E 1F

2A 2B 2C 2D 2E 2F

3A 3B 3C 3D 3E 3F

4A 4B 4C 4D 4E 4F

5A 5B 5C 5D 5E 5F

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications

Response Robots (E54.09) | http://RobotTestMethods.nist.gov

HAZARDOUS MATERIAL LABEL TEST CHART

30 RANDOM, RANDOMLY ORIENTED 100 MM (4 IN) LABELS A This bar should be 150mm long.

1

A B C D E F

2

3

4

5

1A 1B 1C 1D 1E 1F

2A 2B 2C 2D 2E 2F

3A 3B 3C 3D 3E 3F

4A 4B 4C 4D 4E 4F

5A 5B 5C 5D 5E 5F

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications

Response Robots (E54.09) | http://RobotTestMethods.nist.gov

COLOR ACUITY TEST CHART 6 RANDOM HAZMAT LABEL COLORS - 30 REPETITIONS &RORUVDVVSHFL¿HGLQ&)5&RGHRI)HGHUDO5HJXODWLRQVG

Colors: 

Red 3DQWRQH8

Orange 3DQWRQH8

Yellow 3DQWRQH8

Green 3DQWRQH8

Blue 3DQWRQH8 A Black
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Visual, Color, Motion, Thermal, and Operationally Significant Objects

LICENSE 

PLATES

OBJECTS OF 

INTEREST

THERMAL 

HAND WARMER

MOTION

DETECTION

HAZMAT 

LABELS

LICENSE 

PLATES 

OBJECTS 

INTEREST

OF 

THERMAL 

ACUITY

VISUAL & COLOR ACUITY

Aerial Sensing: Point and Zoom Camera Test

ASTM WK33261
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Sensing: Point and Zoom Camera Test

ASTM WK33261

z

ALTITUDE 2(S)

2

1

MID RANGE = 4(S) 

FAR RANGE = 6(S)

A

B

C

D

3

NEAR RANGE = 2(S)

ALIGN

ALIGN

TOP

( T )

( B )

BOTTOM 

( TR ) TOP RIGHT

( R ) RIGHT

( BR ) BOTTOM RIGHT

TOP LEFT ( TL )

LEFT ( L )

BOTTOM LEFT ( BL )

A

Gap size (mm) Gap size (inches) Equivalent Avg. Human vision at 6 m (20 !)

40 mm 1.58 in 4.5 %

16 mm 0.63 in 11 %

6.4 mm 0.25 in 28 %

2.6 mm 0.10 in 70 %

1.0 mm 0.04 in 180 %

0.41 mm 0.016 in 440 %

For accurate calibration, the outer diameter of the largest Landolt C optotype should be 200 mm (7.9 in). 

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

Nested Landolt C Vision Test Chart

#3

1. Visual Acuity

2. Color Acuity

3. Hazmat Label Identification

4. Motion Detection

5. Thermal Acuity

THERMAL 

ACUITY

MOTION

DETECTION

HAZMAT 

LABELS

LICENSE 

PLATES 

OBJECTS OF 

INTEREST

VISUAL & COLOR ACUITY

THERMAL

ACUITY
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Ground Sensing: Point and Zoom Camera Test

ASTM WK33261

Point/Zoom Cameras

SENSING

TOP

( T )

( B )

BOTTOM 

( TR ) TOP RIGHT

( R ) RIGHT

( BR ) BOTTOM RIGHT

TOP LEFT ( TL )

LEFT ( L )

BOTTOM LEFT ( BL )

A

Gap size (mm) Gap size (inches) Equivalent Avg. Human vision at 6 m (20 !)

40 mm 1.58 in 4.5 %

16 mm 0.63 in 11 %

6.4 mm 0.25 in 28 %

2.6 mm 0.10 in 70 %

1.0 mm 0.04 in 180 %

0.41 mm 0.016 in 440 %

For accurate calibration, the outer diameter of the largest Landolt C optotype should be 200 mm (7.9 in). 

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

Nested Landolt C Vision Test Chart

#3

MID RANGE 

4(S) 

FAR RANGE 

6(S) 

NEAR RANGE

2(S) 

NEAR RANGE

2(S) 

A

B

C

D

ROBOT ROTATES IN PLACE AND

STAYS ENTIRELY BEHIND THE LINE 
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

7

Aerial Sensing: Point and Zoom Camera Test

ASTM WK33261

Thermal acuity circular hole patterns. The large holes are 1 inch 

diameter and small holes are 1/2 inch diameter. One of the 8 

directions is missing, like the gap on the visual acuity targets.

There is a sticker template to drill through in the Disk Insert file.

A simpler approach is to fold a hand warmer into roughly a line 

and staple it to the panel vertical, horizontal, or diagonal

WATCH MOVIE OF ASSEMBLY PROCESS HERE
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Sensing: Motion Detection

ASTM WK_______

8
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Sensing: Thermal Image Acuity

ASTM WK57967

9

Heated reptile pads or hand warmers behind laser cut or drilled facades

(Indoor or outdoor use – typical sticker targets warmed by the sun also work)

Drill Holes (1in, 1/2in, 1/4in) through plastic disks 

with hand warmers heating a metal disk backing. 

Concentric Cs laser cut into MDF with a reptile 

heater. A metal backing helps diffuse the heat.

An array of Concentric C thermal targets

placed throughout a scenario (needs power).
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Sensing: Video Latency

ASTM WK46478

Latency test with flashing “SOS” beacon or other light

High speed camera video (240 fps) captures flash in field AND flash on display views simultaneously.

10

Standard Test Methods for Response Robots

ASTM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09) 

http://RobotTestMethods.nist.gov Email: RobotTestMethods@nist.gov

deo Latency

dates including those from ASTM Collaboration site: 

Update graphics to show option of test performed with 

ight at the operator station. 

 

m Robot Operator 

Control Unit

Next to Strobe

High Speed 

Recording 

Camera

Picture from 

High Speed 

Recording Camera

Radio or 

Tether

Count the frames 

between flashes
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Sensing: Remote Latency and Packet Loss

ASTM WK46478

11

UP RANGE WITH OCU

(VIDEO CAPTURE WITH INTERFACE)

DOWN RANGE WITH ROBOT 

(VIEWED THROUGH INTERFACE)

Standard Test Methods for Response Robots

ASTM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09) 

http://RobotTestMethods.nist.gov Email: RobotTestMethods@nist.gov

Latency Standard Test Methods for Response Robots

TM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09) 

otTestMethods.nist.gov Email: RobotTestMethods@nist.gov

UP RANGE WITH OCU

(VIDEO CAPTURE WITH INTERFACE)

DOWN RANGE WITH ROBOT 

(VIEWED THROUGH INTERFACE)

COMPUTER READABLE CODES

SYNCRONIZED AT BOTH ENDS

HUMAN READABLE CLOCKS

SYNCRONIZED AT BOTH ENDS
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Sensing: Audio Acuity (2-Way)

ASTM WK60783 

Alpha-numeric list read by a computer voice 

AUDIO ACUITY TEST1.

0	MISSES	IN	2	LINES ALLOWED. 0	IN	10	NUMBERS.

1	MISS IN	3	LINES ALLOWED. 1	IN	15	NUMBERS.

2	MISSES	IN	5	LINES ALLOWED. 2	IN	25 NUMBERS.

3	MISSES	IN	6	LINES ALLOWED.	3	IN	30 NUMBERS.

A! 1. 2. 3. 4. 5. _____ _____ _____ _____ _____

B! 6. 2. 3. 5. 4. _____ _____ _____ _____ _____

C! 2. 5. 9. 8. 7. _____ _____ _____ _____ _____

D! 7. 2. 8. 9. 5. _____ _____ _____ _____ _____

E! 3. 4. 9. 1. 0. _____ _____ _____ _____ _____

F! 5. 8. 0. 2. 9. _____ _____ _____ _____ _____

G! 6. 9. 7. 3. 8. _____ _____ _____ _____ _____

H!	 2. 0. 5. 2. 7. _____ _____ _____ _____ _____

I!	 3. 5. 2. 8. 9. _____ _____ _____ _____ _____

J!	 7. 2. 6. 1. 6. _____ _____ _____ _____ _____

K!	 8. 3. 3. 4. 5. _____ _____ _____ _____ _____

Loudness set to 75-80 dB

1 2 3 4 5

6 2 3 5 4

2 5 9 8 7

7 2 8 3 5

3 4 9 1 0

5 8 0 2 9

6 9 7 8

2 0 5 2 7

3

LISTEN

TALK

TALK

LISTEN

Point

SENSING

NEA

2

ROBOT ROTATES IN PLACE AND

STAYS ENTIRELY BEHIND THE LINE 
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Sensing: 3D Range Imagers and Scanners

ASTM WK____

13

ROOTOP 

FLUSH

OBSTACLE / SUNNY

RECESSED

OPEN / SUNNY

OPEN / SHADY

CONFINED

Resolution Mapping
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Inspect objects of interest

using lighted buckets (red or white)

Identify objects

lighted from the aircraft

Measure additional 

sensor capabilities

Position accuracy for range to target

using lighted buckets (red or white)

Aerial Sensing: Light Emissions

ASTM WK______

Thermal Acuity

Inside Buckets

14

WHITE OR RED HEADLAMPS

WRAPPED AROUND BUCKETS POINTED INWARD
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Sensing: Combined Sensor Crates (aka “Victim” Crates)

WK _____

15
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Configuration: Noise Emissions

WK _____

16

• Conducted during a practice Recon test flying 

straight and level toward/away from the 

microphone S distance from the landing (on 

the A-frame)
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Radio Comms Range Tests

Aerial Test Methods

Test Director:

Adam Jacoff

Intelligent Systems Division

National Institute of Standards and Technology

U.S. Department of Commerce

Sponsor:

Phil Mattson

Science and Technology Directorate 

U.S. Department of Homeland Security

Internet

RobotTestMethods.nist.gov

Adam	Jacoff

Intelligent	Systems	Division

NATIONAL	INSTITUTE	OF	STANDARDS	AND	TECHNOLOGY

U.S.	Department	of	Commerce

Email

RobotTestMethods@nist.gov

17
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Radio Comms: Line-of-Sight Range

ASTM WK______

gyg 

rface of sufficient length is not available, the test shall be considered a non-standard 

perational test and the test form annotated with a statement to the effect that the robot met o

ceeded the maximum range of the test apparatus. There must be no obstructions on the pav

rface or radio reflective metal objects within 50 m (165 ft) of the centerline to minimize ef

om multi-path radio transmissions. A roadway, airstrip, or parking lot can be used dependi

n the overall length required (see Figure 1).

 

gure 2: Left) The aerial robot maneuvering tasks include four locations around a central point, each denoted b

1

TOP

FRONT 1A

LEFT 1B 1C B

ALL “A” BUCKETS

POINT TOWARD THE 

LAUNCH/LAND

HANDLE

CENTERLINE

ONLY 

CHAN

OMNI 

Open Test Lane

POST

LEGS 

FLUSH WITH TOP

SHOWN NOT ALIGNED 

WHITE BUCKETS & GREEN RINGS 

IN STANDARD TEST LANES
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Radio Comms: Line-of-Sight Range

ASTM WK______

surface of sufficient length is not available, the test shall be considered a non-standard 

operational test and the test form annotated with a statement to the effect that the robot met or 

exceeded the maximum range of the test apparatus. There must be no obstructions on the paved 

surface or radio reflective metal objects within 50 m (165 ft) of the centerline to minimize effe

from multi-path radio transmissions. A roadway, airstrip, or parking lot can be used depending 

on the overall length required (see Figure 1).

 

Figure 2: Left) The aerial robot maneuvering tasks include four locations around a central point, each denoted by 

buckets. The aerial robot is shown at the (A) position. Middle) The center bucket stand, buckets limit the view

qy 

propagation effects, although the same test can be conducted in any operationally significant 

environment as a practical measure of line-of-sight radio communications range.

351 radio communications range. Variants of this test can be conducted with the operator co

352 inside the armored personnel carrier to determine the maximum line-of-sight range with 

353 hatches open or closed, facing down range through the front window or otherwise, and 

354 engine on or off. The specific configurations used shall be noted on the trial form. 

355

356 Figure 6: An example of an operational variant of the test apparatus setup. This is an operational variant of t

357 to the use of the enclosed armoured vehicle as an attenuating operator station enclosure, the grass surface, an

358 presence of the fence, which can also affect the radio by introducing reflections and multipathing. 

359

360 7. Hazards

361 7.1 Functional emergency stop systems are essential for safe remote or autonomou

362 robot operation. The emergency stop on the operator control unit shall be clearly marke

363 accessible. For larger aerial robots that may pose a wider danger, it is highly recommen

OPTIONAL ATTENUATION

FROM INSIDE AN ARMORED 

PERSONNEL CARRIER
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Radio Comms: Non-Line-of-Sight Range

ASTM WK__________

ods for Response Robots

mittee for Homeland Security Applications;

Robots (E54.09) 

Aerial apparatuses (shown here) use angled buckets.

Ground robot apparatuses use horizontal buckets.

OPERATOR STATION UP RANGE 

HALF THE LINE-OF-SIGHT MAX RANGE
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Radio Comms: Non-Line-of-Sight Range

ASTM WK______
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Radio Comms: Non-Line-of-Sight Range

ASTM WK______
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aerial Radio Comms: Non-Line-of-Sight Range

ASTM WK______

Standard Test Methods for Response Robots

ASTM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09) 

Radio Communications

•Guidance for embedding into building scenario.

Guidance fo
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Line-of-Sight Range

ASTM E2854-2013

24

100m

50m

AIRSTRIP OR PAVED ROAD

OLD VERSION
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Line-of-Sight Range

ASTM E2854-2020

25

5

Figure 1: Overview of the test site showing a roadway, airstrip, or parking lot with a centerline and measured 

incremental distances between the omnidirectional robot tasks and a movable remote operator interface. 

151 totals 20 points for overall acuity. 

152 4.5 There are four performance metrics to consider when calculating the results of a test t

153 They should be considered in the following order of importance: Line-of-Sight R

154 Communications Range, Reliability, Average Visual Acuity, and Efficiency. 

155 4.6 This test is performed with appropriate safety precautions to mitigate any potent

156 dangerous robot behaviors due to lost communications. The operator performs the maneuve

157 and visual acuity tasks from a standoff distance near where loss of either control or video is evid

158 The test is then repeated closer to the robot along the centerline at incremental distances of 

159 (20 ft) until all omnidirectional maneuvering and visual acuity tasks are performed successfu

160 The maximum distance from the remote operator interface and its co-located antenna to the ce

161 of the circle is considered the maximum line-of-sight radio communications range. 

162

163 Figure 3: Left) The robot is shown successfully straddling the circular line by having at least one ground conta

164 both sides. Middle) When ground contacts are touching the circular line the robot is still considered success

EXXXX

Alternatively, the robot could have left the circular line toward the inside which would also be considered 

unsuccessful.

Figure 4: Successful alignment with each bucket requires visibility of a continuously inscribed ring on the interior 

bottom of the bucket. The inscribed ring is shown as green with a black inner edge to increase contrast. When the 
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Line-of-Sight Range

ASTM E2854-2020

26

familiar with the test procedures and to capture baseline performance. Practice 

view of the robot in the apparatus. Then repeat using only the operator interface

he tasks as if beyond visual line-of-sight (see Figure 7). 

al robot tasks including robot maneuvering and visual 

t in the apparatus. Then repeat using only the operator interface

yond visual line-of-sight (see Figure 7). 

PERPENDICULAR BUCKETS BUCKET IDENTIFICATION POSITION

Maneuvering Tasks (5 points) and Visual Acuity Tasks (20 Points)
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Non-Line-of-Sight Range

ASTM E2855-2013

27

100m

50m

AIRSTRIP OR PAVED ROAD

OLD VERSION
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Non-Line-of-Sight Range

ASTM E2855-2021

28
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Non-Line-of-Sight Range

ASTM E2855-2021

29

STACKABLE SHIPPING CONTAINERS WITH LINE FOLLOWING, 

BUCKET ALIGNMENT, AND ACUITY TASKS BEHIND.

OPERATOR STATION UP RANGE 

HALF THE LINE-OF-SIGHT MAX RANGE
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Non-Line-of-Sight Range

ASTM E2855-2021

30
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sub Committee Chair:

Adam Jacoff

Intelligent Systems Division

National Institute of Standards and Technology

U.S. Department of Commerce

Committee Chair:

Phil Mattson

Science and Technology Directorate 

U.S. Department of Homeland Security

Internet

RobotTestMethods.nist.gov

Adam	Jacoff

Intelligent	Systems	Division

NATIONAL	INSTITUTE	OF	STANDARDS	AND	TECHNOLOGY

U.S.	Department	of	Commerce

Email

RobotTestMethods@nist.gov

31