Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sensor Tests

Ground Robots

Test Director:

Adam Jacoff

Intelligent Systems Division

National Institute of Standards and Technology

U.S. Department of Commerce

Sponsor:

Science and Technology Directorate 

U.S. Department of Homeland Security

Internet

RobotTestMethods.nist.gov

Adam	Jacoff

Intelligent	Systems	Division

NATIONAL	INSTITUTE	OF	STANDARDS	AND	TECHNOLOGY

U.S.	Department	of	Commerce

Email

RobotTestMethods@nist.gov

1

Version 2020C
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

2

DIFFERENT LEVELS OF ACUITY

Interface

LIVE

Streamed

BETTER

Recovered

BEST

Sensing: Visual Acuity Test

ASTM E2566-2017

VISUAL/THERMAL ACUITY TEST CHART CONCENTRIC LANDOLT RINGS – 30 REPETITIONS

A

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications

Response Robots (E54.09) Website: RobotTestMethods.nist.gov

3

4

5

B C E F Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Test ChartCombined Vision 

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01) A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long. Combined Vision Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Test ChartCombined Vision 

Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above should be 50 mm (1.97 in) long.

Combined Vision Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Combined Vision Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Test ChartCombined Vision 

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Test ChartCombined Vision Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

Methods for Response Robots

ards Committee on Homeland Security Applications;

nal Equipment; Robots (E54.08.01)

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. A 

2

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Test ChartCombined Vision 

ASTM International Standards Committee on Homeland Security Applications;

Standard Test Methods for Response Robots

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

should be 50 mm (1.97 in) long.

For proper calibration, the line above 

Combined Vision 

Test Chart

Standard Test Methods for Re

ASTM International Standards Committee on Homeland Secur

y 

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Test Chart

Combined Vision 

Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above should be 50 mm (1.97 in) long.

Combined Vision Test Chart Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above should be 50 mm (1.97 in) long.

Combined Vision Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. A For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart ASTM International Standards Committee on Homeland Security Applications;

Standard Test Methods for Response Robots

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

should be 50 mm (1.97 in) long.

For proper calibration, the line above 

Combined Vision 

Test Chart

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. For proper calibration, the line above should be 50 mm (1.97 in) long.

Test ChartCombined Vision 

1

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above should be 50 mm (1.97 in) long.

Combined Vision Test Chart

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. A For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Combined Vision 

Test Chart

for Response Robots

ee on Homeland Security Applications;

ent; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

Combined Vision 

Test Chart

ASTM International Standards Committee on Homeland Security Applications;

Standard Test Methods for Response Robots

Operational Equipment; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

Combined Vision 

Test Chart

for Response Robots

ee on Homeland Security Applications;

ent; Robots (E54.08.01)

A

Landolt C represents features of size 

20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

For proper calibration, the line above 

should be 50 mm (1.97 in) long.

ard Test Methods for Response Robots

ernational Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

A Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm. 

sion 

Standard Test Methods for Response Robots ASTM International Standards Committee on Homeland Security Applications; Operational Equipment; Robots (E54.08.01) A

Landolt C represents features of size 20 mm, 8 mm, 3.2 mm, 1.3 mm and 0.52 mm.

Combined Vision Test Chart

2A 2B 2C 2D 2E 2F

3A 3B 3C 3D 3E 3F

4A 4B 4C 4D 4E 4F

5A 5B 5C 5D 5E 5F

D

GAP SIZES:

RING 1: 38 mm (1.5 in)

RING 2: 13 mm (0.5 in)

RING 3: 5.0 mm (0.2 in)

RING 4: 2.5 mm (0.1 in)

RING 5: 1.3 mm (0.05 in) Visual Acuity Test Char

Landolt C Optotypes representing features from 

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; Response 

http://robottestmethods.nist.gov | robottestmethods@nist.go

100% 

Human 

6/6 (20/20)

Vision at 

3m (10ft)

100% Human 

6/6 (20/20)

Vision at 

6m (20ft)

0.6mm 0.6mm 0.6mm 0.6mm

0.8mm 0.8mm 0.8mm 0.8mm

0.9mm 0.9mm 0.9mm 0.9mm

1.1

mm 1.1mm 1.1mm 1.1mm

1.5

mm 1.5mm 1.5mm 1.5mm

1.8

mm 1.8mm 1.8mm 1.8mm

2.2

mm 2.2mm 2.2mm 2.2mm

1.3

mm 1.3mm 1.3mm 1.3mm

2.5

mm 2.5mm 2.5mm 2.5mm

3

mm 3mm 3mm 3mm

3.6

mm 3.6mm 3.6mm 3.6mm

4.3

mm 4.3mm 4.3mm 4.3mm

5

mm 5mm 5mm 5mm

6

mm 6mm 6mm 

7.27.27.2

2.2

mm 

2.5

mm 

3

mm 

3.6

mm 3.6mm 

4.3

mm 4.3mm 

5

mm 5mm 

6

mm 6mm 6m

7.2

mm 7.2mm 7.2mm 

100% Human 

6/6 (20/20)

Vision at 

6m (20ft)

SENSORS  AUTONOMY  PROFICIENCY  DURABILITY

 ENERGY  MOBILITY  DEXTERITY  COMMS  SAFETY 

RESPONDERS

MANUFACTURERS

ADMINISTRATORS

AERIAL  GROUND  AQUATIC

CHALLENGE  TRAIN  EVALUATE

STANDARDTEST METHODS

RESPONSE ROBOT CAPABILITIES

1A 1B 1C 1D 1E 1F

MORE CONTINUOUS TARGETS 5 DEEP CONCENTRIC C TARGETS
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

3

Sensing: Color Acuity Test

COLOR DOTS ASTM WK54755 ROTATING HAZMAT LABELS

This bar should be 150mm long.

1

A B C D E F

2

3

4

5

1A 1B 1C 1D 1E 1F

2A 2B 2C 2D 2E 2F

3A 3B 3C 3D 3E 3F

4A 4B 4C 4D 4E 4F

5A 5B 5C 5D 5E 5F

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications

Response Robots (E54.09) | http://RobotTestMethods.nist.gov

HAZARDOUS MATERIAL LABEL TEST CHART

30 RANDOM, RANDOMLY ORIENTED 100 MM (4 IN) LABELS A This bar should be 150mm long.

1

A B C D E F

2

3

4

5

1A 1B 1C 1D 1E 1F

2A 2B 2C 2D 2E 2F

3A 3B 3C 3D 3E 3F

4A 4B 4C 4D 4E 4F

5A 5B 5C 5D 5E 5F

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications

Response Robots (E54.09) | http://RobotTestMethods.nist.gov

COLOR ACUITY TEST CHART 6 RANDOM HAZMAT LABEL COLORS - 30 REPETITIONS &RORUVDVVSHFL¿HGLQ&)5&RGHRI)HGHUDO5HJXODWLRQVG

Colors: 

Red 3DQWRQH8

Orange 3DQWRQH8

Yellow 3DQWRQH8

Green 3DQWRQH8

Blue 3DQWRQH8 A Black
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Visual, Color, Motion, Thermal, and Operationally Significant Objects

LICENSE 

PLATES

OBJECTS OF 

INTEREST

THERMAL 

HAND WARMER

MOTION

DETECTION

HAZMAT 

LABELS

LICENSE 

PLATES 

OBJECTS 

INTEREST

OF 

THERMAL 

ACUITY

VISUAL & COLOR ACUITY

Sensing: Point and Zoom Camera Test

ASTM WK33261
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sensing: Point and Zoom Camera Test

ASTM WK33261

Point/Zoom Cameras

SENSING

TOP

( T )

( B )

BOTTOM 

( TR ) TOP RIGHT

( R ) RIGHT

( BR ) BOTTOM RIGHT

TOP LEFT ( TL )

LEFT ( L )

BOTTOM LEFT ( BL )

A

Gap size (mm) Gap size (inches) Equivalent Avg. Human vision at 6 m (20 !)

40 mm 1.58 in 4.5 %

16 mm 0.63 in 11 %

6.4 mm 0.25 in 28 %

2.6 mm 0.10 in 70 %

1.0 mm 0.04 in 180 %

0.41 mm 0.016 in 440 %

For accurate calibration, the outer diameter of the largest Landolt C optotype should be 200 mm (7.9 in). 

Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications;

Operational Equipment; Robots (E54.08.01)

Nested Landolt C Vision Test Chart

#3

MID RANGE 

4(S) 

FAR RANGE 

6(S) 

NEAR RANGE

2(S) 

NEAR RANGE

2(S) 

A

B

C

D

ROBOT ROTATES IN PLACE AND

STAYS ENTIRELY BEHIND THE LINE 
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

6

Sensing: Point and Zoom Camera Test

ASTM WK33261

Thermal acuity circular hole patterns. The large holes are 1 inch 

diameter and small holes are 1/2 inch diameter. One of the 8 

directions is missing, like the gap on the visual acuity targets.

There is a sticker template to drill through in the Disk Insert file.

A simpler approach is to fold a hand warmer into roughly a line 

and staple it to the panel vertical, horizontal, or diagonal

WATCH MOVIE OF ASSEMBLY PROCESS HERE
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sensing: Motion Detection

ASTM WK_______

7
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sensing: Thermal Image Acuity

ASTM WK57967

8

Heated reptile pads or hand warmers behind laser cut or drilled facades

(Indoor or outdoor use – typical sticker targets warmed by the sun also work)

Drill Holes (1in, 1/2in, 1/4in) through plastic disks 

with hand warmers heating a metal disk backing. 

Concentric Cs laser cut into MDF with a reptile 

heater. A metal backing helps diffuse the heat.

An array of Concentric C thermal targets

placed throughout a scenario (needs power).
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sensing: Video Latency

ASTM WK46478

Latency test with flashing “SOS” beacon or other light

High speed camera video (240 fps) captures flash in field AND flash on display views simultaneously.

9

Standard Test Methods for Response Robots

ASTM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09) 

http://RobotTestMethods.nist.gov Email: RobotTestMethods@nist.gov

deo Latency

dates including those from ASTM Collaboration site: 

Update graphics to show option of test performed with 

ight at the operator station. 

 

m Robot Operator 

Control Unit

Next to Strobe

High Speed 

Recording 

Camera

Picture from 

High Speed 

Recording Camera

Radio or 

Tether

Count the frames 

between flashes
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sensing: Remote Latency and Packet Loss

ASTM WK46478

10

UP RANGE WITH OCU

(VIDEO CAPTURE WITH INTERFACE)

DOWN RANGE WITH ROBOT 

(VIEWED THROUGH INTERFACE)

Standard Test Methods for Response Robots

ASTM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09) 

http://RobotTestMethods.nist.gov Email: RobotTestMethods@nist.gov

Latency Standard Test Methods for Response Robots

TM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09) 

otTestMethods.nist.gov Email: RobotTestMethods@nist.gov

UP RANGE WITH OCU

(VIDEO CAPTURE WITH INTERFACE)

DOWN RANGE WITH ROBOT 

(VIEWED THROUGH INTERFACE)

COMPUTER READABLE CODES

SYNCRONIZED AT BOTH ENDS

HUMAN READABLE CLOCKS

SYNCRONIZED AT BOTH ENDS
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sensing: Audio Acuity (2-Way)

ASTM WK60783 

Alpha-numeric list read by a computer voice 

AUDIO ACUITY TEST1.

0	MISSES	IN	2	LINES ALLOWED. 0	IN	10	NUMBERS.

1	MISS IN	3	LINES ALLOWED. 1	IN	15	NUMBERS.

2	MISSES	IN	5	LINES ALLOWED. 2	IN	25 NUMBERS.

3	MISSES	IN	6	LINES ALLOWED.	3	IN	30 NUMBERS.

A! 1. 2. 3. 4. 5. _____ _____ _____ _____ _____

B! 6. 2. 3. 5. 4. _____ _____ _____ _____ _____

C! 2. 5. 9. 8. 7. _____ _____ _____ _____ _____

D! 7. 2. 8. 9. 5. _____ _____ _____ _____ _____

E! 3. 4. 9. 1. 0. _____ _____ _____ _____ _____

F! 5. 8. 0. 2. 9. _____ _____ _____ _____ _____

G! 6. 9. 7. 3. 8. _____ _____ _____ _____ _____

H!	 2. 0. 5. 2. 7. _____ _____ _____ _____ _____

I!	 3. 5. 2. 8. 9. _____ _____ _____ _____ _____

J!	 7. 2. 6. 1. 6. _____ _____ _____ _____ _____

K!	 8. 3. 3. 4. 5. _____ _____ _____ _____ _____

Loudness set to 75-80 dB

1 2 3 4 5

6 2 3 5 4

2 5 9 8 7

7 2 8 3 5

3 4 9 1 0

5 8 0 2 9

6 9 7 8

2 0 5 2 7

3

LISTEN

TALK

TALK

LISTEN

Point

SENSING

NEA

2

ROBOT ROTATES IN PLACE AND

STAYS ENTIRELY BEHIND THE LINE 
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sensing: 3D Range Imagers and Scanners

ASTM WK____

12

ROOTOP 

FLUSH

OBSTACLE / SUNNY

RECESSED

OPEN / SUNNY

OPEN / SHADY

CONFINED

Resolution Mapping
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Inspect objects of interest

using lighted buckets (red or white)

Identify objects

lighted from the aircraft

Measure additional 

sensor capabilities

Position accuracy for range to target

using lighted buckets (red or white)

Sensing: Light Emissions

ASTM WK______

Thermal Acuity

Inside Buckets

13

WHITE OR RED HEADLAMPS

WRAPPED AROUND BUCKETS POINTED INWARD
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sensing: Combined Sensor/Dexterity Crates (aka “Victim” Crates)

WK _____

14
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms Range Tests

Ground Robots

Test Director:

Adam Jacoff

Intelligent Systems Division

National Institute of Standards and Technology

U.S. Department of Commerce

Sponsor:

Phil Mattson

Science and Technology Directorate 

U.S. Department of Homeland Security

Internet

RobotTestMethods.nist.gov

Adam	Jacoff

Intelligent	Systems	Division

NATIONAL	INSTITUTE	OF	STANDARDS	AND	TECHNOLOGY

U.S.	Department	of	Commerce

Email

RobotTestMethods@nist.gov

15
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Line-of-Sight Range

ASTM E2854-2013

16

100m

50m

AIRSTRIP OR PAVED ROAD

OLD VERSION
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Line-of-Sight Range

ASTM E2854-2020

17

5

Figure 1: Overview of the test site showing a roadway, airstrip, or parking lot with a centerline and measured 

incremental distances between the omnidirectional robot tasks and a movable remote operator interface. 

151 totals 20 points for overall acuity. 

152 4.5 There are four performance metrics to consider when calculating the results of a test t

153 They should be considered in the following order of importance: Line-of-Sight R

154 Communications Range, Reliability, Average Visual Acuity, and Efficiency. 

155 4.6 This test is performed with appropriate safety precautions to mitigate any potent

156 dangerous robot behaviors due to lost communications. The operator performs the maneuve

157 and visual acuity tasks from a standoff distance near where loss of either control or video is evid

158 The test is then repeated closer to the robot along the centerline at incremental distances of 

159 (20 ft) until all omnidirectional maneuvering and visual acuity tasks are performed successfu

160 The maximum distance from the remote operator interface and its co-located antenna to the ce

161 of the circle is considered the maximum line-of-sight radio communications range. 

162

163 Figure 3: Left) The robot is shown successfully straddling the circular line by having at least one ground conta

164 both sides. Middle) When ground contacts are touching the circular line the robot is still considered success

EXXXX

Alternatively, the robot could have left the circular line toward the inside which would also be considered 

unsuccessful.

Figure 4: Successful alignment with each bucket requires visibility of a continuously inscribed ring on the interior 

bottom of the bucket. The inscribed ring is shown as green with a black inner edge to increase contrast. When the 
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Line-of-Sight Range

ASTM E2854-2020

18

familiar with the test procedures and to capture baseline performance. Practice 

view of the robot in the apparatus. Then repeat using only the operator interface

he tasks as if beyond visual line-of-sight (see Figure 7). 

al robot tasks including robot maneuvering and visual 

t in the apparatus. Then repeat using only the operator interface

yond visual line-of-sight (see Figure 7). 

PERPENDICULAR BUCKETS BUCKET IDENTIFICATION POSITION

Maneuvering Tasks (5 points) and Visual Acuity Tasks (20 Points)
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Non-Line-of-Sight Range

ASTM E2855-2013

19

100m

50m

AIRSTRIP OR PAVED ROAD

OLD VERSION
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Non-Line-of-Sight Range

ASTM E2855-2021

20
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Non-Line-of-Sight Range

ASTM E2855-2021

21

STACKABLE SHIPPING CONTAINERS WITH LINE FOLLOWING, 

BUCKET ALIGNMENT, AND ACUITY TASKS BEHIND.

OPERATOR STATION UP RANGE 

HALF THE LINE-OF-SIGHT MAX RANGE
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Radio Comms: Non-Line-of-Sight Range

ASTM E2855-2021

22
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sub Committee Chair:

Adam Jacoff

Intelligent Systems Division

National Institute of Standards and Technology

U.S. Department of Commerce

Committee Chair:

Phil Mattson

Science and Technology Directorate 

U.S. Department of Homeland Security

Internet

RobotTestMethods.nist.gov

Adam	Jacoff

Intelligent	Systems	Division

NATIONAL	INSTITUTE	OF	STANDARDS	AND	TECHNOLOGY

U.S.	Department	of	Commerce

Email

RobotTestMethods@nist.gov

23