Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Aquatic System Tests

Introduction

Sub-Committee Chair:

Adam Jacoff

Intelligent Systems Division

National Institute of Standards and Technology

U.S. Department of Commerce

Committee Chair:

Phil Mattson

Science and Technology Directorate 

U.S. Department of Homeland Security

Internet

RobotTestMethods.nist.gov

Adam	Jacoff

Intelligent	Systems	Division

NATIONAL	INSTITUTE	OF	STANDARDS	AND	TECHNOLOGY

U.S.	Department	of	Commerce

Email

RobotTestMethods@nist.gov

1

Online Only Meeting

February 4, 2021

10:00am – 2:00pm EST

Version 2021A
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

2

• Reminder that electronic recording of ASTM meetings is prohibited.

• This meeting will run in accordance with the ASTM Antitrust Statement (see minutes).

Antitrust Statement

ASTM International is a not-for-profit organization and developer of voluntary consensus standards. ASTM's leadership in international 

standards development is driven by the contributions of its members: more than 30,000 technical experts and business professionals 

representing 135 countries.

The purpose of antitrust laws is to preserve economic competition in the marketplace by prohibiting, among other things, unreasonable 

restraints of trade. In ASTM activities, it is important to recognize that participants often represent competitive interests. Antitrust laws 

require that all competition be open and unrestricted.

It is ASTM's policy, and the policy of each of its committees and subcommittees, to conduct all business and activity in full compliance with 

international, federal and state antitrust and competition laws. The ASTM Board of Directors has adopted an antitrust policy which is found 

in Section 19 of ASTM Regulations Governing Technical Committees. All members need to be aware of and compliant with this policy. The 

Regulations are accessible on the ASTM website (http://www.astm.org/COMMIT/Regs.pdf) and copies of the antitrust policy are available 

at the registration desk. For a complete list of standards, see: http://www.astm.org/COMMIT/SUBCOMMIT/E5409.htm

Call To Order

Aquatic System Tests
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

3

Agenda

Aquatic System Tests

.

10:00 am EST Committee Overview, Use Cases, Validation Exercises

11:00 am EST Dexterity, C-IED/EOD, and Other Tests

12:00 pm EST Open Discussion 

POST

T-TOP

LEGS

POST

4x4x24in (10x10x60cm)

LEGS

2x4x20in (5x10x50cm)

with 45° opposing ends

T-TOP

2x4x12in (5x10x30cm)

BUCKETS

2 or 5 gal (7 or 19 liter)

Launch/Land

12in (30cm) Radius Center

Bucket Stand #1

(A,B,C,D)

10 ft (3 m) Spacings

FLIGHT LINE

SIGHT LINE

Launch/Land

12in (30cm) Radius Center

FLIGHT LINE

SIGHT LINE

Bucket Stand #1

(A,B,C,D)

Bucket Stand #2 

(E,F,G,H)

Bucket Stand #3

(I,J,K,L)

10ft (3m) Spacings

10 FT (3 M)

ALTITUDE LAUNCH LAND

Test Director 

Adam Jacoff

Intelligent Systems Division

Nat’l Institute of Standards and Technology 

U.S. Dept. of Commerce 

RobotTestMethods@nist.gov | 301-975-4235 

Sponsor 

Philip Mattson

Science and Technology Directorate 

U.S. Dept. of Homeland Security

These test methods are primarily intended for 

vertical takeoff and landing systems with an 

onboard camera and remote pilot display. Some 

are also applicable to fixed wing systems when the 

lane dimensions are extended to accommodate 

the orbit radius of forward flying systems. 

Bucket stands on a level surfaces ensure the top bucket is 

vertical and the angled buckets are 45 degrees. 

TRAVERSE TEST

Right and Left Ovals Around All Three Bucket Stands

ALIGNED

WHEN RING IS 

COMPLETELY 

VISIBLE

GROUND

AQUATIC

AERIAL
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

4

Membership

Committee Overview

POST

T-TOP

LEGS

POST

4x4x24in (10x10x60cm)

LEGS

2x4x20in (5x10x50cm)

with 45° opposing ends

T-TOP

2x4x12in (5x10x30cm)

BUCKETS

2 or 5 gal (7 or 19 liter)

Launch/Land

12in (30cm) Radius Center

Bucket Stand #1

(A,B,C,D)

10 ft (3 m) Spacings

FLIGHT LINE

SIGHT LINE

Launch/Land

12in (30cm) Radius Center

FLIGHT LINE

SIGHT LINE

Bucket Stand #1

(A,B,C,D)

Bucket Stand #2 

(E,F,G,H)

Bucket Stand #3

(I,J,K,L)

10ft (3m) Spacings

10 FT (3 M)

ALTITUDE LAUNCH LAND

Test Director 

Adam Jacoff

Intelligent Systems Division

Nat’l Institute of Standards and Technology 

U.S. Dept. of Commerce 

RobotTestMethods@nist.gov | 301-975-4235 

Sponsor 

Philip Mattson

Science and Technology Directorate 

U.S. Dept. of Homeland Security

These test methods are primarily intended for 

vertical takeoff and landing systems with an 

onboard camera and remote pilot display. Some 

are also applicable to fixed wing systems when the 

lane dimensions are extended to accommodate 

the orbit radius of forward flying systems. 

Bucket stands on a level surfaces ensure the top bucket is 

vertical and the angled buckets are 45 degrees. 

TRAVERSE TEST

Right and Left Ovals Around All Three Bucket Stands

ALIGNED

WHEN RING IS 

COMPLETELY 

VISIBLE

GROUND

AQUATIC

AERIAL
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

5

Minutes

Committee Overview

The presentations shown today will serve as the minutes of this 

meeting. They will be posted as PDFs to the ASTM E54.09 

committee website, and on NIST RobotTestMethods.nist.gov

website under Meetings where there are also pointers to the 

next/previous meetings.

• ASTM E54.09 Aquatic Tests – Descriptions (v2021A)

• ASTM E54.09 Aquatic Tests – Validation Exercises (v2021A)

POST

T-TOP

LEGS

POST

4x4x24in (10x10x60cm)

LEGS

2x4x20in (5x10x50cm)

with 45° opposing ends

T-TOP

2x4x12in (5x10x30cm)

BUCKETS

2 or 5 gal (7 or 19 liter)

Launch/Land

12in (30cm) Radius Center

Bucket Stand #1

(A,B,C,D)

10 ft (3 m) Spacings

FLIGHT LINE

SIGHT LINE

Launch/Land

12in (30cm) Radius Center

FLIGHT LINE

SIGHT LINE

Bucket Stand #1

(A,B,C,D)

Bucket Stand #2 

(E,F,G,H)

Bucket Stand #3

(I,J,K,L)

10ft (3m) Spacings

10 FT (3 M)

ALTITUDE LAUNCH LAND

Test Director 

Adam Jacoff

Intelligent Systems Division

Nat’l Institute of Standards and Technology 

U.S. Dept. of Commerce 

RobotTestMethods@nist.gov | 301-975-4235 

Sponsor 

Philip Mattson

Science and Technology Directorate 

U.S. Dept. of Homeland Security

These test methods are primarily intended for 

vertical takeoff and landing systems with an 

onboard camera and remote pilot display. Some 

are also applicable to fixed wing systems when the 

lane dimensions are extended to accommodate 

the orbit radius of forward flying systems. 

Bucket stands on a level surfaces ensure the top bucket is 

vertical and the angled buckets are 45 degrees. 

TRAVERSE TEST

Right and Left Ovals Around All Three Bucket Stands

ALIGNED

WHEN RING IS 

COMPLETELY 

VISIBLE

GROUND

AQUATIC

AERIAL
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

6

“Start Remote, Stay Remote?”

Aquatic System Tests

Mission Success = Robotic System Capabilities + Remote Operator/Pilot Proficiency

• Focus operator training and measure proficiency to track very perishable skills over time and enable 

comparison across squads, regions, or national averages. To date, the suite of ground robot test methods have 

been used by more than 250 civilian and military bomb technicians.

The suite includes 15 draft standard test methods for underwater remotely operated vehicles (UROV) that 

measure essential capabilities for countering improvised explosive devices (C-IED) and other hazardous 

underwater tasks.

Figure 1: A) Several of the underwater test methods are mounted to submerged C-frame apparatuses to present 5 

different task panel orientations to the UROV, including +90°upward, +45°upward, 0° forward, -45° downward, -90° 

downwardThey can be performed in any orderpresumably from easiest to hardestB) Multiple cameras capture all th

Validation Exercises

Flow Tank

2-4 knots

Simple Apparatuses Omnidirectional Tests System Interfaces
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

7

“Practice Makes... Relatively Reliable!”

Aquatic System Tests

Mission Success = Robotic System Capabilities + Remote Operator/Pilot Proficiency
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

8

“Practice Makes... Relatively Reliable!”

Aquatic System Tests

Mission Success = Robotic System Capabilities + Remote Operator/Pilot Proficiency
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

9

“Practice Makes... Relatively Reliable!”

Aquatic System Tests

Mission Success = Robotic System Capabilities + Remote Operator/Pilot Proficiency
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Omni Directional Tests

Aquatic System Tests

Inspect Grasp Hook Cut Affix/Aim

C-Frames are typically WOOD for periodic field exercises, ALUMINUM or PVC for longer term submerged facilities
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

11

Committee Vision

Committee Overview

Remotely operated robots, including ground, aerial, and aquatic 

systems, enable emergency responders to perform extremely 

hazardous tasks from safer stand-off distances. 

Standard test methods help robot manufacturers and users 

objectively evaluate system capabilities to align with mission 

requirements. 

So this committee is developing the measurements and standards 

infrastructure necessary to quantitatively evaluate and compare 

robotic system capabilities and remote operator proficiency. 

POST

T-TOP

LEGS

POST

4x4x24in (10x10x60cm)

LEGS

2x4x20in (5x10x50cm)

with 45° opposing ends

T-TOP

2x4x12in (5x10x30cm)

BUCKETS

2 or 5 gal (7 or 19 liter)

Launch/Land

12in (30cm) Radius Center

Bucket Stand #1

(A,B,C,D)

10 ft (3 m) Spacings

FLIGHT LINE

SIGHT LINE

Launch/Land

12in (30cm) Radius Center

FLIGHT LINE

SIGHT LINE

Bucket Stand #1

(A,B,C,D)

Bucket Stand #2 

(E,F,G,H)

Bucket Stand #3

(I,J,K,L)

10ft (3m) Spacings

10 FT (3 M)

ALTITUDE LAUNCH LAND

Test Director 

Adam Jacoff

Intelligent Systems Division

Nat’l Institute of Standards and Technology 

U.S. Dept. of Commerce 

RobotTestMethods@nist.gov | 301-975-4235 

Sponsor 

Philip Mattson

Science and Technology Directorate 

U.S. Dept. of Homeland Security

These test methods are primarily intended for 

vertical takeoff and landing systems with an 

onboard camera and remote pilot display. Some 

are also applicable to fixed wing systems when the 

lane dimensions are extended to accommodate 

the orbit radius of forward flying systems. 

Bucket stands on a level surfaces ensure the top bucket is 

vertical and the angled buckets are 45 degrees. 

TRAVERSE TEST

Right and Left Ovals Around All Three Bucket Stands

ALIGNED

WHEN RING IS 

COMPLETELY 

VISIBLE

GROUND

AQUATIC

AERIAL
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

12

When We Started

Committee Overview

POST

T-TOP

LEGS

POST

4x4x24in (10x10x60cm)

LEGS

2x4x20in (5x10x50cm)

with 45° opposing ends

T-TOP

2x4x12in (5x10x30cm)

BUCKETS

2 or 5 gal (7 or 19 liter)

Launch/Land

12in (30cm) Radius Center

Bucket Stand #1

(A,B,C,D)

10 ft (3 m) Spacings

FLIGHT LINE

SIGHT LINE

Launch/Land

12in (30cm) Radius Center

FLIGHT LINE

SIGHT LINE

Bucket Stand #1

(A,B,C,D)

Bucket Stand #2 

(E,F,G,H)

Bucket Stand #3

(I,J,K,L)

10ft (3m) Spacings

10 FT (3 M)

ALTITUDE LAUNCH LAND

Test Director 

Adam Jacoff

Intelligent Systems Division

Nat’l Institute of Standards and Technology 

U.S. Dept. of Commerce 

RobotTestMethods@nist.gov | 301-975-4235 

Sponsor 

Philip Mattson

Science and Technology Directorate 

U.S. Dept. of Homeland Security

These test methods are primarily intended for 

vertical takeoff and landing systems with an 

onboard camera and remote pilot display. Some 

are also applicable to fixed wing systems when the 

lane dimensions are extended to accommodate 

the orbit radius of forward flying systems. 

Bucket stands on a level surfaces ensure the top bucket is 

vertical and the angled buckets are 45 degrees. 

TRAVERSE TEST

Right and Left Ovals Around All Three Bucket Stands

ALIGNED

WHEN RING IS 

COMPLETELY 

VISIBLE

GROUND

AQUATIC

AERIAL

• Lack of Coordinated Innovation and Commercialization

• Difficult to assess performance of robotic systems

• Robots were not addressing end-user needs

• No mechanism to tangibly communicate operational requirements

• No structured training for operators to improve proficiency

• No credentialing of remote operators and pilots

• Standard Test Methods Need To...

• Communicate operational needs to robot researchers and developers.

• Promote innovation through commercial manufacturers.

• Enable users to understand emerging robot capabilities. 

• Guide robot purchasing, acceptance testing, and deployment decisions.

• Focus training and measure operator proficiency for credentialing.
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

13

Stakeholders & Priorities

Committee Overview

Identifying our priorities:

• Requirements workshops with all interested stakeholders identify capability gaps and priorities.

• Test validation exercises with users refine and validate apparatuses, procedures, and data collection.

• Robot evaluations with manufacturers capture statistically significant capabilities data.

• Standards committee meetings and exercises prepare the tests for balloting and adoption.

• Research competitions validate and disseminate tests, inspire innovation, and measure progress.

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09) http://RobotTestMethods.nist.gov

ors

 and Developing 

ndard Test Methods

International Eco-System Developing/Deploying Robots

Challenge | Measure | Communicate

8

Standard Test Methods for Small Unmanned Aircraft Systems

ASTM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09) http://RobotTestMethods.nist.gov

Sponsors

Adopting and Developing 

New Standard Test Methods

Robot Purchases

Using Robot Capabilities

Data From Test Methods

$70M to date

An International Eco-System Developing/Deploying Robots

Challenge | Measure | Communicate

FEDERAL SPONSORS 

AND COLLABORATORS

COMMERCIAL

PROCUREMENTS

STANDARDS

ORGANIZATIONS

ROBOTICS RESEARCH

COMPETITONS

$200 M THROUGH 2020

7

Standard Test Methods for Small Unmanned Aircraft Systems ASTM International Standards Committee for Homeland Security Applications;

Response Robots (E54.09) http://RobotTestMethods.nist.gov

The Same Standard Test Methods Help Different Users

Challenge | Measure | Communicate

Robot Developers

§ Understand missions through tangible test apparatuses

§ Practice and refine robot designs, make trade-off decisions

§ Highlight “Best-In-Class” capabilities

Responders, Soldiers, and Other Users

§ Compare robots with objective data, not marketing

§ Specify purchases based on existing combinations of capabilities

§ Align expectations with deployment decisions

Program Managers 

§ Describe objectives with a collection of tangible tasks

§ Challenge conventional approaches and inspire innovation

§ Measure baseline capabilities and document progress
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

14

Our Approach

Committee Overview

• Develop test methods

• Representative

• Repeatable

• Reproducible

• Science-based

• Inexpensive & easy to conduct

• Enable innovation 

• Competition challenges

• Identify and communicate gaps

• Measure performance

• Compare different system capabilities

• Track and compare operator proficiency

50+ TESTS FOR

Maneuvering

Mobility

Sensing

Endurance

Radio Comms

Dexterity

Durability

Logistics

Safety

Mapping

Autonomy

POST

T-TOP

LEGS

POST

4x4x24in (10x10x60cm)

LEGS

2x4x20in (5x10x50cm)

with 45° opposing ends

T-TOP

2x4x12in (5x10x30cm)

BUCKETS

2 or 5 gal (7 or 19 liter)

Launch/Land

12in (30cm) Radius Center

Bucket Stand #1

(A,B,C,D)

10 ft (3 m) Spacings

FLIGHT LINE

SIGHT LINE

Launch/Land

12in (30cm) Radius Center

FLIGHT LINE

SIGHT LINE

Bucket Stand #1

(A,B,C,D)

Bucket Stand #2 

(E,F,G,H)

Bucket Stand #3

(I,J,K,L)

10ft (3m) Spacings

10 FT (3 M)

ALTITUDE LAUNCH LAND

Test Director 

Adam Jacoff

Intelligent Systems Division

Nat’l Institute of Standards and Technology 

U.S. Dept. of Commerce 

RobotTestMethods@nist.gov | 301-975-4235 

Sponsor 

Philip Mattson

Science and Technology Directorate 

U.S. Dept. of Homeland Security

These test methods are primarily intended for 

vertical takeoff and landing systems with an 

onboard camera and remote pilot display. Some 

are also applicable to fixed wing systems when the 

lane dimensions are extended to accommodate 

the orbit radius of forward flying systems. 

Bucket stands on a level surfaces ensure the top bucket is 

vertical and the angled buckets are 45 degrees. 

TRAVERSE TEST

Right and Left Ovals Around All Three Bucket Stands

ALIGNED

WHEN RING IS 

COMPLETELY 

VISIBLE

GROUND

AQUATIC

AERIAL
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

15

• Develop suites of reproducible test methods that are quick and easy to conduct 

repeatedly.

• Measure combinations of existing capabilities and emerging technologies.

• Inspire innovation using tests to communicate operational needs and 

technological gaps.

• Guide purchasing and deployment decisions with objective capabilities data.

• Focus training with repeatable tasks to measure and compare operator 

proficiency.

• Identify readiness issues with equipment and/or training through local, regional, 

or national averages.

REPEAT

Our Process

Committee Overview

POST

T-TOP

LEGS

POST

4x4x24in (10x10x60cm)

LEGS

2x4x20in (5x10x50cm)

with 45° opposing ends

T-TOP

2x4x12in (5x10x30cm)

BUCKETS

2 or 5 gal (7 or 19 liter)

Launch/Land

12in (30cm) Radius Center

Bucket Stand #1

(A,B,C,D)

10 ft (3 m) Spacings

FLIGHT LINE

SIGHT LINE

Launch/Land

12in (30cm) Radius Center

FLIGHT LINE

SIGHT LINE

Bucket Stand #1

(A,B,C,D)

Bucket Stand #2 

(E,F,G,H)

Bucket Stand #3

(I,J,K,L)

10ft (3m) Spacings

10 FT (3 M)

ALTITUDE LAUNCH LAND

Test Director 

Adam Jacoff

Intelligent Systems Division

Nat’l Institute of Standards and Technology 

U.S. Dept. of Commerce 

RobotTestMethods@nist.gov | 301-975-4235 

Sponsor 

Philip Mattson

Science and Technology Directorate 

U.S. Dept. of Homeland Security

These test methods are primarily intended for 

vertical takeoff and landing systems with an 

onboard camera and remote pilot display. Some 

are also applicable to fixed wing systems when the 

lane dimensions are extended to accommodate 

the orbit radius of forward flying systems. 

Bucket stands on a level surfaces ensure the top bucket is 

vertical and the angled buckets are 45 degrees. 

TRAVERSE TEST

Right and Left Ovals Around All Three Bucket Stands

ALIGNED

WHEN RING IS 

COMPLETELY 

VISIBLE

GROUND

AQUATIC

AERIAL
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Same Tests Help Different Users and Robots

Committee Overview Robot Developers

§ Understand missions through tangible test apparatuses

§ Practice and refine robot designs, make trade-off decisions

§ Highlight “Best-In-Class” capabilities

Responders, Soldiers, Other Users

§ Compare robots with objective data, not marketing

§ Specify purchases based on existing combinations of capabilities

§ Align expectations with deployment decisions

Program Managers 

§ Describe objectives with a collection of tangible tasks

§ Challenge conventional approaches and inspire innovation

§ Measure baseline capabilities and document progress

POST

T-TOP

LEGS

POST

4x4x24in (10x10x60cm)

LEGS

2x4x20in (5x10x50cm)

with 45° opposing ends

T-TOP

2x4x12in (5x10x30cm)

BUCKETS

2 or 5 gal (7 or 19 liter)

Launch/Land

12in (30cm) Radius Center

Bucket Stand #1

(A,B,C,D)

10 ft (3 m) Spacings

FLIGHT LINE

SIGHT LINE

Launch/Land

12in (30cm) Radius Center

FLIGHT LINE

SIGHT LINE

Bucket Stand #1

(A,B,C,D)

Bucket Stand #2 

(E,F,G,H)

Bucket Stand #3

(I,J,K,L)

10ft (3m) Spacings

10 FT (3 M)

ALTITUDE LAUNCH LAND

Test Director 

Adam Jacoff

Intelligent Systems Division

Nat’l Institute of Standards and Technology 

U.S. Dept. of Commerce 

RobotTestMethods@nist.gov | 301-975-4235 

Sponsor 

Philip Mattson

Science and Technology Directorate 

U.S. Dept. of Homeland Security

These test methods are primarily intended for 

vertical takeoff and landing systems with an 

onboard camera and remote pilot display. Some 

are also applicable to fixed wing systems when the 

lane dimensions are extended to accommodate 

the orbit radius of forward flying systems. 

Bucket stands on a level surfaces ensure the top bucket is 

vertical and the angled buckets are 45 degrees. 

TRAVERSE TEST

Right and Left Ovals Around All Three Bucket Stands

ALIGNED

WHEN RING IS 

COMPLETELY 

VISIBLE

GROUND

AQUATIC

AERIAL
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

17

Previous Meetings with Validation Exercises

Committee Overview

ASTM E54.09 Meeting and Test 

Validation Exercise

Host:

Virginia Beach Fire Dept., 

Virginia Beach, VA

January 2017

Interview with Kathy Morgan, President of ASTM International
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

18

Previous Meetings with Validation Exercises

Committee Overview

ASTM E54.09 

Meeting and Test 

Validation Exercise

Host: Canadian 

Explosives 

Technicians 

Association

Hamilton, Ontario, 

Canada

June 2017 (AND 

JUNE 4-7, 2021)

“The tremendous work in design, validation, and delivery of NIST’s Emergency Response Robots 

Project has opened the door to operationalization of a training/qualification environment. Similar 

to weapon qualifications, CETA sees the NIST program as just that for public safety 

professionals, minimum standards.”
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

19

Previous Meetings with Validation Exercises

Committee Overview

ASTM E54.09 

Meeting and Test 

Validation Exercise

Host: San Dieg, Fire 

Dept. and Navy, 

SPAWAR

San Diego, CA

June 2018 
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

20

Previous Meetings with Validation Exercises

Committee Overview

ASTM E54.09 

Meeting and Test 

Validation Exercise

Host: San Diego, Fire 

Dept. and Navy, 

SPAWAR

San Diego, CA

June 2018 
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

21

Annual Test Validation Events

Committee Overview

ANNUAL EVENTS VALIDATING OUR TESTS YEARS ‘21 ‘20 ‘19 ‘18 ‘17 ‘16 ‘15 ‘14 ‘13 ‘12 ‘11

© RoboCupRescue International Championships X – X X X X X X X X X

© RoboCupRescue Regional Open, Germany X – X X X X X X X X X

© RoboCupRescue Regional Open, Japan X – X X X X X X X X X

© RoboCupRescue Regional Open, Thailand X – X X X X X X X X X

Robot Test Facility Openings (1+ per year), Int’l X – X X X X X X X X X

© DARPA Robotics Challenge for Disaster Response, USA X X X

© World Robot Summit Disaster Challenge, Japan X – X X

European Robotics League / Eurathlons, Europe X – X X X X X X X

Robot Rodeos and Raven’s Challenges, USA/Canada X X – X X X X X X X X X X X

Conference Expos with Demonstrations, USA X X – X X X X X X X X X XX X X X X X X X X

ASTM E54 Response Robot Exercises, USA X X – X X X X X X X X X X X X X X X X X X X

We go where the robots are! 
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

22

Test Facilities Worldwide

Committee Overview
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Conducting Standard Test Methods

Safety | Capabilities | Proficiency

Conducting Standard Test Methods

Safety | Capabilities | Proficiency
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Example of Quantitative Performance Comparisons

Robots Capabilities, Operator Proficiency, and/or Autonomous Behaviors

MOBILITY TEST METHODS

MOBILITY: Confined Area Terrains (figure-8 path, minimum 150 meters) 

 Average rate of advance comparison of robots on increasingly complex terrains: 

0

10

20

30

40

50

60

70

80

ReconScout FirstLook Armad-Base SandFlea DR10-Base Armad-Manip DR10-SCK Pointman

0 3 0 0 8 4

41

11 9 6 7

25

13 11

38

14 17 19 19

30

21

15

64

19 17 19

8 12 14 11

75

38

30

38

32 30 30 30

38 38

25

38 38

30

38

0

(meters per minute)

Sand Gravel Flat Line Following Continuous Ramps Crossing Ramps Stepfields 

ROBOT A ROBOT B ROBOT C ROBOT D ROBOT E ROBOT F ROBOT G ROBOT H

Robot Capabilities Data to Support Procurement/Deployment

Average Rate of Advance on Terrain for At Least 100m 

(meters/minute)

Compare Robot Capabilities or Proficiency

Safety | Capabilities | Proficiency

Thrust Inspect Grasp Cut Hook Affix
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Example of Quantitative Performance Comparisons

Robots Capabilities, Operator Proficiency, and/or Autonomous Behaviors

Compare Robot Capabilities or Proficiency

Safety | Capabilities | Proficiency

Sampling of Bomb Tech Readiness Nationwide

Sequence of 10 Basic Maneuvering and Dexterity Test Methods in 30 Minutes

Score

100+ Bomb Techs Using Their Operational Robots (USA & Canada)

Civilian Average

Best In Class Robot with 

Expert Operator (Orange)

Conventional Robot with 

Expert Operator (Orange

Military Average
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Simple Rules for Comparing Scores 

Focus Training and Evaluate Proficiency for Credentialing

• Compare scores in the same tests or embedded scenarios

Use ALL tests and scenarios that apply to the robot or to the intended missions.

• Compare scores with the same trial times

Trial time limits help normalize fatigue across several tests, so novices don’t get worn 

out unnecessarily. They provide enough time for an “expert” to easily perform a 

complete trial. Longer times can be used for scenarios with embedded tests.

• Compare scores to “average” or “best-in-class” across organizations

• ”Expert” operators designated by the manufacturer capture the system’s best 

possible performance in each test. This is the 100th percentile of remote operator 

proficiency for that system. Your score is 1-100% of that “expert” score

• For training, use the entire time limit and track your scores. When they become 

repeatable, your learning phase is over. The best indicator of your proficiency in 

each test is the average of your last 5 trials. 

• We will post “average” and ”expert” scores and rates as we collect them. 26

POST

T-TOP

LEGS

POST

4x4x24in (10x10x60cm)

LEGS

2x4x20in (5x10x50cm)

with 45° opposing ends

T-TOP

2x4x12in (5x10x30cm)

BUCKETS

2 or 5 gal (7 or 19 liter)

Launch/Land

12in (30cm) Radius Center

Bucket Stand #1

(A,B,C,D)

10 ft (3 m) Spacings

FLIGHT LINE

SIGHT LINE

Launch/Land

12in (30cm) Radius Center

FLIGHT LINE

SIGHT LINE

Bucket Stand #1

(A,B,C,D)

Bucket Stand #2 

(E,F,G,H)

Bucket Stand #3

(I,J,K,L)

10ft (3m) Spacings

10 FT (3 M)

ALTITUDE LAUNCH LAND

Test Director 

Adam Jacoff

Intelligent Systems Division

Nat’l Institute of Standards and Technology 

U.S. Dept. of Commerce 

RobotTestMethods@nist.gov | 301-975-4235 

Sponsor 

Philip Mattson

Science and Technology Directorate 

U.S. Dept. of Homeland Security

These test methods are primarily intended for 

vertical takeoff and landing systems with an 

onboard camera and remote pilot display. Some 

are also applicable to fixed wing systems when the 

lane dimensions are extended to accommodate 

the orbit radius of forward flying systems. 

Bucket stands on a level surfaces ensure the top bucket is 

vertical and the angled buckets are 45 degrees. 

TRAVERSE TEST

Right and Left Ovals Around All Three Bucket Stands

ALIGNED

WHEN RING IS 

COMPLETELY 

VISIBLE

GROUND

AQUATIC

AERIAL
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Set Your Minimum Thresholds for Pass/Fail

Focus Training and Evaluate Proficiency for Credentialing

• Organizations can set their own threshold for pass/fail in these tests based on their tolerance for reliability 

and/or efficiency. Complete trials are assumed.

• Measure everybody repeatedly over time and graph the results to help people understand their strengths and 

weaknesses. Then set minimum thresholds relative to the average or “expert” scores. Or adopt other 

organization’s thresholds as a central credentialing reference.

• At deployment time, each organization needs to consider their environmental variables, and mission 

complexity (night ops, BVLOS, etc.) to select a robot and operator that is likely to succeed.

Average Military 66%

Average Civilian 55%

Example proficiency data shown from bomb squads in ground robot tests 27
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Related Awards

Committee Overview

28

Presidential Gears of Government Award (2020)

For developing the first ever comprehensive suite of emergency response robot test methods and data collection tools to 

evaluate and improve bomb-disposal robots and operators. These efforts led to enhanced testing and use of advanced robot 

capabilities that enable emergency responders to perform extremely hazardous missions from safer standoff distances. 

Secretary Ron Brown Excellence in Innovation Award, U.S. Department of Commerce (2019)

Gold Medal Award, U.S. Department of Commerce (2019)

Award of Merit, ASTM International Standards Society (2015)

Measurement Science Award, National Institute of Standards and Technology (2014)

Commendationsfrom Dept. of Justice (2020, 2016), Air Force (2018), DARPA (2015), Dept. of Homeland 

Security (2014), Dept. of State (2014), JIEDDO (2010), several state and local responder organizations and 

international organizations.

This same process will work similarly 

well for AERIAL and AQUATIC systems.
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

29

Validation Exercises

Committee Overview

Disaster-Resilient Buildings, Infrastructure, 

and Communities Goal

Technical Expertise: Standards Committees

Slide37

Assess the portfolio of scientific expertise

Chair/Fellow, ASTM International Standards Committee on Homeland 

Security Applications: Response Robots (ASTM E54.09) 

This committee is standardizing 50 test methods for ground, aerial, and 

aquatic systems. It hosts two meetings annually to facilitate development 

and balloting of standards. Typically we conduct an associated test 

method validation exercise.

Member, NFPA Standards Committee on Small Unmanned Aircraft 

Systems Used for Public Safety Applications (NFPA 2400)

This committee has adopted our test methods to support Job 

Performance Requirements for remote pilot proficiency and will likely set 

thresholds of performance for firefighting and other public safety 

applications. 

Member, NATO SCI-ET-052 Research Task Group on EOD Robot TeleManipulation This committee is developing NATO’s roadmap for 

dexterous robot applied to C-IED/EOD applications.

2019 Host: Houston Fire Dept

2017 Host: Canadian CETA

2018 Host: San Diego Fire Dept

• 2020.08 DHS/DOJ sUAS Procurement Testing ($35M), Montgomery County Police Facility, MD (1 days)

• 2020.10 Air Force Large Ground Robot Procurement ($70M), Tyndall AFB, FL (Weeks)

• 2020.08 DHS/DOJ sUAS Procurement Testing ($35M), Montgomery County Police Facility, MD (5 days)

• 2020.09 Canadian Fire Training Facility Opening Exercise, Toronto Airport, Ontario, Canada (4 days)

• 2020.08 World Robot Summit Disaster Response Championship, Fukushima, Japan (4 days)

• 2020.06 RoboCupRescue International Championship, Bordeaux, France (5 days)

• 2020.05 AUVSI Exponential Conference (netted aviary), Boston, MA (3 days)

• 2020.04 Fire Dept. International Conference (FDIC) Hands-On Training, Indianapolis, IN (3 days)

• 2020.03 UTAC UAS Conference, Guardian Center, Perry, GA (4 days)

• 2020.03 Public Safety UAS Conference Validation Exercise, Crozet, VA (5 days)

Color Key: Ground Aerial Aquatic Multiple Standards
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

30

Validation Exercises

Committee Overview

Disaster-Resilient Buildings, Infrastructure, 

and Communities Goal

Technical Expertise: Standards Committees

Slide37

Assess the portfolio of scientific expertise

Chair/Fellow, ASTM International Standards Committee on Homeland 

Security Applications: Response Robots (ASTM E54.09) 

This committee is standardizing 50 test methods for ground, aerial, and 

aquatic systems. It hosts two meetings annually to facilitate development 

and balloting of standards. Typically we conduct an associated test 

method validation exercise.

Member, NFPA Standards Committee on Small Unmanned Aircraft 

Systems Used for Public Safety Applications (NFPA 2400)

This committee has adopted our test methods to support Job 

Performance Requirements for remote pilot proficiency and will likely set 

thresholds of performance for firefighting and other public safety 

applications. 

Member, NATO SCI-ET-052 Research Task Group on EOD Robot TeleManipulation This committee is developing NATO’s roadmap for 

dexterous robot applied to C-IED/EOD applications.

2019 Host: Houston Fire Dept

2017 Host: Canadian CETA

2018 Host: San Diego Fire Dept

Color Key: Ground Aerial Aquatic Multiple Standards

• 2020.02 ASTM E54.09 Response Robots Meeting and Exercise, Atlanta, CO (3 days)

• 2020.01 Ohio Fire Training Facility Opening, Ohio (2 days)

• 2020.01 FDIC Fire/Rescue East, Daytona, FL (2 days)

• 2020.01 Los Angeles Fire Dept. Training, Los Angeles, CA (3 days)

• 2019.12 FAA Requirements Workshop for Fire Depts and Emergency Services, NIST (1 day)

• 2019.11 Atlantic Future Forum, UK HMS Queen Elizabeth, Annapolis, MD (2 days)

• 2019.11 DHS Familiarization Exercise, Army Camp Shelby, MS (5 days)

• 2019.10 World Robot Summit, Fukushima, Japan (5 days)

• 2019.09 NATO Aerial and Ground Exercise, Base Borden, Ontario, Canada (3 days)

• 2019.07 Aerial Validation Exercise at NIST (3 days)

• 2019.06 RoboCupRescue International Championship, Sydney, Australia (5 days)
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

31

Validation Exercises

Committee Overview

Disaster-Resilient Buildings, Infrastructure, 

and Communities Goal

Technical Expertise: Standards Committees

Slide37

Assess the portfolio of scientific expertise

Chair/Fellow, ASTM International Standards Committee on Homeland 

Security Applications: Response Robots (ASTM E54.09) 

This committee is standardizing 50 test methods for ground, aerial, and 

aquatic systems. It hosts two meetings annually to facilitate development 

and balloting of standards. Typically we conduct an associated test 

method validation exercise.

Member, NFPA Standards Committee on Small Unmanned Aircraft 

Systems Used for Public Safety Applications (NFPA 2400)

This committee has adopted our test methods to support Job 

Performance Requirements for remote pilot proficiency and will likely set 

thresholds of performance for firefighting and other public safety 

applications. 

Member, NATO SCI-ET-052 Research Task Group on EOD Robot TeleManipulation This committee is developing NATO’s roadmap for 

dexterous robot applied to C-IED/EOD applications.

2019 Host: Houston Fire Dept

2017 Host: Canadian CETA

2018 Host: San Diego Fire Dept

Color Key: Ground Aerial Aquatic Multiple Standards

• 2019.06 ASTM E54.09 Response Robots Meeting and Exercise, Denver, CO (5 days)

• 2019.05 Western Regional Robot Rodeo, Sandia/Kirtland, Albuquerque, NM (5 days)

• 2019.05 Canadian Police College Training Exercise, London, ON Canada (7 days)

• 2019.04 Thermite RS2 firefighting robot capabilities evaluation (1 day)

• 2019.04 Army Tank Automotive Research and Development facility fabrication (remote)

• 2019.04 Fire Dept Training Conference (FDIC), Indianapolis, IN (3 days)

• 2019.04 Guardian Center Training, Perry, GA (2 days remote)

• 2019.04 Reveille Ranch Calibration, Texas Dept of Public Safety, Burnet, TX (2 days)

• 2019.04 InstantEye UAS capabilities evaluation, NIST (3 days)

• 2019.03 ASTM F38 standard balloted referencing 6 of our aerial test methods

• 2019.03 Navy Explosive Ordinance Disposal Tech Division facility fabrication (remote)

• 2019.03 Virginia UAS Summit on Public Safety, Crozet, VA (3 days)
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

32

Flow Tank (4 knots), Carderock, MD

Validation Exercises
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

33

Port Hueneme, CA

Validation Exercises
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

34

Virginia Beach Fire, Dept., VA

Validation Exercises

Aquatic

Rented Frac Tank or Pool Add Turbulence with 

Submerged Pumps

Coastal	Trident	Coastal	Trident	2016	– Regional	Port	and	Maritime	Security	Program,	Port	Hueneme,	CA 7

pg)

b. Inspect Floating	Objects:	Below	Water Line	(10 minute	time	limit):

The	purpose	of	this	test	is	to	evaluate	the	maneuvering	and	camera	pointing	

capabilities	of	the	system	below	the	water	line	by	performing	a	series	of	tasks	to	

inspect	targets	from	multiple	angles.	This	test	is conducted on	the	water	surface.	It	

should	be	performed	in	still	water	and	with	turbulence	to	challenge	auto	stationkeeping	capabilities.

• Start	2m	(6ft)	from	the	designated	task	location on	the	water	surface.

pect below	the	water	line inside	downward	facing	pipes	10cm	(4in) in	

meter x	10cm	(4in)	deep to	correctly	identify the number	of	randomly	placed	

s	1.2cm	(0.5in)	wide	x	5cm	(2in)	long	on	the	inner	surface. Capture	images	of	

h	to	verify,	if	possible.	

peat	in	any	order	to	completely	identify	the	set	until	time	expires.	If	

mpleted,	start	identifying	a	second	set	in	any	order.	

Floating	Objects	(10	minute	time	limit):

rpose	of	this	test is	to	evaluate	the	maneuvering,	dexterity, and	gripper	

h	capabilities	of	the	system	by	performing	a	series	of	tasks	to	grasp soft	

s	of	a	floating	object and	tow	some	distance.	This	test	is	conducted	on	the	

urface.	It	should	be	performed	in	still	water	and	with	turbulence	to	challenge	

ation-keeping	capabilities

rt	2m	(6ft)	from	the	designated	task	location on	the	water	surface.

sp at	water	level	a	soft	strap made	of	seat	belt	loops	10cm	(4in)	in	diameter	

ddthbjt5(15ft)itditdd

• Start	2m	(6ft)	from	the	designated	task	location on	the	water	surface.

• Grasp at	water	level	a	soft	strap made	of	seat	belt	loops	10cm	(4in)	in	diameter	

and	drag the	object	5m	(15	ft)	into	a	designated	end	zone.	

• Release	and	grasp the	opposite	soft	strap and drag	back	to	start zone.	

• Repeat	until	time	expires.

d. Hook	Floating	Objects	(10	minute	time	limit):

The	purpose	of	this	test	is	to	evaluate	the	maneuvering,	dexterity,	and	tool	

deployment	capabilities	of	the	system	by	performing	a	series	of	tasks	to	hook	

carabiners	to a	floating	object as	a	mock	attachment	of	a remote	pull	line.	This	test	

is conducted on	the	water	surface.	It	should	be	performed	in	still	water	and	with	

turbulence	to	challenge	auto	station-keeping	capabilities

• Start	2m	(6ft)	from	the	designated	task	location on	the	water	surface.

• Hook at	water	level	a	carabiner	to	a D-ring 10cm	(4in)	in	diameter	leaving the

carabiner attachedwithouttheassociated remotepullline

)()y

y	collection	of	system	cameras	as	an	indicator	of	omni

nal	awareness.	But	the	level	of	acuity	is	part	of	the	

g	the best	cameras	is	advised.

esist	extracting	a	seat	belt	strap	from	a	closed	gripper	will	be	

emoved	from	the	tank	after	testing	is	complete	(time	

ts:	Below	Water Line	(10 minute	time	limit):

t	is	to	evaluate	the	maneuvering	and	camera	pointing	

em	below	the	water	line	by	performing	a	series	of	tasks	to	

ultiple	angles.	This	test	is conducted on	the	water	surface.	It	

n	still	water	and	with	turbulence	to	challenge	auto	station

	the	designated	task	location on	the	water	surface.

Floating Surface Tasks
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

35

SAAB Manufacturer Site, Sweden

Use Case Examples
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

36

NATO SCI-342 Research Task Group

Use Case Examples

• Adopting and extending our tests to measure robot 

dexterity and remote operator proficiency necessary 

to perform explosive ordinance disposal (EOD) 

missions from remote standoff distances. 

• Will replicate the modular dexterity tests in several 

collaborating NATO countries, including the NATO 

Center of Excellence for C-EOD Operations in 

Slovakia.

• Adam Jacoff leads the Subgroup 4 Evaluation (Testing 

& Metrics)

• The charter will be active 2020-2023 

Subgroup 4 Evaluation (Testing & Metrics)

• Adam Jacoff

• Aurélie Lepoil

• André Volk

• Johannes Pellenz

• Eric den Breejen
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

37

Canadian CETA and CERRA Training/Credentialing

Use Case Examples

Lead Agencies;

CETA- Canadian Explosives Technicians Association

CERRA- Canadian Emergency Responders Robotics Association

Primary Locations:

Pearson International Airport ( Toronto Canada)

Grimsby Regional Training Centre ( Grimsby, Ontario , Canada)

CETA

CETA is the national association for police/military/government agencies 

tasked with response to explosives , chemical, biological, and radiological 

incidents in Canada. Current projects include EOD Standard training methods 

for both robots and bomb techs deployed in bomb suits.

CERRA

Spring 2020 established with focus on the public safety deployment of 

ground, air, water based robotics. Membership is open to any current or 

former public safety member or agency or any supporting government agency 

with an interest in response robots.
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

38

World Robot Summit, Fukushima, Japan (2018-2020)

Use Case Examples

Standard Disaster Robotics Category, Fukushima Robot Test Field, Fukushima, Japan
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

“Standard Test Methods in a Box”

Idea for ISO Container Facility Put a Frac Tank Facility in Your Parking Lot (Rent or Buy)

Safety | Capabilities | Proficiency
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

“Standard Test Methods in a Box”

Idea for ISO Container Facility Put a Frac Tank Facility in Your Parking Lot (Rent or Buy)

Safety | Capabilities | Proficiency

Not so hard... get a firefighter to do it!
Standard Test Methods for Response Robots

ASTM International Standards Committee on Homeland Security Applications; 

Response Robots (E54.09) | Website: RobotTestMethods.nist.gov

Sub Committee Chair:

Adam Jacoff

Intelligent Systems Division

National Institute of Standards and Technology

U.S. Department of Commerce

Committee Chair:

Phil Mattson

Science and Technology Directorate 

U.S. Department of Homeland Security

Internet

RobotTestMethods.nist.gov

Adam	Jacoff

Intelligent	Systems	Division

NATIONAL	INSTITUTE	OF	STANDARDS	AND	TECHNOLOGY

U.S.	Department	of	Commerce

Email

RobotTestMethods@nist.gov

41