Stimulating Robotic 

Technology Development for 

First Responder Applications

Elena Messina 

Intelligent Systems Division

National Institute of Standards and Technology

Gaithersburg, MD 20899

http://www.isd.mel.nist.gov/US&R_Robot_Standards/

usar.robots@nist.gov

October 23, 2008
National Institute of 

Standards and Technology

NIST strengthens the U.S. economy and improves the quality of life 

by working with industry to develop and apply technology, 

measurements, and standards

NIST carries out its mission through a

portfolio of four programs:

Measurements 

and Standards

Technology 

Innovation

Manufacturing 

Extension 

Partnership

Baldrige National 

Quality
What Must Response Robots Do?

 (mobility, power, sensors, communications, operator interfaces)

What are the requirements?

How can we quantify robot performance in specific areas?

How can we abstract domain challenges?

How can we make them reproducible, repeatable? Building Confidence that new 

Technology is up to the 

Challenge
Toward Performance Standards for 

Homeland Security Robots

Standard Test 

Methods

“Consumer’s 

Guide”

Response Robots Exercises

Requirements 

from FEMA 

Teams & Bomb 

Squads
Responder Requirements

Results from 3 FEMA US&R TF Workshops

Requirements Category

Number of 

Individual 

Requirements

Category Definition

Human-System Interaction 23 Pertaining to the human interaction and operator(s) control of the 

robot

Logistics 10 Related to the overall deployment procedures and constraints in 

place for disaster response

Operating Environment 6 Surroundings and conditions in which the operator and robot will 

have to operate

System

The main body of the robot, upon which additional components and 

capabilities may be added. This is the minimum set of capabilities 

(base platform)

Chassis 4 The main body of the robot, upon which additional components and 

capabilities may be added.

Communications 5

Pertaining to the support for transmission of information to and 

from the robot, including commands for motion or control of 

payload, sensors, or other components, as well as underlying 

support for transmission of sensor and other data streams back to 

operator

Mobility 12 The ability of the robot to negotiate and move around the 

environment

Payload 7 Any additional hardware that the robot carries and may either 

deploy or utilize in the course of the mission

Power 5 Energy source(s) for the chassis and all other components on 

board the robot

Sensing 32 Hardware and supporting software which sense the environment

Safety 1 Pertaining to safety of humans and potentially property in the 

vicinity of robots
Sensing Real-time Video

Resolution of the image will be tested 

using visual acuity tests at given range. 

Image should be in color. Quality is 

evaluated through entire system (i. e., 

not standalone).

Logistics

Field 

Maintenance: 

Tools

Scale Defined: 1= Requires Special 

Tools; 3=Simple Tools (e.g., screw 

driver); 5=No Tools Required

Power (Energy) Working Time

System working time beyond mobility 

requirements. Assumes one power 

charge; one out and back mission. 

Scale defined: 1=1hr; 3=4hrs; 

5=12hrs.

Example Responder-Defined Requirements
Robot Deployment Categories

Ground: Peek Robots

Ground: Collapsed Structure--Stair/Floor climbing, map, spray, breach Robots

Ground: Non-collapsed Structure--Wide area Survey Robot

Ground: Wall Climbing Deliver Robots

Ground: Confined Space, Temporary Shore Robots

Ground: Confined Space Shape Shifters

Ground: Confined Space Retrieval Robots

Aerial:High Altitude Loiter Robots

Aerial: Rooftop Payload Drop Robots

Aerial: Ledge Access Robot

Aquatic: Variable Depth Sub Robot

Aquatic: Bottom Crawler Robot

Aquatic: Swift Water Surface Swimmer
Example Deployment Categories for 

Robots

Some commercial products are shown for 

illustration purposes. This does not imply 

endorsement by NIST.

Robot 

Category

Ground: 

Peek Robots

Employment 

Roles(s)

Provide rapid audio 

visual situational 

awareness; provide 

rapid HAZMAT 

detection; data logging 

for subsequent team 

work

Deployment 

Method(s)

Tossed, chucked, 

thrown

pneumatically, w/

surgical tubing; 

marsupially deployed

Tradeoffs

Trade mobility, 

duration, sensing for 

increased expendability

Omnitech

Toughbot
Example Deployment Categories for 

Robots

Robot 

Category

Ground: NonCollapsed 

Structure –Wide 

Area Survey

Employment 

Roles(s)

Long range, human 

access stairway & upper 

floor situational 

awareness; contaminated 

area survey; site 

assessment; victim 

identification; mitigation 

activities; stay behind 

monitoring

Deployment 

Method(s) Backpacked; self driven; 

marsupially deployed

Tradeoffs

Experience form factor for 

increased mobility, 

sensing, manipulation; 

mapping variant; spraying 

variant; breaching variant iRobot Packbot
Example Deployment Categories for 

Robots

Robot 

Category Aerial –Wide Area 

Survey (& Loiter)

Employment 

Roles(s)

Provide overhead 

perspective & sit. Awareness; 

provide HAZMAT plume 

detection; provide comm 

repeater coverage

Deployment 

Method(s) Released; balloon or F/W; 

tethered; launched; VTOL

Tradeoffs

Trade penetration capacity 

for vertical perspective (in 

some cases); trade simplicity 

for greater sit. Awareness.
ASTM E54.08.01 Working Groups

• Terminology 

• Logistics

• Safety and Operating Environment

• Communications 

• Human-System Interaction

• Sensing

• Mobility

• Power (renamed Energy)

Homeland Security Applications, Operational 

Equipment, US&R Robot Performance 

Standards
ASTM E54.08.01 Working Groups

• 6 Work Items introduced; 3 balloted

✴ Visual Acuity and Field of View (E 2566) 

✴ Terminology (E2521-07a)

✴ Logistics, Cache Packaging (E2592-07)

✴ Communications: Line of sight and Non-line of 

sight wireless

✴ Human-System Interaction: Usability

✴ Mobility

• Additional ones in queue

✴ Safety

✴ Power
Example: Visual Acuity & FOV Test Method

Requirements

Illumination Adjustable

Video Real-time remote video system (near)

Video Real-time remote video system (far)

Video Field of View

Video Pan

Video Tilt

Snellen Eye chart correlated to 

Relevant Visual Targets
Example: Wireless Communications Range

Requirements Addressed

Communications Range - Line of Sight

Communications Range - Beyond Line of Sight
Example: Directed Perception Test Method

(eye charts, hazmat labels, thermal, chemical, radiological, explosive)
Cache Packaging

Human Factors

Visual Acuity

Radio Comms

Inclined Plane

Random Stepfields

Confined Space

Stairs

Mobility/Endurance

Remote Situational 

Awareness

Directed Perception

Grasping Dexterity
Common Underlying Artifacts & Measurement Infrastructure

Quad Screen, Time-tagged video capture of tests, including ultra 

wide band tracking of robot position within test areas
Pocket Guide - Per Event

http://www.isd.mel.nist.gov/US&R_Robot_Standards/

• Program Overview

• Event Introduction

• Site Overview

• Safety

• Test Methods and Artifacts

• Participating Robots

- Ground 

- Wall Climbers

- Aerial

- Aquatic

• Sensors

• Index
Response Robot Exercises: Validating 

the Tests; Characterizing the Application 

- Held at FEMA 

US&R Training 

Facilities

• Nevada (8/05)

• Texas (4/06, 6/07, 

11/08)

• Maryland (8/06)

- 23 FEMA Task Forces 

have Participated

- 34 organizations have 

brought 46 robots 

(aerial, ground, 

aquatic)
Response Robot Evaluation Exercise

TX-TF1 Training Facility - Disaster City

College Station, TX

April 4-6, 2006

(with a standards meeting April 7, 2006)

www.isd.mel.nist.gov/us&r_robot_standards

usar.robots@nist.gov

RUBBLE

PILE 2

(#132)

STEEL

STRUCTURE

(#133)

WOOD 

PILE

(#136)

STRIP

MALL

(#131)

THEATER

(#134)

HAZMAT

TRAIN WRECK

(#116)

PASSENGER

TRAIN WRECK

(#126)

RUBBLE

PILE 1

(#128)

HOUSE OF

PANCAKES

(#130)

LAKE

RIG

MAIN STREET

LAKE STREET

1ST STREET

PARKING

CHECK-IN

NIST

ICP

AERIAL

GROUND

BRIEFS

TEES

LUNCH

LOCATION COORDINATES

GATE

[E]

DISASTER CITY

[SHORING, BREACHING, LIFTING]

PORTO POTTY

PORTO POTTY

SUAV AREA OF INTEREST

LAUNCH/RECOVERY

SUAV 

OPERATOR

STATION

250 FT

LAUNCH/RECOVERY

250 FT

DWELLING

(#129)

SUAV FLIGHT PERIMETER

(2000 FT x 2000 FT)
Foster-Miller Talon Finding 

Simulated Victim

Marsupial Deployment of 

Omnitech Toughbot by Talon
Bomb Squads Robots - Updating 

Requirements, Testing the Tests: 

MetroTech Meeting at NIST
Mobile Manipulation: Directed Perception

Example of an Abstract Test Method

Apparatus: Shelves with face/top holes (up to 4 levels) and terrains: flat, roll, stepfields

Method: Identify targets inside holes (visual, CBRNE) with three constraints: open, over, under

Measure: Number targets identified at each level with each constraint, time
Mobile Manipulation: Grasping Dexterity

Example of an Abstract Test Method

Apparatus: Shelves with top holes (up to 4 levels) and terrains: flat, roll, stepfields

Method: Place blocks into holes with three constraints: open, over, under

Measure: Number of blocks placed at each level with each constraint, time
Robotics and associated technologies provide a diverse 

and evolving set of capabilities for emergency response.

To get these advanced tools into the hands of emergency 

responders, we are:

✴ Measuring performance of robots in reproducible, 

repeatable ways that can correlate to use in the field

✴ Developing concepts of operation and match the right 

characteristics to different deployment needs

✴ Moving toward statistically significant repetitions to 

capture performance and reliability 

✴ Standardizing performance test methods through ASTM 

International

Summary
Summary

Robotics and associated technologies provide a diverse 

and evolving set of capabilities for emergency response.

To get these advanced tools into the hands of 

responders, we are:

✴ Measuring performance of robots in reproducible, 

repeatable ways that can correlate to use in the field

✴ Developing concepts of operation and match the right 

characteristics to different deployment needs

✴ Moving toward statistically significant repetitions to 

capture performance and reliability 

✴ Standardizing performance test methods through ASTM 

International
Acknowledgments

• The FEMA USAR Task Force members, representing 23 Task Forces, who ably advise us 

throughout the process

• The Metrotech and Michigan bomb squads, the National Bomb Squad Commander’s 

Advisory Board, the Hazardous Devices School instructors, TSWG, and others who have 

defined the requirements for bomb disposal robots

• All the manufacturers and researchers who have voluntarily participated in the exercises 

and subjected their robots to testing

• The Department of Homeland Security, Science and Technology Directorate, Standards 

Office

• The National Institute of Justice Office of Science and Technology 

• The Standards Working Groups and Support Team (at NIST and elsewhere) including, but 

not limited to: Adam Jacoff, Brian Antonishek, Stephen Balakirsky, Tony Downs, John 

Evans, Hui-Min Huang, Galen Koepke, Alan Lytle, Philip Mattson, Bill McBride, Mark Micire, 

Kate Remley, Debra Russell, Jeanenne Salvermoser, Salvatore Schipani, Craig Schlenoff, Jean 

Scholtz, Chris Scrapper, Ann Virts, and Brian Weiss

This work would not be possible without the contributions of:
Thank You!

For more information about 

Performance Standards

for 

Response Robots:

http://www.isd.mel.nist.gov/US&R_Robot_Standards/

usar.robots@nist.gov