U.S. flag
        An official website of the United States government
          Here’s how you know
        Here’s how you know**Official websites use .gov**
              
              A **.gov** website belongs to an official government organization in the United States.
        **Secure .gov websites use HTTPS**
              
              A **lock** (  A locked padlock
) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites. https://www.nist.gov/el/intelligent-systems-division-73500/standard-test-methods-response-robots/events-and-meetings-8
  Search NIST
    Search
              Menu

      


Close
    Topics
        All Topics
                                                        Advanced communications
                                                        Artificial intelligence
                                                        Bioscience
                                                        Buildings and construction
                                                        Chemistry
                                                        Climate
                                                        Cybersecurity
                                                        Electronics
                                                        EnergyEnvironment
                                                        Fire
                                                        Forensic science
                                                        Health
                                                        Information technology
                                                        Infrastructure
                                                        Manufacturing
                                                        Materials
                                                        Mathematics and statisticsMetrology
                                                        Nanotechnology
                                                        Neutron research
                                                        Performance excellence
                                                        Physics
                                                        Public safety
                                                        Resilience
                                                        Standards
                                                        Transportation
            Publications
            Labs & Major Programs
        Laboratories
                                Communications Technology Laboratory
                                  Engineering Laboratory
                                  Information Technology Laboratory
                                  Material Measurement Laboratory
                                  Physical Measurement LaboratoryUser Facilities
                                NIST Center for Neutron Research
                                  CNST NanoFab
                                                        Research Test Beds
                                                        Research Projects
                                                        Tools & InstrumentsMajor Programs
                                Baldrige Performance Excellence Program
                                  CHIPS for America Initiative
                                  Manufacturing Extension Partnership (MEP)
                                  Office of Advanced Manufacturing
                                  Special Programs Office
                                  Technology Partnerships Office
            Services & Resources
        Standards and Measurements
                                Calibration Services
                                  Laboratory Accreditation (NVLAP)
                                  Quality System
                                  Standard Reference Materials (SRMs)
                                  Standards.gov
                                  Time Services
                                  Office of Weights and MeasuresSoftware
                                                        Data
                                Chemistry WebBook
                                  National Vulnerability Database
                                  Physical Reference Data
                                  Standard Reference Data (SRD)
                                                        Storefront
                                                        License & PatentsComputer Security Resource Center (CSRC)
                                                        NIST Research Library
            News & Events
        News
                                                        Events
                                                        Blogs
                                                        Feature Stories
                                                        AwardsVideo Gallery
                                                        Image Gallery
                                                        Media Contacts
            About NIST
        About Us
                                Leadership
                                  Organization Structure
                                  Budget & Planning
                                                        Contact Us
                                                        Visit
                                                        Careers
                                Student programsWork with NIST
                                                        History
                                NIST Digital Archives
                                  NIST Museum
                                  NIST and the Nobel
                                                        Educational Resources
 
 Engineering Laboratory
      Intelligent Systems Division
    
      
  Standard Test Methods for Response Robots
                            Expand or Collapse
          NIST's Robotics Test Facility
                      Collaborating Facilities
                      Ground Systems
                      Aerial Systems
                                              Expand or Collapse
                Aerial Program Overview
                                  Open Test Lanes and Scenarios
                                  Obstructed Test Lane and Scenarios
                                  Confined Test Lane and Scenarios
                                  Sensor Test Lane
                                  Simulation
                                  Forms
                                  Frequently Asked Questions
                      Aquatic Systems
                      Robot Competitions
                                              Expand or Collapse
                RoboCup Rescue
                                  Aerial Competitions
                      Events and Meetings
                                              Expand or Collapse
                2023.01 ASTM E54.09 Meeting San Antonio, Texas
                                  2022.06 ASTM E54.09 Virtual Meeting
                                  2022.01 ASTM E54.09 Virtual Meeting
                                  2021.10 UTAC Conference, Guardian Centers, Perry, GA
                                  2021.06 ASTM E54.09 Virtual Meeting
                                  2021.02 ASTM E54.09 Virtual Meeting
                                  2018.06 ASTM E54.09 San Diego, CA
                                  2018.02 ASTM E54.09, Gaithersburg, MD
                                  2017.06 ASTM E54.09, Toronto, Canada
                                  2017.01 ASTM E54.09/Virginia Beach Exercise, Virginia Beach, VA
                                  Prior Events and Meetings2017.01 ASTM E54.09/Virginia Beach Exercise, Virginia Beach, VA

Share
  Facebook
                      Linkedin
                      Twitter
                      Email

**2017 - January 30 - February 3**

**ASTM E54.09 Standard Committee Meeting**

**Virginia Beach Fire Training Center**

FEMA US&R VA-TF2Special Operations Training Facility

927 S Birdneck Rd

Virginia Beach, VA  23451

**Invitation**
	**Overview of Meeting and Exercise**
	**Registration**
	**Agenda**
	**Venue**

	**Directions to the Venue**
	**Dress Code**
	**Robot Developers**

	**Ground Systems**
		**Aerial Systems**
		**Aquatic Systems**
		**Additional Information**
		**Shipping Address**
	**Test Administrators**
	**Hotel (NEW)**
	**Questions/Contacts****Invitation**

We invite you to participate in a meeting and associated test method validation exercise with theASTM International Standards Committee on Homeland Security Applications; Response Robots (E54.09).This is our same committee elevated one level to that of a sub-committee due to recognition of our overall productivity.  In attendance will be members of the newDHS FEMA Urban Search and Rescue (US&R) Robotic Standards Group.This will be the first meeting of this group, which is tasked with identifying test methods to evaluate robots for purchase for FEMA US&R task forces. Robot manufacturers will be bringing their robots to demonstrate capabilities within the test methods. Test administrators from around the world will be there as well to help conduct the testing and collaborate on new test methods.



Key discussion topics will include:

50 test methods for ground, aerial, and aquaticroboticsystems
	New collaborating test facilities worldwide
	Test method standardizationballoting status and what's next
	Response Robots Capabilities Compendium(filtering test results to suit your needs)
	Trainingwithin the test methodsas standard measures of operator proficiency



The associated test method evaluation exercise will be set up to demonstrate almost alltest methods under development.  We will validate and refine the test methods with participants by conducting data collection trials anddemonstratingtraining with measures of operator proficiency.



Thisexercise will include all the test methods in a single indoor facility, along with outdoor US&R scenarios embedded with key test methods.The test methods will be organized asseveral concurrent stations, including:

Configuration Identification photo booth and robot dispatch board to show where and when each robot is testing in whichtest method.
	30 Ground system test methods for mobility, dexterity, sensors, etc. In three different scales:

	120cm (48in) wide lanes forming aBasic Maneuvering and Dexterity Skills Coursefor Bomb Squad Robot Operatorsthat will be used by local bomb squads on their own robots.
		60cm (24in) wide lanesto evaluate a variety of issues inside dwellings, buses, trains, between parked cars, etc. These test methods have optional confined space "ant farm"entries for throwable robots.
		30cm (12in) wide lanes for throwable robots working in confined spaces.  Also for new initiatives into truly disposable,rapidly manufactured,and 3D printed robot kits.
		5 Sensors test methods for visual acuity, color acuity, latency, dynamic range, and speech intelligibility.
		2 Radio comms attenuation and interference test methods in a 40ft ISO container.
		Several other test methods for Sensors, Situational Awareness, Endurance.
	10 Aerial test methods for maneuvering, inspecting, perching, etc. in a netted aviary.
	15 Aquatic test methods for a range of underwater C-IED tasks in a rented frac tank (50x8x8ft; 18,000gal).

The FEMA US&R Robotics Standards Group is tasked with reviewing all available test methods for ground, aerial, and aquatic systems that will evaluaterobots to be purchased for US&R equipment caches nationwide. We will help the Groupthrough the following:

Select which test methods will be used to evaluate and specify robots for various missions.
	Review existing robot capabilities demonstrations in standard test methods.
	Use existing robot purchasesthat used data from the test methods to specify robot capabilities asa model for related roles in US&R missions.
	Review the entire Response Robot Capability Compendium and filtering interface to learn aboutsetting thresholds to best reflect particular mission requirements.

Local bomb squads will be using their own robots to perform theBasic Maneuvering and Dexterity Skills Coursefor Bomb Squad Robot Operatorsthat was used in 3 regional Robot Rodeos in 2016, enabling direct comparison of their own proficiency to regional and national averages.





 
      Test Methods for ground robots
  
 

 
      Test Methods for aerial robots

 
      Aquatic  Apparatus with underwater robot

[Top]



**Overview of Meeting and Exercise**

The objective of this meeting of the ASTM International Standards Committee on Homeland Security Applications, Sub-Committee on Response Robots (E54.09) is to provide an update on each working group’s effort with an emphasis on getting the existing test methods to ballot. Draft standards and prototype apparatus designs will be reviewed and exercised by more than 20 robots and operators provided by manufacturers and others.  Topics will include recent test method validation exercises, robot purchases,  competitions, training events, and new collaborating test facilities around the world.

Each working group will review and validate their suite of test apparatuses, address comments to draft standards, and discuss the balloting status.  As always, we will be briefing the entire E54 standards committee on our progress, but the real work at these events gets done during the focused working group meetings conducted throughout the week as noted in the agenda below.  A large contingent of robot manufacturers, test administrators, emergency responders and other users will be there to contribute.
 
This meeting and exercise will be conducted concurrently from Monday through Friday with ongoing robot testing and focused meetings each day discussing the details of each  test method suite. There will be  ample time to watch the test methods and robots in action. We will try to enable people with different interests and familiarity with our process to ramp up and lean into their chosen test suites.
 
Focused meetings on each test suite will be conducted in the auditorium or conference rooms depending on the number of attendees. These will be scheduled according to the agenda below. GROUND robot test methods will be exercised and discussed every day.  AERIAL and AQUATIC system test methods will be exercised every day but the focused meetings will be on different days.
 
Robot testing with "expert" operators provided by the manufactures will be conducted constantly within the test facility for ground, aerial, and aquatic systems. Robot capabilities data will be captured to support the standardization process and to set the 100% percentile of operator proficiency that others can use to measure and compare their own proficiency.
 
Lunches will be served in the test facility each day so we can eat and watch the concurrent test stations in action.

[Top]



****Registration

**Online Pre-Registration is REQUIRED to participate in the ASTM International Standards Committee on Homeland Security Applications; Response Robots (E54.09) Meeting and Associated Test Method Validation Exercise:**

Register for the upcoming meeting and exercise online. It is FREE if you choose not to pay the VOLUNTARY ACTIVITY FEE on the registration page.  When the dialogue box asks again you can simply PROCEED WITH PRE-REGISTERING.  Of course, you may contribute to ASTM if you wish. You do not need to join ASTM to attend the meeting and exercise, it is open to the public. Registration closes Wednesday, Jan. 25th:

**http://tinyurl.com/ASTME54-9-Reg**



Our new ASTM International Standards Committee on Homeland Security Applications; Response Robots (E54.09) is official.But we have no current voting members!



Join or transfer to our new sub-committee on Response Robots (E54.09) here:

https://www.astm.org/MEMBERSHIP/index.html



If you were a member of the Operational Equipment (E54.08) sub-committee, you are not automatically a member of the new Response Robots (E54.09) sub-committee.  So please go online and either shift or add your affiliation to our new sub-committee.  You do not need to give up your position on any other sub-committee if your interests remain.  Prior to the January 30th meeting, we will need to transfer all members and then go through a process to ensure the balance remains equal between manufacturers, users, test administrators, etc.  But we cannot even call the committee into order unless we have a quorum of members signed up, and at least 10% of them on site.



We will also need to go through a process of transferring over the pertinent standards and work items currently residing within Operational Equipment.  So when you look up Response Robots (09) on the website don’t think that we’re starting from a blank slate.  That should be accomplished by the meeting as well.

[Top]

Agenda ****

**Monday, January 30, 2017**

**0900 FINAL PREPARATIONS**

Travel for attendees
	Test administrator training
	Robot practice in test methods1200 Lunch served on site, robots continue testing so everybody can watch.**1300 MEETING STARTS  [Conference Room #_______]**

Introductions around the room
	Collaborating test facilities worldwide, their particular use cases, and other standard test method initiatives and adoption stories
	Robot purchases recently specified within our suite of standard test methods
	Response Robot Capabilities Compendium (v2015-16) overview and filtering interface, bar charts, etc.
	Robot competitions using our standard test methods as challenge tasks to guide innovation and measure progress in emerging capabilities and autonomy.
	Responder and military training events using our standard test methods to focus training and measure operator proficiency.**1800 DONE**
1900 After action meeting locations will be posted each day.



_____________________________________________________________

**Tuesday, January 31, 2017**
Robots continue operating in dozens of their chosen standard test methods attempting to establish statistically significant performance for responders to consider.
 **0830 SAFETY BRIEFING** for all the day's attendees**0900 GROUND SYSTEMS**

Configuration Identification and Sensors
	Maneuvering (basic training)1200 Lunch served on site, robots continue testing so everybody can watch.**1300 GROUND SYSTEMS**

Mobility
	1500 E54.09 Committee Business
	Dexterity**1800 DONE**
1900 After action meeting locations will be posted each day.

_____________________________________________________________

**Wednesday, February 1, 2017**
Robots continue operating in dozens of their chosen standard test methods attempting to establish statistically significant performance for responders to consider.**0830 SAFETY BRIEFING for all the day's attendees
0900 GROUND SYSTEMS**

Energy/Power
	Safety, Durability**1100 E54 Main Committee Business**
1200 Lunch served on site, robots continue testing so everybody can watch.**1300 AERIAL  SYSTEMS**

Impact Forces (video only)
	Sensors, Agilities, Maneuvering
	Inspect, Avoid, Land/Perch**1800 DONE**
1900 After action meeting locations will be posted each day.

_____________________________________________________________

**Thursday, February 2, 2017**
Robots continue operating in standard test methods. Systems demonstrated to be reliable in the test methods can be selected by responders to perform mock deployments into the scenarios.**0830 SAFETY BRIEFING**for all the day's attendees**0900 GROUND SYSTEMS**

Situational Awareness and Mapping
	Radio Comms (Structure Penetration Attenuation, Interference)1200 Lunch served on site, robots continue testing so everybody can watch.**1300 AQUATIC SYSTEMS**

Bollard Thrust, Sensors, Sonar Resolution
	Floating: Inspect, Grasp; Hook
	Bottom: Grasp-Relocate-Place
	Oriented: Inspect; Grasp; Hook; Aim Disruptors**1800 DONE**
1900 After action meeting locations will be posted each day.

_____________________________________________________________

**Friday, February 3, 2017**
Robots continue operating in standard test methods. Systems demonstrated to be reliable in the test methods can be selected by responders to perform mock deployments into the scenarios.
 **0830** **SAFETY BRIEFING** for all the day's attendees**0900** **CONTINUE ROBOT TESTING**

Capture capabilities, proficiency, and training data
	Responders deploy chosen robots into scenarios
	1200 Lunch served on site, robots continue testing so everybody can watch.
	1300 BREAKOUT MEETINGS
	US&R Robotics Standards Group
	RoboCupRescue Rules Committee
	Japan 2020 Initiative
	Potential new facilities**1600 DONE**
1700 After action meeting locations will be posted each day.
(Travel in the evening or celebrate with us and travel the next day!)
 
Note: *Veggie options will be available for lunches hosted at the test facility.

Note: Agenda V2

[Top]

**Venue**

**Virginia Beach Fire Training Center**

**VA-TF2 Special Operations Training Facility**

**927 S Birdneck Rd**

**Virginia Beach, VA  23451**

**There will be several concurrent standard test method stations set up:**

Configuration Identification photo booth and robot dispatch board to show where and when each robot is testing in which apparatus.
	10 Ground maneuvering and mobility test methods in an array of 60cm (24in) wide lanes for operations inside dwellings, buses, trains, between parked cars, etc.
	10 Aerial test methods for maneuvering, inspecting, etc. in a netted aviary.
	15 Aquatic test methods for a range of C-IED tasks in a rented frac tank (50x8x8ft; 18,000gal).
	2 Radio comms attenuation and interference test methods in a 40ft ISO container.
	10 basic maneuvering and dexterity test methods in 120cm (48in) wide lanes forming a sequence called the Standard Basic Skills Course for Bomb Squad Robot Operators.  It's a 30 minute time limited trial to measure proficiency of local civilian and military bomb technicians using their own robots.  This has been replicated at three regional Robot Rodeos in 2016 enabling direct comparison against regional and national averages.





 
      Training Center Overview

 
      VABeach facility

Some scenarios on site will be available for mock robot deployments with responders. This is only for robots that excel in the standard test methods.  These scenarios will be outfitted with the embeddable set of standard test methods to demonstrate how to capture quantitative measures  within the scenario's environmental complexity.

For example, the Burn Building scenario and Rubble Pile scenario will be outfitted with combined target panels to search for and identify. They include tests for Video Image Acuity (human and machine readable), Hazmat Label  Identification (color and symbols), Thermal Image Acuity, Motion Detection, Audio Acuity, and Timestamps for screen captures from the remote operator station.

These combined targets come in near and far field sizes to embed in different situations. They can be in open areas or confined spaces for ground systems. Or they can be viewed through windows or from defined altitudes by aerial systems. When the viewing distances are known such as from a defined doorway,  line on floor, or altitude, these can provide strict measures. Or they can simply be known targets to find and identify as indicators of performance prior to more operationally significant tasks nearby.

Other examples of easily embeddable test methods include a variety of dexterity tests, terrain elements, mapping fiducials, etc.



 
      Overview of VABeach facility[Top]

Directions to the Venue



 
      Driving Directions[Top]

**Dress Code**

This event will consist of equal parts meetings and test method evaluations with robots. Some additional mock deployments into US&R training scenarios will also be conducted later in the week.

Responders please wear your insignias so people can know where you're coming from throughout the US.



Some personal protective equipment (PPE) will be required to take part in the robot testing at the Virginia Beach Fire Training Center. Minimal PPE consists of the following:

Long pants
	Long sleeve shirt
	Closed toe shoes

For those deploying robots into training scenarios, additional PPE will be required.

Helmets (available to borrow on site)
	Boots
	Gloves
	Eye protection[Top]

**Robot Developers**

**Participating Robots:**



**Ground Systems**

**Crystal Cam**

	Delta Extreme(14 lbs / 6.2 kg)
	**Endeavor Robotics**

	FirstLook 110 (5.2 lbs/2.4 lbs)
		310 SUGV (30 lbs/13.6 kg)
	**ICOR Technology**

	Mini-CALIBER ( 64 lbs/29 kg)
		Caliber T-5 (150 lbs/68 kg)
	**InCommand Robotics, LLC**

	Veratrax 150(88 lbs /40kg)
		VT100 MicroMag
	**Remotec**
	HD-2 (245 lbs/111kg)
	**Roboteam, Inc**
	IRIS (3.6lbs/1.6kg)
		MTGR (28.6 lbs/13 kg)
	**Super Droid**
	MLT "Jack Russell" (< 9lbs/<4kgs)
		LT2 / LT2-F - "Bloodhound" with arm
	**TeleRob**
	TeleMax (175lbs/79Kg)_____________________________________________________________

**Aerial Systems**

**AeroViroment**

	Qube (5.5 lbs/2.5 kg )
	**DJI**
	Mavic (1.64 lbs /743 g)
		Phantom Professional 3  (2.8 lbs /1280 g)
		Inspire 1   (6.74 lbs /3060 g)
	**Dragonfly**
	Dragonflyer
	**Parrot**
	BEEBOP II(1.1lbs/500 g)_____________________________________________________________

**Aquatic Systems**

**DeepTrekker ROV**

	DTG2(18.7lbs/8.5kg)
		DTX2(58lbs/26kg)
	**OpenROV**
	**Strategic Robotic Systems**
	Fusion(55lbs/25kg )
	**TeleDyne Marine - Seabotix**
	LVB300 (28.7 lbs/13 kg)
	**VideoRay**
	VideoRay4  (85lbs/38.5)_____________________________________________________________

**Developers please bring:**

Tethers to avoid radio conflicts.

Spare batteries and charger to conduct nearly continuous operations.

Spare parts and tools.

We will provide a table, chairs and power.

You may bring your pop up marketing backdrops and handouts.

Shippers MUST deliver with drop tailgates. There is no loading dock or forklift available for unloading.

_____________________________________________________________

**Shipping Address:**

Attn: Rex Gurley / Jim Ingledue 757-288-2867
NIST Robot Exercise
VA US&R Task Force 2 Warehouse
513 Viking Drive, Suite 9
Virginia Beach, VA 23452-7322

[Top]

**Test Administrators**

**SET UP FOR TEST ADMINISTRATORS**
Fabrication of test apparatuses, the aerial aviary, and water tank in and around the building will be the priority two weeks prior to the event. Then embedding standard props into on-site training scenarios.

Tuesday-Thursday, January 17-19, 2017 (Monday is a holiday, Friday is inauguration)Final test method and attendee preparations will be set up the weekend prior to the event:

Saturday-Sunday, January 28-29, 2017[Top]

Hotel Information

Hilton Garden Inn Virginia Beach Oceanfront
3315 Atlantic Avenue
Virginia Beach, VA 23451

Reservations: 757-305-9000http://www.hiltongardeninnvirginiabeach.com





[Top]

Questions/Contacts

In addition to ASTM staff, you can visit RobotTestMethods.nist.gov  and/or email RobotTestMethod [at] nist.gov(RobotTestMethod[at]nist[dot]gov).

[Top]

**Contact Information:**

Adam Jacoff - Project LeaderRobotTestMethods [at] nist.gov(RobotTestMethods[at]nist[dot]gov(link sends e-mail))Adam.Jacoff [at] nist.gov(Adam[dot]Jacoff[at]nist[dot]gov(link sends e-mail))
301-975-4235
  Resilience

    
    Created November 8, 2016, Updated February 25, 2022 ‹ 2017.06 ASTM E54.09, Toronto, Canada
              Prior Events and Meetings ›
 HEADQUARTERS
            100 Bureau Drive
              Gaithersburg, MD 20899301-975-2000
            Webmaster | Contact Us | Our Other Offices
        Twitter
            
Facebook
            
LinkedIn
            
Instagram
            
YouTube
            
Giphy
            
RSS Feed
            
Mailing List
                              How are we doing? Feedback

  
  Site Privacy
                      Accessibility
                      Privacy Program
                      Copyrights
                      Vulnerability Disclosure
                      No Fear Act Policy
                      FOIA
                      Environmental Policy
                      Scientific Integrity
                      Information Quality Standards
                      Commerce.gov
                      Science.gov
                      USA.gov
                      Vote.gov