U.S. flag
        An official website of the United States government
          Here’s how you know
        Here’s how you know**Official websites use .gov**
              
              A **.gov** website belongs to an official government organization in the United States.
        **Secure .gov websites use HTTPS**
              
              A **lock** (  A locked padlock
) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites. https://www.nist.gov/el/intelligent-systems-division-73500/standard-test-methods-response-robots/events-and-meetings-7
  Search NIST
    Search
              Menu

      


Close
    Topics
        All Topics
                                                        Advanced communications
                                                        Artificial intelligence
                                                        Bioscience
                                                        Buildings and construction
                                                        Chemistry
                                                        Climate
                                                        Cybersecurity
                                                        Electronics
                                                        EnergyEnvironment
                                                        Fire
                                                        Forensic science
                                                        Health
                                                        Information technology
                                                        Infrastructure
                                                        Manufacturing
                                                        Materials
                                                        Mathematics and statisticsMetrology
                                                        Nanotechnology
                                                        Neutron research
                                                        Performance excellence
                                                        Physics
                                                        Public safety
                                                        Resilience
                                                        Standards
                                                        Transportation
            Publications
            Labs & Major Programs
        Laboratories
                                Communications Technology Laboratory
                                  Engineering Laboratory
                                  Information Technology Laboratory
                                  Material Measurement Laboratory
                                  Physical Measurement LaboratoryUser Facilities
                                NIST Center for Neutron Research
                                  CNST NanoFab
                                                        Research Test Beds
                                                        Research Projects
                                                        Tools & InstrumentsMajor Programs
                                Baldrige Performance Excellence Program
                                  CHIPS for America Initiative
                                  Manufacturing Extension Partnership (MEP)
                                  Office of Advanced Manufacturing
                                  Special Programs Office
                                  Technology Partnerships Office
            Services & Resources
        Standards and Measurements
                                Calibration Services
                                  Laboratory Accreditation (NVLAP)
                                  Quality System
                                  Standard Reference Materials (SRMs)
                                  Standards.gov
                                  Time Services
                                  Office of Weights and MeasuresSoftware
                                                        Data
                                Chemistry WebBook
                                  National Vulnerability Database
                                  Physical Reference Data
                                  Standard Reference Data (SRD)
                                                        Storefront
                                                        License & PatentsComputer Security Resource Center (CSRC)
                                                        NIST Research Library
            News & Events
        News
                                                        Events
                                                        Blogs
                                                        Feature Stories
                                                        AwardsVideo Gallery
                                                        Image Gallery
                                                        Media Contacts
            About NIST
        About Us
                                Leadership
                                  Organization Structure
                                  Budget & Planning
                                                        Contact Us
                                                        Visit
                                                        Careers
                                Student programsWork with NIST
                                                        History
                                NIST Digital Archives
                                  NIST Museum
                                  NIST and the Nobel
                                                        Educational Resources
 
 Engineering Laboratory
      Intelligent Systems Division
    
      
  Standard Test Methods for Response Robots
                            Expand or Collapse
          NIST's Robotics Test Facility
                      Collaborating Facilities
                      Ground Systems
                      Aerial Systems
                                              Expand or Collapse
                Aerial Program Overview
                                  Open Test Lanes and Scenarios
                                  Obstructed Test Lane and Scenarios
                                  Confined Test Lane and Scenarios
                                  Sensor Test Lane
                                  Simulation
                                  Forms
                                  Frequently Asked Questions
                      Aquatic Systems
                      Robot Competitions
                                              Expand or Collapse
                RoboCup Rescue
                                  Aerial Competitions
                      Events and Meetings
                                              Expand or Collapse
                2023.01 ASTM E54.09 Meeting San Antonio, Texas
                                  2022.06 ASTM E54.09 Virtual Meeting
                                  2022.01 ASTM E54.09 Virtual Meeting
                                  2021.10 UTAC Conference, Guardian Centers, Perry, GA
                                  2021.06 ASTM E54.09 Virtual Meeting
                                  2021.02 ASTM E54.09 Virtual Meeting
                                  2018.06 ASTM E54.09 San Diego, CA
                                  2018.02 ASTM E54.09, Gaithersburg, MD
                                  2017.06 ASTM E54.09, Toronto, Canada
                                  2017.01 ASTM E54.09/Virginia Beach Exercise, Virginia Beach, VA
                                  Prior Events and Meetings2017.06 ASTM E54.09, Toronto, Canada

Share
  Facebook
                      Linkedin
                      Twitter
                      Email

**2017 - June 12-14**

**ASTM E54.09 Standard Committee Meeting**

**Hamilton Port Authority; Toronto, Canada**



Committee Overview
	Media Highlights
	Meeting Outcomes (NEW)
	Invitation
	Registration
	**Agenda**
	**Venue**
	**Dress Code**
	**Hotel**
	**Questions/Contacts**Committee Overview

Over the past few years our suite of 30 ground robot test methods have been adopted widely across the U.S. and internationally to support purchasing and training. We are now expanding our efforts to develop test methods for small unmanned aircraft systems (sUAS) defined by the FAA as under 25kg (55lbs) and for underwater remotely operated vehicles for hazardous environments.

Remotely operated robots enable emergency responders to perform extremely hazardous tasks from safer stand-off distances. Standard test methods help robot manufacturers and users objectively evaluate system capabilities to align with mission requirements. This improves the safety and effectiveness of emergency responders as they attempt to save lives and protect property in our communities.

The ASTM International Standards Committee on Homeland Security Applications: Response Robots (E54.09) is developing the standards infrastructure necessary to inspire innovation, inform purchasing decisions, and focus training with measures of operator proficiency. Adam Jacoff from NIST’s Intelligent Systems Division chairs the sub-committee with his team of robotics researchers and collaborators from all over the world. The standards committee is made up of robot manufacturers, emergency responders, test administrators, program managers, and others interested in developing the underlying measurement science necessary to quantitatively evaluate and compare robotic system capabilities and operator proficiencies.

This effort has been sponsored by the Department of Homeland Security, Science & Technology Directorate, Capability Development Support Group, Office of Standards. They along with other sponsors have facilitated these standards committee meetings and associated test method validation exercises for nearly fifty test methods for ground, aerial, and aquatic systems.

[Top]

Meeting Outcomes

**Summary of****Standards Committee****Meeting and Validation Exercise hosted in****Hamilton, Ontario**

This meeting and exercise was hosted at the Hamilton Port Authority, Ontario, Canada. More than twenty robots provided by manufacturers, emergency responders, and military organizations evaluated their robotic capabilities and operator proficiencies within the test methods. Test administrators from collaborating facilities around the world conducted trials to learn about and validate the test methods. They also helped devise new tests to address emerging requirements articulated by responders at the meeting.





 
      Hamilton Port Authority Building
A) Front view of the Hamilton Port Authority boat storage facility used to host the event. It was mostly empty for the summer. B) Back view of the same facility. They also provided dock space in the Lake Ontario to embed our underwater test methods to conduct trials in the opaque water more typically associated with response operations.
  
 





 
      Ground Robot Test Methods
The apparatuses set up to the left are the Basic Maneuvering and Dexterity suite of test methods. They are performed sequentially in time limited trials. B) The Access Buildings suite of test methods includes Terrains, Stoops, Stairs, Doors, and Hallway Labyrinths all with embedded Dexterity and Mapping tasks.



 
      Ground Robot Test Methods
The main service robot used by these regional bomb squads is manufactured in Canada. A) It is shown performing the Dexterity: Rotate task using the parallel apparatus at door knob elevation. B) Here it is performing the Dexterity: Inspect task using the omni-directional apparatus.



 
      Standard Test Methods for Aquatic
The C-frame apparatuses provide five different orientations for each task (0°-45°-90°-135°-180°). B) The walkways on top of the tank appear helpful, but they actually hindered inserting the large apparatus frames into the tank. So open top tanks are preferred.  C) The same test methods were replicated off the dock in the brackish water with natural currents and sediment in the harbor. We embed the apparatuses into these operationally relevant scenarios to measure the degradation in performance due to the uncontrolled environmental variables.



 
      Standard Test Methods for Aerial
The indoor netted aviaries provided safe flight testing while validating the test methods and collecting essential data for the standardization process. B) A wall of fans provided turbulence for the aerial systems, encouraging automatic holding of position and orientation to improve performance of tasks.



 
      Standard Test Methods for Aerial
The outdoor testing took place in a more remote section of Ontario where the local responders are cleared to fly their systems outdoors. B) We set up our Situational Awareness test methods to practice and validate them using an aerial system manufactured in Canada and used by all the responders. C) Because aerial systems need to handle weather conditions, we adapted a sprinkler system to the Spiral Inspect test apparatus to enable testing resilience of the system to rain

To download the complete report:

dhs-nist-astm_e54.09_response_robots_meeting_and_exercise_toronto_canada.pdf

[Top]

Media Highlights



News interview with President of the Canadian Explosives Technicians Association

https://www.thespec.com/videopopup/7369896?popUp=true

Local news

http://www.chch.com/bomb-disposal-robots/

Other media

https://motherboard.vice.com/en_us/article/these-robots-will-explore-tunnels-dug-by-looters-in-egypt

[Top]

**Invitation**



The *ASTM International Standards Committee on Homeland Security Applications; Response Robots (E54.09)*will be hosting a meeting and associated response robot exercise to validate test methods for remotely operated and autonomous ground, aerial, and aquatic systems. The resulting standard test methods can be used by anybody to measure and compare system capabilities and operator proficiency.

Key discussion topics will include:

Fifty test methods including apparatuses, procedures, and performance metrics
	Collaborating test facilities worldwide that host the standards
	Test method balloting status and priorities going forward
	Response Robots Capabilities Compendium allowing filtering of test results to address mission requirements
	Training within the test methods as standard measures of operator proficiencyThe associated test method validation exercise will involve most of the test methods under development. International test administrators from collaborating facilities will help set up and administer the test methods.  We will validate and refine the test methods with regional emergency responders and manufacturers using their own systems to practice the tests. Manufacturers will measure baseline system capabilities. Regional responders will focus their training and measure their proficiency in head-to-head trials. They will be using their own robots to perform a sequence of standard tasks used during four other regional Robot Rodeos with more than one hundred bomb technicians throughout the U.S. in 2016-2017.  This enables direct comparison of their proficiency to regional and national averages.

The test methods will include:

**15 ground system test methods** focusing on Maneuvering, Mobility, Dexterity, Tool Deployment, Sensing, Endurance, and Radio Comms necessary to conduct missions involving improvised explosive devices, urban search and rescue, accessing and mapping buildings, and others.
	**15 small unmanned aircraft system (sUAS) test methods** for  Maneuvering, Situational Awareness, Sensing, Endurance, and Mapping necessary to conduct missions involving search and rescue, infrastructure inspection, accident reconstruction, and others.
	**15 Aquatic system test methods** focusing on maneuvering, dexterity, tool deployment, and other capabilities necessary to conduct missions involving underwater improvised explosive devices.

[Top]

Registration

Online Pre-Registration is REQUIRED to participate in the ASTM International Standards Committee on Homeland Security Applications; Response Robots (E54.09) Meeting and Associated Test Method Validation Exercise: 
 
Register for the upcoming meeting and exercise online. It is FREE if you choose not to pay the VOLUNTARY ACTIVITY FEE on the registration page.  When the dialogue box asks again you can simply PROCEED WITH PRE-REGISTERING.  Of course, you may contribute to ASTM if you wish. You do not need to join ASTM to attend the meeting and exercise, it is open to the public.

http://tinyurl.com/ASTME54-Reg-June2017
 
Our new ASTM International Standards Committee on Homeland Security Applications; Response Robots (E54.09) is official.
 
Join or transfer to our new sub-committee on Response Robots (E54.09) here:https://www.astm.org/MEMBERSHIP/index.html
 
If you were a member of the Operational Equipment (E54.08) sub-committee, you are not automatically a member of the new Response Robots (E54.09) sub-committee. So please go online and either shift or add your affiliation to our new sub-committee. You do not need to give up your position on any other sub-committee if your interests remain.



[Top]

Agenda ****

**Monday, June 12 2017**

Robots begin operating in ground, aerial, and aquatic test methods either for practice or to capture proctored test method validation trials. Regional responder VIPs meet and tour prior to meeting.

0900 **Regional Responder VIP Meeting and Tour**

Travel for attendees
	Test administrator training
	Robot practice in test methods1200 Lunch served on site, robots continue testing so everybody can watch.

1300 **E54.09 Committee Business**

Introductions around the room
	Collaborating lest facilities worldwide, particular use cases, and other standard test method initiatives.
	Robot purchases recently specified within our suite of standard test methods
	Response Robot Capabilities Compendium (v2015-16) overview and filtering interface, bar charts, etc.
	Robot competitions using our test methods as challenge tasks to guide innovation and measure progress in emerging capabilities and autonomy.
	Responder and military training events using our standard test methods to focus training and measure operator proficiency.1600 **Tour Test Method Exercise**

1800 **Done**.

_____________________________________________________________

**Tuesday, June 13 2017**

Robots continue operating  ground, aerial, and aquatic test methods either for practice or to capture proctored test method validation trials.

0830 **SAFETY BRIEFING** for all the day's attendees

0900 **GROUND TEST METHOD MEETING*(Ground system testing will be continuous all day every day)***

Configuration Identification and Sensors
	Maneuvering (basic training)
	Radio Comms
	Mission Combo Sequences: Building Access1200 Lunch served on site, robots continue testing so everybody can watch.

1300 **AERIAL TEST METHOD MEETING*(Aerial system testing will be continuous all day every day)***

Impact Forces (video only)
	Maneuvering Suite
	Situational Awareness Suite
	Outline of next day's outside aerial testing (off-site)1800 **Done**.

_____________________________________________________________

**Wednesday, June 14 2017**

Robots continue operating in dozens of chosen standard test methods attempting to establish statistically significant performance for responders to consider. (Note: 0800-1300 Adam is at Exec Committee meeting in Toronto.)

0830 **SAFETY BRIEFING** for all the day's attendees

0900 **AQUATIC TEST METHOD MEETING**

***(Aquatic system testing will be continuous all day every day)***

C-IED suite1200 Lunch served on site, robots continue testing so everybody can watch.

1300 **AERIAL  SYSTEM TESTIN OUTDOORS (OFF SITE)**

Relocate and set up test apparatuses
	Situational Awareness Suite
	Mapping Test Method (stitched images containing 10-30 objects of interest)1800 **Done**.

[Top]

**Venue**

**Hamilton Port Authority**





Harbor West Marina





210 Hillyard Street





Hamilton, Ontario L8L 6B6











They are providing access to a new 150 x 270 ft boat hangar to contain our ground tests, netted aviary, and a 21,000 gallon water tank.





[Top]

**Dress Code**

This event will consist of equal parts meetings and test method evaluations with robots indoors and outside.  So proper dress is required, including long pants and closed toe shoes. Responders please wear your insignias.

[Top]
 

Hotel Information

**Waterfront Hotel**





2020 Lakeshore Road, L7R 4G8 Burlington, Canada





http://www.waterfronthotelburlington.com/





Only has rooms starting on June 11. Onsite Restaurant, lakefront area, walkable, 15 minutes from venue.











**Hilton Garden Inn Toronto/Burlington**





985 Syscon Road, Burlington, Ontario, L7L 5S3, Canada





Phone: 1-905-631-7000





http://hiltongardeninn3.hilton.com/en/hotels/ontario/hilton-garden-inn-toronto-burlington-YYZBUGI/index.html




(link is external



[Top]

Questions/Contacts

In addition to ASTM staff, you can visit RobotTestMethods.nist.gov  and/or email RobotTestMethod [at] nist.gov(RobotTestMethod[at]nist[dot]gov(link sends e-mail)).

[Top]



**Contact Information:**

Adam Jacoff - Project LeaderRobotTestMethods [at] nist.gov(RobotTestMethods[at]nist[dot]gov)Adam.Jacoff [at] nist.gov(Adam[dot]Jacoff[at]nist[dot]gov)
301-975-4235
  Manufacturing, Robotics in manufacturing and Resilience

    
    Created March 8, 2017, Updated February 25, 2022 ‹ 2018.02 ASTM E54.09, Gaithersburg, MD
              2017.01 ASTM E54.09/Virginia Beach Exercise, Virginia Beach, VA ›
 HEADQUARTERS
            100 Bureau Drive
              Gaithersburg, MD 20899301-975-2000
            Webmaster | Contact Us | Our Other Offices
        Twitter
            
Facebook
            
LinkedIn
            
Instagram
            
YouTube
            
Giphy
            
RSS Feed
            
Mailing List
                              How are we doing? Feedback

  
  Site Privacy
                      Accessibility
                      Privacy Program
                      Copyrights
                      Vulnerability Disclosure
                      No Fear Act Policy
                      FOIA
                      Environmental Policy
                      Scientific Integrity
                      Information Quality Standards
                      Commerce.gov
                      Science.gov
                      USA.gov
                      Vote.gov