1

Minutes

ASTM F45 Committee Meeting, April/May 2019

Recording Secretary: Adam Norton, UMass Lowell

Location: UMass Lowell NERVE Center, Lowell, MA 

Meeting Schedule:

Tuesday, April 30, 2019 Notes

F45.90 Executive Subcommittee

ASTM Committee F45 Officers:

Registered Meeting Attendees:

UMass Lowell NERVE Center

Start

d

u

r

Event Lead

8:00 AM Greetings, Introductions, Agenda Bostelman

8:10 AM UMass Lowell Greeting and Presentation UML, Norton

8:30 AM

F45.90 Executive Committee meeting - history 

overview, status, officers, ASTM information, 

etc.

Bostelman, Murphy

9:00 AM F45.01 Environmental Conditions Bostelman

9:30 AM Coffee Break

9:45 AM F45.91 Terminology - A-UGV Capability Levels Bostelman

11:45 AM Lunch

12:45 PM F45.02 - Navigation and Docking Roberts

2:45 PM Coffee Break

3:00 PM Day 1 wrap-up and next day agenda Bostelman, Norton

3:15 PM

Demonstrations: Vehicle capabilities, 

environmental conditions, docking, navigation, 

obstacles... (communication impairment?)

Norton

4:45 PM Adjourn

6:00 PM Dinner - The Keep

Tuesday, 

Apr 30 UMass Lowell NERVE Center

Start Event Lead

8:00 AM Day 2 agenda review, logistics, etc. Bostelman

8:15 AM F45-03 Object Detection & Protection - 

Describing Obstacles (Static and Dynamic) Norton

10:15 AM Coffee Break

10:30 AM F45-04 Communication & Integration - Comm. 

Impairment

Holmberg

12:30 PM Lunch

1:00 PM Fleets Reynolds

1:30 PM Building block standards Norton

3:30 PM Coffee Break

3:45 PM Main Committee - Subcommittee Reports, 

Future works

Bostelman

4:15 PM Adjourn

4:00 PM to 

5:20 PM

ASTM interviews and video recording LaFary, Roberts, 

Norton, Bostelman

6:00 PM Dinner - Lowell Burger Co.

Wednesday, 

May 1
2

Membership:

F45 Committee and Subcommittee memberships have remained relatively constant and were not 

updated for this meeting.

Key Roles to fill:

• F45.01 Chairman – Environmental Effects

Committee Accomplishments:

Name Affiliation

1 Bostelman, Roger NIST

2 Ferman, Ahmet M Omron Adept Technologies

3 Gates, Rusty Universal Parks and Resorts

4

Ghataore, Gurpreet The Manufacturing Technology Centre

5 Holmberg, Robert Google

6 Jordan, Adam The Manufacturing Technology Centre

7 LaFary, Matt 

8 Li, Qiang Shenzhen Click Technology

9 Marino, Anthony SOSV HAX

10 Messina, Elena NIST

11 Murphy, Karen ASTM International

12 Noelte, Greg Sensata Technologies

13 Norton, Adam University of Massachusetts Lowell

14 Pedersen, Christian Have Mobile Industrial Robots

15 Reynolds, Todd Omron Adept Technologies

16 Roberts, Malcolm T BR2 Consulting

17 Sarbak, Emre Mediate

18 Sparrow, Mary Ellen T Next Shift Robotics

19 Vogtman, Randall Oceaneering International, Inc

20 Yanco, Holly University of Massachusetts Lowell

21 Davis, Chris ASTM International

22 Huang, Ahing ASTM International
3

• Our ballots are producing minimal votes, meaning not everyone is responding. Please respond just to 

abstain rather than no response at all. Inactive members cause standards to not pass ballot, halting 

months of work and progress.

• Discussing method of how to deal with repeated lack of votes from voting members; after 3 strikes, send 

them a message that they could be downgraded to a non-voting member

o Idea to send an alert after 2 instances of non-response stating that if it happens a 3rd time then 

they get downgraded 

o Motion to adopt this policy, seconded, all in favor, no opposed, so it is

• Where to hold next meetings? (discussed again in Main)

ACTION ITEMS:

• Implement new policy for non-responsive voters; after 2 strikes given a notice that 3rd strike will result in 

downgrading of member status to non-official voter

• Find location/date for next in-person meeting(s)

F45.01 Environmental Effects

• Reviewed updates to WK54576 for title, adding test method references, including static/dynamic, etc.

• Discussed changing the Grade level definitions: should level 1 be 1-3% instead of up to 5%? Etc.

• What about polarization of lighting and/or external sensor emissions?

ACTION ITEMS:
4

• Changes to figure 2 (gap/step):

o Length change to width

o Length is the opposite of width

o Depth (into the ground) and height (above the ground)

o Add angle of gap/step

o Remove A-UGV path marker

o Add axes for coordinate system (X,Y)

• Make sure years are included when referencing other standards

• Updates to grade level values; level 1: 1-3%, level 2: 4-7%, etc.

• Updates to characteristics and/or description of ground such that it includes floor grates, transparent 

flooring, etc.

• Ensure synchronization between categories and characteristics in the document and the report form

• Redo sample filled out report using updated report form design

• Align angular scales of Figure 1 such that 0 is on the right and A-UGV front is on the right for both top and 

side view (this applies to the Lighting diagram and Air Quality diagram)

• Figure numbers need to be updated; there are two figure 2s

• Review how to integrate polarization of lighting and/or external sensor emissions

• Karen to check on ballots having pixelated images.

F45.91 A-UGV Capabilities

• Localization: doesn’t this depend on some level of uncertainty?

o Yes, but a test method can be used to determine if the A-UGV has some capability for localization

and measure it

• Docking: adding capability for coordinated movements in order to differentiate between docking with 

something that changes its position throughout a task, or something that is actively moving and must be 

docked with

• If a vehicle does not have a capability, does the user check a box to say so? “N/A”?

ACTION ITEMS:

• Propagate “Infrastructure Dependence” capability as a subset for higher level capabilities such as 

Navigation, Localization, etc.

• “Localization” phrasing should not use the word “kidnapped” or only refer to physical movement of the AUGV to become kidnapped (specifically cells 5E and 5F); need to reword this so the lack of localization 

could be done virtually, etc.

o This capability should really just refer to the fact that the A-UGV can become re-localized after 

being lost/un-localized

• Continue to review the obstacle dimensional levels (rows 9 and 10); are they appropriate?

• Information Sharing: Propagate the “one or more A-UGVs in its fleet” to all cells in row 11

• Information Sharing: Spell out “A-UGV System” in cells D and E instead of “A-UGVS”

• Localization: cell B should use “its” instead of “it’s”

F45.91 Terminology

• What does infrastructure mean? We have a draft definition that we came up with during previous 

meetings, but it is not accepted yet

• Can we bold our terminology in other standards such that the reader knows to reference the F45 

definition of the term?

• Reviewed proposed new terms: infrastructure, object, path, zone, navigation aids, fleet, etc.:
5

ACTION ITEMS:

• Put together statement for ASTM editors to rally for bolding, capitalizing, etc. F45 defined terms in 

standards

• Need new definitions for “envelope” and “contour” and other terms captured in the “Capabilities” 

document

• Modify definitions of “A-UGVS” and “fleet” (which is a new term) such that A-UGVS refers to a single 

vehicle only

• Ballot the new terms added from the Capabilities document

F45.02 Docking

• Reviewed latest version of the standard and make live updates

ACTION ITEMS:

• Harmonize the drawing design for all figures so they are readable and consistent

• Update route parameters variable names from Xl, Xr, etc., to X1, X2, etc.

• Fix figure 10 dimensions (17mm supposed to be 18mm)

• Add the ability to set success criteria based on time: maximum amount of time per repetition, average 

time across repetitions, etc.

o This option for success criteria can be used for other test methods, too

• Harmonize use of lower or uppercase variable letters
6

Demonstrations:

 

a b

Figure 1. (a) Test robot. (b) Test space. 

Wednesday, May 1, 2019 Notes

F45.03 Obstacle Detection and Protection

• Mentioned grid-video standard

• Overview of describing obstacle document

• Not going for standard set of obstacles (a la YCB) right now, but could do it later (probably as a separate 

guide)

• For overall, industry niche, or sensor types

• Klaus Hiddenburg was working on a standard artifact that’d he presented at the ICRA workshop that was 

supposed to cover many cases in one, Roger will follow up with him to see if he kept working on it

• Negative ballot review

• There were persuasive comments from the negative and from the affirmative with comment, so we need 

to reballot it (but we can do it concurrently)

• No document for dynamic obstacles yet

• Discussion about moving/moveable instead of dynamic

• Moveable ones will be in other test methods related to building blocks (a stationary object that’s in 

different positions between tests)

ACTION ITEMS:

• Review responses to negative ballot comments with voter

• Re-ballot Stationary Obstacles standard after updating

• Change new work item to use “Moving” obstacles in the title

• Start draft for Moving Obstacles work item

• Implement survey regarding obstacle types that could be captured in a usage guide for a reference set of 

standard obstacles

• Roger to follow up with Klaus Hiddenburg to see if he kept working on the standard obstacle artifact

F45.04 Communication and Integration

• Do you need 29 repetitions for the baseline test?

o Text for establishing a baseline

• Reviewing the characteristics of communication impairments to be recorded

• Traffic is being impaired, not the content that is being transmitted

• Where in the diagram does the impairment simulator go?
7

ACTION ITEMS:

• Figure 1 add connection to cloud connection in diagram, correct the diagram such that no antennas are 

being removed, add a new measurement figure showing the actual connections with the measurement 

device – Roger to work with Bob on this

• Adding a second connection diagram to show where the traffic control module is implemented

• Karen to organize document review with ASTM editor and Bob for formatting help

Testing of Multiple A-UGVs / Fleets

• Todd Reynolds led a discussion on fleets and their accuracy/repeatability measurement 

• Repeatability may be the same across A-UGVs of the same make/model, but the accuracy of each 

individual A-UGV may not be equivalent due to variability in manufacturing, etc.

• Two issues: 1) how we present the data that our test methods record now, 2) how do we conduct testing 

that measures the repeatability of performance across multiple A-UGVs

• How is 2) done in industry manufacturers? Some have their own internal tests for verifying repeatability; 

in general A-UGVs should all operate in the manufacturer setting before they get deployed (should)

• Repeatability can be achieved using external equipment vs. non-external equipment

• Some boilerplate into all standards about “the results of this test are for one vehicle” 

• Some information can be shared across vehicles when conducting repeatability testing; that information 

needs to be presented in the context

• “Repeatability of Performance Across a Collection of Homogeneous A-UGVs” Practice or Usage Guide

ACTION ITEMS:

• Roger to register new work item for usage guide on measuring Repeatability of Performance Across a 

Collection of Homogeneous A-UGVs (F45.01/F45.91)

• Roger, working with Todd Reynolds, to put together one page overview of what this standard is aiming to 

be used for

• Get NIST statisticians scheduled for a virtual meeting

• Send out note to committee about contributing to discussion area on testing of fleet capabilities; e.g., 

moving in a caravan, sharing information across vehicles, etc.

Combining F45 Standards - “Building Blocks”

• Went through Roger’s new drawings then Adam’s slides

• How do we combine them? Do we put all the various criteria in a table? Don’t want to have to reballot

building blocks every time we add a new standard (so maybe an extra thing we publish but don’t ballot)

• We start with a few examples

• We need to go back and massage the various tests/practices/etc, probably starting with navigation

• Need to a navigation standard that works with multiple paths (to represent the demo Adam did Tuesday 

afternoon)....

• Maybe separate existing navigation test into:

o Describing defined areas

o Simple navigation test (that does everything the existing standard is)

o Complicated navigation test (that does what Adam has demo’d)

• Maybe we actually just expand the navigation test to be able to include these more complicated spaces

• This might lose us the minimum distance for like Zx and such, but we could just have that be the minimum 

distance

• Maybe we make all tests not be methods, but instead they’re all practices and then each person builds a 

given example that has all those specific criteria (perhaps only with one of the tests)

• We probably want to have an ‘imported success criteria’ that pulls the criteria from all the building blocks 

being combined... saying ‘in play’ or not, or possibly ‘imported stock’ ‘imported and modified’ or ‘ignored’
8

• Some of them will be compound, like ‘no human intervention except when comms is interrupted’

• Some of them will just be imported, like ‘don’t hit boundaries’ and ‘cross goal line’

• We should probably add timing failure criteria to all the various tests where we’re recording it

• We probably need to number all the success criteria and failure criteria in each standard...

• We probably only have to go back and modify navigation then modify docking for now...

ACTION ITEMS:

• Adam to register new work item for Standard Usage Guide of Combining Standards

• Adam to start draft

F45 Main

• F45 Main Officers discussion:

o serve 2 yr timeframe, 3 terms in a row allowed

o The same Main committee officers were again nominated through motion and 2

nd vote with no 

opposition.

o Karen will send nomination ballot ~ October for anyone else to become a Main Officer should 

they choose.

• Roger briefly discussed F45 promotion for standards development organization liaison and for 

membership:

Joint Standards 

• ITSDF B56.5 Driverless Automatic Guided Industrial Vehicle Safety

o Roger presented F45 update at the February 2019 meeting, Sarasota, FL

• MHIA requested F45 update for their AGV Focused meeting

• Old Business:

o RIA 15.08 Mobile Robot and Mobile Manipulator Safety

o Roger presented F45, October 2018 meeting at NIST towards harmonization of terminology

Video

• work with editors to produce F45 promotion video

o ASTM interviews followed the meeting for F45 officers

Future Meetings

• International (most members from USA) - promoting F45 face-to-face is ideal 

• Demonstrations and/or real A-UGV installations to observe - also ideal

Upcoming Meetings:

Monthly E-meetings:

F45.02 1st Tuesday /month 1-3 PM ET

F45.03 2nd Tuesday /month 1-3 PM ET

F45.04 3rd Tuesday /month 1-3 PM ET

F45.91/.01 4th Tuesday /month 1-3 PM ET

Face-to-Face Meeting 

10th Bi-Annual Face-to-Face Meeting:

Where ?

When: Fall (September/October) 2019

11th Bi-Annual Face-to-Face Meeting:

Where ?

When: Spring (March/April/May) 2020

The following were discussed as potential hosts/locations for the next bi-annual meetings:

Suggestions for this meeting were:

• Fetch – San Jose, CA, USA 
9

• Robotnik – Valencia, Spain (Roger sent Roberto Guzman a LinkedIn message – said “we could be 

interested”)

• Balyo – near Paris, France (Roger sent Fabien Bardinet a LinkedIn message – no reply)

• NIST, Gaithersburg, MD, USA

• Omron –San Francisco, CA, USA

• Omron – Netherlands

ASTM-Hosted Committee Weeks:

• Oct 21-25, 2019 - Houston, TX

• Mar 30-Apr 3, 2020 - Boston, MA

• May 11-15, 2020 - Boston, MA

Suggestions for this meeting were:

• MTC (Catapult centers) Dyson - UK

• MIR, also robot cluster - Denmark 

• Disney, Universal Studios – Orlando, FL

• Mitsubishi, Diafuku, Sansei – Japan

• Rocla – Finland

• Technalia – Spain

Post-meeting potential next locations were:

• MIR - Denmark 

• MTC - UK

ACTION ITEMS:

• Karen, Roger, Matt to follow-up with scheduling the next meeting and with a conference call with Sara 

Gobbi, ASTM Europe on Europe membership.