1

ASTM F45 Committee Meeting, December 2018 Notes

Recording Secretary: Adam Norton, UMass Lowell

Location: Omron Europe, Annecy, France

Attendance: 

Agenda:

Tuesday, December 11, 2018 Notes

F45.90 Executive Subcommittee

Review of committee officers and ASTM staff

Roles that need to be filled: 

• F45 Committee Vice Chairman

• F45 Membership Secretary

• F45.01 Chairman – Environmental Effects

• F45.03 Chairman – Obstacle Detection and Protection

1 Roger Bostelman NIST

2 Karen Murphy ASTM

3 Matt LaFary

4 Adam Norton UMass-Lowell

5 Malcolm Roberts BR2 Consulting

6 Bob Holmberg X

7 Bruno Adam Omron Mobile Robots Europe

8 Paola Serna Lizarraga Omron Mexico

9 Todd Reynolds Omron Massachusetts

10 Mufit Ferman Omron California

11 Parker Conroy Omron California

12 Claus Jørgensen MIR

13 Randall Vogtman Oceaneering

14 Sara Gobbi ASTM Europe
2

• F45.91 Chairman – Terminology 

Review current state of standards, work items, committee accomplishments, web resources, and meeting schedule

Welcome presentation from Omron Adept general manager

Presentation and overview of ASTM Europe

• We have the ability to work with other standards bodies (such as ISO) to develop standards 

collaboratively

• Good to note that the F45 standards are largely test methods, rather than specifications, which can 

support requirement standards to determine if a specification is met

• Sometimes it is hard to communicate that our standards are not requirements, and that they are 

voluntarily used for measurement; that message can get lost on the end user or developer that may 

reference the standard

Example of this relationship: B56.5 standard requirement and the F45.03 Grid Video test method

• The test method can be used to prove parts of the B56.5 requirement

Some considerations needed for collaborating with others around the world

• Standard meeting day/time has helped, but other resources could be utilized, such as recording our 

meetings and Webex calls

• Could better utilize the collaboration sites to involve those on different time schedules

Chasing down non-voters: a good project for the membership secretary

Mitchell Weiss stepped down from his leadership positions of F45 Vice Chairman and F45.03 Chairman. 

Votes for filling roles:

• Motion to propose Matt LaFary as F45 Committee Vice Chairman

o Seconded around the room; accepted

• Motion to propose Matt LaFary as F45 Membership Secretary

o Seconded around the room; accepted

• Motion to propose Adam Norton as F45.03 Subcommittee Chairman

o Seconded around the room; accepted

ACTION ITEMS:

• Next face-to-face bi-annual meeting: where? When?

• Increase recording methods from meetings: video, minutes of Webex meetings (not just bi-annual), 

increased use of collaboration site?

o Maybe designate someone as a notetaker for each meeting

o ASTM can record task group meetings

• Method for dealing with non-voters; is there something we can do to increase participation?

• Update leadership roles on ASTM F45 webpage

• Matt to talk to people who came last time and maybe 2 ago and see if they’ll come if we’re in the US, 

otherwise go overseas.
3

Sara Gobbi, ASTM Europe Presentation – Sara provided background of her ASTM involvement. She will work with 

F45 to help promote the committee in Europe.

F45.01 Environmental Effects

Review of balloted changes to WK54576 to update F3218 standard practice for recording environmental effects

(due by 12/14; not closed as of this meeting)

• Some of the updates include changing from “effects” to “conditions”, adding boundaries, references,

boundaries, terminology, etc.

• Review of negative vote from Norton; removing 4.4.5 Vehicle Motion characteristic, and instead add a 

higher level characteristic of static or dynamic environmental condition in a part of the apparatus

o Likely to be removed and the Continuous/Transitional explanation will be expanded to include 

this characteristic

o Updating examples to include use of “between area”

• Editorial comments from other members will be addressed

• Comment about last sentence in 1.1 saying that recording environmental conditions is not required

o We have adjusted language to mention that the practice can be used to demonstrate compliance 

with requirements as set forth by a test requestor

• Language in section 4. Significance and Use needs to be unified in its subsections such that it is clear to 

the reader that they should be recording something on each of these characteristics

Other suggested changes to listing of conditions, what order they go in, separating gap and step from elevation 

change, etc.

Grade: other standards (like B56.5) specify that 3% grade is what qualifies a surface as a ramp; the F45.01 standard 

does not use 3%

Lots of discussion about each category of environmental condition; each set of characteristics needs to be 

reviewed

ACTION ITEMS:

• Ballot will be closed and re-balloted after updates made due to negative vote, including other comments 

brought up during the discussion; all categories of conditions should be reviewed during the next meeting

F45.91 Terminology

Might be worth adding notes to terms to address international differences in terminology; not translations, but 

making sure to include notes of how terms are used in different parts of the world, etc.

• This could open a can of worms, though, in that it’ll be hard to satisfy everyone

Discussion of A-UGV Capabilities document

• Use of “levels” and numbered values could be problematic, so maybe they instead use letter identifiers

• Clarify that a vehicle could have multiple capabilities in a particular classifier, not just one

• Situation Awareness

o Many parts of SA: environment interaction, obstacle definition, human avoidance, planning, etc.

o Might be better to decompose into separate capabilities (rather than couched in SA)

o Or, actually could break out parts of SA and associate them with each of the different A-UGV 

capabilities, like the elements of SA that are associated with navigation, those that are associated 

with obstacle avoidance, with docking, etc.; rather than have SA be its own classifier

• A good exercise would be to have members of the committee list out the capabilities of their vehicle that 

they think are important and/or that their customers think are important

• Ran through list of capabilities for “Obstacle Avoidance” as a classifier

• Ensure that these capabilities are in reference to “outside” actions (observable) rather than “inside” 

operation

• Capability distinctions should be presented alongside the test method performance that was used to 

meet that level 
4

ACTION ITEMS:

• Get more European members to contribute to standards development

• A good exercise would be to have members of the committee list out the capabilities of their vehicle that 

they think are important and/or that their customers think are important

F45.02 Navigation and Docking

WK57000 Docking Test Method

• Is mating a separate term/process from docking and positioning?

• Some confusion over the difference between docking and positioning; positioning is to a known location, 

whereas docking may be to a known location with an object, but most importantly it is relative to another 

object or infrastructure

o Example: an A-UGV dropping a pallet on the floor is positioning; an A-UGV moving to a pallet to 

pick it is docking

o The test method can be used for confirming both docking and positioning performance, but the 

standard refers to it as docking only

• There is a similar pose and positioning standard measurement in the ISO service robots standard

o Can it be leveraged? Are we doubling efforts?

o It is important to state that there is a similar standard addressing a similar capability

• The reference frame of the dock is aligned with the XY frame

• Do we have ways to indicate when tests are performed some multiple starting positions, or across 

multiple vehicles?

o Probably in a different standard; some type of standard guide that describes presenting a 

combination of tests

o Same consideration for when testing across multiple vehicles, across the same vehicle as it ages, 

degrades, etc.; will be referenced in a different standard that describes how to run those types of 

tests using these test methods

• How many repetitions? 30 minimum; then refer to the chart to get higher confidence and probability 

• Figure 13: may be confusing to include A-UGV route in a diagram (given that it is not specified as part of a 

test, and is rather recorded at the end of a test repetition)

• Building block method will include adding obstacles, etc.

• Do the fiducial marks need to be on the vehicle? They can be abstracted to just be the corners of the 

vehicle, rather than an additional marking; or motion capture reflector balls

• The distance between the fiducials on the vehicle and those on the TPMs (vertically) should be small; they 

need to be close to one another so as to not introduce potential errors over a large distance

o What is “close”? Is it defined? An explanation of this sensitivity should be added

• What is our precision and bias statement? Probably mostly in reference to the TPM grid sizes

o The resolution should be sufficient to be able to accurately measure the orientation and roll

ACTION ITEMS:

• Add ISO TC299 18646-2 as a reference; contains a similar test process for measuring positioning 

repeatability

• Add variable letters (XYZ) to the 6-dof diagram and RGB color scheme should be used to match those 

dimensions in the diagram

• Ensure there is language in the document that presents why this method is designed the way it is

• Add some language to the document that mentions that this test method can be used for testing 

repeatability of docking across a fleet of vehicles, across the same vehicle as it degrades over time, etc.

• Add example test report that shows diagram of a space layout, route of the A-UGV drawn over, etc.

• Remove figure 13; add note to figure 3 that labels the outer black line as the border of the operating 

space

• Add spot on the form to note which measurement method was used (grid on the floor with human 

observation, cameras, motion capture, etc.)
5

• Ensure the description of the vehicle fiducials allows for markings, actual points on the vehicle (like the 

corners of a panel), motion capture reflectors, etc.

• Add explanation for the sensitivity of the vertical distance between the fiducials on the vehicle and the 

grids on the floor; explain why large distances is problematic, and why a close distance should be used

• Clarify what parts of the environment are noted as being used as part of the A-UGV navigation; natural, 

unnatural, etc.; what qualifies as which?

• Write Precision and Bias statement to include resolution of TPM in order to accurately measure 

orientation and roll

Wednesday, December 12, 2018 Notes

F45.03 Obstacle Detection and Protection

(Matt LaFary taking notes)

• High level review of standard.

• Faked filling out the test form for a table.

• Measuring reflectivity is hard, getting specified reflectivity materials is VERY expensive.

• White kent sheet is 100% reflective, so retroreflectors are like 1000%.

• We should have a simple accessible version for surface quality and such, but then also give people the 

pointer to the other standard as a rabbit hole they can do down if they want (like we do for F45.01).

• We moved the more complicated versions of surface quality into an appendix so we don’t scare people 

away.

• Surface quality->surface.

• Changed the ‘persistence’ clause to accommodate fabric.

• Added other relevant characteristics to cover things like fans, or light sticks, or lamps (that are different 

depending if it’s on or off).

• Discussion about obstacle maybe not being the best term since the objects may not obstruct 

movement. Added ‘or impair’ to the introductory paragraph to clarify this.

• Reviewing comments from John Wooly. Added a comment that this can be used for indoor or outdoor 

obstacles to address his comments.

• Discussion of if we need the baseline test in the appendix.

ACTION ITEMS:

• Update document according to comments in the draft and turn it in for ballot

F45.04 Communication and Integration

Standard Practice for Implementing Communication Impairments

• Can be problematic to structure this such that this practice has its own success criteria that may differ 

from that of the test method it is applied to; Bob proposing to have only the success criteria of the target 

test method be the one that counts

• Might be best to leave out success criteria and only focus on how to implement impairments, not about 

what the success criteria is (similar to the Describing Stationary Obstacles standard practice); then both of 

those documents are structured in the same manner, establishing a type of framework for how these 

standards can be written and function together (see Building Blocks discussion later on)

• What about optical sensor interruptions? It can be obstructed via mechanical/physical means, so should 

be included here

• Should break out interruptions into loss and delay of information; two different scenarios
6

• Difference between causing impairments between components vs. on that component’s antenna; 

diagram may need to be edited down; the standard states we are not going to be interrupting RF but the 

diagram may imply that we are

o Using attenuators on an antenna could allow us to interrupt RF, though; would it be similar to 

losing the antenna completely?

• What about physically removing the antenna from the vehicle? It can cause real damage, and can be nondeterministic, so probably not a good idea

• Can be problems with communication between access points and vehicles; good to consider as a method, 

or is it for a different standard?

o Do we need to clarify our scope about what subset this standard covers? As in, it doesn’t include 

AP loss?

• Does it matter what the A-UGV is communicating with? If it all uses the same communication method... 

• This standard only covers a small subset of implementation details for how to impair a communication 

signal

• We need to keep in mind that this standard only covers how to cause a subset of an impairment, NOT 

about what the effect of that impairment is; that is for the building blocks document

ACTION ITEMS:

• Update standard and test report to only record what impairment was used, not anything about the 

performance of the vehicle, success criteria, etc.; match structure of obstacle description practice

• Add language to include mechanical interruption of optical communication links

• Break out interruptions into loss of info due to an interruption vs. delayed transmission of info after an 

interruption

• Add link in diagram to cloud infrastructure

• Update diagram to remove/edit links between components; some are not appropriate

• Add option of using attenuators to disrupt antennas; keep language about not doing “over the air” RF

• Update communication link diagram to clarify links between components, antennas to components, etc.

• Ensure there is language in the document that presents why this method is designed the way it is

Workshop: Combining F45 Standards - “Building Blocks”

Combining test method with practices for obstacles, communications impairments, etc.; is it still pass/fail? How?

• Pass/fail on the navigation test, pass/fail on the communication impair (i.e., was the response from the AUGV what was expected or not?)

It is up to us to determine the best way to do this: pulling out clauses from each standard is one option, coming up 

with another method to replace success criteria, etc.; because our group is the first to do it

“The success criteria of a combined F45 test is set forth by the test requestor and overrides any success criteria set 

by the standards which are being combined.”

• When creating new success criteria of a combined test, is it worth noting which of the “child” success 

criteria are still in play? Or is it confusing? 

Ran through an exercise trying to combine standards as a way to try and define the process

Is there text that we can add to each standard that allows them to be more easily combined?

Two ways to use this:

• A test method with multiple practices applied to it

• Multiple test methods combined (maybe also with practices)

Cannot have competing success criteria; meaning all pieces of success criteria must be achievable in a repetition; 

you cannot have partial completion; all the success criteria must be AND, not OR

Any standards that are combined, the potential effects of those standards must be addressed in the success 

criteria

• For example; navigation TM + obstacles and comms Prs = success in the presence of obstacles and comms 

impairments
7

What is the best method for how to “cut and paste” from each standard that is combined in order to present the 

information in a singular way?

Is there language we can inject into currently developing standards that speaks to how they are able to be 

combined?

• Need to be sensitive to the fact that the standard guide for combining isn’t published yet

Example structure showing combining Comm Impairment with Navigation test method and applying practices:

ACTION ITEMS:

• We still need to write up placing obstacles standards

• Add block diagram of relationship between test methods and practices to a collaboration space

• Adam to write out first draft of standard guide and test report form; upload to the collaboration area

(WK65141)

Promoting F45

Joint standards: RIA 15.08, ITSDF B56.5, ASTM E57

Future meetings, maybe international again? Ideal to have demonstrations of A-UGVs; in a lab or real deployments

ACTION ITEMS:

• Everyone get videos to Roger

• Roger to get info to people about how they can transfer video files to him

• Roger/Karen to e-mail the committee asking about logos to place on the web for a membership page