Industrial Autonomous Vehicle Project Report

Roger Bostelman

Maris Juberts

Sandor Szabo

Robert Bunch

John Evans

Intelligent Control of Mobility Systems Program

Intelligent Systems Division

Manufacturing Engineering Laboratory

National Institute of Standards and Technology

Technology Administration

Department of Commerce

June 7, 2001

NISTIR 6751
Table of Contents

1 INTRODUCTION..................................................................................................................................1

1.1 INTELLIGENT VEHICLE RESEARCH....................................................................................................1

1.2 INDUSTRIAL AUTONOMOUS VEHICLE (IAV) PROJECT......................................................................2

2 COMMERCIAL END-USERS OF AGVS VISITED .........................................................................3

2.1 POLYGRAM, CHARLOTTE, NC, DECEMBER 13, 2000........................................................................4

2.2 KEEBLER, CHICAGO, IL, FEBRUARY 13, 2000..................................................................................4

2.3 BOEING, SEATTLE, WA, APRIL 10, 2001..........................................................................................4

3 GOVERNMENT END-USERS OF AGVS VISITED.........................................................................5

3.1 LETTERKENNY ARMY DEPOT, CHAMBERSBURG, PA, SEPTEMBER 20, 2000 ...................................5

3.2 US POSTAL SERVICE, FT. MYERS, NOVEMBER 29, 2000..................................................................6

4 AGV/MOBILE ROBOT VENDORS VISITED ..................................................................................7

4.1 FMC, CHALFONT, PA, JULY 17, 2000..............................................................................................7

4.2 AGV PRODUCTS, CHARLOTTE, NC, DECEMBER 12, 2000 ...............................................................8

4.3 NDC, CHARLOTTE, NC, DECEMBER 13, 2000..................................................................................8

5 MOBILE ROBOT VENDORS/RESEARCHERS...............................................................................8

5.1 PYXIS, DANBURY, CT, APRIL 2001 ..................................................................................................9

5.2 SERVUS ROBOTS, RICHMOND, VA, APRIL 2001...............................................................................9

5.3 SPAWAR, (SPACE AND NAVAL WARFARE SYSTEMS CENTER), SAN DIEGO, CA, APRIL 23, 2001 .9

6 OTHER RESEARCH VENUES ...........................................................................................................9

6.1 NORTH AMERICAN MATERIAL HANDLING CONFERENCE, DETROIT, MI, APRIL 12, 2000 ...............9

6.2 ASRS/AGV USERS CONFERENCE, RENO, NV, JUNE 12-14, 2000 .................................................12

6.3 HK SYSTEMS CONFERENCE, PARK CITY, UT, SEPTEMBER 25-27, 2000 ........................................13

6.4 CSEM, SWITZERLAND AND S-TEC, GERMANY, SEPTEMBER 30-OCTOBER 10, 2000 .....................14

6.5 DEFENSE MANUFACTURING CONFERENCE 2000, TAMPA, FL, NOVEMBER 27-28, 2000................15

6.6 LOCKWOOD GREENE, SPARTENSBURG, SC, DECEMBER 12, 2000..................................................15

6.7 PROMAT 2001, CHICAGO, IL, FEBRUARY 12-14, 2001...................................................................16

6.8 DIMENSIONAL PHOTONICS, BOSTON, MA, APRIL 11, 2001............................................................17

6.9 MIT/LINCOLN LABS, BOSTON, MA, APRIL 12, 2001 .....................................................................17

7 COMMENTS ABOUT AGVS.............................................................................................................18

7.1 FROM USERS ...................................................................................................................................18

7.2 FROM VENDORS ..............................................................................................................................18

8 STANDARDS EFFORTS ....................................................................................................................18

8.1 AGV BUMPERS...............................................................................................................................18

8.2 OTHER STANDARDS ISSUES TO CONSIDER ......................................................................................20

9 ADVANCED TECHNOLOGY TRANSFER.....................................................................................20

9.1 IAV TESTBED .................................................................................................................................20

9.2 LADAR MEASUREMENTS ..............................................................................................................22

9.3 OTHER LADAR RESEARCH............................................................................................................23

9.3.1 Background ........................................................................................................................23

9.3.2 First Cut at a Requirements Specification for a Scannerless Ladar..................................23

9.3.3 Research Labs Doing Work in Scannerless LADAR..........................................................24

9.3.4 Other Developers................................................................................................................28

10 PLANNED PROJECT ACTIVITIES ............................................................................................29
11 REFERENCES .................................................................................................................................30

12 ACKNOWLEDGEMENTS.............................................................................................................30

13 APPENDIX: OTHER PROJECT MATERIALS..........................................................................31

13.1 SENSOR TYPES AND MOUNT IDEAS.............................................................................................31

13.2 POTENTIAL AGV CAPABILITIES IDEAS .......................................................................................34

Disclaimer:

Commercial equipment and materials are identified in order to adequately specify certain procedures. In no

case does such identification imply recommendation or endorsement by the National Institute of Standards

and Technology, nor does it imply that the materials or equipment identified are necessarily the best

available for the purpose.
1

1 Introduction

This report documents a National Institute of Standards and Technology (NIST) Intelligent Systems Division (ISD)

effort to determine the status of Automatic Guided Vehicles (AGVs) in the United States. This report is intended to:

1) provide information to management for guiding future ISD and MEL projects and 2) review ISD mobility project

research that was focused mainly on the industrial vehicle industry. The particular information collected parallels

NIST’s main mission goals, standards and measurements, as well as ISD advanced technology strengths: sensing,

modeling, and planning for autonomous vehicle navigation.

This work falls under the Intelligent Control of Mobility Systems program. Details of the program can be found at

http://www.isd.mel.nist.gov/research_areas/mobility/index.htm.

The methods for determining the state of the AGV industry’s measurement, standards and technology needs were as

follows:

1) Search Internet, read trade-magazines, attend trade shows, and visit AGV vendors and users.

2) Develop initial list of AGV manufacturers and visit key manufacturers. Determine size, capabilities and plant

locations. Gather as much information as possible regarding current state-of-the-art in sensing, modeling,

planning and navigation.

3) Develop list of AGV end-users and visit key-users, particularly government users. Determine scenarios, needs,

limitations and desires of end-users.

1.1 Intelligent Vehicle Research

Vehicle intelligence has been advancing for many years, mainly in sensing and navigation. Researchers have

developed many key advancements in vehicle navigation including this small subset:

• J. Evans, B. Krishnamurthy, Helpmate Robotics, Inc. [4]– developed an autonomous mobile robot courier for

material transport in hospitals and used around the world operating in uncontrolled and unsupervised

environments up to 24 hours per day. Navigation via sonar, vision and other sensors was used.

• H. Schneiderman, M. Nashman, NIST [11] – demonstrated vision-based, autonomous, road-following of a

HMMWV (high mobility, multi-wheeled vehicle) on highways and rural roads at full speeds using a real-time,

image-processing algorithm reading lane-markings and confined only to the 2-D plane.

• D. Raviv, Florida Atlantic University and M. Herman, NIST [9]– demonstrated vision-based autonomous road

following in the laboratory using optical-flow-based theory and generating motion commands directly from a

visual feature or cue, consisting of the projection into the image of the tangent point on the edge of the road,

along with the optical flow of this point. This approach requires no 3-D scene reconstruction and uses relatively

simple computations.

• J. Albus, NIST, 4-D/RCS Reference Model Architecture for Unmanned Ground Vehicles, [1]- currently

developing a reference model architecture for the Demo III Experimental Unmanned Vehicle program where 4-

D/RCS (4 dimensional/real-time control system) integrates the NIST RCS with the German (Universitat der

Bundeswehr Munchen) VaMoRs 4-D approach to dynamic machine vision, including computational nodes

containing behavior generation, world modeling, sensory processing, and value judgment processes.

• K. Murphy, et. al., NIST [8]– demonstrated a HMMWV (high mobility, multi-wheeled vehicle) with sensors

and computer controlled actuators that detects, using a 30 x 60 pixel field of view laser range scanner, and

avoids obstacles while it drives off road at speeds up to 35 km/h. The planner computes smooth, obstacle free

paths that follow an operators commanded path.

• X. Deng, E. Milios, et. al. York University [3]– demonstrated landmark selection for minimal sensing cost

(detection and tracking) and a strategy that verifies a map in edge traversals only, using a single edge marker,

that solves the issue of cumulative odometry errors which may cause a robot to lose track of its position in the

real world.
2

• R. Simmons, Carnegie Mellon University [12]– developed a robot architecture for reliable office delivery robots

that is composed of four layers: obstacle avoidance, navigation, path planning, and task planning.

• K. Konolige, SRI International [7]– built a team of four robots to perform cooperative mapping of building and

near-building spaces using real-time vision to perform mapping and localization.

• J. Evans, Transition Research Corporation and Johann Borenstein, U of Michigan [5]– demonstrated feasibility

of controlling a four degree of freedom vehicle using dual differential drive with linear compliance for large

non-holonomic vehicle translation in any direction and to simultaneously rotate about any instantaneous center

of rotation.

• S. Thrun, et. al., Carnegie Mellon University [13]– demonstrated a talking robot designed to accommodate

people in public spaces as it perceives and interacts with it’s environment through a computer and multiple

sensors (cameras, laser range finders, ultrasonic sensors) allowing reliable robot control in crowded

environments.

Vehicle navigation is only possible by first sensing the environment, processing the information received from the

sensors, comparing the information to stored world-model (or building world-model) information and then acting on

the environment by moving the vehicle, for example as generalized in Figure 1.

Figure 1 – The NIST RCS (real-time control system) generic architecture for intelligent control, (e.g., sensor-based

vehicle navagation). Boxes are control nodes and rows of boxes are levels of control.

Goal

Sense Model Act

A variety of sensors have been considered for vehicle navigation, such as: acoustic, vision, structured light,

microwave, and most recently, laser ranging or LADAR and LIDAR. All of these, and some other sensors not

listed, can provide the vehicle with knowledge of the environment to which the vehicle will act upon. Vehicle

vendors and users desire improvements in vehicle safety, navigation, and other vehicle functions. The cost of

sensors is steadily decreasing while the amount of information they provide steadily increases. The bottleneck

appears now to be in the processing of the sensor data to facilitate planning and control. Additionally, vendors are

faced with the challenge to improve vehicle capabilities in a cost-effective manner. One approach may be to use

sensors that can serve many uses. For example, LADAR bumpers are used more and more by AGV vendors and

users, and they may serve as a navigation sensor as well. This would currently save users thousands of dollars

alone. A key to this type of dual-usage capability is modular system design and well-defined interfaces

1.2 Industrial Autonomous Vehicle (IAV) Project

To further the intelligence of vehicle platforms for navigation via measurements, standards and advanced technology

developments, ISD has recently begun a new mobility project. The Industrial Autonomous Vehicle (IAV) Project

began in early 2000 and is presently ongoing. The project objectives are to provide measurement, standards and

advanced technology to the manufacturing industry of AGVs and mobile robots. Other AGV users such as,

warehousing, postal services, and army depots, can also benefit from project findings as AGVs are used in many

non-manufacturing settings as well. As advancements through previous and ongoing ISD Intelligent Control of

Mobility Systems Program projects (Army Demo I through Demo III and Department of Transportation Projects)
3

have evolved, ISD can now apply these research findings toward the industrial sector. Dr. Hadi A. Akeel, Fanuc

Robotics NA stated, “Robotic support for the army promises to enhance military advantage and save the lives of

soldiers. This is perhaps the most realistic motivator for future robotic development and may define the course of

robotic technology in the 21st century.” (Integrated Manufacturing Technology Roadmap (IMTR) [6] and other

advanced manufacturing studies determined that the computer is a key advanced technology impacting the

manufacturing industry over the past 50 years as used for design, planning, execution and corporate functions in

manufacturing. ISD works toward application of computers to control manufacturing processes, for example in

planning, sensing and world modeling. ISD seeks to improve the mobility aspects associated with manufacturing.

The ability to traverse structured and unstructured environments requires advanced sensing and control to navigate

around obstacles that block typically used paths. An example data capture from a planar LADAR that could be used

for navigating in unstructured environments is shown in Figure 2.

Figure 2 – (left) Photo of hallway that a testbed vehicle traversed and (right) data capture from a LADAR sensor

mounted on the testbed vehicle while traversing the hallway. The sensor information was used to build a map of the

world that can be used to plan vehicle paths (e.g., if a door was closed/blocked, another path potentially could have

been chosen).

 

Figure 2 shows that using a single LADAR sensor for obstacle detection as well as vehicle navigation, the sensor

can provide robust environmental information for intelligent vehicle control (i.e., multiple paths can be considered

should one become blocked) and without a priori knowledge of vehicle surroundings. The sensor, with its 180° field

of view, “looked” inside the left room while moving forward down the hallway.

This document provides several sections discussing findings from visits to commercial and government end-users of

AGVs, AGV vendors and other research venues (conferences, exhibitions and consultants). In the Comments about

AGVs section, feedback from several (anonymous) AGV users and vendors is generalized. Following are sections

on standards efforts and advanced technology transfer from the military and transportation projects to the IAV

project. Expected next project steps are then stated. Appendices follow with presentations to the AGV community

and some ideas toward advancements and potential intelligent vehicle niche areas to consider. A list of references

concludes the document.

2 Commercial End-Users of AGVs Visited

The following section generalizes trip reports from visits to commercial AGV end-users and each visit is dated

accordingly.
4

2.1 Polygram, Charlotte, NC, December 13, 2000

Brian Keiger took Roger Bostelman and Bob Bunch to Polygram records, now Universal, in Charlotte. They met

with Ken Gantt, senior engineer with Universal who led a tour of the facility. This is the largest music CD

manufacturing facility in the world. During the busy season, before Thanksgiving, they produce and ship 700,000

CDs per day and would like to get to 1 M per day. This innovative facility makes music CDs through injection

molding, metal foil, plastic coating and silk-screening. CDs are then packaged and boxed through semi-automated

cells as orders come to the cell operators. The operator puts in a request for say, Johny Mathis or Bon Jovi CDs, and

they arrive via AGVs to the packaging area. Once boxed, the CDs are conveyed to a robot arm that loads the boxes

onto pallets. Pallets are then taken to the loading dock area ready for shrink-wrapping and truck loading. Prior to

wrapping, some manual box consolidation is done as pallets are rarely filled to capacity. They use 14 AGVs, half of

which are Mentor vehicles that have been upgraded by Universal and NDC engineers. Universal adds work cells as

needed to grow the facility. They add their own vehicle-paths as necessary. Hence, AGV systems and robot cells

are getting much friendlier to duplicate and add to existing facilities by the user.

2.2 Keebler, Chicago, IL, February 13, 2000

Randy Winger, HK Systems AGV Market Head, set-up and escorted Bostelman to the Keebler Distribution facility

near Chicago. Chester Zimbauer, Lead Maintenance Technician led us on a tour of the facility. They use 9 AGVs

and a 13,000 location AS/RS (Automated Storage and Retrieval System) all built and installed by HK Systems.

Pallets of Keebler brownie mix, crackers, cookies, ... were being moved by AGVs. Twenty truck bays supported by

manual fork-lifts load/unload trucks and feed AGVs via conveyors. Pallets are automatically checked for oversize

prior to AS/RS storage to eliminate binding in the AS/RS. The AGVs use inertial guidance (embedded floor

magnets that error correct vehicle position and between which uses dead reckoning) and were upgraded from wireguided vehicles. The vehicles and AS/RS have been used for 9 years without major down times. This is due to their

consistent preventive maintenance (PM). Also, the PM has allowed them to not replace a single battery in 9 years!

Very impressive. The only bottleneck in the system is the logging data of pallets as they are stored/retrieved. One

person does this manually.

2.3 Boeing, Seattle, WA, April 10, 2001

Roger Bostelman, Adam Jacoff, Bob Bunch and John Evans visited Boeing to present the IAV project and to gain an

understanding of Boeing's needs for autonomous vehicles for materials transport in manufacturing.

Craig Battles, a Boeing Technical Fellow, Robotics and Automation, MR&D (Manufacturing Research and

Development), hosted the meeting. Present was a group from Phantom Works that studies vehicle navigation

(CHAOS agent architecture, not an engineering project), Everett manufacturing, a representative from Space and

Defense in Los Angeles, and three representatives from Boeing Hawker de Haviland in Australia. Participating by

Netmeeting were St. Louis, Long Beach, and Wichita.

Hawker de Havilland (Ron Spiteri)

Looking at robots and automation for assembly, for composites, and for metal processing. In the past there has been

no strategic direction, "islands" of automation. Now looking more toward coherent strategy of flexible, low-cost,

custom tailored automation to make a more flexible manufacturing process.

For Assembly: Mobile, "non-monument" machines and tools

Robots for painting and part sealing

Large part manipulation

Automated part supply and moving assembly lines

For Composites: ASRS for tools

Automated parts delivery

For Metals: Painting

Automated parts supply

Automated parts supply/delivery and moving assembly lines are vehicle applications.
5

They want large part manipulation; mobile, 5-axis drill/ream/fastener; painting and part sealing robots; automatic

part supply and moving assembly lines.

They have used the "chucka-chucka" manufacturing layout with minimal impact. This is a Japanese term meaning

point-of-use manufacturing, where all tools needed for a family of parts are put in one place; parts are made

individually as needed. They have considered placing a robot to feed the chucka-chucka process instead of a human

if the assembly interface for a robot arm could be designed.

Boeing Robotics (Craig Battles)

Examples of current robot projects. Fairly standard. Large accurate gantries, special purpose machines.

Strategic directions: Data driven manufacturing

Asset optimization

Process characterization/accreditation to reduce inspection to zero

Material handling is a big issue. Cranes are the main mechanism and are a bottleneck in manufacturing process.

One project under consideration is adding arms to mobile base so the robot can move to the parts. One use of this

idea is a "mechanic's assistant" which would carry parts and tools for worker within a workstation. Working with

Khatib at Stanford on this.

Tour of Everett Plant

This is the largest building in the world where 747/767/777 jets are built. Indoor “roadways” with stop and street

signs at intersections!

Cranes are bottleneck. 26 cranes on 50 km of track. Considering moving assembly lines and AGVs.

Lots of people on bikes, lots of small trucks.

Need for four different materials handling functions:

ß Small part and kit delivery (small payload in size and weight, on demand)

ß Fork lift (there are 100 fork lifts now in use)

ß Tug (many wheeled carts and tools for moving parts and sub assemblies around; this could be the same

vehicle as the fork lift operating as a tug)

ß Moving assembly line (very large part movement)

For navigation, could use more clearly painted lines defining aisles. These used to be very well defined, but they

have not been maintained well and are chipped and worn and poorly defined in some areas. Still good enough to

navigate, but surprisingly poor maintenance. Fewer aisles than there used to be, Everett is much more crowded than

it was 20 years ago.

One problem: Rules do not allow people or vehicles under a crane payload. So, need to know where all the cranes

are. This is more or less known but would need to be organized and available on computer network.

Boeing AGV Project

Boeing is considering forming an AGV project for autonomous navigation. Phantom Works is already doing

research on agent-based control architectures. It would start about June 2001. NIST would like to be a part of that

project. Proposed that a first step would be to run a demonstration/test vehicle around the Everett site.

3 Government End-Users of AGVs Visited

The following section generalizes trip reports from visits to US government AGV end-users and each visit is dated

accordingly.

3.1 Letterkenny Army Depot, Chambersburg, PA, September 20, 2000

Sandor Szabo and Roger Bostelman visited Letterkenny Army Depot in Chambersburg, PA. Letterkenny is

responsible for maintenance and upgrade of missiles, launchers, radar and support equipment. In 1990, 6000 people

worked at Letterkenny and now only 1200 people are employed there due to base closings that occurred in the 90’s.

They use automated guided vehicles (AGVs) to carry pallets of parts to workers throughout the maintenance facility.

Szabo and Bostelman were there to tour the facility and learn about their needs and challenges with AGVs. Upon
6

receiving a tour, Szabo presented the Demo III Project efforts and Bostelman presented the IAV Project and briefly

the RoboCrane and Boom Lift projects as other agency/industry collaboration examples. It was informative to view

and discuss AGVs working in an industrial environment, and although the vehicle sophistication is basic, it is

adequate for their operations.

Letterkenny first installed nine Eaton-Kenway (E-K) vehicles in 1984. The systems were upgraded to H-K

controllers (H-K purchased E-K) in 1994. They are currently down to 5 vehicles. The vehicles use wire-guided

navigation. All motion commands are from a central controller running on two 466 MHz PCs. One of three

remaining trained operators is able to edit configuration files that tell the controller the location of transfer stations

and wire loops. The controller then automatically coordinates deliveries to ensure vehicles don’t collide. Picks

(deliveries) are generated when a worker requests a part. The parts are stored in an ASRS (automated storage and

retrieval system). A vehicle picks up the part and delivers it to the worker.

Points to note:

1. Letterkenny personnel say the vehicles are very reliable.

2. Their biggest challenge is upgrading computers and converting from Fortran to C. An original estimate from a

vendor was $1.2M. They believe the effort could be done for $120K – if they had the source code. Since there

is no source code, the software must be built essentially from scratch (starting with analysis, design, etc.).

3. Improvements they suggest: creation of a standard specification for material-handling systems that government

or industry could use to ensure that the right interfaces (hooks) are in place to facilitate future upgrades.

4. They didn’t see any great need for technical advances, e.g., autonomous obstacle navigation.

5. Although, pick stations are located near workers, handcarts are used to move the boxes closer to their

workbenches. Also, forklifts unload trucks, buffer the parts bins in a middle area, and AGVs then take the parts

bins to an ASRS. But, they are still happy with this type of multiple-handling system.

6. They would like to work with NIST, either providing technical services (mechanical, circuit cards production,

painting, etc.) or teaming in some type of demonstration project. Potentially, Letterkenny could be an outlet for

ISD to test IAV Project developments in a “real world” environment in the future.

3.2 US Postal Service, Ft. Myers, November 29, 2000

Roger Bostelman met with Roger McLarry, Operations Support Specialist, Ft. Myers US Postal Service who is onsite and heading implementation of automated equipment into their mail distribution facility. Bostelman toured the

large, 24-hr facility including automated letter handlers rated at 30,000 pieces/hour, two ABB gantry robots to be

used for sorting, several AGV Products AGVs that deliver bulk mail to sorters, and AGV “tuggers” that pull up to 5

daisy-chained carts (of about 5 different varieties) from the loading dock area to the sorting and cancellation areas.

Letter, flat and box mail all run through the facility where 41 truck bays transition mail to southwest Florida.

Contractors are now in the facility installing an overhead conveyance system and have interrupted the use of AGVs

temporarily. McLarry suggested that Bostelman talk with Patrick Reardon, Automation Project Manager, and said

he may be interested in discussing our Mobility Program further. One possibility might be for ISD to provide

standard interfaces and advanced technology for their variety of equipment from multiple vendors. Advanced

Technology and Research Corp. is working with them.

Points to note:

• Re: AGV bumpers: A damaged AGV bumper was seen. The mechanical bumper obviously hit an object and

needed repair. Non-contact bumpers may have potentially avoided the damage.

• Re: AGV sensor configuration: Unless an object/person is directly in front of the AGV, the forward-looking

sensors may not “see” it/them. McLarry was interested in our LADAR implementation approach.

• Re: Carts: If carts are not aligned properly for pick-up by the AGV tuggers, the vehicle can not access them.

They even have alignment strips bolted to the floor for this purpose. One concern is that in some older

facilities, all of these strips, as well as the proposed embedded navigation magnets, must go through the

asbestos tile floor. This could enable OSHA to force a lengthy and expensive asbestos removal process prior to

any infrastructure change toward automation. Instead, ladar or other automated vehicle navigation preventing

this infrastructure change seems promising for improved cart and pallet access without infrastructure change.
7

4 AGV/Mobile Robot Vendors Visited

The following section generalizes trip reports from visits to AGV/mobile robot vendors AGV end-users and each

visit is dated accordingly.

4.1 FMC, Chalfont, PA, July 17, 2000

Maris Juberts, Sandor Szabo and Roger Bostelman visited FMC Corporation in Chalfont (near Philadelphia), PA to

gain background information for the Industrial Autonomous Vehicle Project. The group met with Mark Longacre,

Marketing Manager, and Barry Douglas, General Manager. The visit was informative and provided answers to a

few generic questions (see Q and A shown below). The group toured the AGV manufacturing area, which turns out

200-250 vehicles per year to a broad user-community of industries.

The following information was gathered:

• FMC is the originator of the laser-guidance navigation system. They purchased Caterpillars’ laser guidance

system. The capabilities of the laser guidance system is enhanced when combined with the capabilities of the

FMC developed “Layout Wizard” software program. Layout Wizard is used by FMC engineers or directly by

the customer to establish vehicle paths and perform other configuration tasks.

• FMC is the largest US manufacturer of AGV’s competing head-to-head with several companies who are trying

to be number one in supplying systems with laser navigation in the US.

• AGVs are less than 1 % of FMC’s business (chemicals are the largest).

• FMC was part of the NREC (National Robotic Engineering Consortium). The purpose of the consortium is to

transfer technology developed by Carnegie Mellon University (CMU). CMU implemented vehicle

control/navigation without great success at Ford. For example, CMU tested navigation using a down-looking

camera to capture the floor patterns . Later, the vehicle was supposed to match the pattern to a vehicle location.

Oil on the floor or even the vehicle could, unfortunately, change the next vehicle-pass frame match.

• The AGV industry as a whole is behind in standards implementation by “a decade.” And no one is calling for

standards.

• They have not used the vehicle to do other manufacturing tasks, such as putting a robot arm on the vehicle for

pick and place tasks where needed.

• They have tested the automated forklift market. They have found that although some predict that market to be

very large, customers typically want more functionality than can be delivered by an automated forklift. Many

of the forklift-type applications can only be done by an AGV.

Q/A

When asked what they would like to see advanced on AGVs, they said:

• For scenarios where the laser guidance system’s view of navigation targets is obstructed, they suggest indoor

GPS guidance systems.

• They want a camera that can accept a CAD-based description of the load to figure out the pallet/load pose for

vehicle alignment to it. Juberts suggested the Lidar system developed for DoD.

• They want to unload tractor-trailers with AGVs.

• Szabo suggested doing a standard vehicle test course at NIST similar to the Urban Search and Rescue approach

to promote the advancement of vehicle navigation. Data from the test course would be posted on a web site for

use in academic, industry or government research. This would provide a venue for difficult navigation

problems to be worked upon by a wide audience.

Approximate AGV advancement costs:

• Example: On an expensive $100K priced vehicle, a $2000 improvement would be acceptable, as long as sales

margin remains the same or slightly above current. Opening up new markets or niches serves as the motivator

and cost justifier here.

When asked if they might participate in a NIST-hosted workshop:

• They said they don’t see the point of this since he envisions several people not talking about their proprietary

issues with competitors. Instead, he feels the one-on-one approach as ISD did here is best.

• They suggested talking with Material Handling Industry of America who may do workshops on occasion.
8

4.2 AGV Products, Charlotte, NC, December 12, 2000

Roger Bostelman and Bob Bunch visited AGV Products, Inc., Charlotte, NC, to tour their facility and meet with

Mats Herstromer, AGV Products President. Mats, originally with NDC Automation, started his own AGV

manufacturing company 12 years ago. They have moved to a larger building next door, still own the previous one,

and they are busy with several vehicles in their manufacturing and test area. They provide AGVs of a wide variety,

including fork, unit-load, tow, custom and forklift retrofits with laser, wire and inertial guidance navigation. They

replace hydraulic components with electric motors on retrofits to provide smoother joint control.

In a meeting with Mats and two other staff members, Bostelman presented the IAV project as well as findings from

vendors and users to date for vendor-perspective input.

4.3 NDC, Charlotte, NC, December 13, 2000

On Thursday, Roger Bostelman and Bob Bunch met with Tommy Hessler, VP AGV Business and Brian Keiger,

Sales Engineer of NDC. NDC is a Swedish company that manufactures AGVs and is a top supplier in Europe with

a large presence in the US as well. They specialize in laser guidance; their LazerWay® guidance product allows the

user to attach reflectors to the walls and the vehicle to follow a CAD-developed, pre-taught path with position

feedback from a spinning laser to the reflectors. With the latest algorithms, they can install the reflectors quickly,

position the vehicle in the workspace, and run the algorithm allowing the vehicle to determine where it is based on

approximate reflector positions. As the vehicle moves, it recalibrates the position feedback information and can

follow the planned path. At Polygram, Bostelman and Bunch saw accurate vehicle positioning and traversal using

this system as the vehicles traveled between building pillars and robot cells with approx. 0.13 mm (1/2 in) tolerance.

Another important information tidbit is that NDC is unloading trucks in Europe with their laser guidance system and

other sensors. They began several years ago by placing reflectors in the trucks. They now can get accurate enough

position with reflectors outside the truck and using the Sick laser scanner.

Bostelman presented the IAV project findings to NDC showing vendor and user feedback. They provided some

input here and offered to comment on slides prior to presentation at future conferences.

The following information was gathered:

• Source code is provided to customers as needed/requested to allow them to make their own changes.

• Vehicles are smart enough to allow path changes from remote planners through RF (radio frequency_.

• NDC said that it would be advantageous for ISD to visit their Sweden office where 100 engineers and

researchers produce innovative technology in AGVs, as well as other European user facilities.

• Laser guidance works and is being used extensively in the US and Europe.

5 Mobile Robot Vendors/Researchers

In April, ISD visited Boeing Everett Facility, Seattle, WA (see section 2.3 Boeing, Seattle, WA, April 10, 2001) to

discuss the use of AGVs for aerospace manufacturing. The feedback received for their manufacturing plant was that

AGVs would have to use landmark navigation, would have to do point-to-point routing and on-demand dispatch, and

would have to variously perform tug, fork lift, and small payload messenger functions.

Since that visit, John Evans had the opportunity to explore current service technology for floor cleaning robots,

merchandising robots, hospital materials transport robots, and security robots. These are all applications where

products exist and where many units, in some cases hundreds of units, have been installed and supported in the field

although none have been profitable markets to date.

John’s conclusions are that all of these applications are successfully using ultrasonics and laser ranging for obstacle

detection and avoidance, and that non-contact sensing is a proven approach for bumpers for autonomous machines

moving safely in close proximity with people.

Non-contact sensors for obstacle detection and avoidance have been successfully deployed on several service robots

for floor cleaning, hospital deliver, and security. The hospital robot in particular has shown that safe operation in

crowded, uncontrolled environments is possible on a 24/7 basis.
9

Sonar and the Sick lidar are the most common sensor systems used by these products.

The following are trip reports from visits to Pyxis, Servus Robots and SPAWAR (Space and Naval Warfare Systems

Center).

5.1 Pyxis, Danbury, CT, April 2001

Pyxis, a subsidiary of Cardinal Healthcare, has purchased the hospital delivery robot technology from HelpMate

Robotics. There are over 100 units in the field and production is eight per month, mostly rebuilds of older units.

The original HelpMate robot used 28 Polaroid sonar transducers and structured light vision for obstacle detection

and avoidance, plus Tapeswitch contact bumpers. They are currently adding a Sick LADAR to the robot to replace

the structured light vision.

5.2 Servus Robots, Richmond, VA, April 2001

Servus has purchased the RoboKent floor cleaning machines from Kent, a former subsidiary of Electrolux AB of

Sweden and now a part of Nillfisk Advance. Approximately 300 of these machines have been built, although the

number in active use is probably only half of that. These robots used the Polaroid sonar sensors for obstacle detection

and avoidance, plus Tapeswitch contact bumpers. They are also planning to add a Sick LADAR to their robots and

want to form a CRADA with NIST to use the LADAR mapping code that was developed and demonstrated.

Servus has also been working with Carnegie Mellon to develop a "merchandising robot." This design uses two Sick

LADARs plus the Polaroid sensors plus contact bumpers for obstacle detection and avoidance.

5.3 SPAWAR, (Space and Naval Warfare Systems Center), San Diego, CA, April 23,

2001

The DoD (Department of Defense) MDARS-I (Mobile Detection Assessment and Response System-Interior)

security robot project has deployed several prototypes for evaluation. They use a Cybermotion platform plus an

array of security sensors. Sonar has been the principle sensor used for obstacle detection and avoidance. This robot

uses a Sick lidar to detect reflective targets as landmarks for navigation. The Navy Space and Naval Warfare

Systems Center in San Diego has performed most of the technical work.

6 Other Research Venues

The following section generalizes trip reports from visits to research venues (conferences and meetings) and each

visit is dated accordingly.

6.1 North American Material Handling Conference, Detroit, MI, April 12, 2000

Roger Bostelman attended the North American Material Handling Show at Cobo Hall in Detroit, MI, to gain

background information for the Industrial Autonomous Vehicle Project. This informative show focused on

industrial material handling equipment such as Automatic Guided Vehicles (AGVs), pallet lifters and jacks, lifting

equipment, automated storage and retrieval systems (AS/RS), Automatic ID Systems and Data Collection

Equipment, and many other smaller parts and equipment. The main focus for Bostelman was on AGVs and sensor

technology (laser and ID systems). In short, the trip was a very useful information-gathering venture for this and

other ISD projects with applications in material handling and/or mobility. Also, all (vendor) company

representatives that Bostelman talked to are interested in: becoming involved with the IAV project, ISD hosting a

workshop, and ISD becoming move involved in the industry.

There were about twenty AGV-related booths, several displaying their AGVs statically or in motion. The two

guidance technologies displayed were wire and laser guidance (see Figure 3).
10

Figure 3: Laser Triangulation (left) vs. Wire-guided graphics from AGV Products website.

 

One impressive demonstration involved the Swedish NDC (Netzler &Dahlgren Co) vehicle. NDC also has a US

entity in Charlotte, NC. NDC provides the sensor system and controls for most other AGVs. They had a large

booth allowing their 1814 kg (4000 lb) capacity vehicle to navigate throughout the booth and to stop at a roller table

(see Figure 4). A spinning laser was used to determine vehicle positioning within a group of reflectors based on

triangulation and position updates are provided at 20 Hz. See Figure 5 for their control scheme and is an example of

an AGV controller. Their vehicle used differential steer/drive wheels along with passive wheels to allow the vehicle

to turn in place, crab and drive in a variety of directions. It easily approached the tray table by crabbing. Bostelman

spoke with Michael Thornton and was later introduced to Anders Dahlgren, one of the company owners. Bostelman

explained NIST’s role in promoting this industry and Mr. Dahlgren expressed his delight at NIST’s involvement.

Figure 4 – NDC booth layout at NAMHC Exhibition.

batteries

controls

roller/lift table

vehicle

safety bumpers batteries

controls

laser reflectors

obstacles spinning laser
11

Figure 5 – NDC Vehicle Control Scheme (from NDC company website)

A list of other companies at the show, their representatives, and some brief product descriptions were:

• Portec (Donny Kirtland), Cincinnati, OH, demonstrated their vehicle following a track and pulling articulating

joint trailers from Ederer (Mel Terry). The joint allows the trailer to follow the same tracks as the vehicle

wheels.

• AGV Products, Inc. (Mats Herrstromer, President) Charlotte, NC, displayed their laser-guided forklift-style

vehicle.

• Jervis B. Webb Co. (was Control Engineering Co.) (Earl Raynal, Jr.) of Harbor Springs, MI displayed their

vehicle, called Prontow.

• FMC Corp. (Patrick Conway) showed their SGVS2000 forklift–style vehicle moving back and forth and

displayed their laser guidance capability. A video of their SGV2000 vehicle was obtained. Patrick Conway

mentioned that he is on the AGVS committee as part of NAMH (North American Material Handling). FMC’s

AGV work is performed within their Airport Products and Systems Division.

• Mentor AGVS (Matthew Curry) Bedford Heights, OH, displayed no vehicles at the show. They offer a newly

designed laser-guided vehicle to transport coils. Loads are 45 kg to 108,862 kg (100 lb to 240,000 lb).

• Sick Inc. Bloomington, MN, displayed their laser ranging systems. They displayed two types, one with obstacle

detection capability and one with ranging capability. The range system was good to 50 m with a 1 cm

resolution.

• RF-ID, Inc. (John Martinez) Aurora, CO displayed their RF ID (identification) tags. For AGV applications, the

tags are buried in the floor to update the vehicle position. They are about the size of a role of nickels and

include a microprocessor and transmitter.

Bostelman visited several other AGV-related booths, including:

• AFT – Automation and Conveying Systems

Ltd.

• Bishamon Industries

• Creform Logis-Tech Corp.
12

• Eisenmann Corp.

• Frog Navigation Systems

• HK Systems

• Insul-8 Corp.

• Katlyn International, Inc.

• Keogh Consulting

• Material Handling Industry of America

• PEEM Forderanlagen Ges Mbh

• ProMat 2001

• Rapistan Systems

• SI Handling Systems

• Schlafhorst Automation

• Spectra Precision

• Visolux

Bostelman also spoke with Charles Switzer of the ASRS/AGVS User’s Association who pointed out a

conference from June 11 -14, 2000, in Reno, Nevada, (http://www.asrs.org/calendar.htm) called 2000

Annual ASRS/AGVS Users' Association Conference "The Economics of Automated Material Handling in

the New Millennium.” Another show mentioned was the International Manufacturing Technology Show

(http://www.imts.org) in Chicago, September 6-13, 2000 that had a pavilion on Lasers and Laser Systems.

It was also suggested that ISD should contact Richard Ward of the Material Handling Industry of America

who deals specifically with the AGV industry.

Information was requested from nearly all the companies listed above. Also, a Material Handling Resource

Catalog 2000 of titles and prices of material handling literature and a Material Handling Product and

Membership Directory 2000 was also obtained. Several of these items will be ordered, including “An

Overview of a Recommended Industry Practice for Calculating an Approximate Measure of AGVS

Performance” and the CD-ROM of “A Personal Guide to Automated Guided Vehicle Systems” which

includes case studies and sub-system explanations.

6.2 ASRS/AGV Users Conference, Reno, NV, June 12-14, 2000

Maris Juberts and Roger Bostelman attended the ASRS/AGV User’s Conference at John Ascuaga’s Nugget

Hotel in Reno, NV, from June 12-14 as background information for the Industrial Autonomous Vehicle

Project. This informative conference joined both automated storage and retrieval system and automated

guided vehicle (ASRS/AGV) users and vendors together for information exchange. It also served as an

ASRS/AGV business meeting. ISD provided handouts including a presentation called “NIST and the

Industrial Autonomous Vehicle Project,” a list of references for previous NIST vehicle research, and an

“Intelligent Control of Mobility Systems” program brochure. Our main focus was to learn about navigation

and control problems encountered by AGV users and vendors. Several presentations were given along with

ASRS- and AGV- focused breakout sessions. The AGV breakout session included only three users and

focused on their battery, static- electricity, and forklifts side-by-side with AGV problems. Low session

turnout seemed to be due to a low AGV user turnout at the conference (i.e., mostly ASRS users/vendors).

Juberts and Bostelman met with June Powers, User’s Group President (retiring after this year) and Leslie

Cumiford (Sandia Lab’s), the Groups Vice-President (future President). Discussed was the possibility of a

NIST-hosted Industrial Autonomous Vehicle (IAV) Project workshop. They felt that getting users and

vendors to participate/attend a workshop might be unlikely based on the turnout they got for this

conference (94 total attendees = 56 users + 32 vendors + 6 academic/government). Conference attendees

represent 26 users with 23 vendors and 4 government agencies/universities. It was suggested that NIST

participate in the next User’s Conference, which was planned for June 2001 in Columbus, OH. Also, for

near-term AGV user’s/vendors needs research, it was suggested that NIST conduct (with assistance from

the User’s Group) an industry survey. They receive about 90 % returned surveys upon follow-up calls to

survey recipients. Without follow-up calls, they receive about 5 % back. As for the positive response to a

NIST-hosted IAV Project workshop that Bostelman received, from the vendors at the material handling

show in Detroit April 2000, Cumiford said they typically hear the same thing and still get lower turn-out

than expected.

Bottom line perspective from the conference:

• Users want system reliability/high-“availability” (i.e., up and running time), upgrades to the newest

available technology at the best price, and 24/7 maintenance.

• Vendors want to sell new and/or retrofit existing products for a reliable (low maintenance) system. For

example, millions of dollars worth of equipment was sold at this conference alone last year.
13

• Neither discussed further technological advancements. A likely scenario is that neither group has

followed/tracked technological advancements beyond their current systems and may not be aware of

what recent advancements can provide (e.g., navigation advancements, additional vehicle uses, flexible

shop configurations, etc.).

Some of the papers presented at this conference were:

• “Overview of the Atlas Services Operations ...” by Mike Kenney, Atlas Services. This paper talked

about a two-year-old facility with ASRS and how production efficiency increased as a result. The

local plant was toured on Tuesday.

• “The Impact on E-Commerce on Our Industry” by Tom Hughes, HK Systems. This paper provided

useful hints to the users such as “get involved in the e-commerce arena or be left behind.” For

example, control software is to be web-enabled and configurable; users need to plan on and design for

ultimate flexibility.

• “E-Commerce and Supply Chain Management” by Dale Rogers, U of Nev./Reno. This paper was

consistent with the HK Systems’ talk and also said that universities must educate students of the

impact of E-Commerce (i.e., provide a solid basis with e-commerce integration taught to new students

focused on material handling).

• “Moving to SAP: Impact to Automation” by Nadra Nicholas, Hewlett Packard (HP). This paper

discussed HP’s movement to the Systems, Applications, and Projects (SAP) in Data Processing

approach. This is similar to RCS where there is hierarchical control of the warehouse that runs the

Equipment Control Software and the management system. Conveyors, AGVs, cranes and towers reside

below the ECS. These multi-vendor systems appear to be interoperable and will feed data back to the

SAP for warehouse updates and decision making control. SAP is to be implemented in a year.

• “OK Grocery ASRS Retrofit Project” by John Weber, OK Grocery. This paper and video described an

HK Systems retrofit project that included an upgrade of an old DEC PDP 11/44 and PDP 11/03

controller with an advanced S/RM main controller and several Sick Laser vertical and horizontal axis

positioning sensors providing ±1 mm positioning accuracy.

In short, the trip was a very useful information-gathering venture for the IAV Project. Based upon findings

from the April 2000 material handling show and this conference, it is recommended that ISD obtain

additional feedback from users directly through mailings, site visits, and industry research. Afterwards,

when an agenda has been established, a workshop can be scheduled. Foreign vendors, such as Frog and

Swisslog Munck, which provide AGV and sensor technology to US companies, can offer potentially useful

input to the IAV project as well. These foreign entities should also be used as a benchmark for assessing

further research needs of the AGV and sensor industries in the US.

6.3 HK Systems Conference, Park City, UT, September 25-27, 2000

Roger Bostelman attended the HK Systems User’s Conference at the Grand Summit Resort Hotel in Park

City, UT as background information for the Industrial Autonomous Vehicle Project and to network with

industry participants. This informative conference focused on AGV issues, mainly HK Systems’ AGVs,

including a tour of their facility. But, also presented were user reports on these systems from a variety of

users and battery upgrade points from AGV Products, Inc. Conference proceedings are available on CD if

there is further interest in the papers presented. As background information for the IAV Project, the most

interesting talks attended were the Equipment Roundtable and the Preventive Maintenance talk, both

regarding AGVs. These talks allowed users to discuss their challenges with AGVs and how problems can

be prevented. The problems that arose and the prevention of problems presented gave users a chance to

voice their positive or negative experiences with these machines. Mats Herrstromer, AGV Products

President, claimed there were few reliability problems with their units. Similar responses from HK

Systems staff supported Herrstromer’s assertion.

Bostelman also met with Herrstromer and presented to him the IAV Project and asked him questions about

their systems. Herrstromer stated that they do provide source code to their customers and build several

systems custom tailored to customer needs. For example, AGV Products has modified Hyster forklifts for

autonomous operation at Sandia National Lab to reliably move nuclear waste containers in a warehouse.
14

Bostelman also spoke at length with the Sick Laser sales representatives (Ted Hemmelgarn and Patrick

Bornstein) and presented the IAV Project to them. Bostelman received a very favorable response

suggesting the possibility of a future collaboration with Sick. They suggested Bostelman send them the

presentation to be forwarded to their management.

Bostelman spoke with Randy Winger, HK Systems AGV expert. In an email returned by his boss, the

reply was to “nurture the NIST relationship.” Randy seems very knowledgeable of this industry, including

the following feedback from him:

• Bumpers have been troublesome since their inception. Material modifications are all that have been

done since the B56 standard required a mechanical bumper on AGVs. Bostelman sent Umberto

D'Urso, ASME B56 Committee member, email pertaining to the B56 standard, specifically the AGV

bumper related standard - B56.5, since there was interest by the AGV industry to get the committee to

meet again. The committee has not met in four years. The committee has now been reformed and will

meet again soon.

• Secondary markets are nearly the only way vendors can make money in this $70M market. There are

basically 10 big companies and 20 little ones that make up the market. The development of custom

source code is a large secondary market. In cases when source code is provided to a customer, it is

rarely used. They typically call the vendor back for upgrades, etc.

• Quality forklifts have been successfully implemented as AGV forklifts. Unloading/loading trucks has

been considered and tested in approximately 1988 by Caterpillar for Chrysler. The challenge is that the

last few pallets on the truck near the loading dock remain difficult to retrieve without articulated forks.

Also, with 40-50 different truck types, standard unload procedures won’t work. Unloading is more

difficult than loading since the procedure for unloading is not often know a priori.

• Fork lifts cost $100k, but the hydraulics are difficult to work with – instead electric drives are used

due to increased precision. Tilt is virtually non-existent for AGVs and might be expensive for forks.

• RF has been tested for indoor navigation.

Follow-up will include additional discussions with Randy Winger, HK Systems, and a visit to the US Post

Office Distribution Center, which uses AGVs, while in Florida for the DMC 2000 Conference.

6.4 CSEM, Switzerland and S-Tec, Germany, September 30-October 10, 2000

Maris Juberts traveled to Germany and Switzerland with the following itinerary:

• Attend a presentation of capabilities of battlefield robotics (PRIMUS, Demo C +) by the German

Ministry of Defense in Friedrichshafen, Germany.

• Present a brief report on the status of Flash LADAR technology during a technical session at the

PRIMUS workshop in Friedrichshafen.

• Join Boeing in a meeting with KMW (Krauss-Maffei Wegmann) Company in Germany. Technology

Assessment and Concept Design Support are part of NIST’s responsibilities under a sub-contract to

Boeing in the Future Combat Systems (FCS) program.

• While in Europe, meet with research labs developing next generation Flash LADAR imaging cameras.

Maris has been tracking the progress in development in Flash LADAR

• Maris has been in contact with LADAR developers during the past several years; they have been

willing to provide him information on development progress and availability. There are two labs of

primary interest to NIST for UGV (unmanned ground vehicle) applications, the FCS program and the

Industrial Autonomous Vehicles project. They are:

• CSEM (Swiss Center for Electronics and Microtechnology) in Zurich, Switzerland.

• S-Tec (Sensor TEChnik) in Siegen, Germany.

These two site reports are explained below.

 CSEM (Swiss Center for Electronics and Microtechnology)

Juberts met with Dr. Peter Seitz at Swiss Center for Electronics and Microtechnology, Inc. (CSEM) in

Zurich, Switzerland, pertaining to a scannerless, time-of-flight, range-imaging camera using demodulation

pixels in CCD and CMOS technologies. The availability of such a sensor for NIST’s use was also

discussed. Seitz could provide NIST with a prototype camera available in 2002. It will be a 128 X 64 pixel,

5-50 fps camera. Juberts will work with him to get a more definite date.
15

A condensed description of their developed technology is in Section 8.3 Other LADAR Research.

S-TEC,University of Seigen, Germany

Juberts attended a meeting at S-TEC located in Siegen, Germany at the University of Siegen with Bernd

Buxbaum (developer of 3-D solid state range imaging system), Zhanping Xu (project leader for

optoelectronics and microsensors) and Stephan Zoll (marketing manager). The meeting was arranged to

discuss S-TEC’s approach for developing a scannerless LADAR electro-optical sensor – their Photonic

Mixer Device (PMD) in particular. Juberts provided an overview of NIST and the NIST program in

Mobility Systems.

A condensed description of their developed technology is in Section 8.3 Other LADAR Research.

6.5 Defense Manufacturing Conference 2000, Tampa, FL, November 27-28,

2000

Roger Bostelman attended and gave a presentation at the Defense Manufacturing Conference (DMC) 2000

at the Tampa Convention Center in Tampa, FL, November 27 and 28. His talk, entitled “Intelligent Control

of Mobility Systems and the Industrial Autonomous Vehicle Project,” included general information about

the Mobility Program, Army Demo III, past DOT (Department of Transportation) efforts, FMR (Field

Material-handling Robot), and how these efforts provide sound basis for the IAV Project.

Dave Stieren co-hosted this “Gee Whiz Technologies” session with Dr. Bill Alzheimer of Sandia National

Labs. Other talks in this session, besides Bostelman’s talk, were from Sandia and NIST (Kevin Lyons)

including:

• Engineering Energetic Materials for Novel Applications (i.e., micro-explosives).

• Meso-Scale Machining Capabilities and Issues.

• Complex Simulations Tools for Improved Manufacturing Processes.

• BioMems: The promise of Microsystems in the Biological and Medical Sciences (i.e., microscopic

motors).

• On the Path Toward Nano-Manufacturing.

After the session, Bostelman talked with Maurice Smith, Honeywell Kansas City (KC), who was very

interested in Bostelman’s talk and requested that he visit KC’s facility and discuss collaboration. They are

a government facility run by Honeywell, similar to an AMRF (Automated Manufacturing Research

Facility)-type manufacturing facility that produces parts for the nuclear industry. They have two transport

vehicles (AGVs) feeding parts and materials to the manufacturing area. But the vehicles have limited

intelligence and miss targets occasionally due to incorrect alignment with tables, etc. We discussed a

potential collaboration between NIST and Honeywell that would provide us with a focus toward a real

facility implementation and they would receive advanced vehicle control expertise with standards and

performance metrics efforts leveraging both NIST and Honeywell funds. A third partner (e.g., an AGV

vendor) is envisioned to include more industry support. Bostelman will follow up on this with Smith.

Bostelman also spoke with Dr. Calvin Jaeger of Sandia who would like to compare notes sometime about

their vehicle project and ours. They are apparently doing some advanced vehicle control and capabilities as

well, e.g., two manipulators mounted on a HMMWV.

A CD of the proceedings and a list of pre-registered conference attendees, including names, mailing and

email addresses, and phone numbers, were included in the conference materials package.

6.6 Lockwood Greene, Spartensburg, SC, December 12, 2000

Roger Bostelman and Bob Bunch met with Kirk Redwine, a marketing manager with LG in Spartensburg,

SC, from whom NIST had just procured an AGV study [Redwine, 2000]. Redwine provided some insights

into the AGV industry. Interesting facts he had found are that sales in the US are at 400 units annually as

compared with 1800 for Europe and 2300 for Japan. He says, “the US manufacturing community has not

fully embraced the value that an AGV system can bring to the manufacturing processes.” But, he feels that
16

upon studying the market, the US has the “opportunity to significantly grow their US sales” through

innovation, finding new markets and educating manufacturing [Redwine, 2000]. It appears that Japanese

systems are not readily used or marketed outside of Japanese markets as Kirk Redwine suggests.

6.7 Promat 2001, Chicago, IL, February 12-14, 2001

Roger Bostelman attended ProMat 2001 in Chicago, IL, and visited the Keebler distribution facility near

Chicago on Feb. 12 - 14. ProMat is a large material handling show that displays conveyers, automated

guided vehicles (AGVs), wheels, forklifts, hand carts/trucks, etc.

Bostelman’s main focus was on AGVs. He talked with nearly all the AGV vendors (NDC, FMC, AGV

Products, Mentor, FROG, HK Systems, Egemin) of which ISD has had contacts with most of them. Also,

he made contact with Bell and Howell, a mail delivery AGV that was new information to the NIST – IAV

Project. Also, Bostelman spoke with Sick (Eric Hanson, LMS product Specialist). These are some

highlights from Bostelmans’ conversations with company representatives:

• NDC: Tommy Hessler, VP AGV Business. They were running an AGV demo with embedded voice

alerts of commanded vehicle actions and showed the vehicle crabbing.

• FMC: Mark Longacre, AGV Sales. They demonstrated a large fork vehicle.

• Mentor: They have an agreement with NDC to use their laser guidance.

• AGV Products: Mats Herstromer, AGV Products President. He asked about progress on the bumper

standard. They had two booths and were displaying a spinning IR sensor for use as an obstacle

detector for AGVs, etc. They were demonstrating a large Hyster forklift/AGV hybrid moving pallets

in a small AS/RS.

• FROG: Jaap van der Werff, Managing Director. They demonstrated their grid of floor-magnets that

they use for vehicle guidance. They claim they can navigate around obstacles. Bostelman said he

would forward information to them that NIST generates for the industry.

• HK Systems: Mike Kotecki, Marketing/Sales head and Randy Winger, AGV head. They were

demonstrating a quiet conveyor system. Mike said he wants to ensure they are keeping up with the

IAV project and new AGV niches and leads that we can help provide, such as shipyards and aircraft

manufacturing. They like the idea of supporting us toward future collaboration potential.

• Egemin: Chad Bruhn, applications engineer, and Bostelman said he would forward information to

them that is generated for the industry.

• Bell and Howell: Richard Paske, VP Mailmobile Business Unit, who knows John Evans and would

like to keep up with status of our project.

Other than AGV Vendors:

• Sick: Eric Hanson, LMS product Specialist. Bostelman asked if the safety LADAR (yellow) and the

precision measurement LADAR (blue) could be put together in the same unit. Within 1-1/2 hrs of the

visit with Eric, two other AGV companies had asked Eric the same question. This appears to represent

the notion that AGV vendors want to do more with LADAR and their vehicles. Eric will discuss the

idea with his management.

• Comtrol: The company sells dual port RS232 to Ethernet converters called the PocketPort Serial Hub.

The unit is new and soon to be in production.

• Material Handling Industry of America: Dick Ward, AGV Head Representative. They discussed the

ASME B56.5 standard regarding an effort to change the bumper standard from requiring mechanical

contact to allowing for non-contact bumpers. Ward organized a quick, on-the-fly AGV vendor

meeting at the conference to discuss the “bumper” standard issue. They met and decided to hold a

more formal meeting in Charlotte, NC, on April 3 and 4, 2001, to discuss this issue. Bostelman should

plan to attend as an invited guest by the industry.

Presentation

On Wednesday, Bostelman attended Kirk Redwine’s informative talk, entitled: “How New Technology is

Impacting the Specification and Performance of AGV Systems.” Kirk mentioned several markets,

applications, and types of AGVs, including a slide showing the NIST Demo III XUV. He also mentioned

the work we are doing in bumper standards and potential LADAR navigation of AGVs.
17

6.8 Dimensional Photonics, Boston, MA, April 11, 2001

Contact: Lyle Shirley, President and CEO

Purpose of trip: See demo of a new laser-based high resolution/accuracy non-contact 3D imaging system.

Shirley provided overview of the Accordian Fringe Interferometry (AFI) concept. This technology was

initially developed at MIT/LL but is being commercialized by Dimensional Photonics – a Lincoln Labs

spin-off company. The approach overcomes limitations of standard light projection structured light

imaging techniques – which suffers from depth-of-field limitations for imaging a pattern on the surface of

the object with a lens. AFI provides an illumination pattern with infinite depth-of-field by projecting

interference fringes. AFI can achieve high resolution by projecting very small interference fringes and

modulating their phase to subdivide fringes into many resolution cells. Shirley claims that this will lead to

low cost systems ($20-30K) for high speed and high accuracy 3D metrology/inspection applications.

The system was used to scan a complex 3-blade fan ( 25.4 cm (10 in) across.) The measurements took

about 90 s and a 3D model was generated. Dimensions and surface features were measured to an accuracy

of about 10 μm.

Juberts described NIST’s work and experience with LADAR obstacle detection/avoidance sensors. Shirley

was eager to learn more about NIST specific needs for mobility systems. They plan to expand their work in

high accuracy 3D measurement systems to other applications.

Shirley provided Juberts with a paper (Proceedings of SPIE) titled “High-precision surface profiling with

broadband accordion fringe interferometry.” All the authors are currently with Dimensional Photonics but

did this work while employed at MIT Lincoln Labs. Another paper titled “Video-rate surface profiling

with acousto-optic accordion fringe interferometry” is available from Optical Engineering Vol 39, No 1,

January 2000.

6.9 MIT/Lincoln Labs, Boston, MA, April 12, 2001

Contact: Rich Heinrichs, Program Manager for 3D Laser Radar Technology

Purpose of Trip: Invited committee member for review of the Lincoln Labs program in active optical

sensors.

Juberts was asked by Lincoln Labs to join a government agency review committee to evaluate their work.

The committee consists of B. Blumberg – Air Force (Hanscom), G. Spiers – JPL, M. Juberts – NIST, A.

Dalcher – IDA (Institute for Defense Analyses), and M. Rankin – Navair.

The committee was very impressed by the work at Lincoln Labs in their program on Active Optical

Surveillance and Detection. All parts of the program are well managed and showed great synergy and

cooperation. Lincoln Labs seems to be the only laboratory proceeding with the development of Geiger

mode APD detector arrays for LADAR applications. Good performance has already been achieved.

Although the committee presented initial comments, B. Blumberg (committee head) will collect additional

comments by committee members and will prepare a report by May 9, 2001.

Session 1 was on 3-D Laser Radar Technology development.

Areas covered included:

• APD/CMOS Array Technology Development

• APD Performance Measurements

• LADAR sensor development and data-collection activities

• Multi-function laser radar with photon counting arrays

• InGaAs APD Array development

Session 2 was on Active Hyperspectral Sensor System development

Areas covered included:

• STRIKE system concept

• Measurement highlights and plans
18

• Measurements supporting Bio-detection

A condensed description of their developed technology is in Section 9.3 Other LADAR Research.

7 Comments about AGVs

Sections 7.1 Users and 7.2 Vendors summarize comments received from AGV users and vendors. They do

not necessarily represent comments about all AGV systems. They are individual comments when asked

one-on-one to anonymous users and vendors about the bulleted general headings, e.g., Challenges/Needs.

Some comments are direct quotes from anonymous users/vendors.

7.1 From Users

• Challenges/Needs...

Improved vehicle reliability, open proprietary-software, reduced procurement times, improved bumpers

(less false information).

• Open-Architecture Controls...

Open-architecture controls or suitable "hooks" should be supplied; AGVs are a relatively small market with

a few large control system suppliers that tend to dominate the market; “We absolutely demand source code,

electronic, and mechanical documentation.”

• Navigation...

Should not require “buried wire” or other types of fixed path guides; should allow for vehicle “passing” of

vehicles waiting and around other temporary obstructions; vehicles should convey their intentions to casual

human observers in the area, the more precisely the better.

• In the future, ...

Faster ways to program vehicle paths, “it can take the supplier weeks to set up all the paths and turns even

with something as simple as wire guidance;” Easier to add third party hardware; Establishing the controls

and communication with the AGV could potentially be simplified; Easier to program new capabilities (“On

the other end, we have used several XXX vehicles which are fairly advanced in their high level controls

and programming; however, that level of technology has not really made it into industrial vehicles.”); “If

we ever have to buy more..., it would be nice to have a controller/PLC onboard the AGV which we can

plug a laptop into and make our own changes.”

7.2 From Vendors

• There are no standards in the industry to speak of, and no push for them.

• Open architecture control that would allow plug-and-play of other vendors' boards in your system, is

non-existent in this industry.

• Mechanical bumpers are problematic and US standard ASME B56.5 limits vehicle capability (false

stops, material hysteresis, slower speeds for short stopping distance).

• Laser bumpers are prevalent in Europe and nearly non-existent in the US.

• At least one vendor can unload trucks with AGVs.

• A potential user would typically choose AGVs over conveyers for heavy parts, low volumes and

longer distances and choose AGVs over forklifts due to their decreased labor expense. Also, fork lifts

and AGVs don't mix well due to human judgements.

• Controls are getting friendlier for users.

• Laser guidance systems are used in and between buildings (mainly in Europe.)

8 Standards Efforts

8.1 AGV Bumpers

Discussions with users, vendors and others (as noted in trip reports and in user and vendor comments

above) pointed out problems with mechanical bumpers. Problems included false information from
19

mechanical bumpers and hysteresis due to bumper materials among other problems. The ASME standard

B56.5 that directly relates to AGV bumpers, and specifies what can be used as a bumper, is not clear (see

Figure 6).

Figure 6 - NIST IAV Project Presentation Slide that shows the associated line number and potential

confusion of the current standard with regard to allowing non-contact sensors on AGVs.

ASME B56.5a-1994 Standard

w.r.t. bumpers (potential confusion?)

8.10 Vehicle Emergency Controls and Devices

8.10.2 Mandatory emergency control functions and devices shall

include the following:

(g) sensing device or combination of devices to prevent

contact of the object sensed with the vehicle structure in the

direction of travel.

Yet, call-out specifically for bumpers:

Definition: bumper - mechanically actuated device, which when

depressed, causes the vehicle to stop

8.10.4 Bumpers: If used as emergency devices, bumpers shall not

exert a force greater than ...

Instead of this traditional bumper system, the AGV industry wants more reliability and more information

from bumpers; for example, look-ahead capability for decrease/increase speed regions and non-contact

emergency stop capability. Improved, non-contact bumper technology is available off-the-shelf (e.g., Sick

LMS) and can also provide some world-model information.

The Spring MHIA AGV Industry meeting included an AGV Working Group meeting on the first of two

days and a semi-annual Spring meeting on the second day that, focused not only on the bumper issue, but

also on AGV industry related issues. The Working Group meeting focused only on the B56.5 standard

regarding braking, object-detection, and/or bumpers. Several B56.5 sections were studied in the meeting

that posed some confusion to the reader, including Figure 6 points. These points will be the focus of the

first submitted change to ASME as agreed upon during the formal Spring meeting on Day two. Other

standard points not related to bumpers, but posed as suggested points to be changed were discussed as well.

These other standard points to be considered are shown in Figure 7.
20

Figure 7 - NIST IAV Project Presentation Slide that lists efforts and ideas for clarifying the ASME B56.5

standard, mainly with regard to bumpers and with points compared to the European standard EN1525:1997.

Bumper Standard Efforts and Ideas

• Support MHIA, AGVS Vendors: April Meeting - agreed to

submit update to B56.5 standard (allow non-contact sensors as

vehicle “bumpers.”)

• Upon approval from ASME, further clarification ideas include:

– Specifying what size object (test piece) to be sensed (e.g., EN 1525:1997;

5.9.5.2), and under what conditions

– Specifying slow-down and e-stop range w.r.t. vehicle velocity

– Toning down “bumpers” and instead specifying that objects must be

sensed (e.g., EN 5.9.5 detection of persons in the travelling path)

– Perhaps modeling parts of EN 1525 standard but, state smaller forces

(i.e., lower than 750 N (169 lb) to B56.5 force of 133 N (30 lb))

– System restart from emergency state after X amount of time

8.2 Other Standards Issues to Consider

Discussions with AGV users and vendors, as part of an IAV Project focus regarding other standards in use

on AGVs above the component level, conclude that there are no standards. De facto standards of hardware

computer platforms are just recently being considered. Limited interface standards or communication

standards appear to be used on these systems. Interface and communication standards would allow plugand-play capability of cross-vendor equipment, for example: computer processors, input/output boards,

integrated circuits, sensors, wiring harnesses, networks, and radio frequency boards are a few of these

standard interface and communications components. With these standards, users and vendors would not be

forced to buy new or replacement components from a single source allowing competitive, open-market

supply of vehicle components.

9 Advanced Technology Transfer

9.1 IAV Testbed

The Industrial Autonomous Vehicle Project requires a testbed to:

• show industry users and vendors NIST credibility in industrial, mobility applications,

• help NIST understand industry challenges (i.e., standards, measurement, and technology),

• provide NIST with a demonstration mechanism of advanced industry applications based on past and

present Mobility Program advances.

The testbed (see figure 8) includes a Labmate mobile robot with power conversion system, sensors

(cameras and LADAR), PC Laptop running Win2000 OS and interface wiring. Spread spectrum-based

Ethernet and multiplexed RS-232 radios were also mounted on the testbed and are planned for near-future

use as shown in the design in Figure 9.

Currently, the testbed is used for analysis of various sensors such as the Sick PLS LADAR sensor. Direct

technology transfer from the Army Demo III project was demonstrated by using the testbed controller to

build a world model from Sick sensor data. In the future, the IAV project is planning to combine the

vehicle controller with the data capture from the world model. The two software modules would
21

communicate via standard interfaces so that any equivalent sensor could be integrated into the system (see

Figure 1).

Figure 8 – Photo of the IAV Project testbed.

Figure 9 – IAV Testbed Near-Future Design. This new design will allow remote control and/or remote

web-based control of the vehicle to allow NIST, other research institutions, and industry to control the

vehicle and gather data as they choose.

Arlan

Data Radio

Labmate

Command/Status

SICK Ladar Data

(coincident with 

stationary camera

image)

Video Camera 

Pan/Tilt

Control

LAN ethernet radio

world

Pan/Tilt Video Camera

(for operator control)

ethernet

Map

goal

PC/Linux OS

Video Camera

with Pan/Tilt

TRC Labmate

CONTROLLER TESTBED

Laser Scanner

Stationary

Camera

Axis 240

multi-port

video server ethernet radio

power distribution

hub

Arlan Data 

Radio

010

and 120

(9600 bd.)

IR Camera

LADAR (Sick PLS)

[sensor PC not shown)

Pan-Tilt Camera

Labmate Mobile

Robot

Traditional AGV

bumper-level sensor

mounting structure

PC Laptop

(controller and

sensor interface)

Spread Spectrum

Data Radio

Power converters
22

9.2 LADAR Measurements

World modeling and mapping algorithms have been transferred from the Army Demo III and previous

military mobility projects to the IAV project. These algorithms generate maps off-line or in real-time using

only LADAR data and vehicle position and orientation knowledge as the vehicle moves along a

programmed path. A world model is built by commanding the vehicle to travel to various goal points and

storing the information for future use. An example of collected LADAR data and a map generated from this

data is shown in Figure 10.

The mapping algorithm uses the following constraints:

• 10 cm2 (4 in2) tile, grid-based elevation map. Maps can be regenerated with different tile sizes.

• Each map tile contains data including maximum, minimum & average elevation and classification.

• Each cycle includes update and fuse elevation data from the LADAR sensor.

• Display elevation map is 2-1/2 D (distance from floor to object height is filled).

Maps were generated in 2 1/2 dimensions. This means that there is only a single surface elevation at each

X, Y location. For example, in Figure 10, a railing is projected the entire distance from the railing down to

the floor. 2 1/2 D maps are used since they represent where objects and obstructions to the vehicle path are

located. For example, although there is clear space beneath the railing, the vehicle cannot enter this area

due to its height being taller than the railing. As shown in Figure 10 (right), the LADAR can detect both

the glass and even through the glass (note post away from hallway) in some locations.

Figure 10 – (right) Photo of testbed vehicle in glass corridor (arrows show direction of vehicle travel);

(middle) Single-scan Sick PLS LADAR data-capture and plotted using demonstration software provided by

Sick; (right) A map generated from multiple LADAR data scans (reversed data set from middle) and

displayed using MatLab software.

Future planned technology transfer from military and transportation projects includes path planning

algorithms as described in [Murphy, 2000]. This technology allows the vehicle to plan paths based on

information in the map and weights map blocks according to approved, disapproved and possible vehicle

traversal areas. As shown in Figure 2, the unblocked path through the left door could be a priori knowledge

stored by the vehicle controller as to whether the left path is an approved path should the forward path

become blocked. The path planner can provide this information as well as inform the vehicle that it must

make a turn with a particular radius given the current vehicle speed.
23

9.3 Other LADAR Research

9.3.1 Background

During the last few years, there has been a growing demand for low cost, non-contact, high resolution and

fast (real-time) scannerless 3D range imaging cameras. The needs expressed by industry, the military, and

other government programs have been for robotics, autonomous vehicles, construction automation,

automated manufacturing, topological mapping and automotive applications. Some of the identified uses

include: obstacle/object detection and identification, crash avoidance/path planning, indoor and outdoor

autonomous navigation and map generation, safety management and control and product integration and

dimensional control and inspection. Maris Juberts has been studying and tracking the development of

scannerless and Flash LADARs for about four years. He has been in touch with many of the developers and

they have been willing to provide him with information on the technical approaches being used, the

progress being made and possible dates for availability of prototypes and products.

Conventional Time-of-Flight (TOF) systems have been available since the 70s. They use mechanically

moving parts in order to generate a raster scan (steer a laser beam(s) over a target scene) and produce a

range image. Although conventional LADARs can provide high quality imagery, they generally have

limited frame rates and are large in size, heavy and expensive ($30K and higher). Range measurements are

typically made sequentially by steering the laser beam(s) over the target scene. These scanning mirrors and

assemblies are sensitive to vibration and shock.

Scannerless Laser Radar or Flash LADAR represents the next step in non-contact range imaging

technology. They use pulsed or modulated lasers to floodlight illuminate the target scene – this is similar to

using an electronic flash to take a snapshot with a photographic camera. The TOF of the laser illumination

source to various surfaces in the scene is measured in parallel by each pixel of the detector to generate a

range image. Because of the parallel measurements, very fast frame rates can be achieved enabling small,

lightweight, low voltage, power-efficient sensors to be built.

The characteristics of Scannerless LADAR make them ideal for Mobile systems.

• They have no moving parts,

• Produce range images at high frame rates (up to many kHz),

• Some have demonstrated high resolution (640 X 480),

• Most have B/W reflectance image which is registered at pixel level,

• Have achieved good ranges. 100 m or more for navigation and several kilometers for target detection,

• Are more rugged than scanning LADARs,

• Are lower in weight, power consumption and cost,

• Produce little or no image blur and are highly programmable.

Of utmost interest is the anticipated cost of such sensors. Most of the developers that we have contacted

mention a target price of $2000 or less per unit in large quantity production. This seems to be consistent

with the needs of AGV and servicing robot manufacturers. Some of the developers working on LADAR

sensors for automotive applications have put the cost estimates as low as a few hundred dollars per unit.

9.3.2 First Cut at a Requirements Specification for a Scannerless Ladar

The following is a first cut at a requirements specification for a scannerless LADAR to be used for

Intelligent Unmanned Ground Vehicle applications in the military. Requirement specifications for an

Industrial Autonomous Vehicle may be different (lower maximum range, lower weight, may not need to

operate in sunlight, may not need an adjustable light output, and may not need such a wide temperature

operating range.)

- Range: Minimum< 2 m, Maximum > 50 m (100 m desired)

- Range Resolution: < 0.07 m (0.03 m desired)

- Range Accuracy: +/- 0.03 m

- Angular Resolution: < .004 rad (0.25°) {0.0017 rad (0.1° desired)}
24

- FOV (field of view): - Azimuth ~ 2 rad (120°) Elevation ~ 0.5 rad (30°)

- Frame Rate: > 10 Hz (20 Hz desired)

- Wave Length: (900 nm to 1500 nm)

- Power: < 50 W

- Weight: < 13.6 kg (30 lb)

- Operate in full sunlight

- Adjustable light output to reduce probability of detection

- Temperature Operating Range: -40 °C to +85 °C

9.3.3 Research Labs Doing Work in Scannerless LADAR

The following sub-section describes the work and progress of some of the laboratories and companies

developing scannerless LADARs. These are developers who have been contacted by Juberts and have

demonstrated working prototypes in the lab.

MIT/Lincoln Labs

• Basic Description: Geiger - Mode Avalanche - photodiode array detector bonded to CMOS Timing

Circuitry for each pixel and using a solid-state Fiber Pumped Laser for pulse illuminations of the

scene. Each pixel measures TOF directly.

• Brass Board:

- Major components

30 J, 1000-pulses per second microchip laser (1064nm, shifted to 532nm)

4 x 4 APD array with timing circuitry

- single photon sensitivity

- low dark-count rate

adjustable optics

scanning mirrors to generate up to 128 x 128 pixels per frame

- Range

500 m with 30μJ/pulse

- Range Precision

2 cm (1 )

• FY 2000 Activities

- complete development of 32 x 32 fully hybridized array

- measure performance of new array

- design and build brass board system to utilize new array in non-scanning mode

- collect high frame rate range data on moving objects (several thousand fps)

- conduct multi-sensor data fusion experiments (vibration sensing laser, IR,

 hyperspectral imaging)

- eye-safe (1.5 μm) APD technology development

• Future Development Activities

- compact camera development using 32 x 32 array detector (FY01)

- possibly build compact camera for a 128 x 128 array detector

- Field testing of compact cameras

• Range Images (Fig. 11 and 12) taken with APD Laser range Imager
25

Figure 11 – Range Image taken with APD Laser Range Imager of a Ford Bronco automobile with wires at

100 m (3 μJ/pulse, 128 X 128 pixel range image.)

 

Figure 12 – Range Image taken with APD Laser Range Imager of a Chevy Van at 60 m in bright sun (3

μJ/pulse, 128 X 128 pixel range image.) Recorded image is rotated as if viewed from different aspects.
26

Daimler-Chrysler Aerospace AG (DASA)

• Method of Range Measurement: Indirect Time-of-Flight

• Basic Approach

- Scene is illuminated by a series of laser pulses (10 ns to 80 ns in direction)

- A micro-lens assembly spreads illuminations over a desired FOV

- A standard high resolution CCD camera (having a high-speed electronic shutter) is

used to collect back-scattered light whose travel time is dependent on distance to objects in the

scene

- Three images are taken at slightly different times

- measure ambient illumination with laser off

- after the laser is pulsed, a distance image is captured

- a gray-scale image is captured over the entire pulse sequence

- To reduce effects from laser power and surface characteristics, a quotient image is computed by

dividing range image by intensity image (after ambient image is subtracted from gray-scale image)

- Final range image is found through a look up table created during calibration

• Technical Data

Operating Range (Depends On Optics Used) 0.5 m to 30.0 m

Depth of Field (Depends on Optics Used) 1.5 m to 15 m

FOV (field of view)Horizontal 0.73 rad, 0.37 rad (42o, 21o)

Vertical 0.56 rad, 0.28 rad (32o, 16o)

Lateral Resolution 640 x 480

with hor./ver. Binning 320 x 240

Image Frequency 6.7Hz for max resolution

10Hz with hor./ver. binning

Laser Illuminator five solid state laser diodes

(850nm, 30 W each max)

Light Pulse Duration Is Adjustable 10 - 80ns

Uses a Quick Electronic Shutter to Achieve Combined Time of Flight Intensity Measurement

Range Image Measurement Time 150ms

• Picture of Range camera (Fig. 13-left) and Range image (Fig. 13-right) taken with camera

Figure 13 – (left) Photo of DASA Camera and (right) Range Image taken with DASA Camera. Range

image shows a poster board as a target at 8 m distance in a NIST distance measurement calibration lab.

CSEM (Zurich, Switzerland)

• Method of Range Measurement: TOF (time-of-flight) - by measuring phase delay of modulated light

reflected from the target scene.

• Basic Principle: Combines fast detectors with electronic demodulation circuit at each pixel. Pixels

have been fabricated using modified CMOS process which allows for overlapping CCD gates.

• Prototype Range Cameras:

500m W LED light source modulated with 20MHz
27

108 pixel line camera - uses lock-in CCDs

25 x 64 pixel area camera - uses lock-in CCDs

Integration time per range measurement – 50 ms 20 Hz frame rate

Current max range: 20 m

Range resolution: +/- 2 cm to 3 cm

• Future Activities:

To reduce background illumination effects (sunlight) - replace LED flasher with higher power laser

diode flasher

• Picture of Camera (Fig.14) and Range image (Fig. 15) taken with camera

Figure 14 – Photo of the CSEM Range Imaging Camera (prototype) 25 X 64 pixel array.

Figure 15 – Range Images taken with CSEM Camera. This strip of five range images, taken with the 25 X

64 pixel camera, shows a person entering a room through a door (frame rate is 7.5 Hz).

 
28

Sandia National Labs, Department of Energy

• Method of Range Measurement: Modulated laser illumination and a focal plane receiver to phase

encode the laser TOF for each pixel.

• Basic Principle: The reflected laser light from a scene is imaged onto the cathode of an image

intensifier tube.

- The intensifier optical gain is sinusoidally modulated at the laser modulation frequency.

- A fiber optic taper couples the intensified image (phosphor screen) to a CCD detector.

- The modulation enables the sensor to produce a range image from successively 

acquired intensity images. A minimum of three are required.

- A sine wave is then fit to the intensity values for each pixel and used in phase delay

measurement.

• Current Camera: Built for NASA Johnson Space Center

- 640 x 480 pixels at 7.5fps.

- 150 ms per range measurement

- 40° FOV

- Absolute range accuracy of 2.5 mm (0.1 in) (average of 6 measurements) at 45.7 m (150 ft)

interval

- 12 W, 805 nm diode laser

- 2.3 kg (5 lb), 58 mm x 150 mm x 292 mm (2.3 in x 5.9 in x 11.5 in), 28 VDC (18 W)

• Next Version: Improve frame rate to 30fps - to reduce image blurring and errors due to motion.

Replaces single CCD detector with four detectors - for parallel processing.

• Long Term: Develop solid state focal plane array detector which takes snapshot images at 30 fps.

• Picture of Camera (Fig. 16)

Figure 16 – Photo of Sandia National Labs Camera

9.3.4 Other Developers

The following is a slide from Juberts presentation, entitled: Status Report of Next Generation Laser Radar

systems, October 5, 2000, at a Primus Workshop at Dornier in Friedrichshafen, Germany.
29

S-TEC

Siegen, Germany

Contact

Dr. Bernd Buxbaum

Buxbaum@nv.et-inf.um-seigen.de

Concept

Measures phase delay of

reflected light from laser

which has been modulated

with a square-wave or

pseudo-noise. Have

developed a novel sensor

element (Photonic Mixer

Device) uses fast

correlation process to

measure phase delay.

Characteristics

Sampling rate achieved

2ms

range currently: 7.5m

accuracy ~ 2cm

Radiation

Monitoring Devices,

Inc.

Boston, Mass

Arieh Karger

Akarger@rmdinc.com

Uses APD detector to

directly measure TOF

range

Very similar to Lincoln

Labs approach

ARL

Adelphi, Maryland

Bill Ruff FM/CW Ladar Developing 1 x 32 array

with 10kHz fps

Concluding remarks: In addition to the developers mentioned above, there are now other research labs

developing scannerless LADARs. These will be contacted over Summer 2001. Juberts is trying to

determine the availability of such sensors for integration and testing on the NIST IAV testbed vehicle

(described earlier.) Prototypes are expected from CSEM and S-TEC, possibly even in 2001.

10 Planned Project Activities

• Continue site and conference visits to provide industry direction to NIST research and publicize NIST

efforts.

• For example: ASRS/AGVS Users Conference, Columbus, OH, June ’01.

• Continue standards efforts based on industry discussions and suggestions for standards (e.g., noncontact bumpers, interfaces)

• Continue testbed efforts toward LADAR-mapping technology transfer to industry,

• Combine testbed controller module with mapping algorithm module on one system PC.

• Transfer planning algorithm module to testbed system controller PC.

• Host workshop at NIST for AGV users in FY2002. Then invite both AGV users and vendors.

• Initial workshop will invite AGV users and potential users

• Second workshop will invite AGV users, vendors and associated organizations.
30

11 References

[1] Albus, James; 4-D/RCS Reference Model Architecture for Unmanned Ground Vehicles, Proc. of SPIE

Vol. 3693, Aerospace Session on Unmanned Ground Vehicle Technology, April 7-8, 1999.

[2] Deng, X., Milios, E. et al.; Robot Map Verification of a Graph World, Department of Computer

Science, York University, Toronto, Ontario, Canada, April 12, 1998.

[3] Deng, X., Milios, E. et al.; Landmark Selection Strategies for Path Execution, Department of Computer

Science, York University, Toronto, Ontario, Canada, March 17, 1997.

[4] Evans, John, Krishnamurthy, Bala; HelpMate, The Trackless Robotic Courier: A Perspective on the

Development of a Commercial Autonomous Mobile Robot, p182 in Autonomous Robotic Systems, Ed.

Anibal T. de Almeida and Oussama Khatib, Springer Verlag Lecture Notes in Control and Information

Sciences 236, 1998.

[5] Evans, John, Transitions Research Corporation and Borenstein, Johann, U of Michigan; Innovative

Design for a Four Degree of Freedom Mobile Robot, Department of Energy SBIR Phase 1 Final Report,

Contract No. 20418-92-II, March 1993.

[6] Integrated Manufacturing Technology Roadmap (IMTR), (draft) Matrix for Processes and Equipment,

Intelligent Systems Division, National Institute of Standards and Technology, December 2000,

[7] Konolige, Kurt; SRI International, SenseNet: Immersive Vision for Small Robots presentation,

http://www.ai.sri.com/~konolige/sensenet/index.htm, June 2000.

[8] Murphy, Karl, et al.; Intelligent Control for Unmanned Vehicles, World Automation Congress

Conference, Maui, HI, June 11-16, 2000.

[9] Raviv, Daniel and Herman, Martin; A “Non-Reconstruction” Approach for Road Following, Florida

Atlantic University and the National Institute of Standards and Technology, October 1991.

[10] Redwine, Kirk; Automated Guided Vehicle Study, Lockwood Greene Engineering and Construction,

December 2000.

[11] Schneiderman, H., Nashman, M., Visual Processing for Autonomous Driving, Proceedings of the

IEEE Workshop on Applications of Computer Vision, Palm Spring, CA, November 30-December 2, 1992,

(1993)

[12] Simmons, Reid, et al.; Xavier: Experience with a Layered Robot Architecture, Carnegie Mellon

University, 1997.

[13] Thrun, Sebastian, et al.; Minerva: A second generation mobile tour-guide robot, Robotics Institute,

Carnegie Mellon University, Proc. of the IEEE International Conference on Robotics and Automation,

1999.

12 Acknowledgements

The authors would like to thank Intelligent Automation, Inc., for their assistance with preliminary LADAR

experiments and Tsai Hong and Tommy Chang for their LADAR and mapping experiments using the NIST

IAV Testbed. Also, the authors would like to thank all information sources, including: AGV vendors,

users, mobile robot vendors/researchers, Material Handling Institute of America, other research venues

listed herein, sensor researchers, and the ASME B56.5 standards committee who provided invaluable input

to this research.
31

13 Appendix: Other Project Materials

13.1 Sensor Types and Mount Ideas

The following are presentation slides that list some sensor types and show potential sensor mounting

locations.

Some Sensor Types to Consider for Use

as/with AGV Bumpers

To provide more info. than contact sensors alone

• Ultrasonics - acoustic range

• Vision - image

– 2D, stereo, optical flow, structured light

• LADAR - Laser Detection and Ranging

– 1D single line, 2D raster-scan, flash

• LIDAR - Pulsed Light Detection and Ranging

• Microwave - radar to object within set range

• Combinations - e.g., color vision/ladar (CMU)

• Others??
32

Sensors/Mounting Ideas

Mount

Fixed-array sensor(s)

- e.g., ultrasonic detects

clear/obstructed path data;

- much data on this concept

Combo. Sensors examples:

- Front and Rear ultrasonics

(long for fast speeds)

- or Rear mech./other (slower

speeds)

dc/dc

24/12

PWM

Amps

battery

battery

top

NIST IAV Testbed -

Labmate Mobile Robot

45"

(1.1 m)

Sensors/Mounting Ideas

(to allow more/enhanced vehicle applications)

Mount

- Fixed-horizontal

- Fixed-tilt: collects

more path data

dc/dc

24/12

PWM

Amps

battery

battery

top

Combo. sensors:

- Front, Rear ladar

 (long range for fast speeds)

- or Rear mech./other

 (short range for slower speeds)

4"

(sensor max. range)

11-1/16"

0"

10°

20-1/4"

(10 cm)

(28 cm)

(51 cm)
33

Sensors/Mounting Ideas

Advanced Mount/Sensor (combined with bumper?)

- Controlled-tilt: collects much more path/surrounding info.

- Or 3D Ladar without tilt

3-7/8"

(sensor max. range)

minimum vehicle height clearance

47.7°

(9.8 cm)

(0.83 Rad)

IAV Project: Ladar Data at Bumper Height

• Fixed-Horizontal or Slightly Angled Ladar Mounting

• Info. is limited at this sensor mount height, but could

be dual-used as bumper.

1-1/2"

4"

SlightlyAngled Ladar 102 mm 38 mm

 (4”) (1.5”)
34

13.2 Potential AGV Capabilities Ideas

The following are presentation slides showing advanced applications suggested for AGVs and mobile

robots using demonstrated NIST and other researcher-tested advancements.

Some Potential AGV Applications

• Shipbuilding

• Aircraft manufacturing

• Automobile manufacturing

• US Post Office

• Military Storage/Warehousing

• Assembly-line worker (mobile vehicle with arm (or pair))

.... Most places, structured or unstructured, that require

people to frequently retrieve parts, materials, tools for use

within a reconfigurable, long distance environment (a

helper for a worker or set of workers).

Shipbuilding Potential AGV Uses

Stock Materials Input

Manufacture Assemblies

Parts

storage

building

Parts

distribute

building

Truck unload

Machine

repair/

small

assembly

machining

Building

obstacles

material

delivery

truck

bridge/trolley cranes

AGV's

Buildingto-building

vehicle

navigation

as needed

(i.e., when

called by

worker.)
35

Shipbuilding

AGVs continuously feed the manufacturing process

For example, they could deliver: • weld process materials • plasma-cut materials • small steel pieces • machined parts

AGVs can distribute goods that are delivered from a

relatively long distance away (storage, distribution

areas) to drop-off stations within the

manufacturing facility and directly to or near

workers.

Aircraft Manufacturing

AGVs continuously feed the manufacturing process

For example, they could deliver: • small parts bins - process materials • large parts - engines

• continuous flow of parts, tools, and equipment

from storage

• parts could be from outside of manuf. facility

 (from trucks, storage buildings, ISO containers)
36

For these and other similar

scenarios, AGVs require:

• retrofit to existing facilities (need clear paths between

aircraft assembly areas/dual-use worker paths.) • relatively smooth to uneven and/or inclined mobility • system knowledge (e.g., are parts there for pick-up

before it drives to get them or can it do something else?) • logistics of other vehicles, tray-tables, pallets • work around algorithms • safety system to work with people • system interrupts from prioritized worker requests • potentially unstructured navigation closer to

aircraft and between buildings

parts, materials, tools, 

cleaning supplies, ... 

can be brought

directly to workers

instead of "near" worksite

beacons mounted on traytables/pallets can tell AGV's 

their location, including: 

waste bins, rag bins, ..., 

to be emptied or removed

frequently from the site

for efficient manufacturing 

obstacles are avoided 

with work around solutions

providing little delay

of package drop-off

(the same vehicle-clearance 

paths are required 

as for workers)

portable AGV stations

can be set-up so that 

stations move with the 

manufacturing process

New Concept: Aircraft

Manufacturing Potential AGV Uses
37

Vehicle Modification for

Alternative Uses

6 DOF Manipulator

attached to AGV

aircraft engine

being attached

to wing

aircraft engine

at transport

height on AGV