Derived Performance Metrics and Measurements

Compared to Field Experience for the PackBot

Tom Frost, Christopher Norman, Scott Pratt, Brian Yamauchi

iRobot Corporation, 22 McGrath Highway, Suite 6, Somerville, MA 02143

Bill McBride, Greg Peri

South West Research Institute, San Antonio, Texas

ABSTRACT

Preparing an Unmanned Ground Vehicle for missions

in abusive, dangerous environments requires suitable

tests to define the system capabilities. Well-designed

performance metrics can provide the government and

industry designers with an understanding of how the

system should be used in the field and how the system

can be improved. This paper describes the metrics and

measurements used for testing the PackBot system and

compares those metrics and measurements against

insights gained in field experience.

I. INTRODUCTION

The PackBot System, shown in Figure 1, is a

ruggedized, man-transportable Unmanned Ground

Vehicle system that provides a remote presence in

dangerous locations. Reconnaissance and manipulation

of a remote environment can be performed while the

operator remains safe. The PackBot was designed

primarily for Mobile Operations in Urban Terrain

(MOUT). Designs for situational awareness

capabilities and obstacles

negotiation capabilities are driven

by anticipated urban combat

scenarios. The MOUT

requirements have resulted in a

system that also has many

applications in other dangerous

combat operations and Urban

Search and Rescue (USAR)

operations.

To prepare the PackBot for the

hazardous duties that it will

encounter in the real world, the

PackBot has been tested at iRobot’s

facility and at the Small Robotic

Vehicle Test Bed at the South West

Research Institute, SwRI, at San

Antonio, Texas. The PackBot

system was also exercised at the Army simulated

MOUT city at Fort Drum, New York. The predecessor

to the PackBot system was tested at a testing ground in

Rockville, Maryland. These tests helped to understand

and measure the performance of the PackBot before the

system was used in the real world. The real world

scenario where the PackBot was employed was

operations at the World Trade Center (WTC) disaster

site. These experiences at the WTC disaster site held

many lessons for future Unmanned Ground Vehicle use

in the real world. This paper will look at the derived

performance metrics and compare these to field

experience.

II. PACKBOT HISTORY

The PackBot was developed by iRobot Corporation

under DARPA’s Tactical Mobile Robotics (TMR)

program. iRobot began developing mobile robots for

the TMR program in 1997 by creating a proof-ofconcept robotic platform designed for MOUT called the

Urban Robot. The Urban Robot was designed to be a

small man-portable surveillance robot that could

negotiate urban terrain. Under

subsequent DARPA contracts, the

Urban Robot platform became

continually more rugged and

sophisticated. The later versions

incorporated sonar and infrared

rangefinders with a more powerful

CPU for onboard sensor

processing. Under the PackBot

contract, iRobot was assigned to

develop a more robust and

complete robotic system capable

of surviving the abuses of real

operations. Developing the

PackBot became one of the

primary focuses of the TMR

program and iRobot Corporation

was selected as the system

integrator.

Figure 1. PackBot System.
III. MOBILITY

The PackBot is a tracked robot vehicle designed for use

in both urban and wilderness environments. PackBot is

equipped with two main treads, used for locomotion,

and two articulated flippers with treads that are used to

climb over obstacles The PackBot can be fitted with

extra treads for additional mobility, see Figure 2. Since

the robots developed under DARPA’s TMR program

were primarily designed for urban environments, this

commonality between USAR and MOUT is a large

contribution to ease of mission transferability.

Urban environments typically include open spaces such

as city streets and building interiors. Common obstacles

that robots encounter in urban environments include:

curbs, stairs, small rubble piles, pipes, railroad tracks,

furniture, and wires. The ability to surmount these

obstacles is essential to the success of these platforms.

The PackBot system was capable of traversing all of

terrain encountered at the MOUT city at Fort Drum.

With very little training, operators were able to drive

the PackBot up stairs and through doorways. However,

the MOUT city did not have piles of rubble that would

be encountered from buildings that have been damaged

in explosions or earthquakes.

The SwRI tests consisted of outdoor obstacles. Many

of the obstacles, such as pipes and rubble piles, are also

informative for predicting performance in urban

environments. The obstacle course at the SwRI site

consisted of various natural and man-made obstructions

selected as a representative subset of robot-scale

impediments to cross country movement. The

following list is a description of the obstacles used for

testing the PackBot and a brief description of the

PackBot’s performance with each obstacle:

• Railroad ties. No difficulty traversing

• Pipes of ten different diameters ranging from 1.25

inch to 9 inch. The robot had no difficulty

traversing these pipes. The robot traversed the

largest pipe by lifting itself onto the pipe using its

flippers and doing a “backflip”.

• Drainage Culvert. 24-inch wide culvert with two

45-degree bends. No difficulty traversing.

• Bamboo Forest. The bamboo forest consists of a

matrix of 2 inch PVC pipes on 6 inch centers. The

maze width is one pipe-width larger than the width

of the robot. Under tele-operation, the PackBot

failed to get through the maze on the first try in

under the 20 minutes time limit. On the second try

the robot was able to pass through the maze in 17

minutes using lessons learned from the first

attempt and using the pose capabilities of the

PackBot.

• Rock channel. The obstacle has rocks the size of a

typical man-packable robotic vehicle. The obstacle

was traversed without any problems. The

articulators, power, and low center of gravity of the

PackBot contributed to the successful negotiation.

• Large, medium, and small rock beds. These beds

are populated with rocks that are football sized,

softball size, and hockey-puck sized, respectively.

The robot was able to traverse all three beds

successfully. However, it went out of bounds (off

the bed) in one out of the six runs.

• Dirt furrows. The furrows were dry, loose dirt

formed into ridges in a 7 ft wide by 30 ft long

obstacle. The robot had no difficulty moving

through this obstacle.

• Vegetation obstacle. The course was divided into

four sections with crops ranging in size from lawn

grass to heavy crops, greater than 18 inches high.

The light and medium crops were traversed with no

difficulty. The heavy crops made the direction of

the PackBot difficult to determine and the PackBot

moved outside the course in several of the runs.

The articulated head/neck unit being developed for

Figure 2. PackBot with additional flippers.
the TMR program will make navigation in thick

brush easier.

• Flat Sand Pit. The pit was filled with dry sand and

the robot transitioned the pit without difficulty

• Sand Furrows. Dry sand was formed into sand

dunes. The robot had no difficulty with this

obstacle.

• Mud pit. A pit 4 in by 7 ft by 16 ft was filled with

a mud slurry. Figure 3 shows the PackBot after

traversing the mud pit. The PackBot could go

straight through the mudpit. If the PackBot turned

6-8 times in the center of the pit, the PackBot

would become mired. The SwRI report stated “the

robot has very good mobility in mud, despite its

failure to complete the entire test.”

• Inclined ramp. The ramp is adjustable from 0 to 60

degrees. With no payload, the PackBot climbed

the ramp, was able to hold position, and was able

to skid steer in both directions. With 22.5 pounds

of payload, the PackBot was able to ascend and

descend up to 55 degrees and able to traverse 45

degrees.

• Curb Height. The PackBot climbed 13 inch curbs.

• Cattle Grating. Metal pipes, 2 inches in diameter,

can be moved to different positions. The PackBot

traversed this obstacle at all possible gap settings

(other than unreasonable ones spaced farther apart

than would be expected in an actual animal guard.

• Stairs. The stairs were sets of: wooden 9 inch

risers and 11 inch runs, wooden 7.5 inch risers and

11 inch runs, metal 6.5 inch risers and 12 inch

runs. The PackBot was able to climb all of the

stairs.

• Speed Runs. The speed tests recorded an average

(cruising) speed of about 5 mph. With its power

booster, the PackBot can achieve burst speeds over

8 mph.

The SwRI tests provided the PackBot designers with

excellent information on the PackBot’s mobility

characteristics tested against precise metrics. Results of

the tests were very positive and there was no critical

design changes required because of mobility

shortcomings.

Since disaster areas and urban terrain are covered in all

types of debris, having as much mobility as possible

greatly increases the success of USAR missions.

Although tracks provide the core of essential mobility,

having additional modes of mobility is advantageous.

When negotiating rough terrain, robots often flip over.

The PackBot’s flippers enable it to perform selfrighting. The flippers on the PackBot also provide

extra mobility, since they are coupled to the main drive

tracks, creating a larger adjustable contact surface. The

articulated flippers help prevent the robot from being

immobilized due to high-centering, enable the robot to

climb taller objects, and can help propel the robot

forward through dense vegetation through continuous

rotation.

At the WTC disaster site, the terrain where USAR

operations took place could be divided into two types,

the rubble pile, see Figure 4, and buildings for

clearance. The rubble pile was not something that these

robots had previously encountered and the robots were

not specifically designed for the rubble pile terrain. The

extreme conditions presented the robots with problems

in various areas. The wreckage site had such an

incredible amount of debris that mobility was very

difficult. The huge pile of twisted steel was challenging

for humans to climb, and more difficult still for robots

Figure 3. PackBot after Traversing the Mud Pit.

Figure 4. World Trade Center Rubble Pile.
to negotiate. The mere size of the pieces of steel could

be insurmountable. It was very difficult for small

mobile robots to traverse such an environment. The

PackBot was not able to negotiate the rubble pile

except for specific places. For the most part, there were

no crevasses in the rubble pile big enough for the

PackBot to explore. Usefulness of the PackBot was

demonstrated in the surrounding area for building

clearance.

The sites requiring clearance and inspection around the

rubble pile were strewn with paper and debris. A layer

of dust several inches thick covered the area. The

PackBot’s debris rejection system on the treads was

successful, allowing the PackBot to drive through large

amounts of debris. This system was previously tested

at SwRI with the small rock bed, dirt tracks, and rock

channel. The PackBot did not detrack in the WTC

operations. The PackBot was used to demonstrate

building clearance in the buildings surrounding the

WTC. A building clearing operation was performed in

an area that had been previously cleared by rescue

workers and the PackBot system was shown to be

useful for clearing buildings. The PackBot moved

through the environment looking for people in the

buildings and inspecting structural integrity of

staircases. USAR personnel at the WTC observing the

demonstration deemed the building clearance capability

useful. The PackBot moved through the buildings

including many staircases without difficulty except for

one staircase. The problematic staircase was covered

with dust and the metal was slippery such that the

PackBot was not able to get traction on the steps.

The WTC disaster highlighted that specific terrain

encountered with USAR operations is difficult to

predict, but having a well designed general mobility

capability that will climb rubble, self-right, climb

staircases, and is maneuverable enough to go through

hallways is useful in many scenarios. The designers of

the PackBot feel that the terrain such as the rubble pile

could not be negotiated without a large increase in

complexity and expense in the mobility mechanisms.

IV. DURABILITY

The PackBot was designed from the ground up with

considerations for impact resistance, waterproofing,

vibration resistance, electromagnetic resistance, low

electromagnetic signature, and a wide operation

temperature range. The SwRI tests examined the

following metrics.

• System Shock. The system survives shocks of 400

G’s. 400 G’s translates to a 10 ft drop onto

concrete. The system was fully operation after

multiple drops from 10 ft in positions of rear,

forward, and belly down.

• Waterproof. The system is rated for water depths

of up to three meters. At SwRI the system was

tested in a water channel just covering the

PackBot, see Figure 5, and in a pond with depths of

up to 5 ft. The PackBot had no problem with the

water tests.

• Electromagnetic. The electromagnetic resistance

capabilities of the PackBot are restricted from

public release.

• Temperature Range. The known operation range

of the PackBot is 20 degrees Fahrenheit to 120

degrees Fahrenheit. While the temperature range

was not explicitly tested at the SwRI tests, the

ambient temperature reached 116 degrees

Fahrenheit.

The PackBot has an onboard health sensor suite that

sends messages back to the Operator Control Unit on

the status of the system. Examples of messages sent

back to the operator are the thermal sensor readings

throughout the system. The system automatically

scales back power to motors if the motor is in danger of

damage from overheating. The operator can override

the thermal protection in the case of a critical mission.

The waterproof capability makes the PackBot more

practical for use in the field. The system can be used in

rain and can be cleaned with a water hose. The system

is also sealed against dust. Many of the combat

situations where a robot would be used, such as in

Afghanistan, are very dusty environments.

Figure 5. PackBot Emerging From Water Test
At the WTC, the PackBot’s durability contributed to

the systems usefulness. The general ruggedness of the

PackBot translates to less maintenance and more

readiness. The results from the SwRI tests indicated

that the PackBot would be a system with excellent

durability and throughout the WTC deployment, the

PackBot proved to be a durable system.

V. COMMUNICATIONS

The PackBot requires a single 802.11b digital link from

the Operator Control Unit to the PackBot vehicle. This

link carries the real-time digital video stream as well as

the other status information from the robot. In testing,

the PackBot’s communications link has reached

distances greater than one kilometer line of sight. For

missions where the system will not be able to maintain

line of sight or in an electro-magnetically sensitive

environment such as an ordnance deactivation mission,

the PackBot system has a fiber-optic spooler that

releases fiber optic cable and draws the cable back in.

The PackBot also has a payload for RF amplifiers for

extended range RF missions. The payload system

allows other communication systems to be developed

and integrated in a payload slot with ease.

The WTC area, in particular the area directly on the

rubble pile, presented significant obstacles for RF

communication. The rubble was heavily strewn with

attenuating metallic debris and the airwaves were

awash with RF radiation from a multitude of radios and

equipment. For these reasons, and the fact that tether

operations guarantee full frame rate video feedback, the

tethered robot systems were used almost exclusively on

the WTC rubble pile and were inherently more reliable

during operations. Tests that demonstrate a robots

ability to pay out its tether and keep the tether from

becoming ensnared would be useful. Radio Frequency

interference was anticipated as a major and crippling

issue once the robots penetrated deep within the rubble.

Exercises were conducted with RF-controlled robots in

the area directly surrounding the rubble pile. These

areas consisted of blown out, unsafe buildings, wrecked

vehicles, and rubble-and-dust-strewn streets. Most of

the buildings in this area had not collapsed and, as a

result, the areas of robot operation were not as densely

congested with metallic debris. This permitted the

freedom of RF operation well inside many buildings

that were structurally unsound or not yet deemed safe.

Although the robots operated freely within many areas

of these structures, the operations were conducted with

the knowledge that any robots losing communications

within unsafe structures were to be considered

irretrievable. The dynamic and intermittent nature of

RF communications highlights the need for robot-based

autonomy to assist the operator during communication

blackouts. iRobot is working on autonomous behaviors

that detect these situations and react with algorithms

designed to navigate or backtrack the robot platform to

areas of better communications. Test ranges testing

this ability would be of value. In addition, iRobot is

working on the ability to use multiple robots relaying

communications one-to-another from deep within a RFimpenetrable building. Tests that evaluate this

capability would be useful.

VI. SITUATIONAL AWARENESS

Situational awareness, in the context of PackBot

missions, refers to the ability of the operator to

understand the environment that the PackBot is

exploring. This understanding depends on the

particular mission and may include:

• Presence of disaster victims, enemy fighters,

friendly fighters, civilians.

• Medical condition of detected people.

• Location of Booby traps and mines.

• Integrity of building structure.

• Layout of the area.

• Location of items of interest within the area.

Test courses specifically designed for the above aspects

of situational awareness, would be useful.

Selecting the most appropriate sensors for robot

situational awareness and understanding how to exploit,

merge and interpret the various data provided by the

sensors is an extensive area of research. The PackBot

can be equipped with various sensors including:

cameras, sonars, infrared sensors, and laser scanners.

The primary source of information for situational

awareness on the PackBot has typically been standard,

low-light, or infra-red video cameras.

The PackBot is equipped with a differential GPS

system. At the test course in Rockville, autonomous

waypoint navigation was demonstrated, see Figure 7.

Using only the GPS system, the Packbot demonstrated

back-tracking through maze-like courses resembling

minefields.

The PackBot is equipped with a compass, roll sensor,

tilt sensor, and 3-axis accelerometers. These sensors
increase situational awareness by providing the

operator with additional information on the PackBot’s

status and position

Many of the tests conducted at SwRI indirectly tested

some aspects of situational awareness. For example,

the vegetation test, examined the PackBot’s ability to

determine where it was so that it could move through

the vegetation course.

At the WTC disaster, the PackBot used cameras as the

main source of data for situational awareness. Both

color and b/w video was used. As expected, the color

was beneficial for the operator’s understanding of the

PackBot’s environment. The PackBot also used a

forward-looking infrared (FLIR) camera. The FLIR

was useful in both dark and light environments. Since

the environment at the WTC was covered in a thick

dust, trying to discern various objects and details of the

environment became difficult. The gray dust tended to

mute all colors, but having a thermal based view of the

world provided an alternate perspective on the

immediate environment. Certain aspects of the

environment were not readily evident using visible light

or low light cameras; however, thermal imaging made

these details apparent. The FLIR was particularly

useful in detecting the presence and location of people

in low-light environments. Tests examining the

operator’s ability to discern details through a camera

would be useful.

VII. DEPLOYMENT

The deployment of a robot system can be broken down

into:

• Delivering the robotic platform to the deployment

area

• Setting up the control system

• Operating the robot

All of these areas must be well thought out for an

Unmanned Ground Vehicle’s mission to be successful.

The PackBot was developed for ease of deployment.

The PackBot is shipped in a padded case that can be

lifted by a single person. The Operator Control Unit

(OCU) and battery chargers are shipped in a separate

case. The robot is deployed by pulling the base chassis

out of the case and pushing the flippers onto the

chassis. The deployment does not require any tools.

The base chassis without batteries weighs 28 pounds so

the pieces of the robot can be distributed among a team

for carrying to the operation site. Ease of deployment

of the PackBot was not tested in a formal setting.

Our experience at the WTC served as an education in

the deployment issues associated with real emergency

situations. A wide range of scenarios was present at the

disaster site, requiring different deployment strategies.

Automobiles were used to transport the robot systems

and operators from the Javitts Convention Center to the

WTC site. At other times automobiles were used to

transport the robot systems and operators to a standoff

point from which the operators then transported the

system to the operation area. Not all areas were easily

accessible and some required the PackBot to be carried

on foot for long distances (up to approximately one

mile). The robot systems needed to be carried over

debris and rubble that the robot themselves could not

traverse. This highlights the need for robot systems to

be lightweight, compact, and man-portable. In other

cases, operators carried equipment and robots in their

arms and strapped to their backs while riding on ATVs

that ferried them to the operation area. The robots did

not deliver themselves to the operation area to conserve

battery life and because some of the operating areas

were not directly accessible.

After arriving at the area of operation, the control

system had to be set up. Control stations for the

PackBot at the WTC consisted of a human-machine

interface of joysticks and buttons for sending

commands to the robot. At the time of the WTC effort,

the OCU hardware was not rugged or weather-resistant.

This resulted in instances where the robot was not used

for fear that the system’s components were inherently

unreliable. Since that time, the OCU software has been

ported to more rugged OCU hardware, including a

wearable OCU that has been used in military exercises.

Having a man-portable OCU was shown to increase the

type of missions where the PackBot could be used.

Figure 7. GPS Waypoint Autonomous Navigation
Experience at the WTC illustrated the need to transport

the PackBot, OCU, and any other equipment, by foot to

the area of operation and the necessity for it to be setup

in a short time. A well thought-out container with

intuitive locations for each piece of equipment is a

necessity. These containers should be capable of being

roughly transported in the back of a truck and carried

easily. Minimizing the number of cables and plugs is

also important. Every piece of equipment and tool

needed at the control site must be included in the

deployment plan.

During a deployment, the operation of the robot

requires an operator’s undivided attention. Additional

people are needed at the control station to alert the

robot operator if the operation area becomes too

dangerous. All of the robots were driven under direct

tele-operation without any additional autonomous

behaviors aiding the operator.

One of the lessons learned at the WTC was the need for

a rugged, waterproof robot system that is man-portable

and easily set up. As a result, the development of wellpackaged rugged OCUs has been mandated for all

Tactical Mobile Robots. In order to be effective during

these operations and flexible enough to adapt to the

varied and extreme conditions associated with USAR

missions, the robot systems should require a minimal

amount of well-packaged support gear and cannot be

limited by the logistics of their support or deployment.

VIII. MODULARITY

The PackBot platform is a versatile platform that can

deliver a wide range of payloads that sense and

manipulate the

environment.

Modularity is central to

the PackBot design, see

Figure 8. Each robot

has eight payload

interface connectors

providing a variety of

standard buses and

system power. Each

standard payload port

contains 10/100 full/half

duplex Ethernet,

FARnet (an iRobot

networking protocol),

two differential analog

video channels, two

general purpose digital

pins (serial, if needed), USB, and power sourcing or

sinking. Payloads that have been designed or are being

designed include cameras and lighting units,

manipulators, fiber optic spoolers, and hybrid-electric

generators. The on-board computer is a 700 MHz,

Mobile Pentium III Processor with a 100 MHz system

bus and 256 MB of SDRAM. The computing power of

the onboard computer is available for running software

required for controlling payloads.

The PackBot implements a methodology for “snap-on”

modular payloads that are quickly and easily

interchanged to suit the particular or unique mission at

hand. The flexibility and importance of this concept

was proven at the WTC site when robots were able to

change cameras, lights and tethers as the buildings and

areas that were searched presented various technical

and physical challenges.

Due to the modular design of iRobot’s PackBot, a

payload was developed that specifically addressed

challenges that were anticipated for the WTC and

USAR operations in general. Despite being designed

with MOUT operations in mind, the PackBot was

adapted to address USAR missions. The payload

developed had multiple cameras, infrared illumination,

2-way audio, and a lens cleaning system. In addition to

the payloads that were developed in preparation for

deployment to the WTC, the standard buses provided

on the PackBot payload connectors allowed for several

technologies to be incorporated on site. This allowed

the robot to make use of sensors and equipment specific

to USAR missions. The experiences at the WTC

demonstrated the flexibility of a modular architecture

and the necessity for robot configuration on a per

mission basis.

Payloads used at the WTC were limited to

stationary cameras (including a FLIR), light

sources (IR and Halogen), up to four battery

packs, and Cat 5 cable spoolers. But the

versatility, the advantages, and the

disadvantages of the overall modular

concept were apparent. Even though all

payloads were developed for harsh

environmental exposure, great care had to

be taken when attempting to replace, adjust,

or modify payload configurations. Fine

concrete dust and debris covered every

square inch of the robots’ surfaces after

every deployment.

Useful metrics for modularity include the

Figure 8. PackBot Modularity.
amount of time and cost required to add a simple

payload to the system.

IX. ENDURANCE

A critical metric for field operations is the maximum

length of a mission that can be performed. At the SwRI

tests, the PackBot demonstrated run times of 2 hours

with constant activity. The maximum mission times

can be extended to over 10 hours if the robot is

stationary for most of the mission. These mission times

represent the standard usage of two batteries. Four

batteries can be used to increase the mission time.

Endurance tests at SwRI demonstrated that the PackBot

can carry payloads more than doubling its 35-pound

base weight without significant impact to its mission

time. The PackBot achieves its long run times with the

selection of power saving components throughout the

vehicle.

At the WTC, the lengthier mission times translated to

less down time for the robot returning to swap batteries.

Although the mission length was not an issue on the

specific PackBot missions, there were scenarios at the

WTC where a long mission length would be

advantages. For example, in conducting extensive

building clearance, the operation may be slowed down

by the robot returning to the home base every two to

three hours for a battery exchange.

X. CONCLUSION

The PackBot performed well at the World Trade Center

disaster site in each area of mobility, durability,

situational awareness, communications, deployment,

modularity, and endurance. Many of the characteristics

of the PackBot were well understood from the testing

conducted at SwRI, iRobot, and other exercises such as

the Fort Drum MOUT city. The application of PackBot

technology at the WTC disaster demonstrated that

technology developed for MOUT is transferable to and

useful in USAR missions although these operations

presented unique challenges. The robust mobility and

the flexibility provided by the modular nature of the

PackBot enabled the robots to operate effectively and

adapt to new situations.

The widely varying conditions and environments

encountered at the WTC disaster site confirmed that no

single size or configuration of robot could address all of

the difficulties that an USAR mission can present,

however the PackBot provides a useful capability for

many situations encountered. The TMR program has

made great progress in advancing robot technology and

much of what it has done is applicable to areas outside

of MOUT missions. The experiences at the WTC show

that there is great benefit in using robotic technology in

search and rescue operations, yet there remains work to

be done in developing specific USAR robot technology.

Additional metrics specifically designed for USAR

applications would be helpful. For example, metrics

could measure mobility over rubble and building

debris, as well as communications range in indoor

environments (both intact and damaged). Metrics could

also test the ease of deployment by measuring weight

and time to setup. The performance metrics developed

by SwRI for testing mobile robots provided valuable

information about the PackBot’s capabilities. The

SwRI metrics indicated that the PackBot would

perform well in a wide variety of rugged environments,

and our experiences at the WTC confirmed the SwRI

tests.