NAT'L INST. OF STAND & TECI

II II II

I

AlllOb OMfiSTB eferenc?

National Bureau

of Standards

%

'

? TTsr An Overview of Artificial Intelligence and Robotics

Volume II - Robotics

March 1982

NBSIR 82-2479

—

luu

I Udd

^s2-2479

19J2

Prepared for National Aeronautics and Space Administration

Headquarters

Washington, DC 20546
G7 1 82K 1 F‘ B82 204439 NT I S F‘ r- i c. -s s s pC A06 / MF" A0

TTri Ove r v'l e Oj' 75 F’T^rTTfl^rl. a

1

' 'TriT'i T 1 i a a n c a n d R o b o tics

^T2>trc"

Vo 1 urns "Tls Rcihdl:'ic5

Ge Va r t a r ^ ki i 1 1 i am

"^Tm^ i o n a I B u o f S t a n d a r- d s7 Wa s h i n s t o ri ? BC„ -s-r^ar i o na 1 Ae r o n a. u t i c: b- a vi d

Spac-a Admi n i st rat i on 7 Washington? DC„

liar 82 103 p*

F^ a p- 1 N o s MBS I R—82~ 247
NBSIR 82-2479

IIATIONAI. BVRHAU

OF 9TANOAIU)8

UBBABT

WAY 14 1982

AN OVERVIEW OF ARTIFICIAL

INTELLIGENCE AND ROBOTICS

VOLUME II - Robotics

William B. Gevarter

U.S. DEPARTMENT OF COMMERCE

National Bureau of Standards

National Engineering Laboratory

Center for Manufacturing Engineering

Industrial Systems Division

Metrology Building, Room A127

Washington, DC 20234

March 1982

Prepared for: National Aeronautics and Space Administration

Headquarters

Washington, DC 20546

U.S. DEPARTMENT OF COMMERCE. Malcolm Baldrige. Secretary

NATIONAL BUREAU OF STANDARDS, Ernest Ambler, Director
: r .K'.v'.i ' . 't . *:

'

i r' '

i
Preface

This is the second volume of a two volume NBS/NASA report on an Overview

of Artificial Intelligence and Robotics . This report is intended to be read

by the interested technically-oriented la3nnan, such as engineering managers,

government funding officers, and others who desire an overall perspective of

the field but are unable to obtain it from the highly technical and unintegrated

literature in the field, or from the more flamboyant but non-technical feature

articles in the popular press.

The report incorporates a summary of the basic concepts utilized in each

of the many technical areas, a review of the state-of-the art, an indication

of the organizations involved together with their funding sources and principal

activities, research requirements, and a 5-10 year forecast of the emerging

technology

.

As robotics is a very rapidly changing field - with new companies, products

and researchers entering the field each month - a report like this can only be

regarded as an imperfect snapshot in time, and would benefit from periodic updating. Thus, quantitative information shown in this report may perhaps best

be regarded as indicative rather than authoritative. Despite such shortcomings,

this report should be helpful in providing a base, for organizations and individuals

in and out of the government, for planning future robotic research and development, applications and utilizations.

1
Acknowledgments

I wish to thank the many people and organizations who have contributed to

this report, both in providing information, and in reviewing the report and

suggesting corrections, modifications and additions. However, the responsibility

for any remaining errors or inaccuracies must remain with the author.

It is not the intent of the National Bureau of Standards to recommend or

endorse any of the manufacturers or organizations named in this report, but simply

to attempt to provide an overview of the robotics field. However, in a diverse

and rapadly changing field such as robotics, important activities and products

may not have been mentioned. Lack of such mention does not in any way imply

that they are not also worthwhile. The author would appreciate having any such

omissions or oversights called to his attention so that they can be considered

for future reports.

11
AN OVERVIEW OF ROBOTICS

Table of Contents Page

I Introduction 1

II Primitive Functions Accomplished by a Robot 8

III Manipulator Arm Configurations 10

IV Actuators 12

V Mobility 13

VI End Effectors 14

VII Control 15

VIII Sensor-Controlled Robots 17

IX Robot Vision 18

A. Methods for Determination of Pose 18

1. Range 18

2. Orientation 18

B. Feature Extraction 21

C. Object Recognition 21

1. Template Matching 21

2. Use of Edge and Region Statistics 21

3. Fourier Transform 22

D. Future Techniques 22

X Programming a Robot 23

XI Kinematics and Dynamics 24

A. Coordinate Transformations 24

B. Trajectory Selection 24

C. Dynamics 25

ill
Table of Contents (cont.)

Page

XII Research Requirements 26

A. Different Views 26

B. Synthesis of Needed Research 33

1. Manipulator and Actuator Design 33

2. End Effectors 34

3. Control 34

4. Sensory-Controlled Manipulation 35

5. Vision 36

6 . Tactile and Force Sensing 37

7 . Programming a Robot 37

8 . Intelligence 37

a. Communication 38

b. Knowledge Representation 38

c. Planning 38

9. Robot Mobility 39

10. Modularity 40

XIII Commercially Available Robots and Components 42

XIV Commercially Available Vision Systems for Robots 47

XV Current Robot Research 52

XVI Applications - Now and in the Future 76

XVII The Expanding Robot Population 78

XVIII Future Directions 86
ROBOTICS

I. Introduction

Robots in use today are primarily machines with manipulators that can be

easily programmed to do a variety of manual tasks automatically. The robot consists of one or more manipulators (arms), end effectors (hands), a controller,

power supply, and - increasingly - sensors to provide information about the environment and feedback of performance in task accomplishment. The motivation

for most current robotic development is industrial: to increase productivity,

reduce costs, overcome skilled labor shortages, provide flexibility in batch

operations, improve product quality and to free humans from boring, repetitive

tasks or operations in hostile environments.

Table 1 gives the Japanese standard classifications for Industrial robots

(IR's) by input information and teaching method, and relates this to the U.S.

classification by control method - non-servo or servo.

As indicated in Table 2, the Japanese are the world’s largest user of robots.

Counting all classifications of robots the Japanese had approximately 77,000 robots

in use at the end of 1980 (Yonemoto, 1981, p. 1). Table 3 presents the Japanese

robotic use by industry, based on robot costs. It seems that the electrical

machinery and automobile industries are the largest users of robots. Less than

1% of IR's are utilized outside of manufacturing. Table 4 presents the 1979

Japanese usage by production process. It is apparent that over 60% of the usage

is for moving or positioning a workpiece. This is reflected (see Table 5) in the

predominance in 1980 robot production of the fixed sequence type of robots.

Of the many reasons for installing industrial robots, the most prominent

ones are to increase productivity and reduce costs. A commonly used measure for

determining the ecomonics of installing robots is the payback period. This is

1
Table 1 Definitions and Classifications of Industrial Robots

Japanese Industrial Standard JIS B0134-1979 U.S. View

Manipulator A device lor handling objects as desired without touching

with the hands,and it has more than two ol the motional capabilities such as revolution, out-in, up-down, right-lelt

travelling, swinging or bending, so that it can spatially trans- poitanobject by holding, adhering to. and so on.

INDUS 1 RIAt. ROBO 1 S all have armlike projections and grippers that perlorm lactory work customarily done b) humans. The term is usually reserved lor machines w ith somelormol built-in control system and capable

ol stand-alone operation. But in Japan, It also includes manipulators

operated by humans, cither directly or remotely.

Robot A robot is delined as a mechanical system which has Ilexibic motion lunctions analogous to the motion lunctions ol living organisms or combines such motion functions with intelligent lunctions. and which acts in response to the human will. In this context, intelligent functions mean the ability to perlorm

at least one ol the lollowing: judgment, recognition, adaptation or learning.

Classification by input information

and teaching method

Manual A manipulatoi that is directly operated by a man.

manipulator

Scv|uence A manipulatoi . the working step of which operates sequentially in compliance with preset procedures, conditions and

positions.

f ixed A sequence robot as defined above, lor which the preset sequence inlormation cannot be easily changed.

Variable A sequence robot as delined above, for which the preset .sequence information can be easily changed.

Rlayback

robot A manipulator that can repeat any operation after being

instructed by a man.

Numerically

controlleu robot

A manipulator that can execute the commanded operation

in compliance with the numerically loaded working information on e g. position, sequence and conditions.

Classification by Servo Type

NON-SERVO ROBOrS

A I’ICK-AND-PLAC'E ROBO l is the simplest version accounting for about one-third ol all U.S. installations. I he names comes from its usual application in materials handling, picking something from one spot and

placing it at another. Ereedom ol movement is usually limited to two or

three directions in and out. lelt and right, and up and down. I he control

sy stem is electromechanical. Prices range I rom S5.000 to S.JU.OOO.

A SERVO ROBOl is the most common industrial robot because it can

include all robots described below. I he name stems I rom one or more servo- mechanisms that enable the arm and gripper to alter direction in midair, without having to trip a mechanical switch. Eive to seven directional movements are common, depending on the number ol "Joints." or articulations,

in the robot 's-arm.

A PROGRAMMABLE ROBOl is a serv o robot directed by a programmable controller that memori/esa sequence of a rm-and -gripper movements;

this routine can then be repeated perpetually. I he robot is reprogrammed

by leading its gripper through the new task. The price range is $25,000 to $90,000. A COMPU I ERIZED ROBOl is a servo model run by a computer. 1 he computer controller does not have to be taught by leading the arm-gripper through a routine, new instructions can be transmitted electronically. 1 he programming loi such "smart "robots may include the ability to optimize, or improve, its work-routine instructions. Prices start at about $.15,000.

Imclligeiu robot A robot that can determine its ow n actions through its sensing and recognitive abilities.

A SENSORY ROBOl is a computerized robot with one or more arlilicial senses, usually sight or touch Prices for early models start at about $75,000. AN ASSEMBIA ROBO l is a computerized robot, probably a sensory model, designed spccilically for assembly -line jobs. Lor light, batch-manufacturing applications, the arm's design may be fairly anthropomorphic.

Source: Yonemoto. 19X1, P. 5 Source: Business Week, lune 9. 19X0. P, (i4

2
Table 2

IRs in Operation

(As of the End of 1979)

Japan 1U,000 Units (69^) ”

United States 3,255 (16%)

West Germany 850

Sweden 600 Total

20 ,268

Italy 500 { 100%)

Poland 360 Western

Europe

France 200 3,013

(15^0

Britain 185

Norway 170

U.S.S.R. 25

Belgium 13

(Note) Manual Manipulators and Fixed Sequence

Robots are not included.

Source: Sadamoto, I98I, p. 52.

3
Table 3

Principle Users of Industrial Robots in Japan

Share in I 98O

(Based on Value)

Electric Machinery Industry 36%

Automobile Industry 30%

Plastic Moulding Products Industry 10%

Metal Working Industry 5%

Metal Working Machinery Industry 4%

Steelmaking, Textile, Chemical,

Shipbuilding and Misc. 15%

Source: Yonemoto, I 98I, p. 2

4
Table 4

JAPANESE ROBOT USAGE BY PRODUCTION PROCESS

Process % of Total

Loading & Unloading 40

Transfer & Sorting 21

Palletizing 9

Welding 6

Work Maintenance 4

Assembly 3

Spraying 2

Other (including

Pouring, Screving,

Riveting, etc.

15

Total 100

Source: Research Div. , Long-Term Credit Bank of Japan

1981 , p. 28 .
Tab le 5

JAPANESE PRODUCTION OF ROBOTS BY TYPE IN I 98 O

Type Unit

Value

($ Million)

Manual Manipulators 1,9^0 (loy») 13 (i+%)

Fixed Sequence

Control Robots

13,i+H0 (6 t%) loU ( 31%)

Variable Sequence

Control Robots

l,3h0 in) h3 (12%)

Playback Robots 2,030 ( 10%) 71 (21%)

NC Robots 990 (5%) 97 ( 29%)

Intelligent Robots 130 (1%) 10 (3%)

1 Sub-Total (lOOfO

1

Other Robot -Related

Devices 10

Total 19,870 (100%) 3U 8

Source: Ogawa, lOQl, P» 132.

6
usually given (see for example, Simons, 1980, p. 32) as the cost of the robot

and accessories (including fixtures, etc.) divided by the annual savings:

I P L-E

where p = payback period in years

I = total investment

L = total annual savings in labor

E = total annual robot upkeep cost.

Typically, automation investments have required that the payback period be three

years or less, with actual payback periods reaching as low as one year when

multiple shifts are utilized.

7
II . Primitive Functions Accomplished by a Robot

Industrial robots are primarily designed for manipulation purposes. The

primitive actions imposed at the hand by the manipulator are:

1. Moving

- from point to point

- following a desired trajectory

- following a contoured surface

2. Changing Orientation

3. Rotation

Non-servo robots can provide a sequence of point to point (PTP) motions.

For each motion, the manipulator members move full tilt until the limits of travel

are reached. Thus they are often referred to as "bang-bang", "pick and place"

or "limited sequence" robots. "Programming is done by setting up the desired

sequence of moves and by adjusting the end stops for each axis" (Tanner, 1979,

p. 7). The use of mechanical stops and limit switches yields good positional

accuracy, which is typically repeatable to better than + 0.5 mm (Simons, 1980,

p. 20).

Servo-controlled robots can be designed to provide a sequence of PTP motions,

with controlled intermediate velocities. They can also be used to follow a programmed trajectory or by using sensors to follow a contoured surface.

If we consider end effectors, the robot can also be made to grasp, push

and pull, twist, use tools, perform insertions and assembly, and do other manipulations done by humans.

If sensors and appropriate computations are added, the obot can perceive,

inspect, recognize, test and do many of the other perceptua. functions now done

by humans

All applications require that the robot interact with environment in

the execution of its programmed task. Thus the robot is of , designed to

8
switch machines on and off and to accept information from machines as to their

status. The robot may also have provisions for accepting signals from a moving

assembly line, so it can track it. Similarly, the robot may have interface

provisions for tactile, visual or other sensory inputs for feedback control.

9
III. Manipulator Arm Configurations

To accomplish these primitive actions, four basic configurations of

manipulator arm designs have evolved:

(a) Rectangular (Figure la)

(b) Cylindrical (Figure lb)

(c) Spherical (Figure Ic)

(d) Anthropomorphic, articulated or jointed arm (Figure Id)

These arms utilize both rotary ("revolute") and linear ("prismatic") joints, as

appropriate, to accomplish their function. Six degrees of freedom (6 DOF requires

6 joints) are necessary to locate the end effector (hand) at any point and orientation in the work volume. More DOF's are sometimes used to prevent interference

of the arm with the work piece or other obstacles. As few as 2 or 3 DOF may suffice

for a simple "pick and place" robot.

As indicated in Figure 1, only three DOF's are required to position the end

effector. However for applications requiring a high degree of wrist flexibility,

as many as three of the arms DOF's are located in the wrist. These DOF's are

usually referred to (using aeronautical terms) as pitch, yaw and roll.

The basic four manipulator arm configurations can be combined in various ways

to produce a bewildering set of manipulator combinations. These combinations can

be further extended by employing angled arms and offsets. Often built-in

mechanical linkages are added to do such things as constrain the motion of nominally

articulated arms to cylindrical coordinates for ease of control.

In addition, the arms may be designed to be mounted on their sides, hung

from the ceiling, or mounted on rails for mobility, to further multiply the

range of installed configurations.

10
Figure

1

Basic

Manipuiator

Geometries

11

Spherical

Articulated

or

Jointed

Spherical

(Polar

Coordinates)

(Revolute

Coordinates)
IV. Actuators

There are three major ways today of powering the actuators that move

the manipulator joints: pneumatic, hydraulic and electrical. Pneumatic is

simple and inexpensive but "squooshy" and is best used for "pick and place"

robots whose trajectory motions are controlled by mechanical stops. Hydraulic

actuation (currently the most popular system) is capable of high-power for a

given size, but suffers from leakage problems as well as requiring pumps,

storage tanks and other accompanying paraphenalia. Electrical-mechanical

actuation is clean but doesn’t offer good power to weight ratios. Stepper

motors are inexpensive but sometimes lose pulses and therefore accuracy.

D.C. torque motors are very reliable, offer good controllability, but can be

expensive. Because the overall cost advantage of hydraulic systems diminishes

with size, electrical drives are to be preferred for the small robot.

Actuators are the most problem-prone area of robotics. Thus considerable

research is needed in this area to further improve reliability (robots have

an approximately 98% up-time these days) and to reduce costs and weight.

12
V. Mobility

At present, most robots in industrial applications are fixed inplace or

ride along rails, guideways or conveyors. Simple robot carts for transporting

materials are now being used in industrial and commerical applications. Robot

vehicles using wheels and treads have been researched by NASA for planetary

exploration purposes.

Crawling robots, using their manipulator arms to grasp handholds, are being

explored by NASA for use in constructing large structures in space. Walking

machines are still in the experimental stage, with research going on at Ohio State

University, Carnegie Mellon University (CMU) , Tokyo Institute of Technology, and

in the Eastern Block countries. Engelberger (1980, p. 63) states that "Practical

robot devices with legs could have advantages in such applications as arctic transport, mining, agriculture, forestry, fire-fighting, explosive ordance disposal,

unmanned ocean-floor surveys, and planetary exploration".

13
VI. End-Effectors

The hand or gripper, sometimes called the "end-effector", (attached to the

robot wrist) can be a mechanical, vacuum or magnetic device for part (or object)

handling or for tool manipulation. At present, most of the these interchangable

devices are grippers which are simple open-and-close devices. The variety of

tools and grippers that can be adapted for robot use is virtually unlimited (see

for example, Engelberger, 1980, pp. 41-58). The end-effectors are usually unique

to the robot application and thus customarily provided by the user, though several

manufacturers offer a broad selection of devices for grasping parts that might be

adaptable to the particular task being performed. Nevertheless, though hundreds

of different special-purpose end-effectors now exist, the end-effector remains

one of the major limiting factors in universal robot manipulation due to lack of

dexterity and programmability of the hands. Extensive research and development

is now underway to produce grippers that can handle a wide variety of part

configurations

14
VII. Control

Control of robot arms ranges from highly repeatable open loop devices

to servo-controllers that utilize external sensors in the control of robot

actions.

Open loop devices may be as simple as a step sequencer and positionable

mechanical stops for a pick and place robot, to more complex devices using

stepper motors to reproduce a desired motion.

Servo-devices can use internal sensors, such as joint position sensors,

or external information sensors such as force, proximity devices, and even

vision (operating under computer control)

Servos operating only on internal sensors require very careful positioning

of the work-piece (which can require expensive fixturing or feeding devices).

Servos utilizing data from sensing the external environment, require more

complex processing of the signal, but yield much more flexible systems.

The controller, in addition to controlling the manipulator motion, often

also serves as an interface to the outside world - coordinating the robots

motions with machines and assembly lines and turning on and off machines that it

is operating.

A. Pick and Place Robots

The simplest controllers are non-servo (open loop) devices that. rely on

sequencers and mechanical stops to control the manipulator end-point positions

along each of the axes. These "limited sequence" robots are also referred to

as "pick and place," "bang-bang," or "fixed stop" robots. These robots have no

provision for trajectory control between the end points.

B . Programmable Robots

Programmable robots are servo controlled robots of two basic types: "pointto-point" and "continuous path."

15
Point-to-point robots are directed by a programmable controller that memorizes

a sequence of arm and end-effector positions. Hundreds of points may be memorized.

The robot moves in a series of steps from one memorized point (set of point

positions) to another under servo control, using internal joint sensors for feedback. Because of the servos, trajectory control between the memorized points is

possible and relatively smooth motions can be achieved.

Continuous path robots do not depend on a series of intermediate points to

generate a trajectory, but duplicate during the playback process the continuous

motions recorded during the teaching process. Thus, these robots are used for

painting, arc welding and other processes requiring smooth continuous motions.

C . Computerized Robots

Computer-controlled robots are capable of being programmed "off-line" using a

high level programming language and do not have to rely on being physically taught.

D. Sensory Robots

These are computerized robots that interface to the outside world thru external

sensing such as sight or touch. These "intelligent robots" are capable of adapting

to a variety of conditions by changing their actions based on relating the sensed

information to their goals or preprogrammed decision points.

E. Robots as Fart of a Flexible Manufacturing System

A flexible manufacturing system (FMS) is a programmable batch-processing

arrangement that contains programmable machine tools and transfer devices all

under central computer control. When robots are used in such a system, the robot

controller may itself be just the bottom layer of a hierarchical control system

for the FMS (c.f. Topperwein, et al., 1980, pp. 96-105, or Albus, 1981A, pp . 261-

280) .

16
VIII. Sensor-Controlled Robots

Sensor-controlled robots are the first step on the path to truly

intelligent robots - robots that can determine their own actions based on

their perception and planning abilities.

The simplest versions of sensor-controlled robots utilize contact

switching for stopping the arms and for opening and closing grippers.

More sophisticated robots are now beginning to use touch, force and

torque sensing. Forces and torques can be derived from stress measurements

or from internal signals (e.g. derived from back EMF) . Tactile information

may be derived from deflection- induced resistance variations of a special sensing

surface, or by various other ingenious approaches.

Work in France and at MIT on "artificial skins" with embedded

arrays of touch sensors, opens the possibility of a robot eventually being

able to determine position, orientation and identity of parts by touch

alone. This may be particularly important for difficult assembly

operations (Kinnucan, 1981). Proximity and range sensing are also now

being considered.

Vision, the most complex of all the sensory modalities, is now

beginning to find its way into robotic control.

17
IX. Robot Vision

In the industrial manufacturing arena, to circumvent the problem of requiring

that the workpiece be in a prescribed position and orientation (pose) for the robot

to operate upon it, sensory systems are now beginning to be employed. Vision

provides perhaps the most flexible approach to avoid all the fixturing that

would be required to achieve a fixed pose.

Figure 2 lists the desired functions of a machine vision system to achieve

fully flexible sensor-controlled manipulation. Figure 3 indicates potential applications for such a system.

A. Methods for determination of pose :

1. Range can be determined in four principle ways:

Stereo

Triangulation

Active ranging (e.g. time of flight of light or sound)

Using optical focusing.

2 . Orientation can be determined by :

Observing the relationship of three (or more) known object

points that are not all colinear in the viewing field, provided that the relative ranges connecting these points are

known

Deriving surface normals using the intrinsic image concept

(Barrow and Tennenbaum, 1981)

Deducing it based upon the response of lighting on the object,

utilizing known characteristics of the object. (This is simplified

if structured lighting such as sheets of light are employed)

Interferometry measuring the phase differences of an oscillating

signal received at multiple points.

18
Figure 2

Desired Functions of Machine Vision for Sensor-Controlled

Manipulation in Industrial Manufacturing

** Recognition of workpieces /assemblies and/or

recognition of the stable state where necessary.

" Determination of the position and orientation of

workpieces /assemblies relative to a prescribed set

of coordinate axes.

* Extraction and location of salient features of a workpiece /assembly to establish a spatial reference

for visual servoing.

® In-process inspection — verification that a process

has been or is being satisfactorily completed.

Source: Rosen, 1978
Figure 3

Block Diagram of Potential Vision Applications in Robotics

Souim: Sown, 197S, p8, (tIiiMtralMl Vtniofi Omwii It From Sovorlone,

1980, pis, ond It Utod by Pormittlen et WoboUct Ag»).

20
B. Feature Extraction

Though it is possible to operate directly on an analog output of a vidicon

imager (by such approaches as optical correlation, or template matching) , it is now

fashionable to use solid state imaging arrays, such as CCD or CID devices, which

directly provide points (pixels) in the image.

These pixels are then digitized by intensity levels (usually 2 to 16) so

as to be put in a convenient form for operation upon by a computer. From this

data, edges and regions are determined. Edges are considered to occur when there

is an intensity gradient exceeding a selected threshold. These gradients are

derived from intensity differences between adjacent pixels (often using an averaging

scheme to reduce noise effects) . Regions are grown by annexing ("clustering")

adjacent pixels that are close to the same intensity level. Various schemes

(heuristics) exist to connect lines having gaps, smooth irregularities, and remove

discontinuities in regions (all resulting from noise in the image). The. result

(which can be considered as a form of data compression) is a reduced image that

consists of edges and regions. Further data compression might be achieved by representing edges by sets of straight or curved lines (the ends and curvature of

which are all that need to be known)

C. Object Recognition

Having determined edges and regions, it is possible using the methods

of Section A to determine position and orientation. However, often it is

necessary to determine which object is being observed from a collection of

possible objects. This may be accomplished by

1. Template Matching : Using a stored image in the computer.

2. Use of Edge and Region Statistics : This may be curvature of edges,

area moments of inertia, ratio of square of perimeter to area, number

and/or location of holes or notches, etc.

21
3. Fourier Transform : Determining the coefficients of the spatial

Fourier transform and comparing them to the coefficients of known

objects

D. Future Techniques

In the future we may see:

Greater use of direct range measurements - e.g. scanning laser

radar, or LED's and photo diodes using phase or intensity measurements.

Parallel processing to obtain feature extraction in real-time.

Improved methods for determining edges and regions.

Use of optical markings for recognition, for optical code reading or for

determination of pose.

Use of intrinsic image analysis - to determine surface orientation,

depth, relationship of foreground to background, and texture.

Scene analysis for less-structured situations.

22
X. Programming a Robot

There are four primary methods for programming a robot

Physical-setup , in which the operator sets up programs by physically fixing

stops, setting switches, arraying wires, etc. This is characteristic of the

simpler robots.

Lead-through , in which the operator leads the robot through the desired

positions and locations by means of a remote teach box. (These points are

recorded and used to generate the robot trajectory during operation.)

Walk-through , where the robot arm is physically manipulated through

the desired motions (which are recorded and then played back by the

robot-control during operation)

Writing a software program , which is then executed when desired.

The emphasis in programming research today is on software programming of

computer-controlled robots. Work on sensor-controlled manipulation is extending

the scope for programmability. Interacting with the robot by means of software,

provides more flexibility then the other programming methods, and allows for

conditional actions or flexible adaptations. Various high level robot programming

languages such as VAL (Unimation) and AML (IBM) are now beginning to become available

to aid in the software generation.

Simons (1980, p. 107) notes that software programs can be divided into two

types: "In explicit programming , the user requires explicit instructions for

every action the robot must take. In world modelling , the robot is more

autonomous and can make decisions according to its knowledge. World-modelling

systems, largely in the research stage, tend to require a considerable amount

of computer power but are able to carry out complex tasks. The most important

world-modelling languages are AL, AUTOPASS, and LANA..."

23
XI. Kinematics and Dynamics

A. Coordinate Transformations

If the arm is programmed by walking-thru, then it is simply necessary to record

the simultaneous joint positions during the walk- thru. If the arm is led-thru, then

in more sophisticated systems, that can operate in tool point, world, or other coordinate systems, it is necessary to do the needed coordinate transformations for

each joint in order to obtain the desired end-effector orientation and position.

This is particularly true when the arm is being programmed by writing software.

When the arm is controlled via feedback from a vision or other sensor,

it is necessary to do coordinate transformations to convert the camera or

hand coordinates to base or computer coordinates. Fca; vision (or other sensor)

control, the sensor interpretation and coordinate transformations need to be done

in real time. As the mathematics for exact transformations requires time-consuming

calculations, approximate simplifications have been developed to speed the calculations.

In some systems, Individual microprocessors are used to process the kinematics at

each joint.

Note that arms are often designed so that the wrist motions are mechanically

constrained to cylindrical or rectilinear coordinates to simplify control and reduce coordinate transformation requirements.

B . Trajectory Selection

For moving the end effector from one position (or point) to another, various

schemes are possible for choosing the motions for moving the joints.

Approaches for articulated arms include:

Variations from preprogrammed paths.

The "dominant axis" approach: some joints may be allowed to adapt

while others follow fixed movements.

24
* All joint motions move proportionately relative to the slowest joint, so that

all joint motions terminate at the same time.

Considerations of this sort become particularly Important for off-line or real-time

programming of computer-controlled robots.

C. Dynamics

At present, all robot arms are made relatively stiff so that structural

flexibility is minimized (to promote repeatability). However, this requires

stiff and usually heavy arms with high power actuators, lowered carrying capacity,

and reduced speed. Research is now underway on the control of structurally

flexible arms to help overcome these limitations, and in active control to control

flexure for achieving on-line precision under load as during assembly or drilling

operations. The 15 meter long Romote Manipulator System (RMS) for the Space Shuttle

is probably the most extreme case of a structurally flexible manipulator thus far.

25
XII. Research Requirements

A. Different Views

In considering research requirements It Is very helpful to differentiate

between near-term needs and far-term needs. Figure 4 Indicates the near-term

needs for Improvement In IR’s, as perceived by Japanese users. Note the

emphasis Is on robot flexibility and versatility of movement, speed, and ease

of programming.

In observing how private enterprise Is responding to the IR user desires,

Umentanl (1981, p. 81) Indicates that the robot manufacturers most Important

targets for development Include:

reduced production costs

smaller Installation space requirements

greater flexibility

larger carrying capacities

modular robots components to enable a building block approach to

diversification of functions

visual and tactile feedback and recognition systems

Table 6 lists the main research subjects being pursued by Japanese

Robot Manufacturers. These may be regarded as relatively near-term.

Note the correspondence with user desires In their emphasis on speed,

flexibility and programming versatility.

As we move towards the longer term, we encounter robot research

such as Is being pursued In Japanese Universities and Japanese Government

26
Figure 4

1980 JIRA User Survey of Desired

improvements in industriai Robots

Greater Degrees of Freedom

More Compact Size

Increased Speed

Larger Assortment

of Attachments

Easier Reprogramming

Greater Reliability

50

Increased Working Volume

Increased Payload

Capability

Increased Accuracy

Tactile Sensing

Vision

Pattern Recognition

Increased Memory

Mobility

Coordinated Control of Multiple

Robots and Machines

100

~r

Others

Source: Umentani, 1981, p. 84.

Cases

27
Table 6

Main Research Subjects In IRs

1980 JIRA Survey of Japanese Robot Manufacturers

Order of

Importance Subj ect

No. of

Research

Subj ects

I0 of

Total

1 Increased Speed 1+9 13.1

2 More Compact Size 36 9 .T

3 Computer Control 32 8.6

I4 Lighter Weight 30 8.0

5 Interchangeability

by Module 25 6.7

6 Object Recognition 19 5.0

T Increased Payload IT 1+.6

8 Improved Actuators II+ 3.8

9 Self Diagnosis 1I+ 3.8

10 Adaptive Control lU 3.8

Source: Umentani, I98I, p. 98.

28
Research Institutes. Here a 1980 JIRA research survey indicates (see

Umentani, 1981, pp. 95-96) that their primary research topics (in decreasing

order by number of research projects) are:

control

locomotion

vision

tactile sensing

manipulators

end effectors

programming methods and tools

Figure 5 lists the desirable goals for future robots (from a U.S.

vantage point) as seen by Engelberger (1980). He states (p. 125) that

of the missing robotic attributes, the two most cruical ones are vision

and tactile sensing. As indicated in Figure 6, this view is shared by the

particpants in the NBS/RIA Robotics Research Workshop.

Kelley and Huston (1980) , Birk and Kelley (1981) , and Albus (1981B)

have also attempted to come up with research requirements for future

robots, though they did not attempt to prioritize them. Their requirements

are summarized in Figure 7. In the following discussion we will combine these

research needs into categories for ease of presentation.

29
Figure 5

Desirable Goals for Future Robots

1. Rudimentary sense of vision to provide

a) recognition data

b) orientation data

2. Tactile sensing to provide

a) recognition data

b) orientation data

c) physical interaction data

3. Computer interpretation of the visual and tactile data

U. Multiple appendage hand-to-hand coordination

5 . Computer directed appendage trajectories

6. Mobility

T. Minimized spatial intrusion

8. Energy conserving musculature

9 . General purpose hands

0 •

1

—

Man-robot voice communication

11. Total self diagnostic fault tracing

12. Inherent safety (Asimov’s Laws of Robotics)

13. All the capabilities above available for a price which allows

purchase and operation within the traditionally accepted rules

for economic justification of any new equipment

Source: Engelberger, I98O, p. II8.

30
Figure 6

Recommended Distribution of U.S. Robotics Research Funds/Effort

%

Binary Vision 14

Gray Scale Vision 12

Off-line Programming 12

Control Systems 11

Touch, Force, Torque Sensing 11

End Effectors 10

Integration Into CAM Systems 8

Mechanical Systems 7

Simulation and Modeling 6

Safety 4

Mobility/Line Tracking 4

Others 1

Source: NBS/RIA Robotics Research Workshop, 1979, p. 49.

31
Figure 7

Basic Research Needs in Robotics (not prioritized)

Kelley and Huston (1980)

Manipulator Kinematics

Control

Locomotion

Sensing

Artificial Intelligence

Higher-Order Languages

Smaller Actuators

Albus (1981B)

Accuracy

Speed and Dexterity

Sensing

Control

Knowledge Representation

Programming (Higher Order Languages)

Standardized Interfaces (Modularity)

Mobility

Birk and Kelley (1981)

1. Intelligence

Knowledge Representation

Auto, Prob, Solving,

Decision Making, Planning

2. Control

Flexible Structures Control

Modeling

Real-time Sensory Control

Fault Tolerance, Monitoring

and Repair

3,

Actuation

4. Manipulation and Hands

Performance Evaluation

Kinematic Planning for

Assembly

Speed

5. Dexterity

6. Locomotion

7. Sensing

Vision

Tactile

Real-time Computation

8. Programming

9. Adapation and Learning

32
B. Synthesis of Needed Research

1 . Manipulator and Actuator Design ;

To achieve the higher speeds, flexibility, accuracy, efficiency and

dexterity that is desired in future robots, additional research in manipulator

arms are needed. For large manipulators and precision work, joint and arm structural

flexibilities can be a problem. Therefore stiff, but lightweight arms are

desirable to minimize this problem and to aid in increasing speed. However as

variable compliance in different directions is often desirable to provide

the "give" needed for inserting parts and other assembly operations, unique

design approaches may be required.

At present, actuators are one of the more problem-prone areas of robots.

In addition, actuator efficiencies are in the order of one tenth that of human

musculature. Smaller, more reliable and efficient actuators and drive mechanisms,

with high load to weight ratios, are needed for the high speed, lighter weight

robots of the future.

For greater dexterity and less intrusion into the work-zone, it is also

necessary to understand the mutual interdependence of end effector and manipulator kinematics. Also research on micro-manipulators is needed for applications

to micro-electronics and biology.

A unified theory is needed on how to design an arm for smooth universal

movements, satisfaction of functional requirements, and to provide the required

load capacities throughout its motion spectrum. A standardized approach to performance measurement and evaluation is required to enable comparison of robots

not only in such factors as working volume, load capacity and reliability, but

also in aspects such as speed, smoothness and accuracy throughout their operating

range

33
To help determine manipulator accuracy (now a limiting factor in off-line

programming of open-loop manipulator operations) , a universal stochastic model

of mechanical errors of manipulators needs to be devised.

2 . End Effectors

Umetani (1981, p. 110) states that "the most difficult barrier in industrial

robot design... is the design of dexterity in hands and fingers." Thus, much

work is needed to devise designs for general purpose hands (and the means to

control them) , such that the whole arm need not move to provide the desired

dexterity. Because general purpose hands are not yet available, multiple end

effectors may be required for a single application. This in turn requires some

type of quick change wrist design.

3 . Control

Kelley and Huston (1980) indicate that further research is needed on the

basic robot problem: given the desired external or absolute displacement,

velocity and acceleration of the hand, determine the optimal physical and

internal motion parameters. (Birk and Kelley, 1981) state that procedures

have been recently developed for efficiently obtaining and solving governing

dynamic equations for robotic systems with flexible links and joints. However,

optimizing the arm motion still presents problems of convergence and uniqueness

(Kelley and Huston, 1980)

Though the repeatibility of arm positioning via open-loop (no external sensory

feedback) robot control has been good, accuracy for initially positioning the arm

via open-loop control has been poor. In addition, no efficient means of robot calibration has yet been determined. Thus it is important that more work be done on

improving open-loop control accuracy. Accuracy degradation due to flexure can be

a particularly troublesome problem when trying to perform precision operations under

load, such as drilling without the aid of jigs.

34
An important area of control not yet solved is control of structurally

flexible manipulators. This problem, important to high speed manipulation with

manipulators having high load to weight ratios, may require real-time simultaneous

management of a large number of command and control variables. An associated

problem with such manipulators may be the large changes in moments of inertia

as the manipulator is extended and a load applied. Another problem associated

with high speed manipulation, requiring additional research, is the generation of

time-optimal trajectories.

An area for which little is known about ideal solutions, is the control of

redundant-axis manipulators used to provide additional manuever capability.

Design of control systems for fault tolerance will become particularly important as robots become more autonomous.

Finally, lack of suitable strategies for control of multi-fingered hands is

still one of the principle factors retarding design of dexterous hands.

35
4 . Sensory-Controlled Manipulation

One of the most desirable features for improving robot flexibility is

sensory capability. Vision, the most popular U.S. robot research area, appears

to be the most desirable sense for present and future IR's. Next in importance

is touch.

To take full advantage of sophisticated sensory data, real-time control using

sensors is needed. Hierarchical control may be the appropriate means to handle

such data, though control logic systems such as pattern reference feedback logic may

be appropriate in certain contexts.

In addition to vision and tactile sensing, dynamic control using force feedback

is an important area of research. Also of increasing importance as robots become

more autonomous, is proximity sensing as used before final contact and to avoid

obstacles. Other sensors will increasingly have roles in inspection and in

determining the type of material being observed. Research is needed in the use of

models in sensory processing and interpretation.

It is also necessary to explore the tradeoff between the use of sensors and

imposing more structure on the robot environment. As sensory control is perfected,

the tradeoff will undoubtedly shift towards the direction of using sensors.

5 . Vision

Vision becomes more important as the environment becomes less structured.

Therefore vision is needed more for fire-fighting, household servants, robot

explorers, etc., than for IR's. Vision can also reduce manipulator accuracy requirements, by making real-time adjustments practical.

At the current state of the art, additional research in recognition and geometric

representation is needed. As greater capability is desired, three dimensional vision

combined with world models and spatial reasoning will become important. Current

36
3-D research in vision systems center on the use of range-finding, structured light,

and binocular vision systems. Further-out, research is required in the analysis of

image sequences (optical flow: shape from motion) , intrinsic image analysis (determining shape from a single image by disambiguating the response to light) , and

improved methods to recognize objects in clutter. The bin-picking problem is a

classic one in this latter area.

One of the most important research areas in vision is approaches to improve

the speed of visual processing. This includes research in computational elements

structured for visual processing, which Includes parallel-processing CPU's, and

special chips for edge-finding, Fourier analysis, etc.

6 . Tactile and Force Sensing

What is needed is better use of contact sensing data. This includes better

resolution in touch and force sensing. An important result could be the ability

to recognize parts and to determine their relative position and orientation through

such methods.

7 . Programming a Robot

Programming by teaching is not practical for small lot production, particularly

when sensory interaction is Involved. Thus, improved techniques are needed for

generating robot control data. These could include high-level programming languages,

software debugging tools, world models, and the ability to directly utilize CAD

data bases and sensory inputs. Robot simulation tools can aid in off-line programming.

Also needed are better geometric modeling systems. As robot usage advances, it

would be desirable to produce aids for writing programs to orchestrate

multiple arms and multiple sensors, and for systems with distributed control.

8 . Intelligence

Plantier and Bodmer et al. (1981, p. 63) assert, "Artificial Intelligence is the

technological area which needs most to be developed and mastered to accelerate

robot evolution." Birk and Kelley (1981) indicate that an intelligent robot is

37
one capable of:

receiving communication

understanding its environment by the use of models

formulating plans

executing plans

monitoring its operation

Thus research in all these areas is important.

Communication : Use of a natural language interface, or computer graphics or

even voice commands would all be helpful in simplifying communication with a robot.

In addition, intelligent aids for robot users to help plan and develop robot usage

are also needed.

Knowledge Representation : A key ingredient in advanced robotics is knowledge

representation. However, the tradeoff between control algorithm complexity and

representation complexity needs to be explored. Sophisticated internal models

of the environment are needed to generate expectations, to interpret sensory data,

and for use in automatic generation of robot control data (such as for real-time

decision-making)

Representations are also needed to enable qualitative reasoning about physics

and space for use in planning and problem solving.

Parallel processing will be needed to make real-time modeling a reality.

Planning : Automatic problem-solving or planning is one of the keys to future

intelligent robots. Planning is even needed for acquiring information from

sensors. Needed is research on ways to use large amounts of domain-specific

knowledge to guide planning and responses to recognized situations.

One of the difficult, but important areas of research needed is in

developing plans that deal with time.

38
For certain tasks, the software for handling contingences is becoming

excessive. Thus means for automatic monitoring and correction are needed.

In addition, plan-monitoring and repair is also in need of research. It would

also be useful to be able to store successful plans in categories so that they

can be reused. In general, it is desirable that the system be capable of adaptation

and learning.

Automatic assembly in non-highly structured environments is a key research

area. Development of kinematic strategies for assembly is needed. Assembly could

be aided by model-based generation of programs that use local sensory information

in assembly tasks. Research is also needed for planning and orchestrating the

actions of cooperating robot systems for assembly and other purposes.

Intelligent controls can be thought of as coupling control theoretic approaches

with advanced AI methodolgies. To advance this area, better understanding of modular

hierarchical structures is needed as well as how to generate hierarchical plans

associated with such structures.

To make intelligent robots really useful, it is necessary to develop methods

that will enable them to function in real-time (not having to pause for computation)

with realistic computers.

9 . Robot Mobility

For industrial robots tending machines, furnaces, etc., the robots are often

in use only a fraction of the time, the remainder being in a wait mode. To make

better use of the robots, machines are frequently arranged so that one robot

can tend several machines. Sometimes this involves moving the robot (motion along

tracks being the usual means). Locomotion is needed to further extend the working

range and for use in hazardous environments and in remote locations. In many cases,

a truly mobile robot would be preferred that can be used where needed. Mobile robots

39
would be particularly advantageous in environments less structured than that found

in factories, such as in the construction of ships, buildings and roads, in

mining, and in undersea or space use.

New mechanical mechanisms, other than wheels, are needed for mobility over

uneven terrain. However, mobility systems tend to amplify compliance and

vibration problems (which can be minimized by careful design and the use of sensordriven control). Active suspensions are needed for uneven terrains, climbing, etc.

For legged vehicles, the key problems are stability, control and optimal motion

synthesis. In walking machines very little is known about the stability of various

gaits. Advances are needed in the hierarchical control of multi-body systems.

Efficient algorithms need to be developed for solving the governing dynamic equations.

If the mobile robot is to be supervisory-controlled , then optimal partitioning

of the task between the operator and machine becomes an important research area.

10. Modularity

Further work is needed not only to develop modular manipulators, but also

to enable integrated modular systems that can consist of a variety of robots, tools,

sensors and computers all connected together. To do this, standardized interfaces are

needed. It will be necessary to partition the control problem into modular

components and to develop interface standards for communication between these

components. An important goal is to be able to write an off-line program

for one robot and be able to use it with a differently designed robot.

The 1980 NBS/Air Force ICAM Workshop on Robot Interfaces (1981) considered the

following items for standardization.

simple sensor interface between simple peripheral devices and a robot

control system.

40
' Wrist interface between the robot wrist and the end effector.

* Complex sensor interface that covers vision, complex touch

and other such sensors.

‘ Common robot control Interface, providing robot independent

trajectory descriptions.

Robot Programming Languages

Integration of robots into robot systems for automated manufacturing.

It was concluded that both the simple sensor and wrist interfaces were

ready for standardization, but that it was too early in the development of

the other areas to begin a standards effort at that time.

41
XIII. Commercially Available Robots and Components

Engelberger (1980, p. 117) indicates that many of the attributes required

to make industrial robots practical in an industrial environment are available

today. These (see Figure 8) include good positioning repeatability, reliability,

flexibility, easy programmability and adequate load capacity, all at a affordable

price

The rapid growth of industrial robots (roughly 35% per year) and the entry

of new manufacturers and new products makes its inappropriate to try to list or

even summarize all the robots currently available. However, the following

are good sources of information to keep current with available robot products:

Robotics Age , a bimonthly magazine popularizing robots, P.O. Box 725, La Canada,

CA 91011.

Robotics Industry Directory published yearly by Robotics Age , P.O. Box 725,

La Canda, CA 91011. This directory lists complete information on over

one hundred different models of currently available robots, robot components

and peripherals. It also provides information on robot distributors,

robot research institutes and consultants in the robot field. Robot Systems

and Products , providing a monthly new products update, is also available from

Robotics Age .

Specifications and Applications of Industrial Robots in Japan published by

Japan Industrial Robot Association ( JIRA) , 3-5-8, Shiba Koen, Minato-ku,

Tokyo, Japan. A 1982 edition is currently available. JIRA is a good source

for all information on Japanese robots.

Robotics Today , a quarterly magazine on industrial robotics published by the

Society of Manufacturing Engineers in cooperation with the Robot Institute

of America, One SME Drive, P.O. Box 930, Dearborn, MI 48128.

42
Figure 8

Roliot Attributes Nov Commercially Available

1. Work space command with six infinitely controllable

articulations between the robot base and its hand

extremity

2. Teach and playback facilities - realizing fast,

instinctive programming

3. Local and library memories of any practical size desired

U, Random program selection possible by external stimuli

5. Positioning accuracy repeatable to within 0.3 mm

6. Weight handling capability up to 150 kilos

7. Point-to-point control and continuous path control,

possibly intermixed in one robot

8. Synchronization with moving workpieces

9. Interface allowing compatibility with a computer

10. Palletizing and depallet izing capability

11. High reliability — with not less than 400 hours MTBF

12. All the capabilities available for a price which allows

purchase and operation within the traditionally accepted

rules for economic justification of any new equipment

Source: Engelberger, I 98O, p. 117
The Industrial Robot , a quarterly magazine with a European orientation toward

industrial robots, published by IFS Ltd., 35-39 High St., Bedford MK42 7BT,

England.

An excellent recent guide to industrial robots for perspective users -

Sanderson, R. J. et al. , Industrial Robots: A Summary and Forecast for Manufacturing Managers , 1982 - can be purchased from TECH TRAN Corp., 134 N. Washington

St., Naperville, IL 60540.

In addition, various robotic newsletters (such as "Robot News International,"

also available from IFS Ltd., "Industrial Robots International," available from

Technical Insights, Inc., P.O. Box 1304, Ft. Lee, N.J. 07024; and the "Robotics

Newsletter," available from Bache, 100 Gold St., New York, NY 10038) offering

information on new robot products and investments have recently begun to appear.

Datamation , Byte , Business Week , Time and other technical and popular magazines

have also recently begun to have feature articles on robotics.

At the end of 1981, there were some 50 U.S. based firms marketing or preparing to market robots (Conigliaro, 1981), the leaders being:

Unlmation Inc. (projected to have 32% of the estimate $215 million

(3075 units) U.S. market in 1982) and Cincinnati Milacron (projected to

have 25% of the U.S. market in 1982).

Several of the large U.S. auto, electrical and computer firms have been building

robots for internal use and are now entering or considering entering the commerlcal robot market.

44
Japan at the end of 1981 had over 130 different manufacturers producing

robots, many for their own internal use. Table 7 presents the robot production

by major Japanese manufacturers for the years 1980 and 1981, Of the estimates by

JIRA of 19,400 robots, valued at 77 billion yen, produced in Japan in 1980, only 3% were

exported

In 1979, approximately 2400 robots were sold in Europe. European manufacturers

having a major share of this market include:

ASEA - Sweden

Olivetti - Italy

The Eastern Block Communist countries have been moving in the direction

of robots to make up for their serious labor shortages. Business Week (Aug. 17,

1981) reported that the Soviets plan to manufacture 40,000 robots by 1985 to

counter not only the shortage of skilled labor, but also absenteeism associated

with drinking.

45
Table 7

Number of Robots Produced by Major Japanese Robot Manufacturers

Note: The figures in brackets indicate approximate value.

1980 1981 Major Product

Company Achievement Target Function

1

Kawasaki Heavy 450 650

t Spot welding

Industries ($22 M) ($31 M) Arc-welding

Yaskawa Electric 360 630 Arc-welding

($16 M) ($29 M)

Hitachi 150 480 Spot welding

($22 M) Arc-welding

Mitsubishi Heavy 120 240 Spot welding

Industries Print-spraying

Kobe Steel 120 200 Arc-welding

($7 M) ($13 M) Pain t-s praying

Star Seiki Plastic mould

($13 M) picking

Shinmeiwa 360 480

Industry ($4 M) ($9 M) Arc-welding

Fuji Electric 130 200 Inspection

($9 M)

Fu ji tsu-Fanuc 480 600 Machine processing

1

($4 M) ($7 M) Assembling

1 Sankyo Seiki 300 Assembly

1 ($4 M)

Nitto Seiko 120 Assembly

1

' ($3 M)

Source: August 1981 Survey (Survey Japan, 1981, pp. 210 211).

46
XIV. Commercially Available Vision Systems for Robots

Present commercially available vision systems for use with robots are basically devices that generate and analyze 2D images. These systems recognize

objects by their silhouettes. Principle current manufacturers of sophisticated

computer vision systems are:

Machine Intelligence Corporation (M.I.C.)

Sunnyvale, CA

Automatix Inc.

Burlington, MA

Octek Inc.

Burlington, MA

M.I.C. Vision Module

This Vision Module is based upon prototype software and hardware developed

at SRI under NSF and industrial affiliate support (see Gleason and Agin, 1979)

The Vision Module consists of a solid-state TV camera, a preprocessor interface and a LSI-11 microcomputer. The vision software, stored in the LSI-11 memory,

includes the entire library of vision subroutines previously developed at SRI,

several application programs (such as training-by-showing and execution of part

recognition), as well as newly developed capabilities. The capability to develop

application programs by calling the vision-library subroutines makes the Vision

Module a general-purpose vision system. The Vision Module is intended for a variety of industrial vision applications such as inprocess inspection, sensor- controlled manipulation, and visual servoing.

The Vision Module can recognize parts on the basis of their size and shape

regardless of their position or orientation. The part (lying in one of its

stable states previously used to train the module) is recognized on the basis of

a set of shape descriptions, including area, perimeter, number of holes,

length of major and minor bounding ellipse axes, first and second moments of

area, etc., and various algebraic combinations of these.

The Vision Module operates by appropriately thresholding the image in the

preprocessor to achieve binary silhouettes of the objects in the scene. A frame

buffer stores the binary image (consisting of zeros and ones) to achieve timing

47
independence between the camera and the controlling microcomputer. Each line

is sequentially scanned and "run-length" encoded (i.e., edge points recorded

where the pixels change from zero to one or vice versa) . Each run-length segment

on a line is matched against segments of the previous line to determine their

overlapping relations hips. Using these relation ships, the program traces the

appearance and disappearance of blobs (regions) as the image is processed from

top to bottom. Only one line at a time of image data needs to be stored in the

computer.

At the heart of the vision processing are connectivity analysis routines

which segment the image into contiguous regions (called "blobs") even in the

presence of noise. During the connectivity analysis, sequences of perimeter

points are extracted. Using this blob boundary information, other desired

parameters such as area, area moments, shape, position and orientation of the

blob can be computed.

The system is trained by showing the object to the TV camera resulting in

all potentially useful shape descriptions being automatically calculated and

stored. Then the same object is moved and photographed several times and the

mean and variance of these descriptors calculated and stored. This resulting

set of means and variances, can be used as a "prototype" for that object or

class of objects. New objects can then be recognized in either of two ways.

One, by using a normalized nearest-neighbor method, where the best fit of the

features of the new object to the set of prototypes is selected. Two, using

a binary decision tree (which can be built by the Vision Module software upon

command) recognition is done by measuring one feature at a time, and at each

stage subdividing the recognition problem into two smaller problems until a

single choice remains. The time to read in, process and implement recognition

is in the order of one second.

48
Complete vision systems cost from $30-40 thousand dollars. M.I.C. also

markets the VS-110 Machine Vision System version for real time inspection tasks.

M.I.C. which is currently rapidly expanding, is placing increased emphasis

on turnkey systems, and improved man-machine interfaces. The problems of resolution

speed and cost are also being vigorously attacked.

The Automatix Autovision System

The Autovision System (Rheinhold and Vanderbrug, 1980) also uses the SRI

algorithms as its main base for vision processing and in many ways is similar

to the MIC system. However, the Autovision System’s preprocessor (Scheinman,

1981) has a pixel memory capable of storing several images at once, or several

bits of grey level information, or any combination. The versatility of the

system comes from the system's powerful AI-32 control processor, which with its

32 bit internal design and megabyte memory address space, can operate in parallel

with the preprocessor and has the ability to interface with multiple preprocessors

and various other input and output devices. The control processor operates under

an Automatix developed operating system. Most of the system software is factory

programmed in PASCAL, except where speed or compactness is so critical that assembly

language is used.

The primary interface for the user is via the Automatix application language,

RAIL, which is designed for the non-computer oriented user. This powerful and

flexible robot programming language makes it possible to sequence the operations

of the Autovision system so that the desired task can be performed.

Octek Vision System

Octek has several vision systems based on a circuit board developed in

1980. It can handle two high resolution solid state TV cameras with 320 x 480

pixel arrays. The system has an on-board microprocessor that can manipulate

images at four frames per second. The system can handle a 16 gray level scale.

49
There are four memory planes so that similarities and differences between images

can be determined. The system relies on a Data General or DEC computer for

computation. The basic circuit board costs about $4500, with the cost of a

complete vision system ranging from $16,000 to $40,000.

KEYSIGHT AND CONSIGHT

At General Motors, several different computer vision systems are now

operational (Rossol, 1981). KEYSIGHT is an example of a system for Inspection

tasks in automobile manufacturing. It uses a low-resolution solid state camera

to inspect for the presence or absence of valve spring cap keys on automobile

engines. Ingeniously applying fairly standard computer vision algorithms

for edge finding, etc., the system locates the center of the valve spring assembly and then, based on the keys being brighter than the holes they fit over,

determines if the keys are present.

More interesting is CONSIGHT, an example of a robot vision system that

uses structured light to detect the silhouette of passing objects. Using a robot

arm, CONSIGHT transfers a moving stream of non-touching, randomly oriented parts

from a moving conveyor belt to a predetermined location.

Structured light provides a means to deal with situations that require

both high spatial resolution and fast response. It allows part geometry to be

sensed directly, as opposed to being inferred through grey level pictui^e processing .

CONSIGHT is logically partioned into independent vision, robot and monitor

subsystems. When the vision subsystem has seen the entire object, it sends to

the monitor the object’s position and belt position reference value. The robot

subsystem then executes a previously ''taught” robot program to transfer the

part from the belt to a fixed position.

50
A single narrow and intense line from two parallel, separated sources

is projected across the belt surface perpendicular to the belt's direction of

motion. A linear array camera builds a conventional two dimensional silhouette

imate of passing parts by collecting a sequence of these image strips. For each

object detected, a small number of numerical descriptions is automatically

extracted for part classification and position determination.

CONSIGHT is now commercially available, having been licenced to Unimat ion,

Cincinnati Milacron, M.I.C. and Automatix.

51
XV. Current Robot Research

In Japan

Robot related research in Japan can be placed into three categories - research

done by universities and research institutions, by industry, and by special government sponsored task forces. A 1980 JIRA survey found that a total of 88 research

projects were then underway at universities and public and private research

institutions. However unlike the U.S., there were no large-scale research efforts

at the universities or technology colleges.

Areas of emphasis in research projects at Japanese universities and research

institutions are (Hasegawa, 1981, pp. 162-163):

increasing robot speed

coordination of multiple arms

dexterous robot hands with tactile recognition capability for

automatic assembly

sensory development for intelligent robots

robot locomotion

robot control and application software development

In the United States

Robot research in the U.S. is primarily aimed in the direction of making

robots more intelligent and versatile, with vision research being given a high

priority. Albus (1981B, p. 7) states, "Much remains to be done in sensor technology

to improve the performance, reliability and cost effectiveness of all types of

sensory transducers. Even more remains to be done in improving the speed

and sophistication of sensory processing algorithms and special purpose hardware for recognizing features and analyzing patterns in space and time."

Robotic research efforts at U.S. universities (roughly reflecting FY-82

funding) are indicated in Table 8. Note that the research is centered on greater

52
Table 8A

Robotics Research

Institution: Stanford University

Principal

Researchers: Thomas 0. Binford, Bernard Roth,

Robert Cannon, Larry Lelfer

(23 Grad Students, 4 PostDocs)

Research Funding: Approx, $1,400 K/year from NSF, ARPA

and NASA

Manipulation

• Control of flexible manipulators

• Control of multiple manipulators

• Design of three-fingered hand

Perception

• Robot Vision

• Stereo Mapping

• Special purpose VLSI for vision

• Segmentation

• Force sensing and control

Programming and Intelligence

• AL (arm language) one of the most advanced robot programming

languages available today,

• ACRONYM system for geometric modeling and reasoning for modelbased vision and for programming robots,

• Work on CAD, resulting in a simulator for mechanical systems

and robot arms, and for off-line programming.

Mobility

• A continuing program of navigation and perception.

53
Table 8B-1

Institution : Artificial Intelligence Lab., M.I.T.

Research Funding: $1.75M DARPA and ONR.

I. Robotics Mechanisms and Control

Principal Researchers: Hollerback, Seering, Mason, Pubrick, Salisbury,

Asada, Lorenzo Perez, Brooks, Hillis,

Manipulation

Design of Dexterous Hands

- Tendon-actuated three-fingered hands (for use with PUMA) lighter,

nimbler, more compliant.

- Controllers for tendon-actuated fingers and shoulders.

Manipulator Design

- Direct drive actuation

- Developed a recursive Lagragian formulation of the dynamics of

manipulators

- Developed a simulation for robot linkages and biological arms.

- Hybrid controller that allows the available degrees of freedom to be

divided into position servoing and force feedback servoing as needed

to achieve the compliance required for assembly and operation under load.

Perception

Force and tactile sensors for automating compliant assembly task.

- High resolution tactile sensor.

- Strain gage force sensor.

Programming and Intelligence

Developing a high level language based on a geometric modeling system

developed at the AI Lab.

- Allows errors to be modeled, as well as keeping track of spatial relationships in ways programmers find natural.

54
- Connecting a PUMA directly to a LISP machine to be able to program it

in this geometric modeling language - to Improve robot speed, accuracy

and Interfacing with sensors and effectors.

II . Robot Vision for Parts Assembly and Repair

Principal Researchers: Brady, Crimson, Horn, Larson, Nishihara, and Poggio.

Perception

Acquire possibly overlapping laminar workpieces off a textured background.

- Real time edge detection carried out in hardware using a camera developed

at M.I.T.

- M.I.T. constructed frame grabber provides a Interface to commercially

available area array cameras for real time processing of image files.

Utilize 3D vision for guiding the acquisition of objects from a bln of

parts

- Photometric stereo to determine local surface orientation.

(1) Using multiple light sources and assuming Lambertian reflectance.

(2) Use of diffuse light sources to determine layout of scences containing

specular objects.

- Binocular stereo vision utilizing matching of detected edges to obtain

depth.

55
Table 8B-2

Institution: Mechanical Engineering Dept., M.I.T.

Principal Researchers: Thomas B. Sheridan, Nam P. Suh, Nathaniel H. Cook,

Neville Hogan, (10 grad, students)

Research Funding: Approximately $500K/Yr. from NOAA, ONR, NSF and NIHR.

I . Use of Robotics (Teleoperators) Underseas

Manipulation

Teleoperators under computer control.

Perception

Tactile sensing for oceanbottom operations.

Measurement arm attached to base of robot to subtract out base motion.

Vision tradeoffs of video frame rate, resolution and number of gray

scale levels.

I I . Upper Extremity Prosthetics

- movement control

- adaptive control stiffness

56
Table 8C

INSTITUTION: Robotics Institute, Carnegie-Mellon University

DIRECTOR: Raj Reddy

RESEARCHERS: 15 Faculty

6 Research Associates

18 Graduate Students

23 Staff

RESEARCH FUNDING: Approx. $4 M/year from Westlnghouse, Digital

Equipment Corp, ONR, DARPA, NSF, and other

industrial sponsors.

Manipulation

- Direct drive rare-earth magnet electric actuators for manipulator joints

- Multi-processor control systems

- Dexterous end effectors for handling compliant parts

Perception

- Laser range sensor

- Proximity sensors

- Binary and gray scale image analysis and understanding

- Touch, force, pressure, vibration, temperature, and chemical sensors

- 3D sensors

Mobility

- Mobile cart

- Gait and legged motion analysis

Manufacturing

- Factory of the Future Project

- integrated factory system

- research testbed for manufacturing problem

- Intelligent Factory Management System

- Job-shop planning and scheduling

- Factory modelling and analysis

- Flexible user interfaces

- Flexible machining

- multiple robot and machine integration and control

- Flexible Assembly

- multiple robot and sensor integration and control

- multiple processor controllers

- electronic assembly

Programming and Intelligence

- Knowledge representation

- Problem-solving techniques for planning and scheduling

57
Table 8D

Institution: Rhode Island University

Principal

Researchers: John Birk

Robert Kelley

Research Funding: Approx. $200 K/yr. from NSF

Approx. $1 M/yr. from Industrial Associates

Manipulation

• Dexterous end effectors

• Articulated hands (includes manipulation within the hand itself).

• Design methods for hands

Perception

• Visual Pose estimation

• Touch sensing

Programming and Intelligence

• Robot programming language

Manufacturing

• Successfully demonstrated bin-picking

• General methods to enable robots with vision to acquire, orient and

transport workpieces

• Design of production systems utilizing robots

58
Table 8E

Institution; University of Florida

Principal

Researcher: Del Tesar (+10 faculty from 5 disciplines

and 20 grad, students)

Research

Funding: Approx. $1 M/yr. from DOE, NSF and State of

Florida

Manipulation

• Human augmentation with real-time digital computer assistance

(computer-based teleoperators)

• Robot geometry, kinematics and dynamics - to achieve high-level

dexterity

• Control of structurally flexible manipulators

• On-line robot precision under load (for batch processing)

• Actuator modules - to enable construction of customized manipulators

Programming and Intelligence

• Hierarchical control in an unstructured environment - using force feedback

assistance

59
Table 8f

Robot Research at Other Universities

University

$

Yearly

Funding

Manipulation,

Perception,

Mobility

Programming

and Intelligence

Purdue lOOK

NSF

• Robot Vision • Robot Control System

* Robot Programming

Languages

U. of Mass 300K

NSF

U. of Mich 500K

State

of MI

• To Implement a new Robot Lab

U. of MD • Robot Vision

U. of

Rochester

200K

NSF,0NR

• Machine Vision • Computer graphics

language - PAUL

U. of

Arizona

lOOK • Teleoperators

Ohio State

( Robert

McGhee

)

IM NSF

& DARPA

• Legged locomotion • Manipulator Dynamics

and Controls

U. of IL

U. of PA

U. of WA

U. of Texas

• Small projects in

robotics and

related work

R.P.I.

Manufacturing

• Modeling of

parts flow thru

in industrial

plants

• Design of parts

for automatic

assembly

• Economic application of assembly

robots

• Automated

Manufacturing

60
dexterity, improved sensory perception, higher-order robot programming languages,

increased robot intelligence, and computerized manufacturing and assembly. As

indicated in Table 9, the focus of robotics research in the U.S. non-profit research laboratories is on manufacturing, with emphasis on computerized planning,

factory management, assembly and inspection.

Table 10 indicates robotic research efforts in commercial U.S. companies.

The emphasis is on manufacturing, assembly and computer vision for part recognition

and inspection.

Information on research and development in some of the U.S. companies specializing in sophisticated computer vision is indicated in Table 11. The emphasis is on

vision for inspection and robot control, and on simplified command capability.

Table 12 gives an indication of the research activities of the principle U.S.

robot manufacturers. Emphasis is on advanced control systems, improved programming techniques and mechanical design.

Though there is no central focus for robotics in the U.S. Government, several

government agencies are involved in robotics research and a number of others are

looking into applications.

Table 13 (roughly reflecting FY-82 funding) provides an overview of robotics

research performed within the U.S. Government. In general, these activities can

be viewed as exploratory research into robotic applications.

Table 14 presents an overview of robotics research (roughly reflecting FY-82

funding) supported by the U.S. Government. NSF primarily supports basic research,

ONR supports a mixture of basic and applied research, while the V.A. is focussed

on potential applications for the handicapped.

Table 15 (again roughly reflecting FY-82 funding) indicates some of the government supported robotics applications development in the manufacturing and rework
The various U.S. military services are considering future robotic research

efforts that could lead to replacing humans, or leveraging their capabilities,

for dangerous jobs or missions and for increased productivity.

It is estimated that the current (FY-82) overall U.S. Government support

for robotics research and development is approximately 18 million dollars per year.

Research in Europe and the Soviet Union

The Western Europeons are estimated to be spending two to four times as

much as the U.S. on robotics and related research. (A recent survey of teleoperator

and robotics research in Europe indicates that $10 million per year is being spent

in telemanipulation R&D alone, (Plantier and Bodmer et al., 1981, p. 8).)

As in the U.S., the emphasis is on making robots more intelligent and versatile.

"Germany, Italy, Sweden, France, Switzerland and the United Kingdom, either in

national research laboratories or in robot manufacturers, are presently the most

advanced European countries in computer vision and sensor-based robots" (Plantier

and Bodmer et al., 1981, p. 12).

No figures are available on robotic research in the U.S.S.R. and the Eastern

Block Countries, but they appear to be making a strong effort in robotics in order

to improve productivity and overcome labor shortages. However, the Soviets lack

of ready access to digital and computer circuitry is hindering their research effects

in advanced robots (Teschler, 1981, p. 48).

62
Table 9A

Robotics Research

Non-Profit Lab: Charles Stark Draper Lab, Inc.

Principle

Researchers: Jim Nevins

Dan Whitney

Research Funding: Approx. $1 M/yr. from NSF and Industry

Manufacturing

• Part-mating science for assembly operations

- Sensory feedback

- Adaptive learning

- Passive compliance

-Models of assembly process

• Economic and configuration modeling for making choices

among competing technologies in designing industrial

systems

• Flexible assembly system design

• Operational strategies

• Product design for producibility

Applications, with industry, range from precision products with

low annual volume to products with annual volumes in excess of several

million. Techniques developed are also applicable to other environments,

such as space, underwater, and medical.

63
Table 9B

Robotics Research

Non-Profit Lab: SRI International

Principal

Researcher: David Nitzan

Research Funding: Approx. $1 M/yr. from NSF and

Industrial Affiliates

Perception

• Machine vision for inspection and recognition

• Robot vision involving structured light, a combination of

binary and grayscale vision, and local features to

distinguish overlapping parts

Manufacturing

• Vision-guided arc-welding

• Printed circuit board inspection

• Programmable assembly

64
Table 10

Company

GM

Warren, Ml

GE

Schenectady, NY

Westinghouse

Pittsburg, PA

IBM

Texas

Instruments

Martin Marrietta

Denver , CO

($3M/yr.

)

Robot Research in Commercial U.S, Companies

Research Activities

• Robot vision systems for inspection and part transfer

(KEYSIGHT, CONSIGHT)

• Automatic small parts assembly

• Flexible computer-controlled high-speed precise assembly,

to handle different versions of a one product family from

a data base

• Evaluation of existing robots for use in GE manufacturing

• Robotics Research Laboratory evaluating use of commercial

robots

• Supports productivity research at CMU ($lM/yr.)

• Adaptable Programmable Assembly System (APAS) ($lM/yr),

cost-shared with NSF

• Supports Research on Teleoperators for Nuclear Power Plants

at U. of Florida

• Developed own robots for its manufacturing operations

• Developed robot programming languages - AUTOPASS and

EMILY

• Developed own robot for assembly and testing of hand

calculators

• Robotics research directed at DOD and NASA interests

- Autonomous devices

- Automated diagnosis/checkout

- Speed requirements of space manipulators

- Simplified high-speed coordinate transformations

- CAD robot program planning system

• Two arm coordinated manipulation (for DOE)

65
Table 11

Near-Term

Automatix

Burlington, MA

Machine Intelligence

Corp.

Sunnyvale , CA

R&D in Sophisticated Computer Vision Systems

• Robot Vision

• Robot Microcomputer Control Systems

• Robot Programming Language (RAIL)

• Commercial Robot Applications - arc welding

• Computer Vision Systems for Turnkey Inspection

• Materials Handling and Assembly Systems

• Univision System developed in cooperation

with Unimation for the PUMA robot - programmed

in VAL robot language. This is the first

commerically-available seeing robot.

• Research in person/robot communication for

non-computer oriented people.

66
Table 12

Research Activities of U.S, Robot Manufacturers

Unimation

• Advanced control systems

• Calibration techniques

• Programming techniques

• Mobility systems

• UNIKATM vision for arc welding

Cincinnati Milacron

• New control system architectures

• Programming language

• Mechanical design

Other Robot Manufacturers

• Similar activities - all aggressively developing new and

improved product lines.

67
Table 13

Organization

National Bureau

of Standards

$1-1/2 M/yr.

In-House Government Robotic Research

Research Activities

Standards

• Robot Interface Standands

• Robot Performance Measures

• Programming Language Standards for

- robot systems

- integrated computer-aided manufacturing

Advanced Concepts for

• sensory-interactive control systems

• modular distributed systems

• sensor interfaces to the control systems

of robots and machine tools

• vision systems

Focussed Application

• Automated Manufacturing Research Facility

- planning

- scheduling

- routing

- inspection

- demonstration of a completely automated

machine shop consisting of computer

controlled NC machines served by

robots and robot carts, producing

mixed configurations of parts on demand

68
Table 13 (cont.)

Organization

NASA

$lM/yr.

12 man years

Naval Ocean Systems

Center

San Diego, CA

$65 OK

In-House Government Robotics Research

Center Activity

Langley Research Center • Robotics Laboratory to

Hampton, VA develop teleoperated and

automated spacecraft

servicing techniques

- Trajectory planning and

multi-arm coordination

- Man-machine interface

for remote systems

- Operating system for

concurrent processes

in a distributed computer

network

- Robotics simulation

Marshall Space Flight Center • Research on end-effectors

Huntsville, Ala.

• Man-machine interface for

teleoperation

JPL • Research in Hand-Eye

Pasadina, CA Coordination

• Control of structurallyflexible manipulators

• Teleoperators

• 3 fingered hand with

Stanford U.

Exploring application of robot and teleoperator systems

• submersibles

• land vehicles

• underwater manipulators

• stereo optic and acoustic vision

• remote presence

• autonomous robots - knowledge representation

and automatic decision-making

• complex robot systems specification and

verification

69
Table 13 (cont.)

In-House Government Robotics Research

Naval Ocean Systems Center Exploring the use of telepresence

Hawaii (teleoperation with sensory feedback

$300K directly to the operator, giving a sense

of being at the work site) for use in

materials handling aboard ship.

70
Table 14

U.S, Government Supported Robotic Research

Organization: NSF

Washington, D.C.

Program (FY-81) Organizations Supported and P.I.

Production Research U. of R.I. - Birk

$2M

(W. Spurgeon) C. N, K, Corp. - Hayes

(Emphasis on robotics SRI - Barnard

for assembly) SRI - Nitzan

Merrick Corp. - Cook

Taylor Sys. Engr. - Kerth

Stanford U. - McCarthy

M.I. Corp.

& Binford

- Rosen

U, of Mass, - Boothroyd

Mechanical Systems U.C, Berkeley - Tomizuka

(a. Strauss)

$700K - Dornfield

(Emphasis on UCLA - Dubowsky

control, dynamics.

kinematics, and U of FL - Sandor

design) It - Hole

U. of Nebraska - Schade

Rut gers - Lee

Stevens - Tricano

Columbia - Freudenstein

Ohio State - Srinivasan

U, of Conn, - Huston

Program (FY-81) Organizations Supported and P«I.

Automation, Bioengineering and

U. of PA - Rozina

Sensing Systems Purdue - Kashyap

$750K CMU - Sanderson

(Emphasis on SUNY - Shapiro

robotics vision U. of S. Carolina - Cannon

and intelligent U. of IL - Ahuja

controls) Purdue - Funkunaga

U, of Minn, - Wechsler

U. of IL - Cose

RPI - Saridis

Yale - Narendra

Activity

Part acquisition using

vision

Part Recognition

Auto, Visual PCB Inspection

M.I, applied to Industrial

Auto.

Adaptive arc welding

Adaptive arc welding

Integrated Programmable

Auto

.

Natural Language

Economic applications

Digital Control of

Industrial Process

Control of arc welding

Dynamics of high speed

machines

Kinematic analysis

Nonlinear dynamic structual

synthesis

Mechanism motions

Design of high speed

mechanisms

Optimum design of

mechanisms

.

Forces in high speed

linkages

Control of machine tool

feeds

Multi-body dynamics

Activity

Scene understanding

Image modeling

Parallel signal processing

Edge detection in images

Auto signal processing

Dot pattern processing

Image feature evaluation

Texture analysis

Auto, scene analysis

Intelligent controls

Self-learning systems

71
Table 14 (cont.)

U.S. Government Supported Robotic Research

Organization: NSF

Washington, D.C.

Program (FY-81) Organizations Supported and P.I,

Public Technology Westinghouse

(A. Schwarzkopf)

$450K

Computer Engineering U. of Rochester

(B. Chern)

$300K CMU

Intelligent Systems VPI

(Y. T. Chien)

$100K U. of PA

- Abraham

- Voelcker

& Requicha

- Chang

- Shapiro

& Haralick

- Bajczy

Activity

Programmable assembly

Geometric modeling

Robot memory

Geometric reasoning

Scene understanding

using active sensing

72
Table 14

U.S, Government Supported Robotic Research (cont.)

Organization

Office of Naval Research

Arlington, VA

$1.7M/yr.

(Marvin Denicoff)

Defense Advanced Research

Projects Agency

Arlington, VA

(Robert Kahn)

$300K/Yr.

Veterans Administration

Washington, D.C,

(Dr. Margaret Gianni)

$400K/yr.

Organization Supported

M.I.T.

$700K/yr.

CMU

$700K

SRI

$25 OK

U. of

Rochester

$100K

M.I.T.

$30 OK

Applied Physics Lab

John Hopkins Univ.

$100K

Rehabilitation Engr. Center

Stanford Univ.

$200K

VA Hospital - West Roxbury

$5 OK

VA Hospital - White River

Junction, N.H.

$65K

Activities

• Fundamental

research in

- vision

- manipulation

- reasoning

• Autonomous

Unmanned Underwater

Vehicles

• Communication and

negotiation among

distributed robots

• Machine Vision

• Fundamental research

in robotic vision,

manipulation and

reasoning (in cooperation

with ONR)

• Robotic work station

for quadriplegics

• Robotic aids for severely

disabled individuals

• Supervisory voice and

head-orientation

command of a PUMA

robot

• Human/Machine Interface

• Myographic wheelchair

control

• Communication control

aid

73
Table 14 (cont.)

Organization Program Organizations Supported & P.I. Activity

A.F.O.S.R. Manufacturing Developing Educational • Assembly in

(A.F. Off. of Science Research centers of manufacturing

Scientific (Tom Walsh) excelle nee Research) $2 M/yr.

Bolling AFB,

D.C.

74
Table 15

U.S. Government Robotic Applications Development

Organization Activity

Naval Air Rework Facility • Development of robots to

San Diego, CA

$3 M/yr. - remove rivets and fasteners from

airplane wings

- strip and repaint aircraft

- perform wing assembly

Air Force Materials Lab.

Wright Patterson AFB

Dayton, Ohio

$lM/yr.

U.S. Army Human Engr. Lab.

Aberdeen Proving Grounds, MD

Office of Manufacturing Technology

DARCOM

• Robotic applications in manufacturing

(Part of ICAM)

• Formulating a plan for military

applications of robotics

• Robot applications in manufacturing

75
XVI. Applications Now and in the Future

Figure 9 indicates current and anticipated future use of robots in manufacture

in Japan. Currently, machine tool processing is dominant, followed by welding,

assembly, and plastic molding. In % of value, assembly, and measurement and

inspection are expected to increase while spot welding, plastic molding and

machining processes are expected to decrease.

Table 16 indicates current and 1990 estimates of major uses for industrial

robots in the U.S. In general the same trends are visible in the U.S. as in Japan,

with welding and materials handling being currently dominant, moving towards

assembly being the dominant application in 1990.

A recent JIRA forecast of robot applications development, introduction and

diffusion is shown in Figure 10. Observe that by 1985 it is anticipated that robots

for assembly, inspection, reactor maintenance, cleaning and ocean development, as

well as all phases of manufacturing, will be in commercial use, with diffusion

relatively complete by 1990. In the last half of this decade it is forecast that

construction and mining robots, robots for fire fighting and even robots to assemble

soft goods like textiles will begin commercial diffusion. In the same time period,

prosthetic arms and legs, and robots for use in patient care and other medical

applications will become widespread.

In the late 1980's, robots for use in various military applications, such

as use in dangerous missions and hostile environments are also expected to appear.

Also in the late 80' s, robots for use in servicing of space satellites are anticipated.

In the 1990's, robots for use in the construction and assembly of large space

structures and in space manufacturing are forecast. Starting around 1995, semiautonomous robots are foreseen for use on extra terrestrial surfaces for exploration,

mining, materials processing, manufacture, and the construction, tending and

maintaining of space installations.

76
Figure

9

JIRA

Forecast

of

Japanese

User

Needs

for

IR’s

by

Production

Process

c

a>E

o>

k.3

o>

O)

in

00

o>

oCO

O)

(0o

>-

eniBA *0 o/„

77

Source:

Research

Div.

of

the

Long-Term

Credit

Bank

of

Japan,

1981,

p.

74.
Table 16

Estimated U.S. Robot Order Trends By Usage

Through 1981 1990

Materials handling

including machine

loading and unloading

, .25-30% 30-35%

Assembly 10% 35-40%

Spot-Welding 35-45% 3-5%

Arc-Welding 5-8% 15-20%

Paint spraying 8-12% 5%

Other 8-10% 7-10%

Source: (Conigliaro, 1981, p. 6)

78
Figure 10

Robot Development,

Introduction and Diffusion

’80 ’85 ’90

Painting

Metai Parts Assembiy

Machining ^

Casting & Forging ^

Piating ^

Car Assembiy

Home Appiiances Assy

Agricuiturai Machinery Assy

Soft Parts Assembiy

Warehousing (

inspection

industriai Cieaning

Agriculture & Forestry

Atomic Energy Development

Construction & Mining

Medical Field

Prosthetic Arms & Legs

Ocean Development

Fire Defense

Development

Diffusion

VUJU

X ^ X y ^ ^

UZZZZZZZZZ21

^zzzzzzzzz

U } 77 J

V///A Introduction Into the Field

Source: JIRA (Hasegawa, 1981, p.171)

79
The general trend is toward robots with ever increasing intelligence and sensory

capabilities. Engelberger (1980, p. 135) suggests that robots will eventually be

entering the service industries, with such diverse applications as garbage collection and fast food preparation and dispensing. He even suggests that "...a

[limited] household robot may be practicable before the end of the 1980's decade."

80
XVII. The Expanding Robot Population

Table 2 presents JIRA’s (The Long-Term Credit Bank of Japan, 1981, p. 52)

estimate of the world’s robot population as of the end of 1979. The figures

shown are for high-quality robots and exclude manually manipulated robots (teleoperators) and fixed sequence robots. Note that Japan with 14,000 robots has

nearly 70% of the world’s total, compared with 16% (representing a little

over 3,000 units) for the U.S.

Figure 11 projects the U.S. robotics industry growth through 1990, when it

is estimated that the U.S. will have roughly 100,000 IR’s in operation, or

some 30 times that of 1979. This estimate is predicated on major diversified

companies - such as GE, Westinghouse , GM and IBM - entering the market as now

appears to be happening.

Table 17 presents JIRA’s forecast for the production of IR’s in Japan through

1990, while Figure 12 indicates how this production will be distributed.

Integrating the results from Tables 2 and 17 and Figure 11, Figure 13 presents

the composite forecast of the industrial robot population through 1990. It appears

that the world’s robot population will pass 1,000,000 in the early 1990’s, and will

therefore have a major impact on the world’s productivity.

A recent article in Business Week (Dec. 14, 1981) indicates that the robot

pace is continuing to accelerate and even the figures quoted here may be very

conservative

81
Millions $

Year

Figure 11

U.S. Robotics Industry Growth

1980-1990

Year 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990

Estimated Sales

($ Millions) $ 100 150 215 280 395 540 725 875 1125 1520 2070

Estimated Units

Sold 1450 2100 3075 4000 5600 7715 10350 13000 17000 23000 31350

(Assumed Avg. Robot Cost Approx. $70K Each)

Compound Growth of Revenue Per Year: 35%

Source: Congllaro , 1981

82
Table 1?

Forecast of Aggregate Demand in Manufacturing for All Classes of

IRs in Japan

(Unit $ Billion)

1

1

1

!

1 1980 1985 1990

Annual

1985/1980

Growth Rate

1990/1985

1

1

i Domestic Demand

1 0.3 0.8 to 1.1 1.6 to 2.2 30% lk%

! Exports

1

I

Negligible 0.2 0.4 - 15%

Total 0.3 1.1 to 1.3 2.0 to 2.T 35% lk%

Annual

Production

(Thousands of

Units

19.T h3 to 55 75 to 9^

(Note) The export ratio is based on the JIRA target value of 20% in 1990.

Source: The Long-Term Credit Bank of Japan, I98I, p. 69.

83
Figure

12

JIRA

Estimate

of

Composition

of

iR

Production

in

Japan

(Based

on

Vaiue)

84

Source:

Research

Div.,

The

Long-Term

Credit

Bank

of

Japan,

1981,

p.

73.
Thousands

of

Robots

85
XVIII. Future Directions

Hasegawa (1981, pp. 169-170) makes the forecast of unfolding robot technology

given in Figure 14. Based on these technological trends, he states that

"...industrial robots will become smarter, smaller, quicker, lighter, stronger,

more ingenious, easier to operate, more intelligent and less expensive that they

are now."

Engelberger (1980, p. 118) lists in Figure 5 the qualities sought in future

robots. He indicates that these qualities will be, for the most part available

during the current decade. Of the missing robotic attributes, he considers the

two crucial ones to be vision and tactile sensing.

JIRA (The Long-Term Credit Bank of Japan, 1981, p. 59) indicates that the

technology for robot miniaturization, modularization, high speed operation, and

computer control has advanced to the point where it is now available and will grow

in popularity and acceptance in the first half of this decade. Technology for part

recognition and position determination is also currently available, but won't

receive widespread popularity until the second half of this decade. Technology

for color and behavior recognition, automated assembly, and machine tool and

casting robotics won't become adequately advanced until later in the first half

of the decade and therefore won't achieve widespread popularity until the second

half of the 1980's.

One of the greatest advantages of robotics and automation occurs in the

cases where their use greatly increases equipment utilization. Albus (1981B)

indicates that a robot arc welder cannot weld faster than a human, but by keeping

its torch on the work about 90% of the time, versus no more than 30% of the time

for a human, it can turn out 3 times as much work in the same time. If through

automation more shifts can be worked, productivity is even further increased.

Unfortunately, present day robots cannot set up their own work, so this reduces

their productivity advantage. Albus conjectures:

86
Forecast of Robot Technology

Materials

Lighter and stronger materials for robot structures will be developed.

These will include both organic and non-organic substances. As a consequence, the size and weight of robots will decrease.

Power Sources

Small, high-capacity power sources such as nuclear powered generators or fuel cells will allow robots greater freedom of location and movement.

Actuators

Small, high performance actuators will appear. Robot designs will be more stylish.

Sensors

Small, reliable, high-performance, low-cost artificial eyes, ears, and

tactile sensors will be developed and even low-cost robots will have

recognition capability.

Control Systems

Advances in microcomputer technology will dramatically improve robot

control. Control system costs will also be greatly reduced.

Communications Systems

Remote control of industrial robots will be enhanced through development

in fiber optics, wireless communcations , etc.

Processing Methods

New, non-mechanical processing methods such as laser beam cutting,

welding and other high-energy processing methods will reduce the processing

load on robots and make the design work for them simpler.

Robot Body Structure

Utilization of biomechanical concepts will result in superior robot

structures.

Software

Robot use will become easier through the development of robot languages,

self-diagnosing systems and robot application technologies. Software

will contribute to reducing the design work needed on robots.

87

(continued on next page)
Figure 14 (cont’d)

° Integrated Systems

Techniques for integrating production systems composed of men, machines

and robots as well as their peripheral devices will be found. Further,

their design processes will be computerized.

Source: Hasegawa, 1981 PP 169-170.
Eventually, welding robots will be sufficiently sophisticated

to work from plans stored in computer memory and to correct errors which may occur during a job. Welding robots will then be able

to work nights and weekends (four shifts per week) completely without

human supervision. At that point productivity improvements over present methods of many hundreds of percent become possible. Unfortunately, we are a long way from that today. There are many

difficult research and development problems that must be solved

first. Unless the level of effort in software development is

increased many fold, these improvements will not be realized for

many years.

In the metal cutting industry, Albus (pp. 16-18) predicts.

During the 1980 ’s, robot sensory and control capabilities

will improve to the point where robots can find, and load unoriented

parts, or in some cases, even pick parts out of a bin filled with

randomly oriented parts lying on top of each other. This may improve productivity by hundreds of percent because it will make it

possible to install robots in many existing plants without major re- engineering of production methods....

By 1990 robots may begin to have a significant impact on mechanical assembly. There has been a great deal of research effort

spent on robot assembly. Unfortunately, the results have not been

spectacular—yet. On the one hand, robots cannot compete with

classical so-called "hard automation" in assembly of mass produced

parts. General purpose machines like robots are still too slow and

too expensive to be economical for mass production assembly tasks.

On the other hand, robots cannot yet compete with human assembly

workers in small lot assembly. Humans are Incredibly adaptable,

dexterous, as well as fast, skilled, and relatively cheap compared

to robots. .

Nevertheless, progress is being made and will continue.

Robot capabilities will gradually increase. Sensory systems will

become more sophisticated and less expensive. The cost of computer

hardware is dropping rapidly and steadily with no sign of bottoming

out. Software costs are likely to be the major impediment to robot

development of the foreseeable future, but even these are slowly . yielding to the techniques of structured programming and high level

languages

Eventually, extremely fast, accurate, dexterous robots will

be programmed using design graphics data bases which describe the

shape of the parts to be made and the configuration of the as- semblies to be constructed. Eventually, robots will be able to

respond to a wide variety of sensory cues, to learn by experience

89
and to acquire skills by self optimization. Such skills can then

be transferred to other robots so that learning can be propagated

rapidly throughout the robot labor force.

Sometimes, perhaps around the turn of the century, robot

technology will develop to the degree necessary to produce the

totally automated factory. In such factories robots will perform

most, if not all, of the operations that now require human skills.

There will be totally automatic inventory and tool management,

automatic machining, assembly, finishing, and inspection systems.

Automatic factories will even be able to reproduce themselves.

That is, automatic factories will make the components for other

automatic factories.

Once this occurs, productivity improvements will propagate

from generation to generation. Each generation of machines will

produce machines less expensive and more sophisticated than

themselves. This will bring about an exponential decline in

the cost of robots and automatic factories which may equal the

cost/performance record of the computer industry...

Eventually, products produced in automatic factories may

cost only slightly more than the raw materials and energy from

which they are made.

In summary, what we see emerging are robots with increasing intelligence,

sensory capability, and dexterity. Initially, we will see an increasing use

of off-line programming of computer-controlled robots, using improved robot command

languages. Provision will be made to include the role of sensors, such as

vision and touch, in this programming. Later, self-planning will emerge as

higher and more general commands are given to the robot. At this point, the

marriage of robotics and artificial Intelligence will be virtually complete.

At the same time as all this is taking place, robotic hands with improved dexterity

(so that it will not be necessary to move the entire arm for fine motions)

and advanced control systems to support this dexterity, will emerge. Also emerging

will be robots with coordinated multiple arms and eventually even legs, supported

by even more sophisticated control systems.

90
As this evolution progresses, information and intelligence will become

the dominant factor in robotics, with the manipulator devices and sensors

shrinking in importance to the skeleton that undergirds this dominating

"ghost in the machine".

91
References

1. Albus, J. S., Behav^orj_ an^ Peterborough,

N.H.: BYTE Books, 1981A.

2. Albus, J. S., "Industrial Robots and Productivity Improvement," Summ ary and Report o f an Exploratory W orkshop on the

Social Im pact o f Robot ics , Office of Technology Assessment,

Washington, D.C., July 24, 1981B,

3. Barrow, H. G., and Tenenbaum, J. M,, "Computational Vision,"

P£0 £££^^n^£ Vol. 69, No. 5, May 1981, pp. 575-

579 .

4. Birk, J. R., and Kelley, R. B., "An Overview of the Basic

Research Needed to Advance the State of Knowledge in

Robotics," I.^E E T££n S y s t .M £n_j_ Vol. SMC-11, No.

8, Aug. 1981, pp. 571-579.

5. Congliaro, L., "Robotics Presentation Institutional Investors Conference; May 28, 1981," Bache Robotics Ne w s letter

81-429, New York: Bache Halsey Stuart Shields, Inc., Oct.

28, 1981.

6. Engelberger, J. F., Robots in Practice , New York: American

Management, 1980.

7. Gleason, G. J., and Agin, G. J., "A Modular Vision System

for Sensor-Controlled Manipulation and Inspection," ProceedR o b o _t£

Washington, D.C., March 1979.

8. Kelly, F., and Huston, R., "Recent Advances in Robotics

Research," SAE Technical Paper 800383, Feb. 1980.

9. Kinnucan, P., "How Smart Robots are Becoming Smarter," High

Z2£il22l.2£Z> Vol. 1, No. 1, Sept/Oct. 1981, pp. 32-40.

10. Ogawa, E., "Technical Development in Small and Medium-sized

Enterprises," in Ro^b£_t£ i^n £h£ JapaneseEconomy,Sadamato,

K. (Ed.), Tokyo: Survey Japan, 1981, pp. 113-150.

11. Plantier, M. and Bodmer, R., et al., " Te 1 e o p e r a t i o n and

Automation: A Survey of European Expertise Applicable to

Docking and Assembly in Space," ESTEC Contract No.

4402/80/NL/AK( SC) , Geneva; EUROSTAT, S.A., May 1981.

£l.2£22.^i2££ 2^. Z,2£££ K2£li£^2£ 22 ^2k££

I_n £££^3£££ , NBSIR 80-2152, January 1981.

13. Pro ceedings o f the NB S / R I A Robo tic s Re search W orkshop , Held

at NBS, Gaithersburg, MD , Nov. 13-15, 1979, NBS SP 602.

92
14. Rheinhold, A. G., and Vanderbrug, G., "Robot Vision for

Industry; The Autovision System," Robotics Age , Vol. 2,

No. 3, Fall 1980, pp. 22-28.

15. "Robots Join the Labor Force," Business W eek , June 9, 1980,

pp . 6 2-76 .

16. Rosen, C. A., "Machine Vision and Robotics: Industrial

Requirements," TN 174, SRI International, Menlo Park, CA,

November 1978.

17. Rossol, L., "Vision and Adaptive Robots in General Motors,"

Controls , Stratford-upon-Avon, April 13, 1981, pp. 277-287.

18. "Russian Robots Run to Catch Up," Bu£i.n£££ Aug. 17,

1981, p. 120.

19. Saveriano, J. W., "Industrial Robots Today and Tomorrow,"

Robot i c s Age , Vol. 2, No. 2, Summer 1980, pp. 4-15.

20. Scheinman, V., "The Autovision System Computerized Vision

for Factory Automation," Presented at the 11th International

Sympo s i urn on Industrial Robot

, Tokyo, October 1981.

21. Simons, G. L., Robo^^ i^n I.ndu£^ry^, Manchester, England: The

National Computer Center Ltd., 1980.

22. Tanner, W. R., "Basics of Robots", in I.nd u£_t £^a 1^ Robots , Z£l.£££ i ££££!.£> Tanner, W. R. (Ed.), Dearborn, MI:

SME, 1979, pp. 3-12.

23. Toepperwein, et al., "ICAM Robotics Application Guide

(RAG)," ARWAL-TR-80-4042 , Vol. II, Air Force Wright Aero.

Labs, Wright Patterson AFB, Ohio, April 1980.

24. Teschler, L., "How Good is Soviet Robot Technology,"

Machine Design , Oct. 8, 1981, pp. 43-48.

25. "The Push for Dominance in Robotics Gains Momentum,"

Bus ine s s Week , December 14, 1981, pp. 108-109.

26. Umentani, Y., "Research and Development of IR's in Japan,"

in Rok£££ ££ £ll£ £.££.£££££ l.££££5IZ> Sadamato, K. (Ed.),

Tokyo: Survey Japan, 1981, pp. 79-104.

27. Yoneraoto, K. , "The Art of Industrial Robots in Japan," in

—£££££—££££ £^. £ll£ l.£££££££i.£££l. ^ZTM.2.££l.£TM ££

In du s trial Robots , Tokyo, 1981, pp. 1-7.

93
NBS-n4A (REV. 2 -ec) U.S. DEPT. OF COMM.

4.

1 . BIBLIOGRAPHIC DATA

SHEET (See/ nstruction s)

PUBLICATION OR

REPORT NO.

2. Performing Organ. Report No. 3

NBSIR 82-2479

TITLE AND SUBTITLE

, Publication Date March 1982

ON OVERVIEW OF ARTIFICIAL INTELLIGENCE AND ROBOTICS

Volume II -Robotics

5. AUTHOR(S)

Dr. William B. Gevarter

6. PERFORMING ORGANIZATION (If joint or other than NBS, see instructions) 7. Contract/Grant No. national bureau of standards

DEPARTMENT OF COMMERCE 8. Type of Report & Period Covered

WASHINGTON, D.C. 20234

9.

SPONSORING ORGANIZATION NAME AND COMPLETE ADDRESS (Street. City. State, ZIP)

National Aeronautics and Space Administration

Headquarters

Washington, D.C. 20546

10.

SUPPLEMENTARY NOTES

[^21 Document describes a computer program; SF-185, FIPS Software Summary, is attached.

11.

ABSTRACT (A 200-word or less factual summary of most significant information. If document includes a significant

bibliography or iiterature survey, mention it here)

This report provides an overview of the rapidly changing field of robotics. It is

intended to be read by the technically oriented layman, such as engineering managers,

government funding offices, and others who desire an overall perspective of the field

but are unable to obtain it from the highly technical and unintegrated literature in

the field, or from the more flamboyant but non-technical feature articles in the

popular press.

The report incorporates definitions of the various types of robots, a summary of the

basic concepts utilized in each of the many technical areas, review of the state-ofthe-art and statistics of robot manufacture and usage. Particular attention is paid

to the status of robot development, the organizations involved, their activities, and

their funding. A 5-10 year forecast of the emerging technology is also included.

The majority of the material in this report is drawn from the activities in the U.S.

and Japan, the principle players in the world of robotics.

12.

KEY WORDS (S/x to twelve entries; alphabetical order; capitalize only proper names; and separate key words by semicolon s) Applications; Forecast; Japan; Overview: Research and Development; Robot;

State-of-the-art

13.

AVAILABILITY

[X]] Uni i mi ted 03] For Official Distribution, Do Not Release to NTIS rn Order From Superintendent of Documents, U.S. Government Printing Office, Washington, D.C.

20402.

14. NO. OF

PRINTED PAGES

15. Price

[IQ Order From National Technical Information Service (NTIS), Springfield, VA. 22161

USCOMM-DC 6043-P80

y

I

I