Measurement and Control. in Tele-Existenceand Artificial Reality
Susumu Tachi*, Hirohiko Arai** and Taro Maeda**
事 RCAST, THE UNIVERSITY OF TOKYO
4-6-1, KOMABA, MEGURO-KU,TOKYO 153 JAPAN帥 MECHANICAL ENGINEERING LABORATORY, TSUKUBA 305 JAPAN

Abstract
Tele-existence is a cone泡pt named for the technology which enables a human being to have a real time
sensation of being at the place other than the place where he or she actually exists, and is able to interact
with the remote and/or virtual environment. He or she can tele-exist in a real world where the robot exists
or in a virtual world which a computer has generated. It 1s possible to tele-exist in a combined
environment of real and virtual. Artificial reality or virtual reality is a technology which presents a humanbeing a sensation of being involved in a realistic vIItual environment other than the environment where
he or she really exists, and can interact with the virtual environment. Thus tele-existence and artificial
reality are essentially the same technology expressed in different manners. In this key note paper, theconcept of tele-existence and/or artificial reality is considered from the standpoint of measurement andcontrol, and an experimental tele-existence system is introduced, which enables a human operator to havethe sensation of being in a remote real environment where a surrogate robot exists and/or virtualenvironment synthesized by a computer.
1. Introduction
It has long been a desire of human beings to project themselves in the remote environment, i.e., to havea sensation of being present or exist in a different place other than the place they are really exist at thesam巴 time.
Another dream has been to amplify human muscle power and sensing capability by using machines while
reserving human dexterity with a sensation of direct operation. In the late 1960s research and development
progrむn was planned on a powered exoskeleton that a man would wear like a garment. A concept ofHardiman was proposed by General Electric Co., for example, that a man wearing the Hardiman
exoskeleton would be able to command a set of mechanical muscles that multiply his/her strength by afactor of 25, yet in this union of man and machine he would feel object and forces almost as if he or she
were in direct contact. Howev巴r, the project was unsuccessful because of the following reasons: (1) It is
potentially quite dangerous to wear the exoskeleton when we consider the malfunction of the machine. (2)Space inside the machine is quite valuable to store computers, controllers, actuators and energy source ofthe machine. Thus it is not at all a practical design to use it for a human operator.
With the advent of science and technology, especially measurement and control technology, it has become
possible to challenge for the realization of the dreams. The conc巴pt of projecting ourselves by using
robots, computers and cybernetic human interfac巴 is 伺lled tele-existence or telepresence. Adding toproject ourselves or tele-exist in a remote real world, projecting ourselves or tele-existing in a computer
generated virtual world is becoming possible. The latter concept is usually called artificial reality or virtual-1241-

reality.
Tele-existence aims at a natural and 巴 fficient remote control of robots by providing白e operator with areal time sensation of presence. It is an advanced type of teleoperation system which enables a humanoperator at the con位。ls to perform remote manipulation tasks dexterously with the feeling出at he or she
exists in one of the remote anthropomorphic robots in the remote environment, e.g., in a hostile environ­ment such as those of nuclear radiation, high temperature, and deep space.百1e authors have been working
on the research for the improvement of出e teleoperation by f田ding back rich sensory information which
the remote robot has acquired to白e operator with a sensation of presence, the concept which was bornindependently both in Japan and in the United Stat凶. It is dubbed tele-existence [1-6］泊Japan andtelepresence [7-9] in the United States.
In the previous reports [1,2], the principle of the tele-existence sensory display was proposed. Its desi伊procedure was explicitly defined. Experimental visual display hardware was built, and the feasibility ofthe visual display with the sensation of presence was demonstrated by psychophysical experiments using
the test hardware. A method was also proposed to develop a mobile tele-existence system, which C加 be
remotely driven with the auditory and visual sensation of presence. A prototype mobile tel巴vehicle system
was constructed and the feasibility of th 巴 method was evaluat巴d [2]. The principle of active power
assistance was applied for controlling the visual display with two degrees of freedom [3,4]. In order tostudy the use of the tele-existence system in the artificially constructed environment, the visualtele-existence simulator was designed, a quasi-real-time binocular solid model robot simulator was made,and its feasibility was experimentally evaluated [5].
In the recent paper [6], the first prototype teleーは 1stence master slave system for remote manipulation
experiments was designed and developed, and a preliminary evaluation experiment oftele-existence wasconducted.
In this key note paper, the concept of tele-existence and/or artificial reality is considered from thestandpoint of measurement and control, and an experimental tele-existence system is introduced, whichenables a human operator to have the sensation of being m a remote real environment where a suηogate
robot exists and/or virtual environment synthesized by a computer. An experimental tele-existence system
in real and/or virtual environment is designed and developed, and by conducting an experiment comparing
a tele-existence master slave system with a conventional master slave system, efficacy of thetele-existence master slave system and the superiority of the tele-existence method is demons仕ated
experimentally.
2. Tele-Existence Master Slave System
Figure 1 shows the schematic diagram of the tele-existence master slave manipulation system. 百1etele-existence master slave system consists of a master system with a visual and auditory sensation ofpresence, computer control system and an anthropomo中hie slave robot mechanism with an arm having
seven degrees of freedom and a locomotion mechanism. 百1e operator's head movement, right arm
movement, right hand movement and other auxiliary motion including feet motion are measured by themaster motion measurement system in real time without constraint. The measured head motion signal,arm
motion signal, hand motion signal, and auxiliary signal are sent to the four computers, respectively. Each
computer generates the command position of the slave head movement， 出e arm movement, hand
movement or locomotion of the slave robot.τne servo con仕oiler controls the movement of 出e slave
anthropomorphic robot. A six axis force sensor mstalled at the wrist joint of the slave robot measur，回theforce and torque exerted upon contact with an o吋ect.百1e measured signal is fed back to the computerin charge of the arm control through A to D converters. Force exerted at the hand when grasping an o吋ect
is also measured by a force sensor installed on the link mechanism of the hand.百1e measured signal isalso fed back to the computer in charge of the hand control through another A to D converter.
A specially designed stereo visual and auditory mput system mounted on the neck mechanism of the slave-1242-

robot gathers visual and auditory infonnation of the remote environment.百1ese pieces of infonnation aresent back to the master system, which are applied to the specially designed stereo display system to evoke
sensation of presence of the operator.
Measured human movements (head, arm, hand, and auxiliary) are also applied to another computer whichis in charge of the generation of computer graphi田（Silicon G raphics IRIS 120GTX) throu酔a dedicated..
computer for measurement.τ'he graphiαcomputer generates two shaded graphic images which are applied
to批3D visual display through superimposers. The measured pieces of infonnation on the human
movement are used to change the viewmg angle, distance to· the o吋ect, and condition between the object
and山hand in real time (10-20 Hz). the operator se回 the three dimensional virtual environment in frontof his view, which changes according to his movement. He or she can interact with either 出e realenvironment which the robot observes, or the virtual environment which the computer generates. The
virtual environment 伺n be superimposed on. the real environment.
Figure 2 shows a general view of出e tele-existence master slave system designed and developed. The
slave robot has a locomotion mechanism and a hand mechanism.百1e robot has also a three degree of
freedom neck mechanism on which a stereo camera is mounted. It has an arm with seven degrees of
freedom, and a torso mechanism with one degree of freedom (waist twist）. τ'he robot's structuraldimensions are set very clos巴 to those of a human [4].The weight of the robot is 60 kg, and the arm cancarry a I kg load at the maximum speed of 3 m/s.τ'he precision of position control of the wrist is ± 1 mm.
The dimensions and arrangement of the degree of fr号edom of the robot are designed to mimic those of
the human being.百1e motion range of each degree of freedom is set so that it will cover白c movementsof a human, while the speed is set to match the moderate speed of human motion (3 mis at th巴 wrist
position
A combination of a D. C. servo motor and a hannonic drive reduction mechanism is used as an actuator
for each joint 巴xcept the elbow extension/flexion, which includes conventional gears.百1e location of the
motors is designed so that the appearance of the ann resembles a human ann as closely as possible, andthe range and the speed of the manipulator satisfy the necessary specifications.
百1e impedance control is used in this experiment. The computer generates a pulse sequence to assign thedesired position based on the calculation of the desired impedance assigned at the hand and measured
force, and the impedance based position control is conducted by counting the difference between thecomputer generated pulse and the measured pulse from an encoder.
A schematic figure of the locomotion mechanism designed is shown in Fig. 3. It is a planar motion
・mechanism whose position is assigned by polar coordinate (r, 0), where r = 500 1500 [mm] and 0 = 0
270 [deg]. The orientation of the robot is assigned by the waist rotation angle of the robotゆ，where・= -150 150 [deg]. A hand mechanism of one degree of freedom, which can either pinch or grasp, hasbeen newly d回igned. It is designed to be able to pinch small objects (from diameter of 2 mm) and rather
big o吋ect (up to diameter of 114 mm). It uses a parallel link mechanism and ball screw as shown in Fig.
4. The grasping of cylindrical objects with a minimum diameter of 15 mm can be done with contact atthree points as is shown in the figure. This makes the grasping stable. Strain gauges are placed on two
finger links, respectively, which measure the grasping force. The average grasping force is 5 Kgf.
Measurement of the opening is done by an encoder attached to the DC motor. Position control with anaverage resolution of 0. 01 mm is attained. A six-axis force sensor is installed at th巴 wrist position. The
hand is made of durable aluminum and weighs 620 g including the force sensor.

。

幽百1e vision system of the slave robot consists of two color CCD video heads from TV cameras. Each CCD
has 420,000 pixcells and has its optical system with a focal length of f=12 mm (field of view 40 [deg])
and an aperture of F 1.6. Focus is automatically controlled by the TTLAF method. The separation of two
cameras are set at the distance of 65 mm, and the two cameras are aligned parallel to each other.
As for the auditory system, two microphones are placed 243 mm apa口from each other, and the same
-1243-

locational relation is used for the auditory display of the master system. A small speaker is placed at thelocation of出e mouth, which transmit出e operator’ s voice.
A human operator wears the 3D display with a sensation of presence.百1e audio visual display is carried
by a link mechanism with six degrees of freedom.百1e link mechanism cancels all gravitational force
伽ough a counter balancing mechanism with a relatively wide range of operation. It also enables thedisplay to follow the operator’ s head movement precisely enough to ensure his/her ordinary head
movem叩t. Maximum measured inertial force is within 5 kgf.百花master arm consists of ten degrees of
freedom. Seven degrees of freedom are allocated for the arm itself, and an additional three are used tocomply with the body movement.
Ster巴o visual display is designed according to the developed procedure which assures that the three
dimensional view will maintain the same spatial relation as by direct observation [1,2]. Adding to the fun­damental design procedure, an optical system using polarizers and analyzers was introduced to eliminate
the unnecessary reflected image 伺used by the internal mirror[6]. In the new model a helmet is used tofit on the head.百1ree sizes are available and can be changed easily by a buckle mechanism. Six inch
LCDs (H720 x V240 pixcells) are used. Two mirrors are arranged so that the LCDs can be placed on theupper side in front of the operator. These made possible the compact aπangement of the display system
suitable for the manipulation master system.
3. Experiments
百uee experiments which demonstrate the typical characteristics of the tele-existence master slave systemwere conducted.
(1）百1e most important featur回 of the tele-existence include the natural three dimepsional vision (close
to direct observation), which follows an operator's head movement in real time. Another feature is thenatural correspondence of visual information and kinesthetic information, iふ，an operator observes theslave's anthropomorphic arm at the position where his/her arm is supposed to be.τ'his allows the operator
at the control to perform tasks which need coordination of hand and eye quickly as in the case of direct
operation. Figure 6 shows a general view of an experimental manipulation task of building blocks
randomly placed on a table against a natural background under natural lighting condition. Block building
is usually done within few seconds without training, whereas conventional teleoperation using the same
master, the slave and a conventional two dimensional TV as a monitor takes training. A仕ained operator
takes several minutes to attain the same task.
(2)The combination of fur
virtual environm巴 nt display and impedance control makes it possible to use robots in hazardous
environments. Figure 7 shows that the robot works on the supposition that a pipe of a chemical plant isleaking and the plant is filled with toxic gas.百1巴 operator analyzes the situation using a virtual model
environment of the plant generated by the computer according to the blueprint of the plant while the robot
goes to the plant. The model environment is displayed by using th巴 same display which is used for thetele-existence operation. When the robot amves at the plant, the operator observes the situation throughthe robot's sensors as if he/she were at出巴spot.百1e operator conducts the emergency action by closing
the valve and pushing the switch of the exhaust fan.百1e model environment can be superimposed on thereal scenery. Impedance control of the slave robot ’ s manipulator helps conduct quick manipulation tasks
like closing valves and pushing switches.
(3)By using tele-existence, natural human robot communication becomes possible. In other words, robots
can be used in such situations that human robot collaboration is necessary. Figure 8 shows an example
of human robot commumcation.百1e robot presents a bunch of flower to a lady on behalf of a person at
the control.
By conducting th回 e expenments, efficacy of the tele-existence mast巴r slave system was verified and the-1244-

superiority of the t巴le-existence method was experimentally demonstrated.
Cone ’ usions
百1e concept of tele '- existence and/or artificial reality is considered from the standpoint of measurement
and control, and an experimental tele-existence system is introduced, which enables a human operator tohave the sensation of being in a remote real environment where a suπogate robot exists and/or virtualenvironment synthesized by a computer. A tele-existence master slave system for remote manipulation
experiments is designed and developed, and an evaluation experiment of a tele-existence master slave
system is conducted. By making a comparison of a tele-existence master slave system with aconventional master slave system, efficacy of the t巴le-existence master slave system is verified and thesuperiority of the tele-existence method is demons回ted through several experimental tasks. By conducting
these experiments, efficacy of the tele-existence master slave system has been verified and the superiorityof the tele-existence method has been experimentally demonstrated.
References[1 )S.Tachi et al.，＇’Tele-existen田 (I）一Design and evaluation of a visual display with sensation ofpresence － ，” Proceedings of the 5th Symposium on Theory and Practice of Robots and Manipulators(RoMa凶� 84), pp.245-254, CISM-IFfoMM, Udine, Italy, June 1984.
[2)S.Tachi, H.A凶，I.Morimoto and G.Seet，”Feasibility experiments on a mobile tele-existence system，”
The International Symposium and Exposition on Robots (19th ISIR), Sydney, Australia, November 1988.
[3)H.Arai, S. Tachi and I.Miyazima, Development of a power-assisted head-coupled display system using
a direct-drive motor，” Advanced Robotics, vol.3, no.2, pp.123-130, 1989.
[4)S. Tachi, H.A凶and T.Maeda， ”Development of an Anthropomorphic tele-existence山ve robot,”Proce巴dings of th巴 International Conference on Advanced Mechatronics (ICAM), pp.385-390, Tokyo,
Japan, May 1989.
[5)S. Tachi, H.Arai and T.Maeda， ”Tele-existence simulator with artificial reality (1), -Desi伊and
evaluation of a binocular visual display using solid models － ，”Proceedings of the IEEE International
Workshop on Intelligent Robots and Systems (IROS'88), pp.719-724, 1988.
[6)S.'rachi, H.Arai and T.Maeda，”Tele-Existence Master Slave System for Remote Manipulation，” IEEE
International Workshop on Intelligent Robots and Systems (IROS'90), pp.343-348, 1990.
[7)J.D.Hightower, E且Spain et al.， ”Telepresence: A hybrid approach to high-performance robots，”Proceedings of the '87 International Conference on Advanced Robotics (ICAR), pp.563-573, Versaiト
les, France, October 1987.
[8)L.Stark et al.，”Telerobotics: Display, Control, and Communication Problems，”IEEE Journal of Robotics
and Automation, vol.RA-3,no.1,pp.67-75 ,1987.
[9)S.S.Fisher et al.， ”Virtual environment display system，” ACM 1986 Workshop on Interactive 3D
Graphics, pp.1-11, Chapel Hill, North Carolina, October 1986.

ー 1245-ROBOT
刊巳陣

H
E
A

｜

出

I n_:·-rF-

���：…可叩＼ 1
KAR－· －－－��＼ �

un,nc;cMrn目白r.

MOTORノENCODER
附加／問ODER，飢Tna ＇＂�n=a

Ti
82
T‘

MOTOR/ENCO旺R,,

I

D除＝＝ヰlCOMPUTERI NECKENCOOERPULSE

C加TROLLER

6 叫 ：15 FORCE SEN釦R

AMP

lN'\Tnc,rM「nrucロ

CONTROLLERCOUNTER」「UCOMPUTER

A/0

33剛z

COVNTERENCO慌R

R｜口お5

R

ENCODERCOMPUTER
PULSE

怯＝ゴI

I

ARM

I

HANO

�
MOTORノENCOCER
MJTOνErJCO医R
脚旧TOR,ENC口DERT

COMPUTER
co付TROLLER

同λSE
C,fNERA TワR

286ノマ

10MHz

FOOT円TENTIOMETER

Aノロ
DノD

KNEE SWITCH

Fig.1 Block Diagram of the Tele-Existence Master Slave Manipulation System.

Fig.3 Schematic diag1澗n of the locomotion
system.
Fig.2 General view of the master system (left) and anthropomorphic slave robot (right).
ー1246-

Fig. 7 Experiment in hazardous environment.

Fig. 8 Human robot communication.

-1248-