Tele-Existence

Review:

Tele-Existence
Susumu Tachi
Research Center for Advanced Science and Technology , University of Tokyo
4-6-1, Komaba, Meguro-ku, Tokyo, 153 Japan[Received December 23 ,1991; accepted January 12,19921

Keywords: Tele-existence, Telepresence, Artificial reality,
Virtual reality

technology, people could go wherever they wish or experience whatever they want without having to move from theirpresent location. Before artificial reality andlor tele-existence can be realized, it is essential to integrate and take fulladvantage of achievements in the following fields of appliedsciences: robotics, computer engineering, three-dimensional
3D image processing, neuro-computer engineering, biomedical engineering, psychophysics, and cognitive science. Allof these are based on measurement and control technology,
electronic!;, communication, information processing, andmechatrornics.
This paper will highlight several approaches to the concept of tele-existence, identify its expected applications, andpresent the current trend of research activities with the discussion centered on the authors research.

1. Introduction
Recently, the concept of artificial reality (virtual reality)or tele-existence has evolved, and attempts to achieve practical applications of the related technologies are actively
being made.')-") For example, it has become possible toenter data representing human actions into a computer or to
make the user feel as if he is entering the world of computer
graphics displayed before him. Furthermore, there is a technique, which is nearing completion, that allows the operator
to control robots, which are remote from him, on a real timebasis by the sensation of presence. Through the use of
artificial reality it may be possible to touch a product thathas not yet been completed, to walk in to a house which isstill in the design stage, or to manually construct matter froma vast quantity of molecules. To some people, such ideas
are beyond all imagination. Tele-existence could make itpossible to allocate tasks, which are traditionally done byskilled human workers in dangerous or harsh working environments, to robots. In addition, through tele-existence

Artificial reality provides a basis for the technology
which ena.bles humans to experience events and acts in a
virtual environment just as if they were in the real world.
Tele-existence is similar to artificial reality, but takes a dif-Man-machine interface

Character display/Keyboard

I

3D videoholographic art

I
Human interface\
Graphic display:mouse
Multimedia display
Direct manipulation

\

/

I

Interactive
3 0 video art

ACybernetic'

Virtu@
consoleComputergraphics

2.Evolution of Artificial Reality and TeleexistenceinterfaceI

I-CG 3D

Real time Real time
3D CG -' interactive
3 0 CG

Simulator

Computersimulation(CS)

I

interactive
3 0 CADrobot control

41/
3D'cAD
Computer-aided
design CAD^ -I

Tele-robotics/supervisory control

Telephone/
Teleoperation

Fig. 1. Progress in artificial reality and tt:le-existence.

Journal of Robotics and Mechatronics Vo1.4 No. l , 1992

Tachi, S.

for communication with a more realistic sensation of presferent point of view. It represents a new concept that allows
ence.
humans, who are assumed to be emancipated from the reFurthermore, the simulation field is experiencing thestrictions of time and space, to exist in a "location" defined
rapid development of a real-time interactive 3D computerby inconsistent time and space, or a virtual environment.
ized simulation system which is intended for real-time opOne of reasons for artificial reality or tele-existence aterations in a near-real-experience basis.
tracting worldwide attention is that scientific subjects,
The art and amusement industries are no exception. Artwhich have been believed to belong to completely different
ists and amusement designers are viewing artificial reality
fields of research, are likely to be united by the concept ofas a new art medium that could exceed existing ones withdifferent scientific subjects, as shown in Fig.1. In the caserespect to the power of artistic expression.
of remote-controlled robots,the development of nuclear
In the conventional man-machine interface design, mantechnologies and orthotics techniques for manufacturing
has to adapt himself to the machine because man is more
medical braces for handicapped people (like artificial leg)flexible. However, a more human friendly man-machine
were combined after the world war I1 to give rise to teleinterface has been recently advocated. The next step is a
operation technology. In turn, this technology evolved into
cybernetic interface in which the machine comes unilatersupervisory control in the 1970s, by taking in robotics, further developed into telerobotics in the 1980s. As a result,ally closer to man's natural sensation. This will finally attain the artificial reality. Rapid progress in computer andan idea of tele-existence-based remote control, which relies
sensor technologies and increasing findings about humanon tele-existence (telepresence) for a higher degree of sensensation, brought about by advances in human science,sation of presence on a real time basis, rapidly evolved laterin the 1980s. Thus we can clearly see these fields of robothave made artificial reality possible. Recently, many fields
of scientific research, which have advanced independently
ics approaching the world of artificial reality.
of each other, have begun to focus on the concepts of artiIn the field of computer graphics (CG), the conventional
2.5-dimensional display system, in which a solid model isficial reality and tele-existence and to view these concepts
shaded by perspective-transformation and then displayed,
as key technologies of the 21st century. This encourages
has advanced to 3D display which provides the user withfirms and organizations related to such fields to motivate
stereopsis or stereoscopic vision. This is now developing
intensive research/development programs about artificial reality and tele-existence.
into an interactive 3D display system in which the image
Furthermore, the concepts of artificial reality and tele-excan be changed according to the user's viewpoint. Thisistence are not simply common to the foregoing fields, theallows him to look sideways or obliquely downward/upward
concepts themselves have a common elemental technology,
into the image on the display screen as in a hologram. Theinteractive 3D CG currently under development is in theas will be described in Chapter 4. Therefore, a basic technology developed in one field can be readily available toshortest distance from the world of artificial reality.
In the field of CAD, attempts are being made to realizeanother field. This makes it more important to study all of
the related fields as a single generic technology. In fact,
a design support/evaluation system based on virtual prodJapanese and foreign researchers are actively studying artiucts by combining CG, tactile sensation feedback, and force
ficial reality because they see a great importance in it.
sensation feedback. Virtual products could allow designers
to evaluate utilization prior to manufacturing and to easily
make design changes with ease if necessary. Design change
data stored in computer memory is readily available to pro3. Tele-existence in the Real and Virtual
duce a "real"product if the memory is linked to CIM. The
Worldsconcept of artificial reality has an increasingly greater importance to industrial production because it could help produce easy-to-use products, or much more advanced
products, that are expected to be in great demand in the
Artificial reality or tele-existence may be divided into
future, and are more suited to individual user preference.
two categories: tele-existence in the real world that actuallyexits at a distance, and it is connected via robot to the place
In the computer field, a more user-friendly interface iswhere the user is located; and tele-existence in the virtualdesired. In addition to the currently prevalent character disworld that does not actually exist but is created by computer.
play by keyboard, there are many other possible interfaces
The former is called also remote sensation of presence or
including graphical display on microcomputer, mouse input,
object-oriented programming, multele-reality, and the latter is virtual reality in a narrow sense.
timedia display and 3D mouse, andinput/output operations by direct
manipulation. Now, it has been
Tele-Egstence -l---Tele-Existencein Real World -r-- Standard Tele-Existence
proposed to enter information by
L Augumented Tele-Existence
using virtual console and virtual(Size)
display. Thus, artificial reality is(Sensation)
likely to be incorporated in ahuman-computer interface.
(Time)In the communication field, theTele-Existence in Virtual Worldold telephone system has evolved- Quasi physical world
into video phone. Tele-communicaNon physical worldtions are now under intensive research, and a great demand isFig. 2. Tele-existence in real and virtual worlds
expected in the coming b-ISDN era

8

Journal of Robotics and Mechatronics Vo1.4 No. 1, 1992

Tele-Existence
However, there is usually no strict distinction between the
two. Figure 2 illustrates this classification.

3.1. Tele-existence in the Real World
The tele-existence in real world, which relies on a robot,
is further divided into standard and augmented tele-existence. The former is intended to facilitate user's presence
in a place, which is distant from actual location, in real timeor within a negligible time delay by means of a robot. The
robot is assumed to have the same external shape and sensory functions as a human. In other words, the purpose is
to make it possible for the use to perform tasks in a remoteenvironment through the help of a robot. The tasks are donewith the sensation of presence and in real time. In contrast,
augmented tele-existence is intended for cases in which theshape and/or sensory functions of the robot are different
from a human's or the time interval is too large between
events in the user's location and the remote location. Augmented tele-existence may apply to cases in which the user
controls an micro-robot or controls a gigantic robot.
Although the standard tele-existence handles only signals
in the frequency range among sensory channels throughwhich the user normally detects stimuli, it is possible to lift
such restrictions to augment human sensory functions. Thisaugmentation need not be limited to sensory functions
within a human's inherent senses; it can be applied to sensory functions beyond that limit. For this purpose, standard
tele-existence employs such sensor information obtained
through X-rays, ultra violet rays, infrared rays, micro waves,supersonic waves, and ultra-low frequencies sensed by the
robot (also referred to as super-sensory information). For
example, if information obtained by a robot, which scans adark environment for an object using infrared rays, can bedisplayed by means of computer graphics and 3D presentation technologies, then the user would be able to see theobject on the display screen as if it were bright illumination.
The super-sensory information may be superimposed over
the ordinary visual display image, not in ordinary superimposition but in 3D superimposition. Possible applications
include the following cases: when the distance between theuser and object is displayed in the form of an aerial image
which is superimposed over the location of the object exists;
and when only the portion of an image, which has undergone change for some reason is displayed with the added
sense of presence, and the rest of the image originally visible
to the user is subtracted.
Augmentation in terms of time is also possible. For applications of tele-existence to planetary exploration andother space activities, the length of communication time isan important factor for consideration. The maximum permissible time delay, which does not interfere with control
in ordinary teleoperation, is believed to be about 0.1 second.
Anderson et a1 have proposed an alternative that can keep
the system stable with a time delay of up to about two
seconds by converting the transmission block to a loss-freetransmission line.
This makes the transmission block nearly equal to a passive element which appears to be independent of time delay.
However, in a system with a time delay greater than 2 seconds, this method cannot provide proper control.
Even in such cases, augmented tele-existence may enableproper control, at least theoretically, if the following method
is used with the help of the remote robot which is autonomous. The robot first scans the environment and prepares
Journal of Robotics and Mechatronics Vo1.4 No. l, 1992

an environment model, which is sent to the remote computer
system. The system displays visual, auditory, or tactile information with a sensation of presence according to the stateof both the object and the user. The user performs a tasks
in this virtual environment with a sensation of presence, andthe essence of user's work is transmitted to the remote robot.
The robot, in tele-existence mode, decodes the information
or instructions for performing tasks and makes additions toor corrections in the environment model, if necessary. Any
disorder produced by tasks performed strictly in accordancewith the directions implies that there are defects in themodel. Therefore, the robot stops operation in a safe state
and then estimates an alternative model. The robot thenreports its state and the new model to the user. The user
performs the task over again in the virtual environment
which provides sensation of presence based on the newmodel. Thus, augmentation in terms of time is theoreticallypossible. However, there still remain several unsolved
problems such the vast amount of calculation required andthe estimation of the model.

3.2. Tele-existencein Virtual WorldA virtual world created by computer can be divided into
the categories of physical world, quasi-physical world, andnon-physical world. The physical world is where the same
physical laws as those of our planet apply. Design support
and the evaluation of virtual products must be done in thephysical world. The virtual environment used for training
simulators also must be set in this physical world.
In the quasi-physical world the same physical laws thatgoverns the real world also apply; however, it contains virtual worlds such as the moon, the microscopic world where
the laws of quantum mechanics rule, or a world where theprinciple of relativity has predominant effects over events.
The concept of the quasi-physical world is useful for training/educational programs to provide trainees with experiences in a world which is utterly unknown to them.
Considering applications to leisure activities and arts, thepotential applications for such fields are not necessarily limited to the physical and quasi-physical worlds. Rather, aworld of imagination is often required by amusement-related industries.
The non-physical world can meet such demands. This
world is technically easier to realize than the former two,
although a higher artistic sense is necessary. Artistic expression by virtual reality created by the non-physical worldprovides a potentially new medium that, still embracing linguistic and picturized expressions but transcending them,
may represent ideas, thoughts of sensibility. In this sense,
the non-physical world attracts attention from artists.
3.3. Applications for Tele-existence
Research into artificial reality and tele-existence is anattempt to release the user from spatial restrictions. This isachieved not by providing information with the user in a
passive state, in TV watching, but by providing an artificial,
but very realistic environment where the user can feel andact as if he were there.
Based on this perspective, the goals of tele-existence may
be arranged as follows:(1) To provide substitutes for manual labor in potentially
dangerous working environments such as nuclear facilities, ocean engineering, disaster-prevention, andspace activities; and to apply to construction work andTachi, S .
mining.
(2) To apply to secondary industries, manufacturing industry, such as tele-machining as a new production
support;(3) To apply to primary industries such as agriculture(tele-farmer) and fishing;(4) To apply to tertiary industries including cleaning,
maintenance, and other services;(5) To apply to leisure, amusement and game industries
as a tele-existence trip;(6) To apply to medical fields as in micro-surgery;(7) To apply to communications such in communication
with a sensation of presence;(8) To apply to education, for example, an ultimate simulation including an electronic simulator;(9) To apply virtual products to design support (CAD,
IMS);(10) To apply virtual environment to the design field,
including interior design;(11) To apply scientific visualization as a tool for scientific-engineering research;(12) To apply display with a sensation of presence as atool for research of the functions of humans and otherliving creatures;(13) To provide a new medium that, embracing linguistic
and picturized expressions and going beyond them,
may express ideas and sensibility.

4. Organization of Tele-existence and its Related Problems
Figure 3 shows the organization of tele-existence. The

most notable distinction of artificial reality or tele-existenceis that the virtual environment where the user is supposed
to exist:(1) is a 3D space which is natural to the user; (2) allows
the user to act freely; and (3) allows the interaction to takeplace in a natural form and in real time.
The basic technologies necessary to put tele-existence
into practice include:(i) the estimation of the user's state (including the external state represented by user movements and tone of voiceand the internal state represented by brain wave and electrocardiogram) and the estimation of the human decision
making process; (ii) the interaction between the robot .andthe natural environment and/or the interaction between thevirtual human and the virtual environment; and (iii) the
presentation to the user of the process described in (ii) andthe results with the sensation of presence and in real time.
These are what any possible applications of tele-existence
should have in common. An in-depth investigation of suchbasic technologies is essential to the future development oftele-existence. What is characteristic of the study of teleexistence is that the achievements of one basic technology
are readily available to the others.
In order to prevent artificial reality from becoming amere application of simulation technology, it is important toconnect the virtual and real environments in good harmony.
The necessary technology is one of problems awaiting asolution.
Furthermore, the following concept provides a new andinteresting research subject: a system of a virtual or realenvironment that could accept a virtual human or robot of
another tele-existence system and allow them to exist withthe original virtual human or robot. This will give rise to
new interaction between virtual humans or robots of the two

Fig. 3. Organization of tele-existence system.

Journal of Robotics and Mechatronics Vo1.4 No..l, 1992

Tele-Existenceor more systems, in addition to the existing interaction between the environment and robot.

5. Present State of Research and Development of Tele-existence
Because a detailed description of tele-existence in virtualworld is provided in the other commentary, the author will
present tele-existence in the real world with the discussions
focused on the author's work.
A series of intensive research on viable design procedures of visual display system3) has been conducted withgood results. For a visual display system to be regarded as
an ideal system with a sensation of presence, it must allow
the user the same "clues," or means to act on it, that he hasin actual 3D space through direct-viewing. If a 3D display
system which has the same clues as a direct-viewing systemsuch as visual convergence, parallax, magnitude of image,and eye lens accomodation, is given an additional function
which enables images on the display screen to change according to the movement of the viewer, then it is defined as
an active 3D display system. A tele-existence system requires this type of 3D display.
For the ideal visual display system, two cameras are installed apart from each other the same distance as between
a human's eyes. The mechanism equipped with the pair ofcameras is controlled by the neck motions of the user. First,
head movement is measured. According to the results, theangles of convergence for the cameras and display image(hand er) are controlled so that 8n = er. At the same time,the distance to the object, Xr, is determined, which enables
the cameras to focus their lenses on the object. In the display unit, the lens system mounted in front of the CRT iscontrolled so that Xn = Xr, for the position of the virtual
image, and In = Ir, for the magnitude of the image.
There are three parameters which govern human visual
perception of space. In monochrome, surrounding him: (a)the contraction/relaxation of ciliary muscle for adjusting the
thickness of the crystalline lens; (b) the magnitude of theimage on the retina; and (c) the angles of convergence ofthe eyes. In an ideal system, the values of these parameters
should be equal to their respective values for ordinary direct
viewing. Accordingly, the ideal system can provide the user
with the same visual perception as by direct viewing.
Next, the author discusses how to simplify such an ideal
system. Taking the characteristics of the human optic function into consideration, process (a) can be executed in a
simpler manner. Even if the crystalline lens is set to operate
only for a fixed distance to an object of, 20cm as an example, its images on the two retinas are naturally fused to forma single image of the object for the convergence range from10 to 50cm. In other words, with the focal length fixed tolm, the optical mechanism can accommodate the convergence range from 20cm to infinity. Therefore, even if the
distance Xn is taken as lm, the sensation of presence isrnaintair~ed.~)If the distance displayed, Xn, is kept constant,then the number of variables that need to be controlled isreduced to two, the magnitude of image and the angles of
convergence of the eyes. This would make it possible toconstruct a simpler tele-existence system.3) Such a system
could re-create the same space around the user as that where
robot is operating by using a wavefront re-creation system.
Journal of Robotics and Mechatronics Vo1.4 No. l, 1992

Accordingly, there is no need to control the robot's eyes bymeasuring the movement of the eyes of the user.
Remote-controlled robot systems have been manufactured as an experiment and tested to verify the basic conceptof tele-existence.
A tele-existence system intended for a manipulative task
is shown in Fig.4. Tele-existence master equipment, asshown in Fig.5, was combined with a human-shaped slave
robot, which was constructed so that its shape and degrees
of freedom were similar to those of a human, and was put
to a task test. The results were compared to the abilities ofa conventional two-dimensional (2D) display system, providing the superiority of the tele-existence system over the
2D system.6) Figure 6 shows another tele-existence system
which experimentally presents a bouquet to a girl. The
success of the bouquet presentation ceremony has provedthat any user can operate a tele-existence system without
special training to coordinate the eyes and hands of therobot. It also suggests that such a system may coexist withhuman beings by serving useful functions.
A mobile tele-existence system intended for carrying
such a manipulator is currently under study. A running test
conducted on a mobile system on the premises of a laboratory has shown that the user can control a mobile tele-existence system as if driving his car, even in an environment
with obstacles.
The study of tele-presence is being conducted on theNaval Ocean Systems Center (NOSC) tele-operated vehicle
and on the NOSC tele-operated robot at NOSC in the Unitedstates.') Furthermore, NASA has launched a researchldevelopment program of a tele-presence geologist aimed at
identifying its application to activities on Mars.

6. Conclusion
There is an increasing interest in tele-existence involving artificial reality by researchers of both Japan and Overseas countries. As our original technology which evolved
from the Mechanical Engineering Laboratory of the Agency
of Industrial Science and Technology, tele-existence has
played a rnajor role in the large national project entitled
Advanced Robot Technology (Research and Developmentof Task Robot for Potentially Dangerous Working Environments). The technical feasibility of the tele-existence system has been proved both theoretically and experimentally
through a series of tests conducted on prototype hardware
models.
These tests have also helped to establish design procedures for basic systems. In addition, the results of theauthor's work have been incorporated in an experimental
system intended for marine tasks, which is almost on theverge of practical application.
M.L. Minsky, of the MIT Media Lab, calls this idea
tele-presence, which is recently a focus of study by NASA.
In the U.S., the concept of artificial reality and virtual reality
has also come into being, and efforts are being actively
made towiud its research and development as well as its
application. In this context, Japanese researchers have spentapproximately ten years in nourishing its original idea andbasic technology. It is now widely regarded as one of the
key technologies which was born in Japan and is expected
to play a significant role in the 21st century.

Tachi, S .

Fig. 4. Tele-existence slave robot

Fig. 5. Tele-existence master equipment.

Fig. 6. Interractiove works between man and machine with
Tele-Existance environment.

The author believes that Japan should continue to claim
the originality of its idea and technology to avoid anotherincidence of Japan-U.S. friction that might arise from amisunderstanding. Looking back to the first principles oftele-existence reminds us that it is aimed at a human. This
enables the robot to perform task that only humans can do,
unites the ability of the robot with the user on a cybernetic
basis, and finally substitutes the robot for the user in performing tasks that even man cannot do. Therefore, the teleexistence system is expected to be one of leading
alternatives to human laborers in doing dirty or dangerous
tasks which they dislike.
12

Tele-existence seems to represent a technology which has
been long yearned by people who believed they could display their potential power to the maximum or become asuper man. Tele-existence technology will make it possible
for man to go through experiences which had been deemed
impossible. In addition, it can evolve into something likean artificial experience culture. While containing conventional expressions by character, picturized expression andmusical expression, this culture transcends them and develops into a new medium for expressing thoughts and sensibility. In the United States, there is a plan to establish a
consortium administered chiefly by the University of Washington, the publication of Presence. (a new scientific journal
from MIT Press), and a virtual network plan connecting
MIT, VPL and the University of North Carolina. This consortium is giving impetus to research and development activities.
In Japan, the Image Laboratory has been established, andthe Study Committee of Artificial Reality and Tele-existence
has been inaugurated, attracting increasingly strong attention from the government and related industries. With thisbackground, the author expects that tele-existence and artificial reality will find applications in a variety of fields.
References:
1) S. Tachi and Abe: Study Report on Tele-existence, No.1 - Designof Visual display -,Proceedings of the 21st Conference of the Societyof Instrumentation and Control Engineers (SICE), 1671168 (1982).
2) S. Tachi et al.: Tele-existence (1)Design and Evaluation of a Visual
Display with Sensation of Presence, Proc. of the 5th Symposium on
Theory and Practice of Robots and Manipulators (RomanSy 84),
2451254, CISM-IToMM, Udine, Italy (1984).
3) S. Tachi, H. Arai: Design and Evaluation of a Visual Information
Display by Tele-existence, the Journal of Japan Robotics Society, 7-4,
3141326 (1989).
4) D.L. Akin, M.L. Minksy et al.: Space Application of Automation:
Robotics and Machine Intelligence Systems (ARAM1S)Phase 11,
NASA Contract Report 3734 (1983).
5) J.D. Hightower, E.H. Spain et al.: Telepresence: A Hybrid Approach
to High-Performance Robots, Proceedings of the '87 International
Conference on Advanced Robotics (ICAR), 5631573, Versailles,
France (1987).
6) S. Tachi, H. Arai and T. Maeda: Development of Anthropomorphic
Tele-existence Slave Robot, Proc. of International Conference on
Advanced Mechatronics, 3851390, Tokyo, Japan (1989).
7) S. Tachi and T. Sakaki; Impedance Controlled Master Slave System
for Tele-existence Manipulation, Proceedings of the First International Symposium on Measurement and Control in Robotics (ISMCR'90), Houston, Texas (1990).
8) S. Tachi, H. Arai and T. Maeda: Tele-existence Simulator with Artificial Reality, Proc. of IEEE International Work Shop on Intelligent
Robots and Systems, 719/724, Tokyo, Japan (1988).
9) S.S. Fisher et al.: Virtual Environment Display systems, ACM 1986
Workshop on Interactive 3D Graphics, Chapel Hill, North Carolina,
1/11 (1986).
10) M. Hirose et al.: Man-Machine Interface using Artificial Reality for
Tasks in Three-Dimensional Space, Proc. of the Symposium on
Human Interface, Tokyo, 2011206 (1988).
11) H. Iwata: Artificial Reality for Walking in a Large-Scale VirtualSpace, Human Interface, 5,49152 (1990).
12) H. Takemura and F. Kishino: Human Interface using Artificial Reality, Journal of Television, 44-8, 9811985, (1990).
13) M. Sato et al.: Virtual working Space for Modelling Three-dimensional Shapes, 3D Image, 4-2, 27/35, (1990).

Journal of Robotics and Mechatronics Vo1.4 No. l, 1992