International Conference on Artificial Reality and TelexistenceEurographics Symposium on Virtual Environments (2019), pp 1-8

Y. Kakehi and A. Hiyama (Editors)

Forty Years of Telexistence—From Concept to TELESAR VI

Susumu Tachi†1
1

The University of Tokyo, Japan

Abstract
Telexistence is a human-empowerment concept that enables a human in one location to virtually exist in anotherlocation and to act freely there. The term also refers to the system of science and technology that enables realizationof the concept. The concept was originally proposed by the author in 1980, and its feasibility has been demonstrated
through the construction of alter-ego robot systems such as TELESAR, TELESAR V, and TELESAR VI, which weredeveloped under the national research and development projects “MITI Advanced Robot Technology in Hazardous
Environments,” the “CREST Haptic Telexistence Project,” and the “ACCEL Embodied Media Project,” respectively.
Mutual telexistence systems, such as TELESAR II & IV, capable of generating the sensation of being in a remoteplace using a combination of alter-ego robotics and retro-reflective projection technology (RPT), have been developed, and the feasibility of mutual telexistence has been demonstrated. Forty years of telexistence development are
historically reviewed in this keynote paper.
CCS Concepts • Human-centered computing→ Mixed/augmented reality • Human-centered computing→ Virtual
reality • Human-centered computing→ Haptic devices • Human-centered computing→ Telexistence

1. INTRODUCTION

and telepresence are very similar concepts proposed independently in Japan and the USA, respectively.

Telexistence technology enables a highly realistic sensationof existence in a remote location without any actual travel.
The fundamental concept refers to the general technologythat allows a human being to experience the real-time sensation of being in another place interacting with the remote environment, which may be real, virtual, or a combination ofboth [Tachi 2015b]. It also refers to an advanced type of teleoperation system that allows an operator at the controls toperform remote tasks dexterously with the feeling of being
in a surrogate robot working in a remote environment. Telexistence in a real environment through a virtual environment is also possible.

However, the concept of telexistence in virtual environments or in a real environment through a virtual environmentis the unique feature of telexistence. Telepresence, in contrast, indicates remote presence only in the real world. In
addition, telexistence uses even an autonomous robot as its
avatar [Tachi et al. 1984], while telepresence uses only a teleoperated robot.

Sutherland [Sutherland 1968] proposed the first headmounted display system, which led to the birth of virtual reality in the late 1980s. This was the same concept as telexistence in computer-generated, virtual environments. However, it did not include the concept of telexistence in real,
remote environments. The concept of providing an operator
with a natural sensation of existence in order to facilitate
dexterous remote robotic manipulation tasks was termed"telexistence" by Tachi [Tachi et al. 1980; Tachi et al. 1981]
and "telepresence" by Minsky [Minsky 1980]. Telexistence

Figure 1 illustrates the emergence and evolution of the concept of telexistence. Teleoperation emerged in Argonne National Laboratory soon after World War II to manipulate radioactive materials [Goertz 1952]. In order to work directlyin the environment rather than work remotely, an exoskeleton human amplifier was invented in the 1960s. In the late1960s, a research and development program was planned todevelop a powered exoskeleton that an operator could wear
like a garment. The concept for the Hardiman exoskeleton
was proposed by General Electric Co.; an operator wearing† Professor Emerirus of The University of Tokyo

invited to ICAT-EGVE (2019)

In this keynote paper, the development of telexistence
technology and telexistence systems is historically reviewed
from 1980 through 2019.
2. HISTORY OF TELEXISTENCE

S. Tachi / Forty Years of Telexistence—From Concept to TELESAR VI

the Hardiman exoskeleton would be able to command a setof mechanical muscles that could multiply his strength by afactor of 25; yet, in this union of a man and a machine, theman would feel the object and forces almost as though he isin direct contact with the object [Mosher 1967].
However, the program was unsuccessful for the following
reasons: (1) wearing the powered exoskeleton was potentially quite dangerous for a human operator in the event of amachine malfunction; and (2) it was difficult to achieve autonomous mode, and every task had to be performed by thehuman operator. Thus, the design proved impractical in its
original form. The concept of supervisory control was proposed in the 1970s to add autonomy to the human operations[Sheridan and Ferrell 1974]. In the 1980s, the exoskeleton
human amplifier evolved into telexistence, i.e., into the virtual exoskeleton human amplifier [Tachi and Komoriya
1982].

Avatar robots can be autonomous and controlled by supervisory control mode while the supervisor can use one of therobots as his virtual exoskeleton human amplifier by using
telexistence mode.
2.1 How Telexistence was Conceptualized and Developed

The author came up with the idea of telexistence while hewas walking one morning in the corridor in the laboratory
on September 19, 1980. He was suddenly reminded of thefact that human vision is based on light waves and that humans only use two images that are projected on the retina;
they construct a three-dimensional world by actively perceiving how these images change with time. The robot
would only need to provide humans, by measurement andcontrol, with the same image on the retina as the one thatwould be perceived by human eyes. As he somewhat rediscovered this fact, all his problems dissipated and he started
to tremble with excitement. He went back to his office immediately and wrote down what he had just discovered andall the ideas that sprang up following that discovery (Fig.3).
Surprising enough, he simultaneously came up with the ideaof how to design telexistence system and how to apply thetechnology to the evaluation of mobility aids for the blind[Tachi et al. 1980] and also to the dexterous remote manipulation with sensory feedbacks [Tachi et al. 1981].

Figure 1: Evolution of the concept of telexistence through
Aufheben or sublation of contradictory concepts of exoskeleton human amplifier and supervisory control.
In using a telexistence system, since it is not necessary
for a human user to wear exoskeleton robot and be actuallyinside the robot, the human user can avoid the danger ofcrashing or exposure to hazardous environment in the caseof a machine malfunction, and also make the robot work in
its autonomous mode by controlling several robots in the supervisory control mode. Yet, when the robot is used in thetelexistence mode, the human user feels as if he is inside therobot, and the system works virtually as an exoskeleton human amplifier, as shown in Fig. 2.

Figure 2: Exoskeleton Human Amplifier (left), and telexistence virtual exoskeleton human amplifier (middle and right),
i.e., a human user is effectively inside the robot as if wearing
the robot body.

Figure 3: Sketch of the idea when the author invented theconcept of telexistence.
2.2 Third Generation Robotics
The concept of telexistence was the fundamental principle
behind the eight-year large-scale Japanese project entitled“Advanced Robot Technology in Hazardous Environments,”
which was established in 1983 together with the concept ofthird-generation robotics. A typical potential use of a thirdgeneration robot is as a substitute for humans in potentiallyhazardous working environments. These include, for example, work in nuclear power plants, underwater operations,and rescue operations in disaster areas. One type of robot
system which can be used in these areas is a human-robot
system consisting of several intelligent mobile robots, a supervisory control subsystem, a remote operator, and a communication subsystem linking the operator and the robots.
Figure 4 shows the author’s concept of a human-intelligent
robot system, which performs essential work in hazardous
working environments [Tachi and Komoriya 1982].
Planning, scheduling, and task sharing operations forseveral robots can be handled by a supervisory controller

invited to ICAT-EGVE (2019)

S. Tachi / Forty Years of Telexistence—From Concept to TELESAR VI[Sheridan and Ferrell, 1974]. Each robot sends its work progress report to the supervisory controller in a consecutive
manner. The reports are compiled and processed by the supervisory controller, and selected information is transmitted
to the human operator through visual, auditory, and tactile
channels. The operator issues macro commands to each of
the robots via a voice recognition device.
When an intelligent robot is confronted with a task which
is beyond its capabilities, the control mode is switched to ahighly advanced type of teleoperation, namely telexistence.
Telexistence attempts to enable a human operator at the controls to perform remote manipulation tasks dexterously withthe feeling that he or she is present inside a remote anthropomorphic robot in a remote environment.

Figure 6: Head-linked stereo display providing a sensationof driving a televehicle as if on borad the vehicle (left), andthe mobile telexistence vehicle (right).
2.4 Telexistence Surrogate Anthropomorphic Robot:
TELESARThe first prototype telexistence master-slave system for performing remote manipulation experiments was designed anddeveloped, and a preliminary evaluation experiment of telexistence was conducted [Tachi et al. 1989].
The slave robot employs an impedance control mechanism for contact tasks and for compensating for errors thatremain even after calibration. An experimental operation ofblock building was successfully conducted using a humanoid robot called TELExistence Surrogate Anthropomorphic
Robot (TELESAR).

Figure 4: Telexistence system architecture.
2.3 First and Early Telexistence Hardware Systems
Our first report proposed the principle of a telexistence sensory display and explicitly defined its design procedure. The
feasibility of a visual display providing a sensation of presence was demonstrated through psychophysical measurements performed by using an experimental visual telexistence apparatus. Figure 5 illustrates the first prototype telexistence visual display [Tachi and Abe 1982; Tachi and
Komoriya 1982; Tachi et al. 1984].

Experimental studies of the tracking tasks quantitatively
demonstrated that a human being can telexist in a remote environment using a dedicated telexistence master-slave system [Tachi and Yasuda 1994]. Because measurements are
performed using goniometers and potentiometers and all
programs are written in C, the system is operated at an extremely high-speed cycle time of 3 ms, even when the operation is conducted as impedance control mode. Virtually, nodelay is observed. Human operators perceive as though they
were inside the robots. Figure 7 illustrates a telexistence
master-slave manipulation system.

Figure 7: TELESAR building blocks (left) and telexistence
master system (right).
2.5 Augmented Telexistence

Figure 5: First prototype telexistence visual display, which
creates out-of-the-body sensation.
In 1985, a method was also proposed for the development of a mobile telexistence system, which can be driven
remotely with both an auditory and a visual sensation ofpresence. A prototype mobile televehicle system was constructed (Fig. 6), and the feasibility of the method was evaluated [Tachi et al.,1988].

Telexistence can be divided into two categories: telexistence
in a real environment that actually exists at a distance and isconnected via a robot to the place where the operator is located (transmitted reality), and telexistence in a virtual environment that does not actually exist, but is created by a computer (synthesized reality). Combining transmitted reality
and synthesized reality, which is referred to as mixed reality,
is also possible, and it has great significance for real applications. This is referred to as augmented telexistence, and it
can be used in several situations, for instance, controlling aslave robot in an environment with poor visibility.

S. Tachi / Forty Years of Telexistence—From Concept to TELESAR VI

An experimental augmented telexistence system wasconstructed using mixed reality. An environment model was
also constructed from the design data of the real environment. When augmented reality is used to control a slave robot, modeling errors of the environment model must be calibrated. A model-based calibration system using image
measurements was proposed for matching the real environment with a virtual environment. An experimental operationin an environment with poor visibility was successfully conducted using TELESAR and its virtual dual. Figure 8 illustrates the virtual telexistence anthropomorphic robot used in
the experiment [Yanagida and Tachi 1993].

Figure 8: Virtual TELESAR at work.
2.6 R-Cubed: Realtime Remote Robotics
In order to realize a society wherein everyone can freely telexist anywhere through a network, the Japanese Ministry ofInternational Trade and Industry (MlTI) and the Universityof Tokyo proposed a long-range national research and development program that was dubbed R-Cubed (R3) in 1995.
R3 means real-time remote robotics. The concept of this program is the research and development of technologies thatallow human operators to telexist freely by integrating robots, virtual reality, and network technology. Figure 9 illustrates an example of an R3 robot system.

Figure 9: Diagram for RCML and RCTP processes. Robots
and control devices are placed virtually everywhere in theworld and connected by Internet. Any user can use any robot
when the robot is in use.
Each robot site has a server as its local robot. The typeof robot varies from a humanoid (high end) to a movable
camera (low end). A virtual robot can also be a local controlled system. Each client has a teleoperation system. The
system can be a control cockpit with master manipulators
and a head-mounted display (HMD) or a CAVE Automatic
Virtual Environment (CAVE) at the high end. It is also possible to use an ordinary personal computer system as a control system at the low end. In order to assist low-end operators with controlling remote robots through networks, R-

Cubed Manipulation Language (RCML) and R-Cubed
Transfer Protocol (RCTP) were developed [Tachi 1998].
2.7 Humanoid Robotics Project (HRP)
On the basis of the R3 program and after conducting a twoyear feasibility study called Friendly Network Robotics(FNR) from April 1996 until March 1998, a National Applied Science & Technology Project called "Humanoid andHuman Friendly Robotics," or "Humanoid Robotics Project(HRP)" was launched in 1998. This was a five-year project
toward the realization of an R3 Society by providing humanoids, control cockpits, and remote-control protocols.
In phase 1 of the project, a telexistence cockpit for humanoid control was developed, and the telexistence system
was constructed using the developed humanoid platform. In
order to evaluate the usability of the cockpit for HRP robots,
an experiment was conducted where an operator was given
the task to navigate the environment of a biped humanoid
robot and manipulate objects by utilizing the robot as a surrogate. The operator navigated the robot as if he or she were
inside the robot, and manipulated the robot’s arms and hands
to handle a stuffed animal, stack blocks, open and close asliding glass door of a cupboard, pick up a can from inside
the cupboard, place the can into a basket, and so forth. While
the robot walked, real images captured by the multi-camera
system for the wide field of view were displayed on four
screens of the surround visual display. This provided the operator with a feeling as if he were inside the robot in the remote environment. Figure 10 presents the robot descending
stairs. Since the series of real images presented on the visualdisplay are integrated with the movement of the motion base,the operator experiences a real-time sensation of walking or
stepping up and down.
This was the first experiment in the world which succeeded in controlling a humanoid biped robot by using telexistence [Tachi et al. 2003].

Figure 10: HRP humanoid biped telexistence robot descending stairs (left), and a human operator at the telexistence cockpit feeling the sensation of stepping down (right).
2.8 Mutual Telexistence Communication System:TELESAR II & IV
A method for mutual telexistence based on the projectionof real-time images of the operator onto a surrogate robot
using retroreflective projection technology (RPT) was first
proposed in 1999 [Tachi, 1999], and the feasibility of thisconcept was demonstrated by the construction of experimental mutual telexistence systems using RPT in 2004
[Tachi et al., 2004]. In 2005, a mutual telexistence masterslave system called TELESAR II was constructed for the
Aichi World Exposition. Nonverbal communication such as
gestures and handshakes could be performed in addition to

invited to ICAT-EGVE (2019)

S. Tachi / Forty Years of Telexistence—From Concept to TELESAR VI

conventional verbal communication because a master-slave
manipulation robot was used as the surrogate for a human[Tachi et al., 2008]. Moreover, a person who remotely visited the surrogate robot’s location could be seen naturally
and simultaneously by several people standing around thesurrogate robot (Fig. 11).
Figure 13: TWISTER to TWISTER mutual telexistence.
2.9 Telexistence Avatar Robot System: TELESAR
III and TELESAR V

Figure 11: Mutual telexistence surrogate robot TELESAR
II (left), and an operator at the telexistence cockpit (right).
The mobile mutual telexistence system, TELESAR IV,
which is equipped with master-slave manipulation capability and an immersive omnidirectional autostereoscopic 3D
display with a 360-degree field of view know as TWISTER,
was developed in 2010 [Tachi et al., 2012]. It has a projection of the remote participant’s image on the robot by RPT(Fig. 12).

Figure 12: Mobile telexistence surrogate robot TELESAR
IV communicating with local participants (left), and a remote participant at the TWISTER cockpit (right).
Face-to-face communication was also confirmed, as local
participants at the event were able to see the remote participant’s face and expressions in real time. It was further confirmed that the system allowed the remote participant to not
only move freely about the venue by means of the surrogate
robot, but also perform some manipulation tasks such as ahandshake and several gestures.
Face-to-face telexistence communication has been developed using TWISTER booths. The following systems wereadded to the original TWISTER’s omnidirectional 3D autostereoscopic display: a 3D facial image acquisition systemthat captures expressions and line of sight, and a user motion
acquisition system that captures information about arm andhand position and orientation. An integrated system has been
constructed whereby communication takes place via an avatar in a virtual environment.
The results of having two participants engage in
TWISTER-to-TWISTER telecommunication verified thatparticipants can engage in telecommunication in the sharedvirtual environment under mutually equivalent conditions as
is shown in Fig. 13 [Watanabe et al. 2012].

In telexistence, an operator can feel his/her slave robot as
an expansion of his/her bodily consciousness and has theability to move freely and control the slave robot in a similar
way to his/her body movement. In order to certify this concept in telexistence, the TORSO (TELESAR III) system,
which can acquire visual information in a more natural andcomfortable manner by accurately tracking a person’s headmotion with 6 DOF, was constructed [Tachi et al. 2004].
TELESAR V, a master-slave robot system for performing
body movements including 6-DOF head movement such asthat of TORSO, was developed in 2011 (Fig. 14) [Tachi etal. 2011]. In addition, an operator can feel the fingertip tactile and thermal sensations when touching objects remotely.
The TELESAR V system satisfies the following functions[Fernando et al. 2012].
(1) The operator can freely and independently movehis/her head, upper body, and both arms and hands. In contrast, the slave robot’s arms and hands follow similar movements with 53 DOF while maintaining 6-DOF arm endpoint
accuracy.
(2) The operator can have a clear wide-angle stereo viewof the remote site with 6-DOF point-of-view robot vision accuracy. The operator also has the ability to perform binaural,bi-directional verbal communication.
(3) The operator can grasp and manipulate objects in asimilar manner to a human hand.
(4) The operator feels fingertip tactile and thermal sensations when touching objects remotely.
In July, 2012, it was successfully demonstrated that the
TELESAR V master-slave system can transmit fine haptic
sensations, such as the texture and temperature of material,from an avatar robot’s fingers to the human user’s fingers.

Figure 14: TELESAR V transferring small balls from one
cup to another cup.

S. Tachi / Forty Years of Telexistence—From Concept to TELESAR VI

3. TOWARD A "TELEXISTENCE SOCIETY"[TACHI 2016]
Working at home remotely to date has been limited to communications and/or paperwork that transmit audio-visual information and data, as well as conversations. It was impossible to carry out the physical work at factories or operationsat places such as construction sites, healthcare facilities, or
hospitals; that cannot be accomplished unless the person inquestion is actually on site. Telexistence is a technology thatdeparts from the conventional range of remote communications that transmit only the five senses, and it realizes an innovative method that transmits all the physical functions ofhuman beings and enables the engagement of remote work
accompanying labor and operations that was impossible until now.
If a telexistence society that can delegate physical functions were realized, the relationship between people and industry and the nature of society would be fundamentallychanged. The problems of the working environment would
be resolved, and it would no longer be necessary to work inadverse environments. No matter where a factory is located,workers would be assembled from the entire country or theentire world, so the conditions for locating factories will see
revolutionary changes compared to the past, and population
concentration in the metropolitan area can be avoided. Since
foreign workers would also be able to attend work remotely,
the myriad problems accompanying immigration as a mere
labor force, separate from humanitarian immigration, can beeliminated. Moreover, it will be possible to ensure a 24-hourlabor force at multiple overseas hubs by making use of timedifferences, rendering the night shift unnecessary. Both men
and women will be able to participate in labor while raisingchildren, and this will help to create a society in which it iseasier to raise children.
The time-related costs due to travel in global business will
be reduced. Commuting-related travel will become unnecessary, and transportation problems can be alleviated. It is predicted that it will no longer be necessary to have a home near
one’s workplace, the concentration of population in the metropolis will be alleviated, the work-life balance will be improved, and the people concerned will be able to live where
they wish and lead fulfilling lives.
In addition, owing to additional functions of an avatar robot, which is the body of the virtual self, even the elderly
and handicapped will not be at a disadvantage physicallycompared to young people, since they can augment and enhance their physical functions to surpass their original bodies, and thus they can participate in work that gives full play
to the abundant experience amassed over a lifetime. The
quality of labor will rise greatly, thereby reinvigorating Japan. The hiring of such specialists as technicians and physicians with world-class skills will also be facilitated, and optimal placement of human resources according to competence can also be achieved.
With a view to the future, it will be possible to respondinstantly from a safe place during disasters and emergencies,
and this technology can also be used routinely to dispatch
medical services, caregivers, physicians, and other experts
to remote areas. In addition, owing to the creation of new
industries such as tourism, travel, shopping, and leisure, itwill greatly improve convenience and motivation in the livesof citizens, and it is anticipated that a healthy and pleasant
lifestyle will be realized in a clean and energy-conserving
society (Fig.15).

Figure 15: Telexistence Society.
4. AVATAR XPRIZE
On August 3, 2016, XPRIZE Foundation's Visioneers Prize
Design team visited the Tachi Laboratory at Miraikan (theNational Museum of Emerging Science and Innovation) located in Odaiba. The XPRIZE Foundation was founded in
1995 by Peter H. Diamandis, selected as one of the world's
50 Great Leaders in 2014, who is regarded as a charismatic
persona in the world of innovation. So far, the Foundation
has helmed a number of large-scale projects, such as amanned ballistic spaceflight contest, and cleanup of seawater surface oil. Like Charles Lindbergh's solo nonstop
transatlantic flight, which has explosively expanded new areas of human travel and tourism, the attempt to create a new
industry through a global prize competition and the creationof the current private space industry are considered to be theresults of the XPRIZE.
In one room of the Tachi Laboratory in a private researchbuilding, those who have experienced TELESAR V wore ahead mount display, and at the second they put the gloves onand move, collectively exclaimed, “Yes, this is exactly what
I was looking for!” About 2 months later, the XPRIZE Foundation opened the Summit. About 300 mentors in various
fields: investors, scholars, businessmen, philanthropists, artists, and engineers gathered over 2 days to judge the 9 nominated themes for the next prize competition. Here,TELESARV, which was developed by the author and his
team, was invited by the foundation to give a demonstration(Fig.16). Because of this, it was decided that the theme forthe next prize competition is ANA AVATAR XPRIZE. After that, details of the development goals and evaluation
were solidified by the XPRIZE Foundation, and a formal
opening of the competition was declared on March 12, 2018at the SXSW Hall in Austin, Texas, USA. The plan is to accept entries from around the world until the end of September 2019, narrow down the teams to 150 by February 2020
through a proposal document screening, open the qualifying
tournament in May 2021, and select 20 teams; the main battle will be held in January 2022. In the preliminaries, thephysical examination of the man-machine synergy telexistence system is carried out; during the main competition, candidates will perform tasks in the fields of welfare, disaster
response, and laboratory environment and they will compete
in terms of degree of achievement.
This competition uses cutting-edge technology such as virtual reality, robotics, AI, and network technologies. The aim
is to industrialize telexistence technology where human beings can exist in a number of different places by using theinvited to ICAT-EGVE (2019)

S. Tachi / Forty Years of Telexistence—From Concept to TELESAR VI

robot's body to physically touch objects and work with them.
Thus, AVATAR XPRIZE will lead the way to the Telexistence Society.

Figure 16: Demonstration of TELESAR V at the XPRIZE
Visioneers Summit 2016.
5. TELESAR VI
TELESAR VI (TELExistence Surrogate Anthropomorphic Robot VI) is a newly developed telexistence platform for the ACCEL Embodied Media Project. It was designed and implemented with a full body mechanically unconstrained master cockpit and a 67 degrees-of-freedom(DOF) anthropomorphic slave robot. The system provides afull-body experience of our extended “body schema,” whichallows a human to maintain up-to-date representation in
space of the positions of his/her various body parts, including his/her head, torso, arms, hands, and legs. Body schema
can be used to understand the posture of the remote body andto perform actions with the belief that the remote body is theuser’s own body. With this experience, users can performtasks dexterously and feel the robot’s body as their own,
through visual, auditory, and haptic sensations, which provide the most simple and fundamental experience of a remote existence.

ger, a 2-DOF little finger, and 1-DOF abduction of four fingers. Each leg has 3 DOF for its hip, 1 DOF for its knee, and2 DOF for its ankle. The hands are custom designed, humansized, anthropomorphic robotic hands, which have a similar
number of joints as real human hands. The robotic fingers
of each hand are driven by 16 individual DC motors, anddynamically coupled wires and a pulley-driven mechanism
connect the remaining joints that are not directly attached toa motor. All of the DC motors are connected to the custommade DC motor driver with a combination of optical encoders and potentiometer readings as position measurements.
Furthermore, voltage and current consumption are monitored at each motor, and torque at the motor shaft is calculated.
The robot has two cameras (See3CAM CU135), two microphones (AT9912), and a speaker (SoundCore Boost).
Cameras for capturing wide-angle stereo vision, and stereo
microphones are situated on the robot’s ears for capturing
audio information from the remote site. The operator’s voice
is transferred to the remote site and outputted through asmall speaker installed in the robot’s body area for conventional, verbal, bidirectional communication.
On the master side, the operator’s movements are captured with a motion capturing system (OptiTrack) and sent
to the kinematic generator PC. Finger postures are captured
to an accuracy of 16 DOF using IGS-Cobra gloves with
OptiTrack active markers.
6. CONCLUSIONSTelexistence was conceptualized in 1980 and has been developed through several research and development projects.
Recently, as start-ups specializing in telexistence have been
established, momentum has built toward the industrialization of telexistence and its application to a variety of industrial fields. Contributing to this trend, the XPRIZE Foundation has launched the ANA Avatar XPRIZE as its next challenge. Intense worldwide competition within the emerging
telexistence industry, which integrates AI, robotics, VR, andnetworking, has just begun. In this keynote, forty years of
research on telexistence was overviewed and cutting-edge
technologies of telexistence were explained. We also discussed several problems to be solved to attain a telexistencesociety and foresee the future that telexistence will develop
and create.
ACKNOWLEDGEMENT

Figure 17: General view of TELESAR VI avatar robot.
The TELESAR VI system consists of a master (local) andphysical avatar (remote). As shown in Fig.17, the 67-DOF
dexterous robot was developed with a 6-DOF torso, a 3-DOF
head, two 7-DOF arms, two 16-DOF hands, and two 6-DOF
legs. Its torso has 2 DOF for its chest motion, 1 DOF for its
spine’s telescopic motion, and 3 DOF for its waist motion.
Each arm has 3 DOF for its shoulder, 1 DOF for the elbow,
and 3 DOF for the wrist. Each hand has a 5-DOF thumb, a
3-DOF forefinger, a 3-DOF middle finger, a 2-DOF ring fin-This study on TELEXISTENCE was conducted in collaboration with many former and current students, affiliates, andstaff members of the Tachi Laboratory at the University ofTokyo and Keio University, including H. Arai, T. Maeda, Y.
Inoue, K. Yasuda, T. Sakaki, E. Oyama, Y. Yanagida, N,
Kawakami, I. Kawabuchi, M. Inami, D. Sekiguchi, H.
Kajimoto, H. Nii, Y. Zaitsu, Y. Asahara, K. Sogen, S. Nakagawara, R. Tadakuma, K. Watanabe, K. Sato, S. Kamuro,
T. Kurogi, K. Hirota, M. Nakayama, Y. Inoue, and F. Kato.
This work is partly supported by JST-ACCEL and theauthors declare under their sole responsibility that the experimental protocol was compliant with International Ethics
regulations.

S. Tachi / Forty Years of Telexistence—From Concept to TELESAR VI

References[Fernando etal. 2012][Goertz
1952][Minsky
1980][Mosher
1967][Sheridan
and Ferrell
1974][Sutherland
1968][Tachi et al.
1980][Tachi et al.
1981][Tachi and
Abe 1982][Tachi and
Komoriya
1982][Tachi et al.
1984][Tachi et al.
1988][Tachi et al.
1989]

C. L. Fernando, M. Furukawa, T. Kurogi, K.Hirota, S. Kamuro, K. Sato, K.
Minamizawa, and S. Tachi: TELESAR
V: TELExistence Surrogate Anthropomorphic Robot, ACM SIGGRAPH
2012, Emerging Technologies, Los Angeles, CA, USA, 2012.
R. C. Goertz: Fundamentals of generalpurpose remote manipulators, Nucleonics, vol. 10, no.11, pp. 36–42, 1952.
M. Minsky: Telepresence, Omni, vol.2,
no.9, pp.44-52, 1980.
R. S. Mosher: Handyman to Hardiman,
SAE Technical Paper 670088,
doi:10.4271/670088, 1967.
T. B. Sheridan, and W. R. Ferrell:
Man–Machine Systems, MIT Press,
Cambridge, MA, 1974.
I. E. Sutherland: A Head-Mounted
Three Dimensional Display, Proceedings of the Fall Joint Computer Conference, pp.757-764, 1968.
S. Tachi, K. Tanie and K. Komoriya:
Evaluation Apparatus of Mobility Aidsfor the Blind, Japanese Patent 1462696,
filed on December 26, 1980.
S. Tachi, K. Tanie and K. Komoriya:
Operation Method of Manipulators with
Sensory Information Display Functions,Japanese Patent 1458263, filed on January 11, 1981.
S. Tachi and M. Abe: Study on tele-existence (I), Proceedings of the 21st Annual Conference of the Society of Instrument and Control Engineers (SICE),
pp. 167-168, July 1982.
S. Tachi and K. Komoriya: The Third
Generation Robotics, Measurement andControl, Vol.21, No.12, pp.1140-1146,
1982.
S. Tachi, K. Tanie, K. Komoriya and
M. Kaneko: Tele-Existence (I): Design
and Evaluation of a Visual Display with
Sensation of Presence, Proceedings of
the 5th Symposium on Theory andPractice of Robots and Manipulators(RoManSy ’84), pp.245-254, Udine, Italy, June 1984.
S. Tachi, H. Arai, I. Morimoto and G.
Seet: Feasibility Experiments on a Mobile Tele-existence System, Proceedings of The International Symposiumand Exposition on Robots, pp. 625-636,
Sydney, Australia, 1988.
S. Tachi, H. Arai and T. Maeda: Development of an Anthropomorphic Teleexistence Slave Robot, Proceedings ofthe International Conference on Advanced Mechatronics, pp.385-390, Tokyo, Japan, 1989.

[Tachi and
Yasuda1994][Tachi
1998][Tachi
1999][Tachi et al.
2003][Tachi et al.
2004][Tachi et al.
2008][Tachi et al.
2012][Tachi2015][Tachi
2016][Watanabe
et al. 2012][Yanagida
and Tachi
1993]

Susumu Tachi and Ken-ichi Yasuda:
Evaluation Experiments of a Telexistence Manipulation System, Presence,
Vol.3, No.1, pp.35-44, 1994.
S. Tachi: Real-time Remote Robotics Toward Networked Telexistence, IEEE
Computer Graphics and Applications,Vol.18, No.6, pp.6-9, 1998.
S. Tachi, “Augmented Telexistence,” inY.Ohta and H.Tamura ed., Mixed Reality - Merging Real and Virtual Worlds,
Springer-Verlag, ISBN3-540-65623-5,
pp. 251-260, 1999.
S. Tachi, K. Komoriya, K. Sawada, T.
Nishiyama, T. Itoko, M. Kobayashi and
K. Inoue: Telexistence Cockpit for Humanoid Robot Control, Advanced Robotics, Vol.17, No.3, pp.199-217, 2003.
S. Tachi, N. Kawakami, M. Inami andY. Zaitsu: Mutual Telexistence System
Using Retro-reflective Projection Technology, International Journal of Humanoid Robotics, Vol.1, No.1, pp.4564, 2004.
S. Tachi, N. Kawakami, H. Nii, K.
Watanabe and K. Minamizawa: TELEsarPHONE: Mutual Telexistence Master Slave Communication System basedon Retroreflective Projection Technology, SICE Journal of Control, Measurement, and System Integration, Vol.1,
No.5, pp.335-344, 2008.
S. Tachi, K. Watanabe, K. Takeshita,
K. Minamizawa, T. Yoshida and K.
Sato: TELESAR4: Mutual Telexistence
Robot System using Omnidirectional
Autostereoscopic 3D Display andRetroreflective Projection Technology,
Trans. of the Virtual Reality Society ofJapan, Vol.17, No.1, pp.11-21, 2012.
S. Tachi: Telexistence 2nd Edition,
World Scientific, ISBN 978-981-461806-9, 2015.
S. Tachi: Memory of the Early Daysand a View toward the Future, Presence, Vol.25, No.3, pp.239-246, 2016.
K. Watanabe, K. Minamizawa, H. Nii
and S. Tachi: Telexistence into cyberspace using an immersive auto-stereoscopic display, Trans. of the Virtual Reality Society of Japan, Vol.17, No.2,
pp91-100, 2012.
Y. Yanagida and S. Tachi: Virtual Reality System with Coherent Kinesthetic
and Visual Sensation of Presence, Proceedings of the 1993 JSME International Conference on Advanced Mechatronics (ICAM), pp.98-103, Tokyo, Japan, August 1993.

invited to ICAT-EGVE (2019)