3Inνited Presentations
Paper presented at Sensors and their Applications XI, September 200 I©2001 IOP Publishing Ltd

Networked Telexistence: Real-time Remote Robotics with aSensation of Presence
S Tachi
The University of Tokyo , Department of Information Physics & Computing
7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656 Japan
http://www.star.t.u tokyo.ac.jp/
Abstract: Telexistence (tel-existence) is fundamentally an advanced type of telerobotics
system that enables an operator at the control to perform remote tasks d巴xt巴rously with thefe巴ling that he or she exists in a su汀ogate robot that is working in the remote environment. It
is also a concept nam巴d for the technology which 巴nables a human being to have a real time
sensation of being at the place other than the place wh巴re h巴 or she actually exists, and isable to interact with the remote and/or virtual environment. It enables a human operator tohave a sensation of being in a remote real environment wh巴re a suπogate robot exists, while
being augment巴d by a virtual 巴nvironment synth巴sized by a computer, whos巴 structure isbas巴d on th巴 sensor information on board the robot. The concept of telexist巴nce, i e., virtual
existenc巴m a remot巴 or computer-gen巴rated environment, has developed into a national
R&D scheme of R-Cub巴d (Real-time Remote Robotics) for the advanced and comfortabl巴
life of the 21st century network society. R Cubed can be said to be the effort toward thenetworked telexistence. Based on the scheme of R-Cubed, in the fiscal y巴ar of 1998,
Ministry of Economy, Trade and Industry (METI) of Japan launched a national five-year
project call巴d th巴 Humanoid Robotics Project (HRP). This is an effort to integrate
t巴lerobotics, network t巴chnology and virtual reality into network巴d telexistence. This pap巴rreviews how telexistenc巴 technology has been developed, and d巴scribes a n巴wly d巴veloped
t巴lexistence cockpit for humanoid robot control, and shows a technical demonstration toevaluate the dev巴loped cockpit and the robot. A human operator controls the robot within th巴
remote cockpit as if (s)he w巴re inside th巴 robot itself.
田Keywords telexistence, tele-existence, telep,esence, virtual reality, master-slave manipulation,
humanoid robots

1. Introduction
Technology of projecting ours巴Ives in real time into remote environments by using surrogate
robots, remote/local computers and cybernetic human interfaces is called telexistence (tel-existence).
This concept has been expanded to include projection into computer-generated virtual environments,
and also th巴 use of a virtual environm巴nt for the augmentation of the real environment.
The concept of th巴 t巴lexist巴nc巴was propos巴d by the author in 1980 and it played the role of thefundamental principle of the eight year Japan巴se National L 紅ge Scale Project of "Advanced Robot
Technology in Hazardous Environment，＇’which started in 1983 together with the concept of theThird G巴neration Robotics. Through this project th巴oretical consideration has been done andsystematic design procedure has been established. Experimental hardware telexist巴nce syst巴m has
been made and the feasibility of the conc巴pt has been demonstrated.

4Sensors and their Applications XI

一Bb、

1・i 川w

d しF

一、v ’

一 剛 、，

一J

ji
ぷ ／じ！
一In our first report [1], the principle of the telexistence sensory display was proposed, and its
design procedure was explicitly defined. Experimental visual display hardware was built, and thefeasibility of th巴 visual display with a sensation of presence was demonstrated by psychophysicalexperiments using the test apparatus.
A method was also proposed to develop a mobil巴 t巴lexistence system, which can be remot巴ly driven
with th巴 auditory and visual sensation of presence. A prototype mobile t巴lev巴hicle system wasconstructed and the feasibility of the method was 巴valuated [2].
Th巴 first prototype tel巴xistence master slave system for remote manipulation experiments wasdesigned and d巴veloped, and a preliminary evaluation experiment of telexistence was conduct巴d.
An experimental telexistence system for real and/or virtual environm巴nts was designed anddevelop巴d, and by conducting an experiment comp紅ing a tel巴xistence mast巴r-slave system withconventional master-slave system, efficacy of the telexistence master-slave system and thesuperiority of the telexistence method were d巴monstrat巴d experimentally (Fig.1) [3, 4, 5].

一・一

Fig. 1 Telexistence Surrogate Anthropomorphic Robot (TELESAR) at Work.

2. Augmented Telexistence
T巴lexist巴nce can be divided into two categories: telexistence in the real world that actually exists
at a distance, and is connected via a robot to th巴 place where the user is located; and telexistence inthe virtual world that does not actually exist but is synthesized by a computer. The former can be’’＇called ”transmitted r巴ality ，＇ while the latter is synthesized reality." The synthesiz巴d reality can beclassified into two, i.e., a virtual environment as a model of th巴 real world and a virtual environmentof an imaginary world. Combination of transmitt巴d reality and synthesized reality, which is call巴dmixed reality, is also possible and has a great importance in real applications. This we callaugmented telexistence to clarify the importanc巴 of harmonic combination of real and virtualworlds.
Augment巴d telexistence can be used in several situations. By takeing for instanc巴，of controlling aslave robot in a poor visibility environment, an experimental augmented telexistence system using avirtual environment has be巴n constructed. The environment model has also been constructed fromth巴 design data of th巴 r巴al environment. Wh巴n augment巴d reality is used for controlling a slave
robot, the modeling e町ors of th巴 environm巴nt model must be calibrated. A model-bas巴d calibration
system using image measurements has been proposed for matching th巴 real environm巴nt and avirtual environm巴nt. The slave robot has impedance control mechanism for contact tasks and forcompensating for the errors that remain even after the calibration. An experimental op巴ration in apoor visibility environm巴nt was successfully conducted by using Telesar (Fig. I) and the virtual

6Sensors and their Applications XI

In an R-Cubed system, each robot site has its server of its local robot. Th 巴 robot type varies from ahumanoid (high 巴nd) to a movable camera (low end). A virtual robot can also be a controlled
system to be telexisted.
Each client has its teleoperation system. It can be a control cockpit with master manipulators and ahead mounted display (HMD) or CAVE Automatic Virtual Environment (CAVE) on th 巴 high 巴nd.
It is also possible to use an ordinary personal computer system for its control system on the low end.
In order to support th巴 low end users to control remot巴 robots through networks, RCML/RCTP (R­Cubed Manipulation Language / R-Cubed Transfer Protocol) is now under development [9].
To standardize th巴 following control scheme, a language call巴d RCML (http://www.rcml.org/),
which describes a remote robot's features and its working environm巴nt, has b 巴en proposed. A
communication protocol RCTP, which is d巴signed to 巴xchang巴 control commands, status data, andsensory information between the robot and the user, has also been developed.
With Web browsers a user accesses a Web site describing information of a robot in the form ofhype巾xt and icon graphics using WWW browser. Clicking on an icon downloads the description
file, which is written in RCML format, to the user's computer and launches the RCML browser. TheRCML brows巴r parses the downloaded file to process th巴 geometry information, including the包rnngement of the degrees of freedom of the robot, controllable parameters, available motion
ranges, sensor information, and other pertinent information. The browser decides what kind andhow many devices are required to control the remote robot. It then generates a graphical user””
interface (GUI) p加el to control the robot, plus a vid巴 O window that displays the images seen by’
the robot and a monitor window that lets users obs巴rve the robot s status from outside the robot. If
出e user has a device, such as 6 degrees of-fre 巴dom (DOF) position/orientation sensor, to indicate
the robot-manipulator's endpoint, th 巴 user can employ that instead of the conventional GUI panel（日g.5).

teleoperated robot s
remote site in th巴 fol
he or she exists insic
Bas巴d on R-Cubed
Robot (FNR), whid
Technology Project,a five-ye 訂 project 1control cockpits andFor METI's natiom
a new type of cockpi
The telexistence c,subsystems: a auc
communication subi…

亘書

幽－!>"(_P"_P"

S町l'OUII

Fig.(lllllli理

Fig 目 5 An Example of an RCML Browser .

4. Humanoid Robotics Project (HRP)
A novel robot system capable of assisting and cooperating with people is necessary for any
human-centered system to b巴 used for activities such as the maintenance of plants or power stations,th巴 operation of construction work, the supply of aid in case of emergency or disaster, and the car巴
for elderly people. If we consider such systems from both a technical and a safety point of view,
however, it is clearly intractable to develop a completely autonomous robot system for theseobjectives. The robot system should therefore be realized with the combination of autonomous
control and teleoperated control. By introducing tel 巴xist 巴nce techniques, as an advanced typ 巴 of盟

Fig.7 Telexistece Co
In order to addres
Display), a suロounCAVE), has recent
images captured bywhich allows the orwalk around. In acrobot site, he or sh(system with a head