TelesarPHONE- Communication Robot based on Next Generation Telexistence Technologies Naoki Kawakami, Dairoku Sekiguchi,
Hiroyuki Kajimoto and Susumu Tachi
Department of Information Physics & Computing
The University of Tokyo
7-3-1 Hongo, Bunkyo-ku Tokyo 113-8656 JAPAN{kawakami,dairoku,kaji,tachi}@star.t.u-tokyo.ac.jp

Riichiro Tadakuma
Japan Science and Technology Corporation
c/o Department of Information Physics & Computing
The University of Tokyo
7-3-1 Hongo, Bunkyo-ku Tokyo 113-8656 JAPANtadakuma@star.t.u-tokyo.ac.jp

TELEsarPHONE is a conceptual prototype of mutual- telexistence system, designed for a face-to-face tele-communication
via robots. Thanks to the development of telexistence techonologies. We can have feelings that we are present in several realplaces and can work and act. However, those people in the place where someone telexists using a robot see only the robot
but they can’t feel that the person who presents. To solve this problem, the new prototype of a telexistence master-slave
system for remote manipulation experiments is being designed and developed. And we apply RPT (retro-reflectiveProjection Technology) system to TELEsarPHONE system. In the TELEsarPHONE system, the face and the breast of theslave robot TELESAR II is covered by retro-reflective material. To provide the feeling of existence, the real-time image ofthe operator is projected onto there. Peeping from the RPT projector.

by phone (or sound) and television is communication byvision. In the next generation of communication method iscommunication by existence. This is our final goal.
To realize above concept, the new prototype of atelexistence master-slave system for remote manipulation
experiments is being designed and developed. The robot
built for this system is called “TELESAR II.” In this project,
we focus on producing human-like, realistic movement ofTELESAR II. TELESAR II has bath arms and both hands
and a head. It is important that master can present an exact
force to an operator and slave has contact safety with humanin a remote environment. Impedance control typemaster-slave we adopted can realize an exact presentationof force. Moreover, we can realize contact safety withhuman on slave, because slave has sufficient compliance byimpedance control.
To control the slave arm, we developed the master arm ofTELESAR II. For the reason that an operator can gesturesmoothly, this arm has 6 degree of freedom structures tofree an operator's elbow. To control the redundant degree of
freedom of the anthropomorphic slave arm, we put a small
orientation sensor on the operator’s arm. Moreover, thismaster arm is made light and impedance control is appliedso that an operator doesn’t feel a big burden.
Important factor to feel the existence of an operator is not
only motion of the operator but also visual image of theoperator. The robot is painted with retro-reflective materialand can act as screens for HMP (Head Mounted Projector).
So the robot is seen as if it is the operator by the projectionof real image of him or her on it with a sensation ofpresence. (See Fig.1)

1. INTRODUCTION
Telexistence[1-3] is fundamentally a concept named for thegeneral technology that enables a human being to have areal-time sensation of being at a place other than where heor she actually exists, and being able to interact with theremote environment, which may be real, virtual, or a
combination of both. It also refers to an advanced type ofteleoperation system that enables an operator at the control
to perform remote tasks dexterously with the feeling ofexisting in a surrogate robot working in a remoteenvironment. We proposed the concept of Telexistence in1980 and have been working for the realization of theconcept and application to the robotics, design andcommunication. And finally we have realized al prototype
system “TELEsarPHONE “ and exhibited it at the Prototype
Robot Exhibition of EXPO ’05 in Aichi.
2. CONCEPT OF TELEsarPNONE SYSTEM
TELEsarPHONE is a conceptual prototype of mutualtelexistence system, designed for a face-to-face telecommunication via robots. Almost a quarter of a century
have past since our first idea and concept of telexistence
was proposed, and it is now possible to telexist in theremote environment and/or virtual environment with asensation of presence. We can have feelings that we arepresent in several real places and can work and act.
However, those people in the place where someone telexists
using a robot see only the robot but they can’t feel that theperson who presents. It is useless to use TV display on
board the robot to show the face of the user. It would be justcomical and far from reality.
In the history of communication, communication method
has been developing, telephone is a communication method-1-

Head, right/left arms and right/left hands are composed
master-slave systems individually. In the case of
autonomous robots, system needs hard computation toavoid crash of each arms/hands or body. But in the case oftelexistence robot, system needs no collision detection. The
operator calculates it unconsciously. This is very
remarkable factor of telexistence system.
The most distinctive feature of TELEsarPHONE system
is RPT (retro-reflective Projection Technology) system. In
our laboratory at the University of Tokyo, a newtype ofvisual display is being developed called, which uses
retro-reflective material as its projection surface. The
retro-reflective surface works as a special screen. Under theRPT configuration, a projector is arranged at the axial
symmetric position of a user's eye with reference to ahalf-mirror, with a pinhole placed in front of the projector toensure adequate depth of focus. In the TELEsarPHONE
system, the face and the breast of the slave robot TELESAR
II is covered by retro-reflective material. To provide thefeeling of existence, the real-time image of the operator isprojected onto there. Peeping from the RPT projector.
According to the safety restrictions of AICHI EXPO, wedemonstrate by using fixed type HMP, but we also
developed wearable HMP.
If there are enough space and money, we will construct
the 3-D Display system by immersive wide-angle screensystem. But we don’t. So In AICHI EXPO, we decide to use
simple 3-D Display system. The system consists of four 3-D
displays (17inch Lenticular type) arranged in front, left,right and bottom (like T-shape). Because of lenticular, theoperator can see stereoscopic without any glasses. And 3-D
camera system is located above the TELESAR II (not ONhead). Because the display system is fixed, camera systemshould be fixed. If we want use stereoscopic camera locatedon eye poison of TELESAR II, the operator must wear
HMD (Head-Mounted Display), which hide the operator’seyes, it makes a poor communication. The 3-D camera
system consists of four couple of CCDs (totally eightCCDs). Each couple of camera is for front, right, left andbottom view. Each of captured images by those cameras aretranslated to respective displays. The system provides
approximated first-person’s view for the operator.

Fig.1 Conceptual Image of TELEsarPHONE
3. DESIGN OUTLINE OF TELEsarPHONE SYSTEMThe TELEsarPHONE system is composed of 4
sub-systems, slave robot TELESAR II, master cockpit, 3-D
display system for cockpit and RPT system. (Fig.2,3)

Fig. 2 System Outline

3. DETAILS OF SLAVE ROBOT ”TELESAR II” AND
MASTER COCKPIT

Fig.3 System Layout(left: Cockpit Booth / right: Robot Booth)The slave robot TELESAR II has 2 DOF head, which canrotate pitch and roll axes. There are 2 CCD Cameras forstereoscopic inside its head. (Not used in Aichi-Expo) The
motion of the head is synchronized with motion ofoperator’s head, which measured by head tracker of Mastercockpit. TELESAR has not only head but also both arms,
and both hands. The operator can control these arms andhands easily, because the motion of the operator‘shands/arms are measured by master cockpit and are thesynchronized with TELESAR’s.

-2-

3.1 Master-Slave Arms[4]From the viewpoint of simplification of function andcontrol, high rigidity, small inertia, and the fact that thepower to be applied to the operator’s hand is along at most 6axes, we designed the master arm to specialize in the power
presentation function in these 6 axes, and also made its
mechanism 6-DOF. (Fig. 4)
Since the measurable movement of the master arm that
follows the operator's hand has 6 DOF, we decided to use a
new lightweight posture sensor composed of twoacceleration sensors to measure the final DOF, which iscritical to identifying the posture of the operator’s wholearm, including his or her elbow.
Altogether, the mechanism serves as a master systemwith 7 DOF for measurement of the arm’s posture, and 6
DOF for power presentation. Since the posture sensor isvery light compared with mechanical restraints on theoperator’s elbow, the sensor enables much freer movementof his or her arm without any undesirable load on it.
Moreover, since the master arm has less than 7 DOF, itachieves power presentation with high rigidity and stability.

3.3 Master-Slave Hands[5]We have developed a new type of master-slave hand toimprove these disadvantages. (Fig.5,6) Our master hand has
two features as follows.

Fig.5 Master Hand (right: structure left:overview)Fig.4 Slave Arm (right: structure left:overview)As described above, we should leave the motion of theoperator as unrestrained as possible in telexistence.
Therefore we adapted an exoskeleton-type mechanism forthe master arm. Although the master arm does notnecessarily need an exoskeleton-type structure to present
forces along 6 axes, it has a mechanism similar to the humanbody in general, and it always follows human motion. So,
we decided to use the structure because we considered thatit could be widely adapted for movement of an operator
with minimal size requirements, which is an essential
characteristic to correspond to a human's various actions ineveryday life. In our system, the number of degrees offreedom in the master arm is less than that of a human arm,so it is impossible for the master arm to follow an operator's
whole arm completely, and it introduces the possibility ofmutual interference between the human arm and the masterarm.(Fig. 4)

Fig.5 Slave Hand
One is the compact exoskeleton mechanism of themaster hand’s finger contrived to cover wide workspace ofan operator. Possible ways to place an exoskeleton
mechanism are to place it over (parallel joint) or beside(coaxial joint) an operator’s finger. To place it over theoperator’s finger, there is a problem that master finger
disturbs the motion of the operator’s finger when the finger
is bent. However, to place it beside an operator’s finger isdifficult, because there is little space to place it. To solve
this problem, we propose “circuitous joint” that coincides
joint axis of the master hand with that of the operator byextending the link length in proportion to the joint angulardisplacement (Fig.7).

Fig.7 Master Hand (left: stretch right: bend)
The other is the encounter-type force feedback.
Encounter-type device stays at the location of the object in
the remote world and waits for an operator to encounter it.
As shown in Fig. 8, our encounter-type master hand’s finger

Fig.4 Master Arm (right: structure left: overview)-3-usually follows the operator’s finger without contact. It
enables him to touch nothing when the slave hand contacts
nothing. When slave hand touches an object, master finger
contacts the operator’s finger and provides resistive force.
Therefore, our master hand is able to provide both perfectunconstrained motion and natural touch sensation.

5. CONCLUSION
In this paper, we finally built up the prototype telexistencesystem. And we exhibited it at AICHI EXPO as “TELEsarPHONE.”
Fig. 10 shows overview of slave-robot
TELESAR II and master cockpit.

Fig.10 right: TELESAR II,
Left: Master Cockpit (operator’s view).
Peeping from RPT projectors as shown in Fig. 11,
visitors can see the real-time operator’s image on thesurface of TERESAR II’s body.

Fig.8 Encounter-type master-slave hand
4. DETAILS OF RPT SYSTEM
Fig.9 shows the principle of RPT system, a projector isarranged at the axial symmetric position of a user's eye withreference to a half-mirror, with a pinhole placed in front of
the projector to ensure adequate depth of focus. [2][3]Fig.11 TELSAR II(left: normal view right: Peeping from RPT)
Until now, we have studied elemental technologies oftelexistence. Now we are getting a clue for the next
generation of telexistence. We will keep studying about
essence of the telexistence.
6. REFERENCES[1] S. Tachi, H. Arai, and T. Maeda: “Tele-existence masterslave system for remote manipulation,” Proc. Int’lWorkshop on Intelligent Robots and Systems (IROS’90),
pp. 343-348, 1990.
[2] S. Tachi: “Telecommunication, Teleimmersion andTelexistence”, Ohmusha, IOS Press[3] S. Tachi: “Telecommunication, Teleimmersion andTelexistence 2”, Ohmusha, IOS Press[4] S. Nakagawara, R. Tadakuma, H. Kajimoto, N.
Kawakami, and S. Tachi: "A Method to Solve Inverse
Kinematics of Redundant Slave Arm in Master-Slave
System with Different Degrees of Freedom",Proc. of.
ISMCR2004[5] S.Nakagawara, I.Kawabuchi, H.Kajimoto, N. Kawakami, and S.Tachi: "An Encounter-type Multi-fingered Master
Hand Using Circuitous Joints", Proc. ICRA2005

Fig.9 RPT (left: Principle right: overview)
The face and breast of TELESAR II is covered byretro-reflective materials. The ray comes from a direction
reflect the same direction on the surface of retro-reflectivematerial. Thanks to this characteristic, image is projected
onto the surface of TELESAR II without distortion. And if
we use many RPT projectors in different direction andproject difference images, observer can see corresponding
image. In AICHI EXPO we set up 3 RPT projectors in front,
right side and center of these. Observers simultaneously see
the front face, right face and slant face of the operator.

-4-