     

Study on Telexistence LXX:- Position estimation of head, body and hand in Telexistencewith an anthropomorphic slave robot Charith Lasantha FERNANDO 1), Masahiro FURUKAWA1), Tadatoshi KUROGI 1), Kyo HIROTA1),
Sho KAMURO 2), Kouta MINAMIZAWA1) , and Susumu TACHI 1)

1) Graduate School of Media Design, Keio University(223-8526, 4-1-1 Hiyoshi,Kohoku-ku,Yokohama-city,{charith, m.furukawa, kurogi, nippachi, kouta, tachi}@tachilab.org)
2) Graduate School of Information Science and Technology, The University of Tokyo,(113-8656, 7-3-1 Hongo, Bunkyo-ku, Tokyo, kamuro@tachilab.org)

Abstract: In Haptic Telexistence, posture of the operator body, head and hand should be closely mapped tothe slave robot in order to transfer realistic haptic sensation. However, present Telexistence robots do provide
position accuracy, but due to the limited DOF, it is difficult to keep the same posture of each joint close to theoperator. We introduce a 3 DOF head, 6DOF torso, 7 DOF arms in TELESAR V so that the individual joints
can be kept in a close posture to the operators’ at all time. This structure enables us to maintain a synchronizedposture w.r.t to the operator even when he is moving his entire body and maintain the position accuracy bycompensating with the upper arm as it cannot be seen through the robot eyes via a HMD. With auto
stereoscopic vision, binaural audio and haptic sensation, operator can feel robot’s head, body and hand, as hisown. Thus can perceive his bodily consciousness is with the robot’s body boundaries.
Key Words: Telexistence, Haptic Telexistence, Robotics, Dynamic Trajectory Generation

1. INTRODUCTION
Telexistence is a concept that refers to the technology, whichenables a human to have a real-time sensation of being
somewhere else than where he actually exists and to interact
remotely with real or virtual environments [1]. In 1988, anexoskeleton type master-slave system [2] called TELESAR I
was developed for performing remote manipulation experiments.
In 2005, a mutual telexistence master-slave system calledTELESAR II [3][9] was developed with human-like arm andhand movements. The system had the functionality to performconventional bi-directional verbal communication with a remoteparticipant as well as nonverbal gestures such as handshaking.
In 2007, a TORSO with a head [4] was developed which cantransfer visual information in a more natural and comfortablemanner by tracking the head motion of a person with 6-DOF
accuracy.
In general, humans experience the conscious self as localized

Fig.1. (a) Master-Slave configuration, (b) HD wide angle

within their bodily borders. Due to high level of spatial unity

view, (c) Slave robot

perceived with multi sensory inputs makes human to think thatsearches have proved that if the same spatial unity is kept with athe body they see,and can feel touch is their own body. Also, rehigh level of multi sensory applied to a human, neurologicalconditions such as Rubber Hand Illusion (RHI) [5] and Body

operator can freely move in his space while able to mimic theTransfer Illusion [6], [7] can be felt.

spinal movements and perform human-like body assisted stroke

It has been evident from the previous telexistence researchmotion. As shown in Fig. 1(b) robot can maintain a 6 DOF point[2][3][4], if the operator can feel the slave robot as an expansionof view and arm endpoint accuracy so that the vector betweenof his own body and ability to freely move and control the slave

the two points of vision and arm endpoint is seamlessly mapped

robot in a similar way to how his body moves, it will increase

to the operators same points.

the throughput of the performed remote task. In addition with
fingertip haptic and thermal sensation [10] synchronized with

3. IMPLEMENTATION

his kinematic model, operator can perceive a natural haptic
sensation when touching objects remotely [12].
In order to achieve this requirement in telexistence, it isBelt driven
Haptic Actuator
PeltierGel Layer +Thermochromic Ink

essential to have the following fundamental conditions satisfied.
1. Operator should be able to freely and independently move

Vision Based
Force Sensor

Wide-angle
CameraMotorshis head, upper body, arms and hands. In contrast, slave
robot’s hands should follow similar movements while
maintaining 6 DOF accuracy of arm and head endpoint.
2. Robot trajectories should seamlessly maintain the exact
vector from eye-to-hand to be same with operators’.
3. Operator should clearly see a wide-angle stereo view of

MASTER (LOCAL)SLAVE (REMOTE)

Fig. 2. TELESAR V: System Overview

the remote site and feel the robot vision is same as his own
vision in a dynamic trajectory.

As shown in Fig. 2, TELESAR V system consists of aIn order to address the above points, a slave robot with higher

Master (Local) and Slave (Remote) system. A 53 DOF

level of dexterity is necessary. We developed “TELESAR V”, adexterous robot is developed with 6 DOF torso, 3 DOF head, 7telexistence master-slave robot system with a conjunction of 53

DOF arms and 15 DOF hands. Robot also has Full HD (1920 ×DOF to perform full body movements.

1080 pixels) cameras for capturing wide-angle stereovision andstereo microphones situated on robots ears for capturing audio

2. DESIGN CONSIDERATIONSConventional

tele-operation

systemfrom the remote site. Voice from operator is transferred to thesometimes

uses

remote site and output through a small speaker installed on

exoskeleton based master system [2][8] to capture therobots mouth area for conventional verbal bi-directional

movements of operator. These systems limits the operator’scommunication.

movements to the exoskeleton system’s mechanical constrains.
Thus in order to satisfy the first condition, a non-mechanical,
not direct attachable measurement system is necessary. This

3.1 Development of 53 DOF human sized anthropomorphic
robot

also gives full flexibility to move head, body and arms
independently as desired.

As shown in Fig. 3, TELESAR V slave robot consists of 4

In order to map the point-of-view and arm endpoint 6 DOF

main systems. (torso, head, arms and hands). Torso is developed

and to maintain the eye-to-hand vector, a higher DOF robot isbased on a modified “Mitsubishi PA 10-7C Industrial Robot

necessary. Conventional dexterous robots [8] are not capable ofManipulator” placed upright. First 6 joints of the manipulator

achieving this due to not having enough DOF in the torso. A

arm is used as torso and last joint with a separately attached 2robot that can mimic similar spinal movements (extension,DC motors are used as the 3 DOF (roll, pitch, yaw) head.

flexion, lateral flexion, and axial rotation) similar to a human is

A custom designed 7 DOF human sized anthropomorphic

required as robot’s torso. In order to address the above point

robot arm is fixed between the Torso joints 5 and 6 to make itboth torso and an anthropomorphic robot arm should have

similar to human sized dexterous robot. In order to increase theminimum of 6 DOF. Furthermore robot’s eye coordinateslevel of dexterity of the slave robot arm, it is designed withshould be exactly following the operator’s eye coordinates insimilar limiting angles of each joints compared with ordinary

order to feel that the robot vision is exactly a replicate of hishuman arm. However we have included a position based[11].

electrical limit overriding the mechanical limit to provide extra

We developed a system called “TELESAR V” that satisfies

safety in case of a joint angle overshoot. Table. 1 shows thethe above mentioned 3 conditions. As shown in Fig. 1(a)mechanical and electrical joint angle limitations in positive and

negative direction.

drivers
All dimensions are in millimeters (mm)380Overall Body
Weight：50kg

witha

combinationof

opticalencoders

andpotentiometer reading as position feedback. Communication

65

between the motor drivers and PC is carried out through aPCI-Express x1 bus.

110

Neck Roll(±30°)
Neck Pitch,Yaw

300

5030100



280

330

3.2 Trajectory mapping Algorithm
240

In the trajectory generation for torso, first condition is to200

freely move without any constrains while robot should follow

500

280

TELESAR V ARM

the operator’s spinal movements (extension, flexion, lateral

11450

42453025

following condition (Eq.1) has to be satisfied.

102210.5

ZH
15

that the robot’s point-of-view is exactly same as the operator’s,

I

Overall Weight : 500g
(Excluding Force andTemperature Sensors)

20

II

XH

78

III

20

T = ms−eyewT.................(1)wsl−eye

18

200.5
14

IV

66

Secondly, when robot head, body and arms are dynamically20315

numerical kinematics for 6 DOF serial-chain. In order to feel

185

25
30
11.5

flexion, and axial rotation). This is achieved by modified

XH2

1620

33.5
241565240

YH

V
9moving, it is necessary to maintain the eye-to-hand vector

TELESAR V HAND

seamlessly mapped between the operator to robot’s. (Eq. 2)

TELESAR V BODY and HEAD

ms−eyeT = ms−hand
T.................(2)sl−eyesl−handFig. 3. Kinematic Configuration

This is kept tracked by dynamically generating the arm
Arm joints are driven with 12V DC motors and first 3 joints
(J1, J2, J3) implements Harmonic Gears to maintain a very low

trajectories based on the slave robot’s shoulder joint coordinate
frame satisfying Eq. 3.

backlash and vibration while provide the necessary torque. As−1 ms−eye
T = sl−eye
sl−shT . ms−handT .................(3)sl−shsl−handfor the hand, a custom designed human sized anthropomorphic
robot hand is used. The hand is having similar number of joints

Apart from the above conditions, operator should see thecompared to an ordinary human hand. 15 individual DC motors

robot’s forearm vector similar to his own. However since

drive robots fingers and dynamically coupled using wires and aoperator cannot see his upper arm it is not necessary to maintainpulley driven mechanism couples the remaining joints that does

the same vector between shoulder to elbow. In the dynamic

not directly attach to a motor.

trajectory generation, any compensation is appended to theupper arm while keeping the forearm vector mapped.

Table. 1. JOINT LIMITS OF 7 DOF
ANTHROPOMORPHIC ROBOT ARM

Joint

Mechanical Angle
limit

3.3 Communication protocol for telexistenceElectrical Angle Limit

Negative

Positive

Negative

Positive

J1-90◦

145◦

-90◦

+145◦

J2-100◦

+20◦

-100◦

+18◦

J3-152◦

+32◦

-150◦

+30◦

J4-135◦

-2◦

-130◦

0◦

J5-93◦

+93◦

-90◦

+90◦


Fig. 4. Communication ArchitectureWhen designing the telexistence master-slave platform, we
had to consider a multi platform PC environment in order toJ6-15◦

+45◦

-15◦

+40◦

support all the hardware limitations. Thus, it was necessary toJ7-45◦

+60◦

-40◦

+60◦

share the same data among 4 workstations and synchronize thedata with real-time. We have been trying UDP datagrams at thebeginning, but it was slowing down the entire execution cycle ofAll the DC motors are connected to standard DC motorthe PC due to rapid data receive. After lot of experiments,multicast and unicast was selected as the most efficient dataK. Watanabe, I. Kawabuchi, N. Kawakami, T. Maeda, andtransferring protocol. Each workstation has it’s own shared

S. Tachi, “Torso: completion of egocentric telegnosis

memory which contains raw rigid body tracked data, fingersystem,” in ACM SIGGRAPH 2007 emerging technologies,

bending data, joint space data and sensor space data etc… These

ser. SIGGRAPH ’07. New York, NY, USA: ACM, 2007.

data are real-time synchronized between 4 workstations as seen

[Online].Available:http://doi.acm.org/10.1145/1278280.12

from Fig. 4. All the communication speed has a constant refresh

78302

rate of 1Khz because for our application it is not necessary to

M. Botvinick and J. Cohen, “Rubber hands feel touch thathave a higher update rate. In order to preserve the higher details

eyes see.” Nature, vol. 391, no. 6669, p. 756, 1998.

of tactile data and to keep a low latency, currently the setup is[Online].

constructed in a way that it works on a local environment. Available:

http://www.ncbi.nlm.nih.gov/pubmed/9486643



4. CONCLUSION

M. Slater, B. Spanlang, M. V. Sanchez-Vives, and O.
Blanke, “First person experience of body transfer in virtual

reality,” PLoS ONE, vol. 5, no. 5, p. e10564, 05 2010.

We have constructed a 53 DOF anthropomorphic robot for

B. Lenggenhager, T. Tadi, T. Metzinger, and O. Blanke,

performing telexistence experiments. With a combination of 3

“Video ergo sum: manipulating bodily self-consciousness.”

DOF head, 6DOF torso, 7 DOF arms in TELESAR V theScience, vol. 317, no. 5841, pp. 1096–1099, 2007.

individual joints can be kept in a close posture to the operators’[Online].

Available:

at all time. Furthermore the eye-to-hand vector is seamlessly

http://www.ncbi.nlm.nih.gov/pubmed/17717189

mapped between master and slave in order to keep the same

B. Bauml, F. Schmidt, T. Wimbock, O. Birbach, A.

viewpoint while maintaining the forearm vector similar toDietrich, M. Fuchs, W. Friedl, U. Frese, C. Borst, M.

operators’. With this setup, we were able to transfer haptic

Grebenstein, O. Eiberger, and G. Hirzinger, “Catchingsensation (force, tactile and thermal) in a more realistic manner.

flying balls and preparing coffee: Humanoid rollin’justin

Furthermore, with auto stereoscopic vision, binaural audio andperforms dynamic and sensitive tasks,” in Robotics andhaptic sensation, operator can feel robot’s head, body and hand,Automation (ICRA), 2011 IEEE International Conference

as his own. Thus can perceive his bodily consciousness is withon, may 2011, pp. 3443–3444.

the robot’s body boundaries.

S. Tachi, K. Watanabe, K. Takeshita, K. Minamizawa, T.
Yoshida, and K. Sato, “Mutual telexistence surrogate

Note: Mechanical components of Arm, Hand and Head aresystem: Telesar4 - telexistence in real environments usingdeveloped

autostereoscopic immersive display -,” in Intelligent

inKAWABUCHI

MechanicalEngineering

Laboratory, Inc.

Robots and Systems (IROS), 2011 IEEE/RSJ InternationalConference on, sept. 2011, pp. 157 –162.
 K. Sato, K. Minamizawa, N. Kawakami, and S. Tachi,

ACKNOWLEDGEMENT
This project is supported by JST-CREST Haptic Media“Haptic telexistence,” in ACM SIGGRAPH 2007 emerging

Project and collaborated with Kyokko Electric co.,Ltd.

technologies, ser. SIGGRAPH ’07. New York, NY, USA:

ACM,REFERENCESS. Tachi, Telexistence. World Scientific, 2010. [Online].

[Online].

Available:

 S. Tachi, K. Tanie, K. Komoriya, and M. Kaneko,

Available:http://www.worldscibooks.com/compsci/7079.h“Tele-existence (i): Design and evaluation of a visualtml

display with sensation of presence,” in Proceedings. 5th

S. Tachi, H. Arai, and T. Maeda, “Tele-existenceInternational Symposium on Theory and Practice of robots

master-slaveand Manipulators on, june 1984, pp. 245 –254 vol.1.

systemforremotemanipulation,”

inProceedings. IROS ’90. IEEE Interna- tional Workshop on,
2007.

http://doi.acm.org/10.1145/1278280.1278291

 K. Minamizawa, S. Kamuro, S. Fukamachi, N. Kawakami,

jul 1990, pp. 343 –348 vol.1.

and S. Tachi, “Ghostglove: haptic existence of the virtualN. Kawakami and D. Sekiguchi, “TelesarPHONE -world,” in ACM SIGGRAPH 2008 new tech demos, ser.

Communication

SIGGRAPH ’08. New York, NY, USA: ACM, 2008, pp.

RobotbasedonNextGeneration

Telexistence Technologies -,” in International Symposium

18:1–18:1.

on Robotics, 2005, pp. 1–4.

http://doi.acm.org/10.1145/1401615.1401633

[Online].

Available: