䢳䣃䢳䢯䣌䢲䢳

Study of Telexistence LXVIII- Reaching a remote object with
6DOF point of view and arm endpoint accuracy Charith Lasantha FERNANDO†, Masahiro FURUKAWA†, Tadatoshi KUROGI†, Sho
KAMURO†*, Katsunari SATO‡, Kouta MINAMIZAWA†, and Susumu TACHI††Graduate School of Media Design, Keio University‡Graduate School of System Design and Management, Keio University†*Graduate School of Information Science and Technology, The University of Tokyo
email{charith, m.furukawa, kurogi, kamuro, sato, kouta, tachi}@tachilab.org
In Telexistence, point of vision through robot eyes and posture of the robot’s hand should be in a close
correlation in order to feel the extended body sensation and transfer realistic haptic sensation. However, present
Telexistence robots mostly operate its torso in less than 4 DOF, thus it is difficult to map the vector between the pointof vision to arm endpoint in operator to the same vector of slave robot. In order to address this issue we designed anddeveloped TELESAR V, with a 5DOF torso and a 7 DOF arm attached to it so that the slave robot can have 6 DOF
point of vision and 6 DOF arm endpoint accuracy with operator’s posture. This structure enables the operator tofreely move in his space and experience his extended bodily border while operating the robot and perform
manipulatory task remotely. In addition ability to transfer the fingertip haptic and temperature sensation, operator canfeel his bodily consciousness is extended to the robot.
Key Words: telexistence, haptics, dynamic trajectory generationof dexterity in entire upper body.

1. Introduction
Telexistence is a concept that refers to the technology, whichenables a human to have a real-time sensation of being at aplace other than where he actually exist, and to interact withthe remote environment [1]. We have achieved human-likearm and hand movements in a Telexistence operation
through TELESAR II [2] and human-like neck movements tovisually interact and explore 3-dimensional details in aremote object through TORSO [3].
In case of an operator cannot reach a remote object in atelexistence operation, usually he will use the robot’s mobile
base platform to move [4, 5] sides/towards the object thru ajoystick or similar technology because conventional robot
upper body does not allow him to reach through his upper
body. When this locomotive operation is performed, operator
can see the robot’s vision is being changed while he was kept
on the same posture. Due to this conflicting visual feedback,
operator confuse about the feeling of his body is attached tothe slave robot. Thus, when grasping and touching objects inremote environment, operator cannot perceive the fingertip
haptic sensation as his own fingers were touched.
In order to address the above situation two conditions has tobe addressed. i.e point of vision and arm endpoint accuracy.
We developed “TELESAR V” (as shown in Fig. 1) where
the operator has to sit on a stationary chair and user’s spinal,
neck, head and arm movements are mapped into a dexterous
slave robot and allows the operator to feel the vector between
the point of vision and arm endpoint is similar to his own
body vector between the same points. Furthermore we haveadded fingertip haptic and thermal sensation [6] whentouching objects remotely. The ability to control the spinal,
head and arm movements individually, it increases the levelFig. 1 TELESAR V in a glass ball pouring Tele-operation

As a result of synchronized visual, auditory, and haptic
sensation, TELESAR V makes the operator to feel the bodily
consciousness is extended to the slave robot in a
Tele-operation.
Also, been able to freely move the upper-body makes theoperator feel that he is not restrained to a static body andincreases the level of dexterity.

2. Related background study
Humans normally experience the conscious self as localized
within their bodily borders. Due to high level of spatial unity
perceived with multi sensory (audio, visual, kinesthetic andhaptic) inputs makes human to think that the body they see,and can feel touch is their own body. Researches have
proved that if the same spatial unity is kept with a high levelof multi sensory applied to human, neurological conditions䣐䣱䢰䢢䢳䢴䢯䢵䢢䣒䣴䣱䣥䣧䣧䣦䣫䣰䣩䣵䢢䣱䣨䢢䣶䣪䣧䢢䢴䢲䢳䢴䢢䣌䣕䣏䣇䢢䣅䣱䣰䣨䣧䣴䣧䣰䣥䣧䢢䣱䣰䢢䣔䣱䣤䣱䣶䣫䣥䣵䢢䣣䣰䣦䢢䣏䣧䣥䣪䣣䣶䣴䣱䣰䣫䣥䣵䢮䢢䣊䣣䣯䣣䣯䣣䣶䣵䣷䢮䢢䣌䣣䣲䣣䣰䢮䢢䣏䣣䣻䢢䢴䢹䢯䢴䢻䢮䢢䢴䢲䢳䢴䢳䣃䢳䢯䣌䢲䢳䢪䢳䢫such as Rubber Hand Illusion (RHI) [7] and Body Transfer
Illusion [8] can be felt.

3. Implementation

TELESAR V wrist is restricted with 60 to 105 degrees
respectively due to mechanical restrictions. In conventional
robots, even though operator can reach a remote object withposition accuracy, approaching orientation becomes different
from the operators wrist.
Thus, in order to achieve a 6 DOF arm endpoint accuracy ofunder the mechanical limitations of the robot wrist, we use
remaining DOF’s of robot upper arm and torso tocompensate the lower arm orientation error.

Fig. 2 TELESAR V, System Overview

As shown in Fig.2, TELESAR V system consists of a Master(Local) and Slave (Remote) system. A 54 DOF dexterous
robot is developed with 5 DOF torso, 3 DOF head, 7 DOF
arms and 16 DOF hands. Robot also has Full HD (1920 ×1080 pixels) cameras for wide-angle stereovision and stereo
microphones situated on robot’s ears as audio input from theremote site. Voice from operator is transferred to the remotesite and output through a small speaker installed on robot’s
mouth area. Apart from bi-directional verbal communication,
robot fingers have a vision based tactile sensor [11] to sense
both force and temperature of robot fingertip.
In Master side, operator movements are captured using amotion tracking system (OptiTrack) and generates desired
robot joint kinematics. Finger bending is captured with anaccuracy of 5 DOF using “5DT Data Glove 5 Ultra”.
Operator perceives the visual sensation as seen from therobot through a HD (1280×800 pixels) wide angle Head
Mounted Display (HMD). Remote fingertip proprioceptive
sensation is reproduced using vertical and shearing forces
excreted by two motor driven belt mechanism [12] while
fingertip temperature is reproduced using thermoelectric
actuators (Peltier) placed on the bottom side of operator
fingertips.

3.1 Point of view (POV) and arm endpoint accuracy
In Telexistence, it is important to have the ability to see theremote site with the same point of view as if you would see
with naked eye and change the point of view synchronous
according to the head movements of operator. Furthermore
approaching vector of the manipulator arm should be same as
the operator lower arm vector in order to feel the arm is partof your extended body.
In human anatomy, wrist joint has a higher DOF compared toconventional robots. Similar to conventional robots, inSince the 7 DOF arm is attached to the 5 DOF upper, theorientation of the upper body has a close relationship withthe final arm endpoint accuracy. i.e if the robot torso does
not follow the operator spinal movements it will reduce thelevel of dexterity of the slave robot. In order to overcome thissituation we modeled the robot torso to follow the operator’s
all spinal movements (extension, flexion, lateral flexion, andaxial rotation). This does not perform a exact match, but aapproximate model so that the error can be compensated byusing the remaining DOF’s of torso. With correct shoulder
center point and 3 DOF of Head, point of view is accuracy isobtained. Thus with the development of TELESAR V, above
two conditions could be satisfied.

Fig. 3 Remote object’s orientation accuracy obtained through bodyrotation (remote object position is fixed)

With this configuration, as shown in Figure 3(a), operator
tries to reach the remote object through correct orientation,
but due to the joint limitation of the wrist it is not possible.
This kind of a situation is naturally resolved by humans byuse of spine to rotate the body and approach using theextended right hand. As shown in Figure 3(b) withTELESAR V, such situation can be naturally resolved withthe ability of rotating the robot’s body synchronous withoperator’s spinal movements.
Any spinal movement of operator will result in recalculating
the arm trajectories to compensate both position, orientationand maintain good endpoint accuracy. Thus, it cancompensate on both horizontal and vertical planes giving䣐䣱䢰䢢䢳䢴䢯䢵䢢䣒䣴䣱䣥䣧䣧䣦䣫䣰䣩䣵䢢䣱䣨䢢䣶䣪䣧䢢䢴䢲䢳䢴䢢䣌䣕䣏䣇䢢䣅䣱䣰䣨䣧䣴䣧䣰䣥䣧䢢䣱䣰䢢䣔䣱䣤䣱䣶䣫䣥䣵䢢䣣䣰䣦䢢䣏䣧䣥䣪䣣䣶䣴䣱䣰䣫䣥䣵䢮䢢䣊䣣䣯䣣䣯䣣䣶䣵䣷䢮䢢䣌䣣䣲䣣䣰䢮䢢䣏䣣䣻䢢䢴䢹䢯䢴䢻䢮䢢䢴䢲䢳䢴䢳䣃䢳䢯䣌䢲䢳䢪䢴䢫operator to move in x-y and x-z planes. With this setup, wehave achieved a high level of dexterity and less complexity
in object manipulation.

vector of the hand. As shown in Fig. 5, a similar scenario canbe executed with TELESAR V by first keeping arm end
posture fixed and move the body and head around.

4. Tele-Operation scenarios
Due to the independent control mechanism of TELESAR V
head body, arms and hands it defines many new interactionsin Telexistence.
4.1 Body compensated hand movements with fixed headorientation.
As shown in Fig 4(a) conventionally when the operator is notable to reach a remote object by fully stretching the arm, heneeds to move the robot base close enough to reach in it’s
fully extended arm. But TELESAR V can rotate its bodywith an axial rotation as shown in Fig 4(b) and extends thereach easily. This eliminates the use of moving the robot
base and preserves the human natural body movements. In
addition operator can see the remote object clearly and does
not need to change his head orientation while taking the axialrotation.

Fig. 5 Expanding field of view through combined bodyand hand movements.

4.3 Active compliant force
When manipulating objects humans usually use their body asa support structure for arms and legs. (i.e back muscle
supports the arm movements.) It helps to generate impulse
forces when necessary. Similarly as shown in Fig .6 inhuman anatomy the operator can induce an accelerated force
at the end point by a combined movement of body and arms.
This is not possible if the combined forces have a delay or if
the body and arm has different posture from the operator.
As shown in Eq. 1 the tangential force element created by thebody rotation adds to the arm trajectory and accelerates theend point movement. These accelerations help to generatestroke and impulse motions.

Fresul = Farm + Fbody × cos(θ ) ……….. (1)€Fig. 4 Axial rotation of body helps to extend the reach.

4.2 Body compensated hand movements with dynamic
head orientation.
Sometimes operator needs to explore the grabbed objects
carefully before or after picking it up. (For example pickingup a chemical container and reading the label). In a similar
situation, humans will naturally use a combination of headand arm movements to rotate and explore the entire
360-degree rotation. The operator can move his body andhead around to explore while preserving the approaching

Fig. 6 Combined motion results in accelerated force䣐䣱䢰䢢䢳䢴䢯䢵䢢䣒䣴䣱䣥䣧䣧䣦䣫䣰䣩䣵䢢䣱䣨䢢䣶䣪䣧䢢䢴䢲䢳䢴䢢䣌䣕䣏䣇䢢䣅䣱䣰䣨䣧䣴䣧䣰䣥䣧䢢䣱䣰䢢䣔䣱䣤䣱䣶䣫䣥䣵䢢䣣䣰䣦䢢䣏䣧䣥䣪䣣䣶䣴䣱䣰䣫䣥䣵䢮䢢䣊䣣䣯䣣䣯䣣䣶䣵䣷䢮䢢䣌䣣䣲䣣䣰䢮䢢䣏䣣䣻䢢䢴䢹䢯䢴䢻䢮䢢䢴䢲䢳䢴䢳䣃䢳䢯䣌䢲䢳䢪䢵䢫

With the use of active compliant force, TELESAR V candraw Japanese calligraphy. When performing the above
operation, operator was able to use his body and arm
separately (Fig. 7) to maintain a fine brush strokes.

remote slave robot.

5. Conclusion
With TELESAR V Telexistence system we have been able
have an exact mapping of robot’s vision point of view andarm endpoint between the operator and slave robot.
Furthermore the ability to freely move and use spinal
movements increased the level of dexterity in entire upper
body and perform human like motion. With thisconfiguration and ability to feel the multiple sensor inputs atthe same time, operator forgets his own body and feels that
he is inside the robot. Furthermore he can feel that his bodily
consciousness is extended to the remote slave robot.

Acknowledgement
This project is supported by JST-CREST Haptic Media
Project and collaborated with Kyokko Electric co.,Ltd.
Fig. 7 TELESAR V, drawing Japanese Calligraphy


[1]
[2]

4.4 Haptic sensation with 6 DOF end point accuracy

Fig. 8 Haptic Sensation when pouring glass balls.

With all the above enhancements enabled we have performed
few tests to explore how the haptic sensation enriches in a
Telexistence operation. As shown in Fig. 8, Operator grabs 2cups placed on the table and pours the glass balls from one toanother. In this experiment operator cannot only explore theabove explained scenarios of body and arm movements, butalso can feel the tactile feedback generated when glass balls
hit the cup. The ability to see the glass balls falling, hear thesound of glass ball hitting the cup and feel the haptics
sensation makes the operator feels that he is inside the robot
and perceive that his bodily consciousness is extended to theReferences
Tachi, S. “Telexistence” World Scientific 2010.
Kawakami, N. and Sekiguchi, D. “TelesarPHONE - Communication
Robot based on Next Generation Telexistence Technologies”,
International Symposium on Robotics (pp. 1-4), 2005.
[3] Watanabe, K., Kawabuchi, I., Kawakami, N., Maeda, T., and Tachi, S.
“Torso: completion of egocentric telegnosis system”, ACM
SIGGRAPH 2007 emerging technologies, ACM, New York, NY, USA,SIGGRAPH ’07. 2007.
[4] Tachi S., Arai, H., and Maeda, T. “Tele-existence master-slave system
for remote manipulation”, Proceedings. IROS ’90. IEEE International
Workshop on, 343 –348 vol.1. 1990.
[5] Bauml, B.; Schmidt, F.; Wimbock, T.; Birbach, O.; Dietrich, A.; Fuchs,M.; Friedl, W.; Frese, U.; Borst, C.; Grebenstein, M.; Eiberger, O.;
Hirzinger, G.; , "Catching flying balls and preparing coffee: Humanoid
Rollin'Justin performs dynamic and sensitive tasks," Robotics andAutomation (ICRA), 2011 IEEE International Conference on , vol., no.,pp.3443-3444, 9-13 May 2011[6] Sato, K., Minamizawa, K., kawakami, N., and Tachi, S. “Haptictelexistence” ACM SIGGRAPH 2007 emerging technologies, ACM,
New York, NY, USA, SIGGRAPH ’07. 2007.
[7] Botvinick & Cohen, Rubber Hand Illusion (RHI), 1998[8] Slater, M., Spanlang, B., Sanchez-Vives, M. V., and Blanke, O. “FirstPerson Experience of Body Transfer in Virtual Reality”. PLoS ONE,5(5), e10564. Public Library of Science. 2010.
[9] H. Henrik Ehrsson. “The experimental induction of out-of-bodyexperiences” Science, Vol. 317, p. 1048, 2007.
[10] Bigna Lenggenhager, Tej Tadi, Thomas Metzinger, and Olaf Blanke.
“Video ergo sum: Manipulating bodily self-consciousness” Science,Vol. 317, pp. 1096– 1099, 2007.
[11] Katsunari Sato, Kazuto Kamiyama, Naoki Kawakami, Susumu Tachi,"Finger-Shaped GelForce: Sensor for Measuring Surface Traction
Fields for Robotic Hand," IEEE Transactions on Haptics, vol. 3, no. 1,pp. 37-47, Jan.-Mar. 2010[12] Minamizawa, K., Kamutaro, S., Fukamachi, S., Kawakami, N., andTachi, S. “Ghostglove: haptic existence of the virtual world” ACM
SIGGRAPH 2008 new tech demos, ACM, New York, NY, USA,SIGGRAPH ’08, 2008.

䣐䣱䢰䢢䢳䢴䢯䢵䢢䣒䣴䣱䣥䣧䣧䣦䣫䣰䣩䣵䢢䣱䣨䢢䣶䣪䣧䢢䢴䢲䢳䢴䢢䣌䣕䣏䣇䢢䣅䣱䣰䣨䣧䣴䣧䣰䣥䣧䢢䣱䣰䢢䣔䣱䣤䣱䣶䣫䣥䣵䢢䣣䣰䣦䢢䣏䣧䣥䣪䣣䣶䣴䣱䣰䣫䣥䣵䢮䢢䣊䣣䣯䣣䣯䣣䣶䣵䣷䢮䢢䣌䣣䣲䣣䣰䢮䢢䣏䣣䣻䢢䢴䢹䢯䢴䢻䢮䢢䢴䢲䢳䢴䢳䣃䢳䢯䣌䢲䢳䢪䢶䢫