Master-Slave Robot Hand Control Method based on
Congruence of Vectors for Telexistence HandManipulation
Yasuyuki Inoue
Institue of Gerontology
The University of TokyoTokyo, Japany-inoue@tachilab.org

Fumihiro Kato
Institue of Gerontology
The University of Tokyo
Tokyo, Japan
fumihiro.kato@tachilab.org

Abstract— This research focuses on robot hand control oftelexistence robot to realize precise hand manipulation. A
humanoid surrogate robot which possesses large number of DoFand works under master-slave controlling can reproduce
operator’s hand and finger movements, and therefore it willrealize fine and delicate teleoperation as if the work wereperformed by operator’s own hand. However, even though therobot hand moves with dexterity, the body size difference
between operator and robot has great influence to theconsistency of position and configuration of body-parts such as
fingertips between them, and may lead the difficultly of manual
teleoperation. The problem is how to manage various operator’s
body size and maintain the correspondence of the operator tothe robot. To overcome this issue, we propose a novel method ofrobot hand control which is based on congruence of two vectors
bellows: (1) thumb tip to opposite fingers and (2) point-of-viewto thumb/fingers. Considering that the spatial relationshipbetween thumb and opposite fingers are important to perform
precise hand manipulation, to keep the vector makes replicationof operator’s handshape on the robot hand. Also, because theoperator of telexistence robot sees robot’s body-parts including
hand via eye-camera as his/her surrogate body, to match thevector realizes consistent experiences about hand position andaids intuitive operation. The proposed method was implemented
to the control system of newly developed telexistence robot“TELESAR6” and the performance of hand manipulation using
the method was evaluated.
Keywords— telexistence, master-slave control, robot handI. INTRODUCTION
Telexistence is a concept that refers to the technology,
which enables a human to have a real-time sensation of being
at a place other than where he/she actually exist, and to interact
with the remote environment [1]. Telexistence is realized byusing a master-slave controlled robot such as TELESARs[2,3,4,5,6]. By using telexistence robot, operator can virtuallyexist in a remote location, and perform various manual
operations through the robot hands controlled by operator’sfinger movements. For the operator, the robot hands should
work freely and completely like his/her own hands. Therefore,
it is important to develop precise robot hand control system.
For achieving skillful manipulation equivalent to human’shand, a lot of anthropomorphic robot hand systems have beendeveloped for decades [7,8,9,10]. These robot hands consistof many joints and have complicated kinematic structures,
therefore, it is difficult to generate the motion signals fordesired hand manipulation. Therefore, various control
methods for robot hand also have been proposed. Most
common way is master-slave controlling, that is, a humanoperator’s finger movements are measured via motion sensors

Susumu Tachi
Institue of Gerontology
The University of Tokyo
Tokyo, Japantachi@tachilab.org

and are applied to the robot either immediately (real-timeduplication) or later (playback of the motion). This method isuseful to acquire human-resemble motion for the robots.
However, even though the motion duplicated on the robot issimilar to the original motion, the hand shape is not exactly
same due to the difference of body size and joint excursion
between the operator and robot. This is a large problem fortelexistence because skilled hand motions performed by antrained person cannot be reproduced on the robot unless therobot hand imitates the function of the motion. Otherwise, theskilled motion would be degraded. The essential thing is thatthe mismatch of kinematic parameters causes position anddirection errors at the endpoint, that is, fingertip. This leads
impediment of intuitive hand manipulation for the robot’s
operator. To improve the accuracy of reproduced finger
movements, a previous study [11] applied calibration
procedure of kinematic parameter between operator and robot
by using correspondences of predefined finger postures.
However, the accuracy was only ensured for the neighborhoodof the calibrated postures and not certain for other finger
positions.
In this study, we propose a novel master-slave control
method of anthropomorphic robot hand system. The key of themethod is that the geometrical congruence of polygonal chain
consisted of spatial vectors connecting between five fingers is
kept for both operator and robot. This is because the distanceand direction between thumb tip and opposite fingertips are
crucial to perform precise hand manipulation such as pinchingaction. Furthermore, to maintain spatial positions of fingers
seen on the operator’s visual field, the vector between pointof-view (POV; i.e., eye/camera) and fingers is also kept. As
long as the two constraints are satisfied, even though thedifferences of kinematic parameters between operator androbot are not negligible, intuitive and precise robot handmanipulation is realized. Further procedure of proposed
method is described below.
II. ROBOT HAND CONTROL METHOD FOR TELEXITENCE
A. Limitation of Proposed Method
The target of proposed method is anthropomorphic fivefinger robot hand but not limited to the human-like structure,and it can be applied for other types of manipulator as long as
the robot has at least two end effectors which can be regarded
as thumb and opposite finger. Also, it is desirable that theworkspace of each finger is wide enough to cover the moving
area of human’s finger movement.

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEEUSB ISBN 978-1-7281-4898-4, IEEE Catalog number CFP19MCR-USB

A1-1-1

B. Formularization of Telexitence Robot Control
Previous study regarding telexistence robot [6] indicatesthat it is important to keep operator’s eye-to-arm endpoint
vector equal to robot’s eye-to-arm endpoint vector so that theoperator can always see the robot arms like as his/her ownbody. This is realized by satisfying two equations regarding
POV and endpoint as shown below;

are overlapped as wide as possible. However, the adjustment
cannot be applied to all fingers. This is because the thumband finger positions of operator and robot at the same posture
are usually different, though all thumb-to-finger vectors
between them should be kept. Therefore this is applied toonly thumb as expressed below;
𝑠𝑙_𝑒𝑦𝑒𝑤𝑤
𝑠𝑙_𝑒𝑦𝑒𝑻 = 𝑚𝑠_𝑒𝑦𝑒 𝑻()𝑠𝑙_𝑒𝑦𝑒𝑚𝑠_𝑒𝑦𝑒𝑠𝑙_𝑒𝑛𝑑𝑝𝑜𝑖𝑛𝑡 𝑻 = 𝑚𝑠_𝑒𝑛𝑑𝑝𝑜𝑖𝑛𝑡 𝑻()where, 𝐵𝐴𝑻 is 4x4 homogeneous transform matrix whichdescribes position and orientation of frame B relative to A.
Equation (1) brings out exact tracking of operator’s headmotion, whereas (2) leads consistency of end-point’s positionand orientation seen from the operator’s view. Thoughdetailed expressions of them depend on the link structure of
the robot, these are expressed as matrix multiplication offrames in terms of joints such as 𝑁0𝑻 = 01𝑻 12𝑻 ⋯ 𝑁−1
𝑁𝑻. For
simplicity, we assume that Eq. (1) is always satisfied, and only
consider (2) to discuss how to match operator’s fingers to therobot. Although Eq. (2) should be satisfied for all fingers, it isquite difficult to realize the ideal matching because most ofanthropomorphic robotic finger has just 3 – 5 DOF whereas itis required to 6DOF for each finger to totally satisfy theequation. Also, even though the robotic finger has over 6DOF,
size difference between operator and robot (i.e., mismatch offinger length) causes different hand posture between them.
Therefore, it is necessary to compromise some factors.
Considering that thumb plays crucial role for the handmanipulation and can move most flexibly among 5-finger, theposition and direction of robot’s thumb tip must be controlledwith high accuracy.
𝑠𝑙_𝑒𝑦𝑒𝑚𝑠_𝑒𝑦𝑒𝑠𝑙_𝑡ℎ𝑢𝑚𝑏𝑻 = 𝑚𝑠_𝑡ℎ𝑢𝑚𝑏𝑻()On the other hands, other four opposite fingers just have tofollow the positions of the endpoints of operator’s fingertips
relative to thumb tip to mimic operator’s hand manipulation.

𝑷𝑠𝑙_𝑡ℎ𝑢𝑚𝑏0 =𝑚𝑠_𝑒𝑦𝑒𝑷𝑚𝑠_𝑡ℎ𝑢𝑚𝑏0𝑤
In order to satisfy Eq. (5), the frame of robot hand 𝑠𝑙_ℎ𝑎𝑛𝑑𝑻 inwhich the root joints of thumb and fingers are belonging
should be adjusted so that the position error of thumb between
operator and robot at the initial posture relative to the frame
origins of the hands 𝑠𝑙_ℎ𝑎𝑛𝑑𝑷𝑠𝑙_𝑡ℎ𝑢𝑚𝑏0 − 𝑚𝑠_ℎ𝑎𝑛𝑑 𝑷𝑚𝑠_𝑡ℎ𝑢𝑚𝑏0
is offsetted as shown below;

𝑠𝑙_𝑒𝑦𝑒 𝑠𝑙_ℎ𝑎𝑛𝑑
𝑚𝑠_𝑒𝑦𝑒𝑠𝑙_ℎ𝑎𝑛𝑑 𝑻 𝑠𝑙ℎ𝑎𝑛𝑑 ′𝑻 = 𝑚𝑠_ℎ𝑎𝑛𝑑 𝑻

𝑷𝑠𝑙_𝑓𝑖𝑛𝑔𝑒𝑟 =𝑚𝑠_𝑡ℎ𝑢𝑚𝑏

𝑷𝑚𝑠_𝑓𝑖𝑛𝑔𝑒𝑟

()

()

Fig. 2 illustrates the relationship. Thanks to the adjustment,
the motion ranges of fingers are mostly overlapping eachother. It is useful to compensate individual hand size
differences of operators.
III. IMPLEMENTATION OF PROPOSED METHOD
The proposed method discussed in previous section isimplemented into the master-slave robot control system oftelexistence robot “TELESAR6”. Also, virtual-reality (VR)simulator of the robot is developed together to verify thebehavior of entire robot systems. Here, the specification ofTELESAR6 and its VR simulator is briefly introduced toclarify the operating environment. Then, the master-slave
robot control system for the environment is explained.
A. Specification of RobotTELESAR6, developed by JST-ACCEL project [12], is alife-size humanoid robot for telexistence which consists of
total 67 DOF mechanism (body:6, head:3, arm:7x2, leg:6x2,hand: 16x2). The head is equipped with stereo camera and bieyeeyefinger

𝑠𝑙_𝑡ℎ𝑢𝑚𝑏

()

thumb

fingerthumb

where, 𝐴𝑷𝐵 is 4x1 position vector of point B relative toframe A. Eq. (3) and (4) mean that the polygonal chain ofvectors connecting to POV, thumb and fingers is congruence
between operator and robot. Fig. 1 shows the schema of thecongruence of vectors in the method. As long as finger
opposition is kept, the operator can control the robotic fingers
as well as own fingers.
C. Motion Range Adjustment of Robotic Finger
Eq. (3) and (4) are not always satisfied because of manyfactors such as excess of motion range. If the operator’sfinger is longer than the robot’s one, for example, it isdifficult to reach the target fingertip position. In order toutilize the workspace of finger motion effectively, the initial
finger position of operator and robot should correspondregardless of size difference so that the motion ranges of them

Operator

RobotFig. 1. Vector congruence between operator and robot. The polygonal
chain connecting to eye, thumb and finger is identical for both coordinates.
Thumb position is described relative to eye, whereas four finger positions
are described relative to thumb.

A1-1-2

xms
Operator𝑚𝑠_𝑒𝑦𝑒𝑠𝑙_𝑒𝑦𝑒𝑷𝑡ℎ𝑢𝑚𝑏0𝑚𝑠_𝑒𝑦𝑒𝑚𝑠_ℎ𝑎𝑛𝑑𝑻θmsθsl+Robot_𝑷𝑡ℎ𝑢𝑚𝑏0

PositionController

𝑠𝑙_𝑒𝑦𝑒𝑻
𝑠𝑙_ℎ𝑎𝑛𝑑𝑠𝑙_𝑒𝑦𝑒𝑠𝑙_ℎ𝑎𝑛𝑑𝑻Fig. 2. Motion range adjustment of the robot hand. The frame of robot handis determined so that thumbtip positions at initial posture are corresponding.
Red-colorred vector drawn on the right side means the offset of frame
relative to the original location.

naural microphones, and it can move according to theoperator’s head movement in order to satisfy Eq. (1). The arm
can perform reaching movement according to the operator’s
hand position and orientation in order to satisfy Eq. (2). The
endpoint of the arm is equipped with anthropomorphic fivefinger robot hand which has 16 DOF (thumb:5, index:3,middle:3, ring:2, little:2, abduction:1). The number of DOF isenough to apply the proposed method.
B. VR Simulator
A VR simulator is developed to provide equivalentworking environment of TELESAR6. In the simulator, virtual
robot is connected to the measurement and control systems
and works according to the user’s operation same as real robot.
Also, the simulator presents POV image of robot’s eyecamera to the user as though he/she were actually operating
the robot. As shown in previous study [13], VR simulator isuseful to verify the robot system. It can also be used to train
the operators as well. Fig. 3 shows a picture of the simulator
and it’s user.
C. Master-Slave Control SystemFig. 4 shows block diagram of the master-slave system inwhich the proposed method is implemented. Operator’s body
movement is recorded from the measurement systems and theposition and orientation data of target body-parts (head, torso,shoulders, hands, fingertips, etc.) are obtained.

POV image
VR robot

VR simulator

IKSolverFig. 4. Block diagram of the master slave system. IK solver outputs desired
joint angles of the robot in master side, whereas position controller performs
feedback loop to follow the target values in slave side.

These target values are inputted to the inverse-kinematics(IK) solver, and it generates desired joint angles of the robot.
The IK calculation proceeds in stages as shown below;
1.

6DOF body joints and 3DOF head joints in the robot
are determined by using operator’s head position andorientation (6DOF) and torso orientation (3DOF) inorder to satisfy Eq. (1).

2.

7DOF arm joints in the robot are determined by using
operator’s hand position/orientation (6DOF) andshoulder direction (1DOF) in order to satisfy Eq. (2).
In addition, the endpoint of arm is slightly shiftedfrom operator’s wrist as shown (6) in order to satisfy
Eq. (5).

3.

5DOF thumb joints in the robot are determined byusing operator’s thumb position (3DOF) with thumbtip direction (1DOF) in order to satisfy Eq. (3). The
redundant DOF can be used for keeping finger
opposition.

4.

3DOF finger joints in the robot are determined byusing operator’s finger position (3DOF) in order tosatisfy Eq. (4). Note that the robotic finger is a 3-link
planar manipulator, therefore there are only 2DOF interms of position. Thus, this is performed with the aidof an abduction joint to shift laterally.

To apply above, the vector congruence described in Section II
is realized for the robot. As shown in Step 1 and 2, the body(equipped with the head) and the arm (equipped with the hand)
are involved with the method.
IV. DEMONSTRATION OF DUPLICATED HAND POSTURE
By using proposed method, operator’s finger movement isduplicated on the robot hand. However, because of size
difference between operator and robot, the overall finger
shape is not always same. This means that the posture of
middle and proximal phalanxes as well as palm are notcongruent between operator and robot. Still, proposed method
guarantees to match the fingertip positions of them. Fig. 5
demonstrates some examples of corresponding hand postures.
Operator’s position and orientation data were measured by
optical motion capture system (OptiTrack), and calculated
target values were applied to virtual robot in the VR simulator.
As shown in Fig. 4, operator’s hand manipulation (pinchingan eraser on different side) is functionally duplicated by therobotic fingers.

UserV. CONCLUSION
Fig. 3. VR simulator of TELESAR6, develped by using a game engine,
Unity. The 3D model in the simulator has same shape and hierarichical link
structure of real robot, and their joints moves according to the control signals
sent from the master-slave system. Note that the user in this picture is notequipped with finger motion measuement instruments for simplicity

In this study, a novel robot hand control method wasproposed to realize precise hand manipulation under masterslave controlling telexistence operation. This method is basedon the congruence of spatial vectors between operator’scoordinates and robot’s ones.

A1-1-3

and grasping. Inter Conf on Intelligent Manipulation and Grasping.
123-124, 2004.
[9] H. Liu, K. Wu, P. Meusel, N. Seitz, G. Hirzinger, M.H. Jin, Y.W.Liu,S. W. Fan, T. Lan, and Z. P. Chen, Multisensory five-finger dexterous
hand: the DLR/HIT hand II. Proceedings of IROS. pp. 3692-3697,
2008.
[10] T. Mouri, H. Kawasaki, K. Yoshikawa, J. Takai, S. Ito,
Anthropomorphic robot hand: Gifu hand III, Proc. Int. Conf. Control
Autom. Syst., pp. 1288–1293, 2002.
[11] C. L. Fernando, M. Furukawa, K. Minamizawa, S. Tachi, Experiencing
ones own Hand in Telexistence Manipulation with a 15 DOF
Anthropomorphic Robot Hand and a Flexible Master Glove,
Proceedings of the 23rd International Conference on Artificial Realityand Telexistence (ICAT), Tokyo, Japan, pp.20-27, 2013.
[12] http://www.jst.go.jp/kisoken/accel/en/research_project/ongoing/h26_0
5.html[13] M. Y. Saraiji, C. L. Fernando, M. Furukawa, K. Minamizawa andS. Tachi, Virtual Telesar - Designing and Implementation of a Modular
Based Immersive Virtual Telexistence Platform, Proceedings of 2012
IEEE/SICE International Symposium on System Integration(SII2012),
pp.595-598, 2012.

Fig. 5. Examples of corresponding hand postures. Although the overall
hand shape is different, the length and direction between thumb to index
finger keeps same. Note that other fingers (middle, ring, little) are notcontrolled in the example because of the limitation of measurement.

The proposed method was implemented to a robot control
system for a telexistence robot and the effectiveness isevaluated by using VR robot simulator. However, the method
has not been tested on the real robot hand because the robot isunder development. Therefore it is necessary to perform
further evaluations of hand manipulation in the real situation.
ACKNOWLEDGMENT
This research is supported by ACCEL Program fromJapan Science and Technology Agency, JST.
REFERENCES
[1]
[2]

[3]

[4]

[5]

[6]

[7]

[8]

S. Tachi, Telexistence (2nd Edition). World Scientific, 2015.
S. Tachi, H. Arai and T. Maeda, “Tel-existence master-slave systemfor remote manipulation”, Proceedings ofIROS ’90. IEEE
International Workshop, pp. 343–348, 1990.
N. Kawakami and D. Sekiguchi, “TelesarPHONE – Communication
Robot based on Next Generation Telexistence Technologies -”,International Symposium on Robotics, pp. 1 – 4, 2005.
K. Watanabe, I. Kawabuchi, N. Kawakami, T. Maeda and S. Tachi,“Torso: completion of egocentric telegnosis system”, ACMSIGGRAPH 2007 emerging technologies, 2007.
S. Tachi, K. Watanabe, K. Takeshita and K. Minamizawa, “MutualTelexistence Surrogate System: TELESAR4 - telexistence in realenvironments using autostereoscopic immersive display -”, IEEE/RSJ
International Conference on Intelligent Robots and Systems, pp. 25 –30, 2011.
C. L. Fernando, M. Furukawa, T. Kurogi, S. Kamuro, K. Sato, K.
Minamizawa and S. Tachi, “Design of TELESAR V for Transferring
Bodily Consciousness in Telexistence”, 2012 IEEE/RSJ International
Conference on Intelligent Robots and Systems, pp.5112 – 5118, 2012.
L. B. Bridgwater, C. Ihrke, M. A. Diftler, M. E. Abdallah, N. A.
Radford, J. Rogers, S. Yayathi, R. S. Askew, and D. M. Linn. The
robonaut 2 hand‐designed to do work with tools. InRobotics andAutomation (ICRA), 2012 IEEE International Conference, pp. 3425–3430, 2012.
M. Reichel, Transformation of shadow dextrous hand and shadow
finger test unit from prototype to product for intelligent manipulation

A1-1-4