Experiencing ones own Hand in Telexistence Manipulation
with a 15 DOF Anthropomorphic Robot Hand and a Flexible Master Glove
Charith Lasantha Fernando∗

Masahiro Furukawa†

Kouta Minamizawa‡

Susumu Tachi§

Graduate School of Media Design
Keio University

A BSTRACTThis paper describes a new type of robust control mechanism fora 15 DOF anthropomorphic robot hand in telexistence manipulations using a flexible fiber based master glove to experience thevisual-kinesthetic sensation of one’s own hand in remote manipulations. Accordingly, a master-slave telexistence system was constructed with the following: a 14 DOF modified optical fiber based
data glove for capturing the complex finger postures of the master
operator without any mechanical constraints; a novel finger posture mapping algorithm that is independent from the effects of different finger sizes and digit ratios; and a 15 DOF anthropomorphic slave robot hand for reconstructing the operators finger posture. This paper describes the importance of feeling one’s fingers
in a telexistence manipulation, control mechanism for accurate finger posture capture/reconstruction where the effectiveness has beenverified through a set of experiments and a subjective evaluation.

rehearsal to perform tasks remotely. Operators lifelong experienceon doing things (playing games, handling tools etc.) could be continued. In addition, it is possible to use his muscle memories, previous learning so that the training can be minimized or eliminated.
Secondly, since there is no thinking or processing overhead and ahuman brain is used in thinking and processing, it will be able to react to un-excepted dynamic behaviors. With the above advantages,
the task effectiveness of teleoperations could be increased.

Index Terms:
H.4.3 [Information Systems Applications]:
Communications Applications—Computer conferencing, teleconferencing, and videoconferencing; I.2.9 [Artificial Intelligence]:
Robotics—Kinematics and dynamics
1 I NTRODUCTION
Dexterous robot hands are used [7, 3, 2, 6] in traditional teleoperations to perform manipulations that involves fine finger movements.
These robot hands are controlled manually by humans or a combination of artificial intelligence driven hybrid semi-autonomous
control mechanics. Hybrid control systems does provide muchaccurate grasping techniques based on the pre-programmed grasp
techniques. However, it does not provide any realistic sensation ofexperiencing that the hand that they see is their own or their presence in the remote environment. In daily life, the coherent relationship between visual-kinesthetic sensation on various body parts
allow humans to naturally experience their body awareness. In addition, extending the awareness of their body to a tennis racket ora hammer is possible. Similarly, this eye-hand coordination can beextended for secondary tools in remote side through teleoperation
technologies.
The awareness of different body parts in a remote manipulation
can be helpful for various tasks. For example, the awareness ofthe operators head can be helpful to inspect the remote objects ina natural manner with accurate three-dimensional details such as
size and distance to object information. Awareness of one’s arm is
helpful to naturally reach to an object without having to depend on
the visual information accuracy. If this awareness is kept continuously throughout the teleoperation, users will no longer need any∗ e-mail: charith@kmd.keio.ac.jp† e-mail:m.furukawa@kmd.keio.ac.jp‡ e-mail:kouta@kmd.keio.ac.jp§ e-mail:tachi@tachilab.org

Figure 1: (a) Master-Slave Configuration, (b) Slave Hand as seenfrom the HMD, (c) Slave Hand used in a Manipulation

Telexistence systems enable a human to have a real-time sensation of being at a place other than where he actually exists andto interact with the remote environment [8, 14] with his entire
body. With the development of a master-slave telexistence system“TELESAR” [15, 10, 11, 16, 9, 12] a combination of vision, auditory and kinesthetic sensation was achieved by Tachi et al. The
authors also achieved to match the differences of dynamics of therobot and human body by using a force feedback mechanism [13]for arms. With the correct eye-hand coordination and multi-sensory
information acting on a user’s body, telexistence systems allow auser to experience the remote body as their own body during remotemanipulation. The coherent relationship between visual-kinesthetic
sensation allows the user to experience his eye-hand coordinationsynchronization on various body parts. Previous telexistence systems [15, 10, 11, 16, 9, 12] provide these head and arm awareness.
However there is no system to have an awareness of operator’s fingers and hand during remote manipulations.
In telexistence manipulations, to capture the movements of theoperator’s fingers, a master hand is used. They can be divided
into two categories. Exoskeleton type such as CyberGrasp [18]are bulky and requires the operator to wear a glove to measure
the posture of the finger. In contrast, endoskeleton type such as

Rutgers Master II [1] type does not require wearing a glove, butalso lightweight and compact. However, it does not allow capturing
the full finger movements due to the placement of the actuators in
the palm and other mechanical constraints. To overcome the constraints caused by the incorrect placement of the actuators in thepalm, Nakagawara et al. developed a new type of master hand [7].
The compact exoskeleton mechanism called “Circuitous Joint” introduced by Nakagawara et al. allow the operator to use his fingers
in a much wider workspace. However, these systems cannot compensate when the operator’s finger lengths are different and therefore it cannot produce an accurate kinesthetic sensation mapping
between the operator and robot hand finger movements.
With the development of a full upper body avatar robot called“TELESAR V” [4] Fernando et al. was able to extend the sensation of feeling the robot body as one’s own to the entire upper bodyand fingers. In this paper, human like finger posture reconstruction mechanism of “TELESAR V” system [4] was explained andthe advantage of human like finger posture to telexistence was discussed. By implementing a 14 DOF optical fiber based data glove
and a novel position based mapping algorithm can accurately capture the finger postures normalizing the effects of different lengthsof the user’s fingers. Remote side finger posture reconstruction wasachieved by constructing 15 DOF slave robot hand. Furthermore,
the effectiveness of this system with non-expert has been verified
through a set of experiments followed by a subjective evaluation.
2

F INGER P OSTURE C APTURE M ECHANISM

Extension/Flexion

Abduction/Adduction

ungrounded master is necessary where it can capture finger movements without any mechanical constraints.
Since the human hand dexterity is very high compared to any
other body part, it is necessary to capture the full range of movements from the operator side to reproduce the motion back on the
robot side. Fig. 2 shows common dexterity types associated withall fingers of human hand. As shown on Fig. 2(top left) the bending
towards is known as flexion and opening the fingers fully is known
as extension. Also as shown in Fig. 2(top right) all 5 fingers canbe bent in opposite axis and it is called adduction and abduction
respectively. However, human thumb has much dexterity than flexion/extension, adduction/abduction and one of the most common
dexterity that is necessary for manipulation is thumb opposition.
Apart from the above mentioned dexterity, thumb rotation (thumbroll) is also used when the thumb tip needs to contact with any othertip or the palm.
2.1 Optical Fiber Based Data Glove System
Existing data glove systems uses optical or resistive based bend
approximation. Resistive bending technologies used in “Cyber
Glove” [5] products have required sensing capabilities for most of
the fingers except for thumb opposition. However, they are very
expensive and also no after sales support. Thus, our approach was
to use an off-the-shelf data glove system and to modify it so thatthe opposition sensor can be added and repair can take place at any
place. In addition, resistive bending will have a hysteresis decays
over time and effectively reduce the accuracy of the sensors. Therefore, in this design, a data glove from “Fifth Dimension Technology(5DT)” 5DT-14 [17] was used. However this glove was also lacking of thumb opposition and thumb roll sensors. Thus, as shown on
Fig. 3 sensor 10 was added in order to detect the thumb opposition.
With a numerical calculation combining the effect of sensors 1, 2,
3, 10 the thumb roll was calculated. The detail of the calculation is
explained in a later section of the paper. Fig. 3 shows the 5DT-14
sensor placement after the custom sensors have been added in orderto detect the thumb opposition and thumb roll.

Thumb Opposition

Thumb Roll

Figure 2: Human Hand Flexion/Extension, Adduction/Abduction,
Thumb Opposition and Thumb Roll

Grasping is one of the most important feature in remote manipulation. Even though the robot can reach the target accurately, if
it is not possible to grasp objects with normal grasping techniques,operator will frustrate and try to use various techniques based ontrial-and-error until he finds the proper way to grasp. In order toreproduce the operator’s finger postures, it is necessary to capture
all finger movements. Using traditional exoskeleton [18] and endoskeleton [1, 7] master hands it is possible to capture the finger
movements accurately. However, limited dexterity provided by theexoskeleton systems will limit the operator to perform manipulations based on fingers as he would expect to be. Furthermore, depending on different finger lengths and the way of using fingers in amanipulation these unpredicted mechanical constraints would provide a frictional force to the operators’ fingers so that they would
not be able to use the fingers as they would like. Therefore, to grasp
objects correctly and feel the remote hand as your own, first an

Figure 3: Fiber Sensor Placement of TELESAR V Data Glove after
New Sensor Addition

According to human anatomy each finger was structured basedon separate sub bones and these are called phalanges. There are
14 phalanges in a hand where three for each finger, and two forthe thumb. The names of the phalanges of the three rows of finger bones, from the hand out are proximal, intermediate and distal
phalanges while the thumb only contains a proximal and distal phalanx. Fig. 3 blue color sensor shows the flexion type and red color
shows the abduction type sensor placement. The detected bend angle is calculated w.r.t to the points, which is similar to the sensor id
notation circle as shown in Fig. 3. Sensor 2 detects the thumb angle
between proximal and distal phalanx. Sensors 5, 8, 11, 14 detects
the 4 fingers intermediate and distal phalanx where as sensors 1, 4,
7, 13 detects the angle between palm to distal phalanx.

After some trials it was found that the angle between palm andring finger distal phalanx was not commonly used in manipulation.
Therefore, that sensor was moved from its original placement to thesensor 10 position as shown in Fig. 3 to detect the thumb opposition
and thumb roll. To mount the custom sensor, different types ofmaterials were tested to hold the sensor tube. However, the best
performance was given with black stretch lycra material due to theelasticity and compressibility. It was also holding the sensor tightly
to the operators finger and not to move around when bending.

Figure 4: Lycra based Stretchable Sensor Pocket for New Sensor

Fig. 4 shows the new sensor addition and the special pocket created with black stretch lycra to hold the sensor. The pocket was
sewed to the glove inside so that the sensor can be replaced easily
similar to all other sensors on the glove. Due to the limitations of
the A/D converter used in this circuit the maximum bend sensors
connected is limited up to 14.
RX
1-2.4V

TX
1.3V

S1

RubberInsulation

Optical FiberRX

USB 1.1

S14

TX

Hard Plastic
Fiber Guide

The last min, max values will be used for normalizing consecutive
incoming raw data unless a recalibration step is performed. The angle calculation is done at the kinematic generation step because thenormalized bend data is used in few other routines such as to checkthe correct calibration of bend sensors, no calibration detection, andfaulty bend sensors etc.
3

F INGER P OSTURE R ECONSTRUCTION M ECHANISM

To touch or grasp objects and mimic human hand finger movements, a higher DOF anthropomorphic robot hand similar to human
hand is necessary. Thumb, index finger and the abduction is mostly
used in grasping because they have the highest dexterity in human
hand compared to other fingers. When performing a remote taskwith tools the robot hand should be able to grab and use the tools
with similar dexterity to a human hand. In addition, it will enable
the user to feel that the hand that he sees is his own.
It is obvious that the human hand dexterity is very high compared
to existing robotic hands [3, 2, 6]. To control these hands in telexistence manipulations, exoskeleton [18] and endoskeleton [1, 7] master hands can be used without having to consider different finger
lengths. However, they are not be suitable due to the mechanicalconstraints and the reduced work space. In contrast, when using adata glove, depending on the users finger length, the angle calculation can be vary. This becomes complex when it is necessary tocapture all finger motion and especially the thumb motion. Thus,
it is essential to have a mapping algorithm so that different finger
lengths, hand shapes can be normalized and to be mapped to one
specific robot hand while giving the same sensation as the hand
you are controlling is your own.
3.1 15 DOF Anthropomorphic Robot Hand
Compared to the body and arm, it is very difficult to implement thesame dexterity in a robot hand due to the complexity of the mechanics and the smaller size. Thus, TELESAR V hand has mainly
focused on increased index, thumb finger dexterity while giving theability to control the abduction. As shown in Fig. 6 (Right), in
TELESAR V a custom designed 15 DOF human sized anthropomorphic robot hand was used. It’s thumb has 5 DOF, index finger
3, all other fingers 2 DOF, and the abduction. Fig. 6 (Left) shows
the modified light weight data glove that is easily wearable.

Figure 5: Optical Bend Sensor Internal Structure

Fig. 5 shows the internal structure of an individual bend sensor
used on the 5DT-14 Data Glove. At the tip of the sensor, a transmitter IR LED was used to emit the IR light rays. On the otherside of the sensor tube, a proprietary photo transistor was placed
to measure the intensity of the incoming IR light. When there is
no bending on the tube all the light emitted was captured throughthe photo transistor. When a bending occurs the intensity of IR lightwill reduce and a change in electrical signal ranging from 1V - 2.4V
will be generated. The digitized bend electrical signal is digitizedand sent to a PC via a USB connection.
On the PC side, to use the captured raw bending data several
signal conditioning were necessary. The raw data was normalized
at the fiber calibration step. During calibration, the raw data wascaptured and maximum and minimum bending values was calculated. Once the min, max raw data was known according to Eq. 1
the normalized bend values between 0 - 1 was further processed for
calculating the bend angle.
rawval − rawmin.Max(1)
rawmax − rawmin
After the user has been satisfied with the bend calibration he confirms the calibration and no further max, min values were captured.
bendnormalized =Figure 6: Master Glove vs. Slave Hand

Robot fingers are driven by 15 individual DC motors connectedwith dynamically coupled wires, and a pulley driven mechanism
couples the remaining joints that does not directly attach to a motor. Fig. 7 shows the robot hand coordinate frame assignment andthe placement of each joints w.r.t. each finger. As shown on Fig. 7,
robots proximal phalanx is independently working and the intermediate and distal phalanx is mechanically coupled on 4 fingers except
for thumb. This limits the user to move fingers with full flexibility.
However, when humans use their fingers in manipulation tasks thistwo joints are mostly working with coupled motion. Therefore, unless the operator is trying to pose for very specific finger gestures,
this mechanical coupling do not raise any issues during most of themanipulations.

Table 1: Joint Limits of 15 DOF Anthropomorphic Robot hand
D2

Joint

D8

D1
D5

D12

Drive

Mechanical limit

Electrical Limit

Negative

Positive

Negative

Positive

5◦

-120◦

0◦

J1

D12

-125◦

J2

D2

-70◦

2◦

-65◦

0◦

J3

D8

-2◦

47◦

0◦

45◦

J4

D1

-5◦

115◦

0◦

112◦

J5

D5

-45◦

90◦

-40◦

85◦

J6

-

-

-

-

-

J7

D4

-5◦

115◦

0◦

112◦

J8

D3

-45◦

90◦

-40◦

85◦

D4
D6

D3
D7

D13
D10
D14

D11
D15

Figure 7: 15 DOF Anthropomorphic Robot Hand Coordinate Frame
Assignment

Being the robot fingers proximal phalanx to be 25mm, intermediate phalanx to be 30mm, the four fingers have a 45mm long distal phalanx where as thumb distal phalanx is 42mm. Each finger is
20mm wide and thus approximately equal to a human finger. Due tothe two joints of each finger being mechanically coupled with a ratio of 0.7x, it was impossible to obtain a valid joint angle using numerical inverse kinematics. Thus, trigonometric and analytical inverse approach was used to obtain the inverse angle values. Thumb,index, ring and small fingers can move on the z-direction where as
middle finger was fixed. To simplify the problem, z-direction motion on all fingers were fixed and it was later added to the calculations. Eq. 2,3 shows the formula for getting the x,y coordinates
based on the current joint angle. Thus when the bending is provided
by the 5DT sensor, robot joint angles were calculated to reach thetarget fingertip position.

J9

-

-

-

-

-

J10

D6

-5◦

115◦

0◦

112◦

J11

D7

-45◦

90◦

-40◦

85◦

J12

D13

N/A

16◦

0◦

15◦

J13

D10

-5◦

115◦

0◦

112◦

J14

D11

-45◦

90◦

-40◦

85◦

J15

D14

-5◦

115◦

0◦

112◦

J16

D15

-45◦

90◦

-40◦

85◦

Fnx = 45 · cos(θ1.1 ) + 30 · cos(θ1.1 + θ1 ) + 25 · cos(θ1.1 + θ1 + θ2 )(2)

was necessary to implement motor control logic. Unlike mostrobotic applications, a closed loop control between a PC and robot
hardware was used. The most common method is to run PID
loops at hardware layer and control the motors on the hardware.
This method will have advantages over speed. However, dynamicchanges to the control algorithm cannot be done on PC side. Therefore it has to re-program on the hardware level every time when theparameters needed to change. Also to be able to use the system as
a learning tool, the challenge was taken by implementing a closed
loop control between a PC and the hardware Motor. This challenge
was tricky when windows based PC’s were used for all implementation as windows does not provide accurate timing which is requiredby robotics application. A shared memory based dynamic memory
mapping technique and low level C++ routines were used together
with Windows Multimedia Timer to overcome this issue.

Fny = 45 · sin(θ1.1 ) + 30 · sin(θ1.1 + θ1 ) + 25 · sin(θ1.1 + θ1 + θ2 )(3)As shown in Fig. 7 there are 16 Joints driven by 16 drives denoted through D1 - D16 where 2 joints (D9 - J9, D16 - J6) were
not used in the current configuration. Those joints being the palm
to distal phalanx on index and middle finger, it was very fragile andeasy to break. In addition, when in manipulation this palm to distal
joint was not used frequently. Thus, the joint was electrically fixed
at 0 deg. Therefore, the experiments carried out in the next sections
only uses 14 DOF’s out if 15 DOF’s. Table. 1 shows the complete
joint to drive mapping, joint limits of the hand, motor and joint assignment based on the kinematic configuration. Each finger’s motion limit angles are decided based on the maximum working areaof an ordinary human hand.
TELESAR V Hand uses all Brushless DC motors. Thus it

Current controlled PWM logic was used in order to control thecurrent of each motor. Potentiometer reading is sent to the PC as
16-bit value. Motor current is sensed at the motor driver chips using a hall-effect sensor. For communication between the PC andhardware a special communication system called “TexART NCord”
was used. It has both hardware and software C++ support to read
and write data between hardware and PC. The main processor usedin the hand was a FPGA (Xilinx Spartan - 6 XC6SLX75). Sensor data is updated at a frequency of 8KHz parallel for each motor. Each Terminal in FPGA communicates via 6 dedicated full
duplex RS485 buses at a speed of 20MBps. In order to translate the
RS485 signals to PC compatible format, a RS485 to PCI-Express
x1 converter board was used. Thus, when the system is operating
normally PC sends target information (motor current) at every 1ms
and retrieves the buffered position data at every 1ms. The time between each read/write data cycle is 500us. In our setup the RS485
to PCI-Express converter box is placed at the robots’ back.

3.2 Hand Hybrid Control Logic Block
Normalized Data Glove Angle vs. Ideal Angle Trajectory
0

i

i (t)

+

K c .i(t)-−20K c .i(t).e(t)

−30

+

-

t

+

K i e( ).d
0

Limit

Motor

Link

y (mm)e(t)+Ideal Trajectory
Modeled Trajectory−10+d.e(t)
Kd
dt

−40
−50
−60
−70

Figure 8: Control Logic Block Diagram for 15 DOF Hand

−80

Fig. 8 shows the control logic for Hand. The hand controller
outputs 15 channels of current, position and 8 channels of contact
force information. Current controlling for hand was very important.
For example during manipulation some operators will squeeze theirfingers, but if there is no proper current control it will break the objects due to uncontrolled high torque. Due to the higher number ofmotors per one terminal it was not possible to receive current information at the PC side and no feedback current controller was implemented. In order to avoid uncontrolled torque, a position based
feed forward current controller was implemented so that when the
joint is not moving it will not increase the torque. This technique
was successfully implemented for all joints and it was tuned in a
way that the first joint of each finger has the highest limiting torquewhere as 2, 3, 4 joints have much less torque limits. Since there wasno current feedback, if the limits were very low fingers will start tomove very slow. This is again unacceptable on a telexistence ystembecause the operator can easily understand the lagged fingers hesee is not his own. Therefore, first joint torque limit was increased
because to have a faster finger motion.
The very low torque limit on fingertip joints allows to steadily
touch the object surface, then it will stop at the same torque whichit contacted until the operator releases his fingers. Good contact
area between the fingertip and the object not only helps to accurate
grasp, but also to detect the contact force, tactile and thermal information. In TELESAR V system these haptic information was used
to reproduce the fingertip haptic sensation on the operators fingertip. Finally, in order to make finger pinch posture it is necessary
to move the fingertip joint in opposite direction. However, the data
glove does not detect the opposite finger bend. However, when theoperator tries to pinch fingers, robot hand will naturally bend its
finger tips due to very low toque limiting on last joints and higher
torque limits on the first joint. With this method the pinch finger
gestures were naturally achieved without modelling the opposite
direction bend.

−90
−60

−40

−20

0

20
x (mm)

40

60

80

100

Figure 9: Normalized Data Glove Angle vs. Ideal Angle Trajectory

Modeled joints vs. the ideal joints were plotted in order to understand the error of the modelling. As shown on Fig. 9 the modeled
trajectory has a very slight mismatch in modelling the data. However when used with many operators, the bend values read by thesame bend angle of the 2nd joint was different. After analyzing
the data it was found that the differences of finger lengths causes
this issue. An ordinary humans finger length can be vary depending on the ethnicity, sex and even the characteristics of the person.
This was quite a common occurrence in most of the operators andtherefore a solution was studied to overcome it.
The “Digit Ratio” is the ratio of the lengths of different digits
or fingers typically measured from the midpoint of bottom crease
where the finger joins the hand to the tip of the finger. In mostcases the 2nd (index finger) and 4th (ring finger) can be taken as
an index to derive a users other finger length and therefore the digit
ratio can be calculated by dividing the length of the index fingerof the right hand by the length of the ring finger. A longer index
finger will result in a ratio higher than 1, while a longer ring finger
will result in a ratio of less than 1. This ratio has a notation called“2D:4D” digit ratio. In general, women ring and index finger tend
to be about the same length where as in most men the index finger
is usually shorter.

3.3 Hand Trajectory Generation Algorithms
As explained in Eq. 2, 3 the normalized bend data is used to calculate the bend angle. As shown in Table. 1 the 2nd joint angle ofeach finger is limited to 0◦ - 112◦ and the last joint of each finger islimited to -40◦ - 85◦ . Computing forward kinematics on index finger, it was found that fingers can move in a trajectory similar to asshown in Fig. 9. Thus, it was impossible to get an inverse solution
with just position data. Therefore, trigonometric based analytical
Inverse approach was used to determine the joint angle. Knowing
the required bend range accuracy, the operator side bend angle wasmodeled based on the normalized bend values with the form of an
equation as shown on Eq. 4. In this approach, it was found that thenormalized bend values were not linear and thus the closest match
was “arcsin” curve.
Bendmodeled = A + arcsin(Bendnormalized ) · B(4)

Shorter

Equal

LongerFigure 10: Digit Ratio Pattern of Male and Female

Fig. 10 shows a typical digit ratio pattern of distribution among
male and female over various ethnic grouping. More over, the small
finger length and the thumb length is also vary depend on the users.
Thus it was necessary to calibrate the fibers and bend angle for eachfinger.
The 15 DOF hand in TELESAR V was made with the digit ratio
2D:4D of 1. However, out of the different operators the system wastested, no one was found to be of digit ratio of 1. To determine
which type of calibration is needed, robot grasping a cylindricalcoffee can with its 5 fingers were modeled. As shown on Fig. 11

the most important fingers for grasping objects are the index andthumb. The higher dexterity of these two allows to perfectly alignand grasp objects while the palm and other 3 fingers act as a support
to the grasping. Furthermore, middle, ring and small finger works
similar to the index finger in most people.

Index
Finger60°
30°
42°

Virtual
Circle

45°

30°

50°

Figure 12: Counting Examples

Thumb

Figure 11: Finger Digit Ratio Calibration Method

On the robot side, when index and thumb edges touches eachother and it creates an approximate circle as shown in Fig. 11. If
this condition is satisfied when the user makes an “OK Sign” with
his index and thumb finger it should be similar for other fingers.
The circle shape is decided because index, middle, ring and small
fingers cannot independently bend from the 3rd joint and it is coupled with the 2nd joint with an ratio of 0.7. Thus the robot handwill not be able to make any other shapes with the 4 fingers except
for the thumb. The best shape the robot hand can make is a circle
and therefore the counter part (thumb) has to make the other halfof the circle. It also intuitive to the user to ask to pose for the “OKsign” and it is known by many people.
Thus during the finger calibration process the operator was asked
to pose for “OK Sign” with all fingers, i.e first touch thumb andindex finger tips and make other 3 fingers co-aligned with the index
finger. At this point the system will take the normalized bend data
and decide the A and B parameters of the Eq. 4. This will be done
for all fingers and it gave the best calibration. One disadvantage
when using this method is that the user has to create a near equal
circle.
4

E XPERIMENTS AND E VALUATION

4.1 Evaluation of Robot Finger Posture Representation
Accuracy
To experience one’s own hand in telexistence manipulation it is necessary to capture the same finger posture from the operator and reconstruct in an anthropomorphic robot hand. In the previous sections using an optical fiber based flexible master glove and a robot
hand the reproduction of the same finger posture was discussed.
However, based on different users the effectiveness of the normalization algorithm was unknown. In order to find out the effectiveness of the finger-mapping algorithm on multiple users hand, random grasping tests with 4 users were carried out.
At first, the users were briefed about the nature of the manipulation and proceeded to the finger calibration by asking them to posefor the “OK Sign” with all fingers. Once the calibration is verified on a simulated environment they were connected to the slave
robot. As the first verification step, they were asked to check thethumb-index finger pinch and thumb-middle finger pinch. Some
users initial trial was not accurate as the calibration was not successfully performed. Therefore, the finger calibration step was repeated while connected to the robot until the correct pinching wasachieved. Approximately with 3 calibrations users were able topose for the gestures of pinching thumb-index and thumb-middle
fingers.
Fig. 12 shows 9 counting examples that the operator hand was
posing and the resultant slave representation. Each example shows
the master posture on to the right and the slave posture on to the left
side. As can be seen from the Fig. 12, the slave hand and the masterglove is capable of successfully showing the counting sequences byfingers. Furthermore, specific gestures such as “Peace” in Japan and“Rock” gesture in states, fist gestures can be performed. Mimicking
counting gestures on a remote slave robot while correcting throughvisual feedback was possible even in robots with pre-programmed
trajectories. However, the ability to perform these gestures in a repeated and dynamic manner without visual feedback is very helpful
in telexistence manipulations for building awareness of the body.

Figure 13: Fingertip-fingertip Pinching and Rolling Examples

Next, with the same calibration user was asked to do thumb toother finger pinch gesture. As shown in Fig. 13 finger-finger pinch
can be done with finger combinations thumb-index, thumb-middle
and even thumb-ring fingers. thumb-small finger pinch was notpossible as the electrical limit of J1 does not permit to reach thetarget position. However, thumb-small finger pinch usage is veryrare even by bare hands. For each finger combination total of 10
repetitions was performed.
During the previous experiment it is also found that the operator
can even roll their fingers over from index finger to ring finger. This
will allow the operator to roll objects on fingertips. To test the finger
rolling a small rubber ball with a diameter of 15mm was used asshown in Fig. 14. Operator was able to do 2 finger, 3 finger rollwith the 15mm rubber ball and even roll the ball from index finger
to middle finger.
As shown in the above examples to perform these actions, theoperator does not have to be taught. He naturally understands thatwhat fingers to be used, what type of finger movements needed inorder to perform the required task and continue with the grasping4.2.2 Experimental Procedure

4.2.1 Experimental Setup
In the previous experiment the subjects were using visual feedbackin the manipulation. However, most humans can grasp simple objects with eyes closed. A preliminary study was conducted with 10
subjects and a cylinder with diameter 65mm and height 125mm wasplaced in front of them at the reaching distance. They were asked
to grasp the cylinder with eyes open repeated with eyes closed. All
the subjects were able to grasp the cylinder even with eyes closed.
When a visual cue is given they imagine the size, distance information and then with eyes closed they think how the fingers should
be moved to grab that object. Similarly, if the fingers are mapped
accurately and the visual feedback has the same characteristics as
human, grasping might be possible in telexistence manipulations
without visual feedback.

Static Head / Body

Close up Posture

reach
grasp

reach
grasp

3

3

2

1

0

Any Posture
reach
grasp
3

Success Trials

4.2 Accuracy of Reaching and Grasping in Teleoperation without the Visual Feedback

Success Trials

tasks as he wish. Furthermore, the ability of mimicking one’s own
finger motion and seeing it through the robot’s eye will allow theoperator to have the correct eye-hand coordination and also eyefinger coordination over time. Furthermore, having able to manipulate the remote fingers with similar dexterity compared to one’s own
fingers helped the operator to perform manipulations much easily.

Success Trials

Figure 14: Two Finger, Three Finger Grasping and Rolling Examples

The subjects were connected with the slave robot with head, body,right arm and hand only. Once connection was established, they
were asked to check the finger calibration accuracy by touching the
thumb-index and thumb-middle fingers and check the corresponding robot posture. Finger calibration was repeated (max 3 times)
until the correct posture is mapped. As shown on Fig. 15 a redcylinder with diameter 65mm and height 125mm was placed exactly 500mm apart from the robots body, and 200mm down the eyelevel. A similar size white cylinder was placed on the users side.
The experiment was carried out with three conditions. First condition was to confirm the spatial position of the two cylinders arecorrect and for that the subjects were asked to grasp the cylinder
with visual cue. In the first condition subjects were restricted forgoing closer, or back as well as sides. The visual cue was confirmedwith the display shown in Fig. 15 and a black screen was provided
when visual cue is disabled. After confirming the position, the subject side cylinder was removed and the step was repeated until they
could successfully reach, grasp and lift the object up from theirright hand. Next, subjects were asked to position their head (roll,
pitch, yaw) so that they can see the whole object clearly and in a
comfortable manner. Next, they were asked to place their left handand virtually hold the remote object and while holding the vision
was cut-off (provided a black screen) and asked to grab and lift theobject and wait until the vision is back. The step was carried out
with 3 trials.

2

1

1 2 3 4 5 6 7 8 9Subject

02

1

1 2 3 4 5 6 7 8 9Subject

0

1 2 3 4 5 6 7 8 9Subject

Figure 16: Reaching Success vs. Grasping Success(left) No body dexterity, (mid) front/back dexterity, (right) full
upper body dexterity

Figure 15: Experimental Setup

To evaluate this, an experiment with 9 subjects (6 male, 3 female), first time users was conducted. However this experiment is
not meant for evaluating the perception accuracy error grasping error, but just a measure of if the system capable of delivering theexperience to the first time users so that they can reach and grasp
an objects. The average age was 26 (SD = 2.23) and height was
169cm (SD = 9.7) where all the subjects have used 3D displays before such as 3D TV, 3D Cinema etc. However, only 7 subjects have
had used HMD’s. Three of them have controlled robots in the pastand 8 of them was trying TELESAR V system for the first time.
The remaining participant had one time experience 1 year before.
The average reaching speed was around 10cms−1 .

In next conditions, the same steps were carried out. But they
were only allowed to move their body in x-axis. i.e going closer or
far apart and 3 trials were carried out. Finally, in the third condition the subjects were not restricted and were asked to go in to any
posture and perform the same steps. The experiment always started
with condition 1, but the order of condition 2, 3 was randomized
for each subject.
4.2.3 Experimental Results
As shown in Fig. 16(left) the experiment results when movement
freedom limited to only head. The reaching and grasping successaccuracy was found as 89% and 63% respectively. Fig. 16(middle)
shows when the subjects were given freedom in head, and spinal
movements (i.e closeup/far-apart). The reaching and grasping success accuracy was found as 96% and 74% respectively. As shownas in Fig. 16(right), when full upper body freedom was given thereaching and grasping success accuracy was found as 93% and 63%
respectively. Furthermore, the Condition 2 and 3 order was randomized for each users.

Overall result of 9 subjects showed that when the users were
not allowed to move their body the reaching accuracy was less andit was increased when they were given the freedom. This was because, the subjects had different arm lengths and they were not comfortable when they were not given any freedom. Furthermore, it wasobserved that, when they were given full freedom, they naturally
adjust their body as they wish to comfortably perform the manipulation action. When subjects reach closer, the grasping accuracy
was increased. However, when they were given full body motion
freedom some subjects were unable to perceive the size of the object because they were too close. With that it was confirmed thatthe systems accuracy is optimal when the eye-to-object distance isbetween 400mm - 500mm and the full body motion was naturally
used by all subjects. The results can be concluded that there is atrend that the grasping accuracy was increased when the users were
given more freedom. It was further evidenced that, even though
there was an accurate reaching there might be differences in the fingers and could not accurately grasp. The results showed that byallowing the subject to use the body motion without any constraints
resulted in deciding accurate perception distance and the size of theobject. This was evidenced due to the success rate of grasping withno visual feedback. Therefore, the results can be concluded that if
the fingers are mapped accurately and the visual feedback has thesame characteristics as human, humans can grasp remote objects
without visual feedback in telexistence manipulations.
5

C ONCLUSION

This paper describes a new type of robust control mechanism fora 15 DOF anthropomorphic robot hand in telexistence manipulations using a flexible fiber based master glove to experience thevisual-kinesthetic sensation of one’s own hand in remote manipulations. To overcome the work plane constraints and the frictional
component of exoskeleton and endoskeleton type master hands, a
14 DOF optical data glove based master hand was developed. Due
to very low hysteresis errors, easy replacement/modifications andthe higher dexterity of the sensor system allow the capture system
to correctly model the thumb and other fingers and therefore the fullhand posture reconstruction was achieved. The evaluation showedthat the novel finger length and bending normalization calibration
algorithm shows trends that it can be effectively used by even firsttime subjects for object manipulations without visual feedback intelexistence. By mapping the human finger motion to a 15 DOF
anthropomorphic robot hand accurately and providing visual feedback with the same characteristics as human, subjects were able toquickly adopt to remote robot hand as they were using their own
hand for remote manipulation. In the future a follow up experiment should be conducted and if the trend can be generalized, themethod proposed by this paper can be used in telexistence manipulations where the users can adapt to teleoperations reaching andgrasping mush faster through experiencing the remote hand as theirown.
ACKNOWLEDGEMENTS
This project is supported by JST-CREST Haptic Media Project.
R EFERENCES[1] M. Bouzit, G. Burdea, G. Popescu, and R. Boian. The rutgers master ii-new design force-feedback glove. Mechatronics, IEEE/ASME
Transactions, 7(2):256–263.

[2] L. B. Bridgwater, C. A. Ihrke, M. A. Diftler, M. E. Abdallah, N. A.
Radford, J. M. Rogers, S. Yayathi, R. S. Askew, and D. M. Linn.
The robonaut 2 hand - designed to do work with tools. In Proceedings of IEEE International Conference on Robotics and Automation,
ICRA’12, pages 3425 –3430, may 2012.
[3] A. Caffaz and G. Cannata. The design and development of the disthand dextrous gripper. In Proceedings of IEEE International Conference on Robotics and Automation, ICRA’98, volume 3, pages 2075
–2080 vol.3, may 1998.
[4] C. L. Fernando, M. Furukawa, T. Kurogi, S. Kamuro, K. Minamizawa,S. Tachi, et al. Design of telesar v for transferring bodily consciousness in telexistence. In Intelligent Robots and Systems (IROS), 2012
IEEE/RSJ International Conference, pages 5112–5118. IEEE, 2012.
[5] C. S. Inc. Cyberglove iii, next generation data glove technology, Jan.
2010.
[6] M. Jagersand. Hierarchical uncalibrated predictive display for a 16 dof
utah/mit hand. In Proceedings of IEEE/RSJ International Conferenceon Intelligent Robots and Systems, IROS ’99, volume 1, pages 124–129 vol.1, 1999.
[7] S. Nakagawara, H. Kajimoto, N. Kawakami, S. Tachi, and
I. Kawabuchi. An encounter-type multi-fingered master hand using
circuitous joints. In Robotics and Automation, 2005. ICRA 2005.
Proceedings of the 2005 IEEE International Conference, pages 2667–
2672. IEEE, 2005.
[8] S. Tachi. Telexistence. World Scientific, 2010.
[9] S. Tachi and H. Arai. Design and evaluation of a visual display witha sensation of presence in tele-existence system. Journal of Robotics
and Mechatronics, 9(3):220–230, 1997.
[10] S. Tachi, H. Arai, and T. Maeda. Development of an anthropomorphic
tele-existence slave robot. In Proceedings of International Conference
on Advanced Mechatronics, ICAM’89, pages 385–390, may 1989.
[11] S. Tachi, H. Arai, and T. Maeda. Tele-existence master-slave system
for remote manipulation. In Proceedings of 1990 IEEE International
Workshop on Intelligent Robots and Systems. ’Towards a New Frontierof Applications’, volume 1, pages 343–348, jul 1990.
[12] S. Tachi, N. Kawakami, M. Inami, and Y. Zaitsu. Mutual telexistence system using retro-reflective projection technology. International Journal of Humanoid Robotics, 1(1):45–64, 2004.
[13] S. Tachi, N. Kawakami, H. Nii, K. Watanabe, and K. Minamizawa.
Telesarphone: Mutual telexistence master-slave communication system based on retroreflective projection technology. SICE Journal ofControl, Measurement, and System Integration, 1(5):335–344, 2008.
[14] S. Tachi, K. Minamizawa, M. Furukawa, and C. L. Fernando. Telexistencefrom 1980 to 2012. In Proceedings of IEEE/RSJ International
Conference on Intelligent Robots and Systems, IROS ’12, pages 5440–
5441. IEEE, 2012.
[15] S. Tachi, K. Tanie, K. Komoriya, and M. Kaneko. Tele-existence (i):Design and evaluation of a visual display with sensation of presence.
In Theory and Practice of Robots and Manipulators, volume 1, pages
245–254. Springer, june 1985.
[16] S. Tachi and K. Yasuda. Evaluation experiments of a telexistence manipulation system. Presence, 3(1):35 –44, feb 1994.
[17] F. D. Technologies. 5dt-14dg, 5dt data glove 14 ultra, Nov. 2011.
[18] Z. Zhou, H. Wan, S. Gao, and Q. Peng. A realistic force rendering algorithm for cybergrasp. In Computer Aided Design and Computer Graphics, 2005. Ninth International Conference on, pages 6–pp.
IEEE, 2005.