2009 IEEE International Conference on Robotics and Automation
Kobe International Conference Center
Kobe, Japan, May 12-17, 2009

Development of Grip-type Master Hand “MeisterGRIP”
Katsunari Sato, Shuji Komeiji, Naoki Kawakami, and Susumu Tachi, The University of Tokyo

Abstract—We propose a novel grip-type master hand called
MeisterGRIP that measures grip force in terms of a force
vector distribution. This device is expected to allow intuitive
robot manipulation using vision-based haptic-sensing
technology. Furthermore, it can be used for general-purpose
manipulation and is tolerant to individual differences in handsize and grasping posture. We constructed MeisterGRIP and
evaluated the accuracy of the measured grip force.
Furthermore, we constructed and exhibited a complete robot
manipulation system using MeisterGRIP to demonstrate thepossibility of using MeisterGRIP as a general-purpose master
hand.

T

I. INTRODUCTION

ELEXISTENCE or telepresence is an advanced form of
teleoperation that enables humans to remotely performtasks with dexterity, providing the operator with the illusionof being actually present at the remote location [1]. This
technology finds application not only in expert fields, butalso in communication and entertainment for the generalpublic. However, previous telexistence technologies weremainly developed for expert applications such as in the daVinci surgical system [2]. Nevertheless, in the future,telexistence technology will be more common and will beconsidered a general-purpose technology. For example, itwill allow us to shake hands with friends living in a different
country or to climb a mountain—to actually touch the rocks
and plants—while sitting in the living room.
To realize a telexistence system that can be used by a lay
individual, we believe that a simple and universal control
system employing a robotic hand (master hand) should be
developed. The master hand is important when interactingwith a remote environment because it allows the generationof realistic sensations. We believe that there are threerequirements for the general-purpose master hand. First, it
should be easy to use, i.e., it should function intuitively.
Second, the system should be usable by a wide cross sectionof individuals. To be accepted by general households,
anyone—men, women, and children—should be able to use
the master hand easily. Finally, the system should enablerobotic hands to grasp fragile objects gently.
A short survey of conventional master hand systems

Katsunari Sato, Shuji Komeiji, Naoki Kawakami, and Susumu Tachi areaffiliated with the Graduate School of Information and Technology, The
University of Tokyo, Japan; e-mail: (Katsunari_Sato, Shuji_
Komeiji)@ipc.i.u- tokyo.ac.jp, (kawakami, tachi)@star.t.u-tokyo.ac.jp.
This work was supported by Grant-in-Aid for SCOPE and JSPS Fellows.

978-1-4244-2789-5/09/$25.00 ©2009 IEEE

3283

available today reveals that few systems, if any, satisfy theabovementioned requirements. Master hand systems thatallow another fully robotic hand to mimic the operator’s
movements have been developed. These master handsystems measure the operator’s movements directly by using
a wearable device [3] or indirectly by using optical ormagnetic transducers [4] [5]; both methods enable intuitiveand correct robot manipulation. However, both methods
require the use of additional attachments to control thegrasping force. Other types of master hands have beendeveloped that use efferent neuronal signals such as
myoelectric signals [6]. Because vision is superior tosomatic sensation, even if their body does not actually move,
operators will feel as though they themselves have moved,
when they receive visual feedback that the robotic hand has
moved. Therefore, this type of master hand also enables
intuitive manipulation and can be rendered into a compact
system. Furthermore, it allows the control of grasping force.
However, myoelectric signals cannot be easily detected withhigh accuracy using such systems, which can also be
troublesome to wear.
We believe that a master hand should be developed suchthat anyone can use it intuitively and easily. In addition, we
aim to realize gentle grasping by enabling the control ofgrasping force. In order to satisfy these requirements, wehave developed a novel master hand called “MeisterGRIP.”
This device measures the operator’s grip force and uses thisinformation to control the robotic hand (Figure 1.) For thispurpose, the operator is simply required to grasp the device;
therefore, the complexity of the setup is reduced, and there
are fewer restrictions on the operator. Furthermore, byreflecting the grasping force of the operator directly in the
robotic hand, intuitive manipulation is made possible. In thispaper, we introduce MeisterGRIP and describe anevaluation of the accuracy of the measured grip force.

Fig. 1. Principle underlying grip-type master handII. GRIP-TYPE MASTER HAND “MEISTERGRIP”
MeisterGRIP measures the operator’s grip force and uses
it to control the robotic hand. In order to measure the grip
force, we used the three-dimensional force vector
distribution created by the act of grasping. Using the force
vector distribution, we can relate the grip force tocomplicated finger movement. When an operator grasps our
proposed device, the device recognizes the five fingers of
the operator as five input positions on the basis of the force
applied to the device by each finger. Then, the force vectors
resolved to the three axes are measured at each input point
and relayed to each finger of the robotic hand. The input
points could be at any position on the device surface; thisallows the operator to grasp the device at any position and inany posture. The individual differences among the handsizes are also compensated by this feature. Furthermore, thegrasping force of the robotic hand can be adjusted bycontrolling the robotic fingers by the force of operator’sfingers.
In order to measure the force vector distribution, we used
our vision-based haptic-sensing technology [7]. The haptic
sensor consists of a transparent elastic body with two layersof a marker matrix and a color camera. When a force isapplied to the surface of the sensor, the markers move. The
movement of the markers is captured by the camera. We
determine the location of the center of the markers before
and after the application of force. Therefore, we can record
the movements of markers and calculate the applied force
using equation (1).

(

)

−1

(1)
f = H T H H Tu .
In this equation, u, f, and H denote the two-dimensional
movement vectors of the red and green markers as capturedby the camera, the three-dimensional force vector, and theconversion matrix, respectively. The elements of H are
calculated by using elastic theory [8].
It should be noted that we make two assumptions to
calculate the force vector field from the movements of themarkers: (a) the sensor is assumed to be a semi-infinite
elastic body; and (b) the deformation of the elastic body is
assumed to be linear.
III. CONFIGURATION OF MEISTERGRIPTo measure grip force, the device should be easy to grasp
and should be such that the thumb faces the other fingers
while grapsing. For this, we considered a grip-type(cylindrical) device to be most suitable and developed acylindrical vision-based haptic sensor called MeisterGRIP.
Figure 2 shows the configuration of MeisterGRIP. We
wrap an elastic body (resin replicating the properties ofhuman skin: Hitohada gel, EXSEAL Inc.) around atransparent acrylic pipe. The surface of the elastic body iscovered with black silicon rubber (KE-109, Shin-Etsu

3284

Chemical Co.). Two layers of colored markers are also on
the surface. The cylindrical body makes it difficult tocapture the markers on camera. It would be difficult to place
the cameras inside the cylindrical device because of theirsize. Further, if we use wide-angle cameras to track themarkers, adjusting the focus becomes difficult. Therefore,
we use hemispherical mirrors like those used in 360°
panoramic cameras. Two cameras (Firefly MV, ViewPLUS
Inc.) are attached at the two ends of the cylindrical body, andthe hemispherical mirrors are attached at the center to track
the markers effectively.
A fully constructed MeisterGRIP is shown in Figure 2.
The device is designed to correspond to the hand of theaverage adult man: its length is 140 mm; diameter, 80 mm.
An aluminum frame holds the elastic body, cameras, andmirrors in place. The diameter of the mirrors is 30 mm.
Furthermore, to illuminate the red and green markers, we
attach LED arrays to both sides of the cylindrical body. To
capture the markers clearly, their diameters are set to 1.5 mm.
Moreover, in order to prevent the overlapping of the red andgreen markers, they are separated by a distance of 4.0 mm.
The red and green markers are placed at depths of 1.0 and
3.0 mm, respectively, from the surface of the device.
IV. CALCULATION OF GRIP FORCE
Because of the cylindrical shape of MeisterGRIP, we use
different algorithms that are employed in conventional
vision-based haptic sensors to calculate the force vector
distribution. In this section, we introduce the alogorithms
used in MeisterGRIP for calculating the grip force.
A. Summary of Algorithm
The algorithm used to calculate the force vector
distribution is shown in Figure 3. The algorithm consists of
two separate steps: initialization and force calculation.
During initialization, the initial position of the markers is
captured. Then, finger positions used for grasping are
identified. During force calculation, the force vector
distribution at the finger positions is determined on the basisof marker movement. In both steps, the marker position
should be captured and converted approximately.
transparent elastic bodywith colored markers

CCD
cameraCCD
camerahemisphericalmirrors

Fig. 2. Configuration of MeisterGRIP (right: profile, left: outer appearance)

Initialization

Calculation

Capture and conversionof marker position

Capture and conversionof marker position

Grasping

Capture marker movement

Capture and conversionof marker position

Calculate force vectors

Capture marker movement
Recognize finger position

Fig. 3. Algorithm used to calculate force vectors

B. Capture and Conversion of Marker Position
Capture and approximate conversion of the marker
position involves three steps: (a) the camera images should
be captured and binarized; (b) the marker positions should
be converted; and (c) the movement of the center of themarkers should be calculated.
First, the captured color images are converted to grayscale
images with red or green elements. Then, each grayscale
image is binarized in order to capture the marker position. In
the captured images, the luminance of the markers differs
according to the marker position. Therefore, the image isbinarized by using an adaptive thresholding algorithm. The
adaptive thresholding algorithm binarize the image bychanging the threshold dynamically over the image: thethreshold of a pixel is the average luminance of thesurrounding pixels.
Of the abovementioned initial steps, the second one is themost important for MeisterGRIP. The movement of thecenter of the markers should be calculated on a flat surface.
However, the image recorded the reflection of the markers in
the mirrors. Therefore, to the marker positions obtained
from the images should be approximately converted to thosethat would be captured on a flat surface.
Figure 4 shows the image captured by the camera and thecross-sectional diagram of MeisterGRIP. We can calculate
the approximate marker position (X, Y, Z) from the marker
position in the captured image (ui, vi) using equation (2).
X = Zφ(2)
Y = y t − Z / tan ϑ
Z = R − DRIn equation (2), yt, θ, and φ are calculated from ui and vi
using equation (3).
y t = y m + z m / tan θym =(u i − u c )2 + (vi − v c )2zm =rm − y m22

θ = α + 2β
α = arctan (z m / y m )β = arctan ( z m / (l − y m ))φ = arctan (v i / u i )(3)Fig. 4. Captured image and cross-sectional diagram used for determining
the marker position. Q(ui, vi) denotes the marker position in the captured
image; C(uc, vc), the center of the mirror; rm, the radius of the mirror; P(X, Y,
Z), the approximated marker position; R, the radius of MeisterGRIP; DR, thedepth of the marker from the surface; and l, the distance between the mirror
and camera lens.

Using these equations, we can convert the marker positions
obtained form the captured image to those that would havebeen obtained on a flat surface. It should be noted that we
use two cameras to capture the marker positions. Therefore,
we converted and combined the marker positions to a flat
surface.
Using the approximated marker positions, we calculatemarker movement. To calculate the movement, we calculate
the difference between the current and the initial positions ofmarker. The marker position is calculated from its center ofthe gravity.
C. Recognition of Finger Position
Recognition of the finger position is also important for
MeisterGRIP. The system supports all hand sizes, grasping
postures, and positions. Finger position is recognized when
the sensor is first grasped and remains unchanged even when
the operator releases MeisterGRIP.
To recognize the finger position when an operator grasps
MeisterGRIP, we use the movement of the center of themarkers. When MeisterGRIP is grasped, the markers aroundthe five fingers move more than the other markers.
Therefore, we can estimate the finger position from thecenter of gravity of the absolute value of the movement of
the markers.
D. Calculation of Grip Force
We calculate the grip force at each input point from theforce vector distribution. In conventional vision-based
haptic sensors, the force vector distribution over the wholesurface of the sensor is calculated from all the marker

3285

movements. In contrast, in MeisterGRIP, because weapproximate the camera image, using all marker movements
can introduce errors in the calculation. Furthermore,calculating the force vector distribution over the wholesurface can prove to be wasteful. We believe that it isenough to calculate the force vector distribution over thecontact field between the fingertips and the surface of thedevice.
Therefore, in MeisterGRIP, we calculate the force vectors
at nine (three times three) calculation points around the input
points. The vision-based haptic sensor calculates the mostaccurate force vector corresponding to the force applied at
the measurement point. Therefore, calculating the force
vector at the input points improves the accuracy of themeasurement. The distance between the calculation points is
5.0 mm. The markers used in this calculation are within the
15.0 mm from the input points. After the calculation of theforce vectors, we sum all the calculated force vectors. We
use this value as the grip force of each finger (Figure 5). In
the calculation loop, this algorithm consumes only 30 msec
on a desktop computer (2.8 GHz, Intel Pentium D
processor).
V. EVALUATION
It telexistence, the master hand should be able to remotely
perform grasping as if it were an operator’s hand. Therefore,
MeisterGRIP should be capable of measuring the grip forceof each finger correctly and independently. We evaluatedthe accuracy of the measurement of three-dimensional grip
force at each finger and whether the forces are independentof each other.
A. Experimental Setup
The experiment consisted of two pars; evaluation of
accuracy and verification of the independence of forces. In
both the experiments, we applied an arbitrary force to thesurface of MeisterGRIP by using a six-axis force sensor (BLAUTOTEC Inc.). Furthermore, we recorded the calculated
force vector and the output of the six-axis force sensor inorder to compare them. The experimental setup is shown in
Figure 6. We used XYZ-stage (CHUO PRECISION
INDUSTRIAL Co.) to apply the force at the end of the
six-axis force sensor. The diameter of the contact surface ofthe six-axis force sensor was 10 mm.
In MeisterGRIP, to calculate the grip force, markers
reflected by the mirrors were captured. Then, the captured
marker positions were approximated to those that would
have been obtained on a flat surface. This approximation
might cause calculation error in the y and z directions. Thiserror might be particularly prominent at the center and edgeof the device. Therefore, we evaluated the grip force at threedifferent measurement points A, B, and C on the surface(Figure 6). A, B, and C were at 70 mm, 45 mm, and 20 mm,
respectively, from one end of the cylindrical body.

3286

Fig. 5. Measured grip force. Green lines and red arrows represent thecylindrical shape of the device and the grip forces, respectively.
(a) points thatforce is applied
6-axis force
sensor
x

y

x

y

C

6-axis force
sensorBAzz
140 mmMeisterGRIP
xyz-stage(b) points thatforce is measured
C

B

A

Fig. 6. Experimental setup. Blue dots and green crosses represent points
where the force is applied and is measured, respectively.

In order to evaluate the accuracy of the grip force, we
evaluated the linearity of the grip force in the three axes. We
applied force at the input position while changing its
magnitude and direction. We applied force at three points
and evaluated the accuracy of the grip force. Prior to theevaluation, we normalized all the measured force by theaverage measured force at point B for calibration purposes.
In order to verify the independence of the forces, we applieda z-directional force to a point and measured the force vector
at three points: the point at which the force is applied andaround it. Then, we evaluated the influence of the applied
force on points where the force is not applied.
B. Accuracy of Measured Grip Force
We applied the x- and y-directional forces in the range 0 to
500 gf at 100 gf intervals along with a 1000-gf z-directional
force. Then, we applied a z-directional force in the range 0 to1000 gf at 100 gf intervals. In each trial, we measured theforce five times and evaluated the average of themeasurements.
Figure 7, 8, and 9 show the linearity of the measured force
along each axis. In these figures, the horizontal and verticalaxes represent the applied force and measured force
normalized by the maximum value, respectively. The blue
argyles, the green squares, and the red triangles represent the
average of the measured force when we applied force to thepoint A, B, and C, respectively. The bars represent thestandard error.

the applied force and the measured force normalized using
the maximum value, respectively. The blue argyles, green
squares, and red triangles represent the average of themeasured force at point A, B, and C, respectively. The bars
represent the standard error.
When the distance of the nearest measurement point fromthe applied force is more than 10.0 mm, the normalizedmeasured force is less than 40 % of that obtained at theapplied force. This suggests that the grip force can be
independently measured when the input points are separated
by more than 10.0 mm from the applied force.

A: 70 mm
B: 45 mm

600

C: 20 mm

400
200
0

-200
0

100

200
300
applied force Fx (gf)

400

500

Fig. 7. Linearity of measured force Fx.

measured force Fy (gf)800

A: 70 mm
B: 45 mm

600

C: 20 mm

400
200
0

-200
0

100

200
300
applied force Fy (gf)

400

500

800

1000

Fig. 8. Linearity of measured force Fy.

measured force Fz (gf)1750

A: 70 mm

1500
1250

B: 45 mm
C: 20 mm

1000
750
500
250
0
-250
0

200

400
600
applied force Fz (gf)Fig. 9. Linearity of measured force Fz.

The results show that the measured grip force is linear inall directions. The resolved forces in the x-y direction andthe z-direction are approximately 30 gf and about 70 gf,
respectively. The magnitude of the force measured at
different points differs.
C. Independence of Force Vectors
We chose three measurement points in the y-direction andapplied 300-, 600-, and 900-gf z-directional forces at thesepoints. The distance between the applied force and theclosest measurement point was in the range 5.0 to 15.0 mm
at 2.5 mm interval. We normalized the measured force byusing the measured force at force applied point.
Figure 10 shows the independence of the measured force
at each point. In the figure, the horizontal and vertical axes
represent the distance of the nearest measurement point from

3287

D. Discussion
In the experiment investigating the accuracy of themeasured grip force, the linearity of MeisterGRIP was
confirmed. Furthermore, the resolved forces wereconsidered sufficient for the analog control of robotic thehand to grasp an object. However, the magnitude of themeasured force differs among different measurement points:
the measured force at A is smaller than that at B, and themeasured force at C is larger than that at B. It is possible thatthe approximation of marker positions caused some errors.
In MeisterGRIP, we capture the reflected marker image inhemispherical mirrors. Thus, the converted marker
movements in the y-direction were not accurate. Therefore,
the conversion matrix in equation (1) should be modified so
as to reduce the influence of marker movements on thecalculation.
The results of our experiments also revealed that the force
vectors are independent even when the distance between thetwo input points is 10.0 mm, which is corresponds to the sizeof the fingertip of an average adult man. Furthermore, theindependence of the force vectors might be improved byincreasing the density of markers because the independenceo the force vectors is influenced by the spatial resolution of
the measured force vector distribution. Therefore, we canconclude that MeisterGRIP is capable of measuring the grip
force independently regardless of the operator’ssize/posture.
normalized measured force

measured force Fx (gf)800

A: 70 mm1

B: 45 mm0.8

C: 20 mm

0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-15

-10
-5
0
5
10
15
distance of the nearest measurement point fromthe applied force (mm)Fig. 10. Independence of grip force.

VI. ROBOTIC HAND CONTROL
Using MeisterGRIP, we constructed a manipulation
system comprising robotic hands and arms (cockpit). The
configuration of the cockpit is shown in Figure 11. We usedelectrically actuated humanoid robotic hands andair-actuated seven degree of freedom (DOF) robotic arms.
We used the grip force measured by MeisterGRIP to control
the five-fingered robotic hands. To control the robotic hands,
the measured force information was used to calculate thevelocity of movement of the robotic fingers. Therefore, thestronger the grasps of the operator on MeisterGRIP, thefaster is the movement of the robotic fingers. Each robotic
finger has one DOF, so we used only the magnitude of theforce vectors to control each finger. We attached
MeisterGRIP to the master arms, which measure themovement of MeisterGRIP (DOF: 2). When the operator
grasps MeisterGRIP and moves his/her hand, thepotentiometer attached to the master arm measures the hand
movement. This information is relayed to the robotic hand,which moves accordingly from right to left or up and down.
Furthermore, the torque applied by the operator is measured
by MeisterGRIP and mimicked by the robotic hands throughyaw-axis rotation. To control the position and posture of therobotic hands, we control the robotic arm. This isaccomplished by using a PID algorithm.
We exhibited the constructed cockpit at SIGGRAPH
2008 New Tech Demo [9] and interactive-Tokyo 2008 [10].
Through these exhibitions, more than one thousand people
experienced the ability to control robotic hands and arms
using MeisterGRIP. At the exhibitions, visitors were asked
to grasp and pick up balloon animals, and about half thevolunteers, regardless of sex and hand size, could
accomplish this task within a few minutes. Furthermore,some of these volunteers commented on the ease of robotic
control allowed by MeisterGRIP. Therefore, we canconsider MeisterGRIP to be a general-purpose master hand.
However, some of the volunteers had suggested thatreceiving a feedback indicating contact with the object could
assist in accomplishing the task of manipulating the object
more effectively. In MeisterGRIP, operators always receivea reactive force when grasping the device. Thus, operators
can feel force feedback and grasp an object stably. However,they cannot perceive the surface of the object, or thesensation of touching with their fingers. Therefore,
MeisterGRIP should be equipped with a tactile display using
vibration or electro-stimulus. Attaching an electrotactile
display [11] to the MeisterGRIP might assist in realizing
touch feedback.
VII. CONCLUSIONS AND FUTURE WORK
We constructed a novel grip-type master hand called
MeisterGRIP. We evaluated the accuracy of the measured
grip force and constructed a robot manipulation system.

3288

MeisterGRIP was found to allow institutive and dexterous
robot manipulation on the basis of vision-based
haptic-sensing technology. Furthermore, it allows universal
manipulation, i.e., it compensates differences in the handsizes and the grasping postures of the operators.
In the future, to improve the accuracy of the measured
grip force, we intend to improve the conversion matrix of theequation (1). This could reduce the error in theapproximation of the captured image. Furthermore, we
intend to incorporate electrotactile feedback into
MeisterGRIP to obtain a general-purpose master hand.

balloonanimalrobotic arm
MeisterGRIP

robotic handmasterarm

Fig. 11. System controlling robotic hands and arms by using MeisterGRIP.

REFERENCES[1]

S. Tachi and K. Yasuda, “Evaluation Experiments of a Telexistence
Manipulation System,” Presence, Vol. 3, No.1, pp. 35–44, 1994
[2] Intuitive Surgical Inc. da Vinci.
[3] Immersion Technology Inc. Cybergrasp.
http://www.immersion.com/index.php.
[4] K.Hoshino and T.Tanimoto: “Real time search for similar hand
images from database for robotic hand control,” IEICE Transactions
on Fundamentals of Electronics, Communications and Computer
Sciences, E88-A, 10, 2005, pp.2514–2520.
[5] K.Hoshino: “Interpolation and extrapolation of repeated motions
obtained with magnetic motion capture,” IEICE Transactions on
Fundamentals of Electronics, Communications and Computer
Sciences, E87-A, 9, 2004, pp.2401–2407.
[6] M. Yoshikawa, T. Tsujimura, M. Mikawa, and K. Tanaka :“Human-machine Interface Using EMG Signals for Robot Hand
Control,” The Society of Instrument and Control Engineers (SICE)
Annual Conference 2005 in Okayama: Proceedings, 2005,
pp.2853–2858.
[7] K. Kamiyama, H. Kajimoto, N. Kawakami, and S. Tachi, “Evaluationof a Vision-based Tactile Sensor,” in Proceedings of International
Conference on Robotics and Automation 2004, Vol. 2, 2004, pp.
1542–1547.
[8] L. D. Landau and E.M.Lifshitz, “Theory of Elasticity,”
ButterworthHeinemann, 1985.
[9] S. Komeiji, K. Sato, K. Minamizawa, H. Nii, N. Kawakami, and S.
Tachi, “MeisterGRIP: cylindrical interface for intuitional robot
manipulation,” SIGGRAPH Posters and New Tech Demo, 2008.
[10] S. Komeiji, K. Sato, M. Tatezono, K. Minamizawa, H. Nii, N.
Kawakami, and S. Tachi, “MeisterGRIP: cylindrical interface for
intuitional robot operation,” interactive-Tokyo 2008, 2008.
[11] K. Sato, H. Kajimoto, N. Kawakami, and S. Tachi, “Electrotactile
Display for Integration with Kinesthetic Display,” in Proceedings
RO-MAN 2007, 2007, pp.3–8.