IEEE TRANSACTIONS ON HAPTICS,

VOL. 3,

NO. 1,JANUARY-MARCH 2010

37

Finger-Shaped GelForce: Sensor for Measuring
Surface Traction Fields for Robotic Hand
Katsunari Sato, Student Member, IEEE, Kazuto Kamiyama, Naoki Kawakami, andSusumu Tachi, Member, IEEE
Abstract—It is believed that the use of haptic sensors to measure the magnitude, direction, and distribution of a force will enable a
robotic hand to perform dexterous operations. Therefore, we develop a new type of finger-shaped haptic sensor using GelForcetechnology. GelForce is a vision-based sensor that can be used to measure the distribution of force vectors, or surface traction fields.
The simple structure of the GelForce enables us to develop a compact finger-shaped GelForce for the robotic hand. GelForce that isdeveloped on the basis of an elastic theory can be used to calculate surface traction fields using a conversion equation. However, thisconversion equation cannot be analytically solved when the elastic body of the sensor has a complicated shape such as the shape of afinger. Therefore, we propose an observational method and construct a prototype of the finger-shaped GelForce. By using thisprototype, we evaluate the basic performance of the finger-shaped GelForce. Then, we conduct a field test by performing grasping
operations using a robotic hand. The results of this test show that using the observational method, the finger-shaped GelForce can be
successfully used in a robotic hand.
Index Terms—Force sensor, surface traction field, finger shape, robotic hand.

Ç
1

INTRODUCTIONH

UMANOID robots are made to adapt to the environment

in which human beings live and assist humans in their
daily life. Human beings have longed for such robots foundin science fiction and tried to realize them. In recent years,
there has been a remarkable development in humanoid
robots; in addition, robots that are capable of assisting
patients or doing housework instead of human beings havebeen developed. Haptic information acquired by a sensor isimportant in enabling these robots to perform dexterous
operations. In particular, a haptic sensor for a robotic finger
is considered to be an important device because the finger ofa humanoid robot is the most useful part of its body when it
is used to manipulate an object. These robots can stably grasp
an object using their hands on the basis of haptic informationsuch as reflective force, frictional force, and point of contact
with the object. Therefore, a number of haptic sensors for use
in robotic hands have been developed [1], [2].
The objective of this study is to develop a finger-shaped
haptic sensor for a humanoid robot (Fig. 1). While developing such haptic sensor for a robotic finger, we should take
. K. Sato and N. Kawakami are with the Graduate School of Information
Science and Technology, The University of Tokyo, 7-3-1 Hongo,
Bunkyo-ku, Tokyo 113-8656, Japan.
E-mail: {Katsunari_Sato, Naoki_Kawakami}@ipc.i.u-tokyo.ac.jp.
. K. Kamiyama is with the Department of Human Communication, The
University of Electro Communications, 1-5-1 Chohugaoka, Chohu-city,
Tokyo 182-8585, Japan. E-mail: kamiyama@hc.uec.ac.jp.
. S. Tachi is with the Graduate School of Media Design, Keio University,
4-1-1 Hiyoshi, Kohoku-ku, Yokohama City, Kanagawa 223-8526, Japan.
E-mail: tachi@tachilab.org.
Manuscript received 15 Nov. 2008; revised 30 July 2009; accepted 24 Sept.
2009; published online 22 Oct. 2009
Recommended for acceptance by Y. Yokokohji.
For information on obtaining reprints of this article, please send e-mail to:
toh@computer.org, and reference IEEECS Log Number THSI-2008-11-0088.
Digital Object Identifier no. 10.1109/ToH.2009.47.
1939-1412/10/$26.00 ß 2010 IEEE

into consideration its requirements. These requirements aredescribed as follows: First, the sensor should be able tomeasure the magnitude, direction, and distribution of an
applied force or a surface traction field. Human beings
perceive all three parameters and grasp an object safely
and dexterously, even if the object is fragile. Similarly, robots
must be able to detect surface traction fields. Second, the sizeof the sensor should be small. In order to integrate many
haptic sensors in a humanoid robot hand, it is preferable touse compact sensors. Finally, the sensor should be a highperformance device. In order to use this sensor in our daily
life, we need to ensure that the magnitude, spatial, andtemporal sensing performance are the same as thoseperceived by human beings. For example, human begins
can detect approximately 0.5 mN force applied to the skin [3],distinguish two applied forces separating approximately 1.21.7 mm [4], and perceive vibration up to 700 Hz [5].
Nearly all existing haptic sensors for robotic hands [6],[7], [8] belong to one of the following two categories.
Sensors that belong to the first category are used to measure
a force vector at a single point; we call them force vector
sensors. Sensors that belong to the second category are used
to measure the distribution of forces; however, thesesensors can only measure the magnitude of a force andnot its direction. We call such sensors as force distribution
sensors. These sensors are employed with a sensing unit,
where mechanical pressure is exerted. Therefore, sensors
that can be used to measure surface traction fields can be
developed by integrating many force vector sensor elements. However, it is difficult to realize a sensor with small
size and high resolution, because the use of many sensors
can complicate the sensor structure.
Another approach to measure the surface traction fields
involves the use of optics [9], [10], [11], [12]. However, it isstill difficult to satisfy all the abovementioned requirements
using optics. For example, Ferrier and Brockett [10] have
Published by the IEEE CS, RAS, & CES

Authorized licensed use limited to: UNIVERSITY OF TOKYO. Downloaded on March 29,2010 at 01:23:23 EDT from IEEE Xplore. Restrictions apply.

38

IEEE TRANSACTIONS ON HAPTICS,

Fig. 1. Example of operation performed by a robotic hand using ourfinger-shaped sensor. The top and bottom surfaces shown on the laptop
display represent surface traction field measured by the sensors
mounted on the index finger and the thumb of the robotic hand,
respectively.

developed a small tactile sensor that can be used to track
dots on an elastic material in order to compute the3D configuration of a finger-shaped elastic membrane.
Unfortunately, a complex processing method is required forthis sensor to acquire the direction of force, which could
potentially lead to inaccuracy and instability of calculationof the surface traction fields. In addition, Kamiyama et al.
[12] have developed a GelForce that is based on an elastic
theory. GelForce has a simple structure; therefore, itsatisfies the second and third requirements, i.e., it is smallin size, and it has high resolution. However, the elastic
theory is applicable to a semi-infinite elastic body. Therefore, the surface of a conventional GelForce is a plane.
In this study, in order to satisfy all the above mentioned
requirements, we develop a finger-shaped haptic sensor
using GelForce technology. Because of the complicatedshape of a finger, the elastic theory cannot be analyticallyresolved by the finger-shaped GelForce. Therefore, we
propose an observational method to calculate surface
traction fields using the finger-shaped GelForce. In thispaper, we present the principle, design, and observational
method of the finger-shaped GelForce. Then, we present theresult of an evaluation of a prototype finger-shaped GelForceto confirm 1) that proposed method is applicable and 2) thatthe finger-shaped GelForce can be applied to operations
conducted by a robotic hand. We compared the basic
performance of the finger-shaped GelForce with that of theconventional GelForce. We also conducted a field test on thefinger-shaped GelForce using a robotic hand.

2

FINGER-SHAPED GELFORCE

2.1 Basic Principle of GelForceWe developed a finger-shaped sensor using the same
technology used to develop GelForce [12]. This sensor
consists of a transparent elastic body, light (red) and dark
(blue) gray marker matrixes, and a color charge-coupleddevice (CCD) camera that captures the movements of themarkers. When a force is applied to the surface, the markers
move in response to the magnitude and direction of the force

VOL. 3,

NO. 1, JANUARY-MARCH 2010

Fig. 2. Movements of markers in the elastic sensor body caused by force
applied to the sensor surface [12].

(Fig. 2). The movements of the markers are captured by thecamera. We calculate the center of the markers before andafter the application of the force to calculate the movements.
When we assume that the deformation of the elastic body islinear, the relationship between the two-dimensional movements of m-markers u and the three-dimensional n-force
vectors f is given by using conversion matrix H.
u ¼ Hf:2

3uRx
6 uRy 77u¼6
4 uBx 5;uBy

2

3hRxx ; hRyx ; hRzx
6 hRxy ; hRyy ; hRzy 7
7
H¼6
4 hBxx ; hByx ; hBzx 5;
hBxy ; hByy ; hBzy23fx
f ¼ 4 fy 5 :fz

ð1ÞIn this equation, uRp , uBp , hRqp , hBqp , and fq indicate thep-directional captured movement of the light gray (red)marker, that of the dark gray (blue) marker, an element of
the conversion matrix of the light gray (red) marker, that ofthe dark gray (blue) marker, and the q-directional applied
force, respectively. We obtained f from u by solving theinverse of (1).
It should be noted that GelForce uses a single camera for
capturing the movements of the two layers of markers. To
calculate the three-dimensional force vector, it is necessary
to measure the movements of the markers in a threedimensional manner. A stereo camera would be suitable but
would be too bulky and difficult to calibrate. Hence, toovercome these problems, we used two layers of markers,
each of different color, and one camera. The two different
colors make is easier to distinguish between the markers.
However, the measured marker movements for each layer
are not perfectly independent. Therefore, it is difficult tosolve the inverse of (1). Thus, we applied the dumped leastsquares method [13].
f ¼ H g u
H g ¼ ðH T H þ !IÞ1 H T u:

ð2ÞIn this equation, ! and I represent the dumping factor andunit matrix, respectively.
GelForce does not require any electronic components
such as resistors or capacitors. Therefore, it is easy to adapt

Authorized licensed use limited to: UNIVERSITY OF TOKYO. Downloaded on March 29,2010 at 01:23:23 EDT from IEEE Xplore. Restrictions apply.

SATO ET AL.: FINGER-SHAPED GELFORCE: SENSOR FOR MEASURING SURFACE TRACTION FIELDS FOR ROBOTIC HAND

39

Fig. 3. Configuration diagram of the finger-shaped vision-based haptic
sensor.

the shape and size of the GelForce to those of the fingertip.
In addition, new developments in camera technology may
help us to improve spatial and temporal resolutions of thesensor further. The elastic body of the sensor provides anadded advantage. The elasticity enables the sensor tomeasure haptic information accurately by reproducing thephysical interaction involved in the sense of touch. The
sensor is also highly durable.

2.2 Design of the Finger-Shaped GelForce
2.2.1 Configuration of the Prototype
Based on the GelForce technology, we constructed a
prototype of the finger-shaped GelForce. Ideally, theperformance of the sensor should be comparable to that ofhuman beings, as mentioned in chapter 1. However, it wasdifficult to achieve this performance with current (2008)
camera technology. Therefore, we constructed the prototype for the evaluation using a commercially available
camera with a video graphics array (VGA) resolution and aframe rate of approximately 60 fps.
The configuration of the prototype is shown in Fig. 3. The
transparent elastic body and the CCD camera are attachedto the plastic base of the robotic hand. The transparent
elastic body is made from silicon rubber (Shin-Etsu
Chemical Co., KE-109). Because the conventional GelForceis also made from the same silicon rubber, the linearity of
the finger-shaped GelForce appears to be less than 7.5 N[12]. The shape of the elastic body is partly spherical andpartly cylindrical; hence, it resembles a human fingertip.
The dimensions of its cylindrical and spherical parts are
18  9  10 mm3 and 18  9  9 mm3 , respectively. The
boundary between the elastic body and the base of robotic
hand is fixed by a transparent acrylic plate. The camera iscustom made comprising a video compression unit andfour image-capturing units (ViewPLUS Inc.). The size of thevideo compression unit and the image-capturing units are
100  100  10 mm3 and 14  16  1:5 mm3 , respectively.
Each image-capturing unit is connected to the video
compression unit using a flexible cable. In addition, thevideo compression unit is connected to a PC using a USB
cable. The size of the cable is 20  10  0:5 mm3 . The
resolution of each image is 640  480 pixels (VGA). The
frame rate of the camera is 67 fps. We used a camera with awide-angle lens to capture the movements of all themarkers. The lens angle of the camera is 110 degree. The
diameter and the height of the lens are 14 mm and 6.5 mm,
respectively. The constructed prototype and its dimensions

Fig. 4. (a) Constructed prototype of finger-shaped GelForce. This picture
was taken from the same view as that of Fig. 3. (b) Captured image ofmarkers by camera. The white dashed line represents the outline of thefinger-shaped GelForce from the z-directional view.

are shown in Fig. 4a. The total dimension of the prototype is
18  23  36 mm3 . Fig. 4b shows the images of the markers
captured by the camera. To capture the markers clearly, we
set their diameters to 0.5 mm (approximately 15 pixels). To
prevent the overlapping of the light (red) and dark (blue)
gray markers, we also separated them by a distance of
2.5 mm. The light (red) and dark (blue) gray markers wereplaced at a depth of 0.5 and 2.0 mm, respectively, from thesurface of both cylindrical and spherical part. The numbersof light (red) and dark (blue) gray markers are 25 and 23,
respectively. We printed the disk markers using a silkscreen printmaking technique. The markers are made fromsilicon rubber and a colorant. Because the sensor uses
vision-based technology, light from the environment
produces noise. Therefore, the surface of the sensor iscovered with a black sheet made from silicon rubber. In
addition, to capture the marker movements clearly, we
illuminated them using a light-emitting diode (LED)
mounted near the camera. We used eight high-intensity
LED chips (audio-Q Inc., AQ-KS1609WC).

2.2.2 Accuracy of Capturing the Marker
The surface traction field is calculated from the movementsof the centers of the markers captured by the camera.
Therefore, an error that occurs while capturing the movements of the markers may produce noise in the measurement of the force. In this section, we evaluate the accuracyof the measurement of the movements of the markers when
no force is applied to the surface of the sensor.
We capture two images of the markers when no force isapplied. Then, we displace the markers in one of the imagesby 5.0 pixels along the x-direction. Displacement vectors ofthe markers are calculated using these two images. Table 1

Authorized licensed use limited to: UNIVERSITY OF TOKYO. Downloaded on March 29,2010 at 01:23:23 EDT from IEEE Xplore. Restrictions apply.

40

IEEE TRANSACTIONS ON HAPTICS,

VOL. 3,

NO. 1, JANUARY-MARCH 2010

TABLE 1
Measurement of Marker Movements

lists the average and the standard deviations of the x- andy-displacements of the light (red) and dark (blue) graymarkers. This result shows that the movements of themarkers are measured with subpixel accuracy.

2.3 Conversion Matrix H
2.3.1 The Conventional Analytic Method and Its
Disadvantages
In the case of the conventional GelForce, the conversion
matrix H in (1) is analytically constructed. On the basis of
the assumption that the sensor is a semi-infinite elastic
body, we can calculate the elements of H using the elastic
theory [14].
Unfortunately, this assumption is not applicable to afinger-shaped GelForce because the shape of this sensor iscurved. It may be possible to obtain the conversion equation
under a boundary condition that the surface of the elastic
body is curved. However, development of a conversion
equation is quite difficult.
Another approach to constructing the H involves aplanar approximation of the sensor surface. The surface
traction field was calculated using elastic theory under theassumption that the curved sensor surface is flat. However,in the finger-shaped GelForce, the movements of markers
are captured intricately. Unlike the arrangement of markers
in the conventional GelForce, the markers in the fingershaped GelForce are arranged along the surface, which has
a complicated shape. Also, to construct a small sensor, weused a camera with a wide-angle lense. The wide-angle lensdistorts the camera image. Therefore, the patterns of themarker movements captured by the camera differ from one
marker to the other. The intricacy of marker movementscomplicates the conversion of marker movements, which is
required for satisfying the elastic theory.
2.3.2 Observational Method
Due to the above-mentioned problem, we could not
determine H of the finger-shaped GelForce using theanalytical method. Therefore, with the finger-shaped
GelForce, we estimated the elements of H by observational
method. Fig. 5 shows the algorithm used for constructing H.
First, we applied forces at each force sampling point andrecorded the applied force. Then, we measured the movement of markers. From the recorded forces and measured
movement of markers, we estimated H.
Fig. 6 shows the apparatus used for constructing H. The
surface of the sensor was pushed using a probe. The edge ofthe probe is spherical. The probe ideally has a small diameter.
However, when we applied a force using the small probe, theFig. 5. (a) Algorithm for construction of conversion matrix H. The
elements of H are calculated from the measured force fn0 and the
movements of the markers denoted by un . (b) Application of force to zand x- (or y-) direction.

probe slipped on the surface of the finger-shaped GelForce.
Therefore, we used the acrylic probe with a diameter of bothcylindrical and spherical part 5.0 mm. The probe is mounted
on a six-axis force sensor (BL NANO, BL Autotech, Ltd.) tomeasure the magnitude of the applied force. In addition, weused an xyz stage (VSQ-601XYV, Chuo Precision IndustrialCo., Ltd.) to control the applied force. The points at which
force is applied are shown in Fig. 7. The number of the force
sampling points of GelForce depends on the number anddensity of the markers. In the prototype, the numbers ofmarkers are 25 þ 23 ¼ 48 and they are separated by a distanceof 2.5 mm. Therefore, the number of the force sampling points
should be less than 32. In this prototype, we applied forces at
5  5 ¼ 25 points on the surface of the sensor. These points
are also separated by a distance of 2.5 mm.

Fig. 6. Image of apparatus used for constructing H.

Authorized licensed use limited to: UNIVERSITY OF TOKYO. Downloaded on March 29,2010 at 01:23:23 EDT from IEEE Xplore. Restrictions apply.

SATO ET AL.: FINGER-SHAPED GELFORCE: SENSOR FOR MEASURING SURFACE TRACTION FIELDS FOR ROBOTIC HAND

H ¼ ½h1 ; h2 ; . . . ; hn ; . . . ; hN :

41

ð6Þ

When we calculate the H g from the constructed H, thedamping factor causes some errors to the H g in (2) [13].
Therefore, to calibrate the finger-shaped GelForce, we
determined a constant that fits the magnitude of the force
calculated using the finger-shaped GelForce to that of theforce measured using the six-axis force sensor.

3

EVALUATION OF SENSOR PERFORMANCEIn this section, we evaluate the basic performance of theconstructed prototype. We evaluate the following threecriteria: force magnitude, spatial resolution, and the timerequired for carrying out various calculations such as
evaluation of the conventional GelForce [12]. By drawing acomparison between the performance of the finger-shaped
GelForce and that of the conventional GelForce, we examine
the validity of the observationally constructed H. The
apparatus used for evaluating the environment is the sameas that used for constructing H (Fig. 5).
Fig. 7. Force sampling points. (i, j) represents two-dimensional number ofsampling points. The black dashed line represents the same outline of thefinger-shaped GelForce from the z-directional view illustrated in Fig. 4.

First, forces were applied to the one of the force sampling
points along three directions. When we applied a force
along the z-direction, the magnitude of the applied force
was approximately 1.2 N. It is impossible to apply a force
only along the x- or y-direction. Hence, we first applied aforce along the z-direction, and then, we applied suitable
forces along the x- and y-directions (Fig. 5b). The
magnitudes of the applied force along z-direction and xor y-direction were approximately 1.0 and 0.25 N, respectively. Then, we recorded the force elements resulting fromthe application of the force by the six-axis force sensor.
After applying force along three directions, the recorded
forces at the sampling point fn0 were denoted as follows:
2 0
3fx x ; fy0 x ; fz0 x
0000
fn ¼ 4 fx y ; fy y ; fz y 5;
ð3Þ
fx0 z ; fy0 z ; fz0 z
where fp0 q is the q-directional element of the force when an
p-directional force is applied.
Next, we captured the movements of markers un . The
movement of this marker is given by (4).

uxx ; uyx ; uzx
;
ð4Þ
un ¼
uxy ; uyy ; uzy
where upq is the q-directional movement of the marker when
the p-directional force is applied. Using fn and un , we cancalculate the elements of H at the sampling point as follows:hxx ; hyx ; hzx
ð5Þ
hn ¼ un fn01 h ¼
hxy ; hyy ; hzyequation, hn ; un , and fn0

In thisrepresent 2m  3 conversion matrix, 2m  3 matrix of m-marker movements, and3  3 matrix of a force, respectively. Using this equation
at N-force sampling points (in this prototype, N ¼ 25), we
can construct H by (6).

3.1 Magnitude
We evaluate the linearity, resolution, and independency ofthe magnitude of the applied force along the following three
directions: x, y, and z. Because of the small size of thefinger-shaped GelForce, the linearity of its elastic bodyseems not to be assumed when a large force is applied.
Therefore, first, we evaluate the linearity of the sensor alongz-direction. Then, we evaluate the resolution and independency of the magnitude of the force along the abovementioned three directions.
While evaluating the magnitude of the applied force, wedraw a comparison between the measured force vectors F
and the applied force F 0 . The applied force F 0 is measured
using the six-axis force sensor and the controlled xyz stage.
Force is applied at the center of the surface of the sensor,
where x ¼ 5:0 mm and y ¼ 5:0 mm, as shown in Fig. 7. In
this evaluation, we applied a force using a cylindrical probewith a diameter of 10 mm. When a finger touches an object,
the contact area is usually a surface, and not a point.
Therefore, we used a wide probe to evaluate the performance of the sensor in a manner similar to an actual contact.
Furthermore, the measured force from GelForce should be
evaluated by the total of force at every sampling point
because independency of each sampling point is notconfirmed. Therefore, each directional force, denoted by
Fx , Fy , and Fz , is calculated using (7).
Fx ¼

5;5
X

fxij ;

i;j

Fy ¼

5;5
X

fyij;

ð7Þ

i;j

Fz ¼

5;5
X

fzij :

i;j

In this equation, i and j are the numbers of the force
sampling point in Fig. 7, fxij , fyij , and fzij are represented
as one-dimensional force vectors along each direction.

Authorized licensed use limited to: UNIVERSITY OF TOKYO. Downloaded on March 29,2010 at 01:23:23 EDT from IEEE Xplore. Restrictions apply.

42

IEEE TRANSACTIONS ON HAPTICS,

VOL. 3,

NO. 1, JANUARY-MARCH 2010

Fig. 8. Evaluation of linearity of the finger-shaped GelForce inz-direction.

3.1.1 Preliminary Experiment of Linearity
While evaluating linearity, we applied an increasing force
and investigated the limitation of linearity. Because of thecomplicated shape of the sensor, it is difficult to apply a large
force along the x- or y-direction. Therefore, we evaluate thelinearity of the magnitude only along the z-direction.
We applied a force of 1.0-10.0 N along the z-direction
by increasing its magnitude in steps of 1.0 N. Values of thez-directional force obtained by using the finger-shaped
GelForce and six-axis force sensor are recorded. The
measurement procedure is repeated five times.
Graph shown in Fig. 8 presents the results of evaluation.
The horizontal and vertical axes represent the applied force
and the measured force, respectively. The dots show theresults of each trial. The line shows that the applied force isequal to the measured force. From these results, it isobserved that the linearity of the magnitude is below 4.0 N.
3.1.2 Resolution, Independency, and Linearity
While evaluating resolution and independency, we investigated the measurement accuracy of the magnitude anddirection of the force along the three directions. In theevaluation of the conventional GelForce, resolution ofmagnitude along the z-direction and the resolution of angle
are examined. In addition, it is also essential to examine theresolution of the magnitude and the coupling effect of
the sensor along the three directions. It is found that due tothe complicated shape of the sensor, measurement noise isgenerated along the x- or y-direction. Therefore, in the caseof the finger-shaped GelForce, we examine the resolution ofmagnitude instead of the resolution of angle along the xand y-directions.
To investigate the resolution of magnitude along the xand y-directions, we first applied a force of 2.0 N along
the z-direction. Then, we applied the force of 0.2-2.0 N
along the x- or y-direction by increasing its magnitude insteps of 0.2 N. To investigate the resolution of magnitude
along the z-direction, we applied a force of 0.2-2.0 N
along the z-direction by increasing its magnitude in stepsof 0.2 N. From Fig. 8, it can be observed that the fingershaped GelForce seems to be linear under these applied
forces. The force values obtained using the finger-shaped
GelForce and six-axis force sensor is recorded. We
recorded all the directional forces applied for eachFig. 9. Relationship between the applied force and measured force
along the (a) x-direction, (b) y-direction, and (c) z-direction.

experiment. The measurement procedure is repeated fivetimes for each direction.
Graphs shown in Fig. 9 present the results of evaluation.
The horizontal and vertical axes represent the applied force
and the measured force, respectively. The black squares,
triangles, and dots represent the absolute value of Fx , Fy ,and Fz , respectively. Each outline dots represent coupling
effects in each direction. To clearly show the measured force
in each graph, we compensated the values of directional
forces other than the force applied direction. For example,

Authorized licensed use limited to: UNIVERSITY OF TOKYO. Downloaded on March 29,2010 at 01:23:23 EDT from IEEE Xplore. Restrictions apply.

SATO ET AL.: FINGER-SHAPED GELFORCE: SENSOR FOR MEASURING SURFACE TRACTION FIELDS FOR ROBOTIC HAND

43

TABLE 2
Calculation Time

Fig. 10. Force measured by high-density sampling. (a) and (b) graphs
show the results of applying a force at the center and edge of the sensor,
respectively.

when we applied an x-directional force, we subtracted the yand z-directional applied forces Fy0 and Fz0 from Fy and Fz ,respectively. The line in this graph shows that the applied
force is equal to the measured force. From these results, it isobserved that the maximum resolution of the magnitude of
the force is 0.3 N along each direction. The magnitude of theother directional elements of the force is approximately
below 0.3 N, other than y-directional element of graph (c);0.5 N.

3.2 Spatial Resolution
In this section, we evaluate the spatial resolution of thesensor. To carry out this evaluation, we determine themagnitude of each force vector when a force is applied at a
certain point on the surface of the elastic body. The
magnitude of the force vector fij is calculated using (8).
qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ
fij ¼ fx2ij þ fy2ij þ fz2ij :ð8ÞIt should be noted that the sampling interval is too large toevaluate the spatial resolution. Therefore, we adopt thefollowing procedure to evaluate the spatial resolution. Twoadjacent sampling points on the surface of the sensor are
selected. The distance between these two points is subdivided into n points, and a known force is applied along
the z-direction at each point. Then, the force vectors at eachpoint are calculated by considering each point as the origin.
A graph is constructed by plotting the distance from eachorigin and the calculated force magnitude on the horizontal
and vertical axes, respectively. By using the method
explained above, the density of the points in the samplingregion increases. The spatial resolution of the force
distribution can be obtained from the slope of the graph.
The magnitude of the force applied along the z-direction
is fixed at 2.0 N. The diameter of the spherical probe that isused to apply the force is 5.0 mm. In this condition, thedisplacement of the probe into the skin is approximately 1.3
mm. Adjacent sampling points are located at the center andedge of the area in which the force field is measured. The
actual sampling interval is 2.5 mm, and the distance between
adjacent points is subdivided into intervals of 0.5 mm. In thecase of the conventional GelForce, the spatial resolution is
examined only at the center of the sensor. However, in thecase of the finger-shaped GelForce, the shape of the sensor
influences its spatial resolution. Therefore, we evaluate thespatial resolution at both the center (x ¼ 5:0 mm and y ¼ 5:0
mm in Fig. 7) and the edges (x ¼ 5:0 mm and y ¼ 0:0 mm,
x ¼ 0:0 mm and y ¼ 5:0 mm in Fig. 7) of the surface of thesensor and along both the x- and y-directions.
We obtained approximately the same results for all
measurement points and directions. Fig. 10 shows therepresentative results of the experiment. The first graphshows the results obtained by applying a force at thecenter of the sensor. The second graph shows the results
obtained by applying a force at the edge of the sensor. The
values plotted on the vertical axis are normalized using
the maximum value of the force magnitude. The dots in
the graphs represent the points at which the force isapplied (x [mm], y [mm] in Fig. 7).
From the graphs shown in Fig. 10, it can be observed thatthe full width half maximum is approximately 5.0 mm.
Therefore, the spatial resolution of the force is estimated tobe approximately 5.0 mm. These results are identical tothose obtained by carrying out evaluations at other points
and directions.

3.3 Time
To perform a dexterous operation, we need to obtain haptic
information from multiple sensors. Therefore, we must
process a large amount of information obtained fromdifferent sensors simultaneously. In this section, we
evaluate the calculation times of the single finger-shaped
GelForce and four sensor units.
Calculation time is obtained from the frequency of theCPU. In this experiment, the speed of the processor used in
the PC is 2.8 GHz (Intel PentiumD processor). The programruns on Windows XP SP2. The accuracy of measurement is
1 s. The calculation time is measured over 100 trials, andthe average and standard deviation of the calculation time
are listed in Table 2. The sensor units represent the fingershaped GelForce attached to the video compression unit.
In this prototype, the camera unit captures four images
at 67 fps. Therefore, the PC must calculate the surface

Authorized licensed use limited to: UNIVERSITY OF TOKYO. Downloaded on March 29,2010 at 01:23:23 EDT from IEEE Xplore. Restrictions apply.

44

IEEE TRANSACTIONS ON HAPTICS,

TABLE 3
Performance of Conventional and Finger-Shaped GelForce

Value shown in parentheses is the estimated one.

traction fields of the four sensors within 14 ms. The resultof this experiment satisfies this requirement even when
we process information obtained from the four sensor
units simultaneously.

3.4 Discussion
We compared the performance of the finger-shaped
GelForce with that of the conventional GelForce; the resultsof this comparison are summarized in Table 3. Because of
the small size of the finger-shaped GelForce, it is confirmed
the linearity of force is below 4.0 N. However, Fig. 8 showsa one-to-one relationship between the applied and measured forces. Therefore, we can measure a force rather than
4.0 N by compensating for the nonlinearity of the sensor.
The resolution of magnitude of the finger-shaped GelForce is same as that of the conventional GelForce. However,Fig. 9b shows an offset in the y-directional force, and Fig. 9cshows a y-directional element of approximately 0.5 N. We
consider that the shape of the sensor is responsible for theformation of an unintended directional element of force. We
also consider that the shape of the sensor is responsible forthe nonlinearity of the x- or y-directional force greater than
1.0 N. The coupling effect of the sensor is small in all
directions. Furthermore, when we estimate the resolutions ofangle from the results of Figs. 9a and b, the resolution isapproximately 6 degree. Therefore, the three directional
force vectors seem to be calculated correctly. Furthermore,because the magnitude resolution is depend on the camera
resolution in GelForce, we can expect further improvementof the resolution and archive 0.5 mN [3] in the future.
The spatial resolution of the finger-shaped GelForce islower than that of the conventional GelForce. However, in
this prototype, we construct the conversion matrix in 2.5 mm
force sampling interval. From the sampling theory, 5.0 mm
spatial resolution is reasonable. The spatial resolution is
expected to improve with an increase in the density of thesampling points and markers. For example, in the case of thesensor, to obtain a spatial resolution similar to that of thehuman fingertip (approximately 1.2-1.7 mm [4]), the distance of force sampling point should be under 0.9 mm. To
archive this value, the distance between the markers and thediameter of each marker should be approximately 0.2 mm
and 0.9 mm, respectively. In the prototype, the diameter ofthe marker is 0.5 mm or 10-20 pixels, and the error in themeasurement of the marker movement is 0.15 pixels.

VOL. 3,

NO. 1, JANUARY-MARCH 2010

Therefore, we can reduce the size of the markers and the
distance between them in order to meet the desired
specifications. Furthermore, we can expect a further improvement in the spatial resolution with an increase in thecamera resolution. The discontinuous result shown in
Fig. 10a is considered to be obtained due to an error in thecalculation of the pseudoinverse matrix using (2).
Time resolution of the sensor can also be improved.
When we consider the human performance of haptic
perception, it is required to maintain the refresh rate of
the force sensor to greater than 700 Hz [5]. In the case of the
prototype sensor, the calculation time is small, i.e., less than
2.5 ms even after using four sensors. Therefore, a time
resolution of 700 Hz can be easily archived by improving theperformance of the computer and frame rate of the camera.
From these results, we consider that the finger-shaped
GelForce can be used to measure surface traction fields using
the observationally constructed conversion matrix. Therefore, we believe that the implementation of this haptic sensor
will enable robotic hands to perform dexterous operations.

4

FIELD TESTWe developed a finger-shaped GelForce as a haptic sensor
for a robotic hand. In this section, we have discussed theimplementation of the finger-shaped GelForce for use in aconventional robotic hand and performed a grasping of an
object as a field test.
Through the grasping operation, we have illustrated an
operation that is performed using the haptic informationacquired by the finger-shaped GelForce. Furthermore, wehave shown that the finger-shaped GelForce can be used tocorrectly measure the surface traction field in actual use.
When we implement the finger-shaped GelForce for the
robotic hand, the vibration and movement of the robotic
finger can prevent the accurate measurement of the surface
traction fields. Therefore, we have evaluated those surface
traction fields that were measured during the grasping
operation performed by the robotic hand.

4.1 Implementation to Robotic Hand
We use a multifingered robotic hand [15] as a humanoid
robotic hand. Fig. 11 shows a robotic hand with the fingershaped GelForce. This hand has a total of 15 degrees of
freedom (DOFs)—five DOFs for the thumb, one forabduction of other fingers, three for the index finger, andtwo each for the remaining fingers. Each fingertip also has
an independent DOF. The thumb has a number of DOFs,
which enables to cover wide range of movements of thethumb. In addition, this DOF allows the index finger andthe thumb to be countered.
The control unit is placed on the back of the hand. This
unit is connected to the PC through a Controller Area
Network (CAN) interface card (CBI4852, Interface Corp.).
We control the robotic hand by using a PID control at 1 kHz.
However, time resolution of the finger-shaped GelForce isapproximately 67 Hz. This large calculation time causes
instability and hampers the control of the robotic hand.
Therefore, we use two PCs, i.e., one for controlling therobotic hand, and the other for calculating the fingershaped GelForce. We connect both these PCs through LAN
and transmit haptic information at 67 Hz using a UDP.

Authorized licensed use limited to: UNIVERSITY OF TOKYO. Downloaded on March 29,2010 at 01:23:23 EDT from IEEE Xplore. Restrictions apply.

SATO ET AL.: FINGER-SHAPED GELFORCE: SENSOR FOR MEASURING SURFACE TRACTION FIELDS FOR ROBOTIC HAND

45

Fig. 12. Setup for controlling robotic hand. Robotic hand grasps a
Styrofoam ball by its index finger and thumb. Thumb moves along the xor z-direction in each experimental operation.

Fig. 11. Robotic hand with finger-shaped GelForce.

We attach four finger-shaped GelForces at the fingertipsof the thumb, index finger, middle finger, and annular
finger of the robotic hand. Properties of the finger-shaped
GelForce are the same as those of the constructed prototype
discussed in chapter 2. The video compression unit isplaced on the back of the robotic hand.

4.2 Grasping Operation
In order to perform a field test on the finger-shaped
GelForce, we performed simple grasping operations using
the robotic hand. Ideally, the operation using the measured
surface traction field is desired. However, because thesurface traction field contains much information, it isdifficult to clearly show the efficiency of surface traction
field. Therefore, we shrunk the surface traction field andconducted two operations. One operation involved thecontrol of the grasping force using the three-dimensional
force. In this evaluation, we used the finger-shaped
GelForce as a force vector sensor. The other operation
involved the control of the grasping position using thecenter of the force vector distribution. In this evaluation, weused the singer-shaped GelForce as a force distribution sensor.
4.2.1 Control of Grasping Force
The measurement and control of the grasping force avoids
the application of a considerable force to an object. This
control enables soft grasping so as not to damage the object
and the robotic hand. Furthermore, the measurement of africtional force and an increase in the grasping force preventthe dropping of the object. Therefore, controlling thegrasping force using the three-dimensional force information is important in order to ensure the safety and stablegrasping of the object.
In the operation that involves the control of the grasping
force, the robotic hand grasps a styrofoam ball between the

thumb and the index finger, as shown in Fig. 12. The
diameter and weight of the ball are 60 mm and 5.7 g,
respectively. To grasp the ball, we apply a torque to rotate
the index finger and thumb about the x-axis. The grasping
force is controlled at an arbitrary value to ensure that theball does not move. When we apply a force to the ball along
the x-direction, the finger-shaped GelForce detects thefrictional force and increases the grasping force of thefingers. The applied force is approximately 0-1.5 N. The
additional grasping force is proportional to the frictional
force. The proportionality constant is determined by
performing a preliminary experiment whose objective isto ensure that the grasped ball is not dropped. When thegrasping force exceeds 4.0 N, the torque of the thumb is set
to zero. We have measured the grasping force Fz and thefrictional force Fx of the thumb using (7). In addition, we
have measured the torque Tx that rotates the base of thethumb about the x-axis.
The results of the abovementioned experiment are
shown in Fig. 13. In this figure, the horizontal and verticalaxes represent the operation time and the measured value,

Fig. 13. Result of controlling grasping force. Fz, Fx, and Tx represent
grasping force, frictional force, and torque of thumb, respectively.

Authorized licensed use limited to: UNIVERSITY OF TOKYO. Downloaded on March 29,2010 at 01:23:23 EDT from IEEE Xplore. Restrictions apply.

46

IEEE TRANSACTIONS ON HAPTICS,

VOL. 3,

NO. 1, JANUARY-MARCH 2010

we confirmed that the grasping position is correctly
measured and used for controlling the robotic hand.

4.3 Discussion
From the above two examples of grasping operations, we
confirmed that the finger-shaped GelForce can be used inpractice. In these simple operations, we used the fingershaped GelForce as a force vector sensor or force distribution
sensor. We reduced the surface traction fields to the threedimensional force at a point or the center of the force
vectors. Therefore, these two experiments do not show thefull performance of the developed sensor. We believe that it
is possible to perform more complicated operations by
combining this reduced surface traction field or directly
using the surface traction fields. It is a future work to designand conduct an experiment in order to show the full
performance of the developed sensor
Fig. 14. Result of controlling grasping position. Cx and Tz represent thegrasping position and torque of thumb, respectively.

respectively. This figure shows that Fx increase with theapplied force. Furthermore, Fz and Tx increase in proportion to Fx . During this operation, the ball did not move.
Therefore, we confirmed that the force vector was measured
correctly enough to control the grasping force.

4.2.2 Control of Grasping Position
When a robotic hand grasps an object, attention must be
paid to the grasping position on its fingertip. Recognition of
the grasping position enables the robotic hand to control theobject dexterously.
In the operation that involves the control of the grasping
position, the robotic hand grasps the same styrofoam ball asthat used before, as shown in Fig. 12. Then, the robotic handrotates its thumb about the z-axis such that the thumbalternately moves to the right and then to the left along thex-direction. During this movement, the finger-shaped
GelForce measures the grasping position C (Cx ; Cy ) of theball on the thumb. Using (9), C (Cx ; Cy ) is calculated to be
the center of the size of the force vectors.


Cx
Cy



 
5;5
i
1X
fij
¼
;
S i;j
j

S¼

5;5
X

ð9Þ

fij :

5

CONCLUSION

We developed a finger-shaped GelForce as a haptic sensor
for a robotic hand that can be used to measure surface
traction fields. This sensor was developed on the basis oftechnology used in the vision-based haptic sensor GelForce.
Because of the complicated shape of the finger-shaped
sensor, we proposed an observational method in order todevelop a prototype of the sensor. Then, we evaluated thebasic performance of the sensor and conducted a field test
using a robotic hand. The results of the evaluation show thatthe finger-shaped GelForce is applicable to the robotic hand.
The finger-shaped GelForce is a vision-based surface
traction field sensor that has a simple structure andprocessing method. Therefore, the performance of the sensor
can be easily improved using a high quality camera. We
believe that a humanoid robot with the finger-shaped
GelForce will be able to perform dexterous operations inour daily life.

ACKNOWLEDGMENTS
This work had technical support on development of thefinger-shaped GelForce from Nitta Corporation, and work
was partially supported by a grant from the Ministry ofInternal Affairs and Communications SCOPE and Grant-inAid for JSPS Fellows (2010009).

i;j

In this equation, i and j are the numbers of the force
sampling point in Fig. 7, and fij is the size of the force
vector calculated using (8). When the grasping position Cx
exceeds 7.5 mm or is below 2.5 mm, the direction ofrotation of the thumb is inverted. We measure the grasping
position cx and the torque Tz that is responsible for rotating
the thumb.
The results of the abovementioned experiment are
shown in Fig. 14. In this figure, the horizontal and verticalaxes represent operation time and measured value, respectively. When Cx exceeds 7.5 mm or falls below 2.5 mm, Tz is
inverted to move the thumb in the counter direction. This
control ensures that the grasping position is maintained
around the center of the thumb, i.e., at 5.0 mm. Therefore,

REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]

H.R. Nicholls and M.H. Lee, “A Survey of Robot Tactile Sensing
Technology,” Int’l J. Robotics Research, vol. 8, pp. 3-30, 1989.
M.H. Lee and H.R. Nicholls, “Tactile Sensing for Mechatronics—A
State of the Art Survey,” Mechatronics, vol. 9, pp. 1-31, 1999.
R.S. Johansson, A.B. Vallbo, and G. Westling, “Theresholds of
Mechanosensitive Afferents in the Human Hand as Measured
with von Frey Hairs,” Brain Research, vol. 184, pp. 343-351, 1980.
F. Bega-Bermudez and K.O. Johnson, “Differences in Spatial
Acuity between Digits,” Neurology, vol. 56, pp. 1389-1391, 2001.
R.T. Verrillo, “Effect of Contactor Area on Vibrotactile Threshold,”
J. Acoustical Soc. of Am., vol. 35, pp. 1962-1966, 1963.
R.S. Fearing, “Using a Cylindrical Tactile Sensor for Determining
Curvature,” IEEE Trans. Robotics and Automation, vol. 7, no. 6,
pp. 806-817, Dec. 1991.
J.G. Silva, A.A. Carvalho, and D.D. Silva, “A Strain Gauge Tactile
Sensor for Finger-Mounted Applications,” IEEE Trans. Instrumentation and Measurement, vol. 51, no. 1, pp. 18-22, Feb. 1991.

Authorized licensed use limited to: UNIVERSITY OF TOKYO. Downloaded on March 29,2010 at 01:23:23 EDT from IEEE Xplore. Restrictions apply.

SATO ET AL.: FINGER-SHAPED GELFORCE: SENSOR FOR MEASURING SURFACE TRACTION FIELDS FOR ROBOTIC HAND

[8]
[9]

[10]
[11]
[12]

[13]
[14]
[15]

D.T. Pawluk, J.S. Son, P.S. Wellman, W.J. Peine, and R.D. Howe,“A Distributed Pressure Sensor for Biomechanical Measurements,” J. Biomechanical Eng., vol. 120, no. 2, pp. 302-305, 1998.
M. Ohoka, Y. Mituya, K. Hattori, and I. Higashioka, “DataConversion Capability of Optical Tactile Sensor Featuring an
Array of Pyramidal Projections,” Proc. IEEE Int’l Conf. Multisensor
Fusion and Integration for Intelligent Systems, pp. 573-580, 1996.
N.J. Ferrier and R.W. Brockett, “Reconstructing the Shape of a
Deformable Membrane from ImageData,” The Int’l J. Robotics
Research, vol. 19, no. 9, pp. 795-816, 2000.
S. Saga, H. Kajimoto, and S. Tachi, “High-Resolution Tactile
Sensor Using the Deformation of a Reflection Image,” Sensor Rev.,
vol. 27, pp. 35-42, 2006.
K. Kamiyama, K. Vlack, T. Mizota, H. Kajimoto, N. Kawakami,
and S. Tachi, “Vision-Based Sensor for Real-Time Measuring ofSurface Traction Fields,” IEEE Computer Graphics and Applications
Magazine, vol. 25, no. 1, pp. 68-7, Jan. 2005.
W. Menke, Geophysical Data Analysis: Discrete Inverse Theory.
Academic Press Inc., 1989.
L.D. Landau and E.M. Lifshitz, Theory of Elasticity. Butterworth
Heinemann, 1985.
K. Hoshino and Y. Kawabuchi, “Pinching at Finger Tips for
Humanoid Robot Hand,” J. Robotics and Mechatronics, vol. 17,
no. 6, pp. 655-663, 2005.

Katsunari Sato received the BS degree inengineering from Tohoku University in 2006,
and the MS degree in information Science andTechnology from the University of Tokyo in
2008. He has been working toward the PhD
degree at the Graduate School of Information
Science and technology, and a JSPS research
fellow for young scientists since 2008. His
research interests include haptic interfaces fortelexistence. He received the Dean’s Award
from IST, the University of Tokyo, in 2008. He is a student member of
the IEEE.
Kazuto Kamiyama received the BS, MS, and
PhD degrees in Information Science and Technology from the University of Tokyo in 2001,
2003, and 2006, respectively. From 2006 to
2009, he was an assistant professor at the
University of Tokyo. He is currently an assistantprofessor at the University of Electro Communications. His research interests include tactile
sensors and virtual reality.

47

Naoki Kawakami received the MS degree inengineering from Tokyo Institute of Technology
in 1996, and the PhD degree in advanced
interdisciplinary studies from the University ofTokyo in 1999. He was an assistant professor
and a lecturer in the Department of Information
Physics and Computing at the University ofTokyo from 1999 to 2002 and from 2002 to
2009, respectively. His research interest isvirtual reality.
Susumu Tachi received the BE, MS, and PhD
degrees in mathematical engineering and information physics from the University of Tokyo in
1968, 1970, and 1973, respectively. He joined
the Faculty of Engineering at the University ofTokyo in 1973, and in 1975, he moved to the
Mechanical Engineering Laboratory, Ministry ofInternational Trade and Industry, where heserved as the director of the Biorobotics Division. In 1989, he rejoined the University of Tokyoand was a professor in the Department of Information Physics andComputing until March 2009. He is currently a professor emeritus at the
University of Tokyo and a professor of Graduate School of Media Design
at Keio University. From 1979 to 1980, he was a Japanese Government
Award Senior Visiting Scientist at the Massachusetts Institute ofTechnology, Cambridge. His present research covers robotics, virtual
reality and augmented reality with special focus on telexistence(TELESAR and TWISTER), Real-time Remote Robotics (R-Cubed),
and Retroreflective Projection Technology (RPT), including opticalcamouflage. He is a founding director and a fellow of the Robotics
Society of Japan (RSJ), the 46th president and a fellow of the Society of
Instrument and Control Engineers (SICE), a fellow of the Japan Societyof Mechanical Engineers (JSME), and is the founding president of the
Virtual Reality Society of Japan (VRSJ). From 1988, he serves as
Chairman of the IMEKO (International Measurement Confederation)
Technical Committee 17 on Measurement in Robotics. He initiated andfounded ICAT (International Conference on Artificial Reality andTelexistence) in 1991 and IVRC (International-Collegiate Virtual Reality
Contest) in 1993. He is a member of the IEEE VR Steering Committee
and served as general chair of the IEEE VR 2001 and 2002. He is amember of the IEEE.

. For more information on this or any other computing topic,
please visit our Digital Library at www.computer.org/publications/dlib.

Authorized licensed use limited to: UNIVERSITY OF TOKYO. Downloaded on March 29,2010 at 01:23:23 EDT from IEEE Xplore. Restrictions apply.