2008 IEEE International Conference on
Robotics and Automation
Pasadena, CA, USA, May 19-23, 2008

A Decision Method for the Placement of Tactile Sensors
for Manipulation Task Recognition
Kazuya Matsuo, Kouji Murakami, Tsutomu Hasegawa and Ryo Kurazume
Abstract— The present paper describes a decision method
for the placement of tactile elements for manipulation task
recognition. Based on the mutual information of the manipulation tasks and tactile information, an effective placementof tactile elements on a sensing glove is determined. Although
the effective placement consists of a small number of tactile
elements, it has a recognition performance that is as high asthat of a placement consisting of many tactile elements. The
effective placement of tactile elements decided by the proposed
method has been evaluated through experiments involving therecognition of grasp type from grasp taxonomy defined by
Kamakura [1].

I. I NTRODUCTION
A multi-jointed multi-fingered robotic hand [2] is a potentially as dexterous as a human hand [3]. However, it isvery difficult for us to manually and directly write a motion program of multiple fingers moving synchronously andcooperatively to execute a task. Teaching-by-showing is analternative approach: a motion of human fingers is measured
and recognized and is then somehow transformed into a
motion program of a robotic hand. There are two approaches
to teaching-by-showing: direct mapping and symbolic programming.
Generally, there are several differences in structure between the human hand and a robotic hand, including thenumber of fingers, the number of joints, and the dispositionof the fingers. Therefore, it is not straightforward to utilize
the data obtained by human workers in teaching-by-showing
methods. Direct mapping of the joint angle trajectories of
the human hand onto the robotic hand will not generate thesame manipulation task. Another method of teaching is togenerate joint angle trajectories of the robotic hand fromthe fingertip trajectories of the human hand in a Cartesian
coordinate system by solving the inverse kinematics of therobotic hand. However, this technique often fails to maintainthe stable grasp of an object because the possible directions
and magnitudes of the exerted force are not suitable as aresult of the difference in the finger configuration, even if thefingertips of the robotic hand maintain the same positions as
those of the human hand. Force control is therefore requiredin order to overcome error in human motion measurement.
Moreover, some joints may exceed the limits of rotation
K. Matsuo is with the Graduate School of Information Science and Electrical Engineering, Kyushu University, 744 Motooka, Nishi-Ku, Fukuoka
819-0395 JAPAN matsuo@irvs.is.kyushu-u.ac.jp
K. Murakami, T. Hasegawa and R. Kurazume are with the Graduate
Faculty of Information Science and Electrical Engineering, Kyushu University, 744 Motooka, Nishi-Ku, Fukuoka 819-0395 JAPAN {mkouji,hasegawa, kurazume}@irvs.is.kyushu-u.ac.jp

978-1-4244-1647-9/08/$25.00 ©2008 IEEE.

angles. Wang et al. proposed a method of modifying thefingertip positions of the robot according to the limits ofrotation angles when the fingertip trajectories of the human
hand was transformed and processed [4].
A task program for the robotic hand would be autonomously generated if the actual task being executed by ahuman were recognized from the motion data of a human
hand obtained through the teaching-by-showing process.
Related research on human motion recognition has beenreported in [5][6][7][8][9][10][11], and continuous humanmotion is segmented [1][12][13], recognized, and symbolized according to the meaning of the particular motionin the context of the task. A manipulation task by thehuman hand is represented as a sequence of symbols, each
representing a particular manipulation. Adequately abstracted
symbols enable the development of corresponding motion
primitives that can be generated by a robot hand. The robot
system would then be able to autonomously perform various
tasks by executing a sequence of corresponding motion
primitives when a sequence of symbolic descriptions of atask performed by a human is given.
Joint angle trajectories of a human hand are usuallyused for manipulation task recognition. However, recognition
often fails due to the large variation of the joint angle
trajectories when a subject or an object shape is changed.
Therefore, contact information between a hand and an object
is often used to improve the recognition performance ofmanipulation tasks [10][11]. Seki et al. developed a graspingpressure distribution sensor with high flexibility in order tomeasure contact information between a hand and an object[14].
The placement of tactile elements is important whenrecognizing manipulation tasks using contact information.
For example, too many elements would obstruct the human
hand during a manipulation task. In addition, the installationof several elements on a tactile sensing glove is difficult.
The objective of the present study is the development of amathematical decision method for the effective placement oftactile elements for manipulation task recognition. Although
the effective placement consists of a small number of tactile
elements, the method has a recognition performance thatis as high as that of many tactile elements. In previous
research, the placement of tactile elements has been decided
empirically.
The remainder of the present paper is organized as follows.
A decision method of the effective placement of tactile
elements for manipulation task recognition is proposed in
Section II. Section III describes the measurement devices used1641

where xi is an input, y is an output, C is a set of training
data, Xi is the set of xi , vj is a value of xi , n is the numberof vj , Y is the set of y, gain(xi ) is the information gainof xi , H(C) is the information entropy of C, py (C) is theprobability of y in C, and Cij is a subset of C in the caseof xi = vj .
The algorithm of ID3 is explained as follows.
(1) Create a root node ‘N ’ for a tree.
(2) If all the elements of C give the same output, ‘y’, thenlet N be a terminal node labeled by y, and end.
(3) Calculate the information gain of each input ‘xi ’.
(4) Select ‘xk ’ from the inputs so that the information gainof ‘xk ’ be maximized.
(5) Let N be a decision node labeled by xk and createchild nodes ‘Nj ’.
(6) For each child node, Nj → N , Ckj → C, go to (2).

Fig. 1.

Kamakura’s taxonomy of prehension [1].

to obtain the contact information and the joint angle trajectories. Section IV describes the experiments for recognizing
grasp type from grasp taxonomy, as defined by Kamakura(Fig.1). Section V presents the conclusions of the present
study.
II. D ECISION M ETHOD FOR THE P LACEMENT OFTACTILE E LEMENTS

H(C) =
H(C | xi ) =H(C) − H(C | xi )py (C)logpy (C)
−
−

y∈Y
n


|Cij | py (Cij )logpy (Cij )
|C|
j=1
y∈Y

Xi

=

{vj | j = 1, · · · , n},III. M EASUREMENT D EVICES
We use two types of sensors to measure human handmotion, namely, a tactile sensing glove designed by theauthors and a data glove.
A. Contact Information

We propose a decision method for the placement of tactile
elements. First, a human subject performs manipulation tasks
and obtains contact information with the tactile elements
installed on a glove. The decision method then selects aneffective placement of tactile elements using the contact
information.
The proposed method selects an effective placement byusing ID3 (Iterative Dichotomiser 3) [15], which is a kindof supervised learning algorithm. Based on Occam’s razor,
ID3 constructs a small decision tree. ID3 calculates theinformation gain of each input and then makes a decisionnode labeled by the input having the maximum information
gain. The information gain of an input is the expected valueof the information entropy to be given when we decide thevalue of the input. In other words, information gain is themutual information of the outputs and each input and isexpressed as:gain(xi ) =The proposed method constructs a decision tree by using
ID3, where the inputs are the tactile elements and the outputs
are the labels of the recognized manipulation tasks. The setof training data consists of the contact information obtained
through demonstration of the manipulation tasks. The proposed method decides an effective placement for recognizing
the manipulation tasks by using the tactile elements as thelabels for the decision nodes of the tree.

To measure the positions of contact points between a handand a grasped object, we designed a tactile sensing glove. A
total of 160 switches (EVQPLDA15 1.0: Matsushita ElectricIndustrial Corporation) are installed on the glove. A photograph of the glove and the placement of the 160 switches
are shown in Fig.2. The 160 switches are represented bycircles in Fig.2 (b). The circles outside the contour of thehand indicate switches that are distributed on the sides ofthe fingers.
Alternative switches, which output binary data of ‘ON’ or‘OFF’, are used for the glove. The thickness of each switch
is 0.8 mm, and the switches are squares having sides of 5mm. The thickness of the contact mechanism of the switch
is 0.4 mm, and the contact mechanism is circular, having
a diameter of 3.2 mm. When a force of more than 1.0[N] is exerted on the contact mechanism, the switch outputs
the value of ‘ON’. Contact information is 160-dimensional
binary data provided from the 160 switches of the tactile
sensing glove.
B. Joint-angle Information
We use a Cyber Glove (CG1802-R: Immersion Corporation) as an input device for measuring the joint angles of ahuman hand. The appearance and specifications of the glove
are shown in Fig.3 and Table I, respectively. The Cyber

1642

Fig. 4.

(a)Fig. 2.

(b)(a) Photograph of the tactile sensing glove with 160 switches.
(b) Placement of the 160 switches.

Glove can measure the angles of the eighteen joints of ahuman hand. In order to recognize a manipulation task, we
use the angles of sixteen joints, excluding two joints of thewrist. The positions of the sixteen joints are shown in Fig.4.
The proximal joint of the thumb has two DOFs, and the otherfifteen joints have one DOF. The joint-angle information is in
the form of a 16-dimensional vector provided by the Cyber
Glove.

Fig. 3.

Positions of the measured joints of the Cyber Glove.

We collected three data sets, each of which consists ofthe contact information and joint-angle information of the
14 grasp types demonstrated by the subject. The subject
wore the tactile sensing glove over the Cyber Glove while
performing the grasp types. The subject performed eachgrasp type with two objects of different shape. The subject
grasped objects commonly encountered in daily life (Fig.5).
The mass of the objects ranged from 0.9 [g] to 334.9 [g].
The subject reproduced each grasp type 100 times in random
order. Three subjects (Subject-A, Subject-B and Subject-C)
performed the grasping tasks, thus generating three data sets.
The three subjects were males of age 23 to 32 years.

Photograph of the Cyber Glove.

TABLE IS PECIFICATIONS OF THE C YBER G LOVE (CG1802-R).
Number of Sensors
Sensor Resolution
Interface
Maximum Data Rate18
0.5 degrees
RS-232115.2 kbaud

Fig. 5.

IV. E XPERIMENTS

Photographs of the grasped objects.

A. Manipulation Tasks
The goal of the present study is to find an effective
placement of tactile elements in order to recognize manipulation tasks that are frequently executed in daily life,
such as grasping a glass or holding a book. Kamakura, an
occupational therapist, proposed a grasp taxonomy consistingof 14 grasp types (Fig.1) used in daily life [1]. In the present
paper, we used the 14 grasp types defined by Kamakura as
benchmarks for the sensor placement.

Although the thickness and the wiring of each glove may
appear to be cumbersome, we confirmed through visualobservation that the subjects adequately performed the 14
grasp types. Photographs of Kamakura’s grasp types using
the Cyber Glove and the tactile sensing glove are shown in
Fig.6. All of the objects used in the grasping tasks were lightin weight. Therefore, the influence of weight on the sensor
readings is negligible.

1643

PoS

PoH

PoI

PoE

PoD

Lat

Tpd

TVI

TVII

PMF

CMF

Tip

PE

Add

Fig. 6.

Photographs of Kamakura’s grasp types with the Cyber Glove and the tactile sensing glove.

B. Placement Selection Results

subject is shown in Fig.7. The placements consist of 27 to
33 tactile elements.

The method was used to determine the placement of tactile
elements to recognize the 14 grasp types based on the contact
information of each data set. The selected placement for each1644

TABLE IIR ECOGNITION RATES [%] (T HE SEVEN DATA SETS ).

Placement-A(Subject-A)

Placement-B(Subject-B)

Subjects
1 All-Co.

2 Key-Co.

3 NKey-Co.

4 An. + All-Co.

5 An. + Key-Co.

6 An. + NKey-Co.

7 An.


Placement-C(Subject-C)

Fig. 7. Effective placements of tactile elements for recognizing the 14
grasp types.

1

2

3


4

5

6


7


Contact information from all of the elements(All-Co.).
Contact information from the selected elements(Key-Co.).
Contact information from all the non-selected elements(NKey-Co.).
Joint-angle information and contact informationfrom all of the elements (An. + All-Co.).
Joint-angle information and contact informationfrom the selected elements (An. + Key-Co.).
Joint-angle information and contact informationfrom all non-selected elements(An. + NKey-Co.).
Joint-angle information without contact information (An.).

The recognition rates are shown in Table II. The recognition rate is defined as the ratio of the number of successful
data to the number of input data.
The results of the evaluation are summarized as follows.
1
4 :
7
• From input data 
Although the recognition rates obtained by using thejoint-angle information without the contact information
7 or the contact information from all of the elements
1 are approximately 90%, the recognition rates obtained by using both the joint-angle information and the
4 are approximately 100%. Thus,
contact information 
sensor fusion improves the recognition performance.

B
90.0
90.0
67.9
100
100
100
100

C
88.8
88.7
69.1
99.9
99.9
89.9
84.9

1
2 :
3
From input data 
The recognition rates obtained by using the contact
2 are as highinformation from the selected elements 
as those obtained by using the contact information from
1 . On the other hand, the recognition
all 160 elements rates obtained by using the contact information from
3 are reduced. Although the
all non-selected elements selected placements consist of small numbers of tactile
elements, they have recognition performances that are
as high as the rate for many tactile elements.
4
5 :
6
• From input data 
Similar results are obtained when contact information
4
5
6 .
is integrated with joint-angle information
•C. Evaluation of the Selected Placements of Tactile Elements
The selected placements were evaluated through experiments involving the recognition of the 14 grasp types.
We investigated the recognition performance of the selected
placements using the LogitBoost algorithm [16]. The algorithm was implemented using Weka [17], which is acollection of machine learning algorithms for data mining
tasks. Decision stumps were applied as weak learners of thealgorithm. The number of weak learners was 100. Weka’s
default values were applied as parameters of the learning
process. For comparison, we prepared seven different data
sets as inputs of the algorithm. We evaluated the selected
placements through four-fold cross validation on the sevendata sets. The seven data sets are as follows.

A
88.4
88.4
64.6
100
100
91.4
90.0

D. Generality of the Selected Placements of Tactile Elements
In order to evaluate the generality of the selected placement, we investigated the recognition rate of one subject
using the placement of another subject. Table III shows therecognition rates obtained when the evaluation data of one
2 are recognized using the placement of anothersubject subject.
TABLE IIIR ECOGNITION RATES [%]:
T HE EVALUATION DATA ( COLUMN ) OF ONE SUBJECT ARE RECOGNIZEDBY USING THE PLACEMENT ( ROW ) OF ANOTHER SUBJECT.

hhhh evaluation datahhhhhhh
placements
Placement-A
Placement-B
Placement-C

A

B

C

88.4
85.4
85.6

89.6
90.0
89.6

81.6
87.7
88.7

The recognition rates of the evaluation data of each subject
using the placement of the other subjects are lower by 0.47.1 [%] than that obtained using the placement of the same
subject.
E. Comparison of the Effective Placements of Tactile Elements and Locations of Human Mechanoreceptive Units
We compare the selected placements of tactile elementswith the locations of mechanoreceptive units in the humanhand. When the densities of two types of mechanoreceptive
units (FAI and SAI units) are high, the spatial resolution
capacity in the human hand is large [18]. Therefore, we compared the selected placements (Fig.7) with the locations of
the FAI and SAI units [19] (Fig.8). The selected placements
were found to be similar to the locations of the two types ofmechanoreceptive units.

1645

FAI

SAI

Fig. 8. Locations of the receptive field centers of two types of mechanoreceptive units (FAI and SAI units) [19].

V. C ONCLUSIONS
We have developed a decision method for the placementof tactile elements for manipulation task recognition. The
LogitBoost algorithm recognized 14 grasp types from a grasp
taxonomy defined by Kamakura based on the placement oftactile elements decided by the proposed method. Although
the placements consist of only 27-33 tactile elements, theelements have a recognition performance that is as high asthat of a placement consisting of 160 tactile elements.

[11] M. Kondo, J. Ueda, Y. Matsumoto and T. Ogasawara: “Perception of
Human Manipulation Based on Contact State Transition”, Proceedingsof the 2004 IEEE/RSJ International Conference on Intelligent Robots
and Systems, pp. 100-105, 2004.
[12] J. R. Napier: “The prehensile movements of the human hand”, The
Journal of Bone and Joint Surgery, 38B, 4, pp. 902-913, 1956.
[13] M. R. Cutkosky: “On grasp choice, grasp models, and the Design ofhands for manufacturing tasks”, IEEE Transactions on Robotics andAutomation, Vol. 5, No. 3, pp. 269-279, 1989.
[14] Y. Seki, S. Sato, M. Shimojo and A. Takahashi: “Developmentof a New Hand-Grasp Measurement System”, Proceedings of HCIInternational, 20B, pp. 791-796, 1995.
[15] J. Ross Quinlan: “Discovering Rules by Induction from Large Collections of Examples”, In Expert Systems in the Micro-Electronic Age,D. Michie, Editor, Edinburgh University Press, Edinburgh, UK, pp.
168-201, 1979.
[16] J. Friedman, T. Hastie and R. Tibshirani: Additive Logistic Regression:a Statistical View of Boosting, The Annals of Statistics, Vol. 28, No.
2, pp. 337-374, 2000.
[17] Ian H. Witten and Eibe Frank: “Data Mining: Practical Machine
Learning Tools and Techniques with Java Implementations”, Morgan
Kaufmann Publishers, 1999.
[18] A. B. Vallbo and R. S. Johansson: “Properties of cutaneous mechanoreceptors in the human hand related to touch sensation”, Human Neurobiology, Vol. 3, pp. 3-14, 1984.
[19] R. S. Johansson and A. B. Vallbo: “Tactile sensibility in the human
hand: Relative and absolute densities of four types of mechanoreceptive units in glabrous skin”, Journal of Physiology, Vol. 286, pp.
283-300, 1979.

ACKNOWLEDGMENTS
The authors thank Tsuyoshi Yokoya for his assistance withthe data acquisition.
R EFERENCES[1] N. Kamakura et al.: “Patterns of static prehension in normal hands”,
The American Journal of Occupational Therapy, Vol. 34, No. 7, pp.
437-445, 1980.
[2] H. Kawasaki, H. Shimomura and Y. Shimizu: “Educational-industrial
complex development of an anthropomorphic robot hand ‘Gifu hand”’,
Advanced Robotics, Vol. 15, No. 3, pp. 357-363, 2001.
[3] A. Bicchi: “Hands for Dexterous Manipulation and Robust Grasping:
A Difficult Road Toward Simplicity”, IEEE Transactions on Robotics
and Automation, Vol. 16, No. 6, pp. 652-662, 2000.
[4] H. Wang, K. H. Low, M. Y. Wang and F. Gong: “A Mapping Method
for Telemanipulation of the Non-Anthropomorphic Robotic Hands
with Initial Experimental Validation”, Proceedings of the 2005 IEEE
International Conference on Robotics and Automation, pp. 4229-4234,
2005.
[5] S. B. Kang and K. Ikeuchi: “Toward Automatic Robot Instruction from
Perception - Mapping Human Grasps to Manipulator Grasps”, IEEE
Transactions on Robotics and Automation, Vol. 13, No. 1, pp. 81-95,
1997.
[6] T. Mori, Y. Segawa, M. Shimosaka and T. Sato: “Hierarchical Recognition of Daily Human Actions Based on Continuous Hidden Markov
Models”, Proceedings of the 6th IEEE International Conference on
Automatic Face and Gesture Recognition, pp. 779-784, 2004.
[7] K. Ogawara, J. Takamatsu, H. Kimura and K. Ikeuchi: “Generation ofa task model by integrating multiple observations of human demonstrations”, Proceedings of the 2002 IEEE International Conference on
Robotics and Automation, pp. 1545-1550, 2002.
[8] J. Aleotti and S. Caselli: “Grasp Recognition in Virtual Reality for
Robot Pregrasp Planning by Demonstration”, Proceedings of the 2006
IEEE International Conference on Robotics and Automation, pp. 28012806, 2006.
[9] S. Ekvall and D. Kragic: “Grasp Recognition for Programming by
Demonstration”, Proceedings of the 2005 IEEE International Conference on Robotics and Automation, pp. 760-765, 2005.
[10] Keni Bernardin, Koichi Ogawara, Katsushi Ikeuchi and Ruediger
Dillmann: “A Sensor Fusion Approach for Recognizing Continuous
Human Grasping Sequences Using Hidden Markov Models”, IEEE
Transactions on Robotics, Vol. 21, No. 1, pp. 47-57, 2005.

1646