Improvement of Shape Distinction by Kinesthetic-Tactile Integration
Katsunari Sato*

Hiroyuki Kajimoto† Naoki Kawakami* Susumu Tachi*(*)The University of Tokyo, JPN(†)The University of Electro-Communications, JPN
E-mail: Katsunari_Sato@ipc.i.u-tokyo.ac.jp, kajimoto@hc.uec.ac.jp,{kawakami, tachi}@star.t.u-tokyo.ac.jp
Abstract

The sensation of touch is divided into kinesthetic
and tactile sensation. It is important for a touchsensation device to present both types of sensations.
We mounted an electrotactile display on a kinestheticdisplay, and realized a compact display of touchsensation. This system can improve the ability toidentify objects. In this paper, we evaluate the effect ofintegrating kinesthetic and electrotactile sensations foridentifying the shape of an object.

1. Introduction
The sensation of touch can be divided into two
types based on the receptor that acquires the sensory
information. One type is kinesthetic sensation, which isacquired by the proprioceptors that exist in the muscle,tendon, and joint. The other type is tactile sensation,
which is acquired by mechanoreceptors that exist at a
depth of several millimeters from the skin surface.
The sensation of touch is important for virtual
reality environments or telepresence technologies.
When we do work that requires us to remain in contact
with an object, the presentation of touch sensation
significantly influences our working efficiency. Basedon the characteristics of human perception, it would beappropriate to present both types of touch sensation.
However, many conventional displays of touchsensation present only kinesthetic sensation. To
improve actual sensation and the working efficiency, it
is necessary to simultaneously present tactile sensationas well.
This concept is realized by mounting a tactile
device on a kinesthetic device. A number oftechnologies can be employed to achieve such a tactile
device. The mechanical tactile display is one suchtactile device [2] [3] [7]. This display drives a pin
matrix and presents tactile sensation by directly

deforming human skins. Another type of device is theelectrotactile display [4], which directly stimulates thenerves connected to the mechanoreceptors.
Many studies have investigated the use of thesetactile devices along with a kinesthetic device. Typicalexamples include a mechanical tactile display mounted
on a kinesthetic display [1] [5] [10]. In anotherexample, Methil et al. [8] used an electrotactile display
to present tactile sensation. Some studies showed that
presenting both force and tactile sensations canimprove the working efficiency. However, the systems
that accomplish this are too large to be used as aninterface for general-purpose work in virtual reality ortelepresence environments. Therefore, a small andlight-weight system is desirable.
The purpose of this study is to construct a compact
interface that is capable of presenting both kinesthetic
and tactile sensations. In order to achieve this, we
mount an electrotactile display on a kinesthetic display(Figure 1). This system enables us to distinguish theshape of an object. In this paper, we evaluate theefficiency of the electrotactile display to distinguishthe shape of an object.

Figure 1. Conceptual diagram of the interface

field like stimulus with a reactive force presents thefeeling that a plane of the object has been touched.

2. Proposed method
When we hold an object in our hand, we try toascertain its properties such as its size, shape, weight,position, and posture. Recognizing these properties
improves the efficiency of working with the object.
However, in order to recognize these properties, werequire both kinesthetic and tactile sensations to bepresented to us.
The conventional kinesthetic display presents only
the reactive force sensation to its users, and makes us
perceive a global shape. On the other hand, theelectrotactile display makes us perceive a local shape.
The electrotactile display consists of pin electrodes(Figure 2). Each pin electrode sends current inside theskin surface and directly stimulates the nerves that areconnected to the mechanoreceptors (Figure 3). In thismanner, the electrotactile display can present 2dimentional touch sensation, touch position anddistribution.

Figure 2. Electrotactile display (Electrodes)

kinesthetic
sensation

electrotactile
sensation

virtual touchsensation

Figure 4. Virtual touch sensation
The advantage of using an electrotactile display isthat a compact system can be constructed. The
electrotactile display does not require any actuators.
Therefore, it can be fabricated with a size and weightthat are smaller and lighter, respectively, incomparison to a mechanical tactile display. The
compact tactile display is suitable for future integration
with a kinesthetic display.
In the following section, we will confirm theefficiency of integrating an electrotactile sensation
with a kinesthetic sensation. It is considered that thisintegration will simplify object shape recognition.
Further, we evaluate the shape detection efficiency ofthe electrotactile display.

electrotactile display
electrode

3. Experiment 1

mechanoreceptor
nerve
inside the skin
Figure 3. Electrostimulus
Reconstructed from [4]We hypothesize that by integrating these twodisplays, many virtual touch sensations can beachieved (Figure 4). For example, line stimulation witha reactive force presents the feeling that an edge of theobject has been touched. On the other hand, a uniform

3.1. Material and Method
In this experiment, participants were asked to touch
an object and identify it. First, in order to familiarize
the participants with the experimental procedure, they
were made to touch all the objects while conducting
electrical stimulation (ES). Then, the objects werepresented randomly. The participants touched theobject according to the directed procedure andattempted to identify the object. During the experiment,
the participants were not allowed to see the objects.
Each object was displayed five times in a single trial.
Next, the manner of touching and the condition of ES
were changed. The number of correctly recognizedobjects and the time required for recognition wascompared.
Figure 5 shows the experimental setup. The
participants wore a plastic finger case on their fingertip
when they touched the object. The electrode plate usedfor electrotactile display was in the finger case (Figure6). Therefore, they were unable to feel any tactile
sensation other than the ES or the pressure of thefinger case. The thickness of the bottom of the finger
case was 2 mm.
electrotactile display

objectfinger casefinger case

6-axismechanicallink sensor

6-axis mechanicallink sensor

objectFigure 5. Experimental environment

1.25 mmfinger caseused as an electrical ground. The diameter of eachelectrode was 1.25 mm, and the distance between thecenters of the electrodes was 2.5 mm. The pulse widthwas 40 µm, and the pulse frequency was 60 Hz. The
strength of the stimulus could be controlled by theparticipants.
The position of the fingertip was measured by using
a 6-axis mechanical link sensor (ADL-1, Shooting Star,Inc.). Based on the position information, contact
between the finger and object was judged, and thetactile pattern was determined. For example, when all
the bellies of the fingertip are in contact with a flat
surface, a current is sent from all the pin electrodes.
When the middle of the fingertip contacts an edge, themiddle line of pin electrodes matrix sends a current.
The ES is presented when the fingertip made contacts
with the object.
It must be noted that in this setup, “real” kinesthetic
sensation was presented by actual contact, and tactile
sensation was presented by using the object’s VR
model in a PC. This condition is a type of “mixedreality.”
It is difficult to prepare all the shapes of a 3D object.
Therefore, we selected some typical shapes asrepresentative of 3D objects, and conducted
experiments using them. An identical method was usedby Shimojo et al.[9]. For this preliminary experiment,
we prepared three objects with the following
characteristics: flat surface, curved face, and edge
(Figure 7). The objects were prepared from plastic
resin. It is difficult to touch and recognize a large
object using the index finger. Therefore, wedetermined the size of the object from the fingertip
moving range of the index finger when fixing the root.

5 mm
20 mm

17 mm
2.5 mm
12 mm

electrotactile
display

Figure 6.
(Left) Electrodes for electrotactile display(Right) Finger case
The size of the electrode plate used in thisexperiment was 12 mm × 17 mm and it consisted of a
3 × 5 electrode matrix. The surrounding electrode wasflat surface

curve

edge

Figure 7. Objects that participants touched(Above) Dimensions, (Below) picturescorrect answer ratio (%)

In the case of shape recognition, the manner oftouching influences the accuracy and time ofrecognition. In this experiment, we divided the mannerof touching into two modes – pushing and tracing. One
was pushing and another sliding (Figure 8). In thepushing mode, the participants only pushed the object
vertically. In the tracing mode, the participants werepermitted to slide their fingertip horizontally and trace
the surface of the object.

Figure 9 and Figure 10 show the experimental resultof all participants. The horizontal axes represent theabovementioned experimental conditions. The verticalaxes represent the correct answer ratio or recognition
time. The recognition time is the average time required
to recognize one object. Each bar represents the resultfor one of the participants. Each horizontal line
represents the average of participants in eachexperimental condition.
100ABCDEave.

806040200(a) pushing1234

experiment condition

Figure 9. Correct answer ratio(b) tracing
Figure 8. Two modes of touching
The participants included four males and one
female in the age group of 22-25. They were healthy
graduate students volunteered for the experiments.
They were asked to name the object they weretouching (from three choices). The experimentconditions were as follows:
1. Pushing with ES
2. Pushing without ES
3. Tracing with ES
4. Tracing without ES
A total of four trials was designated as 1 set and eachparticipant performed three sets.

3.2. Results

recognition time (s)10ABCDEave.

864
2012

3

4

experiment condition

Figure 10. Recognition time

3.3. Discussion
From the result, it was confirmed that a correct
answer ratio with ES was higher than that without ES,
and a recognition time with ES was shorter than thatwithout ES. Further, this was hardly related to theparticipant and mode of touching. Therefore, it wasinferred that the integration of the electrotactile display
improves the efficiency of shape recognition. However,4. Experiment 2
4.1. Material and Method

2. Touching curved face with edge ES
3. Touching edge with curve ES
4. Touching edge with edge ES
In this experiment, these four trials is one set, and two
sets were performed. Based on the kinesthetic andtactile sensations perceived by participants, theyidentified the object type as “curved face” or “edge.”
In this experiment, participants had to answer within 5s. Other experiment environment conditions in theexperiment were the identical to those in the previousexperiment.

4.2. Results
The average response ratio of the “curve” is shownin Figure 11. For example, if participant always
recognized the object as a curved face, in accordancewith condition 1, the response ratio becomes 1.0. The
horizontal axis represents the experimental conditions.
The vertical axis represents the response ratio of the“curve.” Each dot represents the participants’ answer
ratio, and each bar represents their average. The error
bar represents the standard error.

1
response ratio of"curved face"

this was a preliminary experiment, and hence, only
three different shapes were used, and the size of theobject was also fixed. In the future, we will investigate
in greater detail.
From the differences arising due to the experimentconditions, it can be seen that the recognition time wasshort under condition 2 as well as the condition 1. In
this experiment, we asked participants to answer theshape correctly and quickly. Therefore, someparticipants answered a shape before they recognized
an object completely.
Under condition 4, the participants were able torecognize the object almost correctly even without ES.
The kinesthetic sensation presents the global shape,and shape recognition efficiency is improved bymoving the finger[6]. Therefore, it is considered thatthe kinesthetic sensation is sufficient to recognize anobject in this condition.
If the kinesthetic sensation is sufficient to recognize
an object, why is the efficiency of shape recognition
improved by integrating electrotactile stimulus? As acontradiction to this question, let us assume that theelectrotactile sensation is a transient stimulus. In vision
sensation, a human being is said to be sensitive tochanges in his surroundings. Such sensitivity occurs
with touch sensation as well. It is considered that theparticipants reacted sensitively to the electrotactile
stimulus, and the shape recognition efficiency wasimproved. In the next section, we will investigate thishypothesis.

ave.
AB
C
D
E

0.8
0.6
0.4
0.2
01

If the electrotactile sensation is a transient stimulus,
a number of shape sensations are presented by theelectrotactile stimulus. For example, when thekinesthetic display presents a sensation of an “objectwith an edge” while the electrotactile display presents
that of a “curved object”, a human being would
perceive the latter. In this experiment, we investigate
the participant’s responses presenting stimulation that
confirmed kinesthetic or electrotactile sensation.
The participants traced the object surface in themanner shown in Figure 8 (b). The objects they
touched are an edge and a curve (Figure 7). These areselected as a special example of an identical form. Two
stimulation modes were tested for electrical stimulation.
The first mode stimulated a “curvature”; the second, an“edge”. The experiment conditions were as follows.
1. Touching curved face with curve ES

234experiment situation

Figure 11. Response ratio of “curves”

4.3. Discussion
In this experiment, there was a tendency that theparticipants respond to the object judging from the ES.
Therefore, the response ratio of the “curve” is highunder condition 1 and 3 and low under 2 and 4. This
result supports the hypothesis that the electrotactile
sensation is a transient stimulus. Therefore, it issuggested that the electrotactile stimulus is efficient inpresenting shape sensation, and the result ofexperiment 1 is natural. In addition, we would suggestthat any touch sensation using a typical object shape
can be presented by integrating an electrotactile
display.

5. Conclusion
In this paper, we proposed a touch sensation displaythat presents both kinesthetic and electrotactile
sensations. It is considered that this system will
improve the efficiency of object shape recognition andevaluate it. The results revealed that electrotactile
display can improve the ability to distinguish three
simple objects. Furthermore, we showed the possibility
that the electrotactile stimulus is efficient forrepresenting shape sensation.
In future studies, we will evaluate the efficiency ofintegrating the electrotactile sensation with kinestheticsensation in greater detail. We acquired theinformation on the improvements in the shaperecognition by conducting experiments with threeobjects. Out next step is to experiment with more typeof objects and changing the object size. From theseexperiments, we will clarify the properties required toconstruct the kinesthetic-tactile integrated display andthe possibility of integration. Finally, we hope
construct a simple and compact tactile display.

References[1] M. Fritschi, M.O. Ernst, and M. Buss, “Integration ofKinesthetic and Tactile Display – A Modular DesignConcept”, In Proceedings of the EuroHaptics 2006,Paris, France, 2006.
[2] T. Fukuda, H. Morita, F. Arai, H. Ishihara, and H.
Matsuura, “Micro Resonator Using Electromagnetic
Actuator for Tactile Display”, International Symposiumon Micromechatronics and Human Science, IEEE,
pp143–148, 1997.
[3] C.J. Hasser, J.M. Wesenberger, “Preliminary
Evaluation of a Shape-memory Alloy Tactile Feedback
Display”, In DSC-Vol. 49, Advances in Robotics,
Mecha-tronics, and Haptic Interfaces, pp.73–80, 1993.
[4] H. Kajimoto, N. Kawakami, T. Maeda and S. Tachi,“Electro-Tactile Display with Tactile Primary ColorApproach”, In Int. Conf. on Intelligent Robots andSystems(IROS), 2004.
[5] Y. Kim, I. Oakley, and J. Ryu, “Combining Point Force
Haptic and Pneumatic Tactile Displays”, In
Proceedings of the EuroHaptics 2006, Paris, France,
2006.

[6] R.L. Klatzky, J.M. Loomis, S.J. Lederman, H. Wake,and N. Fujita, “Haptic identification of objects andtheir depictions”, Perseption and Psychophysics 54,
pp.170-178, 1993
[7] M. Konyo, S. Tadokoro, “Artificial Tactile Feel
Display Using EAP Actuator”, Worldwide Electro
Active Polymers, Artificial muscles, Newsletter, Vol. 2,No. 1, Jul 2000.
[8] N.S. Methil, Y. Shen, D. Zhu, C.A. Pomeroy, R.
Mukherjee, N. Xi, and M. Mutka, “Development ofsupermedia Interface for Telediagnostics of BreastPathology”, In IEEE International Conference on
Robotics and Automation, Orlando, Florida, 2006.
[9] M. Shimojo, M. Shinohara, and Y. Fukui, “HumanShape Recognition Performance for 3-D Tactile
Display”, In IEEE Transaction on systems, man, andcybernetics – Part A: Systems and Humans, Vol, 29,
No. 6, November 1999.
[10] C.R. Wagner, D.P. Perrin, R.L. Feller, R.D. Howe, O.
Clatz, H. Delingette, and N. Ayache, “Integrating
Tactile and Force Feedback with Finite Element
Models”, In IEEE International Conference on
Robotics and Automation, pp. 1–10, Barcelona, Spain,2005.