"©2006 IEEE. Personal use of this material is permitted. However, permission toreprint/republish this material for advertising or promotional purposes or for
creating new collective works for resale or redistribution to servers or lists, or toreuse any copyrighted component of this work in other works must be obtained
from the IEEE."

Proceedings of the 2006 IEEE/RSJ
International Conference on Intelligent Robots and Systems
October 9 - 15, 2006, Beijing, China

Real-time Estimation of Touch Feeling Factors
Using Human Finger Mimetic Tactile Sensors
Shogo Okamoto∗, Masashi Konyo∗ , Yuka Mukaibo∗∗, Takashi Maeno∗∗ , and Satoshi Tadokoro∗∗

Graduate School of Information Sciences, Tohoku University
6-6-01 Aramaki Aza Aoba, Aoba-ku, Sendai, 980-8579, Japan{okamoto,konyo,tadokoro}@rm.is.tohoku.ac.jp∗∗

Department of Science and Technology, Keio University
3-14-1 Hiyoshi, Kohoku-ku, Yokohama 223-8522, Japanmukabo@mmm-keio.net, maeno@mech.keio.ac.jp

Abstract— To realize a telepresence system with tactile feedback and force feedback, real-time estimation of various tactile
sensation must be conducted. Because several types of tactile
sensation consist in touch and human tactile feeling has hightime resolution. A man feels active touch extraordinary withtime delayed transmission of tactile information. Our proposing
human ﬁnger mimetic sensor covers three tactile factors, which
are roughness, softness and friction of objects to touch. Current
tactile telepresence systems represent just one tactile sensationin addition to kinesthetic information. For augmented reality,
wider tactile factors must be sensed at a tactile sensor system andtransferred to a tactile display system with small time delay. We
realized quick estimation of vibrational frequencies of the sensor
and softness of objects to regenerate touch feelings to human skinby tactile displays, which usually need time-consuming samples
and make it hard to address tactile telepresence system. Quick
estimation of vibrational frequency was conducted by emulating
impulse emission of Meissner’s corpuscles. Quick estimation ofYoung’s modulus of objects was solved by computing strain
distribution in a sensor.

I. I NTRODUCTION
Tactile telepresence system is expected to realize the palpation in minimally invasive diagnosing, training from experts ofﬁne art by sharing tactile sensation and robotic manipulation
like grasp with small grip force and automation of the otherindustries which needs human tactile perceptions and handling.
Studies of tactile telepresence systems which include all ofsensing, transmission and representation of tactile sensation
have been seen for some purposes. Tactile telepresence system
consists of a tactile display and a tactile sensor. In addition toconventional telepresence systems with kinesthetic feedback,tactile telepresence system regenerates tactile information likeroughness, friction sensation and softness of objects by tactile
displays based on estimated tactile factors by a tactile sensor.
Robotic palpation systems [1] [2] were aimed at locating
nodules on organs and able to represent a pressure distribution
to a ﬁngertip. A master-slave system to transmit roughness
sensation in rubbing motion was reported [3]. Control ofa contact area between a ﬁngertip and a tactile display to
distinguish objects of different Young’s modulus based on asensed or estimated contact area were reported [4] [5]. All ofthese telepresence system sensed tactile factors of objects and1-4244-0259-X/06/$20.00 ©2006 IEEE

installed tactile displays to present transferred factors, neither
in a virtual way nor just force representation. But existing
telepresence systems cover only one tactile sensations, like apressure distribution, roughness or softness of objects to touch
and objects’ properties must be identiﬁed beforehand because
their estimation requires time, which complicates practical use.
There are no tactile sensors to cover the whole various human tactile perceptions. So far have reported studies of sensingof pressure distributions in a ﬁngertip-size area, detection of
incipient slippage [6] and widely distributable pressure sensors
for robots’ skin [7] and so on. Because they can be consideredmost important tactile information for robotic assembly, grasp,
handling of the objects or home robots as partners of humans.
Contemporary tactile sensors are far from adequate for humanto obtain objects’ texture which is classiﬁed into four factors
on material determination as roughness, temperature, friction
and softness of objects [8]. In this paper physical parametersthat could cause these four types of sensation are named tactile
factors. The tactile factors are sensed or estimated individuallyat a tactile sensor system and transfered to a tactile display
system.
High time resolution of human tactile sensation must becontemplated. Human recognition of tactile information havequick time response of less than 1[msec]. Quick estimation andtransmission of tactile factors are also required to implementtactile telepresence systems. If not for quick response when aman rubbed objects, it would lack reality and unnatural feelingwould be sensed to him.
Hence, necessary conditions of tactile telepresence system
are listed as;
1. tactile display to present various tactile factors,
2. tactile sensor to obtain various tactile factors and3. quick estimation of tactile factors.
We have already overcome condition 1. via ICPF tactile
display [9] and condition 2. and 3. are achieved by humanﬁnger mimetic tactile sensor [10] using methods described inthis paper.
ICPF tactile display is formed by array of ICPF actuators, which can stimulate man ﬁnger skin and cause internal
deformation to the skin. ICPF actuators can be performed3581

in enough of broad frequency (0 – 200[Hz]) to correspondto three kinds of tactile mechanoreceptors, Merkel’s discs,
Meissner’s Corpuscles, Pacinian Corpuscles. We proposed themethod to synthesize three tactile factors, softness, roughness
and friction by ICPF tactile display [9].
Human ﬁnger mimetic tactile sensor emulates the physicalstructure of a man ﬁnger. This sensor covers three tactile sensations, roughness, softness and friction of objects. Roughness
sensation can be thought to compose of the vibratory frequency and intensity of applied stimulus to the skin. Especiallyvibrational frequencies take time to be computed by Fouriertransformation and identiﬁcation of Young’s modulus usulally
needs time series relationships between reaction force andnormal displacement. New methods for real-time estimationof vibrational frequency and softness are needed to satisfy thecondition 3.
In this paper, ﬁrstly the basic speciﬁcations of human ﬁnger
mimetic tactile sensor is described, followed by real-time
estimation of vibrational frequency of the sensor by emulating
Meissner’s ccorpuscles’ impulse emission, followed by realtime estimation of Young’s modulus using strain distribution
in a sensor are described.
II. H UMAN F INGER M IMETIC TACTILE S ENSOR [10]Human ﬁnger mimetic tactile sensor emulates physicalstructure of a man ﬁnger. The idea of this sensor is that humanmimetic sensor may be able to sense various tactile factors
quick like humans although its precision may not be accurate.
Fig.1 illustrates a schema of the sensor. The sensor is madeup of a hard silicon rubber outer layer, a soft inner layer andan acrylic base as a bone of a human ﬁnger. The surface of asensor is covered with lots of small lumps emulating dermal
ridges of a man ﬁnger. Five strain gauges are embedded andeach of them is placed on the bound between an outer layer
and an inner layer, just as Meissner’s Corpuscles are found on
the bound between soft fat tissue and a hard outer epidermis
of a man ﬁnger. It is also known that when force is applied
to the surface of a sensor stress distribution is concentrated
around this boundary region most likely.
So far we veriﬁed that the frequency of sensor tissue’s
vibration could be computed by Fourier transformation whena sensor is rubbed against a rough object at a ﬁxed speed, anda variance of the ﬁve gauges’ output reﬂects softness of theobject to touch. And also static and dynamic friction can beobtained by a torque sensor installed on the base of a sensor[10].

Fig. 1.

Human ﬁnger mimetic sensor [10]III. R EAL - TIME E STIMATION OF V IBRATIONALF REQUENCY U SING H UMAN F INGER M IMETIC S ENSOR
Most of the existing tactile displays generate vibratory
stimulus to the skin of a ﬁnger to represent roughness sensation[12]. Therefore tactile sensor must determine the frequencyof vibration to transfer. But generally computing frequency
elements by Fourier transformation takes so long time thatit makes tactile telepresence system lack the reality. Shorttime Fourier transformation (STFT) is a promising approachconcerning this purpose, but we conﬁrmed that STFT can notdeal with active touch because of some problems (elaborated
later). The vibrational frequency corresponding to roughness
sensation (0–100[Hz]) must be estimated as quick as possible
in other ways. Human ﬁnger mimetic tactile sensor may havethe capability of estimating vibrational frequency quickly byusing biomimetic algorithm.
Ohka’s method successfully detected a step of tens of micrometer using a tactile sensor for robots, which implied thatthe methods of mimicking human mechanism for roughness
recognition is also effective for robotic tactile sensors [11].
Ohka’s method did not mean to estimate the vibrational frequency of stimulus. This section describes real-time estimationof vibrational frequency by the method of mimicking impulse
emission of Meissner’s corpuscles.
A. Real-time Vibrational Frequency Estimation by the methodof Mimicking Impulse Emission of Meissner’s Corpuscles
Some physiological aspects are taken into account for
real-time vibrational frequency estimation by our methods.
Meissner’s corpuscles is said to affect on roughness recognition of humans. Meissner’s corpuscles responds to ﬁrst-orderdifferential of mechanical stimulus given to the skin. Impulse
is emitted by a neuron connected to Meissner’s corpuscles withtemporal stimulus change bigger than a certain threshold. If
another impulse is emitted before preceding one attenuatescompletely, membrane potential is superimposed onto an old
signal. This effect is called time summation [11].
Next phenomena caused when a human ﬁnger mimetic
tactile sensor rubs a roughness sample are elaborated. Fig.2 (a)shows output of a strain gauge, when the sensor horizontallyrubbed a roughness sample at a speed of 60[mm/sec]. There
were two grooves with width 1.5[mm] on a roughness sample.
The interval between two grooves was 1.5[mm]. In Fig.2 (a)when the sensor touched the ﬁrst groove its output abruptly
increased and dropped down on contacting a ridge. On passing
the second groove sensor’s output changes as well as on
passing the ﬁrst groove.
Fig.2 (b) shows ﬁrst-order differential value of sensor’s
output (a). There were four spikes in Fig.2 (b) and each ofthem was caused by begining of a groove or contact with aridge. Fig.2 (c) shows the emitted impulses when the ﬁrstorder differential values crosses over a threshold. The timeinterval between impulses tells vibrational frequency. The timeinterval between two impulses caused by two grooves was
0.050[sec]. A sensor moved at the speed of 60[mm/sec], 60[mm/sec] × 0.050[sec]= 3.0[mm], which equals a distance of3582

In an ideal condition impulse emission happens two times
for each groove on a sample, one emission should be emitted
when a sensor enters a groove, the other one should be emitted
when a sensor contacts a ridge. The vibrational frequency ofa sensor f (t) is (3).
c(t)
(3)
2a
Spatial wavelength of the object’s surface to touch at t,λ(t), is estimated as (4) where v(t) is velocity of sensor’s
rubbing motion. This estimation of λ(t) is important to realize
tactile telepresence systems cause communication band width
between a slave side and a master side is not always insured
in a real environment. Even when f (t) is not transmitted froma slave side in a certain instant, a tactile displya system candetermine the vibrational frequency to represent to an operator
based on λ(t) and the velocity of touch motions.
f (t) =

λ(t) =

Fig. 2. Output of a strain gauge and ﬁrst-order differential value of it when
the human ﬁnger mimetic tactile sensor rubbed a roughness sample

two grooves. In this case the vibrational frequency of a sensor
is computed as 1/0.050=20[Hz]. Same story made sense whenusing impulses resuled from contacts with ridges. This is thebasic principle to instantly estimate the vibrational frequencyof the sensor.
Real-time estimation of vibrational frequency is formulated
next. When (t) is the output of a strain gauge, x(t) denotes
absolute value of the ﬁrst-order differential value of (t) as(1).
 d(t) (1)
x(t) = 
dtp(t) denotes pulse emission at t. When x(t) crosses over a
threshold h, p(t) is set to p(t) = 1, which means impulse isemitted, in other cases p(t) = 0. The condition of p(t) = 1 is
not x(t) > h. Only if x(t) crosses h upward then p(t) = 1.
This is slightly different from the ﬁring pattern of Meissner’s
corpuscles. But roughness recognition of humans is affected byboth of intensity and vibrational frequency of stimulus. How
much and how long x(t) exceeds h are ignored deliberately
to extract vibrational frequency in no regard to intensity ofperceived roughness.
Threshold value h is determined to be 1.5 times of a meanvalue of x(t) of a certain period. This period is determined to
be 200[msec] in a heuristic way.
c(t) expresses a number of impulse emissions for a[sec]from t as (2). a is 200[msec]. c(t) is pulse density.
 tp(τ )dτ(2)c(t) =
t−a|v(t)|2a|v(t)|
=
f (t)
c(t)

(4)

B. Preliminary Evaluation of Wavelength Estimation
Here proposed method for real-time estimation of vibrational frequency is evaluated. In these experiments roughness
samples [13] characterized by groove width and ridge width
were used (Fig.3).

Fig. 3.

Roughness sample characterized by groove width and ridge width

All of these experiments were conducted with equipment in
Fig.4. A sensor was mounted on a vertical slide table. They
were fastened to a single-axis arm.

Fig. 4.

Experimental equipment

1) Wavelength Discrimination: λ(t) are shown in Fig.5
when a sensor rubbed the samples of spatial wavelength
3.0[mm], 2.4[mm], 1.8[mm], 1.2[mm] and 0.6[mm] at a ﬁxed
speed of 100[mm/sec] which is a natural speed of humantouch. λ(t) were estimated based on a vibrational frequency
of a tactile sensor f (t) at t. In Fig.5 sampless of different
wavelength were discriminated and each of wavelengths was
identiﬁed unique to some extent. For instance, looking at λ(t)of a roughness sample (GW,RW)=(1.5mm,1.5mm) , its wavelength was 3.0[mm] then wavelength of it was not estimated3583

precisely but it is important that different objects can be judged
to be different when being touched in a same condition.

Fig.7 (b) indicates that f (t) was estimated with time delay of
about 100[msec], no matter whether a touch motion was quickor slow. Fig.7 (c) shows λ(t) derived by λ(t) = v(t)/f (t).
λ(t) was estimated more precisely when v(t) is quick thanslow as well as humans’ roughness recognition works.

Fig. 5. Estimated spatial wavelengths of roughness samples λ(t) when asensor rubbed them at a ﬁxed speed of 100[mm/sec]

2) Robustness evaluation against vertical displacement:
A tactile sensor for a tactile telepresence system has to berobust against the vertical displacement changes to objects totouch or should not be under the inﬂuence of reaction force
from objects. Fig.6 shows λ(t) of roughness samples when thesensor pressed itself to the sampless with vertical displacementd from 0.20[mm] up to 0.80[mm]. As a sensor was just incontact with the object, d was 0[mm]. Reaction force from theobject toward the sensor was 0.70[g] when d was 0.2 and whend was 0.8 reaction force was 38.37[g]. Two roughness samples
used in this experiment were (GW,RW)=(1.5mm,1.5mm) and(GW,RW)=(0.3mm,0.3mm). In Fig.6 these two samples were
judged completely different with different vertical displacements.

Fig. 6. Estimated spatial wavelength λ(t) of two roughness samples withdifferent vertical displacements d3) Robustness evaluation against velocity change: Tactile
sensors for telepresence system must work under active touch,which is natural touch for humans. Changing velocity oftouch motion and high time resolution of humans’ tactile
recognition make it hard to reserve long sampling time forthe estimation of f (t). The velocity of a human’s natural
touch motion is about 1[Hz] and sinusoidal with the max
speed of about 100[mm/sec]. λ(t) of a roughness sample(GW,RW)=(0.9mm,0.9mm) is shown in Fig.7 where the velocity of rubbing motion was natural one as shown in Fig.7 (a).

Fig. 7. Wavelength and vibrational frequency estimation under natural touch
motionC. Comparison With STFT for Vibrational Frequency Estimation
STFT is typical of the real-time frequency estimation methods. We also considered to employ STFT but from someexperimental results some ﬂaws became clear. Fig.8 showsthe estimated frequency of the sensory vibration when a sensor
rubbed the sample (GW,RW)=(0.9mm,0.9mm) at a sinusoidal
velocity with max speed of 100[mm/sec]. Fig.8 (a) is theestimated frequency by the proposed method, Fig.8 (b) isestimated main frequencies by STFT with the window sizeof 200[msec]. Both methods could estimate the vibrationalfrequencies with time delay of about 100[msec]. But theshape of the estimated frequency by the proposed method
was more similar to the theoretical vibrational frequency than
STFT. Three problems about STFT were found, ﬁrstly STFT
took time to be computed and estimated frequency could be
refreshed about every 10[msec] as the proposed methods could
ﬁnish the estimation within 1[msec]. Secondly STFT could not
successfully extract the main frequencies while the vibrationalfrequency changed drastically or the rubbing motion was
accelerated. Thirdly when a sensory vibrational frequency was
low the main frequency was hidden in a lower lobe and could
not be separated. STFT has the capability of specifying theset of composed signals though, there is the more preferencein the proposed method for reasons above.
D. Vibrational Frequency Estimation by Time Interval Between Impulse Emissions
Fig.9 shows the estimated vibrational frequency, ft (t),based on the time interval between two impulse emissions.

3584

Standard deviation of the ﬁve strain gauges’ output changes
when pressing it onto objects with different Young’s modulus[10].
A. Hertz Contact Theory Underlying Real-time Softness Estimation Using Strain Distribution in the Sensor
According to Hertz contact theory, a contact of two spheresof Young’s modulus Ef and Eo generates a circular contact
area A. Relative displacement of the two spheres are wf andwo , a radius of two spheres is R, then (6) describes A.
Fig. 8. Comparison between the proposed method and STFT for vibrationalfrequency estimationft (t) is computed as (5) where the time interval between two
impulse emissions at t is tv [sec].
1
ft (t) =tv

A = πR(wf + wo )The relationship between relative displacements and Young’s
modulus is (7).
wo = wf

Ef
Eo

(7)

By combining (6) and (7), we get (8).
A
Ef
= πR(1 +
)
wf
Eo

(5)

In Fig.9, there was a ﬂicker in the estimated values, it was
noisy. This tells the algorithm from physiological aspects is
effective to make estimated vibrational frequency robust.

(6)

(8)

Human ﬁnger mimetic tactile sensor cannot obtain thecontact area A and relative displacement wf directly, but thenumerical trend of wf /A resembles it of the standard deviationof the output of ﬁve strain gauges. Fig.10 illustrates the contact
between a sensor and a soft object and a hard object. As theYoung’s modulus of an object to touch Eo is larger, the contact
area A is smaller and the standard deviation becomes larger. As
Eo is smaller, A is large and the standard deviation becomes
smaller. Thus the standard deviation of sensory output can beused to estimate the Young’s modulus of objects.

Fig. 9. Estimated vibrational frequency based on the time interval between
impulse emissions

IV. R EAL - TIME E STIMATION OF YOUNG ’ S M ODULUS OF
THE O BJECT U SING S TRAIN D ISTRIBUTION IN A S ENSOR
In robotics to obtain the Young’s modulus of an unknown
object, a set of time-series information of relative displacement
between a sensor and an object and reaction force from an
object is usually required. This method has a few ﬂaws. One
is that this method is time-consuming, and another is that
this method requires a tactile sensor system to have a sensor
apparatus to sense ﬁne relative displacement. It is said that a
man recognizes softness of the objects from both of kinesthetic
and contact information. Some tactile displays successfully
represented softness of objects by changing the contact area
between a ﬁnger and a display [4] [5]. As the contact areachanges, the stress distribution in a ﬁnger also changes and aman may utilize it to know softness of objects quickly without
time delay. The human ﬁnger mimetic tactile sensor employs
this way to estimate Young’s modulus of objects.

Fig. 10. Distribution of reaction force and contact area of the sensor on
contacting with a hard and a soft object

B. Experimental Data Underlying Real-time Softness Estimation and Formulation
Fig.11 (a) is the plots of the standard deviation of ﬁve strain
gauges’ output and the reaction force applied to the sensor
when a sensor pressed three softness samples, and Fig.11 (b) is
a replot of (a) and it shows a relationship between the standard
deviation and Young’s modulus. Three silicon rubber objects
with different Young’s modulus were 0.72[MPa], 0.51[MPa]and 0.089[MPa] softness samples. Fig.11 (a) shows the softness of three samples were discriminated by the standard
deviation in a reaction force range up to about 100[gf]. Fig.11

3585

(a)(b) implies that the standard deviation of the ﬁve strain
gauges’ output ,Sd , were primarily affected by the reaction
force from the object ,f , and Young’s modulus of the objects,E. A regression curve of Sd involving f and E is (9). Young’s
modulus of the object is estimated by (10).

Fig. 11. Experimental results plots between standard deviation of the ﬁvestrain gauges’ output, reaction force from the object and Young’s modulus of
the objects

Sd = −0.0314f 2 + 14.5f + 87.2E 2 + 104.5E − 81.9(9)


0.0115Sd + 0.00036f 2 − 0.166f + 1.3 − 0.6

(10)

E=

[6] Marc R. Tremblay and Mark R. Cutkosky, ”Estimating Friction Using
Incipient Slip Sensing During a Manipulation Task”, Proceedings of theIEEE International Conference on Robotics and Automation, pp. 363368, 1993
[7] H. Kawaguchi, T. Someya, T. Sekitani and T. Sakurai, ”Cut-and-Paste
Customization of Organic FET Integrated Circuit and Its Application toElectronic Artiﬁcial Skin”, IEEE Journal of Solid-State Circuits, Vol.40,
No.1, 2005[8] Hirokazu Shirado, Takashi Maeno, ”Modeling of Texture Perception for
Tactile Displays and Sensors”, The First Joint Eurohaptics Conference
and Symposium on Haptic Interface for Virtual Environment and Teleoperator Systems, 2005
[9] M. Konyo, A. Yoshida, S. Tadokoro and N. Saiwaki, ”A Tactile
Synthesis Method Using Multiple Frequency Vibrations for Representing Virtual Touch”, IEEE/RSJ International Conference on Intelligent
Robots and Systems, 2005[10] Y. Mukaibo, H. Shirado, M. Konyo and T. Maeno, ”Development of a
Texture Sensor Emulating the Tissue Structure and Perceptual Mechanism of Human Fingers”, Proceedings of the 2005 IEEE International
Conference on Robotics and Automation, 2005[11] M.Ohka, T.Kawahara, T.Itahashi, J.Takayanagi, T.Miyaoka and
Y.Mitsuya, ”A Tactile Recognition System Mimicking Human
Mechanism for Recognizing Surface Roughness”, JSME international
Journal, Series C, Vol. 48, No.2, 2005[12] Mohamed Benali-Khoudja, Moustapha Hafez, Jean-Marc Alexandre andAbderrahmane Kheddar, ”Tactile interfaces: a state-of-the-art survey”,
35th International Symposium on Robotics, 2004[13] Takashi Yoshioka, Barbara Gibb, Andrew K. Dorsch, Steven S. Hsiao
and Kenneth O. Johnson, ”Neural Coding Mechanisms Underlying
Perceived Roughness of Finely Textured Surfaces”, The Journal of
Neuroscience, 21(17):6905-6916, 2001

V. C ONCLUSION
We could successfully estimated the three tactile factors
which are needed to regenerate tactile sensation using tactile
display for a tactile telepresence system, just by one sensor
apparatus. The vibrational frequency of stimulus was estimatedby mimicking Meissner’s corpuscles’ ﬁring and other related
physiological aspects. This proposed methods performed realtime estimation of the object’s spatial wavelength. Young’s
modulus of the objects were estimated instantly by the method
considering human’s softness recognition using strain distribution in a sensor. Friction occurring with touch can be sensed
by a torque sensor as a conventional robotic solution. We
are going to work on the realization of a tactile telepresence
system with various tactile sensation feedback.
R EFERENCES[1] R. D. Howe, W. J. Peine, D. A. kontarinis and J. S. Son, ”Remote
Palpation Technology”, IEEE Engineering in Medicine and Biology,
1995[2] P.Dario and M.Bergmasco, ”An advanced robot system for automateddiagnostic tasks through palpation”, IEEE Trans. on Biomedical Engineering, Vol.35, Issue 2, Feb. 1988, 118-126[3] Akio Yamamoto, Shuichi Nagasawa, Hiroaki Yamamoto, and Toshiro
Higuchi, ”Electrostatic Tactile Display with Thin Film Slider and Its
Application to Tactile Tele-Presentation Systems”, IEEE Transactions
on Visualization & Computer Graphics, Vol. 12, No. 2, pp.168-177,
2006[4] Antonio Bicchi, Enzo P. Schilingo and Danilo De Rossi, ”Haptic
Discrimination of Softness in Teleoperation: The Role of the Contact
Area Spread Rate”, IEEE Trans. on Robotics And Automation, Vol.16,
No.5, 2000
[5] Kinya Fujita and Yoshiaki Ikeda, ”Remote haptic sharing of elastic
soft objects”, The First Joint Eurohaptics Conference and Symposium
on Haptic Interface for Virtual Environment and Teleoperator Systems,
2005

3586