Virtual roughness textures via a surface tactile
texture display using vibrotactile andelectrostatic friction stimuli: Improved realism
Kazuya Otake

Hikaru Hasegawa

dept. Mechanical Systems Engineering
Nagoya University
Nagoya, Japanotake.kazuya@j.mbox.nagoya-u.ac.jp

dept. Mechanical Systems Engineering
Nagoya University
Nagoya, Japan
hasegawa.hikaru@c.mbox.nagoya-u.ac.jp

Shogo Okamoto

Yoji Yamada

dept. Mechanical Systems Engineering
Nagoya University
Nagoya, Japanokamoto-shogo@mech.nagoya-u.ac.jp

dept. Mechanical Systems Engineering
Nagoya University
Nagoya, Japan
yamada-yoji@mech.nagoya-u.ac.jp

Abstract—In this study, improved realism of virtual roughness
textures is veriﬁed by using vibrotactile and electrostatic friction stimuli in conjunction. Because these two types of stimuli
have complementary features, they are expected to enhance theperceived quality of stimuli. In our previous research, conjunctstimuli improved the realism of virtual surfaces with sinusoidal
surface roughness proﬁles. In this research, we tested withcomplicated surface roughness, and experimental participants
ranked vibrotactile, electrostatic friction, and conjunct conditionsin terms of realism compared to actual roughness textures. The
conjunct condition was judged to be more realistic for simple
and complicated surface proﬁles consisting of one or two spatial
wave components.
Index Terms—Tactile Texture Display, Vibrotactile Stimuli,Electrostatic Friction Stimuli

I. I NTRODUCTION
According to the spread of touch-panel interfaces, expectations for tactile texture feedback technology have been increasing. Tactile texture displays suitable for touch panels aremainly vibrotactile types [1], [2], [3], [4] and friction-variabletypes [5], [6], [7], [8], [9]. The former type involves driving thepanel mechanically and deforming a ﬁnger pad by touching
the panel. The latter type entails controlling friction on thepanel. Many tactile displays that were previously studied werebased on one of these two types of tactile stimulus, whereasa few perceptual studies about the conjunct vibrotactile andelectrostatic friction stimuli were reported [10].
Because tactile perception of surfaces is expressed in a
multidimensional information space [11], tactile texture displays presenting multiple haptic modalities are effective methods to present texture [1], [12], [13]. These studies entailed
combining multiple tactile stimuli to improve the realism of

978-1-7281-7392-4/20/$31.00 ©2020 IEEEvirtual textures. However, there is little research on the effects
of using multiple tactile stimuli together via tactile texture
displays implementing touch panels.
In this research, we investigated the effects of combining
vibrotactile and electrostatic friction stimuli to present virtual
texture surfaces in terms of realism. In principle, using vibrotactile stimuli is an appropriate method for presenting surface
textures [1], [3]. Use of electrostatic friction stimuli involves
presenting texture using the spatial distribution of friction andvibration caused by friction [5], [6], [7]. These two types ofstimuli are efﬁcient to present surface roughness and friction
characteristics, respectively. Hence, the use of vibrotactile andfrictional stimuli in conjunction widens the texture space tobe covered. However, thus far only limited studies have beenreported [14], [15].
Our previous research indicated the effects of combining vibrotactile and electrostatic friction stimuli in terms ofpresenting virtual roughness with sinusoidal surface proﬁles[14], [15]. In this study, we investigated the same effects
as in previous studies in cases of more complicated texture
surfaces. We adopted surfaces that were three-dimensionallyprinted (3D-printed) by synthesizing two sinusoidal waves
with different wavelengths and natural materials such as cloth
and wood as complicated texture surfaces.

II. P RINCIPLES OF VIBROTACTILE AND ELECTROSTATICFRICTION DISPLAY

In this study, we focused on the effect of using two principles of tactile stimuli that possess complementary features.

147

(a) Vibrotactile stimuli

Load cell(FSS015WNSX, Honeywell Co. Ltd., USA)

Stimuli for movement

ITO panel
Insulator

Position

Finger movement

Voice coil motor(X-1741, Neomax Engineering Co. Ltd., Japan)

Actuator
Vibration

Fig. 2. Schematic view of the tactile texture display used in the experiment.

(b) Electrostatic friction stimuli

III. A PPARATUS : TACTILE TEXTURE DISPLAY USINGVIBROTACTILE AND ELECTROSTATIC FRICTION STIMULI

Load
Friction force

Finger movement---

Insulator
Conductor++++

ChargeFig. 1. Principles of (a) vibrotactile and (b) electrostatic friction stimuli.

A. Vibrotactile display
Vibrotactile stimuli drive the panel using an actuator, as
shown in Fig. 1(a), deforming a ﬁnger pad. Such tactile stimuli
are presented in the normal direction in the present study.
Surface roughness textures can be presented by producing
vibrotactile stimuli pertaining to surface displacement [1],[2], [3], [4]. Because the touch panel itself actively produces
mechanical stimulation, this display does not necessarily need
active ﬁnger movement. However, tactile stimuli synchronizedwith ﬁnger movement are known to be effective method forpresenting stimuli.

B. Electrostatic friction display
Electrostatic friction stimuli control the electrostatic attractive force generated by applying a voltage between the ﬁnger
pad and the panel, as shown in Fig. 1(b). The attractive force
increases the friction between the ﬁnger and panel. The ﬁnger
pad is then deformed along the shear direction by the friction
force when the ﬁnger slides across the panel. Such stimuli
are efﬁcient in presenting surface texture dominated by thefriction force [6], [7]. A display of this type does not needa mechanical drive, thereby simplifying its implementation.
However, the strength of the perceived friction stimuli depends
on the individual, because of the dielectric constant of the
ﬁnger pad.

In this experiment, we used a tactile texture display presenting vibrotactile and electrostatic friction stimuli simultaneously (Fig. 2). There are few tactile displays that can output
both types of stimuli at the same time. This display was usedin our previous research [14], [15] where electrostatic friction
stimuli were presented via a conductive pad to control friction.
In contrast, stimuli are presented to the ﬁnger pad directly inthe current study.
Vibrotactile stimuli were generated by voice coil motors(X-1741, Neomax Engineering Co., Ltd., Japan) located in
the four corners below the panel. These four actuators were
driven synchronously by an audio ampliﬁer (FX-AUDIO-FX502J, North Flat Japan Co., Ltd., Japan) and presented adisplacement normal to the panel. This setup can present
suprathreshold vibrations at up to at least 300 Hz.
Electrostatic friction stimuli were produced by applying
a voltage between the ﬁnger pad and the indium tin oxide(ITO) panel with an insulator ﬁlm attached. The stimuli wereexperienced by touching the panel while holding a stainlessbar connected to ground. The applied voltage was amplitude
modulated over a carrier frequency of 20 kHz. The modulated
voltage signal was ampliﬁed by a voltage ampliﬁer (HJOPS1B20, Matsusada Precision Inc., Japan), whose maximum
output was ±1 kV, with a response of 75 kHz.
Four load cells (FSS015WNSX, Honeywell Co., Ltd., USA)
were placed between the panel and the actuators. Finger
position was measured by the output ratio of these load cells.
Tactile stimuli were controlled for ﬁnger movement.
Two voltage ampliﬁers for the two types of stimuli andload cells were connected to a computer via a data acquisitionboard (TNS-6812, Interface Co., Ltd., Japan). The sampling
frequency was determined to be ∼667 Hz.
IV. E XPERIMENT: COMPARISON OF REALISM AMONGSTIMULI CONDITIONS

We veriﬁed that the realism of the virtual roughness texture
is improved by using vibrotactile and electrostatic friction
stimuli in conjunction. We created a virtual texture that148

imitates roughness texture with one or two spatial frequency
components and natural materials.
A. Ranking task
Participants experienced texture stimuli under the following
conditions: only vibrotactile stimuli were presented, only electrostatic friction stimuli were presented, and both stimuli were
presented. The participants ranked the three stimuli conditionsin terms of realism, and then determined the most realistic
stimuli condition. They were instructed to trace the virtualand real surfaces at the same ﬁnger speed and strength forroughness textures or natural materials to compare them. They
were allowed to experience virtual stimuli and actual stimuli
any number of times in each trial. One trial was conducted
for each roughness texture and each natural material. They
washed their hands and removed extra moisture or oil on theirﬁngertips before the experiment. The panel was lubricatedby talcum powder to reduce the effect of friction in theexperiment.

Fig. 3. Images of roughness textures and natural materials.
TABLE IS PATIAL WAVELENGTH λ AND PEAK - TO - PEAK AMPLITUDE h VALUES OF
ROUGHNESS TEXTURES .
Sinusoidal surface
Single type

B. Participants
Thirteen university students (males in their 20s) participatedin this experiment. Seven of these took part in the experiment on roughness textures (one or two spatial frequency
components). Another six participated in the experiment on
natural materials. None of the students were informed about
the objective of the research.
C. Roughness textures and natural materials
Images of roughness textures and natural materials used inthis experiment are shown in Fig. 3.
Table I lists the characteristics of the ﬁve actual roughness
surfaces used. Three of these are surface roughness textures
created by synthesizing two sinusoidal waves with different
spatial wavelengths. Two other kinds are roughness textureswith sinusoidal surface proﬁles, and these surface types wereused in our previous studies [14], [15]. Listed in the table arespatial wavelength λ and peak-to-peak amplitude h values of
the roughness textures. For synthesizing two different spatial
wavelengths of sinusoidal waves, their phase differences were
set to 0. We prepared 3D-printed actual stimuli for theseroughness textures by using Form2 (Formlabs., Ltd., MA,
nominal resolution: 140 µm). Our previous researches [14],[15] concluded that for relatively large spatial wavelengths (1.0mm≤ λ ≤3.0 mm), vibrotactile stimuli with slight electrostatic
friction stimuli were effective for presenting virtual sinusoidal
surfaces. Therefore, we investigated the effectiveness of conjunct stimuli in this range. Wood, cork, drawing paper, anddenim were used as natural materials, as they are considered
commonly touched surfaces.
D. Virtual roughness textures
Roughness textures were presented by using vibrotactile
and electrostatic friction stimuli. We used a tactile stimulation
algorithm to deform a ﬁnger pad based on the relationshipbetween ﬁnger movement and surface displacement [3].

Synthesized type

λ1 (mm)
1.5
2.5
1.3
1.3
1.9

λ2 (mm)
—
—
1.9
2.5
2.5

h (mm)
1.0
1.0
1.2
1.2
1.2

1) Vibrotactile stimuli simulating roughness textures: To
simulate skin vibration generated by tracing surfaces of theroughness textures, the driving force of the voice coil motors
was set to
(
)
(
)
x(t)
x(t)
Fv (t) = A sin 2π+ B sin 2π
,
(1)
λ1
λ2
where A, B, and x(t) are the gains for each sinusoidal
wave and the ﬁnger position on the touch panel at time t,
respectively, and λ1 and λ2 express spatial wavelengths foreach sinusoidal wave. Similar formulas were used in previous
researches [4], [6], [7]. λ values were determined as listed
in Table I. In case of presenting roughness textures withsingle sinusoidal surface proﬁles, the driving force Fv (t) was
determined by using only the ﬁrst term on the right side of(1).
2) Electrostatic friction stimuli simulating roughness textures: Based on the electrostatic force law and Coulomb’s
law of friction, the relationship between the shear force Fe (t)and the applied voltage Ve (t) generated by tracing the panel
is described as
Fe (t) = µ{W + kVe (t)2 },(2)where µ, W , and k are the coefﬁcient of friction on the panel,
the normal load of the ﬁnger, and a constant of the electrostatic
friction force, respectively. The shear force generated bysliding across an uneven surface with a ﬁnger is a gradientfunction of the surface displacement [16], [17]. This meansthat the shear force phase deviates by π/2 from that of surface

149displacement. Therefore, the applied voltage was set so that
Fe (t) was generated as
)
[
{
(
x(t)
Fe (t) = µ W + k C cos 2πλ() 1}] (3)x(t)+ D cos 2π+C +D ,
λ2
where C and D are the gains for sinusoidal waves. The applied
voltage Ve (t) is presented as follows:
√
(
)
(
)
x(t)
x(t)
+ D cos 2π+ C + D.
Ve (t) = ± C cos 2π
λ1
λ2
(4)
A similar formula was used in a previous study [7], which
can be used regardless of the number of sinusoidal waves
to be synthesized. For roughness textures consisting of asingle sinusoidal surface proﬁle, the applied voltage Ve (t) was
determined by using only the ﬁrst and third terms of (4).
3) Perceived strength of virtual roughness texture: The
strengths of tactile stimuli (A, B, C, and D) were determined
to the value such that the synthesized stimuli were felt to
be close to the sense of real roughness textures by using themethod of adjustment. At this time, three stimuli conditions(vibrotactile stimuli only, electrostatic friction stimuli only,
and conjunct stimuli) were adjusted to have the same perceived
strength.
E. Virtual material textures
We measured the normal and shear forces generated when
tracing natural materials with a ﬁnger [18]. The measured
normal forces correspond to the vibrotactile stimuli that arenormal to the panel for our tactile display. The shear forces
correspond to the friction stimuli for our tactile display. Hence,
we converted the measured normal and shear forces to vibrotactile and electrostatic friction stimuli, respectively. We used
the force components above 15 Hz when the ﬁnger’s normalload and speed were 1 ± 0.3 N and 70–90 mm/s, respectively.
For each of the four types of natural materials, force signals
lasting 500 ms were prepared. In the latter experiment, the
500-ms signal was repeatedly presented to form an endlesstexture.
1) Vibrotactile stimuli simulated natural materials: To
present vibration components generated by tracing surfacesof the natural materials, the driving force Fv (t) of voice coilmotors was determined as
Fv (t) = αfni (t),(5)

where α and fni (t) are gain and normal force generated bysliding material i, respectively.
2) Electrostatic friction stimuli simulated natural materials:
Taking the relationship between the applied voltage and theshear force produced during tracing panel (2) into account,
the applied voltage Ve (t) was determined to be√µi
Ve (t) = ±β fsi (t) + γ
,
(6)
µmin

where β and fsi (t) are the voltage gain and the vibration signalof the shear force generated by tracing natural material i,
respectively; µi is the friction coefﬁcient of natural materiali; µmin is the minimum friction coefﬁcient in those of fourmaterials; and γ is the minimum positive value that satisﬁes
fsi (t) + γ ≥ 0 for all of the materials. The magnitude of DC
component was determined by using the friction coefﬁcient.
µi was normalized by µmin so that the square root of (6)was not negative, i.e., µi /µmin ≥ 1. As in Section IV-D2, thevoltage determined by (6) was amplitude-modulated with thecarrier frequency of 20 kHz.
3) Perceived strength of virtual material textures: The
strength of tactile stimuli (given by α and β) was adjusted byparticipants before the ranking task. Essentially, this shouldmatch the magnitude of the measured contact force. However,this adjustment is difﬁcult because it is inﬂuenced by theconditions of the participant’s ﬁnger. In this research, α andβ were adjusted so that participants could feel that the virtualmaterial and the corresponding real natural material weresimilar. For the conjunct stimuli, α and β were reduced by
1 JND (Just Noticeable Difference), because conjunct stimuli
tend to be perceived as stronger than either single stimuli.
V. R ESULTS
Fig. 4 shows the results of the ranking task. Figs. 4(a), 4(b),and 4(c) show the results of collective analysis of two kinds ofroughness textures with sinusoidal surface proﬁles, three kindsof roughness textures created by synthesizing two sinusoidal
waves with different spatial wavelengths, and four kinds ofnatural materials, respectively. The horizontal axis shows thethree types of stimuli conditions, and the vertical axis shows
the order of realism. The percentage of which the rank was
selected is written in each grid. The higher the percentage,
the darker the color in the grid; the lower the percentage, thelighter the color.
We analyzed whether there were signiﬁcant differencesamong the ranks of stimuli conditions by using a Wilcoxon
rank-sum test with a corrected p value. Signiﬁcance level
was set as p < 0.05. As shown in Fig. 4(a), the case of
conjunct stimuli was selected as signiﬁcantly higher than thatof vibrotactile stimuli for roughness textures with sinusoidal
surface proﬁles. This indicates that conjunct stimuli were
judged more realistic than vibrotactile stimuli only.
As shown in Fig. 4(b), the case of conjunct stimuli wasselected as signiﬁcantly higher than each vibrotactile and electrostatic friction stimuli only for roughness textures created bysynthesizing two sinusoidal waves. Hence, conjunct stimuli
were judged the most realistic of the three conditions.
As shown in Fig. 4(c), no signiﬁcant difference was conﬁrmed among the three stimuli in natural materials. By material, the case of conjunct stimuli was often selected as of ﬁrst
rank for cork and drawing paper. In contrast, no substantial
difference was observed among the three stimuli for woodand denim.

150

an effective method for presenting complex roughness textures
to some extent. In addition, it may be difﬁcult to present
virtual roughness textures by electrostatic friction stimuli onlyin complicated surfaces.
The superiority of conjunct stimuli was not observed in
the case of presenting virtual material textures. This may be
because effective stimuli conditions depend on the kind ofnatural material. Moreover, this result may be inﬂuenced by
the algorithm used for producing the virtual stimuli and themethod of adjusting their intensity.
VII. C ONCLUSION
We investigated whether the realism of virtual roughness
textures was improved by using vibrotactile and electrostatic
friction stimuli in conjunction. A positive effect was conﬁrmed
for complicated roughness textures made by synthesizing two
sinusoidal surfaces; however, this was not the case for stimuli
made from natural textures. Our results are not conclusive
for natural textures because the method for presenting suchtextures has not been established in the community.
ACKNOWLEDGMENTS
This study was in part supported by MEXT KAKENHI(#17H04697 and #15H05923).
R EFERENCES

Fig. 4. Ranks of each type of surface. ∗ indicates p < 0.05.

VI. D ISCUSSION
We expected that using vibrotactile and electrostatic friction
stimuli would have an effect in terms of presenting various
virtual roughness textures. This was conﬁrmed in presenting
virtual textures imitating roughness textures with one or two
spatial frequency components; however, it was not observedin presenting virtual material textures.
It may be necessary to represent both the surface displacement and the spatial distribution of friction by stimuli forpresenting relative macro roughness with spatial wavelengthsof ≥1 mm. This is why combining two kinds of stimuli
contributed to improving the realism of virtual roughness textures. The superiority of conjunct stimuli was also conﬁrmed
for presenting more complicated roughness textures madeby synthesizing two sinusoidal waves with different spatial
wavelengths. Combining the two kinds of tactile stimuli is also[1] H. Culbertson and K. J. Kuchenbecker, “Importance of matching physical friction, hardness, and texture in creating realistic haptic virtualsurfaces,” IEEE Transactions on Haptics, vol. 10, no. 1, pp. 63–74,
2017.
[2] M. Wiertlewski, J. Lozada, and V. Hayward, “The spatial spectrumof tangential skin displacement can encode tactual texture,” IEEE
Transactions on Robotics, vol. 27, no. 3, pp. 461–472, 2011.
[3] S. Asano, S. Okamoto, Y. Matsuura, and Y. Yamada, “Toward quality
texture display: vibrotactile stimuli to modify material roughness sensations,” Advanced Robotics, vol. 28, no. 16, pp. 1079–1089, 2014.
[4] S. Asano, S. Okamoto, and Y. Yamada, “Vibrotactile stimulation toincrease and decrease texture roughness,” IEEE Transactions on Humanmachine Systems, vol. 45, no. 3, pp. 393–398, 2015.
[5] O. Bau, I. Poupyrev, A. Israr, and C. Harrison, “Teslatouch: Electrovibration for touch surfaces,” Proceedings of Annual ACM Symposium on
User Interface Software and Technology, pp. 283–292, 2010.
[6] Y. Vardar, A. Isleyen, M. K. Saleem, and C. Basdogan, “Roughness
perception of virtual textures displayed by electrovibration on touch
screens,” IEEE World Haptics Conference, pp. 263–268, 2017.
[7] T. Fielder and Y. Vardar, “A novel texture rendering approach forelectrostatic displays,” Proceedings of International Workshop on Haptic
and Audio Interaction Design, no. hal-02011782v3, 2019.
[8] T. Nakamura and A. Yamamoto, “A multi-user surface visuo-haptic display using electrostatic friction modulation and capacitive-type position
sensing,” IEEE Transactions on Haptics, vol. 9, no. 3, pp. 311–322,
2016.
[9] J. Jiao, D. Wang, Y. Zhang, D. Cao, Y. Visell, X. Guo, and X. Sun,
“Detection and discrimination thresholds for haptic gratings on electrostatic tactile displays,” IEEE Transactions on Haptics, vol. 12, no. 1,
pp. 34–42, 2019.
[10] S. Ryu, D. Pyo, S.-C. Lim, and D.-S. Kwon, “Mechanical vibrationinﬂuences the perception of electrovibration,” Scientiﬁc Reports, no. 3,
2018.
[11] S. Okamoto, H. Nagano, and Y. Yamada, “Psychophysical dimensionsof tactile perception of textures,” IEEE Transactions on Haptics, vol. 6,
no. 1, pp. 81–93, 2013.

151

[12] M. Strese, R. Hassen, A. Noll, and E. Steinbach, “A tactile computer
mouse for the display of surface material properties,” IEEE Transactions
on Haptics, vol. 12, no. 1, pp. 18–33, 2019.
[13] T. Yamauchi, S. Okamoto, M. Konyo, Y. Hidaka, T. Maeno, and S. Tadokoro, “Real-time remote transmission of multiple tactile properties
through master-slave robot system,” Proceedings of IEEE International
Conference on Robotics and Automation, pp. 1753–1760, 2010.
[14] K. Ito, S. Okamoto, H. Elfekey, H. Kajimoto, and Y. Yamada, “A texture
display using vibrotactile and electrostatic friction stimuli surpasses one
based on either type of stimulus,” Proceedings of IEEE International
Conference on Systems, Man, and Cybernetics, pp. 2343–2348, 2017.
[15] K. Ito, S. Okamoto, Y. Yamada, and H. Kajimoto, “Tactile texture display
with vibrotactile and electrostatic friction stimuli mixed at appropriate
ratio presents better roughness textures,” ACM Transactions on Applied
Perception, vol. 16, no. 4, 2019.
[16] Y. Fujii, S. Okamoto, and Y. Yamada, “Friction model of ﬁngertip sliding
over wavy surface for friction-variable tactile feedback panel,” Advanced
Robotics, vol. 30, no. 20, pp. 1341–1353, 2016.
[17] G. Robles-De-La-Torre and V. Hayward, “Force can overcome object
geometry in the perception of shape through active touch,” Nature, vol.
412, pp. 445–448, 2001.
[18] H. Hasegawa, S. Okamoto, and Y. Yamada, “Phase difference between
normal and shear forces during tactile exploration represents texturalfeatures,” IEEE Transactions on Haptics, vol. 13, no. 1, pp. 11–17,
2020.

152