IEEE GCCE 2022 1570820349

1	  2	  3	  4	  5	  6	  7	  8	  9	  10	  11	  12	  13	  14	  15	  16	  17	  18	  19	  20	  21	  22	  23	  24	  25	  26	  27	  28	  29	  30	  31	  32	  33	  34	  35	  36	  37	  38	  39	  40	  41	  42	  43	  44	  45	  46	  47	  48	  49	  50	  51	  52	  53	  54	  55	  56	  57	  60	  61	  62	  63	  64	  65	  Combined virtual bumps and textures onelectrostatic friction tactile displays
Mirai AzechiShogo Okamoto

Tokyo Metropolitan University
Hino, JapanTokyo Metropolitan University
Hino, Japan

Abstract—Touch panels are commonly used interfaces in consumer electronics, and tactile feedback on touch panels canimprove usability and create new use cases. In this study, weinvestigated whether virtual macroscopic and microscopic surface
features can be recognized when they are presented together
using electrostatic friction tactile displays. Participants wereable to distinguish between ﬁve composite stimuli; however,they tended to struggle distinguishing between a macroscopicbump and dent. The results of this study provides new insights
regarding consumer electronics and human interfaces research.
Index Terms—Touch panel, texture, bump, dent

I. I NTRODUCTION
Tactile feedback functions on touch panels provide surface information regarding bumps and textures. Accordingly,
combining the audiovisual and tactile feedback makes digitalcontents more enjoyable [1]. Recently, electrostatic friction
displays have been intensively studied by many researchgroups [2]–[7]. For example, earlier researches demonstrated
how virtual macroscopic bumps [2] and how virtual microscopic textures are perceived on touch panels with electrostatic
friction feedback functions [3].
However, no existing study has tested bumps and texturesin conjunction using electrostatic friction displays. It remains
unknown if the bumps and textures are correctly recognizedwhen they coexist. One concern is that macroscopic or microscopic features may perceptually mask each other, or they
may mutually interfere consequently, it may be difﬁcult toassess the two surface feature types. The objective of the thisstudy is to clarify human perceptual abilities under combined
bump and texture stimuli. We conducted an experiment whereparticipants identiﬁed ﬁve virtual surface types, a few ofwhich presented a macroscopic bump or dent and microscopic
textures in conjunction.
II. E LECTROSTATIC FRICTION TACTILE DISPLAY
For the experiments, we used an electrostatic friction display
shown in Fig. 1. The display consists of an electrostatic
touch panel (SCT3260, 3M Touch Systems, MA) and fourforce sensors (USLG25, Tec Gihan Co. Ltd., Japan), eachof which was afﬁxed beneath a corner to localize the ﬁnger
on the panel. To unify the frictional conditions between thepanel and ﬁnger for different participants, the participants wereThis study was in part supported by MEXT Kakenhi #21H05819 and#20H04263.

/22/$31.00 ©2022 IEEE

Fig. 1. Electrostatic friction display used in the experiment.

Fig. 2. Five types of virtual surfaces used in experiment, where A, lt , λb ,and λt are the sinusoidal surface amplitude, micro-texture length, and spatial
wavelength of bump and micro-textures, respectively.

provided a thimble with a grounded conductive rubber piece.
An electrostatic friction tactile display modulated friction force
between the ﬁnger pad and panel in the shear direction byapplying a voltage between the conductive rubber and panel.
The applied voltage was ampliﬁed by a voltage ampliﬁer(HJOPS-1B20, Matsusada Precision Inc., Japan, maximum
output: ± 1 kV, response: 75 kHz).
III. E XPERIMENT
A. Participants
Six volunteers participated in this experiment. They werenot informed about the research objective prior to the experiment.

1322

B. Stimuli

TABLE IM EANS ±95% CONFIDENCE INTERVALS OF ANSWER PROPORTIONS **INDICATES p < 0.01.

The relationship between the shear frictional force Fe (t)and applied voltage Ve (t) is determined according to the lawof electrostatic force and Coulomb friction as follows:
Fe (t) = µ{W + kVe2 (t)}Answer(1)
Texturewhere µ, W , and k are the coefﬁcients of friction, load of
the ﬁnger, and electrostatic force constant, respectively [8]. In
the experiment, we presented ﬁve types of virtual surfaces:macroscopic bump only, macroscopic dent only, micro-texture
only, macroscopic bump and micro-texture, and macroscopic
dent and micro-texture as shown in Fig. 2. The shear force
caused by the bump is a function of its gradient. Thus, theapplied voltage Ve (t) could be expressed as follows:√2πx(t) + θA
2πx(t)
5) + sin() + A (2)Ve (t) = A sin(λb4λt4where A and θ were the sinusoidal surface shape amplitudeand phase (0 for bump, π for dent) used to adjust the bumpshape, respectively. Moreover, x(t) denotes the ﬁnger positionon the panel. λb and λt are the spatial wavelength of the bumpand micro-textures, respectively. The ﬁrst term was utilized
when presenting a macroscopic bump; accordingly, the second
term was applied when presenting a micro-texture.
Different virtual surfaces were presented during the training
session and main experiment. During the training session, weset parameters lt = 20 mm and λt = 1.5 mm for the microtexture. For presenting only macroscopic bumps, λb = 25 mm.
For presenting the bump and texture in conjunction, λb = 20mm. During the main experiment, lt = 20 mm, λt = 1.0 mm,
and λb = 20 mm were set as experimental parameters. These
virtual stimuli were presented near the center of the panel.
C. Tasks
During the training session, the participants experienced ﬁvevirtual surfaces and were told which surface they had touched.
The training session continued until the participants were able
to correctly identify the randomly presented virtual surfaces
ten times in a row. The average number of training trials wasapproximately 30.
During the main experiment, the participants identiﬁed
arbitrarily presented stimuli. The participants selected one of
the ﬁve options in a forced-choice manner. They could explore
the stimuli as many times as they wanted during each trial. The
number of trials was 50, and each stimulus was presented 10times in total.

Bump
Dent
Bump +
TextureDent +
TextureTexture

Bump

Dent

0.85 **± 0.180.01± 0.030.00± 0.000.03± 0.040.03± 0.040.03± 0.060.65 **± 0.240.21± 0.180.06± 0.060.00± 0.000.05± 0.090.18± 0.210.66 **± 0.190.00± 0.000.10± 0.19

Bump +
texture0.01± 0.320.10± 0.120.01± 0.030.73 **± 0.150.21± 0.15

Dent +
texture0.05± 0.040.05± 0.060.10± 0.130.16± 0.140.65 **± 0.17

E. Results
Table I lists the means of the answer proportions andtheir 95% conﬁdence intervals. The correct answer proportions
for the ﬁve virtual stimuli ranged from 65–85%. The nullhypothesis that these values were equal to 0.2 (20%) wasrejected (χ2 -test, p < 0.01). Furthermore, the mean of correct
answer proportions for each stimulus was signiﬁcantly greaterthan the chance level (z-test, p < 0.01).
IV. D ISCUSSION
The experimental results demonstrated that combined virtual bumps and textures were correctly recognized. However,regardless of the presence or absence of texture, the bump anddent were occasionally confused. For example, the presenceor absence of micro-textures and those of macroscopic bumps
were correctly recognized at 83%. However, the bumps anddents were correctly assessed only 67% of the time on average.
Although the friction, which is a function of the surface
gradient, is a cue in identifying bumps, the lack of an actual
surface gradient hampers the correct recognition of bumps.
V. C ONCLUSION
This study demonstrated that macroscopic and microscopic
surface features can be recognized when they are presented
in conjunction using electrostatic friction displays. To the bestof our knowledge, such results have not been reported in theliterature. The obtained results will broaden the possible use
cases for electrostatic friction tactile displays. Electrostatic
friction displays can present more complex surface features
than ever considered including macroscopic and microscopic
patterns. On the other hand, the difﬁculty in distinguishing
between a macroscopic bump and dent should be resolved.

D. Analysis

R EFERENCES

First, we performed a chi-square goodness-of-ﬁt test to
determine if the means of answer proportions for individual
stimuli match the chance. The null hypothesis was that there isno signiﬁcant difference between the means of correct answer
proportions and chance probability (0.2). Then, we performed
z-tests to investigate whether the mean of correct answer
proportions exceeded the chance probability for each stimulus.

[1] W. Park, M. H. Jamil, R. G. Gebremedhin, and M. Eid, “Effects of tactile
textures on preference in visuo-tactile exploration,” ACM Transactions on
Applied Perception, vol. 18, pp. 1–13, 2021.
[2] X. Sun, C. Zhang, and G. Liu, “Improved tactile perception of 3D
geometric bumps using coupled electrovibration and mechanical viration
stimuli,” Computer Journal, vol. 65, pp. 621–630, 2022.
[3] R. H. Osgouei, J. R. Kim, and S. Choi, “Improving 3D shape recognition
with electrostatic friction display,” IEEE Transactions on Haptics, vol. 10,
no. 4, pp. 533–544, 2017.

2323

[4] K. Otake, S. Okamoto, Y. Akiyama, and Y. Yamada, “Tactile texture display combining vibrotactile and electrostatic-friction stimuli: Substantial
effects on realism and moderate effects on behavioral responses,” ACM
Transactions on Applied Perception, 2022.
[5] ——, “Tactile texture rendering for electrostatic friction displays: Incorporation of low-frequency friction model and high-frequency texturalmodel,” IEEE Transactions on Haptics, vol. 15, no. 1, pp. 68–73, 2022.
[6] ——, “Virtual tactile texture using electrostatic friction display for naturalmaterials: The role of low and high frequency textural stimuli,” in IEEE
International Conference on Robot and Human Interactive Communication, 2021, pp. 392–397.
[7] K. Ito, S. Okamoto, Y. Yamada, and H. Kajimoto, “Tactile texture displaywith vibrotactile and electrostatic friction stimuli mixed at appropriate
ratio presents better roughness textures,” ACM Transactions on Applied
Perception, vol. 16, p. 20, 2019.
[8] T. Nakamura and A. Yamamoto, “Multi-ﬁnger electrostatic passive haptic
feedback on a visual display,” in IEEE World Haptics Conference, 2013,
pp. 37–42.

3324