Tactile Texture Display Combining Vibrotactile andElectrostatic-friction Stimuli: Substantial Effects on
Realism and Moderate Effects on Behavioral Responses
KAZUYA OTAKE, Nagoya University
SHOGO OKAMOTO, Tokyo Metropolitan University
YASUHIRO AKIYAMA and YOJI YAMADA, Nagoya University
There is increasing demand for tactile feedback functions for touch panels. We investigated whether virtual roughness texture quality can be improved through simultaneous use of vibrotactile and electrostatic-friction stimuli. This conjunctive use
is expected to improve the perceptual quality of texture stimuli, because vibrotactile and electrostatic-friction stimuli havecomplementary characteristics. Our previous studies confirmed that these conjunct stimuli yield enhanced realism for simple
grating roughness. In this study, we conducted experiments using simple and complex sinusoidal surface profiles consistingof one or two spatial wave components. Three different evaluation criteria were employed. The first criterion concerned thesubjective realism, i.e., similarity with actual roughness textures, of virtual roughness textures. Participants compared the following three stimulus conditions: vibrotactile stimuli only, electrostatic-friction stimuli only, and their conjunct stimuli. The
conjunct stimuli yielded the greatest realism. The second criterion concerned roughness texture identification under eachof the three stimulus conditions for five different roughness textures. The highest identification accuracy rate was achieved
under the conjunct stimulus condition; however, the performance difference was marginal. The third criterion concerned thediscrimination threshold of the grating-scale spatial wavelength. There were no marked differences among the results for thethree conditions. The findings of this study will improve virtual texture quality for touch-panel-type surface tactile displays.
CCS Concepts: • Human-centered computing → Empirical studies in HCI
Additional Key Words and Phrases: Tactile texture display, vibrotactile stimuli, electrostatic-friction stimuli
ACM Reference format:
Kazuya Otake, Shogo Okamoto, Yasuhiro Akiyama, and Yoji Yamada. 2022. Tactile Texture Display Combining Vibrotactileand Electrostatic-friction Stimuli: Substantial Effects on Realism and Moderate Effects on Behavioral Responses. ACM Trans.
Appl. Percept. 19, 4, Article 18 (November 2022), 18 pages.
https://doi.org/10.1145/3539733

Kazuya Otake and Shogo Okamoto contributed equally to this research.
This study was partly supported by MEXT KAKENHI (#20H04263).
Authors’ addresses: K. Otake, Y. Akiyama, and Y. Yamada, Nagoya University, Furo-cho, Chikusa-ku, Nagoya, Japan, 464-8603; emails:
otake.kazuya.d6@s.mail.nagoya-u.ac.jp, {yasuhiro.akiyama, yoji.yamada}@mae.nagoya-u.ac.jp; S. Okamoto, Tokyo Metropolitan University,
6-6 Asahigaoka, Hino, Japan, 191-0065; email: okamotos@tmu.ac.jp.

$
This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.
© 2022 Copyright held by the owner/author(s).
1544-3558/2022/11-ART18
https://doi.org/10.1145/3539733
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

18

18:2 •

1

K. Otake et al.

INTRODUCTION

As touch panels have become mainstream in portable user interfaces, many researchers have studied tactile feedback technology that can be used with such panels [2]. Tactile display technologies suitable for implementation
in touch panel displays are mainly divided into two types: vibrotactile [1, 4, 8, 44, 50, 56, 61] and friction-variable[3, 12, 14, 20, 23, 57–59, 62]. For the former, the panel is mechanically driven and the fingertips are deformed to
provoke a sense of touch. For the latter, the frictional force increases or decreases as the finger pad slides on thepanel. Most tactile displays developed for touch panels use either the vibrotactile or friction-variable principle.
Tactile textures are an important component of this technology. Various tactile displays that present multiple
tactile modalities have been developed [8–10, 25, 39, 52, 63], because tactile textures are represented in a multidimensional perceptual space [36]. These studies demonstrated the effectiveness of combining multiple tactile
modalities. For example, the material identification rate can be improved by presenting multimodal cues [63].
However, few studies have investigated the effects of combining multiple tactile stimuli in a tactile display for
touch panels except for [21, 22, 31, 45]. Of the various studies on this topic, two research groups conducted thosemost similar to our work. Ryu et al. presented micro-textures and macroscopic surface profiles on vibrotactile
and friction-variable displays, respectively [45], and investigated the perceptual masking effects of the two stimulus types [46]. They did not aim to improve the texture quality through simultaneous use of vibrotactile andfriction-variable displays. Such microscopic and macroscopic surface features can be presented by using lateralforce display devices [47, 48]. In contrast, Liu et al. demonstrated that the combination of vibrotactile and frictionvariable displays improved the realism of tactile textures [31]. Their friction-variable function could increase anddecrease the surface friction of the panel. Further, they used a similar technique for representing bumps [53].
In our previous studies, we reported that addition of a slight electrostatic-friction stimulus to vibrotactile
stimuli improves the realism of virtual grating scales [21, 22, 39]. Similarity between virtual and actual textures,i.e., realism, is an important criterion, because it is an intuitive feeling generated when interacting with tactile
texture displays. However, realism is just one of multiple evaluation criteria relevant to tactile texture displays(see Section 7). Therefore, we also considered psychophysical behavioral responses that include the ability todiscriminate and identify the virtual texture stimuli. Both criteria must be investigated, because they may be
independent of each other. Therefore, in this work, three experiments were conducted to investigate these criteria. Note that earlier research studies on the combined tactile texture stimuli investigated their effects on thesubjectively reported realism [21, 22, 31, 39]; however, they did not investigate the effects on psychophysicalbehavioral responses.
In this study, we investigate the effects of combined vibrotactile and electrostatic-friction stimuli on texture
presentation. In principle, vibrotactile stimuli are suitable for presenting surface roughness [1, 8, 10, 15, 16];
however, they also affect friction perception [19, 27]. Frictional stimuli present spatial distributions of friction
and frictional vibrations and are also used for roughness presentation [20, 58]. Thus, vibrotactile and electrostaticfriction stimuli can effectively present the roughness and frictional properties of a textured surface, and theperceptual quality of the virtual texture should be improved through simultaneous use of both stimuli, because
actual textured surfaces are largely characterized by their surface roughness and friction.

2

METHODOLOGY

As in Section 4, the first experiment compared three types of virtual stimulus (vibrotactile stimuli, electrostaticfriction stimuli, and their conjunct stimuli) in terms of subjective realism; this experiment was similar to thosereported in our previous studies [21, 22]. However, in this study, complex sinusoidal roughness textures containing two different spatial wavelengths were additionally investigated, whereas the previous studies [21, 22] werelimited to simple grating scales. The complexity of the roughness textures was an important factor in the present
study. Humans can judge the roughness features of simple grating scales simply based on the relationship between the skin vibration frequency and the finger speed used to scan the surface. However, it may be difficult
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

Tactile Texture Display Combining Vibrotactile and Electrostatic-friction Stimuli• 18:3

to apply this simple strategy to complex roughness textures. We expected that presentation of multidimensional
tactile stimuli would improve the texture realism, as reported in previous studies [10, 21, 22, 31, 39]. Hence, we
had a naive expectation that the number of available feedback cues constitutes a determinant factor of subjective realism. Liu demonstrated the effectiveness of conjunct stimuli using vibrotactile and electrostatic-friction
stimuli for complex textures generated from images [31], and our experiment corresponds to a follow-up test
under conditions that can be compared with real stimuli.
As in Section 5, the second experiment evaluated the stimuli in terms of the behavioral aspects pertaining to
texture recognition. We conducted an identification task using five different sinusoidal surface roughness textures. The virtual stimuli were presented using one of three stimulus types: vibrotactile stimuli only, electrostaticfriction stimuli only, and their conjunct stimuli. They were then compared with the actual roughness textures.
We expected that the conjunct stimuli would yield improved textural identification performance based on a multisensory principle, in which behavioral performances are enhanced by combined cues when those for unisensory
conditions are similar [40].
As in Section 6, the third experiment compared the spatial wavelength discrimination thresholds of the threestimulus conditions. Virtual stimuli presented on a tactile display exhibit a larger discrimination threshold thanreal stimuli [28, 32, 38], and the discrimination threshold is one of the indicators used to evaluate tactile display
performance. In this study, we expected that the spatial wavelength discrimination thresholds would be reducedby the conjunct vibrotactile and electrostatic-friction stimuli. Note that this expectation or hypothesis is also
consistent with the principle of multisensory tasks [11, 18, 40], where combination of unreliable sensory cues
improves behavioral performance.
As mentioned above, the main purpose of this study was to investigate the superiority of combined vibrotactile and electrostatic-friction stimuli from multiple perspectives, including subjective realism and behavioral
responses. However, our objective was not to consider all aspects of performance evaluation for tactile texture
displays, as discussed in Section 7. This study was approved by the Institutional Review Board of the School ofEngineering, Nagoya University (#20-8).

3 EXPERIMENT APPARATUS: TACTILE TEXTURE DISPLAY USING COMBINED VIBROTACTILE
AND ELECTROSTATIC-FRICTION STIMULI
We used a tactile display that can present vibrotactile and electrostatic-friction stimuli simultaneously, as shownin Figure 1. The same apparatus was used in our previous studies [21, 22] but the surface was touched using athin conductive rubber pad. In the present study, this pad was not used so as to better replicate actual consumer
usage of touch panels.
A vibrotactile display uses an actuator to drive the panel and deform the finger pads. Here, voice-coil actuators(X-1741, Neomax Engineering Co. Ltd., Japan) located at the four bottom corners of the panel were used as
actuators. These four actuators were driven synchronously by an audio amplifier (FX-AUDIO- FX-502J, North
Flat Japan Co. Ltd., Japan). Vibrotactile stimuli were presented normal to the panel. The mechanical resonance of
our tactile display was 20–30 Hz. Resonance was not a problem during our experiments as stimulus presentation
occurs primarily in a larger band than this. Further, during an active exploration, the finger velocity continuously
changes (e.g., Reference [55]), and the vibration at the resonance band might have occurred only transiently.
Generally, voice-coil actuators attenuate their output at high frequencies; however, in our setup, the stimulus
was easily felt even when rubbing the finest textures without any frequency formation of applied voltages.
In an electrostatic display, the friction force is changed by controlling the electrostatic attractive force createdby applying a voltage between the finger pad and panel. This friction force varies with the magnitude of theapplied voltage and promotes deformation of the finger pad in the shear direction. Therefore, these stimuli
are effective for presenting texture surfaces where friction forces dominate [12, 58]. In the apparatus used inthis study, electrostatic-friction stimuli were presented by applying a voltage between the finger pad and an
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

18:4 •

K. Otake et al.

Fig. 1. Photograph and schematic view of tactile texture display used in experiments.

indium tin oxide panel covered with an 8-μm-thick insulating film (Kimotect PA8X, KIMOTO Co. Ltd., Japan).
The participant experienced the stimuli by touching the panel while holding a stainless-steel rod connected to
the ground. The applied voltage was amplitude-modulated with a carrier frequency of 20 kHz. The modulated
voltage signal was amplified by a voltage amplifier (HJOPS-1B20, Matsusada Precision Inc., Japan, maximum
output: ±1 kV, response: 75 kHz).
Four load cells (FSS015WNSX, Honeywell Co. Ltd., USA) were placed between the voice-coil motors and panel.
Each load cell was located above each voice-coil motor. We estimated the position and velocity of the finger padfrom the ratio of the outputs of the four load cells. The two voltage amplifiers and load cells were connected to acomputer through a DAQ board (PEX-361216, Interface Corporation, Japan). The sampling and control frequency
was 667 Hz.

4

EXPERIMENT 1: COMPARISON OF REALISM AMONG THREE STIMULUS CONDITIONSIn the first experiment, we tested whether combined vibrotactile and electrostatic-friction stimuli improve therealism of virtual textures.

4.1 Ranking Task
The participants experienced vibrotactile, electrostatic-friction, and conjunct stimuli for each of the five types ofsinusoidal or complex surface and ranked the three stimulus conditions in terms of realism for each surface type.
They were instructed to judge the similarity between the virtual and real roughness textures. During the stimulus
experience, the participants could touch the real roughness textures that served as models for the virtual stimuli.
In each trial, the participants were allowed to touch the virtual stimuli and the actual textures freely and no time
limit was imposed. This task was performed twice for each participant under three stimulus conditions. We onlyused the results of the second task, and the first task was considered as a training session. We did not provide theparticipants any instructions on their scanning speed. In fact, they did not trace continuously at a speed slower
than approximately 50 mm/s, because it was difficult for them to perceive the texture stimuli presented by thedisplay at such slow speeds. This was also the case in the later experiments 2–3. The participants washed theirhands and removed excess water and oil from their fingertips before the experiment.

4.2 Participants
Ten students (males in their 20s) participated in the experiment after providing written informed consent. All
participants were paid and were unaware of the purpose of the experiment.

4.3 Actual Roughness Textures
The roughness textures shown in Figure 2 were used as the real stimuli. The parameters of these textures are
summarized in Table 1. Three of these textures had two spatial wave components while the remaining two
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

Tactile Texture Display Combining Vibrotactile and Electrostatic-friction Stimuli• 18:5

Fig. 2. Images of roughness samples.
Table 1. λ and h Values of Grating Scales, Where λ and h
Are the Spatial Wavelength and Peak-to-peak Height of theGrating Scales, Respectively

Sinusoidal surface
Single type
Synthesized typeLabel
λ 1 (mm) λ 2 (mm) h (mm)Texture 1
1.5—
0.5
Texture 2
2.5—
0.5
Texture 3
1.3
1.9
0.3
Texture 4
1.3
2.5
0.3
Texture 5
1.9
2.5
0.3

had one spatial wave component. The latter two textures were also used in previous studies by our researchgroup [21, 22]. These parameters were determined by considering the resolution of the three-dimensional printer(Form3, Formlabs Co. Ltd., USA) used to produce the roughness textures. Here, λ is the spatial wavelength andh is the peak-to-peak amplitude. Sinusoidal waves of two different spatial wavelengths were synthesized with aphase difference of zero:




2πx
2πx
y = h sin
+ h sin,(1)
λ1
λ2where y and x are the surface height and lateral position of the roughness texture as shown in Figure 2 Each of
the five roughness textures were labeled as Textures 1 to 5, as shown in Table 1, and represent them thereafter.

4.4 Virtual Roughness Textures
Virtual roughness was presented using vibrotactile and electrostatic-friction stimuli. We used a tactile stimulation
algorithm to deform the skin based on the relationship between the finger motion and surface displacement, as
described below [1, 63].
4.4.1 Vibrotactile Stimuli Simulating Roughness Textures. The driving force Fv (t ) of the voice-coil actuator
was set as shown in Equation (2) to transmit the skin vibration caused by tracing a roughness texture surface,x (t )x (t )Fv (t ) = A sin 2π+ B sin 2π
,
(2)
λ1
λ2
where A, B represent the gain of each sine wave, x represents the finger position, and λ 1 and λ 2 represent the
spatial wavelengths of the roughness textures (set to the values listed in Table 1). When single-sinusoidal roughness textures were presented, Fv (t ) was given by the intensity of only the first term on the right-hand side ofEquation (2).
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

18:6 •

K. Otake et al.
Table 2. Amplitude of Panel Acceleration and Displacement for
Vibrotactile Stimuli

Label
Texture 1
Texture 2
Texture 3
Texture 4
Texture 5

Acceleration (mm/s2 )No load
1N
3.88
3.44
8.15
4.83
7.31
4.83
8.49
5.32
7.55
5.18

Displacement (mm)No load
1N
0.010
0.009
0.057
0.034
—
—
—
—
—
—

The acceleration was measured under no load and 1 N applied load (assumed
finger load). The displacement results under no load and 1 N applied load arealso listed. However, those for the roughness textures containing two spatial
wavelengths are excluded because of the difficulty in converting acceleration
to amplitude.

Table 3. Amplitude of Panel Acceleration and Displacement for
Conjunct Stimuli

Label
Texture 1
Texture 2
Texture 3
Texture 4
Texture 5

Acceleration (mm/s2 )No load
1N
3.66
3.55
6.55
4.25
6.70
4.72
8.71
5.01
7.22
5.09

Displacement (mm)No load
1N
0.009
0.009
0.046
0.030
—
—
—
—
—
—

The acceleration was measured for no load and 1 N applied load (assumed
finger load). The displacement results under no load and 1 N applied load arealso listed. However, those for the roughness textures containing two spatial
wavelengths are excluded because of the difficulty in converting acceleration
to amplitude.

The vibrotactile stimuli were investigated using a panel-mounted accelerometer (Vibration pickup 2302B,
Showa Sokki Co., Ltd., Japan). The amplitudes of the vibration component of each vibrotactile and conjunctstimulus, or half the peak-to-peak acceleration, are listed in Tables 2 and 3. The panel acceleration was measured
with no load and with an applied load of 1 N (assumed finger load). The frequency used to calculate the amplitude from the acceleration was calculated assuming that the finger pad slides at 150 mm/s. For example, tracingof a roughness texture with λ = 1.5 mm at 150 mm/s yields a frequency of 100 Hz. Note that, for the roughness textures containing two spatial wavelengths, only the acceleration is displayed because of the difficulty inconverting from acceleration to displacement. From Table 2, the intensity of the stimuli containing one spatial
wavelength exceeded the absolute threshold amplitude of the finger pad [5], and all vibrotactile stimuli had sufficiently high magnitudes that they could be perceived by the participants. These magnitudes were determinedusing the method described in Section 4.4.3.
4.4.2 Electrostatic-friction Stimuli Simulating Roughness Textures. The relationship between the shear force
Fe (t ) and the applied voltage Ve (t ) generated by tracing the panel can be expressed as Equation (3), from theelectrostatic force and Coulomb’s friction law [33],1
Fe (t )=

μ{W + kVe (t ) 2 }(3)

1 This equation cannot explain some phenomena [34]. Persson et al. suggested that electrostatic-friction occurring in the human fingertips

is due to a mechanism in which voltage application increases the real contact area and, thus, increases the adhesion friction force [41].
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

Tactile Texture Display Combining Vibrotactile and Electrostatic-friction Stimuli• 18:7

Table 4. Maximum Voltage Applied to Panel (Ve ) for Electrostatic-friction Condition
and Conjunct-stimuli Condition

Label
Texture 1
Texture 2
Texture 3
Texture 4
Texture 5

Electrostatic-friction stimuli (V)
12.2
19.8
20.4
17.6
17.2

Conjunct stimuli (V)
11.2
13.4
17.0
13.8
15.8

The values for the electrostatic-friction stimuli and conjunct stimuli are listed in the second andthird columns, respectively.

where μ, W , and k are the coefficient of friction of the panel, finger load, and constant for electrostatic force,
respectively. The shear force produced when tracing a finger across an uneven surface is expressed as a gradientfunction of the surface displacement [13, 43]. Thus, the phase of Fe (t ) is π /2, different from that of the surface
displacement. Note that, whereas the expression for Fv (t ) (Equation (2)) incorporates the sinusoidal function,that for Fe (t ) (Equation (4)) involves the cosine function:







x (t )x (t )Fe (t ) = μ W + k C cos 2π+ D cos 2π+C + D .
(4)
λ1
λ2
Here, C and D represent the gains of each sinusoidal wave. From Equation (3), Ve (t ) is expressed as
Reference [12]:





x (t )x (t )Ve (t ) = C cos 2π+ D cos 2π+ C + D.
(5)
λ1
λ2This method can be extended depending on the number of sinusoidal waves to be synthesized. In this study,
the actual voltage applied to the panel was Ve (t ) with amplitude modulation applied at a base frequency of 20
kHz. When the single-sinusoidal roughness texture was presented, Ve (t ) was given by the intensities of only the
first and third terms on the right-hand side of Equation (5). The intensities of the electrostatic-friction stimuli
should be determined by measuring the friction force; however, in our device, it was difficult to implement a newload cell to measure the magnitude of the electrostatic-friction force. Instead, the maximum applied voltages arelisted in Table 4.
4.4.3 Adjustment of Virtual Roughness Stimuli Strengths. The amplitudes (A, B, C, and D) were determined bythree people who were familiar with the displays. They included two authors of this article. An informal methodof adjustment was used to tune the tactile stimulus intensities so that they resembled the tactile sensations generated by the real roughness textures. First, each tuner adjusted the gains of A and B in the vibrotactile-stimuli-only
condition. Next, they adjusted the gains of C and D for the electrostatic-friction-stimuli only condition. They
then adjusted the gains of conjunct stimuli while fixing the ratios of A to B and C to D determined in the preceding processes. Finally, the mean gains determined by the three experts was adopted for the main experiment.
As a result, in the conjunct condition, the intensities of the vibrotactile stimuli were stronger than that of theelectrostatic-friction stimuli, which is consistent with the results of user studies in Reference [22]. In this process,the tuners attempted to equalize the subjective magnitudes of the stimuli under the three stimulus conditions.
We implemented this decision method for practical reasons. Other possible decision methods include matching
the normal displacement of the finger pad when sliding over actual and virtual surfaces, and determining thefriction-force magnitude using a feedback control based on measurement of this force; however, our display was
not equipped with load cells. See the last paragraph in Section 7 about the potential problems with these methods
when determining the gain parameters.
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

18:8 •

K. Otake et al.

4.5 Result
Figure 3 shows the proportions of the ranks determined by the participants for each of the five types of stimuli.
Figure 3(f) and (g) summarize the results for the simple sinusoidal and complex textures, respectively, while
Figure 3(h) presents summarized results for all roughness textures. Here, * and ** represent the statistical differences between two stimulus conditions at p = 0.05 and 0.01, respectively, with Bonferroni correction.
Figure 3(a) shows the rank proportions for Texture 1. Seven of 10 participants chose the conjunct stimulus
as the most realistic condition. There was a significant difference between the answers for the vibrotactile andconjunct stimuli (Conjunct-Vibrotactile: z = 2.61, p = 0.027, Wilcoxon signed-ranked test).
Figure 3(b) shows the rank proportion results for Texture 2. Five of 10 participants chose the conjunct stimulus
as the most realistic condition. There was a significant difference between the answers for the electrostaticfriction stimulus and the other two stimulus conditions (Vibrotactile-Friction: z = 2.42, p = 0.046, ConjunctFriction: z = 3.18, p = 0.0045).
Figure 3(c) shows the rank proportion results for Texture 3. Five of 10 participants chose the vibrotactile stimulus as the most realistic condition. There was a significant difference between the answers for the electrostaticfriction stimuli alone and the conjunct stimuli (Conjunct-Friction: z = 2.62, p = 0.026).
Figure 3(d) shows the rank proportion results for Texture 4. Six of 10 participants chose the conjunct stimulus
as the most realistic condition. There were no significant differences between any of the stimulus conditions.
Figure 3(e) shows the rank proportion results for Texture 5. Six of 10 participants chose the conjunct stimulus
as the most realistic condition. There was a significant difference between the answers for the vibrotactile andconjunct stimuli (Conjunct-Vibrotactile: z = 2.53, p = 0.034).
Figure 3(f) shows the total rank proportion of Textures 1 and 2. The answer rates for the conjunct stimuli weresignificantly different from those for the other stimuli (Conjunct-Vibrotactile: z = 2.62, p = 0.026, ConjunctFriction: z = 3.71, p = 6.3 × 10−4 ). In particular, the p-values between the conjunct stimuli and electrostaticfriction stimuli were smaller, and there was a distinct difference in realism for these two stimulus conditions.
Figure 3(g) shows the total rank proportion of the composite roughness textures: Textures 3, 4, and 5. The
answer rates for the conjunct stimuli differed significantly from those for the other stimuli (Conjunct-Vibrotactile:z = 2.88, p = 0.012, Conjunct-Friction: z = 4.23, p = 6.9 × 10−5 ). In particular, the p-values between the conjunctstimuli and electrostatic-friction stimuli became smaller, and there was a large difference in realism quality.
Figure 3(h) shows the total rank proportions for all roughness textures. In this case, the answer rates for the
conjunct stimuli differed significantly from those for the other stimuli (Conjunct-Vibrotactile: z = 3.92, p =2.6 × 10−4 , Conjunct-Friction: z = 5.66, p = 4.7 × 10−8 ). In addition, the answer rates for the vibrotactile andelectrostatic-friction stimuli differed significantly (Vibrotactile-Electrostatic: z = 2.44, p = 0.043).

5

EXPERIMENT 2: ROUGHNESS TEXTURE IDENTIFICATION

We investigated the identification performance for five different roughness textures under each of the threestimulus conditions. For reference, participants also performed a task in which they identified the real roughness
textures. In this experiment, we focused on two points: whether superior identification performance is achievedfor conjunct stimuli compared to vibrotactile stimuli alone or electrostatic-friction stimuli alone and the extent
of the difference in identification performance for real and virtual roughness textures.

5.1 Identification Task Procedure
We conducted a task to identify the corresponding stimuli from five virtual stimuli that simulated each of the five
real roughness textures. After a brief explanation of the task, the participants experienced three virtual stimuli
and real roughness textures for a few minutes as a training session. The participants were asked to choose oneof the five options for the virtual stimuli or real roughness textures that corresponded to the real roughness
textures presented. In a single set, each participant conducted one trial for each of the four stimulus conditions
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

Tactile Texture Display Combining Vibrotactile and Electrostatic-friction Stimuli• 18:9Fig. 3. Rank proportions for each stimulus condition and surface, where ∗ and ∗∗ indicate p < 0.05 and p < 0.01, respectively.

(vibrotactile stimuli, electrostatic-friction stimuli, and their conjunct stimuli plus actual stimulus condition) in
randomized order. In total, 10 sets were conducted for individuals. During the experiment, the participants wore
headphones and listened to pink noise. They also wore sunglasses with tape attached to the lenses. The experiment time was approximately 1.5 h per person, including breaks.

5.2 Participants
Ten university students (males in their 20s) participated in the experiment after providing written informed
consent. Nine of these students also participated in Experiments 1 and 3. All participants were paid and were
unaware of the purpose of the experiment.
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

18:10 • K. Otake et al.

5.3 Analysis
The accuracy rate was calculated by dividing the total number of correct answers by the total number of trials,
i.e., 10. A correct answer was recorded when a virtual stimulus corresponding to a real roughness texture wasselected. First, 95% confidence intervals were used to determine whether the accuracy rate was significant. If thelower boundary of this interval exceeded the chance probability (0.2), then the roughness texture was identified
with significance. Next, we investigated whether there was a difference in the distribution of the mean accuracy
rates among the three stimulus conditions using the chi-square distribution. For any two of the three stimulus
conditions, a Bonferroni correction was applied. Finally, the mean accuracy rates for each stimulus condition
were compared.

5.4 Result
Table 5 lists the identification results for the four stimulus conditions.
For the task where the real roughness textures were identified in Table 5(a), all roughness textures were identified at a rate higher than the chance probability. In particular, all participants correctly identified Textures 1
and 5. For the other three textures, the accuracy rate exceeded 60%. The mean accuracy rate for this condition
was 83%.
The lowest mean accuracy rate was obtained for the vibrotactile-stimulus identification task (Table 5(b)). For
Texture 1, the identification accuracy rate exceeded 90%. However, for Textures 2, 3, 4, and 5, the identification
rates were close to the chance probability. The mean accuracy rate for this condition was 39%.
For the electrostatic-friction stimulus identification task (Table 5(c)), Textures 1, 4, and 5 were correctly identified at rates exceeding the chance probability following consideration of the 95% confidence interval. For theother two roughness textures, the identification accuracy rates were close to the chance probability. The mean
accuracy rate for this condition was 43%.
For the conjunct-stimulus identification task (Table 5(d)), Textures 1, 4, and 5 were correctly identified at
rates exceeding the chance probability following consideration of the 95% confidence interval. For the other tworoughness textures, the identification accuracy rates were close to the chance probability. The mean accuracy
rate for this condition was 46%.
Table 6 lists the test results obtained using the chi-square distribution. The accuracy rates for the five texturesdiffered between the vibrotactile and conjunct stimuli and between the vibrotactile and electrostatic-friction
stimuli. The mean accuracy rates are shown in Figure 4. The mean accuracy rate for the conjunct stimuli exceededthat of the vibrotactile stimuli (d f = 9, t = −4.45, p = 0.0015 < 0.01/3). Further, the mean accuracy rate for thereal roughness textures was far above those for the virtual textures.

6

EXPERIMENT 3: COMPARISON OF SPATIAL WAVELENGTH DISCRIMINATION ABILITY FOR
THREE STIMULUS CONDITIONSIn this experiment, we investigated whether a difference exists in the spatial wavelength discrimination thresholdof sinusoidal roughness stimuli for vibrotactile stimuli, electrostatic-friction stimuli, and their conjunct stimuli.

6.1 Discrimination Task
In this experiment, the spatial wavelength discrimination threshold was investigated using a single sinusoidal
stimulus. This approach was adopted because it is very difficult to perceive changes in spatial wavelengths when
the stimuli have complex shapes that include multiple wavelengths. The standard stimuli were the roughness
textures with Textures 1 and 2 used in Experiment 1. We prepared virtual sinusoidal stimuli with spatial wavelengths (λ) of 0.7–2.3 and 1.7–3.3 mm as comparison stimuli, respectively.
In this experiment, we used the psychophysical method of limits. To prevent errors due to participants’ familiarity and expectations, the initial λ value in each series was randomized. This experiment investigated
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

Tactile Texture Display Combining Vibrotactile and Electrostatic-friction Stimuli• 18:11

Table 5. Accuracy Proportions for Identification Task: Mean ± 95% Confidence Interval

Actual roughness texture
Texture 2
Texture 3
Texture 4
Texture 5
0.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00
(a)
0.64 ± 0.09 0.08 ± 0.05 0.28 ± 0.09 0.00 ± 0.00
Actual0.08 ± 0.05 0.81 ± 0.08 0.11 ± 0.06 0.00 ± 0.00
roughness
0.28 ± 0.09 0.11 ± 0.06 0.61 ± 0.10 0.00 ± 0.00
texture0.00 ± 0.00 0.00 ± 0.00 0.00 ± 0.00 1.00 ± 0.00
Actual roughness texture
Label
Texture 1
Texture 2
Texture 3
Texture 4
Texture 5
Texture 1 0.93 ± 0.05 0.00 ± 0.00 0.03 ± 0.03 0.01 ± 0.02 0.03 ± 0.03(b)
Texture 2 0.03 ± 0.03 0.23 ± 0.08 0.38 ± 0.10 0.22 ± 0.08 0.16 ± 0.07
Vibrotactile Texture 3 0.00 ± 0.00 0.38 ± 0.10 0.11 ± 0.06 0.26 ± 0.09 0.24 ± 0.08
Texture 4 0.02 ± 0.03 0.22 ± 0.08 0.16 ± 0.07 0.31 ± 0.09 0.24 ± 0.08
stimuli
Texture 5 0.02 ± 0.03 0.17 ± 0.07 0.32 ± 0.09 0.19 ± 0.08 0.33 ± 0.09
Actual roughness texture
Label
Texture 1
Texture 2
Texture 3
Texture 4
Texture 5
Texture 1 0.87 ± 0.07 0.10 ± 0.06 0.01 ± 0.02 0.00 ± 0.00 0.02 ± 0.03
(c)
Texture 2 0.03 ± 0.03 0.19 ± 0.08 0.30 ± 0.09 0.36 ± 0.09 0.12 ± 0.06
Electrostatic
Texture 3 0.07 ± 0.05 0.38 ± 0.10 0.15 ± 0.07 0.11 ± 0.06 0.29 ± 0.09
friction
Texture 4 0.03 ± 0.03 0.27 ± 0.09 0.19 ± 0.08 0.44 ± 0.10 0.06 ± 0.05
stimuli
Texture 5 0.00 ± 0.00 0.06 ± 0.05 0.34 ± 0.09 0.09 ± 0.06 0.51 ± 0.10
Actual roughness texture
Label
Texture 1
Texture 2
Texture 3
Texture 4
Texture 5
Texture 1 0.95 ± 0.04 0.02 ± 0.03 0.01 ± 0.02 0.00 ± 0.00 0.01 ± 0.02
(d)
Texture 2 0.01 ± 0.02 0.21 ± 0.08 0.49 ± 0.10 0.17 ± 0.07 0.13 ± 0.07
Conjunct Texture 3 0.02 ± 0.03 0.43 ± 0.10 0.10 ± 0.06 0.29 ± 0.09 0.15 ± 0.07
Texture 4 0.00 ± 0.00 0.21 ± 0.08 0.22 ± 0.08 0.45 ± 0.10 0.14 ± 0.07
stimuli
Texture 5 0.02 ± 0.03 0.13 ± 0.07 0.19 ± 0.08 0.08 ± 0.05 0.57 ± 0.10
Label
Texture 1
Texture 2
Texture 3
Texture 4
Texture 5

Texture 1
1.00 ± 0.00
0.00 ± 0.00
0.00 ± 0.00
0.00 ± 0.00
0.00 ± 0.00

Five different sinusoidal roughness textures defined in Table 1 were compared. Cells with proportions greater thanthe chance are colored light gray and dark gray, considering the 95% and 99% confidence intervals, respectively.

Table 6. Results of Goodness-of-fit Test for Accuracy Rates with
Bonferroni Correction

Vibrotactile
Electrostatic-friction

Electrostatic-friction
χ 2 = 12.52
p = 0.013 < 0.05/3—

Vib. + Fric.
χ 2 = 24.08
p = 7.6 × 10−5 < 0.001/3
χ 2 = 3.34
p = 0.5 > 0.05/3

vibrotactile and electrostatic-friction stimuli and a combination of the two. The three stimulus types were testedon the participants in random order. For each standard stimulus, three ascending and three descending series
were implemented; therefore, a total of 18 series were implemented for individuals. Before the main tasks, a training session was conducted using a stimulus condition that was randomly selected among the three conditions.
Only one ascending and descending series was performed during the training session.
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

18:12 • K. Otake et al.

Fig. 4. Experiment 2 (identification task) results. Mean and standard errors of accuracy rates among the ten participants.

6.2 Participants
Ten university students (males in their 20s), identical to those in Experiment 1, participated in the experiment after providing written informed consent. All participants were paid and unaware of the purpose of theexperiment.

6.3 Analysis
The mean values of the upper and lower thresholds obtained from each of the six series of trials were calculated
for each of the three virtual stimulus conditions. The discrimination threshold was determined by subtracting
the mean of the lower threshold from the mean of the upper threshold and halving that value. For any two of thethree stimulus conditions, we investigated the possible difference between them using a t-test that considered
the Bonferroni correction.

6.4 Result
Figure 5 shows the results of the spatial wavelength discrimination threshold measurement when the standard
stimuli were Textures 1 and 2. For Texture 1, the discrimination threshold of the electrostatic-friction stimuli
was 0.42 ± 0.08 mm (mean ± standard error), which exceeded that of the conjunct stimuli at 0.37 ± 0.08 mm(t = 2.98, p = 0.015 < 0.05/3). There was little difference in discrimination threshold between the vibrotactile
and conjunct stimuli. For Texture 2, the discrimination threshold of the electrostatic-friction stimuli was 0.50 ±0.09 mm, exceeding that of the conjunct and vibrotactile stimuli at 0.39 ± 0.09 mm (t = 3.95, p = 0.003 < 0.01/3)
and 0.38 ± 0.09 mm (t = −4.86, p = 0.001 < 0.01/3), respectively.

7

DISCUSSION

By reviewing studies of tactile texture displays, we can classify their realism evaluation methods into threemain categories: methods based on subjective reports [1, 3, 8, 20–23, 27, 39, 44, 48, 52], on psychophysical behavioral responses [1, 3, 4, 12, 14, 31, 38, 46, 50, 56, 61, 63], and on measurement of the physical quantity ofstimuli [8, 20, 59], as shown in Figure 6. Furthermore, subjective evaluation methods can be divided into direct [1, 8, 22, 23, 31, 39, 44, 48, 52] and indirect [1, 3, 8, 20, 27, 52] methods. The direct method incorporates
ranking tasks to compare the realism of several stimulus conditions and tasks to rate the realism for each of themultiple conditions. The indirect method incorporates tasks to score virtual stimuli along specific perceptual
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

Tactile Texture Display Combining Vibrotactile and Electrostatic-friction Stimuli• 18:13

Fig. 5. Experiment 3 results. The mean and standard errors of the spatial wavelength discrimination thresholds for λ = 1.5
and 2.5 mm are shown. Here, ∗ and ∗∗ indicate p < 0.05 and p < 0.01, respectively.

dimensions, such as roughness or hardness. The magnitude estimation method is a typical approach of this type.
The relationship between the subjective quantity and the actual stimuli is expected to hold for virtual stimuli as
well.
Evaluation methods based on psychophysical behavioral responses consist of methods for estimating psychophysical thresholds [1, 3, 4, 12, 38, 46, 56, 61] and those to classify or identify virtual stimuli [14, 35, 50, 61, 63].
The threshold measurement method is often used to evaluate human responses to a single physical quantity incorporated in virtual stimuli. The closer the thresholds for the actual and virtual stimuli, the higher the qualityof the virtual stimuli. In general, the discrimination thresholds for virtual stimuli exceed those for actual stimuli.
The classification or identification method is used when multiple physical quantities in virtual stimuli are manipulated. It is better if the correct and erroneous classification or identification rates for the actual and virtual
stimuli are similar.
For the physical-quantity measurement method, the actual and virtual stimuli are compared. Note, however,that physical resemblance and perceptual resemblance are different [8, 17, 36, 51].
Of the above-mentioned evaluation methods, this study considered aspects of subjective reports and psychophysical behavioral responses. Note that our tactile display is incapable of measuring shear friction forces
and only a limited range of physical quantities can be evaluated.
In Experiment 1, the realism of virtual stimuli imitating complex and simple roughness textures was enhanced
by combining vibrotactile and electrostatic-friction stimuli for our surface tactile display. As regards the realismof simple sinusoidal surfaces, the conjunct condition was found to be superior to the vibrotactile and electrostatic
conditions. This is consistent with the results of our previous studies [21, 22]. Furthermore, the superiority of
conjunct stimuli was also demonstrated in the case of the complex roughness stimuli. These results suggest that,when simulating textured surfaces, not only the skin deformation caused by the roughness but also the spatial
distribution of the friction should be represented to achieve subjective realism.
In Experiment 2, we investigated whether virtual stimuli combining two principles can improve surfacefeature identification performance. In this identification task, the virtual stimuli do not necessarily have to berealistic. Provided the different surfaces are rendered differently, the identification performance remains high.
Therefore, this task was an independent test from the direct investigation of realism performed in Experiment
1. We expected that the combination of the vibrotactile and friction stimuli would yield accurate identificationof multiple roughness textures, because simultaneous presentation of two types of tactile stimuli increases thenumber of perceptual dimensions, as mentioned in Section 1. However, although the combined stimuli improved

ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

18:14 • K. Otake et al.

Fig. 6. Classification of evaluation methods for tactile texture display realism. The colored items were incorporated in thisstudy.

the surface texture identification performance, the effect was marginal. That is, the conjunct stimuli increased
the identification probability by only 2–7%. In addition, the identification probability for the real roughness textures was 83%, which far exceeds that of the conjunct condition (46%). The main reason for the low identification
probability observed for the virtual stimuli may be related to the surface of the display. That is, the surface display
does not reproduce the skin-deformation spatial distribution that occurs on the finger pad when actual macroscopic roughness textures are touched. These spatial cues are known to be the major determinant for perceiving
macroscopically rough surfaces [24, 49, 60]. In particular, the macroscopic features might have been dominant
in the identification task using real roughness textures. For example, Texture 2 was often confused with Texture
4, as detailed in Table 5(a). This was likely due to the similarity of the spatial skin deformations within the finger
pad for these two surface types, because they contain the same spatial wavelength component, i.e., λ = 2.5 mm.
In contrast, for the identification tasks using virtual stimuli, Textures 2 and 3 were often confused, as detailedin Table 5(b)–(d). The roughness textures produced by an electrostatic-friction display can reverse the effect of
λ on roughness perception [20], where surfaces with greater λ values seem smoother. The perceived roughness
increases as λ increases to at least 3.0–4.0 mm for real roughness textures [6, 29, 54], whereas the virtual stimuli
on the panel become smaller as λ increases. We adjusted the voltage or perceived strength of the virtual stimulus for each texture to avoid this phenomenon, but could not completely avoid it, and a reversal effect like thatreported by [20] might have occurred.
In Experiment 3, we compared three stimulus conditions in terms of the spatial wavelength discriminationthresholds. We expected that the combined condition would yield small discrimination thresholds and foster
discernment of small textural changes. The results reported in Section 6.4 revealed no marked difference in discrimination threshold among the three stimulus conditions. Lederman et al. reported that roughness perception
accuracy is blurred when the spatial wavelength exceeds 1 mm [29, 30]. In our experiments, we used a wave with
λ = 1.3–2.5 mm, and it may be difficult to evaluate tactile texture display performance for this range of λ values.
In addition, the touch screen of the surface tactile display is flat. Thus, the participants could not utilize spatial
variation cues and they determined the roughness degree of the virtual stimuli based on the temporal variationof their skin deformation, i.e., the vibration. Therefore, high discrimination thresholds may not be expected forsurface tactile displays, because the spatial deformation of the skin is particularly important in the context ofmacroscopic roughness [7, 37, 60].
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

Tactile Texture Display Combining Vibrotactile and Electrostatic-friction Stimuli• 18:15

In summary, virtual stimuli combining vibrotactile and electrostatic-friction stimuli were found to have significant effects on realism based on subjective reports, but not necessarily with regard to behavioral responses,
throughout the three experiments.
The present study had several limitations. First, the intensity of each virtual texture was adjusted by threeexperts. However, in general, the friction magnitude in an electrostatic display varies depending on the conditionof the individual fingers. This means that the same voltage does not necessarily yield the same friction force. As
mentioned in Section 4.4.3, two of the authors, along with a third expert, were involved in determining thegain parameters of the stimuli. Hence, we cannot objectively deny that the gain parameters could be biased. We
recognize that we should have employed trained participants that were unaware of the objectives of the study.
Second, the roughness textures tested in this study were limited. In the future, finer roughness textures with
λ smaller than 1 mm should be tested. Third, the lack of gender diversity among participants in the present
study can be a concern when generalizing the results. Especially, our results were obtained from only maleparticipants, whereas there exist known differences in tactile perception between men and women [42]. Fourth,
as mentioned, a limitation of the surface texture display is that it does not simulate the spatial distribution of theskin deformation in the skin-panel contact area, whereas actual roughness textures yield such distribution. We
speculate that such a condition with no spatial distribution of skin deformation is similar to the condition whereroughness textures are scanned by using a stylus or rigid finger sack. Although some earlier studies suggest thatstylus conditions are similar to bare-finger conditions regarding roughness perception [26, 64, 65], the stylus
condition can be more difficult in roughness discrimination because of the limited tactile cues. An experimental
comparison between the stylus condition and virtual textures allows us to test our speculation that the texture
perceived on the panel is similar to the one perceived by using a stylus.

8 CONCLUSION
We attempted to render rough textures on a flat panel. To achieve this, we used a new tactile presentation
method to combine vibrotactile stimuli, which are suitable for presenting a sense of surface roughness, andelectrostatic-friction stimuli, which are suitable for generating a sense of friction. The method efficacy was thenevaluated based on its subjective effects on realism and its effects on behavioral responses for roughness textureswith simple and complex surface geometries. As regards the subjective aspect, we found that the combined
stimuli improved the realism of sinusoidal roughness textures with one spatial wavelength and complex texturesconsisting of two sinusoidal waves. In terms of behavioral responses, an identification task involving sinusoidalroughness textures was performed; the conjunct stimuli results were statistically superior to those obtained
when only the vibrotactile stimuli were used, but the difference was less clear than for the subjective aspect.
Furthermore, combination of the two types of virtual stimuli did not yield smaller discrimination thresholds forthe spatial surface wavelength. Thus, the effects of combined stimuli were prominent for subjectively reported
realism, but marginal or null for psychophysical performance. A combination of vibrotactile and electrostaticfriction stimuli is recommended for improving subjectively reported realism. When applied to natural materials
from regular roughness textures, the stimulus generation algorithm would need to be considered more deeply,
because it involves more fine and complex textures.

REFERENCES[1] S. Asano, S. Okamoto, Y. Matsuura, and Y. Yamada. 2014. Toward quality texture display: Vibrotactile stimuli to modify material roughness sensations. Adv. Robot. 28, 16 (2014), 1079–1089.
[2] C. Basdogan, F. Giraud, V. Levesque, and S. Choi. 2020. A review of surface haptics: Enabling tactile effects on touch surfaces. IEEE
Trans. Hapt. 13, 3 (2020), 450–470.
[3] O. Bau, I. Poupyrev, A. Israr, and C. Harrison. 2010. TeslaTouch: Electrovibration for touch surfaces. In Proceedings of the Annual ACM
Symposium on User Interface Software and Technology. New York, 283–292.
[4] S. Bochereau, S. Sinclair, and V. Hayward. 2018. Perceptual constancy in the reproduction of virtual tactile textures with surface displays.
ACM Trans. Appl. Percept. 15, 2, Article 10 (2018).
ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

18:16 • K. Otake et al.
[5] S. J. Bolanowski, G. A. Gescheider, R. T. Verrillo, and C. M. Checkosky. 1988. Four channels mediate the mechanical aspects of touch.
J. Acoust. Soc. Am. 84, 5 (1988), 1680–1694.
[6] C. E. Chapman, F. Tremblay, W. Jiang, L. Belingard, and E. Meftah. 2002. Central neural mechanisms contributing to the perception oftactile roughness. Behav. Brain Res. 135, 1 (2002), 225 – 233.
[7] C. E. Connor, S. S. Hsiao, J. R. Phillips, and K. O. Johnson. 1990. Tactile roughness: Neural codes that account for psychophysicalmagnitude estimates. J. Neurosci. 10, 12 (1990), 3823–3836.
[8] H. Culbertson and K. J. Kuchenbecker. 2017. Importance of matching physical friction, hardness, and texture in creating realistic haptic
virtual surfaces. IEEE Trans. Hapt. 10, 1 (2017), 63–74.
[9] H. Culbertson and K. J. Kuchenbecker. 2017. Ungrounded haptic augmented reality system for displaying roughness and friction.
IEEE/ASME Trans. Mechatr. 22, 4 (2017), 1839–1849.
[10] H. Culbertson, J. Unwin, and K. J. Kuchenbecker. 2014. Modeling and rendering realistic textures from unconstrained tool-surface
interactions. IEEE Trans. Hapt. 7, 3 (2014), 381–393.
[11] M. Ernst and M. Banks. 2002. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415 (2002),
429–433.
[12] T. Fielder and Y. Vardar. 2019. A novel texture rendering approach for electrostatic displays. In Proceedings of the International Workshop
on Haptic and Audio Interaction Design.
[13] Y. Fujii, S. Okamoto, and Y. Yamada. 2016. Friction model of fingertip sliding over wavy surface for friction-variable tactile feedback
panel. Adv. Robot. 30, 20 (2016), 1341–1353.
[14] D. Gueorguiev, E. Vezzoli, A. Mouraux, B. Lemaire-Semail, and J. Thonnard. 2017. The tactile perception of transient changes in friction.
J. R. Soc. Interface 14, article no: 20170641 (2017).
[15] R. Hassen, B. Guelecyuez, and E. G. Steinbach. 2020. PVC-SLP: Perceptual vibrotactile-signal compression based-on sparse linear prediction. IEEE Trans. Multimedia (2020), 1–1.
[16] R. Hassen and E. Steinbach. 2019. Vibrotactile signal compression based on sparse linear prediction and human tactile sensitivity
function. In Proceedings of the IEEE World Haptics Conference. 301–306.
[17] R. Hassen and E. Steinbach. 2020. Subjective evaluation of the spectral temporal similarity (ST-SIM) measure for vibrotactile quality
assessment. IEEE Trans. Hapt. 13, 1 (2020), 25–31.
[18] M. A. Heller. 1983. Haptic dominance in form perception with blurred vision. Perception 12, 5 (1983), 607–613.
[19] A. Imaizumi, S. Okamoto, and Y. Yamada. 2014. Friction sensation produced by laterally asymmetric vibrotactile stimulus. Hapt.: Neurosci.
Dev. Model. Appl. (2014), 11–18.
[20] A. Isleyen, Y. Vardar, and C. Basdogan. 2020. Tactile roughness perception of virtual gratings by electrovibration. IEEE Trans. Hapt. 13,
3 (2020), 562–570.
[21] K. Ito, S. Okamoto, H. Elfekey, H. Kajimoto, and Y. Yamada. 2017. A texture display using vibrotactile and electrostatic friction stimuli
surpasses one based on either type of stimulus. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics.
2343–2348.
[22] K. Ito, S. Okamoto, Y. Yamada, and H. Kajimoto. 2019. Tactile texture display with vibrotactile and electrostatic friction stimuli mixed
at appropriate ratio presents better roughness textures. ACM Trans. Appl. Percept. 16, 4 (2019).
[23] J. Jiao, Y. Zhang, D. Wang, Y. Visell, D. Cao, X. Guo, and X. Sun. 2018. Data-driven rendering of fabric textures on electrostatic tactile
displays. In Proceedings of the IEEE Haptics Symposium. 169–174.
[24] K. O. Johnson and S. S. Hsiao. 1994. Evaluation of the relative roles of slowly and rapidly adapting afferent fibers in roughness perception.
Can. J. Physiol. Pharmacol. 72, 5 (1994), 488–497.
[25] L. A. Jones and A. Singhal. 2019. Sensory interactions in cutaneous displays. In Proceedings of the IEEE World Haptics Conference. 545–
550.
[26] Roberta L. Klatzky, Susan J. Lederman, Cheryl Hamilton, Molly Grindley, and Robert H. Swendsen. 2003. Feeling textures through aprobe: Effects of probe and surface geometry and exploratory factors. Percept. Psychophys. 65, 4 (01 May 2003), 613–631.
[27] M. Konyo, H. Yamada, S. Okamoto, and S. Tadokoro. 2008. Alternative display of friction represented by tactile stimulation without
tangential force. In Haptics: Perception, Devices and Scenarios, 619–629.
[28] G. D. Lamb. 1983. Tactile discrimination of textured surfaces: Psychophysical performance measurements in humans. J. Physiol.
338 (1983), 551–565.
[29] M. A. Lawrence, R. Kitada, R. L. Klatzky, and S. J. Lederman. 2007. Haptic roughness perception of linear gratings via bare finger orrigid probe. Perception 36, 4 (2007), 547–557.
[30] S. J. Lederman, R. L. Klatzky, and C. L. Hamilton G. I. Ramsay. 1999. Perceiving surface roughness via a rigid probe: Effects of exploration
speed and mode of touch. Haptics-e 1, 1 (1999), 291–299.
[31] G. Liu, C. Zhang, and X. Sun. 2020. Tri-modal tactile display and its application into tactile perception of visualized surfaces. IEEE Trans.
Hapt. 13, 4 (2020), 733–744.
[32] J. W. Morley, A. W. Goodwin, and I. Darian-Smith. 1983. Tactile discrimination of gratings. Exp. Brain Res. 49 (1983), 291–299.

ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

Tactile Texture Display Combining Vibrotactile and Electrostatic-friction Stimuli• 18:17[33] T. Nakamura and A. Yamamoto. 2013. Multi-finger electrostatic passive haptic feedback on a visual display. In Proceedings of the IEEE
World Haptics Conference. 37–42.
[34] T. Nakamura and A. Yamamoto. 2017. Modeling and control of electroadhesion force in DC voltage. Robomech J. 4, 18 (2017).
[35] S. Okamoto. 2014. VR-MDS: Multidimensional scaling for classification tasks of virtual and real stimuli. Attent. Percept. Psychophys. 76,
3 (2014), 877–893.
[36] S. Okamoto, H. Nagano, and Y. Yamada. 2013. Psychophysical dimensions of tactile perception of textures. IEEE Transact. Hapt. 6,
1 (2013), 81–93.
[37] S. Okamoto and A. Oishi. 2020. Relationship between spatial variations in static skin deformation and perceived roughness of macroscopic surfaces. IEEE Transact. Hapt. 13, 1 (2020), 66–72.
[38] S. Okamoto, T. Yamauchi, M. Konyo, and S. Tadokoro. 2012. Virtual active touch: Perception of virtual gratings wavelength throughpointing-stick interface. IEEE Trans. Hapt. 5, 1 (2012), 85–93.
[39] K. Otake, H. Hasegawa, S. Okamoto, and Y. Yamada. 2020. Virtual roughness textures via a surface tactile texture display using vibrotactile and electrostatic friction stimuli: Improved realism. In Proceedings of the 13th International Conference on Human System Interaction.
147–152.
[40] T. U. Otto, B. Dassy, and P. Mamassian. 2013. Principles of multisensory behavior. J. Neurosci. 33, 17 (2013), 7463–7474.
[41] B. N. J. Persson. 2018. The dependency of adhesion and friction on electrostatic attraction. J. Chem. Phys. 148, 14 (2018), 144701.
[42] Ryan M. Peters, Erik Hackeman, and Daniel Goldreich. 2009. Diminutive digits discern delicate details: Fingertip size and the sex
difference in tactile spatial acuity. J. Neurosci. 29, 50 (2009), 15756–15761.
[43] G. Robles-De-La-Torre and V. Hayward. 2001. Force can overcome object geometry in the perception of shape through active touch.
Nature 412 (2001), 445–448.
[44] J. M. Romano and K. J. Kuchenbecker. 2012. Creating realistic virtual textures from contact acceleration data. IEEE Trans. Hapt. 5,
2 (2012), 109–119.
[45] S. Ryu, D. Pyo, B. Han, and D. Kwon. 2016. Simultaneous representation of texture and geometry on a flat touch surface. In Proceedingsof the International Conference on Haptic Interaction—Science, Engineering and Design (AsiaHaptics’16) 432 (2016).
[46] S. Ryu, D. Pyo, S. Lim, and D. Kwon. 2018. Mechanical vibration influences the perception of electrovibration. Sci. Rep. 8, 4555 (2018).
[47] S. Saga and K. Deguchi. 2012. Lateral-force-based 2.5-dimensional tactile display for touch screen. In Proceedings of the IEEE Haptics
Symposium. 15–22.
[48] S. Saga and R. Raskar. 2013. Simultaneous geometry and texture display based on lateral force for touchscreen. In Proceedings of theIEEE World Haptics Conference. 437–442.
[49] K. Sathian, A. W. Goodwin, K. T. John, and I. Darian-Smith. 1989. Perceived roughness of a grating: Correlation with responses ofmechanoreceptive afferents innervating the monkey’s fingerpad. J. Neurosci. 9, 4 (1989), 1273–1279.
[50] S. Smith, G. Smith, and J.-L. Lee. 2015. The effects of realistic tactile haptic feedback on user surface texture perception. J. Vibroeng. 17,
2 (2015), 1004–1016.
[51] E. Steinbach, S. Hirche, J. Kammerl, I. Vittorias, and R. Chaudhari. 2011. Haptic data compression and communication. IEEE Sign. Process.
Mag. 28, 1 (2011), 87–96.
[52] M. Strese, R. Hassen, A. Noll, and E. Steinbach. 2019. A tactile computer mouse for the display of surface material properties. IEEE Trans.
Hapt. 12, 1 (2019), 18–33.
[53] X. Sun, C. Zhang, and G. Liu. 2020. Improved tactile perception of 3D geometric bumps using coupled electrovibration and mechanicalvibration stimuli. Comput. J. (2020).
[54] A. Sutu, E. Meftah, and C. E. Chapman. 2013. Physical determinants of the shape of the psychophysical curve relating tactile roughness
to raised-dot spacing: Implications for neuronal coding of roughness. J. Neurophysiol. 109, 5 (2013), 1403–1415.
[55] Y. Tanaka, W. M. Bergmann Tiest, A. M. L. Kappers, and A. Sano. 2014. Contact force and scanning velocity during active roughness
perception. PLoS One 9, 3 (2014), e93363.
[56] J. L. Tennison, P. M. Uesbeck, N. A. Giudice, A. Stefik, D. W. Smith, and J. L. Gorlewicz. 2020. Establishing vibration-based tactile line
profiles for use in multimodal graphics. ACM Trans. Appl. Percept. 17, 2, Article 7 (2020), 14 pages.
[57] H. Tomita, S. Saga, H. Kajimoto, S. Vasilache, and S. Takahashi. 2018. A study of tactile sensation and magnitude on electrostatic tactile
display. In Proceedings of the IEEE Haptics Symposium. 158–162.
[58] Y. Vardar, A. Isleyen, M. K. Saleem, and C. Basdogan. 2017. Roughness perception of virtual textures displayed by electrovibration ontouch screens. In Proceedings of the IEEE World Haptics Conference. 263–268.
[59] E. Vezzoli, W. B. Messaoud, M. Amberg, F. Giraud, B. Lemaire-Semail, and M. Bueno. 2015. Physical and perceptual independence ofultrasonic vibration and electrovibration for friction modulation. IEEE Trans. Hapt. 8, 2 (2015), 235–239.
[60] A. I. Weber, H. P. Saal, J. D. Lieber, J. Cheng, L. R. Manfredi, J. F. Dammann, and S. J. Bensmaia. 2013. Spatial and temporal codes mediate
the tactile perception of natural textures. Proc. Natl. Acad. Sci. U.S.A. 110, 42 (2013), 17107–17112.
[61] M. Wiertlewski, J. Lozada, and V. Hayward. 2011. The spatial spectrum of tangential skin displacement can encode tactual texture. IEEE
Trans. Robot. 27, 3 (2011), 461–472.

ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.

18:18 • K. Otake et al.
[62] D. Wijekoon, M. Cecchinato, E. Hoggan, and J. Linjama. 2012. Electrostatic modulated friction as tactile feedback: Intensity perception.
In Haptics: Perception, Devices, Mobility, and Communication. 613–624.
[63] T. Yamauchi, S. Okamoto, M. Konyo, Y. Hidaka, T. Maeno, and S. Tadokoro. 2010. Real-time remote transmission of multiple tactile
properties through master-slave robot system. In Proceedings of the IEEE International Conference on Robotics and Automation. 1753–
1760.
[64] T. Yoshioka, S. J. Bensmaïa, J. C. Craig, and S. S. Hsiao. 2007. Texture perception through direct and indirect touch: An analysis ofperceptual space for tactile textures in two modes of exploration. Somatosens. Motor Res. 24, 1-2 (2007), 53–70.
[65] T. Yoshioka and J. Zhou. 2009. Factors involved in tactile texture perception through probes. Adv. Robot. 23, 6 (2009), 747–766.

Received 19 May 2021; revised 29 April 2022; accepted 19 May 2022

ACM Transactions on Applied Perception, Vol. 19, No. 4, Article 18. Publication date: November 2022.