Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

Audio-Tactile Perception of Roughness
Madeline Fery1, Corentin Bernard1,2, Etienne Thoret1,3,4, Richard Kronland-Martinet1,Sølvi Ystad 1
Aix Marseille Univ, CNRS, PRISM, Marseille, France2
Aix Marseille Univ, CNRS, ISM, Marseille, France
3
Institute of Language Communication and the Brain (ILCB), Marseille, France4
Aix Marseille Univ, LIS, CNRS, Marseille, France
1

etienne.thoret@univ-amu.fr; bernard@prism.cnrs.fr

Abstract. Auditory roughness is a perceptual attribute at the basis ofphenomena such as consonance and dissonance in music. The psychophysicalcorrelates of this attribute are often studied by combining two monochromatic
tones slightly separated in frequency, leading to more or less rapid beatings.
Interestingly, roughness is not limited to the auditory modality and it is possible
to evoke the same kind of sensation through the tactile modality by using avibrotactile actuator. Whether or not audio and tactile modalities share the same
perceptual roughness properties is still an open question that may reveal
common sensory processes between the two modalities. Here we investigate
this question in 2 pairwise comparison experiments unveiling roughness curves
in audio and tactile modalities. The results reveal similar roughness curves inboth modalities, which suggests a common way of processing and perceiving
beatings.
Keywords: Audio-tactile, Roughness, Beatings, Critical bands

1Introduction

Auditory roughness probes the fundamental ability of audition to disentangle
harmonic stimuli, a fundamental skill to perceive speech (Arnal et al., 2015), andmusic (Helmholtz, 1885; Plomb & Levelt, 1965) properties. This phenomenon can bedescribed as the perception of very fast fluctuations in sounds. To understand how ourears deal with complex mixtures of harmonics (Vassilaki, 2001), a historical body ofworks has used basic stimuli by combining monochromatic tones. It is now wellknown that for stimuli composed of two monochromatic tones, the sensation ofroughness is driven by the space between the frequencies of the components. Theroughness first increases when the frequency ratio between components increases andreaches a maximum before it decreases with respect to the increasing frequency ratio.
Figure 1 presents a typical auditory roughness curve for a sum of two monochromatic
tones 𝑠𝑠(𝑡𝑡) = 𝑠𝑠𝑖𝑖𝑛𝑛(2π𝑓𝑓1𝑡𝑡) + 𝑠𝑠𝑖𝑖𝑛𝑛(2π𝑓𝑓2𝑡𝑡). When the frequency ratio 𝛼𝛼=f2/f1 is small,the combination of tones tends to be perceived as one tone slowly modulated by theother one. When the frequency ratio increases, a sensation of roughness appears. As 𝛼𝛼245

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

becomes even larger, the perceived roughness falls and two the two tones areperceived separately. This theoretical roughness curve is defined by−𝑏𝑏1𝑠𝑠(𝑓𝑓2−𝑓𝑓1)

𝑟𝑟(𝑓𝑓1, 𝑓𝑓2) = 𝑒𝑒−𝑏𝑏2𝑠𝑠(𝑓𝑓2−𝑓𝑓1)

− 𝑒𝑒

0.24

with b1 = 3.5, b2 = 5.75, 𝑠𝑠 = 𝑠𝑠 𝑓𝑓 +𝑠𝑠 ,1 1

2

s1=0.0207 and s2 = 18.96 (Vassilakis, 2001).
Such a phenomenon reveals the existence of auditory critical bands, a fundamental
characteristic of auditory filters (Terhardt, 1974). While the auditory perception ofsuch phenomena has been largely studied, it is not known whether other modalities
such as touch, elicit similar behaviors (Makous et al., 1995). Interestingly, it ispossible to produce similar stimuli as in auditory experiments with vibrotactile
actuators. It would therefore be interesting to check whether roughness perception isshared between auditory and tactile modalities. This would strongly suggest that theymight also share mechanistic properties during the processing of vibrations. In a larger
multisensory perspective, we may wonder how this information is processed and inparticular how the information is shared between the auditory and tactile inputs. Is it
possible to influence auditory roughness with tactile feedback? And conversely, mighta smooth surface be perceived as rough when touched in presence of a rough sound?

Fig. 1. Typical auditory roughness curve of a sum of two monochromatic tones anddescription of the three kinds of sensations provoked by pairs of pure tones for
f1=200 Hz (Vassilakis, 2001).
In this paper, we investigated the perception of roughness through an experiment thatwas divided in two parts, one with audio stimuli and the other one with tactile stimuli.
The results are presented as audio and tactile roughness curves obtained from pairwise
comparisons.

246

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

2Method

2.1

Participants

18 subjects, 8 women and 10 men (Mean=30 years old, between 21 and 57 years old),
15 right-handed and 3 left-handed, voluntarily took part in the experiment. None ofthem reported having any auditory problems or skin concerns. The participants gave
their informed consent before the experiment. The experiment lasted about 1 hour.
2.2

Audio stimuli

Audio stimuli were pairs of sounds, each composed of the sum of two monochromatic
tones of frequencies f1 and f2=𝛼𝛼f1, of duration 1 second, and separated by 800ms. 𝛼𝛼 is acoefficient between 1 and 2 that determines the frequency ratio between the twofrequencies. When 𝛼𝛼=1, the frequencies are the same ( f1=f2), which corresponds tounison, and when 𝛼𝛼=2 the tones are separated by an octave f2=2f1:
𝑠𝑠(𝑡𝑡) = 𝑠𝑠𝑖𝑖𝑛𝑛(2π𝑓𝑓1𝑡𝑡) + 𝑠𝑠𝑖𝑖𝑛𝑛(2π𝑓𝑓2𝑡𝑡).

Twelve values of 𝛼𝛼 were chosen (1, 1.01, 1.02, 1.03, 1.05, 1.10, 1.15, 1.20, 1.25, 1.35,1.50, 2.00) leading to 66 comparison pairs for one block. Audio stimuli werecompared for 6 frequency conditions (f1 = 50, 100, 200, 300, 600, 1200 Hz) leading to
6 blocks of 66 pairs (=396 pairs). Sounds were presented through Sennheiser HD-650
headphones at a sampling rate of 44100 Hz powered by a Pioneer A-209R audio
amplifier.
2.3

Tactile stimuli

Tactile stimuli were generated with the same procedure as the audio stimuli for 4
frequency conditions only (f1 = 50, 100, 200, 300 Hz). Frequencies above 800 Hz are
indeed not perceptible by the human tactile sensory system (Verrillo, 1969). Hence, 4
blocks of 66 pairs (=264 pairs) were presented through an Actronika HapCoil-One
vibrotactile actuator (dimensions: 11.5 × 12 × 37.7 mm3, acceleration: 8 g-pp,
frequency bandwidth: 10 to 1000 Hz, resonant frequency: 65 Hz). This kind ofactuator has already been used in the literature to render the sensation of textures withvibrations (Rocchesso et al., 2016). The actuator was powered by a Pioneer A-209R
audio amplifier. The subjects were asked to grab the vibrotactile actuator between thethumb and the index of their right hand. During the tactile experiment, participants
wore noise canceling headphones to prevent them from using auditory cues.
2.4

Tasks and procedure

In each experiment, participants were seated in front of a computer screen in a quiet
room. For each subject, the pairs of stimuli were presented in randomized order. For
each pair, the presentation order was also randomized. In each experiment and for247

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

each pair of sounds, subjects were asked to judge which tone combination (or tactilestimulation) was the most “granular” (granuleux in French). As several participants
had a musical background, we avoided the terms rough and pleasant that might also
have been used. Answers were collected with a keyboard and the interface wasdesigned with Max/MSP software to display either audio or tactile stimuli. The
volume and the intensity of audio and tactile stimuli were set constant during thewhole experiment.
2.5

Data analysis

In each experiment, the data were analyzed with the Bradley-Terry model (Hunter,2004). This probabilistic model allows us to predict the outcome of a pairwise
comparison from a win matrix. Practically, for each subject, a win matrix was
obtained from the 66 pairwise comparisons which were sorted as follows: the cell (i,j)
corresponds to the number of times the sound i has been judged rougher than thesound j. The win matrices were then aggregated between subjects and an iterativealgorithm was used to fit the Bradley-Terry probabilistic model. Hence, for eachfrequency ratio, we obtained the probability that the corresponding combination tone
was judged as rough compared to another combination. In the end, for the sake ofcomparison between experiments and with the literature, this probability wasnormalized into a perceived roughness score.

3Results

The results, presented in Figure 2, exhibit that auditory and tactile roughness curves
are very close in the 4 frequency conditions tested. Interestingly, the roughness curves
obtained are also coherent with the theoretical roughness curve proposed byVassilakis (2001). These results might suggest that auditory and tactile modalities
share common principles in the perception of roughness and beatings. It would be ofgreat relevance as it may for the first time lead to a common way between the twomodalities of modeling roughness within the critical band framework. Secondly, it
might further shed light on more fine similarities in the temporal processing ofvibrations through these two modalities. Recent evidences have for instance shownthat rhythm perception is shared between audio and haptics (Bernard et al., 2021). Ourcurrent findings suggest that these results could be extended to the perception of
beating and roughness.

248

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

Fig. 2. Audio (blue line) and tactile (yellow line) roughness curves obtained from theexperiment. The two modalities elicit similar roughness maxima and are coherent
with the theoretical auditory roughness model proposed by Vassilakis (2001) (dashedline). The stimuli with f1 = 600 and 1200 Hz were presented only for the audio
condition, because they are beyond the frequency bandwidth of tactile perception.

4Discussion

As it has already been observed in audition (Vassilakis, 2001), the position ofmaximal roughness perception changes according to the lower frequency f1 also in thetactile modality. It is noteworthy that, in audition, the position of the maximum varies
much more at lower frequencies, when the frequencies are inside the tactile
perception range. In addition to these unimodal studies, it would be interesting to run
a larger study in which subjects are asked to judge the roughness of audio-tactile
stimuli. The ultimate goal is indeed to decipher and to model the way our perceptual
systems combine audio and tactile senses into a coherent percept. We in particularhope to observe interactions between the two modalities and to observe how one
modality may enhance the perception of roughness in the other. This has already been
observed in several multisensory situations (Jousmäki & Hari, 1998, Guest et al.,249

Proc. of the 15th International Symposium on CMMR, Online, Nov. 15-19, 2021

2002), and such cases are of great relevance to understand the fine mechanistic basesof human perceptual systems.

5Acknowledgments

This work was supported by an ILCB/BLRI grant ANR-16-CONV-0002 (ILCB),
ANR-11-LABX-0036 (BLRI), the Excellence Initiative of Aix-Marseille University(A*MIDEX) and the Sound and Music from Interdisciplinary and Intersectorial
Perspectives (SAMI - A*MIDEX) project.

References
Arnal, L. H., Flinker, A., Kleinschmidt, A., Giraud, A. L., & Poeppel, D. (2015).
Human screams occupy a privileged niche in the communication soundscape.
Current Biology, 25(15), 2051-2056.
Bernard, C., Monnoyer, J., Wiertlewski, M. & Ystad, S. (2021) Perception of rhythm
is shared between audio and haptics. In preparation.
Guest, S., Catmur, C., Lloyd, D., & Spence, C. (2002). Audiotactile interactions inroughness perception. Experimental Brain Research, 146(2), 161-171.
Helmholtz, H. L. F. (1885). On the Sensations of Tone as a Physiological Basis for theTheory of Music (2nd edition.) Trans. A. J. Ellis. New York: DoverPublications, Inc. (1954.)
Hunter, D. R. (2004). MM algorithms for generalized Bradley-Terry models. The
annals of statistics, 32(1), 384-406.
Jousmäki, V., & Hari, R. (1998). Parchment-skin illusion: sound-biased touch.
Current biology, 8(6), R190-R191.
Makous, J. C., Friedman, R. M., & Vierck, C. J. (1995). A critical band filter in touch.
Journal of Neuroscience, 15(4), 2808-2818.
Plomp, R. and Levelt, W. J. M. (1965). Tonal consonance and critical bandwidth. The
Journal of the Acoustical Society of America, Vol. 38: 548-560.
Rocchesso, D., Delle Monache, S., & Papetti, S. (2016). Multisensory texture
exploration at the tip of the pen. International Journal of Human-Computer
Studies, 85, 47-56.
Terhardt, E. (1974). On the perception of periodic sound fluctuations (roughness).
Acta Acustica united with Acustica, 30(4), 201-213.
Vassilakis, P. (2001). Auditory roughness estimation of complex spectra—Roughness
degrees and dissonance ratings of harmonic intervals revisited. The Journal of
the Acoustical Society of America, 110(5), 2755-2755.
Verrillo, R. T., Fraioli, A. J., & Smith, R. L. (1969). Sensation magnitude ofvibrotactile stimuli. Perception & Psychophysics, 6(6), 366-372.
250