* Simard, J., Ammi, M., & Auvray, M. (2010). Study of synchronous and colocated collaboration for search tasks. *Joint Virtual Reality Conference (JVRC)*, 27-29 September, Stuttgart, Germany.

* Auvray, M., Gallace, A., & Spence, C. (2010). Cécité aux changements et multimodalité. *Cognition, émotion et société, 52ème congrès de la Société Française de Psychologie*, 7-9 September, Lille, France.

* Auvray, M., Gallace, A., & Spence, C. (2010). Which frame of reference is adopted when interpreting tactile letters presented on the body surface? *12th European Workshop on Imagery and Cognition (EWIC)*, 16-19 June, Helsinki, Finland.

* Gallay, M., Denis, M., Parseihian, G., & Auvray, M. (2010). Egocentric and allocentric reference frames in a virtual auditory environment: Differences in navigation skills between blind and sighted individuals. *12th European Workshop on Imagery and Cognition (EWIC)*, 16-19 June, Helsinki, Finland.

Auvray, M. (2010). Perceptual interactions in a minimalist tactile environment. *Seminar of the Department of Psychiatry and Psychotherapy*, University of Cologne, 15 April, Cologne, Germany.

Deroy, O., & Auvray, M. (2010). A new look on sensory extensions. Workshop on Sensory Substitution, Synaesthesia, Sensation, and Perception, Institute of Philosophy, 31 March, London, UK.

Auvray, M. (2010). Cross-modal change blindness. *Seminar of the Sensory Research Forum*, Institute of Philosophy, 30 March, London, UK.

Auvray, M. (2010). Percevoir avec des dispositifs de compensation: de la substitution sensorielle à l'extension sensorimotrice. *Phiteco*, 18-24 January, Compiègne, France.

* Auvray, M., Gallace, A., & Spence, C. (2009). What perspective do people take when interpreting tactile letters presented on their bodies? *10th International Multisensory Research Forum (IMRF)*, 29 July - 2 August, New-York, USA (poster).

* Auvray, M., Gallace, A., & Spence, C. (2009). Short term memory for tactile stimuli presented on the fingertips and across the body surface. *10th International Multisensory Research Forum (IMRF)*, 29 July - 2 August, New-York, USA (poster).

* Auvray, M., Hoellinger, T., Roby-Brami, A., & Hanneton, S. (2009). Perceptual weight judgments when viewing own and other's movements under minimalist conditions of visual presentation. *10th International Multisensory Research Forum (IMRF)*, 29 July - 2 August, New-York, USA (poster).

Auvray, M., & Myin, E. (2009). Perception with compensatory devices: From sensory substitution to sensorimotor extension. *The Senses Research Workshop*, 19-21 June, Paris, France.

Auvray, M. (2009). NAVIG: Navigation Assisted by artificial Vision and GNSS. International Workshop on Mental Imagery, Spatial Cognition, and Language (MICL 2009), in honour of Michel Denis, 12 June, Padova, Italy.

Auvray, M. (2009). Perceptual interactions in a minimalist environment. *The Third Cargese Consciousness Summer School, Consciousness of the Self - Consciousness of the Others*, 25 May - 4 June, Cargese, France.

Auvray, M. (2009). Replacing one sense by another: Sensory substitution and the classification of our sensory modalities. *Key Issues in Sensory Augmentation Workshop*, 26-27 March, Brighton, UK.
* Auvray, M., Hoellinger, T., & Hanneton, S. (2008). Sensory substitution and the taxonomy of our sensory modalities. *9th Annual Meeting of International Multisensory Research Forum (IMRF)*, 16-19 July, Hamburg, Germany.

* Hoellinger, T., Auvray, M., Roby-Brami, A. & Hanneton, S. (2008). Localisation tasks with a three-dimensional audio-motor coupling based on an electromagnetic motion capture device. *9th Annual Meeting of International Multisensory Research Forum (IMRF)*, 16-19 July, Hamburg, Germany (poster).

Auvray, M. (2008). Remplacer un sens par un autre: les dispositifs de substitution sensorielle. *9ème Journée de la Vision, SFOP*, 13 April, Paris, France.

Auvray, M. (2008). The multisensory nature of spatial attention: Crossmodal change blindness between vision, touch, and audition. *Séminaire du Laboratoire de Neurophysique et Physiologie*, 28 January, Paris, France.

Auvray, M. (2007). Perception spatiale et multisensorialité: Espaces réels et espaces virtuels. *Séminaire du LIMSI: CHM et VENISE*, 04 December, Orsay, France.

Gallace, A., Auvray, M., Hartcher-O’Brien, J., Tan, H. Z., & Spence, C. (2007). The effect of visual, auditory, and tactile distractors on people’s awareness of tactile change. *Body Representation Workshop*, 8-10 October, Rovereto, Italy.

* Auvray, M., Myin, E., & Spence, C. (2007). Crossmodal change blindness and the multisensory nature of spatial attention. *International Conference on Perception and Action (ICPA 14)*, 1-6 July, Yokohama, Japan.

* Lenay, C., Stewart, J., Ali Ammar, A., & Auvray, M. (2007). Direct perception of an intentional subject: A minimalist approach. *International Conference on Perception and Action (ICPA 14)*, 1-6 July, Yokohama, Japan (poster).

* Auvray, M. (2007). Enaction and sensory substitution. *Philosophy of Psychology, Neuroscience, and Psychology*, 25 March, Bristol, UK.

Auvray, M. (2007). Cognition spatiale et appropriation d'outils. *Séminaire du LIMSI: CHM et VENISE*, 06 March, Orsay, France.

Lenay, C., Auvray, M., Sebbah, F.-D., & Stewart, J. (2006). Perception of an intentional subject: An enactive approach, in *Proceedings of the Third International Conference on Enactive Interface*, 20-21 November, Montpellier, France, pp. 37-38.

Auvray, M. (2006). *Perception, interaction, and mutual recognition in sensory substitution*. *Action in Mind, The 6th Annual Conference on the Philosophy of Consciousness of the University of Antwerp*, 21 October, Antwerp, Belgium.

* Lenay, C., Auvray, M., Thouvenin, I., & Sebbah, F. D. (2006). Enactive approach to the recognition of an intentional subject. *Situated Cognition: Perspectives From Phenomenology and Science*, 18-20 August, Durham, UK.

* Auvray, M., Lenay, C., & Stewart, J. (2006). The recognition of others in a simulated environment, the case of minimalist devices. *Association for the Scientific Study of Consciousness 10*, 23-26 June, Oxford, UK (poster).

* Lenay, C., Auvray, M., & Sebbah, F. D. (2006). Seeing oneself: Experimental studies of perceptive interactions. *Association for the Scientific Study of Consciousness 10*, 23-26 June, Oxford, UK (poster).
* Auvray, M., Gallace, A., & Spence, C. (2006). Crossmodal change blindness between vision and touch. *7th Annual Meeting of International Multisensory Research Forum (IMRF)*, 18-21 June, Dublin, Ireland (poster).

* Auvray, M., Gallace, A., & Spence, C. (2006). Crossmodal change blindness. *EPS Birmingham Meeting*, 10-12 April, Birmingham, UK.

Auvray, M. (2005). A commentary of Zenon Pylyshyn: “Spatial representation in the mind/brain: Do we need a global topographical map?” *Frames of Reference in Spatial Cognition: Diversity, Flexibility and Adaptability*, 17-19 November, Paris, France.

* Lenay, C., Auvray, M., Thouvenin, I., & Sebbah, F. D. (2005). Perception of avatars in a space of tactile interaction. *2nd Intuition International Workshop “VR/VE and Industry: Challenges and Opportunities”* (ISBN 960-254-658-1), 24-25 November, Paris, France.

Auvray, M. (2005). Spatial perception and sensory substitution. *Body Representation Workshop*, 30 September, Oxford, UK (Poster).

* Auvray, M., Lenay, C., Hanneton, S., Gapenne, O., & Stewart, J. (2005). Spatial perception and sensory substitution. *International Conference on Perception and Action*, 5-10 July, Monterey, USA.

* Gapenne, O., Rovira, K., Lenay, C., Stewart, J., & Auvray, M. (2005). Is form perception necessarily tied to specific sensory feedback? *International Conference on Perception and Action*, 5-10 July, Monterey, USA.

Auvray, M. (2005). Perception spatiale et substitution sensorielle. *Journées GDR-STIC Santé, Espace et sa Perception: Application à la Conception des Aides Technologiques aux Déplacements de Déficients Visuels*, 10 March, Paris, France.

Auvray, M. (2005). Appropriation d'une prothèse perceptive et constitution de nouveaux espaces de perception et d'action. *Journée Atelier Rescif, Indétermination du Champ Visuel, Anticipations Catégorielles et Thématisation*, 1 February, Paris, France.

* Auvray, M., Hanneton, S., Lenay, C., & O'Regan, J. K. (2004). Exteriorisation in sensory substitution. *First Joint Conference of SPP & ESPP*, 3-6 July, Barcelona, Spain.

* Auvray, M., Hanneton, S., O'Regan, J. K., & Lenay, C. (2004). Distal attribution in sensory substitution. *Association for the Scientific Study of Consciousness 8*, 25-28 June, Antwerp, Belgium.

Auvray, M., Lenay, C., O'Regan, J. K., & Hanneton, S. (2004). Action et Immersion. *Séminaire Savoir ce que l'on Fait*, 27-31 January, Compiègne, France.

* Auvray, M., Hanneton, S., & O'Regan, J. K. (2003). Localisation, form recognition and emergence of sensation in sensory substitution. *ESCOP*, 17-20 September, Granada, Spain.

* Auvray, M., Hanneton, S., & O'Regan, J. K. (2003). Localisation and form recognition in sensory substitution. *ECVP*, 1-5 September, Paris, France. *Perception*, **32**, supplement, 94 (poster).

Auvray, M., O'Regan, J. K., & Hanneton, S. (2003). Repenser la distinction entre nos modalités sensorielles via un dispositif de suppléance perceptive visuo-auditif. *Colloque des Jeunes Chercheurs en Sciences Cognitives*, 18-20 June, Paris, France.

Khatchatourov, A., & Auvray, M. (2003). L'outil modifie-t'il la perception ou la rend-il possible ? *Colloque des Jeunes Chercheurs en Sciences Cognitives*, 18-20 June, Paris, France.
Auvray, M., O'Regan, J. K., & Hanneton, S. (2003). Apprendre à utiliser un dispositif de substitution sensorielle: les différentes étapes. *Séminaire Espaces d'Action, Espaces de Perception*, 21-29 January, Compiègne, France.

Auvray, M., Hanneton, S., & O'Regan, J. K. (2002). Stages in the emergence of sensation in sensory substitution. *Third Workshop on Multimodal Interaction in Perception*, 28-29 October, Paris, France.

Auvray, M., Philipona, D., & O'Regan, J. K. (2001). Tentative d'induire la notion d'espace dans un environnement virtuel simple. *Atelier Perception et Constitution de l'Espace, Journées du RESCIF*, 27-28 October, Paris, France.

Auvray, M., Philipona, D., & O'Regan, J. K. (2001). Notion of space and form recognition in a simulated environment. *Atelier Genèse de la Perception et de la Notion d'Espace chez le Robot et l'Homme*, 18-19 October, Paris, France.

O'Regan, J. K., Clark, J. J., Nadal, J-P., Auvray, M., Bompas, A., & Philipona, D. (2001). Implications of a “sensorimotor” theory of visual phenomenality. *International Workshop: Pathways to Abstraction: From Experience to Concept*, 10-13 September, Gif-sur-Yvette, France.

* Auvray, M., & O'Regan, J. K. (2001). Influence of semantic factors on blindness to progressive changes in visual scenes. *ECVP*, 26-30 August, Kusadasi, Turkey. *Perception*, 30, supplement, 44 (poster).
# Annex

## Selection of articles

1. Pesnot-Lerousseau, J., Arnold, G., & Auvray, M. (submitted). Visualizing sounds: training-induced plasticity with a visual-to-auditory conversion device.

*As this article is in the submitted stage, it is not included for now in the on-line version of the HDR*

2. Arnold, G., & Auvray, M. (2014). Perceptual learning: Tactile letter recognition transfers across body surfaces. *Multisensory Research, 27*, 71-90.

3. Arnold, G., & Auvray, M. (2017). Tactile recognition of visual stimuli: Specificity versus generalization of perceptual learning. *Vision Research*.

4. Deroy, O., Fasiello, I., Hayward, V., & Auvray, M. (2016). Differentiated audio-tactile correspondences in sighted and blind individuals. *Journal of Experimental Psychology: Human Perception and Performance, 42*.

5. Arnold, G., Spence, C., & Auvray, M. (2016). Taking someone else's spatial perspective: Natural stance or effortful decentring? *Cognition, 148*, 27-33.

6. O'Brien, J., & Auvray, M. (2016). Cognition overrides orientation dependence in tactile viewpoint selection. *Experimental Brain Research, 234*, 1885-1892.

7. O'Brien, J., & Auvray, M. (2014). The process of distal attribution illuminated through studies of sensory substitution. *Multisensory Research, 27*, 421-441.
# Perceptual Learning: Tactile Letter Recognition Transfers Across Body Surfaces

Gabriel Arnold<sup>1,2,*</sup> and Malika Auvray<sup>2</sup>

Received 27 September 2013; accepted 15 February 2014

## Abstract

Visual-to-tactile sensory substitution devices are designed to assist visually impaired people by converting visual stimuli into tactile stimuli. The important claim has been made that, after training with these devices, the tactile stimuli can be moved from one body surface to another without any decrease in performance. This claim, although recurrent, has never been empirically investigated. Moreover, studies in the field of tactile perceptual learning suggest that performance improvement transfers only to body surfaces that are closely represented in the somatosensory cortex, i.e. adjacent or homologous contralateral body surfaces. These studies have however mainly used discrimination tasks of stimuli varying along only one feature (e.g., orientation of gratings) whereas, in sensory substitution, tactile information consists of more complex stimuli. The present study investigated the extent to which there is a transfer of tactile letter learning. Participants first underwent a baseline session in which the letters were presented on their belly, thigh, and shin. They were subsequently trained on only one of these body surfaces, and then re-tested on all of them, as a post-training session. The results revealed that performance improvement was the same for both the trained and the untrained surfaces. Moreover, this transfer of perceptual learning was equivalent for adjacent and non-adjacent body surfaces, suggesting that tactile learning transfer occurs independently of the distance on the body. A control study consisting of the same baseline and post-training sessions, without training in between, revealed weaker improvement between the two sessions. The obtained results support the claim that training with sensory substitution devices results in a relative independence from the stimulated body surface.

## Keywords

Perceptual learning, touch, sensory substitution, generalization, tactile letter recognition

This article is part of the 14th IMRF collection, guest edited by A. Amedi, D. I. Shore and S. Levy-Tzedek.

<footnote n="1">¹ LIMSI-CNRS, UPR 3251, rue John von Neumann, 91400 Orsay, France</footnote>

<footnote n="2">² Institut Jean Nicod, UMR 8129, Ecole Normale Supérieure, Paris, France</footnote>

<footnote>* To whom correspondence should be addressed. E-mail: gabriel.arnold@limsi.fr</footnote>
# 1. Introduction

Sensory substitution devices convert stimuli that are normally accessed through one sensory modality (e.g., vision) into stimuli accessible through another sensory modality (e.g., touch or audition). Among them, visual-to-tactile and visual-to-auditory conversion systems were designed to assist blind and visually impaired people by converting visual stimuli into tactile or auditory stimuli.

The first sensory substitution device was the tactile-vision sensory substitution (TVSS) developed by Bach-y-Rita and colleagues (Bach-y-Rita et al., 1969; see also Bach-y-Rita and Kercel, 2003, for a review). The TVSS comprises a camera, a translation system, and a matrix of tactile stimulators (that provides electrical or vibratory stimulation) placed on a body surface such as the back or the tongue (see Bach-y-Rita and Kercel, 2003, for a description). The principles of sensory substitution have been extended to the conversion of visual images into sounds. This led to the design of visual-to-auditory devices such as the vOICe (Meijer, 1992), the prosthesis for substitution of vision by audition (PVSA, see Capelle et al., 1998), and the Vibe (Hanneton et al., 2010).

Since their invention, a variety of devices have been designed and tested across a variety of tasks (see Auvray and Myin, 2009; Bubic et al., 2010; Deroy and Auvray, 2012, for reviews). In particular, studies have revealed that these devices allow their users to perform localization tasks (Jansson, 1983; Levy-Tzedek et al., 2012; Renier et al., 2005) and simple as well as complex shape recognition (Arno et al., 2001; Auvray et al., 2007; Pollok et al., 2005; Sampaio et al., 2001). A defining feature of perception with sensory substitution devices is that extensive training is required. Most users are able, in one or two hours, to explore their environment and to approximate objects' position and shape. However, the training required to obtain more precise object localization and recognition behaviour is estimated at around eight hours with visual-to-tactile devices (Kaczmarek and Haase, 2003) and 10 to 15 with visual-to-auditory devices (Auvray et al., 2007). Perceptual acuity (measured with the Snellen tumbling E test) has also been reported to increase with training, reaching for instance 20/430 after nine hours of training with the Tongue Display Unit (Sampaio et al., 2001; see also Haigh et al., 2013, for a similar study with the vOICe).

One important claim of Bach-y-Rita is that, once trained, TVSS-users no longer feel the stimulation on their skin, where it occurs, but they directly attribute the stimulation as resulting from an external object, i.e. located at a distance (Bach-y-Rita, 1995, p. 181; Bach-y-Rita and Kercel, 2003, p. 543). Furthermore, Bach-y-Rita asserted that, consequent to this externalization process, the tactile stimulator array can be moved from one body surface to
another (e.g., from the back to the abdomen or to the forehead), without loss in
spatial localization abilities or other perceptual capacities (see Note 1) (such
as depth estimates). However, this claim was based solely on users’ verbal
reports and it has never been experimentally investigated.

One way to investigate if perceptual abilities in trained users of visual-to-
tactile sensory substitution devices depend on the stimulated surface or not,
is to study the extent to which there is a transfer of tactile learning across
body surfaces. If indeed what is learned on a given body surface can be trans-
ferred to another one without loss in performance, this would provide reasons
to believe that Bach-y-Rita’s claim of independence from the stimulated body
surface is correct. Perceptual learning is defined as performance improvement
in perceptual tasks resulting from training (Fahle and Poggio, 2002; Gibson,
1969). In tactile perceptual learning, the question of the specificity (i.e., learn-
ing is body-surface dependent) versus generalization (i.e., there is a transfer
of performance improvement from a trained to an untrained body surface) is
still debated. Some studies reported a transfer to untrained body surfaces (Har-
rar et al., 2014; Harris et al., 2001; Kaas et al., 2013; Nagarajan et al., 1998;
Sathian and Zangaladze, 1997; Spengler et al., 1997) whereas other studies did
not find evidence of such transfer (Dinse et al., 2006; Godde et al., 2000). In
addition, when transfer has been reported, it was restricted to adjacent and ho-
mologous contralateral body surfaces only (e.g., adjacent fingers of the same
hand and same fingers of the untrained hand — Harrar et al., 2014; Harris et
al., 2001). In other words, transfer to non-adjacent and non-homologous body
surfaces has never been reported. The extent to which there is a transfer of
tactile learning therefore appears to be related to the topographic organization
of the somatosensory cortex.

There is thus an apparent contradiction between this reported limit to the
transfer of tactile learning (i.e., restricted to adjacent and homologous body
surfaces) and the post-training independence from the stimulated body sur-
face (i.e., occurring even in non-adjacent surfaces such as the back and the
abdomen), as claimed by Bach-y-Rita. One hypothesis to account for such a
discrepancy is that the discrimination tasks used in tactile learning studies in-
volved low-level stimuli that mainly vary along only one feature (e.g., gratings
with different orientations), whereas reports in the field of sensory substitution
were based on more complex objects consisting of combinations of different
features. Discrimination of low-level stimuli may involve somatosensory areas
with a topographic representation of body surfaces. In these areas, receptive
fields are narrow and overlap only with receptive fields of adjacent or con-
tralateral homologous body surfaces (for instance, see Iwamura et al., 1994,
for the topographic organization of the hands). On the other hand, sensory sub-
stitution devices are mainly used to convey tactile information resulting from
higher-level stimuli such as objects, faces, and scenes. These high-level stim-
uli involve less topographically organized areas in which receptive fields are
large and not restricted to a specific body surface. Moreover, according to the
reverse hierarchy theory (Ahissar and Hochstein, 2004), perceptual learning
begins at higher cortical levels and continues at lower levels of processing de-
pendent on the feature learnt. As a result, tactile learning of high-level stimuli
should transfer more easily to different body surfaces than low-level stimulus
features.

The study reported here aims at investigating the extent to which there is a
transfer of tactile learning in the recognition of high-level symbols (i.e., made
of the combinations of several features rather than consisting of single fea-
tures). To do so, a tactile letter recognition task was used. Letters were drawn
on the participants’ body by means of sequential vibrotactile stimulations (see
Yanagida et al., 2004, for a similar method). In order to evaluate learning
effects rather than mere repeated exposure effects, we measured recognition
performance before and after training, as in Harrar et al.’s (2014) study. The
participants first completed a baseline session in which they had to recognize
tactile letters drawn on three different body surfaces: the belly, the front of
the right thigh, and the right shin (see Fig. 1 below). They then underwent
a training session in which the letters had to be recognized on only one of
the three above-mentioned body surfaces. Finally, the participants performed a
post-training session on all three body surfaces. The amount of tactile learning
was evaluated by computing performance improvement (both accuracy and re-
sponse times) between the baseline and the post-training sessions. If there is
a transfer of tactile learning from a trained to an untrained body surface, then
the amount of performance improvement should be similar for trained and un-
trained surfaces. On the other hand, if tactile learning is specific to a given
body surface, then performance improvement should be greater for trained
than for untrained body surfaces. In addition, a control group of participants
performed the baseline and post-training sessions, but without the training ses-
sion in between. For these participants, performance is expected to be either
similar between the two sessions, or to improve in the second one but less than
for the participants in the trained group.

It should be mentioned that the belly, the thigh and the shin are represented
in adjacent areas of the somatosensory cortex (Merzenich et al., 1978; Naka-
mura et al., 1998). Thus, performance improvement, computed as a function
of the trained body surface, allows behaviourally evaluating the topographic
organization of areas in which tactile learning of high-level symbols occurs. If
tactile learning occurs in non-topographic areas, the amount of improvement
should not depend on the trained body surface. On the other hand, if tactile
learning occurs in topographic areas, the amount of improvement should be
greater for two adjacent body surfaces (e.g., the belly and the thigh) than for
two non-adjacent ones (e.g., the belly and the shin).
<figure><img src="image_29.png" /><figcaption>**Figure 1.** Example of the letter F drawn on the three body surfaces (belly, thigh, and shin). The letters' left-right and top-bottom axes were always congruent to the participants' left-right and top-bottom axes.</figcaption></figure>

## 2. Methods

### 2.1. Participants

Forty-five participants completed the experiment (26 females and 19 males; mean age = 25.7 years, range = 18–47 years). Thirty participants were included in the trained group and 15 in the control group. All the participants were naïve to the purpose of the experiment. They provided their informed consent and received payment for their participation. The experiment took approximately one hour and a half to complete (one hour for the control group) and was performed in accordance with the ethical standards laid down in the Declaration of Helsinki (1991).

It should be mentioned that there was a selection process used to determine if participants were included in the study. Given that tactile letters drawn on the skin can be interpreted within different reference frames (see Parsons and
Shimojo, 1987; Sekiyama, 1991), each participant's preferred reference frame was identified prior to the start of the experiment. This was done by means of a task requiring the participants to recognize the tactile letters b, d, p, and q. Recognizing these ambiguous asymmetrical letters requires assigning the top-bottom and left-right axes to the letters, and the reference frame that was adopted can be determined from how the stimulus is interpreted. Only the participants that freely assigned the top-bottom and left-right axes congruently to their body's axes were selected for participation in the study. The letters were then drawn consistently to these body-congruent assignments during the rest of the experiment (see Fig. 1). The reason for this choice is that — as the aim of the study was not to investigate the influence of reference frame on tactile learning — we wanted to avoid any interference due to differences in the reference frames that are freely adopted. We consequently chose the most frequently adopted one among the four possible reference frames. This reference frame with body-congruent assignments of axes was adopted by 52% of participants across all the experiments that were conducted in our laboratory (N = 244). Note that 28% of participants reversed the top-bottom axis, while the left-right axis was consistent with the body. A further 20% of participants reversed the left-right axis, while the top-bottom axis was consistent with the body. No participants reversed both the vertical and horizontal axes.

## 2.2. Apparatus

The tactile stimuli were presented by means of nine rectangular vibrators (Haptuator Mark II, Tactile Labs, Montreal, Canada) arranged in a three-by-three array with a centre-to-centre spacing of 5 cm (see Fig. 2A). The surface area of each vibrator was 0.9 cm vertically by 3.2 cm horizontally. The vibrator array was placed on the participants' body surface, above their clothes, by means of an elastic belt. When the participants were tested on the belly, the vibrator array was placed symmetrically to their body mid-sagittal line

<figure><img src="image_30.png" /><figcaption>**Figure 2.** (A) Schematic figure illustrating the 3 × 3 array of rectangular vibrators. (B) The 8 letters used as tactile stimuli. (C) The sequence of vibrations chosen for drawing the letter P.</figcaption></figure>
(see Fig. 1). In this case, the three lower tactors were located above the waist. When they were tested on the front of the thigh and on the shin, the vibrator array was both horizontally and vertically centred. The position of the vibrator array on each body surface was exactly the same between the baseline and the post-training sessions; this was achieved by means of stickers indicating the positions of the vibrators.

A nine-channel amplifier drove each vibrator independently at a 250 Hz frequency. The intensity of each vibrator was individually selected with an adjustment method at the beginning of each session. Specifically, the participants were instructed to adjust the vibration intensity until each vibrotactile stimulus could be perceived clearly and with the same felt intensity. The activation of the vibrators was controlled through a PC running custom software written in MATLAB R2008a. Instructions and feedback were presented on a 23-inch screen with a 1920 × 1080 resolution. Participants wore noise-reducing headphones with a noise reduction rating of 30 dB, in order to mask any sounds made by the vibrators.

## 2.3. Stimuli

The eight upper-case letters B, C, F, G, K, N, P, and Q were drawn on the participants’ body surfaces (see Fig. 2B). These letters were selected on the basis of previous experiments we conducted, in which they were sufficiently well recognized (above 87% accuracy) and induced little confusion. The order in which the vibrators were successively activated was chosen to approximate the gesture of manual letter drawing (see Fig. 2C). This sequential mode of presentation was chosen over a simultaneous mode because it allows better performance in letter recognition (Yanagida *et al.*, 2004; see also Loomis, 1974). In order to choose the sequence of each letter, 14 persons, who did not participate in the main experiment, were asked to draw the letters on a paper sheet. From this we calculated the most frequent way of writing each of the letters. For instance, the letter P is the most frequently drawn by an initial vertically descending stroke followed by a loop beginning from the top of the vertical stroke. Each vibrator composing the sequence was activated for 250 ms. There was no interval between two consecutive vibrations composing one stroke of the letter (e.g., the three vibrators composing the vertical stroke of the letter P); nor between two consecutive strokes that were spatiotemporally continuous (e.g., the three strokes of the loop of the letter P). However there was a 150-ms interval between two consecutive strokes that were spatiotemporally discontinuous (e.g., the end of the vertical descending stroke and the beginning of the loop of the letter P). This interval avoided possible errors of interpretation caused by an irrelevant grouping of vibrations. Participants were instructed that these temporal intervals corresponded to a
spatial discontinuity in manual letter drawing. The mean duration of letters was 2338 ms.

## 2.4. Procedure

### 2.4.1. Baseline and Post-Training Sessions

The participants were required to stand in front of the computer screen. On each trial, one of the eight letters was drawn on the participant's body surface. The participants were instructed to report which letter was drawn by pressing the corresponding key on the computer keyboard with the index finger of their dominant hand. Stickers were used to represent the eight letters on two lines of four adjacent keys. Participants were required to respond as accurately and as rapidly as possible. Accuracy was emphasized over speed of response. Given that posture has an influence on how tactile symbols are perceived (see Parsons and Shimojo, 1987; Sekiyama, 1991), we made sure that the participant's posture was consistent across sessions. As they had to look at the screen when visual feedback was provided to them, the participants were instructed to look at the empty screen during the conditions in which there was no feedback. However, note that, in all the conditions, the participants were allowed to look at the keyboard when giving their answers. The participants were able to give their responses at any time from the onset of the first vibration and up to 3000 ms after the end of the last vibration (at which point the trial was terminated). At the end of each trial, there was an interval of 3000 ms before the beginning of the next trial. No feedback was given regarding the correctness of the participants' responses.

The baseline and the post-training sessions were composed of three blocks each: one block for each body surface. The six possible orders of these three blocks were counterbalanced across participants. However, the order remained the same for each participant during the baseline and the post-training sessions. The three body surfaces were thus each separated by the same amount of letter expositions (i.e., five blocks of trials) between the baseline and the post-training sessions. Each block was composed of 48 randomized trials corresponding to six presentations of each letter.

In the baseline session, before each of the experimental blocks, the participants were given eight practice trials, corresponding to one presentation of each letter. In this practice block, feedback was given after each response: the correct answer was visually displayed on the computer screen once participants responded.

### 2.4.2. Training Session

The participants in the control group completed the post-training session just after the baseline session. Only the participants of the trained group were trained in between the two sessions. During this training session, the participants were trained on only one of the three body surfaces. This trained surface
was always the surface that was tested in the second position during the baseline and the post-training sessions. Thus, one third of the participants from the trained group were trained on each of the possible body surfaces. The training session consisted of three consecutive blocks of 48 trials. Given that a trial-by-trial feedback has been shown to reinforce learning (Herzog and Fahle, 1998), in this session, the participants were given visual feedback on the correctness of their answers after each trial.

### 3. Results

Trials in which participants failed to make a response before the trial was terminated (less than 0.9% of trials overall) were not included in the data analyses. The accuracy of letter recognition was measured by the percentage of correct responses in each condition. The mean response times (RTs) were also computed. However, RTs corresponding to errors were excluded from the RT analysis. The first analysis presented here evaluated training effects by measuring performance improvement across sessions for the trained and control groups. The second analysis evaluated the transfer of learning by comparing training effects in trained and untrained body surfaces for the trained group only. The third one evaluated the possible topographic organization of perceptual transfer by measuring performance improvement as a function of the different body surfaces in the trained group. For all analyses, an alpha level of 0.05 was used.

In order to evaluate the effect of training on performance, ANOVAs were conducted both on accuracy and on RTs with Group (trained, control) as a between-participant factor and Session (baseline, post-training) as a within-participant factor. Because the sample sizes were unequal, type III sum of squares were used (Shaw and Mitchell-Olds, 1993). With respect to accuracy, there was a significant effect of Session ($F(1, 43) = 45.42; p < 0.001$). Accuracy was greater in the post-training (86.51%, SD = 11.72) than in the baseline (77.49%, SD = 12.67) sessions, thus showing performance improvement. Importantly, there was a significant interaction between Group and Session ($F(1, 43) = 6.35; p < 0.05$), showing that performance improvement was greater in the trained (increase of 11.02 points of percentage, SD = 8.34) than in the control group (5.02 points of percentage, SD = 5.47). As a consequence, global accuracy (i.e., across the two sessions) was also significantly greater ($F(1, 43) = 7.27; p < 0.01$) for the trained (85.07%, SD = 12.69) than for the control group (75.87%, SD = 11.35). Specific comparisons showed that the group difference was significant only for the post-training session ($F(1, 43) = 14.06; p < 0.001$); not for the baseline session ($F(1, 43) = 2.48; p = 0.12$). In other words, prior to training, accuracy in letter recognition was similar for the two groups. However, after training, recognition improved
<figure><img src="image_31.png" /><figcaption>**Figure 3.** Participants' (A) accuracy and (B) response times of the trained group for the trained and untrained body surfaces obtained during the baseline session (in white) and the post-training session (in black). Error bars represent the standard errors of the means.</figcaption></figure>

differentially as the percentage of correct responses was significantly higher for the trained group than those obtained for the control group. With respect to RTs, there was a significant effect of Session ($F(1, 43) = 102.24; p < 0.001$). RTs were shorter in the post-training (3067 ms, SD = 282) than in the baseline (3319 ms, SD = 259) sessions. RTs were also significantly shorter ($F(1, 43) = 21.62; p < 0.001$) for the trained (3088 ms, SD = 268) than for the control group (3403 ms, SD = 238). The interaction between Group and Session was not significant ($F(1, 43) = 1.88; p = 0.18$).

In order to determine if there was a transfer of tactile learning, ANOVAs were conducted with Session (baseline, post-training) and Training (trained, untrained) as within-participant factors for the trained group only (see Fig. 3A and 3B). As with the previously reported ANOVA, accuracy was significantly greater in the post-training than in the baseline sessions ($F(1, 29) = 42.70; p < 0.001$). There was no effect of Training ($F(1, 29) < 1$). More importantly, there was no significant interaction between Session and Training ($F(1, 29) < 1$); which shows that performance improved between the baseline and post-training sessions for both trained (increase of 10.13 points of percentage, SD = 12.32) and untrained (11.47 points of percentage, SD = 7.79) body surfaces. The analysis conducted on RTs yielded the same pattern of results, with a significant decrease in RTs from the baseline to the post-training sessions ($F(1, 29) = 85.10; p < 0.001$). There was a significant effect of Training ($F(1, 29) = 4.32; p < 0.05$) but no significant interaction between Session and Training ($F(1, 29) = 3.37; p = 0.08$). Note that although the interaction approached significance, the RT analysis did not show a greater training effect for trained than untrained body surfaces. The decrease in RTs was 228 ms (SD = 173) for trained and 285 ms (SD = 176) for untrained body surfaces. Thus, for both accuracy and RTs, the performance improvement
obtained with training was similar for trained and untrained body surfaces,
showing that perceptual learning transfers to untrained surfaces.

Finally, in order to evaluate the possible influence of topographic organiza-
tion on tactile perceptual learning, ANOVAs were conducted with the Trained
surface (belly, thigh, shin) as a between-participant factor, and with Session
(baseline, post-training) and Stimulated surface (belly, thigh, shin) as within-
participant factors for the trained group only (see Tables 1 and 2). As with
previous analyses, the performance improvement from the baseline to the
post-training sessions was significant for both accuracy ($F(1, 29) = 49.77$; $p < 0.001$) and RTs ($F(1, 29) = 87.26$; $p < 0.001$). There was also a sig-
nificant effect of the Stimulated surface for both accuracy ($F(2, 58) = 11.92$; $p < 0.001$) and RTs ($F(2, 58) = 5.13$; $p < 0.01$). Tukey's HSD tests showed
that performance was significantly better ($p < 0.01$, for accuracy; $p < 0.05$, for RTs) for the belly (89.72%, SD = 10.85; 3028 ms, SD = 279) than for
the thigh (83.71%, SD = 14.28; 3114 ms, SD = 302) and significantly better ($p < 0.001$, for accuracy; $p < 0.05$, for RTs) for the belly than for the shin
(81.78%, SD = 16.91; 3121 ms, SD = 310), but there was no significant dif-
ference between the thigh and the shin. More importantly, there was neither
a significant interaction between Trained surface and Session ($F(2, 27) < 1$,
for both accuracy and RTs) nor any significant interaction between Trained
surface, Stimulated surface and Session ($F(4, 54) < 1$, for both accuracy and
RTs), indicating that performance improvement did not vary with the distance
between the trained and stimulated surfaces. Performance was significantly
better ($p < 0.05$ for all comparisons) in the post-training than in the baseline
sessions for all the combinations of trained and stimulated surfaces, i.e., in-
cluding adjacent and non-adjacent body surfaces (for both accuracy and RTs).
There was no other significant effect or interaction.

The confusion matrices of the trained participants’ responses (see Table 3)
indicate that before training, the participants made the same types of confu-
sions for the three stimulated surfaces. In particular, the letters K and F were
reciprocally confused and the letter G was frequently mistaken for the letter Q.
In addition, the letter N was frequently mistaken for the letter P when it was
displayed on the shin but not when it was displayed on the belly and the thigh.
Overall, the same types of letter confusion were observed after training, with
however less errors than before training.

**4. Discussion**

The study reported here aimed at investigating the extent to which a perfor-
mance improvement in tactile letter recognition transfers across body surfaces.
Three main results emerged from this study. First, training improved both par-
ticipants’ accuracy and response latency. In addition, participants performing
<caption>**Table 1.**</caption>

Mean accuracy (in percentages) during the baseline and post-training sessions and the corresponding training effects, as a function of stimulated surface (belly, thigh, shin), and trained surface (belly, thigh, shin, untrained). Note that the participants were tested on all three surfaces but trained on only one of them. Standard deviations of the mean are in brackets

<table><caption>**Table 2.**</caption><thead><tr><th rowspan="2">Surface trained</th><th colspan="3">Baseline</th><th colspan="3">Post-training</th><th colspan="3">Training effect</th></tr><tr><th>Belly</th><th>Thigh</th><th>Shin</th><th>Belly</th><th>Thigh</th><th>Shin</th><th>Belly</th><th>Thigh</th><th>Shin</th></tr></thead><tbody><tr><td>Belly</td><td>87.9 (8.8)</td><td>77.5 (11.9)</td><td>71.7 (14.0)</td><td>95.2 (7.8)</td><td>89.6 (6.7)</td><td>87.3 (13.4)</td><td>7.3 (9.9)</td><td>12.1 (10.7)</td><td>15.6 (8.9)</td></tr><tr><td>Thigh</td><td>81.5 (14.7)</td><td>74.8 (22.6)</td><td>74.6 (16.9)</td><td>92.3 (8.6)</td><td>87.5 (11.7)</td><td>87.1 (13.2)</td><td>10.8 (11.0)</td><td>12.7 (13.1)</td><td>12.5 (12.9)</td></tr><tr><td>Shin</td><td>87.1 (10.7)</td><td>81.3 (13.4)</td><td>79.8 (22.3)</td><td>94.4 (8.7)</td><td>91.7 (7.7)</td><td>90.2 (14.9)</td><td>7.3 (8.0)</td><td>10.4 (11.2)</td><td>10.4 (14.2)</td></tr><tr><td>Untrained (control)</td><td>84.6 (9.5)</td><td>69.4 (16.0)</td><td>66.0 (16.3)</td><td>86.2 (11.6)</td><td>73.8 (15.7)</td><td>75.2 (13.6)</td><td>1.6 (6.1)</td><td>4.4 (12.1)</td><td>9.2 (12.1)</td></tr></tbody></table>

Mean response times (in ms) during the baseline and post-training sessions and the corresponding training effects, as a function stimulated surface (belly, thigh, shin), and trained surface (belly, thigh, shin, untrained). Standard deviations of the mean are in brackets

<table><thead><tr><th rowspan="2">Surface trained</th><th colspan="3">Baseline</th><th colspan="3">Post-training</th><th colspan="3">Training effect</th></tr><tr><th>Belly</th><th>Thigh</th><th>Shin</th><th>Belly</th><th>Thigh</th><th>Shin</th><th>Belly</th><th>Thigh</th><th>Shin</th></tr></thead><tbody><tr><td>Belly</td><td>3106 (230)</td><td>3247 (320)</td><td>3286 (293)</td><td>2843 (197)</td><td>2931 (202)</td><td>2964 (199)</td><td>263 (136)</td><td>316 (221)</td><td>322 (232)</td></tr><tr><td>Thigh</td><td>3216 (259)</td><td>3182 (240)</td><td>3311 (266)</td><td>2966 (236)</td><td>3008 (205)</td><td>2971 (201)</td><td>250 (250)</td><td>174 (175)</td><td>340 (277)</td></tr><tr><td>Shin</td><td>3174 (244)</td><td>3279 (349)</td><td>3218 (336)</td><td>2863 (307)</td><td>3038 (349)</td><td>2973 (351)</td><td>311 (237)</td><td>241 (250)</td><td>245 (205)</td></tr><tr><td>Untrained (control)</td><td>3422 (290)</td><td>3551 (260)</td><td>3548 (293)</td><td>3225 (279)</td><td>3335 (207)</td><td>3336 (211)</td><td>197 (251)</td><td>216 (197)</td><td>212 (198)</td></tr></tbody></table>
<caption>**Table 3.**</caption>

Confusion matrices of trained participants' responses for each stimulated surface in the baseline and post-training sessions. The scores are expressed in percentages. Digits in bold indicate correct recognition percentages. Values may not add to 100% due to rounding

<table><thead><tr><th colspan="10">Baseline</th><th colspan="10">Post-training</th></tr><tr><th rowspan="2">Stimulus</th><th colspan="8">Response</th><th rowspan="2">Stimulus</th><th colspan="8">Response</th></tr><tr><th>B</th><th>C</th><th>F</th><th>G</th><th>K</th><th>N</th><th>P</th><th>Q</th><th>Missing</th><th>B</th><th>C</th><th>F</th><th>G</th><th>K</th><th>N</th><th>P</th><th>Q</th><th>Missing</th></tr></thead><tbody><tr><th colspan="20" style="text-align: left;">Belly</th></tr><tr><td>B</td><td><strong>97</strong></td><td>0</td><td>0</td><td>1</td><td>0</td><td>1</td><td>0</td><td>1</td><td>1</td><td>B</td><td><strong>99</strong></td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>0</td><td>0</td></tr><tr><td>C</td><td>0</td><td><strong>91</strong></td><td>1</td><td>5</td><td>0</td><td>0</td><td>2</td><td>1</td><td>1</td><td>C</td><td>0</td><td><strong>97</strong></td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td><td>1</td><td>1</td></tr><tr><td>F</td><td>0</td><td>0</td><td><strong>80</strong></td><td>0</td><td>11</td><td>0</td><td>8</td><td>0</td><td>1</td><td>F</td><td>1</td><td>0</td><td><strong>89</strong></td><td>1</td><td>8</td><td>0</td><td>1</td><td>0</td><td>0</td></tr><tr><td>G</td><td>0</td><td>2</td><td>0</td><td><strong>90</strong></td><td>2</td><td>0</td><td>2</td><td>2</td><td>2</td><td>G</td><td>0</td><td>1</td><td>1</td><td><strong>91</strong></td><td>1</td><td>1</td><td>1</td><td>6</td><td>0</td></tr><tr><td>K</td><td>0</td><td>0</td><td>13</td><td>1</td><td><strong>75</strong></td><td>8</td><td>1</td><td>2</td><td>1</td><td>K</td><td>1</td><td>0</td><td>4</td><td>1</td><td><strong>89</strong></td><td>5</td><td>0</td><td>1</td><td>0</td></tr><tr><td>N</td><td>0</td><td>0</td><td>2</td><td>3</td><td>3</td><td><strong>82</strong></td><td>7</td><td>0</td><td>3</td><td>N</td><td>0</td><td>0</td><td>0</td><td>1</td><td>1</td><td><strong>94</strong></td><td>1</td><td>1</td><td>3</td></tr><tr><td>P</td><td>2</td><td>0</td><td>1</td><td>0</td><td>3</td><td>2</td><td><strong>91</strong></td><td>1</td><td>2</td><td>P</td><td>0</td><td>0</td><td>1</td><td>1</td><td>1</td><td><strong>97</strong></td><td>1</td><td>0</td><td>0</td></tr><tr><td>Q</td><td>1</td><td>0</td><td>1</td><td>18</td><td>1</td><td>0</td><td>1</td><td><strong>79</strong></td><td>0</td><td>Q</td><td>0</td><td>0</td><td>0</td><td>4</td><td>1</td><td>0</td><td>0</td><td><strong>93</strong></td><td>1</td></tr><tr><th colspan="20" style="text-align: left;">Thigh</th></tr><tr><td>B</td><td><strong>92</strong></td><td>0</td><td>0</td><td>1</td><td>4</td><td>1</td><td>0</td><td>2</td><td>0</td><td>B</td><td><strong>99</strong></td><td>0</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><td>C</td><td>0</td><td><strong>89</strong></td><td>1</td><td>5</td><td>1</td><td>0</td><td>1</td><td>1</td><td>3</td><td>C</td><td>1</td><td><strong>92</strong></td><td>0</td><td>7</td><td>0</td><td>0</td><td>1</td><td>0</td><td>0</td></tr><tr><td>F</td><td>2</td><td>0</td><td><strong>67</strong></td><td>1</td><td>19</td><td>4</td><td>6</td><td>2</td><td>1</td><td>F</td><td>0</td><td>0</td><td><strong>80</strong></td><td>0</td><td>13</td><td>2</td><td>3</td><td>1</td><td>0</td></tr><tr><td>G</td><td>1</td><td>6</td><td>1</td><td><strong>79</strong></td><td>2</td><td>0</td><td>2</td><td>9</td><td>1</td><td>G</td><td>1</td><td>3</td><td>0</td><td><strong>92</strong></td><td>1</td><td>0</td><td>1</td><td>3</td><td>0</td></tr><tr><td>K</td><td>1</td><td>0</td><td>16</td><td>4</td><td><strong>58</strong></td><td>14</td><td>3</td><td>2</td><td>2</td><td>K</td><td>0</td><td>0</td><td>6</td><td>0</td><td><strong>86</strong></td><td>7</td><td>2</td><td>0</td><td>0</td></tr><tr><td>N</td><td>0</td><td>1</td><td>1</td><td>1</td><td>7</td><td><strong>79</strong></td><td>9</td><td>0</td><td>3</td><td>N</td><td>0</td><td>0</td><td>1</td><td>1</td><td>1</td><td><strong>90</strong></td><td>7</td><td>0</td><td>0</td></tr><tr><td>P</td><td>3</td><td>1</td><td>2</td><td>3</td><td>2</td><td>3</td><td><strong>82</strong></td><td>2</td><td>2</td><td>P</td><td>1</td><td>0</td><td>1</td><td>0</td><td>2</td><td><strong>89</strong></td><td>2</td><td>1</td><td>0</td></tr><tr><td>Q</td><td>3</td><td>2</td><td>1</td><td>18</td><td>1</td><td>1</td><td>1</td><td><strong>75</strong></td><td>1</td><td>Q</td><td>0</td><td>0</td><td>1</td><td>10</td><td>1</td><td>0</td><td>0</td><td><strong>88</strong></td><td>0</td></tr><tr><th colspan="20" style="text-align: left;">Shin</th></tr><tr><td>B</td><td><strong>86</strong></td><td>0</td><td>0</td><td>1</td><td>1</td><td>5</td><td>2</td><td>3</td><td>3</td><td>B</td><td><strong>96</strong></td><td>0</td><td>0</td><td>1</td><td>0</td><td>1</td><td>0</td><td>1</td><td>1</td></tr><tr><td>C</td><td>0</td><td><strong>92</strong></td><td>0</td><td>4</td><td>0</td><td>0</td><td>1</td><td>1</td><td>1</td><td>C</td><td>0</td><td><strong>95</strong></td><td>1</td><td>1</td><td>1</td><td>1</td><td>2</td><td>0</td><td>0</td></tr><tr><td>F</td><td>1</td><td>0</td><td><strong>77</strong></td><td>2</td><td>15</td><td>1</td><td>2</td><td>0</td><td>2</td><td>F</td><td>1</td><td>0</td><td><strong>86</strong></td><td>0</td><td>11</td><td>1</td><td>2</td><td>0</td><td>0</td></tr><tr><td>G</td><td>1</td><td>10</td><td>0</td><td><strong>79</strong></td><td>1</td><td>1</td><td>2</td><td>6</td><td>2</td><td>G</td><td>1</td><td>2</td><td>1</td><td><strong>92</strong></td><td>2</td><td>0</td><td>1</td><td>1</td><td>1</td></tr><tr><td>K</td><td>2</td><td>0</td><td>28</td><td>2</td><td><strong>57</strong></td><td>8</td><td>1</td><td>1</td><td>2</td><td>K</td><td>0</td><td>0</td><td>13</td><td>0</td><td><strong>80</strong></td><td>6</td><td>0</td><td>1</td><td>0</td></tr><tr><td>N</td><td>1</td><td>1</td><td>2</td><td>3</td><td>5</td><td><strong>61</strong></td><td>21</td><td>1</td><td>6</td><td>N</td><td>0</td><td>0</td><td>2</td><td>1</td><td>1</td><td><strong>76</strong></td><td>19</td><td>1</td><td>1</td></tr><tr><td>P</td><td>3</td><td>0</td><td>3</td><td>1</td><td>4</td><td>4</td><td><strong>78</strong></td><td>1</td><td>6</td><td>P</td><td>2</td><td>0</td><td>1</td><td>1</td><td>1</td><td>3</td><td><strong>92</strong></td><td>0</td><td>1</td></tr><tr><td>Q</td><td>1</td><td>0</td><td>1</td><td>23</td><td>3</td><td>0</td><td>0</td><td><strong>72</strong></td><td>1</td><td>Q</td><td>0</td><td>0</td><td>1</td><td>8</td><td>2</td><td>1</td><td>0</td><td><strong>89</strong></td><td>0</td></tr></tbody></table>

intensive training with trial-by-trial feedback (trained group) improved performance to a greater extent than the participants merely repeating the task twice on each body surface (control group). Second, performance improvement was not restricted to the trained body surface but it transferred to the untrained ones. Third, the obtained transfer of tactile learning was similar for surfaces represented in adjacent (e.g., belly and thigh) and in non-adjacent
(e.g., belly and shin) areas of the somatosensory cortex (see Merzenich et al., 1978; Nakamura et al., 1998, for descriptions of the somatosensory cortex's topographic organization). Taken together, these results reveal that there is a transfer of learning in tactile letter recognition, which occurs independently of the distance between body surfaces. These three results provide support for Bach-y-Rita's claim that, after training with visual-to-tactile sensory substitution devices, perceptual abilities reached on a given body surface transfer to another one (Bach-y-Rita and Kercel, 2003). However, as will be further highlighted below, the extent to which there is no loss in perceptual performance needs to be further qualified.

The degree of learning transfer is considered to reflect the level of involved perceptual processing (Recanzone et al., 1992). Transfer has therefore been reported to depend both on the task and on the stimuli. For example, transfer was not found when using a passive two-point discrimination task (Dinse et al., 2006; Godde et al., 2000) whereas transfer was found in gratings tasks, which involve more active discrimination, i.e. actively scanning the shapes with the fingertip (Harrar et al., 2014; Sathian and Zangladze, 1997). Transfer also depends on the type of tactile stimuli (vibration, pressure, or roughness; see Harris et al., 2001). Moreover, using low-level stimuli only, the above-mentioned studies reported a transfer limited to adjacent and homologous contralateral body surfaces only. The results reported here reveal that, contrary to the discrimination of low-level stimuli, the recognition of high-level symbols transfers across body surfaces without restrictions of their closeness in location. As compared with discrimination tasks of low-level stimuli varying along one feature (e.g., gratings with different orientations), the recognition of letters, which are made of several features in combination, may require the involvement of a holistic process allowing the integration of the different features into a global and unique percept. This holistic process involves higher-level areas than those perceptual processes that operate on simple features (Lerner et al., 2001; Tanaka, 2003). Moreover, in our study, the sequential mode of drawing the letters may have involved a spatiotemporal integration process rather than a purely spatial one that would be involved with a simultaneous mode of presentation. This spatiotemporal integration process also depends on high-level areas (Battelli et al., 2007). High-level perceptual areas are less topographically organized than low-level ones. Consequently, their receptive fields are larger and not restricted to a specific area, facilitating the transfer of perceptual learning across body surfaces that are further apart. The transfer of learning we found regarding high-level symbol recognition is thus compatible with the reverse hierarchy theory, which describe a perceptual learning guided by a top-down process, from high-level to low-level areas (Ahissar and Hochstein, 2004).
In the letter recognition task used in our study, training effects occurred at different levels. During the training session, the repetition of letters improved the recognition of the sequence of tactile stimuli corresponding to each letter. The trial-by-trial feedback provided in this training session also helped participants to associate the sequence of tactile stimuli with the correct letter. In addition, the performance improvement that was observed for the control group indicates that participants are learning the task before the training session. This result is probably influenced both by the use of three different blocks during the baseline session and by the repeated exposure of letters within each block. Harrar *et al.* (2014), using a similar procedure but with a grating discrimination task, did not find such a performance improvement for their control group. This difference between Harrar *et al.*'s and our study suggests that perceptual learning is faster for high-level than for low-level stimuli; which is consistent with the reverse hierarchy theory (Ahissar and Hochstein, 2004). However, the greater performance improvement that was obtained for the trained group than for the control group suggests that explicit training improved performance beyond a mere repeated exposure effect and thereby it supports the claim that a perceptual learning transfer occurred in our study.

It should be mentioned that the perception of the global configuration (e.g., the letter B which is made of one vertical stroke and two loops) was probably sufficient to correctly recognize tactile letters. However, the participants in our study reported more difficulties in correctly discriminating between two letters with similar global configurations (e.g., the letters F and K which are both made of a vertical stroke and two other strokes; see the confusion matrices in Table 3) that require finer discrimination processes (e.g., to discriminate between the horizontal strokes of the letter F and the diagonal strokes of the letter K). The discrimination of feature differences required for recognition of high-level symbols with high similarity may thus involve lower-level processes and reduce the amount of perceptual learning transfer. However, according to the reverse hierarchy theory, expert perceivers (in touch as well as in other sensory modalities), who have had substantial amounts of training, are able to base their perception on high-level processes even in difficult conditions (e.g., discrimination of stimuli with high similarity such as two exemplars of the same bird species); they then show transfer of perceptual learning (Ahissar and Hochstein, 2004). Intensive training should thus allow transfer of tactile learning even in the cases of discrimination of highly similar tactile symbols.

Although the results obtained in the present study support the transfer of tactile learning across body surfaces, they cannot fully support the independence of perceptual abilities from the stimulated body surface that has been claimed to occur after training. The most important counter-argument to this
claim is that there would remain differences in tactile recognition and discrim-
ination abilities as a function of the location in which the stimuli are displayed
on the body surface. For instance, tactile discrimination abilities have been
reported to be higher on the belly than on the legs (Haggard et al., 2003). Simi-
larly, in our study, training improved letter recognition on all body surfaces but
letter recognition remained less accurate on the legs than on the belly. How-
ever, even if the claim must be watered-down in order to include differences in
acuity across body surfaces (i.e., as a function of the differences in resolution
of their respective tactile receptive fields) further investigations would still be
required to demonstrate an independence from the stimulated body surface.
First of all, the participants in our study were given purely sensory learning
whereas sensory substitution devices involve a sensorimotor learning. This
however has been shown to reinforce learning transfer across different sensory
modalities (see Levy-Tzedek et al., 2012). Second, the body surfaces used in
our study remain relatively closely represented in the somatosensory cortex;
thus the use of more distant body surfaces is necessary to more fully assess the
possibility of complete transfer. If perceptual learning can completely transfer
to all body surfaces, this should include distant body surfaces such as head
and legs. Third, the tactile letters we used are more complex than low-level
stimuli varying along only one feature but most sensory substitution devices
convey even higher-level stimuli such as faces, objects and scenes. Moreover,
the sequential mode of presentation may be more appropriate for letters than
for other objects (such as everyday 3-D objects) for which it would be difficult
to sequentially order their different parts in an intuitive way. Further studies
should then extend the actual results to a broader range of stimuli of varying
complexity and to alternative modes of presentation.

Finally, it has been suggested that the body surface independence in visual-
to-tactile sensory substitution is probably due to the fact that, after training,
users no longer feel the stimulation on their skin, but they directly attribute the
stimulation as resulting from an external object (see Bach-y-Rita and Kercel,
2003). This externalization process would reinforce the independence of tac-
tile learning from the stimulated surface (see Hartcher-O’Brien and Auvray,
submitted, for a review on externalization). This claim would need further in-
vestigation as well to be corroborated. However, what can be said for now
is that our study provides the first empirical data to support Bach-y-Rita’s
claim that training in sensory substitution improves perceptual abilities, and
that these perceptual abilities transfer from one body surface to another, re-
sulting in a relative independence from the stimulated body surface. These
results have implications for the use of sensory substitution devices and vibro-
tactile systems by visually impaired people. First, they suggest that training
on one body surface is beneficial to the entire body. As a consequence, users
do not need to undergo an extensive training on all body surfaces. This study
also provides better understanding of the learning process, which is a crucial feature for efficient use of tactile devices. It reveals that transfer of learning for tactile letters occurs within a relatively short time frame, i.e., the 90 min of our experiment. The longer learning reported for recognition of more complex objects with sensory substitution devices (Kaczmarek and Haase, 2003) indicates that the amount of training required to reach accurate object recognition with tactile devices depends on the complexity of the information that is provided. It would be interesting in future research to investigate the extent to which learning transfers between different subsets of stimuli with similar level of complexity, and between stimuli of varying complexity; for instance between moderately complex stimuli such as tactile letters and highly complex stimuli such as everyday 3-D objects.

## Acknowledgements

We wish to thank Jessica Hartcher O’Brien for her careful reading and helpful comments on previous versions of this article. We thank Gaëtan Parseihan and Mathieu Dubois for their help in designing the apparatus. We also thank the editor, Charles Spence, and the two anonymous reviewers for their stimulating comments. G.A. is funded by a grant from the Agence Nationale de la Recherche (ANR-11-JSH2-003-1). The Institut Jean Nicod laboratory finds support in ANR-10-LABX-0087 IEC and ANR-10-IDEX-0001-02 PSL.

## Note

1. Note that Bach-y-Rita also concurrently asserted that the camera could then be moved, for instance from a hand-held to a head-mounted display, without any loss in performance. However, the present study only focused on the effect of training on a change in the stimulated body surface and not on a change in the camera's position.

## References

Ahissar, M. and Hochstein, S. (2004). The reverse hierarchy theory of visual perceptual learning, *Trends Cogn. Sci.* **8**, 457–464.

Arno, P., Vanlierde, A., Streel, E., Wanet Defalque, M. C., Sanabria-Bohorquez, S. and Veraart, C. (2001). Auditory substitution of vision: pattern recognition by the blind, *Appl. Cogn. Psychol.* **15**, 509–519.

Auvray, M., Hanneton, S. and O'Regan, J. K. (2007). Learning to perceive with a visuo-auditory substitution system: localization and object recognition with The Voice, *Perception* **36**, 416–430.

Auvray, M. and Myin, E. (2009). Perception with compensatory devices: from sensory substitution to sensorimotor extension, *Cogn. Sci.* **33**, 1036–1058.
Bach-y-Rita, P. (1995). *Nonsynaptic Diffusion Neurotransmission and Late Brain Reorganization*. Demos, New York, NY, USA.

Bach-y-Rita, P., Collins, C. C., Saunders, F., White, B. and Scadden, L. (1969). Vision substitution by tactile image projection, *Nature* **221**, 963–964.

Bach-y-Rita, P. and Kercel, S. W. (2003). Sensory substitution and the human–machine interface, *Trends Cogn. Sci.* **17**, 541–546.

Battelli, L., Pascual-Leone, A. and Cavanagh, P. (2007). The “when” pathway of the right parietal lobe, *Trends Cogn. Sci.* **11**, 204–210.

Bubic, A., Striem-Amit, E. and Amedi, A. (2010). Large-scale brain plasticity following blindness and the use of sensory substitution devices, in: *Multisensory Object Perception in the Primate Brain*, J. Kaiser and M. J. Naumer (Eds), pp. 351–380. Springer, New York, NY, USA.

Capelle, C., Trullemans, C., Arno, P. and Veraart, C. (1998). A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution, *IEEE Trans. Biomed. Eng.* **45**, 1279–1293.

Deroy, O. and Auvray, M. (2012). Reading the world through sensory substitution devices, *Front. Theor. Philos. Psychol.* **3**, 457.

Dinse, H. R., Kleibel, N., Kalisch, T., Ragert, P., Wilimzig, C. and Tegenthoff, M. (2006). Tactile coactivation resets age-related decline of human tactile discrimination, *Ann. Neurol.* **60**, 88–94.

Fahle, M. E. and Poggio, T. E. (2002). *Perceptual Learning*. MIT Press, Cambridge, MA, USA.

Gibson, E. J. (1969). *Principles of Perceptual Learning and Development*. Appleton Century Crofts, New York, NY, USA.

Godde, B., Stauffenberg, B., Spengler, F. and Dinse, H. R. (2000). Tactile coactivation-induced changes in spatial discrimination performance, *J. Neurosci.* **20**, 1597–1604.

Haggard, P., Taylor-Clarke, M. and Kennett, S. (2003). Tactile perception, cortical representation and the bodily self, *Curr. Biol.* **13**, 170–173.

Haigh, A., Brown, D. J., Meijer, P. and Proulx, M. J. (2013). How well do you see what you hear? The acuity of visual-to-auditory sensory substitution, *Front. Psychol.* **4**, 330.

Hanneton, S., Auvray, M. and Durette, B. (2010). The Vibe: a versatile vision-to-audition sensory substitution device, *Appl. Bionics Biomech.* **7**, 269–276.

Harrar, V., Spence, C. and Makin, T. R. (2014). Topographic generalization of tactile perceptual learning, *J. Exp. Psychol. Hum. Percept. Perform.* **40**, 15–23.

Harris, J. A., Harris, I. M. and Diamond, M. E. (2001). The topography of tactile learning in humans, *J. Neurosci.* **21**, 1056–1061.

Hartcher O’Brien, J. and Auvray, M. (submitted). The process of distal attribution illuminated through studies of sensory substitution, *Multisens. Res.*

Herzog, M. H. and Fahle, M. (1998). Modeling perceptual learning: difficulties and how they can be overcome, *Biol. Cybern.* **78**, 107–117.

Iwamura, Y., Iriki, A. and Tanaka, M. (1994). Bilateral hand representation in the postcentral somatosensory cortex, *Nature* **369**, 554–556.

Jansson, G. (1983). Tactile guidance of movement, *Int. J. Neurosci.* **19**, 37–46.

Kaas, A. L., van de Ven, V., Reithler, J. and Goebel, R. (2013). Tactile perceptual learning: learning curves and transfer to the contralateral finger, *Exp. Brain Res.* **224**, 477–488.
Kaczmarek, K. A. and Haase, S. J. (2003). Pattern identification and perceived stimulus quality as a function of stimulation current on a fingertip-scanned electrotactile display, *IEEE Trans. Neural Syst. Rehabil. Eng.* **11**, 9–16.

Lerner, Y., Hendler, T., Ben-Bashat, D., Harel, M. and Malach, R. (2001). A hierarchical axis of object processing stages in the human visual cortex, *Cereb. Cortex* **11**, 287–297.

Levy-Tzedek, S., Hanassy, S., Abboud, S., Maidenbaum, S. and Amedi, A. (2012). Fast, accurate reaching movements with a visual-to-auditory sensory substitution device, *Restor. Neurol. Neurosci.* **30**, 313–323.

Loomis, J. M. (1974). Tactile letter recognition under different modes of stimulus presentation, *Percept. Psychophys.* **16**, 401–408.

Meijer, P. B. L. (1992). An experimental system for auditory image representations, *IEEE Trans. Biomed. Eng.* **39**, 112–121.

Merzenich, M. M., Kaas, J. H., Sur, M. and Lin, C.-S. (1978). Double representation of the body surface within cytoarchitectonic area 3b and 1 in “SI” in the owl monkey (*Aotus trivirgatus*), *J. Comp. Neurol.* **181**, 41–74.

Nagarajan, S. S., Blake, D. T., Wright, B. A., Byl, N. and Merzenich, M. M. (1998). Practice-related improvements in somatosensory interval discrimination are temporally specific but generalize across skin location, hemisphere, and modality, *J. Neurosci.* **18**, 1559–1570.

Nakamura, A., Yamada, T., Goto, A., Kato, T., Ito, K., Abe, Y., Kachib, T. and Kakigic, R. (1998). Somatosensory homunculus as drawn by MEG, *NeuroImage* **74**, 377–386.

Parsons, L. M. and Shimojo, S. (1987). Perceived spatial organization of cutaneous patterns on surfaces of the human body in various positions, *J. Exp. Psychol. Hum. Percept. Perform.* **13**, 488–504.

Pollok, B., Schnitzler, I., Mierdorf, T., Stoerig, P. and Schnitzler, A. (2005). Image-to-sound conversion: experience-induced plasticity in auditory cortex of blindfolded adults, *Exp. Brain Res.* **167**, 287–291.

Recanzone, G. H., Merzenich, M. M. and Schreiner, C. E. (1992). Changes in the distributed temporal response properties of SI cortical neurons reflect improvements in performance on a temporally based tactile discrimination task, *J. Neurophysiol.* **67**, 1071–1091.

Renier, L., Collignon, O., Poirier, C., Tranduy, D., Vanlierde, A., Bol, A., Veraart, C. and De Volder, A. G. (2005). Cross-modal activation of visual cortex during depth perception using auditory substitution of vision, *NeuroImage* **26**, 573–580.

Sampaio, E., Maris, S. and Bach-y-Rita, P. (2001). Brain plasticity: ‘Visual’ acuity of blind persons via the tongue, *Brain Res.* **908**, 204–207.

Sathian, K. and Zangaladze, A. (1997). Tactile learning is task specific but transfers between fingers, *Percept. Psychophys.* **59**, 119–128.

Sekiyama, K. (1991). Importance of head axes in perception of cutaneous patterns drawn on vertical body surfaces, *Percept. Psychophys.* **49**, 481–492.

Shaw, R. G. and Mitchell-Olds, T. (1993). ANOVA for unbalanced data: an overview, *Ecology* **74**, 1638–1645.

Spengler, F., Roberts, T. P. L., Poeppel, D., Byl, N., Wang, X., Rowley, H. A. and Merzenich, M. M. (1997). Learning transfer and neuronal plasticity in humans trained in tactile discrimination, *Neurosci. Lett.* **232**, 151–154.
Tanaka, K. (2003). Columns for complex visual object features in the inferotemporal cortex: clustering of cells with similar but slightly different stimulus selectivities, *Cereb. Cortex* **13**, 90–99.

Yanagida, Y., Kakita, M., Lindeman, R. W., Kume, Y. and Tetsutani, N. (2004). Vibrotactile letter reading using a low-resolution tactor array, in: *Proc. 12th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems*, pp. 400–406. IEEE Computer Society.
# Tactile recognition of visual stimuli: Specificity versus generalization of perceptual learning

Gabriel Arnold*, Malika Auvray

Institut des Systèmes Intelligents et de Robotique, CNRS UMR 7222, Université Pierre et Marie Curie, Paris, France

## ARTICLE INFO

**Keywords:**

Perceptual learning

Generalization

Learning strategies

Multisensory perception

Sensory substitution

## ABSTRACT

Sensory substitution devices aim at assisting a deficient sensory modality by means of another sensory modality. For instance, to perceive with visual-to-tactile devices, users learn to recognize visual stimuli through their tactile conversion. A crucial characteristic of learning lies in the ability to generalize, that is, the ability to extend the acquired perceptual abilities to both new stimuli and new perceptual conditions. The study reported here investigated the perceptual learning of tactile alphanumerical stimuli. The learning protocol consisted in alternating a repeated list of symbols with lists of new symbols. A first experiment revealed that, when each list consisted of 4 stimuli, recognition performance improved over time only for the repeated list. This result suggests that learning a small set of stimuli involves stimulus-specific learning strategies, preventing generalization. A second experiment revealed that increasing to six the set of learned stimuli results in higher generalization abilities. This result can be explained by greater difficulties in using stimulus-specific strategies in this case, thereby favouring the use of generalization strategies. Feature variability also appeared to be important to achieve generalization. Thus, as in visual perceptual learning, the involvement of stimulus-specific versus general strategies depends on task difficulty and feature variability. A third experiment highlighted that tactile perceptual learning generalizes to changes in orientation. These results are discussed in terms of brain plasticity as they influence the design of learning methods for using sensory substitution devices, with the aim to compensate visual impairments.

## 1. Introduction

Perceptual learning, defined as the performance improvement in perceptual tasks resulting from training (Fahle & Poggio, 2002), reveals the plasticity of humans' sensory systems. Sensory substitution is an illustrative example for which perceptual learning and plasticity play a crucial role. Sensory substitution devices aim at assisting a deficient sensory modality (e.g., vision) by means of another sensory modality (e.g., audition or touch; see Auvray & Myin, 2009, for a review). Even if the question of whether sensory substitution genuinely restores vision to their users is highly debatable, this technique provides a new way to receive sensory information from the environment. Brain plasticity allows users to have access to information in a novel way. For instance, after training with a visual-to-tactile conversion device, the tactile stimulation, initially felt on the surface of the skin, is attributed to external objects (Hartcher-O'Brien & Auvray, 2014).

Numerous studies have reported that sensory substitution devices enable blind and blindfolded sighted people to localize (e.g., Levy-Tzedek, Hanassy, Abboud, Maidenbaum, & Amedi, 2012; Renier et al.,

2005) and recognize simple shapes or complex objects (e.g., Arno et al., 2001; Auvray, Hanneton, & O'Regan, 2007; Pollok, Schnitzler, Mierdorf, Stoerig, & Schnitzler, 2005; Sampaio, Maris, & Bach-y-Rita, 2001). Most users are able, in one or two hours, to approximate objects' location and shape. However, to gain precision in these tasks and to perform more complex ones, a more intensive training is required. The length of the training is estimated at around eight hours with visual-to-tactile devices (Kaczmarek & Haase, 2003) and 10-15 with visual-to-auditory devices (Auvray, Hanneton, & O'Regan, 2007). It would be even more important to achieve a high level of expertise, for instance to be able to recognize facial expressions (Striem-Amit, Guendelman, & Amedi, 2012).

A crucial characteristic of learning lies in the ability to generalize, that is, to extend the acquired perceptual abilities to both new stimuli and new perceptual conditions. For instance, in the case of learning a sensory substitution device, generalization enables their users, trained in specific conditions and with specific shapes and objects, to extract the rules underlying perception with the device and to apply them to new objects or to previously learned objects perceived in new

<footnote>* Corresponding author.</footnote>

<footnote>E-mail address: arnold@isir.upmc.fr (G. Arnold).</footnote>
conditions. In other words, users do not simply associate visual stimuli to their auditory or tactile conversion, but they become able to extract the perceptual characteristics of a scene and of their constituting objects and then they are able to transfer this learning to new scenes and objects.

Studies have reported contrasting results regarding specificity versus generalization of perceptual learning (see Fahle, 2005, for a review). For instance, visual perceptual learning can be highly specific to retinal position, object orientation, and object shape (e.g., Sigman & Gilbert, 2000; Yi, Olson, & Chun, 2006). Tactile perceptual learning has also been reported to be specific to the trained body surface, with a transfer of learning restricted to adjacent or homologous contralateral surfaces (e.g., Harrar, Spence, & Makin, 2014; Sathian & Zangaladze, 1997). On the other hand, studies have reported generalization of learning. For instance, Furmanski and Engel (2000) have found object-specific but size-invariant perceptual learning in visual object recognition. Perceptual learning has also been reported to transfer to non-adjacent body surfaces in tactile recognition of alphanumerical symbols (Arnold & Auvray, 2014).

The reverse hierarchy theory (RHT) of perceptual learning (Ahissar & Hochstein, 2004) has been proposed to account for these discrepant results. According to this theory, the degree of specificity depends on the difficulty of the perceptual task and on the level of cortical processing that is required to perform the task. For a difficult task, involving low-level cortical processes that are highly specific to the characteristics of the task (e.g., fine discrimination of visual orientations), perceptual learning is specific to the characteristics of the task (e.g., the visual orientations that have been discriminated) and it does not generalize to new characteristics (e.g., new orientations). On the other hand, for an easier task, involving higher-level processing, perceptual learning is less specific to the low-level characteristics of the task or of the stimuli. Task difficulty, either during the learning phase or during the evaluation of generalization, has indeed been reported to influence both the specificity and generalization of learning (Jeter, Dosher, Petrov, & Lu, 2009; Liu & Weinshall, 2000; Wang, Zhou, & Liu, 2013). Note however that, in some cases, generalization turns out to be easier with a difficult task than with an easy one (Liu & Weinshall, 2000). Other factors than task difficulty can play a role in influencing specificity versus generalization of learning. For instance, the variability of the learned stimuli contributes to generalization (Hussain, Bennett, & Sekuler, 2012). On the other hand, a long duration of the learning phase contributes to specificity (Jeter, Dosher, Liu, & Lu, 2010), possibly due to sensory adaptation (Harris, Gliksberg, & Sagi, 2012).

Perceptual learning can also transfer from one sensory modality to another. For instance, learning to discriminate tactile intervals has been reported to transfer to the auditory sensory modality (Nagarajan, Blake, Wright, Byl, & Merzenich, 1998). Based on the RHT (Ahissar & Hochstein, 2004), Proulx, Brown, Pasqualotto, and Meijer (2014) made the hypothesis that the broad brain activation of multisensory processing reinforces generalization of learning, especially because multisensory processing involves high-level multimodal brain areas. Sensory substitution, by converting the information from one sensory modality into another, is thus an ideal candidate for observing general perceptual learning. In this field of research, Kim and Zatorre (2008) found that learning to recognize simple shapes and complex objects with a visual-to-auditory conversion device generalizes to new objects. This result was then reproduced with tactile-to-auditory conversion of shapes (Kim & Zatorre, 2010). Learning complex stimuli, corresponding to the auditory conversion of visual shapes, has also been reported to involve faster perceptual learning and greater amplitude of generalization to new stimuli than what occurred when learning simple stimuli (Brown & Proulx, 2013). Similarly Auvray et al., 2007; see also Auvray, 2004, for more details) found that learning every-day life objects transfers to some extent to the recognition of different objects belonging to the same category. Generalization to new stimuli with even more complex objects such as facial expressions has also been reported to occur

(Striem-Amit et al., 2012). Finally, sensory-motor learning with a visual-to-auditory device has been reported to transfer across sensory modalities (Levy-Tzedek, Novick et al., 2012). However, if few studies have investigated the generalization of learning with visual-to-auditory devices, no study to date has investigated the generalization of learning during the use of a visual-to-tactile device.

The study reported here aims at evaluating whether learning to recognize tactile conversion of visual stimuli generalizes to new stimuli and to new perceptual conditions. The methodology is based on a recognition task of alphanumerical symbols presented on the participants' stomach by means of sequences of vibrotactile stimulations. Alphanumerical symbols are usually perceived visually, and rarely by touch. Thus the tactile recognition task requires training, similarly to what occurs when learning to use a visual-to-tactile substitution device. A similar task was previously used to show that learning to recognize tactile symbols transfers from trained to untrained body surfaces (Arnold & Auvray, 2014). In the present study, the tactile symbols were presented on the stomach as, for visual-to-tactile sensory substitution devices, this body part presents a good compromise between the size of the skin surface and its spatial resolution (see Haggard, Taylor-Clarke, & Kennett, 2003, for spatial resolution of the different body surfaces). In addition, it is central on the body and, contrary to the hand, its position and orientation relative to the rest of the body does not change much. Finally, contrary to body surfaces with greater spatial resolution, in particular the fingertips and the tongue, the stomach is rarely used for everyday tactile perception (e.g., haptic object recognition) and it can thus be used for receiving additional tactile information without disturbing people's usual perception and action.

In the first two experiments reported here, the role of stimulus variability in the ability to generalize to new symbols during learning was evaluated by comparing the learning of sets of four symbols (Experiment 1; low variability) to the learning of sets of six symbols (Experiment 2; high variability). If stimulus variability influences the stimulus-specific versus generalized learning strategies, generalization should be better with high than with low stimulus variability. In the third experiment, the ability to generalize to new orientations was evaluated. Orientation is an important source of perceptual variability and changes in orientation frequently disturb the recognition of objects that are learned visually (e.g., Arnold & Siéroff, 2012) or by touch (e.g., Newell, Ernst, Tjan, & Bülthoff, 2001).

## 2. Experiment 1

Experiment 1 was designed to investigate whether learning to recognize tactile symbols generalizes to new symbols when learning a limited number of symbols. In order to investigate generalization to new symbols, a similar procedure to the one used by Dutilh, Krypotos, and Wagenmakers (2011) was used. It consists in alternating blocks with a repeated list of stimuli and blocks with new lists of stimuli. If stimulus-specific learning is involved, recognition performance should increase with practice (i.e., across blocks) only for repeated stimuli. In contrast, if generalized learning is involved, recognition performance should increase for both repeated and new stimuli.

The second aim of this experiment was to investigate whether the generalization of learning depends on the spatial perspective that is naturally adopted when interpreting tactile information presented on the body. Using the graphesthesia task, which consists in recognizing ambiguous asymmetrical symbols presented on the body surface (e.g., the letters b, d, p, and q), previous studies have shown that some observers prefer to adopt a self-centred spatial perspective (i.e., centred on the observer's body) whereas others prefer to adopt a decentred one (i.e., centred on a location different from that of the observer) (see Arnold, Spence, & Auvray, in press, for a review). These individual differences have been reported to reflect the existence of a natural perspective rather than being due to an arbitrary choice (Arnold, Spence, & Auvray, 2016). Moreover, decentred observers are better at
adopting an unnatural perspective than self-centred observers are. Thus, this experiment also investigated whether this superiority of naturally decentred observers over naturally self-centred ones is specific to the adoption of spatial perspectives or whether it reflects a general ease to interpret tactile information instead. When tactile stimuli are presented on people's stomach, three perspectives can be adopted, one decentred and two self-centred (trunk-centred or head-centred). In the present experiment, only those participants that naturally adopt a trunk-centred or a decentred perspective were tested.

## 2.1. Material and methods

### 2.1.1. Participants

Two hundred and fifty-five participants (154 females and 101 males; mean age = 25.2 years, range = 18–47 years; 30 participants were left-handed and 225 participants were right-handed) were initially recruited via mailing lists to participate in 6 different experiments (i.e., the 3 experiments reported in the present study and 3 experiments investigating different research questions reported in different studies). In all these experiments, the graphesthesia task was used to investigate the participants' spatial perspectives. The participants were randomly allocated to one of these 6 experiments in order to generate groups based upon natural perspective. Five participants were excluded from the set of experiments because their natural perspective could not be correctly identified. For the 250 remaining participants, 49.6% adopted a trunk-centred perspective, 29.2% a head-centred perspective, and 21.2% a decentred perspective. For the present study, 20 trunk-centred and 20 decentred participants were included in Experiment 1, 32 trunk-centred participants in Experiment 2, and 24 trunk-centred participants in Experiment 3.

Forty participants completed the present experiment (25 females and 15 males; mean age = 24.8 years, range = 18–41 years; three participants were left-handed and 37 participants were right-handed). Half of the participants (14 females and 6 males; mean age = 24.4 years, range = 18–41 years) had a natural trunk-centred perspective and the other half (11 females and 9 males; mean age = 25.2 years, range = 20–33 years) had a natural decentred perspective. All the participants were naive to the purpose of the experiment. Participants provided their informed consent and received payment for their participation. The experiment took approximately one hour to complete and was performed in accordance with the ethical standards laid down in the Declaration of Helsinki (1991).

### 2.1.2. Apparatus

The tactile stimuli were presented by means of 9 rectangular vibrators (Haptuator Mark II, Tactile Labs, Montreal, Canada) arranged in a 3-by-3 array with a centre-to-centre spacing of 5 cm (see Fig. 1a). Each vibrator measured 0.9 cm vertically by 3.2 cm horizontally for a total surface area of 10.9 cm vertically by 13.2 cm horizontally. A nine-channel amplifier drove each vibrator independently at a 250-Hz frequency. The vibrator array was placed on the participants' stomach, above their clothes, by means of an elastic belt. The vibrator array was placed symmetrically to their body mid-sagittal line, with the 3 lower vibrators located above the waist. Only one layer of clothing was allowed between the skin and the vibrators. The participants individually selected the intensity of each vibrator by means of an adjustment method.

Instructions and feedback were presented on a 23-inch screen with a 1920 × 1080 resolution. The participants wore noise-reducing headphones with a noise reduction rating of 30 dB, in order to mask any sounds made by the vibrators.

### 2.1.3. Stimuli

Twenty asymmetrical symbols were presented on the participants' stomach: the uppercase letters B, C, D, E, F, G, J, K, L, N, P, Q, R, and Z and the numbers 1, 2, 3, 4, 5, and 6. The use of asymmetrical symbols

<figure><img src="image_32.png" /><figcaption>Fig. 1. Illustration of the drawing of alphanumerical symbols with the 3-by-3 vibrotactile matrix. (a) Schematic figure illustrating the vibrotactile matrix. (b) The sequence of vibrations chosen for drawing the letter P.</figcaption></figure>

allowed presenting the symbols according to either trunk-centred or decentred perspectives. According to the trunk-centred perspective, the left and right of the symbols were congruent to participants' left-right axis. According to the decentred perspective, the left and right were opposite to participants' left-right axis. The top and bottom of the symbols were congruent to participants' top-bottom axis for both the trunk-centred and decentred perspectives.

The symbols were traced with sequences of vibrotactile stimulations mapping the trajectory of manual drawing (see Arnold & Auvray, 2014, for more details). Each vibrator composing the sequence was activated for 250 ms and there were 150 ms intervals between two consecutive vibrations that were spatiotemporally discontinuous (e.g., the end of the vertical descending stroke and the beginning of the loop of the letter P; see Fig. 1b). This interval avoids possible errors of interpretation caused by an irrelevant grouping of vibrations. The participants were instructed that these temporal intervals corresponded to a spatial discontinuity in manual letter drawing. It should be underlined that with such a device, the symbols' drawings do not exactly match manual drawing, as curved features are not possible and thus they are replaced by right-angled features. Tactile recognition of alphanumerical symbols has nonetheless previously been reported to be successful with the use of a similar 3-by-3 vibrotactile matrix (Yanagida, Kakita, Lindeman, Kume, & Tetsutani, 2004). The mean duration of symbols was 2215 ms (SD = 618). Five lists of four symbols were created with similar mean durations (List1: PQZ5, 2363 ms; List2: JK36, 2175 ms; List3: FLR4, 2063 ms; List4: BCD2, 2263 ms; EGN1, 2213 ms). Each of these lists was composed of three letters and one number except List2, which was composed of two letters, and two numbers.

In addition, the four ambiguous lowercase letters b, d, p, and q were used for the identification of each participant's natural perspective. The letters were traced beginning from the stem and ending with the loop,
with a sequence of eight 250 ms vibrations without intervals between each vibration.

### 2.1.4. Procedure

The first part of the experiment was designed to identify each participant's natural perspective. The participants sat in front of the computer screen. On each trial, one of the four possible ambiguous letters (b, d, p, and q) was displayed on the participants' stomach. The participants were instructed to report the letter they perceived as spontaneously as possible by pressing the corresponding key with the index finger of their dominant hand. They were informed that each vi-brotactile sequence could be interpreted as one of the four letters, depending on how they assign the left-right and top-bottom axes of the letter, and that there were consequently no correct or incorrect responses. The participants were asked to report their responses as fast as possible. They completed one block of 16 trials (four trials for each of the four letters). If no clear preference for adopting one of the perspectives was identified in the first block, the participants completed additional blocks until their natural perspective could be clearly identified, without exceeding six blocks. Only one block was required for 24 participants, two blocks for 12 participants, and three blocks for four participants. There are three possible causes for this individual variability that can occur individually or in combination. First, the natural perspective may emerge faster for some participants over others. Second, some people might not have a natural perspective. Third, some participants may have had difficulties to understand the instructions, leading them to adopt the most 'logical' perspective (i.e., the experimenter's perspective) rather than the easiest perspective for them.

The second part of the experiment was designed to investigate participants' abilities to recognize unambiguous asymmetrical symbols. On each trial, one symbol was presented on the participants' stomach. The participants were instructed to report which symbol was presented by pressing the corresponding key with the index finger of their dominant hand. Rather than using the real position of each letter and number on the computer keyboard, stickers with the four symbols used in each list written on it were placed on four adjacent keys on the keyboard. The keyboard was located in front of the participants, slightly shifted toward the left for left-handed participants or to the right for right-handed participants. In this part of the experiment, the participants were informed that the symbols would be presented according to their natural perspective and that there were correct and incorrect responses. The participants were asked to give their responses as accurately as possible. Response times (RTs) were recorded as a secondary measure of performance but emphasis was put on accuracy (see Supplementary material for the analyses of RTs). The participants were able to give their responses at any time from the onset of the first vibration and up to 3000 ms after the end of the last vibration (at which point the trial was terminated). At the end of each trial, there was an interval of 3000 ms before the beginning of the next trial.

The participants completed eight blocks of trials for the recognition task. In each block, one of the five lists was presented. One of the lists (the repeated list; labelled A) was presented four times. The other four lists (new lists; labelled B through E) were presented once each. Thus there were four blocks with the repeated list and four blocks with the new lists, which were alternated as follows: ABACADAE for half of the participants and BACADAEA for the other half. Note that each of the five lists served as the repeated list for a fifth of the participants (N = 16). In addition, each list was presented the same number of times across participants as lists B, C, D, and E.

Each block consisted of 40 trials (10 presentations of each symbol). At the end of each block of trials, global feedback indicating the percentage of correct responses and the mean RT was given. The participants had a short pause between each block. At the beginning of each block, the participants were given four practice trials (one trial for each symbol). In this practice block, feedback was given at the end of each trial: the correct answer was visually displayed on the computer screen

once the participants responded.

### 2.2. Results

Trials in which the participants failed to make a response before the trial was terminated (0.56% of trials) were not included in the data analyses. The accuracy of symbol recognition was measured by the percentage of correct responses in each condition. An ANOVA was conducted with Type of list (repeated, new) and Block (block 1, block 2, block 3, block 4) as within-participant factors and with Natural perspective (trunk-centred, decentred) as a between-participant factor. As there was no significant effect of Gender and no interaction between Gender and any of the other factors (in this experiment as well as in Experiments 2 and 3), this factor was not included in the reported analyses.

There was a significant effect of Block, $F(3114) = 3.46, p < .05$, and a significant interaction between Block and Type of list, $F(3114) = 5.40, p < .01$, showing an increase in accuracy across blocks for the repeated list (block 1 = 91.2%, SD = 9.4; block 2 = 94.1%, SD = 7.2; block 3 = 96.4, SD = 5.7; block 4 = 97.6, SD = 2.6) but not for the new lists (block 1 = 92.4%, SD = 8.8; block 2 = 91.3%, SD = 10.3; block 3 = 93.1, SD = 7.3; block 4 = 92.0, SD = 7.6) (see Fig. 2). The linear trend test was significant for the repeated list, $F(1,38) = 23.17, p < .001$, but not for the new lists, $F(1,38) < 1$. There was also a significant effect of Type of list, $F(1,38) = 13.59, p < .001$, with greater accuracy for repeated (94.8%, SD = 7.1) than for new lists (92.2%, SD = 8.6). Finally, the three-way interaction between Natural perspective, Type of list, and Block almost reached significance, $F(3,114) = 2.56, p = .058$. However, the analysis of this interaction revealed no difference between trunk-centred and decentred participants in generalization of learning as the linear trend tests revealed significant increase in accuracy across blocks for the repeated list but not for the new lists for the two groups (repeated list: $F(1,38) = 11.79, p < .01$, for trunk-centred, $F(1,38) = 11.38, p < .01$, for decentred; new lists: $F(1,38) < 1$, for both trunk-centred and decentred).

The number of blocks required to identify the participants' natural perspective individually varied. This might indicate either that the adopted perspective is more or less natural or that the natural perspective emerges faster for some participants over others. This variability was taken into account in order to further investigate the influence of natural perspective on learning and generalization. More specifically, the participants who required only one block of ambiguous symbols (N = 24) were compared with the participants who required

## Experiment 1

<figure><img src="image_33.png" /><figcaption>Fig. 2. Participants' accuracy during Experiment 1 as a function of Type of List (repeated, new) and Block. Error bars represent the standard errors of the means.</figcaption></figure>
several blocks (i.e., two or three blocks; N = 16). An ANOVA was conducted with Number of blocks (one, several) as a between-participant factor and with Type of list (repeated, new) and Blocks (block 1, block 2, block 3, block 4) as within-participant factors. The results revealed a significant effect of Number of blocks, $F(1,38) = 6.73, p < .05$, with greater accuracy for participants with one block (95.0%, SD = 3.3) than for participants with several blocks (91.2%, SD = 6.0). They also revealed a significant interaction between this factor and Blocks, $F(3114) = 4.03, p < .01$, showing a greater increase in accuracy for participants with several blocks (from 87.5% in block 1 to 91.9% in block 4) than for participants with one block (from 94.6% in block 1 to 95.1% in block 4). However, there was no significant interaction between Number of blocks and Type of list, $F(1,38) < 1, n_s$, and no significant interaction between Number of blocks, Type of list, and Blocks, $F(3114) = 2.00, p = .118$. Finally, linear trend tests indicated a significant increase in accuracy across blocks for the repeated list but not for the new lists. This was the case both for participants with one block ($F(1,23) = 14.07, p < .01$ for repeated; $F(1,23) = 2.29, p = .144$ for new) and for participants with several blocks ($F(1,15) = 15.060, p < .01$ for repeated; $F(1,15) = 2.97, p = .106$ for new).

## 2.3. Discussion

The results of Experiment 1 showed that tactile symbol recognition improved with training. However, performance improvement was restricted to those symbols that were repeated throughout the experiment, and did not occur for the new symbols that were introduced. The participants may thus have used stimulus-specific strategies of learning rather than extracting the perceptual rules that enable generalization of learning. In this experiment, the use of only four different tactile symbols in each block may have biased the participants toward adopting stimulus-specific strategies. For instance, it is possible to identify local (e.g., the presence of a curve for only one symbol in a list) or temporal (e.g., the duration of one symbol is shorter than the others) cues to recognize each symbol. With only four symbols the cost of maintaining such specific cues in short-term memory is not too important. However, the counterpart of such stimulus-specific strategies is that the cues specific to the learned symbols are no longer relevant when new symbols are introduced. As stimulus variability during learning facilitates generalization (Hussain et al., 2012), Experiment 2 therefore investigated whether learning a greater number of stimuli in each block encourages the participants to adopt generalized learning strategies.

Experiment 1 was also designed to evaluate whether tactile symbol recognition depends on the participants' natural preference for adopting self-centred or decentred perspectives. The results did not reveal any differences in recognition performance between the participants naturally adopting trunk-centred and those adopting decentred perspectives. This was true even when the variability in the emergence of the natural perspective was taken into account. Thus, both groups have similar results when participants were asked to recognize unambiguous symbols, contrary to the results of a previous study, in which decentred participants were better at recognizing ambiguous symbols (b, d, p, and q) from an unnatural perspective than self-centred participants (Arnold et al., 2016). The superiority of decentred observers in the recognition of ambiguous symbols may consequently be specific to assigning spatial coordinates (left, right, top, and bottom) to tactile stimulation and not to shape processing *per se*. This specificity is in line with the dissociation between processing object identity and processing object orientation, which has been suggested to rely on the ventral and dorsal visual streams respectively (Valyear, Culham, Sharif, Westwood, & Goodale, 2006).

## 3. Experiment 2

In Experiment 1, tactile symbol recognition only improved across blocks for repeated symbols but not for new symbols, revealing stimulus-specific rather than generalized learning. This second experiment investigated whether the learning strategy involved depends on the quantity of information to-be-learned and whether learning a greater number of symbols in each block favours a generalized learning. There were 6 symbols to learn in each list, with the same alternation of repeated and new symbols as in Experiment 1. If a more generalized learning is involved here, a symbol recognition improvement for both repeated and new symbols should be observed. As the recognition of unambiguous symbols did not depend on the participants' natural perspective in Experiment 1, and as naturally trunk-centred participants are more frequent than others (Arnold et al., 2016), only naturally trunk-centred participants were tested in this experiment. The symbols were always presented according to the participants' natural trunk-centred perspective.

### 3.1. Material and methods

#### 3.1.1. Participants

Thirty-two participants (27 females and 5 males; mean age = 22.9 years, range = 18–40 years; eight participants were left-handed and 24 participants were right-handed) that all had a natural trunk-centred perspective completed the experiment. All the participants were naive to the purpose of the experiment. None had participated in the first experiment. Participants provided their informed consent and received payment for their participation. The experiment took approximately one hour to complete and was performed in accordance with the ethical standards laid down in the Declaration of Helsinki (1991).

#### 3.1.2. Apparatus

The apparatus was exactly the same as in Experiment 1.

#### 3.1.3. Stimuli

The 20 symbols used in Experiment 1 and four additional symbols (the uppercase letters H, S, T, and U) were presented on the participant's stomach. The way the letters were traced was the same as in Experiment 1. The mean duration of symbols was 2188 ms (SD = 586). Four lists of six symbols were created with similar mean durations (List1: JKPZ36, 2142 ms; List2: FHLRT4, 2075 ms; List3: BCDQ25, 2392 ms; List4: EGNSU1, 2142 ms). Each of these lists was composed of four or five letters and one or two numbers.

#### 3.1.4. Procedure

The procedure was similar to the one used in Experiment 1. In the first part of the experiment, in order to decrease the number of blocks necessary to identify their natural perspective, participants were explicitly instructed to adopt the easiest perspective for them rather than the most ‘logical’ perspective. To identify each participant’s natural perspective, only one block was required for 25 participants, two blocks for two participants, and three blocks for five participants.

Once the participants’ natural perspective was identified, they completed six blocks for the recognition task. One of the lists (repeated list; labelled A) was presented three times. The other three lists (new lists; labelled B through D) were presented once each. Therefore there were three blocks with the repeated list and three blocks with the new lists, which were alternated as follows: ABACAD for half of the participants and BACADA for the other half. Each of the four lists served as the repeated list for a fourth of the participants (N = 8). In addition, each list was presented the same number of times across participants as lists B, C, and D. Each block consisted of 60 trials (10 presentations of each symbol). At the beginning of each block, the participants were given six practice trials (one trial for each symbol) with a feedback
<figure><img src="image_34.png" /><figcaption>Fig. 3. Participants' accuracy during Experiment 2 as a function of Type of list (repeated, new) and Block. (a) When all the lists were included, accuracy increased across blocks for the repeated list only. (b) When excluding the participants who had List 2 as the repeated list, accuracy increased across blocks for both the repeated and new lists. (c) Details of the results for each list as the repeated list. Accuracy increased across blocks for the new lists except when List 2 was the repeated list. Error bars represent the standard errors of the means.</figcaption></figure>

indicating the correct response at the end of each trial.

## 3.2. Results

Trials in which the participants failed to make a response before the trial was terminated (2.00% of trials) were not included in the data analyses. An ANOVA was conducted with Type of list (repeated, new) and Block (block 1, block 2, block 3) as within-participant factors. The results showed a significant effect of Block, $F(2,62) = 8.27, p < .001$, and, contrary to Experiment 1, no significant interaction between Block and Type of list, $F(2,62) = 1.51, p = .230$. This reveals a global increase in accuracy across blocks (block 1 = 86.5%, SD = 10.8; block 2 = 88.8%, SD = 11.0; block 3 = 90.5, SD = 10.1) (see Fig. 3a). The increase in accuracy was observed for both repeated (block 1 = 88.5%,

SD = 8.6; block 2 = 92.5%, SD = 7.5; block 3 = 93.7, SD = 6.0) and new lists (block 1 = 84.5%, SD = 12.5; block 2 = 85.0%, SD = 12.8; block 3 = 87.2, SD = 12.3). The linear trend test was however significant for the repeated list, $F(1,31) = 18.12, p < .001$, but not for the new lists, $F(1,31) = 1.97, p = .171$. There was also a significant effect of the Type of list, $F(1,31) = 20.40, p < .001$, with greater accuracy for repeated (91.5%, SD = 7.7) than for new lists (85.6%, SD = 12.4).

The absence of interaction between Type of List and Block suggests a global increase in accuracy across blocks but, nevertheless, there was no significant linear increase for the new lists. Thus, to try and account for the results of Experiment 2, the effect of each of the lists was investigated, to see whether the generalization of learning depends on the learned symbols. There was an increase in accuracy across blocks with the new lists when all the lists were repeated except for List 2 (see
<table><caption>**Table 1**
Number of horizontal, vertical, oblique, and curved features for the different lists in Experiment 1 and Experiment 2.</caption><thead><tr><th></th><th></th><th>Horizontal</th><th>Vertical</th><th>Oblique</th><th>Curve</th><th>Total</th></tr></thead><tbody><tr><td rowspan="7">Experiment 1</td><td>List 1</td><td>3</td><td>2</td><td>2</td><td>3</td><td>10</td></tr><tr><td>P Q Z 5</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>List 2</td><td>2</td><td>2</td><td>2</td><td>4</td><td>10</td></tr><tr><td>J K 3 6</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>List 3</td><td>4</td><td>5</td><td>1</td><td>1</td><td>11</td></tr><tr><td>F L R 4</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>List 4</td><td>1</td><td>2</td><td>1</td><td>5</td><td>9</td></tr><tr><td rowspan="4">Experiment 2</td><td>B C D 2</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>List 5</td><td>4</td><td>4</td><td>3</td><td>1</td><td>12</td></tr><tr><td>E G N 1</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>List 1</td><td>4</td><td>3</td><td>3</td><td>5</td><td>15</td></tr><tr><td rowspan="7">Experiment 1</td><td>J K P Z 3 6</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>List 2</td><td>6</td><td>8</td><td>1</td><td>1</td><td>16</td></tr><tr><td>F H L R T 4</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>List 3</td><td>2</td><td>3</td><td>2</td><td>7</td><td>14</td></tr><tr><td>B C D Q 2 5</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>List 4</td><td>4</td><td>4</td><td>3</td><td>4</td><td>15</td></tr><tr><td>E G N S U 1</td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

**Fig. 3c).** The difference in accuracy between the repeated and the new lists was also the strongest when List 2 was repeated (13.4 points of percentage; 4.4, 0.2, and 5.8 points of percentage for the List 1, List 3, and List 4, respectively), showing strong specificity to the stimulus learned in this list. It is interesting to note that List 2 differed from the others by the variability in its letter features. Table 1 shows that the symbols composing List 2 were essentially made of vertical and horizontal features and that there were very few oblique and curved features. On the contrary, the different types of features were more equally distributed in List 1, List 3, and List 4. Thus, feature variability may also have played a role in the generalization of learning, with exposition to a small set of features (List 2) potentially limiting one's ability to recognize new symbols.

Accordingly, an ANOVA was conducted, this time without those participants who had List 2 as the repeated list. In this case, the interaction between Type of List and Block was not significant, F (2,62) = 1.34, p = .271, and the linear trend test was significant for both the repeated, F(1,31) = 15.69, p < .001, and the new lists, F (1,31) = 5.93, p < .05, showing a clear pattern of generalization (see Fig. 3b). Note that this was not the case when the participants who had the List 1, List 3, or List 4 as the repeated list were excluded from the analysis. In order to evaluate the role of feature variability in Experiment 1, the distribution of the different types of features in the different lists was also analysed. Table 1 shows that the different types of features were relatively well distributed in List 1, List 2, and List 5 whereas List 3 was essentially made of vertical and horizontal features and List 4 was essentially made of curved features. Thus an ANOVA was conducted with only the participants who had List 1, List 2, and List 5 as the repeated list. The results revealed a significant interaction between Type of List and Block, F(3,69) = 5.98, p < .01, and the linear trend test was significant only for the repeated list, F(1,23) = 18.96, p < .001, not for the new lists, F(1,23) = 1.75, p = .199, showing no pattern of generalization. In summary, the results of Experiment 2 indicate that participants implemented general learning strategies except when feature variability of the repeated list was relatively low. However, in Experiment 1, with a few learned symbols, stimulus-specific strategies were involved, independently of feature variability.

### 3.3. Discussion

Compared to the results of Experiment 1, the results of Experiment 2 are overall in favour of generalized learning strategies and they are compatible with the effect of stimulus variability on generalization (Hussain et al., 2012). When the number of symbols to learn is

increased (from four to six symbols in each list), stimulus-specific strategies may be too costly. For instance, it may be difficult to keep a specific spatio-temporal cue for each symbol to learn in working memory. As a consequence, participants try to extract general perceptual rules, which favour the subsequent recognition of unlearned stimuli. An alternative explanation would be that task difficulty influenced the degree of generalization. Indeed, the global recognition performance was significantly higher in Experiment 1 (mean accuracy = 93.5%, SD = 4.8; mean RT = 2420 ms, SD = 360), with four symbols to recognize in each block, than in Experiment 2 (mean accuracy = 88.6%, SD = 8.4; mean RT = 2743 ms, SD = 287), with six symbols, for both accuracy, t(70) = 3.14, p < .01 and RTs, t (70) = 4.10, p < .001. However, contrasting results have been reported for the effect of task difficulty, with some studies showing specificity for difficult tasks (Jeter et al., 2009; Wang et al., 2013) but other showing generalization (Liu & Weinshall, 2000).

The results of Experiment 2 also revealed that generalization depends on the symbols' feature variability. For one list of symbols among the four different lists, perceptual learning remained specific to the learned symbols. Compared to the other lists, this list consisted of symbols with a low variability, essentially made of vertical and horizontal strokes, with few oblique or curved features. However, the same analysis of feature variability conducted on the results of Experiment 1 did not show any influence of feature variability: stimulus-specific learning was observed for both homogeneous and heterogeneous lists of symbols. Overall, the results of Experiment 1 and Experiment 2 therefore show that generalization to new symbols depends both on the size of the stimulus set that is learned and on feature variability. When fewer symbols are learned, stimulus-specific strategies are involved, independently of feature variability. When the stimulus set is increased, generalized strategies are involved but feature variability is necessary to make these strategies efficient. This effect of variability during training is consistent with previous studies reporting greater generalization after learning objects from different exemplars (Baeck, Maes, Van Meel, & Op de Beeck, 2016).

Finally, one possibility is that the type of learned features also plays a role. It may be more difficult to generalize from horizontal and vertical to oblique and curved features than the other way round. This effect of symbol feature can be interpreted with the feature analysis theory of visual perception (Treisman & Gormican, 1988). According to this theory, basic visual features such as straight lines are coded in the visual brain as standards and other features such as oblique and curved lines are coded as a deviation from the standard. As a consequence, brain activation is more important when perceiving deviant than standard features. A possibility here is that tactile recognition of visual symbols involved similar low-level processes as in visual recognition and that learning heterogeneous symbols, made of horizontal, vertical, oblique, and curvature features, activated broader brain areas than learning homogeneous symbols. According to the RHT, this broader brain activation has subsequently facilitated learning generalization.

### 4. Experiment 3

Experiment 3 was designed to investigate generalization to changes in orientation. Tactile symbols were learned upright and then they had to be recognized rotated 90° to the left or to the right, or 180° (upside-down). In order to be able to directly compare the generalization to new stimuli and to new orientations, the same learning protocol as in Experiments 1 and 2 was used, with an alternation of repeated (upright) and new (rotated) orientations. If learning generalizes to changes in orientation, that is, if recognizing a learned symbol in a new orientation has no cost, an improvement in recognition performance for both the repeated and new orientations should be observed. On the contrary, if learning is specific to the learned orientation, an improvement in performance only for the repeated orientation should be observed instead. As in Experiment 2, only naturally trunk-centred participants
completed this experiment.

**4.1. Material and methods**

**4.1.1. Participants**

Twenty-four participants (16 females and 8 males; mean age = 23.1 years, range = 19–34 years; one participant was left-handed and 23 participants were right-handed) that all had a natural perspective centred on their trunk completed the experiment. All the participants were naive to the purpose of the experiment. None had participated in Experiment 1 or Experiment 2. Participants provided their informed consent and received payment for their participation. The experiment took approximately one hour to complete and was performed in accordance with the ethical standards laid down in the Declaration of Helsinki (1991).

**4.1.2. Apparatus**

The apparatus was the same as in Experiments 1 and 2.

**4.1.3. Stimuli**

The same four lists of six symbols as presented in Experiment 2 were used. For each symbol, one vibrotactile sequence was created for each orientation (upright, left-rotated, right-rotated, upside-down). The left-rotated orientation corresponded to a 90°-rotation of the symbol toward the participant's left-hand side. Therefore the top of the symbol was located on the left side of the trunk. In contrast, the right-rotated orientation corresponded to a 90°-rotation of the symbol toward the participant's right-hand side. Therefore the top of the symbol was located on the right side of the trunk. The upside-down condition corresponded to a 180°-rotation of the symbol, resulting in the top of the symbol being located on the bottom of the trunk.

**4.1.4. Procedure**

The procedure was similar to the one of Experiment 2. To identify each participant's natural perspective, only one block was required for 22 participants, two blocks for one participant, and three blocks for one participant.

After the identification of their natural perspective, the participants completed six blocks for the recognition task. For each participant, only one list was presented, with the same number of participants (N = 6) for each list. In each block, the symbols were presented with one out of the four orientations. The upright orientation (repeated, labelled A) was presented three times. The other three orientations (unrepeated, labelled B through D) were presented once each. Thus there were three blocks with the symbols presented upright and three blocks with the symbols rotated, that were alternated as follows: ABACAD for half of the participants and BACADA for the other half. Each rotated condition (90° left, 90° right, 180°) was presented the same number of times across participants as blocks B, C, and D. Each block consisted of 60 trials (10 presentations of each symbol). At the beginning of each block, the participants were given six practice trials (one trial for each symbol) with feedback indicating the correct response at the end of each trial.

**4.2. Results**

Trials in which the participants failed to make a response before the trial was terminated (0.26% of trials) were not included in the data analyses. An ANOVA was conducted with Type of orientation (repeated, new) and Block (block 1, block 2, block 3) as within-participant factors. The results showed a significant effect of Block, $F(2,46) = 9.67$, $p < .001$, and no significant interaction between Block and Type of orientation, $F(2,46) < 1$ (see Fig. 4a). The increase in accuracy was observed for both repeated (block 1 = 84.4%, SD = 13.4; block 2 = 88.2%, SD = 13.2; block 3 = 90.4%, SD = 10.7) and new (block 1 = 77.9%%, SD = 16.5; block 2 = 81.8%, SD = 14.6; block 3 = 82.8%, SD = 16.1) orientations. The linear trend test was

significant for both the repeated orientation, $F(1,23) = 12.48, p < .01$, and the new orientations, $F(1323) = 5.88, p < .05$. There was also a significant effect of Type of orientation, $F(1,23) = 25.65, p < .001$, with greater accuracy for repeated (87.7%, SD = 12.6) than for new orientations (80.8%, SD = 15.7).

In order to investigate a possible effect of the symbols' orientation on learning generalization, the accuracy was also computed as a function of the angle of rotation (90° left, 90° right, and 180°) from the upright orientation (see Fig. 4b). The results first revealed that global accuracy was similar for the three angles of rotation (81.3%, SD = 16.5, for the 90° left; 81.3%, SD = 15.0, for the 90° right; 80.0%, SD = 16.2, for the 180°). It should be underlined that this experiment was designed to investigate the participants' ability to recognize learned symbols with a new orientation in each session. As a consequence, the evaluation of performance improvement across blocks as a function of each symbol's orientation is based on between-participant comparisons with a small sample size (n = 6) and can be only descriptive. Fig. 4b illustrates that recognition performance improved across blocks for all the symbols' orientations except for the 90°-left orientation for which performance increased in the second block and then decreased in the third block.

**4.3. Discussion**

The results of Experiment 3 revealed generalization to previously learned symbols presented in a new orientation and they are compatible with previously reported results on generalization to new orientations. Even though perceptual learning is often highly specific to orientation (e.g., Hussain, Sekuler, & Bennett, 2009), generalization to new orientations has been reported to be easier than generalization to new set of stimuli (Baeck, Windey, & Op de Beeck, 2012). The sequential presentation of the tactile symbols in our study may have also facilitated generalization to orientation. Indeed, the spatio-temporal characteristics of each symbol may easily be recognized independently of the way the symbol is oriented on the body surface. For instance, the sequence of one long stroke followed by two short strokes for the letter K is easily perceived in each orientation. That is to say, the static presentation of a symbol might be seen to facilitate generalization to the same extent.

The results of Experiment 3 are also compatible with previous studies on visual recognition of rotated letters and digits, which showed that recognition depends on angular rotation as a function of the task. When the task consists in deciding whether the symbol is normal or mirror-reversed, there is an effect of angular rotation on recognition performance (Cooper & Shepard, 1973). On the contrary, when the task consists in identifying the alphanumerical symbol, there is no effect of angular rotation on recognition performance, showing that alphanumerical symbols are recognized through the extraction of critical features invariant to the symbol's orientation (Corballis, Zbrodoff, Shetzer, & Butler, 1978; Eley, 1982). In the present study, the absence of difference in global accuracy between the different angles of rotation from the upright orientation may be explained by the use of an identification task rather than a verification task.

Regarding the rotation of the symbols on the stomach, it is interesting to note that symbol recognition may have been facilitated for the 180°-rotated orientation because recognizing a 180°-rotated symbol presented on the stomach with a trunk-centred perspective is close to adopting a head-centred perspective. Indeed, adopting a head-centred perspective results in perceiving the symbol upright but mirror-reversed (except for the symmetrical symbols). As the task was not to decide whether the symbol was normal or mirror-reversed, and as there was no pair of symmetrical symbols in the lists of symbols presented to the participants, adopting a head-centred perspective may have been an easier perceptual strategy than trying to recognize rotated symbols.
<figure><img src="image_35.png" /><figcaption>Fig. 4. Participants' accuracy in Experiment 3 as a function of (a) Type of orientation (repeated, new) and Block and (b) Angle of rotation and Block. Error bars represent the standard errors of the means.</figcaption></figure>

**5. General discussion**

The present study investigated whether learning to recognize the tactile conversion of visual symbols generalizes to new stimuli and new orientations. The results show that the generalization to new stimuli depends both on the size of the stimulus set and on feature variability. Stimulus-specific learning strategies are involved when learning a limited number of stimuli whereas generalized strategies are involved when increasing the number of learned stimuli. However, in the latter case, feature variability is necessary to achieve generalization. When the learned symbols are homogeneous and composed essentially of vertical and horizontal strokes, learning does not generalize to new symbols. Our results also reveal that tactile perceptual learning generalizes to new orientations.

The present study reveals that generalization of learning depends on both the quantity and diversity of information presented during learning. Nonetheless further studies would be necessary to identify the specific factors that facilitate generalization. The most likely hypothesis is that increasing the set of learned stimuli prevents observers from involving stimulus-specific learning strategies. As a consequence, observers are forced to identify the general perceptual rules that can be applied to new stimuli. The better generalization performance reached in Experiment 2 than in Experiment 1 can also result from a difference in task difficulty. Note that if this were to be the case, this would go against the RHT of perceptual learning (Ahissar & Hochstein, 2004), which posits generalization for easy tasks and specificity for difficult tasks. In our experiments, the use of short blocks of trials and the alternation of learned and new symbols might also have contributed to generalization by preventing sensory adaptation. However, such use of short blocks and alternation were not sufficient to induce generalization in Experiment 1. Therefore it is unlikely that these characteristics of the learning protocol were solely responsible for the generalization of learning observed in Experiment 2.

The generalization to new tactile stimuli found in our study and the generalization that has been previously reported when using visual-to-auditory conversion devices (Brown & Proulx, 2013; Kim & Zatorre, 2008) can be interpreted in the light of the multisensory RHT proposed by Proulx et al. (2014). According to the classic RHT, generalization of learning benefits from broad brain activations during learning. According to the multisensory RHT, multisensory tasks produce broader brain activation than unisensory tasks in two possible ways. Cross-modal interactions might occur through direct connections between low-level unisensory areas (i.e., visual, auditory, and tactile primary cortex) or through higher-level multisensory areas where the objects are represented independently of the input sensory modality. Therefore, in the case of using a visual-to-tactile or visual-to-auditory substitution

device, direct connections can occur between tactile and visual, or between auditory and visual primary cortices. The other possibility is that the processing of tactile or auditory stimuli first progresses from tactile or auditory primary cortices to high-level object processing areas such as the lateral occipital complex (Amedi et al., 2007) and then cascade down back to visual areas. According to these two possibilities, the multisensory nature of the task produces broad brain activation and may facilitate the generalization to new stimuli.

A question remaining open is whether generalization was indeed facilitated by the multisensory nature of the task, as it corresponds to tactile recognition of visual symbols or, instead, if generalization to new stimuli is a characteristic of tactile perceptual learning itself. This is a difficult alternative to solve given that the generalization to new sets of stimuli in tactile perceptual learning has been the subject of comparatively less investigation than in visual perceptual learning. In one previous study, learning to recognize tactile patterns was found to transfer to new stimuli (Epstein, Hugues, Schneider, & Bach-y-Rita, 1989). However, the task at hand consisted in matching the tactile patterns with their visual analogue. In this case, a similar multisensory processing cannot be excluded either. In further studies, using a direct comparison of a purely tactile task with a visuo-tactile task will allow to evaluate the specific role of multisensory processing in tactile perceptual learning.

Regarding generalization to new orientations, the results obtained in our study are compatible with previous results on visual recognition of rotated alphanumerical symbols (Corballis et al., 1978; Eley, 1982), showing that alphanumerical symbols are learned through the extraction of critical features invariant to the symbol's orientation. The sequential presentation of symbols may also have reinforced orientation-invariant recognition, as spatiotemporal features rather than purely spatial ones are more likely to have been extracted here. However, the hypothesis that people are able to recognize previously learned symbols independently of their orientation seems incompatible with the influence of expertise on orientation effects. In vision, it is well known that inverting faces (i.e., 180°-rotation) dramatically decreases face recognition (Yin, 1969). This inversion effect is partly explained by the strong habit to perceive faces upright rather than rotated. An inversion effect has also been reported for unnatural objects (e.g., Greebles) that have received expertise (Gauthier, Williams, Tarr, & Tanaka, 1998). As faces, alphanumerical symbols are more frequently perceived upright than rotated. Moreover, in the tactile sensory modality, Behrmann and Ewell (2003) have shown that becoming an expert in tactile recognition induces an inversion effect. It is thus surprising that such inversion effects were not observed for the tactile recognition of visual symbols. One possibility for this discrepancy is that the duration of the present study was insufficient for participants to consolidate learning
equivalent to expertise. Another possibility is that the alternation of
blocks with upright and rotated orientations, rather than a complete
learning phase with upright symbols followed by test blocks with ro-
tated symbols, prevents the facilitation effect for the upright condition.
The fact that alphanumerical symbols are less complex items than faces
may also have limited the involvement of specific processes such as
configural processing (i.e., processing not only features, but also the
relations among features), which is often characteristic of expertise
(Gauthier & Tarr, 2002).

Finally, the orientation-independent perceptual learning found in
our study may reflect a distal attribution process, which is an important
characteristic of sensory substitution. Distal attribution corresponds to
the fact that, after training to use a visual-to-tactile device, the tactile
stimulation felt on the skin becomes directly attributed to external
objects, as it is the case for vision. Thus distal attribution is often
considered as a criterion of the involvement of visual-like processes
when using a sensory substitution device. One claim made by Bach-y-
Rita was that, as a consequence of distal attribution, the tactile matrix
of a visual-to-tactile device can be moved from one body surface to
another without loss of perceptual abilities in trained users (Bach-y-Rita
& Kercel, 2003). This claim has been confirmed by a study reporting a
transfer of learning from trained to untrained body surfaces in a tactile
recognition task of visual symbols (Arnold & Auvray, 2014). Distal at-
tribution processes may also contribute here to tactile recognition of
visual stimuli independently of the orientation of the tactile stimulation
on the body surface.

To conclude, the present study provides new insights into the un-
derstanding of tactile perceptual learning and brain plasticity. Tactile
perceptual learning appears to share some characteristics with visual
perceptual learning (influence of stimulus variability and task diffi-
culty, representation of standard features similar to visual features).
This similarity may reflect the involvement of supramodal areas in both
visual and tactile object recognition (Kupers & Ptito, 2011; Pascual-
Leone & Hamilton, 2001). Regarding sensory substitution, the simi-
larity in processes is compatible with the view that perception with a
sensory substitution device is vertically integrated, retaining char-
acteristics of the substituted (e.g., vision) and the substituting (e.g.,
audition or touch) sensory modalities (Arnold, Pesnot-Lerousseau, &
Auvray, in press; Deroy & Auvray, 2012; Deroy & Auvray, 2014). There
are also applied implications, in particular for the design of learning
protocols for sensory substitution, suggesting that short learning ses-
sions with high variability and diversity will facilitate generalization
and will allow for a more optimal use of the device in real-life condi-
tions. However, one remaining question is how to achieve complete
generalization, that is, how to rapidly gain optimal level of performance
for entirely new objects. One of the possible cues is obviously the
amount of learning. Daily use of the device would allow users to be-
come experts. Learning would also benefit from a highly diversified use
of the device, with perception of different categories of complex objects
and of different exemplars of the same category. Finally, an active ex-
ploration of the external world with the device, rather than a passive
reception of stimuli, would also reinforce perceptual learning via the
emergence of sensorimotor contingencies.

Acknowledgment

We thank Gaëtan Parseihan and Mathieu Dubois for their help in
designing the apparatus and Jess Hartcher-O’Brien and Ross Parry for
their careful reading and helpful comments on previous versions of this
article. This work was supported by the Labex SMART (ANR-11-LABX-
65), the Mission pour l’Interdisciplinarité (CNRS, Auton, Sublima Grant,
France), and the Fondation des Aveugles de Guerre. GA is funded by a
grant from the European Research Council (Wearhap, FP7, N°601165).

**Appendix A. Supplementary data**

Supplementary data associated with this article can be found, in the
online version, at http://dx.doi.org/10.1016/j.visres.2017.11.007.

**References**

Ahissar, M., & Hochstein, S. (2004). The reverse hierarchy theory of visual perceptual learning. *Trends in Cognitive Science*, *8*, 457–464.

Amedi, A., Stern, W. M., Camprodon, J. A., Bermphol, F., Merabet, L., Rotman, S., & Pascual-Leone, A. (2007). Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex. *Nature Neuroscience*, *10*, 687–689.

Arno, P., Vanlierde, A., Streel, E., Wanet Defalque, M. C., Sanabria-Bohorquez, S., & Veraart, C. (2001). Auditory substitution of vision: pattern recognition by the blind. *Applied Cognitive Psychology*, *15*, 509–519.

Arnold, G., Pesnot-Lerousseau, J. & Auvray, M. (in press). Individual differences in sensory substitution. *Multisensory Research*.

Arnold, G., & Auvray, M. (2014). Perceptual learning: Tactile letter recognition transfers across body surfaces. *Multisensory Research*, *27*, 71–90.

Arnold, G., Spence, C., & Auvray, M. (in press). A unity of the self or a multiplicity of locations? How the graphesthesia task sheds light on the role of spatial perspectives in bodily self-consciousness. *Consciousness & Cognition*.

Arnold, G., & Siéroff, E. (2012). Timing constraints of temporal view association in face recognition. *Vision Research*, *54*, 61–67.

Arnold, G., Spence, C., & Auvray, M. (2016). Taking someone else's spatial perspective: Natural stance or effortful decentring? *Cognition*, *148*, 27–33.

Auvray, M. (2004). *Immersion et perception spatiale: l’exemple des dispositifs de substitution sensorielle* [Immersion and spatial perception: the example of sensory substitution devices] (Unpublished doctoral dissertation). EHESS, Paris, France.

Auvray, M., Hanneton, S., & O'Regan, J. K. (2007). Learning to perceive with a visuo-auditory substitution system: Localization and object recognition with The Voice. *Perception*, *36*, 416–430.

Auvray, M., & Myin, E. (2009). Perception with compensatory devices: From sensory substitution to sensorimotor extension. *Cognitive Science*, *33*, 1036–1058.

Bach-y-Rita, P., & Kercel, S. W. (2003). Sensory substitution and the human-machine interface. *Trends in Cognitive Sciences*, *7*, 541–546.

Baeck, A., Maes, K., Van Meel, C., & Op de Beeck, H. P. (2016). The transfer of object learning after training with multiple exemplars. *Frontiers in Psychology*, *7*, 1386.

Baeck, A., Windey, I., & Op de Beeck, H. P. (2012). The transfer of object learning across exemplars and their orientation is related to perceptual similarity. *Vision Research*, *68*, 40–47.

Behrmann, M., & Ewell, C. (2003). Expertise in tactile pattern recognition. *Psychological Science*, *14*, 480–492.

Brown, D. J., & Proulx, M. J. (2013). Increased signal complexity improves the breadth of generalization in auditory perceptual learning. *Neural Plasticity*, *2013*, 879047.

Cooper, L. A., & Shepard, R. N. (1973). Chronometric studies of the rotation of mental images. In W. G. Chase (Ed.). *Visual information processing* (pp. 75–176). New York: Academic Press.

Corballis, M. C., Zbrodoff, N. J., Shetzer, L. I., & Butler, P. B. (1978). Decisions about identity and orientation of rotated letters and digits. *Memory and Cognition*, *6*, 98–107.

Deroy, O., & Auvray, M. (2012). Reading the world through the skin and ears: a new perspective on sensory substitution. *Frontiers in Psychology*, *3*, 457.

Deroy, O., & Auvray, M. (2014). A crossmodal perspective on sensory substitution. In S. Biggs, M. Matthen, & D. Stokes (Eds.). *Perception and its modalities* (pp. 327–349). Oxford: Oxford University Press.

Dutilg, G., Krypotos, A. M., & Wagenmakers, E. J. (2011). Task-related versus stimulus-specific practice: A diffusion model account. *Experimental Psychology*, *58*, 434–442.

Eley, M. G. (1982). Identifying rotated letter-like symbols. *Memory and Cognition*, *10*, 25–32.

Epstein, W., Hughes, B., Schneider, S. L., & Bach-y-Rita, P. (1989). Perceptual learning of spatiotemporal events: Evidence from an unfamiliar modality. *Journal of Experimental Psychology: Human Perception and Performance*, *15*, 28–44.

Fahle, M. (2005). Perceptual learning: Specificity versus generalization. *Current Opinion in Neurobiology*, *15*, 154–160.

Fahle, M. E., & Poggio, T. E. (2002). *Perceptual learning*. Cambridge, MA, USA: MIT Press.

Furmanski, C. S., & Engel, S. A. (2000). Perceptual learning in object recognition: Object specificity and size invariance. *Vision Research*, *40*, 473–484.

Gauthier, I., & Tarr, M. J. (2002). Unraveling mechanisms for expert object recognition: Bridging brain activity and behavior. *Journal of Experimental Psychology: Human Perception and Performance*, *28*, 431–446.

Gauthier, I., Williams, P., Tarr, M. J., & Tanaka, J. (1998). Training “Greeble” experts: A framework for studying expert object recognition processes. *Vision Research*, *38*, 2401–2428.

Haggard, P., Taylor-Clarke, M., & Kennett, S. (2003). Tactile perception, cortical representation and the bodily self. *Current Biology*, *13*, R170–R173.

Harrar, V., Spence, C., & Makin, T. R. (2014). Topographic generalization of tactile perceptual learning. *Journal of Experimental Psychology: Human Perception and Performance*, *40*, 15–23.

Harris, H., Gliksberg, M., & Sagi, D. (2012). Generalized perceptual learning in the absence of sensory adaptation. *Current Biology*, *22*, 1813–1817.

Hartcher-O'Brien, J., & Auvray, M. (2014). The process of distal attribution illuminated through studies of sensory substitution. *Multisensory Research*, *27*, 421–441.

Hussain, Z., Bennett, P. J., & Sekuler, A. B. (2012). Versatile perceptual learning of
textures after variable exposures. *Vision Research*, **61**, 89-94.

Hussain, Z., Sekuler, A. B., & Bennett, P. J. (2009). Perceptual learning modifies inversion effects for faces and textures. *Vision Research*, **49**, 2273-2284.

Jeter, P. E., Dosher, B. A., Liu, S. H., & Lu, Z. L. (2010). Specificity of perceptual learning increases with increased training. *Vision Research*, **50**, 1928-1940.

Jeter, P. E., Dosher, B. A., Petrov, A., & Lu, Z. L. (2009). Task precision at transfer determines specificity of perceptual learning. *Journal of Vision*, **9**, 1-13.

Kaczmarek, K. A., & Haase, S. J. (2003). Pattern identification and perceived stimulus quality as a function of stimulation waveform on a fingertip-scanned electromotile display. *IEEE Transactions on Neural Systems and Rehabilitation Engineering*, **11**, 9-16.

Kim, J. K., & Zatorre, R. J. (2008). Generalized learning of visual-to-auditory substitution in sighted individuals. *Brain Research*, **1242**, 263-275.

Kim, J. K., & Zatorre, R. J. (2010). Can you hear shapes you touch? *Experimental Brain Research*, **202**, 747-754.

Kupers, R., & Ptito, M. (2011). Insights from darkness: What the study of blindness has taught us about brain structure and function. *Progress in Brain Research*, **192**, 17-31.

Levy-Tzedek, S., Hanassy, S., Abboud, S., Maidenbaum, S., & Amedi, A. (2012). Fast, accurate reaching movements with a visual-to-auditory sensory substitution device. *Restorative Neurology and Neuroscience*, **30**, 313-323.

Levy-Tzedek, S., Novick, I., Arbel, R., Abboud, S., Maidenbaum, S., Vaadia, E., & Amedi, A. (2012). Cross-sensory transfer of sensory-motor information: visuomotor learning affects performance on an audiomotor task, using sensory-substitution. *Scientific Reports*, **2**, 949.

Liu, Z., & Weinshall, D. (2000). Mechanisms of generalization in perceptual learning. *Vision Research*, **40**, 97-109.

Nagarajan, S. S., Blake, D. T., Wright, B. A., Byl, N., & Merzenich, M. M. (1998). Practice-related improvements in somatosensory interval discrimination are temporally specific but generalize across skin location, hemisphere, and modality. *Journal of Neuroscience*, **18**, 1559-1570.

Newell, F. N., Ernst, M. O., Tjan, B. S., & Bülthoff, H. H. (2001). Viewpoint dependence in visual and haptic object recognition. *Psychological Science*, **12**, 37-42.

Pascual-Leone, A., & Hamilton, R. (2001). The metamodal organization of the brain. *Progress in Brain Research*, **134**, 427-445.

Pollok, B., Schnitzler, I., Mierdorf, T., Stoerig, P., & Schnitzler, A. (2005). Image-to-sound conversion: Experience-induced plasticity in auditory cortex of blindfolded adults. *Experimental Brain Research*, **167**, 287-291.

Proulx, M. J., Brown, D. J., Pasqualotto, A., & Meijer, P. (2014). Multisensory perceptual learning and sensory substitution. *Neuroscience & Biobehavioral Reviews*, **41**, 16-25.

Renier, L., Collignon, O., Poirier, C., Tranduy, D., Vanlierde, A., Bol, A., ... De Volder, A. G. (2005). Cross-modal activation of visual cortex during depth perception using auditory substitution of vision. *Neurolmage*, **26**, 573-580.

Sampaio, E., Maris, S., & Bach-y-Rita, P. (2001). Brain plasticity: 'Visual' acuity of blind persons via the tongue. *Brain Research*, **908**, 204-207.

Sathian, K., & Zangaladze, A. (1997). Tactile learning is task specific but transfers between fingers. *Perception and Psychophysics*, **59**, 119-128.

Sigman, M., & Gilbert, C. D. (2000). Learning to find a shape. *Nature Neuroscience*, **3**, 264-269.

Striem-Amit, E., Guendelman, M., & Amedi, A. (2012). 'Visual' acuity of the congenitally blind using visual-to-auditory sensory substitution. *PLoS ONE*, **7**, e33136.

Treisman, A., & Gormican, S. (1988). Feature analysis in early vision: Evidence from search asymmetries. *Psychological Review*, **95**, 15-48.

Valyear, K. F., Culham, J. C., Sharif, N., Westwood, D., & Goodale, M. A. (2006). A double dissociation between sensitivity to changes in object identity and object orientation in the ventral and dorsal visual streams: a human fMRI study. *Neuropsychologia*, **44**, 218-228.

Wang, X., Zhou, Y., & Liu, Z. (2013). Transfer in motion perceptual learning depends on the difficulty of the training task. *Journal of Vision*, **13**, 1-9.

World Medical Association (1991). Declaration of Helsinki. *Law, Medicine & Health Care: A Publication of the American Society of Law & Medicine*, **19**, 264-265.

Yanagida, Y., Kakita, M., Lindeman, R. W., Kume, Y., & Tetsutani, N. (2004). Vibrotactile letter reading using a low-resolution tactor array. *Proceedings of the 12th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems*, 400-406.

Yi, D. J., Olson, I. R., & Chun, M. M. (2006). Shape-specific perceptual learning in a figure-ground segregation task. *Vision Research*, **46**, 914-924.

Yin, R. K. (1969). Looking at upside-down faces. *Journal of Experimental Psychology*, **81**, 141-145.
# Differentiated Audio-Tactile Correspondences in Sighted and Blind Individuals

Ophelia Deroy

University of London

Irène Fasiello, Vincent Hayward,
and Malika Auvray

University Pierre et Marie Curie, Sorbonne Universités

The aim of the present study is to investigate whether the crossmodal correspondence robustly documented between auditory pitch and visual elevation has analogues in the audio-tactile domain. Across 4 experiments, the compatibility effects between intuitively congruent pairs of stimuli (i.e., outward tactile movement, going from the inside of the finger toward the fingertip and increasing pitch, or inward tactile movement and decreasing pitch) and incongruent pairs stimuli (i.e., the reverse associations) were measured. Two methods were compared to assess the behavioral effects of such a correspondence: One where participants have to respond to either the auditory or tactile stimulus presented simultaneously, while ignoring the other (speeded classification task), and the other where the auditory and tactile stimuli are presented sequentially and associated to different response buttons (implicit association test). No significant compatibility effect was observed under the speeded classification task. The implicit association test revealed a significant compatibility effect. This effect was similar in the conditions where the finger was placed vertically and horizontally. However, this implicit association between pitch and tactile movements was not observed in blind participants. These results have methodological implications for the explanation and testing of crossmodal correspondences, and the origin of the widely discussed association between pitch and vertical elevation.

**Keywords:** crossmodal correspondences, audition, touch, pitch, space

We commonly use vertical elevation to refer to auditory variations: Increases in perceived pitch are experienced and reported as “rising,” while decreases in pitch are considered to “descend” or to “fall.” This spatial mapping does not strictly depend on language (Dolscheid, Hunnius, Casasanto, & Majid, 2012; Parkinson, Kohler, Sievers, & Wheatley, 2012) and can affect experience and behavior in a consistent way, from the perception of sound location (Pratt, 1930; Roffler & Butler, 1968), speeded discrimination experiments (Ben-Artzi & Marks, 1995; Bernstein & Edelstein, 1971; Melara & O’Brien, 1987), and audio-visual interactions (Parise & Spence, 2008, 2009) to the orientation of attention and the perception of ambiguous visual movements (Maeda, Kanai, & Shimojo, 2004). In most cases, the spatial mapping of pitch has been assessed in reference to the location of visual targets or the direction of visual movement (see Occelli, Spence, & Zampini,

2009, for an exception and a review). In stimulus-response compatibility effects in which participants are shown to be faster at responding to a high-pitch sound with the upper key rather than the lower one (Rusconi, Kwan, Giordano, Umiltà, & Butterworth, 2006), vision might also play a role as participants can perceive or imagine the visual organization of the two response buttons.

In the study reported here, we investigated whether the spatial mapping of pitch was intimately tied to vision. Would the perception of a change of pitch interfere with the tactile direction of movement experienced at one’s fingertips, as it does with the visual direction of movement? If the tendency to match or map pitch and gestures seem to be grounded in musical practices, the learning of a similar correspondence between pitch and passive movement, if any, would seem less easy to explain. What’s more, given the many differences in auditory, tactile, and audio-tactile perception which are introduced by visual experience (see Heller & Gentaz, 2013; Hötting & Röder, 2009 for reviews), the existence of a correspondence between pitch and tactile movement should be assessed both in blind and sighted individuals.

Besides its intrinsic interest, evidencing a correspondence between pitch and tactile movement (Deroy & Spence, 2013; Spence, 2011) could help differentiate between the different hypotheses offered to explain the pitch-elevation correspondence documented through audio-visual paradigms. According to the spatial theory of magnitude, differences in pitch are represented as “high” or “low” in an amodal mental space, like differences in brightness or numerical quantities (Rusconi et al., 2006; see also Gevers, Reynvoet, & Fias, 2003; Cohen Kadosh, Lammertyn, & Izard, 2008, for discussion). If this is indeed the case, we would expect pitch to be commonly represented in a similar spatial way, independently of

This article was published Online First March 7, 2016.

Ophelia Deroy, School of Advanced Study, University of London; Irène Fasiello, Vincent Hayward, and Malika Auvray, Institut des Systèmes Intelligents et de Robotique, University Pierre et Marie Curie, Sorbonne Universités.

We thank Merle Fairhurst, Jess Hartcher-O’Brien and the reviewers for their helpful comments on previous versions of the manuscript. Thus research was supported by the Agence Nationale de la Recherche (ANR-11-JSH2-003-1) and by the Mission pour l’Interdisciplinarité (Defisens, Supplé-Sens grant) as well as the RTS-AHRC grant (AH-L007053/1).

Correspondence concerning this article should be addressed to Ophelia Deroy, Centre for the Study of the Senses, School of Advanced Study, University of London, Senate House, Malet Street, London WC1E 7HU, United Kingdom. E-mail: ophelia.deroy@sas.ac.uk
the modality used to evidence this representation. By contrast, if the mapping between pitch and movement is primarily learned through audio-visual associations, it may differ once tested in the tactile modality. Parise, Knorre, and Ernst (2014) have recently showed that the natural statistics of sounds were compatible with high-pitched sounds coming from higher in space, and low-pitched sounds from lower in space. It might therefore be the case that when we hear high-pitched sounds, we associate them to the higher sources of the sounds we look at, and vice versa for low-pitched sounds. In this case, vision would be necessary for spatial mapping to be learned, and we would not expect to find a similar mapping in congenitally blind individuals. More generally, the lack of a correspondence between changes in pitch and direction of tactile movements would reinforce the idea that vision plays a role in the acquisition of the correspondence, as posited by other hypotheses (e.g., Deroy, Crisinel, & Spence, 2013). According to the transitivity hypotheses, other regular associations between sounds and visual objects can explain the pitch-elevation correspondence. For instance, an increased distance in the vertical (or even horizontal) direction correlates with a decrease in visual size: The farther or the higher an object gets, the smaller it becomes. Another environmental regularity is that the smaller an object becomes, the higher the pitch it emits (Evans & Treisman, 2010; Fernández-Prieto, Navarra, & Pons, 2015). The transitivity of associative learning from higher in space to smaller in size and from smaller in size to higher in pitch could then explain how the correspondence between higher in space and higher in pitch is formed through regularities and transitive learning (Spence & Deroy, 2012). Crucially, the mediation of a change in size can only be provided by vision: A change in localization on one's body is not associated to a change in size, which seems to block the transition via "getting smaller." A specific prediction of the transitive hypothesis therefore is that the correspondence between movement and change in pitch should not exist in congenitally blind individuals.

The experiments conducted here were designed to test for the existence of an audio-tactile correspondence between pitch and tactile movements, similar to the well documented audio-visual correspondence between pitch and visual movements. Further experiments tested whether the correspondence could depend on bodily position and on visual experience. The broader goal was to understand the mapping between pitch and space.

Given the importance of attention in crossmodal congruency effects (see Spence & Deroy, 2013 for review), two different methods were used to investigate the existence of an audio-tactile correspondence between tactile movements felt on one's finger and changes in pitch. These two methods focus on different attentional processing levels, one that occurs at early stages and the other at later ones, during response selection. The first method, used in many studies (e.g., Evans & Treisman, 2010; Ludwig, Adachi, & Matsuzawa, 2011; see Marks, 2004, and Spence, 2011, for reviews), draws on a speeded classification paradigm. Participants are presented with two different stimuli in different sensory modalities, and are asked to respond to one type of stimulus (for instance, high or low visual targets) while ignoring the stimulus presented in another sensory modality (for instance, high- or low-pitched sounds). The prediction is that participants are faster to respond to the attended feature when the unattended feature is congruent with it. In other words, the prediction is that people find

it harder (i.e., they are slower) to classify a target stimulus presented in one sensory modality (e.g., audition) when the distractor stimulus presented in a task-irrelevant modality (e.g., touch) varies along a dimension that shares a relation of correspondence with the target dimension (an effect which has similarities with the cross-modal Stroop test, e.g., Cowan & Barron, 1987; MacLeod, 1991; Walker & Smith, 1984). A second method, which has more recently been tested, builds on the implicit association test (IAT), which is meant to measure participants' automatic associations. In this task, participants are required to rapidly categorize two kinds of target stimuli whose values are associated to two response buttons, in both a congruent and an incongruent way. For instance, in the first IAT (see Greenwald, McGhee, & Schwartz, 1998; see Glashouwer, Smulders, de Jong, Roefs, & Wiers, 2013; Teige-Mocigemba, Klauer, & Sherman, 2010, for reviews) targeting the implicit association between skin color and valence, participants would be asked to respond to either positive words or names typical of White people with the same response key (predicted to be a congruent combination) and then asked to respond either to positive words or to names typical of Black people on the same response key (predicted to be an incongruent combination). The difference in response time between the congruent and incongruent conditions reveals the strength of the implicit association. Variants of the IAT have been successfully adapted to nonsocial stimuli and used to test crossmodal correspondences between odors and touch (Dematte, Sanabria, & Spence, 2007), audition and touch (Occelli et al., 2009), taste and pitch (Crisinel & Spence, 2010), and audition and vision (Parise & Spence, 2012). The task always requires sorting four kinds of stimuli using only two response buttons, each of which refers to two of the four stimuli. For instance, for an IAT testing pitch and size, participants have to respond to high or low pitch, and large or small visual targets by pressing one of two keys, associated to either congruent (high-small, low-big) or incongruent (high-big, low-small) combinations. The prediction is that people find it easier (i.e., are overall faster) when the two stimuli that share a response are strongly associated than when they are weakly associated.

The two methods were tested in our study in order to determine whether they would give the same results for the tactile and auditory features that were selected. In the first experiment, participants had to respond to either tactile or auditory stimuli, while ignoring the simultaneous presentation of congruent or incongruent stimuli in the other sensory modality. In the second experiment, participants had to respond to either a tactile or an auditory feature presented one after the other in blocks and with congruent or incongruent responses that were associated to the same response key. In a third experiment, we tested whether the audio-tactile correspondence would depend on the hand position, and thereby if it took place within a hand-centered frame of reference. To do so, the participants completed the experiment with the arm placed in two directions: in the vertical and horizontal planes. Finally, in order to investigate the role of vision, a fourth experiment looked at the existence of audio-tactile crossmodal correspondences in early and late blind participants.

## Experiment 1: Speeded Classification Protocol

A first experiment looked for a crossmodal congruency effect between pitch and tactile movements by means of a speeded
classification paradigm. Participants had to categorize auditory stimuli while ignoring simultaneously presented tactile stimuli and vice versa.

## Method

**Participants.** Sixteen sighted participants (11 men and five women) took part in this experiment.<fnref n="1" /> Their ages ranged from 19 to 40 years (M = 29 years). All of the participants were naive to the purpose of the experiment and reported normal auditory and tactile perception. They received €6 in return for their participation. The experiment took approximately 45 min to complete. The four experiments reported in this article were performed in accordance with the ethical standards laid down in the 1991 Declaration of Helsinki.

**Apparatus and stimuli.** The tactile stimuli were delivered by a distributed tactile stimulator (Wang, & Hayward, 2010) from Tactile Lab, Inc. (Latero). This device induces lateral tractions on the skin's surface by means of a matrix of miniature piezoelectric bending actuators (64 actuators in an 8 × 8 matrix). The active area of the tactile display is 1.0 cm². The Latero was controlled by a 3.40GHz i7 2600 PC equipped with a dedicated Ethernet interface running under the User Datagram Protocol. The software ran in the Xenomai Real-Time Framework on Linux (Ubuntu), guaranteeing a stable rate for the control loop at high frequency. The refresh rate of the display was 1 kHz. Both the tactile and auditory stimuli were run from the same computer.

For the tactile stimuli, a virtual bar was simulated by the tactile display by locally inducing oscillatory strains in the skin of the fingertip at a rate of 80 Hz. The simulated bar was orthogonal to the finger's principal axis (see Figure 1). To minimize positional cues at the start of the stimulation, the tactile stimuli started from the central rows of the 9.6-mm long matrix, and then moved either toward the tip of the finger or toward the palm. From now on, we refer to these movements as outward and inward movements respectively. The stimulus duration was 250 ms, thus giving rise to a bar moving at a speed of 19.2 mm/s. The neutral stimulation consisted in a random activation of the four central rows of the tactile matrix.

The auditory stimuli were presented through a loudspeaker which was located just behind the tactile display in order to ensure a spatial coincidence between tactile and auditory stimuli. The stimuli consisted in three pure tones of linearly increasing (700 to

1,200 Hz), linearly decreasing (700 to 200 Hz), or constant (700 Hz) pitch. The duration of each of the auditory stimuli was 250 ms.

**Procedure.** The experiment was run in a dark, anechoic chamber. The participants sat in front of a table which also supported the tactile device and the loudspeaker, located approximately 25 cm from the participant. Participants completed two main sessions, one where they had to respond to the tactile stimuli, and one where they had to respond to the auditory ones. The order of presentation of these sessions was counterbalanced across participants. Each session was made of a block of training trials, followed by a unimodal block of trials, then a bimodal block of trials.

During the training block, participants learned the mapping between the stimuli and the relevant response keys. They were required to press the left or right button of the mouse in order to listen to the associated auditory stimulus (during the auditory session) or to experience the tactile stimulation (during the tactile session). In all blocks, the mouse buttons were therefore orthogonal to the direction of tactile movement. During this training, only unimodal stimuli were presented, and no verbal label was assigned to them. The stimuli were prompted by the participant: For instance, the participant heard a sound increasing in pitch when pressing the right button of the mouse and a sound decreasing in pitch when pressing the left button of the mouse (and vice versa for half of the participants). Similarly, during the tactile session, the participant had to place either his index or major finger on the tactile display (choosing the one he felt the most comfortable with). He would experience an inward or outward tactile direction of movement when pressing the right or the left key of the mouse. Participants were encouraged to listen to the auditory and experience the tactile stimuli until they were sure that they had learnt the mapping. They were first given 20 practice trials. Those participants who felt they had not learnt the stimulus-response mapping were given additional trials until they reported the mapping without errors.

Following a successful practice block, participants underwent one unimodal block of trials. As in the training block, only two kinds of tactile and auditory stimuli were used: inward or outward tactile stimulation (during the tactile session) and sounds of increasing or decreasing pitch (during the auditory session). Each stimulus was presented 10 times, resulting in a total of 20 unimodal trials for each modality (and 40 for the two modalities).

The participants' task was to respond to the direction of tactile movement or changes in pitch by pressing the corresponding right and left buttons of a mouse (the association between stimuli and buttons were counterbalanced across participants). Participants were instructed to respond as accurately and as quickly as possible, and they could make their responses at any time up to 2,500 ms after stimulus onset. If the participant responded before the stimulus ended, this response would

<figure><img src="image_36.png" /><figcaption>Figure 1. Tactile stimulator device and representation of the tactile stimulus. See the online article for the color version of this figure.</figcaption></figure>

<footnote n="1">¹ Note that two participants performed at chance level in the unimodal tactile baseline condition, which means that they were not able, owing to the novelty of the task, to discriminate the tactile stimuli. These participants did not complete the rest of the experiment and were not counted among the participants reported here. Similar inabilities to discriminate tactile stimuli were found in the subsequent experiments; thus, two participants were excluded from Experiment 2, five participants were excluded from Experiment 3, and none were excluded from Experiment 4.</footnote>
be logged as such but the stimulation was still completed. Participants were given auditory feedback regarding the correctness of their responses: Whenever they provided an incorrect answer, they heard the word “incorrect.” Both the verbal instructions and feedback were prerecorded by the experimenter. There was then a 500-ms intertrial interval; that is, between the offset of the stimulus or the offset of the feedback in case of incorrect trials, and the beginning of the next trial. Participants were given periodic breaks after every 20 trials.

In the bimodal blocks of trials, participants experienced 12 possible pairs of stimuli. When they had to respond to the tactile stimuli, the target tactile stimulus could be either inward or outward, and could be accompanied by one of three possible concurrent auditory stimuli: either increasing, decreasing, or neutral change in frequency (i.e., six possible pairings, see Table 1). In the auditory session, that is, when they had to respond to the auditory stimuli, the target auditory stimulus could be either of increasing or decreasing frequency, and could be accompanied by one of three possible concurrent tactile stimuli: inward, outward, or neutral (i.e., without direction; six other pairings, see Table 1). Each of the bimodal blocks of trials consisted of 30 repetitions of the six possible combinations. Participants thus completed a total of 360 trials over the tactile and auditory sessions.

Participants had their fingertip resting on the tactile display and listened to the sounds presented by the loudspeakers. On each test trial, participants experienced the tactile and the auditory stimuli simultaneously. They were required to pay attention to only one of them while ignoring the other. They were instructed to classify the stimuli as accurately and as quickly as possible by pressing one of the two keys of the mouse. As in the unimodal blocks, participants could make their response at any time up to 2,500 ms after stimulus onset. Verbal feedback was provided to them in cases of incorrect responses and there was then a 500-ms break between the offset of the stimulus and the beginning of the next trial. Participants were given periodic breaks after every 20 trials.

## Results

Trials in which participants failed to provide a response before the trial was terminated (overall less than 0.5% of trials) were not included in the data analyses. The primary dependent measure was the reaction time (RT), measured from stimulus onset, on correct trials only. Note that we also recorded the accuracy of the responses and 92.2% of the responses were correct. An analysis of variance (ANOVA) was conducted on the mean RTs for correct answers given by each participant with two factors: Modality (tactile vs. auditory) and Congruency (unimodal, neutral, congruent, and incongruent). The analysis revealed a significant main effect of Modality, $F(1, 15) = 119.95, p < .00001$, a significant

main effect of Congruency, $F(3, 45) = 9.17, p < .0001$, and a significant interaction between Modality and Congruency, $F(3, 45) = 4.82, p < .006$. A Duncan post hoc test on Modality × Congruency revealed that, when the target stimulus was tactile, there were significant differences between the unimodal condition and the three other conditions ($p < .001$); no other significant difference appeared. When the target stimulus was auditory there was no significant difference between the levels of the congruency factor. Participants were faster overall in the auditory conditions ($M = 54.87 \pm SD$ of 7.63 ms) compared with the tactile ($M = 83.06 \pm 10.47$ ms) condition.

To summarize the results, a significant difference between the unimodal and the bimodal sessions was evidenced, but the difference between the congruent and the incongruent conditions did not reach significance either for the tactile or the auditory condition. In other words, this experiment failed to reveal a crossmodal congruency effect (see Figure 2).

## Interim Discussion

The results did not reveal a congruency effect between direction of pitch and direction of tactile movement although the method used here was similar to other studies which obtained positive congruency effects between direction of pitch and visual movement (Evans & Treisman, 2010). One problem with the joint presentation of stimuli though comes from a lack of control of selective attention. Indeed, nothing guarantees that participants’ attention was focused on the targeted sensory modality only, and that the results do not reflect a failure of selective attention (e.g., Parise & Spence, 2012, for a discussion; see also Melara & O’Brien, 1987). As no significant difference between tones of increasing or decreasing frequency was observed when participants were asked to respond to touch, we can infer that participants did not merely base their answers on the perceived direction of pitch. The second experiment, resting on the successive presentation of stimuli and relying on their association with different response buttons, offers an alternative way to control for the role of attention.

## Experiment 2: Implicit Association Task

The second experiment investigated the crossmodal correspondence between auditory changes in pitch and tactile direction of movement by means of an implicit association test. Participants were presented with either a sound or a tactile stimulus and they had to categorize the stimulation (ascending vs. descending pitch, inward vs. outward tactile direction). They first had to learn the association between the four possible stimuli and the two response buttons that could be either congruent (i.e., the same button for

<caption>Table 1</caption>

The Twelve Possible Combinations of Auditory and Tactile Stimuli Presented in the Bimodal Blocks

<table><thead><tr><th rowspan="2">Response to simultaneous presentations</th><th colspan="2">Tactile stimulation</th><th colspan="2">Auditory stimulation</th></tr><tr><th>Outward</th><th>Inward</th><th>Rising</th><th>Descending</th></tr></thead><tbody><tr><td>Congruent</td><td>Rising pitch</td><td>Descending pitch</td><td>Outward tactile movement</td><td>Inward tactile movement</td></tr><tr><td>Incongruent</td><td>Descending pitch</td><td>Rising pitch</td><td>Inward tactile movement</td><td>Outward tactile movement</td></tr><tr><td>Neutral</td><td>Neutral pitch change</td><td>Neutral pitch change</td><td>Random tactile stimulation</td><td>Outward tactile movement</td></tr></tbody></table>
<figure><img src="image_37.png" /><figcaption>Figure 2. Participants' mean reaction times for the two conditions of stimulus presentation: tactile and auditory; for the four congruency conditions: congruent (upward tactile and rising pitch; downward tactile and descending pitch), unimodal (tactile only; auditory only), incongruent (upward tactile and descending pitch; downward tactile and rising pitch), and neutral (upward tactile and neutral pitch; downward tactile and neutral pitch in the tactile block; neutral tactile and rising pitch; neutral tactile and descending pitch in the auditory block). Error bars represent the standard errors of the means.</figcaption></figure>

increasing pitch and outward direction of movement, the other one for decreasing pitch and inward tactile direction of movement) or incongruent (the same button for increasing pitch and inward tactile direction of movement; the other one for decreasing pitch and outward tactile direction of movement). If there is a cross-modal correspondence between pitch and tactile direction of movement, participants are expected to be more accurate and faster in the congruent condition than in the incongruent one.

## Method

**Participants.** Fourteen sighted participants (seven men and seven women) took part in this experiment. Their ages ranged from 20 to 32 years ($M = 26$ years). All the participants were naive to the purpose of the experiment and reported normal auditory and tactile perception. They received €12 in return for their participation in this experiment which took approximately 90 min to complete.

**Procedure.** The materials (i.e., the tactile device and auditory display) were similar to those reported for Experiment 1. The arm on which the tactile stimuli were presented remained in a horizontal position. The tactile and auditory stimuli had the same durations as in Experiment 1. The tasks and procedure were different: this time an implicit association test was used (see below). Participants went through a training session and then completed a unimodal session made up of one tactile block and one auditory block (whose order was counterbalanced across participants). After the unimodal session, participants completed a bimodal session.

The training was similar to Experiment 1 in that participants had to learn the mapping between the stimuli and the relevant response keys, without explicit verbal labels. In the unimodal tactile block, the participants experienced inward and outward tactile stimulation. Their task was to respond to each stimulus by pressing one of the two mouse buttons. In the unimodal auditory block, the task was the same but participants experienced sounds increasing and decreasing in frequency. Each type of stimulus was repeated 10

<figure><img src="image_38.png" /><figcaption>Figure 3. Participants' mean reaction times for the two conditions of stimulus presentation: tactile and auditory; for the three congruency conditions: congruent (upward tactile and rising pitch; downward tactile and descending pitch associated to the same response key), unimodal (upward tactile and downward tactile associated to two response keys; rising pitch and descending pitch associated to two response keys), and incongruent (upward tactile and descending pitch; downward tactile and rising pitch associated to the same response key). Error bars represent the standard errors of the means.</figcaption></figure>

times, for a total of 40 unimodal trials across the tactile and auditory modalities.

In the bimodal sessions, participants first learned the association between the four possible stimuli and the two response buttons on the computer mouse. The associations were planned so as to be either congruent (i.e., increasing pitch and outward tactile direction of movement on the same button, decreasing pitch and inward tactile direction of movement on the other button) or incongruent (increasing pitch and inward tactile direction of movement on the same button, decreasing pitch and outward direction of movement on the other). These labels were not provided to the participants, who were just trained to respond to the stimuli. Participants completed the two possible associations in separate blocks, with half of the cohort starting with the congruent condition and half with the incongruent one.

On each trial, participants experienced either a tactile or an auditory stimulus and were required to identify them as accurately and as quickly as possible. When the stimulus was auditory, they had to press one of the two mouse buttons depending on whether the sound was increasing or decreasing in pitch. When the stimulus was tactile, they had to press one of the mouse buttons depending on whether the direction of movement was going inward or outward. Each session consisted of 30 repetitions of each of the two tactile stimuli and auditory stimuli presented in a random order. Participants completed 120 trials for each congruent and incongruent session (i.e., 240 in total).

## Results

Trials in which participants failed to provide a response before the trial was terminated (overall less than 0.4% of trials) were not included in the data analyses. The primary dependent measure was the RT, measured from stimulus onset, on correct trials only (Figure 3). The accuracy of the responses was also recorded and reached 93.88% of correct answers. An ANOVA conducted on the
<figure><img src="image_39.png" /><figcaption>Figure 4. The two arm positions used in Experiments 3 and 4: vertical and horizontal. See the online article for the color version of this figure.</figcaption></figure>

mean RTs on the correct answers for each participant with two factors: Modality (tactile vs. auditory) and Congruency (unimodal, congruent, and incongruent) revealed a significant main effect of Modality, $F(1, 13) = 8.62, p < .00001$, a significant main effect of Congruency, $F(2, 26) = 62.90, p < .00001$, and a significant interaction between Modality and Congruency, $F(2, 26) = 35.62, p < .00001$. A Duncan post hoc test on Modality X Congruency revealed that, except between auditory unimodal and tactile incongruent, all the other interactions were significant (all ps < .01). In particular, there was a significant difference for tactile stimuli between the congruent and incongruent conditions ($p = .0002$); and for auditory stimuli as well ($p = .0013$), with participants being faster in the congruent conditions than in the incongruent ones. Overall, participants were faster in the auditory ($68.84 \pm 7.98$ ms) than in the tactile ($M = 85.84 \pm 10.89$ ms) condition. They were faster in the unimodal ($67.02 \pm 7.15$ ms) followed by the congruent ($80.35 \pm 11.04$ ms) and then the incongruent condition ($86.90 \pm 10.40$ ms).

Interim Discussion

In this variant of the IAT, participants were required to focus on one stimulus at a time, while the association was only at the level of response buttons. The results revealed a significant congruency effect between changes in pitch and changes in direction of tactile movement. Participants were for instance faster in responding to a tone increasing in pitch when the response button was shared with the outward tactile movement than when it was shared with the inward tactile movement. This result suggests that an association between tactile movement and direction of pitch operates at the implicit level. Besides the role of attention, this protocol avoids the comparison or direct mapping required by the speeded classification task used in the first experiment. When presented together, the assumption is that a decrease in pitch should correspond to a downward tactile movement, and an increase in pitch to an upward tactile movement. As the primary sensory organs of vision and audition are rigidly attached to the head together with the vestibular

system, and as the head is typically maintained in a vertical position, “upward” and “downward” for vision and audition seem to correspond to the vertical—that is, to the head orientation. In contrast, touch is distributed throughout the entire body: As tactile receptive surfaces are permanently reoriented in space, it is not clear how the direction of tactile movement is encoded. To develop this hypothesis further, we conducted a third experiment, using the successful method used in the second experiment.

Experiment 3: Implicit Association Test: Horizontal Versus Vertical Arm Position

The third experiment investigated the influence of arm position on the crossmodal correspondence between change in auditory pitch and change in tactile direction of movement. The position of the arm on which participants received the tactile stimuli was placed, in two separate conditions, either vertically or horizontally (see Figure 4). If a greater correspondence effect is obtained in the vertical condition, this would suggest that the most natural spatial mapping of pitch corresponds to the vertical plane, as it is the case for the audio-visual correspondence between pitch and vertical elevation in visual space (e.g., Chiou & Rich, 2012; Rusconi et al., 2006, for comparisons between different spatial mappings of pitch). On the other hand, if a similar effect is obtained across the two conditions, the congruency would be effective both in the horizontal and vertical plane. This could mean that, with respect to touch, the bodily reference frame (the hand) is determinant for the correspondence and that the tip of the finger corresponds to “up,” independently of the actual hand position. Alternatively, it could also be the case that there are two different correspondences at stake: one between pitch and the vertical plane, and the other between pitch and the horizontal plane.

Method

**Participants.** Fourteen sighted participants (one man and 13 women) took part in this experiment. Their ages ranged from 19 to
34 years (M = 25 years). All of the participants were naive to the purpose of the experiment and reported normal auditory and tactile perception. They received €12 in return for their participation in this experiment which took approximately 90 min to complete.

**Materials and procedure.** The materials and procedure were similar to those reported for Experiment 2 with the following difference: The participants’ arm position was horizontal in one session and vertical in the other (see Figure 4). The order of these two sessions was counterbalanced across participants. The only difference was the number of trials in the bimodal session which was the same for all the participants, but divided into the two conditions of arm movements. More precisely, the participants completed 15 repetitions of each of the two tactile stimuli and auditory stimuli for each of the congruency conditions (congruent and incongruent) and for each of the arm positions (vertical and horizontal), that is, 240 trials in total.

## Results

Trials in which participants failed to provide a response before the trial was terminated (overall less than 0.53% of trials) were not included in the data analyses. The accuracy of the responses reached 93.10% of correct answers. An ANOVA was conducted on the mean RTs for correct trials for each participant, with three factors: Modality (tactile vs. auditory), Arm Position (horizontal vs. vertical), and Congruency (unimodal, congruent, and incongruent). The ANOVA revealed a significant main effect of Modality, $F(1, 13) = 41.59, p < .0001$, a significant main effect of Congruency, $F(2, 26) = 58.38, p < .00001$, and no effect of Arm Position, $F(1, 13) = .06, p = .81$. There was a significant interaction between Modality and Congruency, $F(2, 26) = 40.36, p < .00001$. There was no other significant interaction (all $F_s < 1$). A Duncan post hoc test on Modality × Congruency revealed that except between tactile unimodal and auditory incongruent all the other interactions were significant (all $p_s < .05$). In particular, there was a significant difference for responses to tactile stimuli between the congruent and incongruent conditions ($p < .001$), with participants being faster in the congruent condition. Similarly, there was a significant difference for responses to auditory stimuli between the congruent and incongruent conditions ($p = .001$) with participants being faster in the congruent condition (Figure 5). Thus, this experiment revealed a crossmodal congruency effect for the two sensory modalities and the two arm positions.

Overall, the participants were faster in the auditory ($72.19 \pm 7.16$ ms) than in the tactile ($M = 90.09 \pm 12.70$ ms) condition. They were also faster in the unimodal ($70.74 \pm 7.96$ ms) followed by the congruent ($84.66 \pm 8.98$ ms) and then the incongruent condition ($90.75 \pm 12.21$ ms).

## Interim Discussion

The third experiment confirmed the existence of an implicit association between changes in pitch and direction of tactile movement. More importantly, it revealed that this association occurs in two different arm positions. Thus, a rising pitch is associated to a tactile movement going toward the fingertip, and a descending pitch to a tactile movement going inward toward the palm, independently of the hand being oriented upward or horizontally. Results are consistent with the correspondence occurring in a hand-centered frame of reference.

## Experiment 4: Horizontal Versus Vertical Arm Position in Early and Late Blind People

The fourth experiment investigated the role of prior visual experience on the crossmodal correspondence between auditory pitch and tactile direction of movement, while still investigating the role of arm position in the correspondence. Early blind and late blind participants completed the experiment as described in Experiment 3.

## Method

**Participants.** Twenty-four blind participants took part in this experiment. Twelve of them were late blind (four men and eight women, becoming blind after the age of five), eight of them were congenitally blinds (five men and three women), and four of them were early blind, that is, they became blind between 1 and 3 years of age (one man and three women). Note that in the results, as there were no significant differences between the eight congenitally blind and the four early blind, their results were subsequently regrouped. All blind participants were completely blind or had light perception without shape perception and used a cane or a seeing-eye dog for walking. For all blind participants, blindness resulted from genetic disease, retinitis, an accident or cancer and was not associated to any additional neurological impairment. All

<figure><img src="image_40.png" /><figcaption>Figure 5. Participants’ mean reaction times for the two conditions of stimulus presentation: tactile and auditory, and for the three congruency conditions: congruent, unimodal, and incongruent (as in Experiment 2). Error bars represent the standard errors of the means. Note that, as there was no effect of arm position, in the graph, horizontal and vertical arm positions were collapsed.</figcaption></figure>
participants were naive to the purpose of the experiment and reported normal auditory and tactile perception. They received €25 in return for their participation in this experiment which took approximately 180 min to complete.

**Materials and procedure.** The materials and procedure were the same as Experiments 2 and 3.

## Results

Trials in which participants failed to provide a response before the trial was terminated (overall less than 0.56% of trials) were not included in the data analyses. The accuracy of the responses reached 93.92% of correct answers. An ANOVA was conducted on mean RTs, for the correct answers given by each participant only, with four factors: Onset of Blindness (early vs. late), Modality (tactile vs. auditory), Arm Position (horizontal vs. vertical), and Congruency (unimodal, congruent, and incongruent). Onset of Blindness was a between-participants factor whereas Modality, Arm Position, and Congruency were within-participants factors. The ANOVA revealed a significant main effect of Modality, $F(1, 22) = 92.72, p < .00001$, a significant main effect of Congruency, $F(2, 44) = 60.12, p < .00001$, no effect of Onset of Blindness, $F(1, 22) = .04, p = .32$, and no effect of Arm Position, $F(1, 22) = .10, p = .75$. There was a significant interaction between Modality and Congruency, $F(2, 44) = 20.65, p < .00001$. There was no other significant interactions, except between Arm Position, Modality, and Congruency, $F(2, 44) = 4.05, p = .024$. A Duncan post hoc test on Modality × Congruency revealed that when the target stimulus was tactile there were significant differences between the unimodal condition and the two other conditions (all $ps < .001$) but not between the congruent and incongruent conditions ($p = .98$). Similarly, when the target stimulus was auditory, there were significant differences between the unimodal condition and the two other conditions (all $ps < .001$) but not between the congruent and incongruent conditions ($p = .90$). Participants were overall faster in the auditory ($M = 61.50 \pm 6.15$ ms) than in the tactile ($M = 76.02 \pm SD$ of 9.34 ms) condition. The participants were faster in the unimodal (61.36 ± 7.41 ms) than both in the congruent (73.23 ± 8.96 ms) and the incongruent (73.11 ± 7.76 ms) conditions. In summary, although the results revealed a significant effect of congruency there were no significant differences between the congruent and incongruent condition in either of the two modalities. Thus there was no crossmodal congruency effect for both early and late blind participants (see Figure 6). Correlations were also conducted and no correlation was found between the amount of previous visual experience and the size of the effect (congruent–incongruent) in the blind participant group, $r(22) = .32, p = .12$. No correlation was found for the participants of the late blind group only, $r(10) = .13, p = .69$.

## Interim Discussion

The performance of blind participants did not show a congruency effect between the direction of tactile movement and changes in pitch, whereas this effect was evidenced in sighted participants in Experiments 2 and 3, which used the same methodology. What's more, there was no effect related to the age of onset. This failure cannot be attributed to a failure of attention, as the experiment relied on stimuli being successively presented. This differ-

<figure><img src="image_41.png" /><figcaption>Figure 6. Participants' mean reaction times for the two conditions of stimulus presentation: tactile and auditory; for the three congruency conditions: congruent, unimodal, and incongruent (as in Experiment 2 and 3); and for the two onsets of blindness: early and late. Error bars represent the standard errors of the means. Note that, as there was no effect of arm position, in the graph, horizontal and vertical arm positions were collapsed.</figcaption></figure>

ence is not linked to arm position either, as blind participants did not exhibit a congruency effect in either the horizontal or vertical arm positions. Vision seems therefore to be necessary to mediate the correspondence between direction of pitch and direction of tactile movement.

## General Discussion and Conclusions

The present study reveals the existence of a congruency effect between auditory changes in pitch and changes in direction of movement in touch. The study also provides further evidence of the usefulness of IAT protocols when testing crossmodal correspondences. Interestingly though, whereas effects of audio-visual congruence between pitch and visual elevation have been successfully tested in speeded classification tasks (Evans & Treisman, 2010; Marks, 2004), no significant effect was observed in our audio-tactile version. This lack of an effect in the speeded classification task, but not in the IAT task, might be explained by the attentional demand needed to ignore the random presentation of one kind of stimuli. It could also come from a lack of strategic information about the informative character of the paired stimuli.

A difference between the present protocol and previous ones comes from the fact that the participants in our study were not explicitly instructed about the informative character of the other cue (e.g., they were told that a high-pitched sound is more likely to be correlated with a high visual target, Chiou & Rich, 2012) or at least trained by using the predicates “high” and “low” to describe the sounds (Evans & Treisman, 2010). Here, we purposely provided only neutral information about these characteristics, to avoid biasing participants on the classification of the tactile stimuli. The lack of a label for the “rising–descending” character of the sounds and tactile stimuli might have been responsible for the lack of effect in the cognitively demanding speeded classification task. As was suggested by various authors (see Spence & Deroy, 2013, for a review), the effects of crossmodal correspondences on behavior, when tested in multisensory settings, might not be automatic and depend therefore on an explicit strategic