<figure><img src="image_1.png" /></figure>

HAL
open science

# Multisensory spatial perception and sensory substitution

Malika Auvray

► To cite this version:

Malika Auvray. Multisensory spatial perception and sensory substitution. Cognitive science. Sorbonne Université, 2018. tel-03975339

HAL Id: tel-03975339

https://hal.science/tel-03975339

Submitted on 6 Feb 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Sorbonne Université

Institut des Systèmes Intelligents et de Robotique

Habilitation à Diriger des Recherches

Specialty: Biology

by Malika Auvray

**Multisensory spatial perception**
**and sensory substitution**

Viva planned for the 25th September 2018, before the following committee:

Dr Michel Denis LIMSI, Orsay, France (Examiner)

Pr Vincent Hayward Sorbonne Université, Paris, France (Examiner)

Dr. Christophe Lopez LNSC, Marseille, France (Referee)

Dr Michael J. Proulx University of Bath, UK (Referee)

Pr Maurice Ptito School of Optometry, Montreal, Canada (Examiner)

Pr Charles Spence Oxford University, UK (Examiner)

Dr Isabelle Viaud-Delmon IRCAM, Paris, France (Referee)
Acknowledgements

I first would like to thank Michel Denis, Vincent Hayward, Christophe Lopez, Michael Proulx, Maurice Ptito, Charles Spence, and Isabelle Viaud-Delmon for agreeing to be part of my HDR committee.

Most of the research presented in this manuscript was conducted by the students and post-doctoral researchers that I have supervised and I want to warmly thank them for their work and for all the things I had the chance to learn from them. I would like to address special thanks to Gabriel Arnold, who stayed under my supervision for 5 years, and to Jessica Hartcher O'Brien and Laura Fernandez who stayed two years. Many thanks as well to the two PhD students Jérémy Roque and Adrien Girard, and to all the master students.

The research I have conducted has always been the fruit of collaborations, and I want to thank my colleagues for the chance I had to work with them, all the work and enthusiastic discussions we had together. I would like to express my deep gratitude to my collaborators, Vincent Hayward, Charles Spence, Michel Denis, Kevin O'Regan, Frédérique de Vignemont, Jérôme Dokic, Pierre Jacob, Ophelia Deroy, Christophe Jouffrais, Sylvain Hanneton, Agnès Roby-Brami, Philippe Tarroux, Alberto Gallace, Barry Smith, Maurice Ptito, Ron Kuppers, Charles Lenay, Erik Myin, Gaëtan Parseihian, Marieke Rohde, Mathieu Dubois. I also thank the researchers I more recently had the opportunity to work with, Fabrice Sarlegna, Nathanaël Jarasse, Wael Watcha, Alessandro Farnè, Mehdi Ammi, Jérémie Lafraire.

I would also like to thank many persons who provided me with invaluable advices and help along these years: Elisabeth Pacherie, Jean-Louis Vercher, Christian Lorenzi, Roberto Casati, Pascal Sommer, Raja Chatila, Sharon Peperkamp, Alva Noë, François Recanati, Joelle Proust, Vincent Gaudefroy.

I would like to thank the heads of the institutions I've successively been in for welcoming me and allowing me to pursue these lines of research. Finally, I would like to thank my colleagues from these institutions: Laboratoire Psychologie de la perception, Université de Technologie de Compiègne, Department of Experimental Psychology in Oxford, Department of Philosophy in Antwerp, Laboratory Neurophysics and Physiology, Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur, Institut Jean Nicod, and Institut des Systèmes Intelligents et de Robotique, Paris for sharing lab life with me and bringing to it joy and enriching discussions.
Table of contents

<table><tr><td>I. Presentation of the scientific work</td><td>1</td></tr><tr><td>I.1. Abstract in English</td><td>1</td></tr><tr><td>I.2. Abstract in French</td><td>2</td></tr><tr><td>I.2. General introduction</td><td>3</td></tr><tr><td>II. Sensory substitution</td><td>4</td></tr><tr><td>II.1. Conceptual understanding of sensory substitution</td><td>5</td></tr><tr><td>II.2. Seeing the thunder while still hearing it</td><td>9</td></tr><tr><td>II.3. Sensory substitution and learning</td><td>13</td></tr><tr><td>II.4. Crossmodal correspondences for the optimization of the devices</td><td>17</td></tr><tr><td>II.5. Prototypes of sensory substitution devices</td><td>20</td></tr><tr><td>II.6. Conclusion on sensory substitution</td><td>21</td></tr><tr><td>III. Multisensory spatial reference frames</td><td>22</td></tr><tr><td>III.1. The variability in our reference frames</td><td>22</td></tr><tr><td>III.2. Adopting a reference frame, a natural process or a mere cognitive choice?</td><td>26</td></tr><tr><td>III.3. The influence of vision on the reference frames that are adopted</td><td>29</td></tr><tr><td>III.4. The embodied nature of spatial perspective taking</td><td>31</td></tr><tr><td>III.5. The role of proprioceptive information on spatial perspective taking</td><td>32</td></tr><tr><td>III.6. The transfer of a learned reference frame</td><td>33</td></tr><tr><td>III.7. The bi-stable palm</td><td>35</td></tr><tr><td>III.8. The processes of distal attribution</td><td>37</td></tr><tr><td>III.9. Conclusion on multisensory reference frames</td><td>40</td></tr><tr><td>IV. Perspectives for future research</td><td>42</td></tr><tr><td>IV.1. Exteriorisation and embodiment</td><td>42</td></tr><tr><td>IV.2. Sensory substitution of vision and proprioception</td><td>44</td></tr><tr><td>IV.3. Interactions between spatial and mental perspectives</td><td>45</td></tr><tr><td>V. References</td><td>46</td></tr><tr><td>CV</td><td>53</td></tr><tr><td>Annex. Selection of articles</td><td>75</td></tr></table>

1. Pesnot-Lerousseau, J., Arnold, G., & Auvray, M. (submitted). Visualizing sounds: training-induced plasticity with a visual-to-auditory conversion device.

2. Arnold, G., & Auvray, M. (2014). Perceptual learning: Tactile letter recognition transfers across body surfaces. *Multisensory Research*, 27, 71-90.

3. Arnold, G., & Auvray, M. (2017). Tactile recognition of visual stimuli: Specificity versus generalization of perceptual learning. *Vision Research*.
4. Deroy, O., Fasiello, I., Hayward, V., & Auvray, M. (2016). Differentiated audio-tactile correspondences in sighted and blind individuals. *Journal of Experimental Psychology: Human Perception and Performance, 42*, 1204-1214.

5. Arnold, G., Spence, C., & Auvray, M. (2016). Taking someone else's spatial perspective: Natural stance or effortful decentring? *Cognition, 148*, 27-33.

6. O'Brien, J., & Auvray, M. (2016). Cognition overrides orientation dependence in tactile viewpoint selection. *Experimental Brain Research, 234*, 1885-1892.

7. O'Brien, J., & Auvray, M. (2014). The process of distal attribution illuminated through studies of sensory substitution. *Multisensory Research, 27*, 421-441.
# I. Presentation of the scientific work

## I.1. Abstract

My research focuses on perceptual processes and in particular on the relations between sensory modalities, both in their correspondences and in their interactions. The conducted research is based on an intrinsic multidisciplinary approach. The tested hypotheses come from a joint consideration of theoretical models, experimental and clinical approaches, with a preoccupation of applied outputs in the field of perceptual rehabilitation. The first axis of my research concerns sensory substitution devices, which aim at compensating one or several functions of a deficient sensory modality by means of information transmitted to an intact sensory modality. My research lies both upstream and downstream the design of sensory substitution devices. In particular, my work involves a conceptual understanding of the nature and possibilities offered by sensory substitution devices, studies of the central factors underlying their learning, their adequacy to the target population, taking into account their individual specificities, and the possibility of improving their design by the use of crossmodal correspondences. I also have collaborated to the design and tests of three sensory substitution devices. The second axis of my work concerns the multisensory groundings of our spatial representations, which vary as a function of the sensory modalities that are involved. Indeed, visual information is initially coded in retinotopic coordinates, auditory information is initially coded according to a head-centred reference frame, and tactile information is coded either in somatotopic or in head-centred coordinates. My aim is to understand how these reference frames interact to give rise to a functional representation of the environment and how these reference frames can be modified by the use of sensory substitution devices and other tools. Taken together these two research axes aim, at the fundamental level, at contributing to the comprehension of the plasticity of our perceptual and bodily representations during tool use. At the applied level, they aim at improving the design and learning of sensory substitution devices.
I.3. Résumé

Mes recherches portent sur les processus perceptifs et en particulier sur la relation entre les
modalités sensorielles dans leurs correspondances ainsi que dans leurs interactions.
L'accent est mis sur une démarche fortement interdisciplinaire, à l'intersection entre
psychologie cognitive et philosophie, avec une préoccupation d'applications pour le
développement de dispositifs de substitution sensorielle. Le premier axe de mes recherches
concerne les dispositifs de substitution sensorielle qui visent à assister ou à remplacer une
ou plusieurs fonctions d'un organe sensoriel défaillant à l'aide d'un autre organe sensoriel.
Mes travaux scientifiques s'inscrivent à la fois en amont et en aval de la conception des
dispositifs. Ils impliquent l'étude de la nature et des possibilités offertes par la substitution
sensorielle, des facteurs centraux à leur apprentissage, de leur adéquation à la population
cible en prenant en compte les spécificités individuelles et de la possibilité d'améliorer leur
conception par l'utilisation des correspondances intermodales. J'ai aussi collaboré à la
mise au point et aux tests de trois de ces dispositifs. Le second axe des mes recherches se
penche sur la construction multisensorielle de nos représentations spatiales en fonction des
modalités sensorielles impliquées. Par exemple la vision se base sur un référentiel centré
sur les yeux, l'audition sur un référentiel centré sur la tête et le toucher à la fois un cadre de
référence centré sur la surface du corps et un centré sur la tête. Mon objectif est de
comprendre comment ces cadres de référence interagissent pour donner lieu à une
représentation fonctionnelle de l'environnement et comment ils se modifient lors de
l'utilisation de dispositifs de substitution sensorielle, et plus généralement d'outils. Pris
ensemble, ces deux axes de recherche visent à un niveau fondamental à éclairer la question
de la plasticité des représentations de nos espaces corporels et distants. A un niveau
appliqué, ils visent à l'amélioration de la conception et de l'apprentissage des dispositifs de
substitution sensorielle.
# I. General introduction

In this manuscript, I present the scientific research conducted since my PhD, in 2004. The presented studies were carried out during a year I've spent as ATER (associate lecturer) at UTC, Compiègne University (2004-2005), during three years as a post-doctoral researcher (2005-2008) at Oxford University (under the supervision of Prof. Charles Spence), at the University of Antwerp (under the supervision of Prof. Erik Myin), and at the Laboratory Neurophysics and Physiology, Paris (CNRS UMR 8119, under the supervision of Agnès Roby-Brami and Sylvain Hanneton). The studies were then conducted after I was hired by the CNRS, first at the LIMSI (CNRS UPR 3251), Orsay (2008-2014), then at the ISIR (CNRS UMR 7222), Paris, where I am since January 2015.

My research focuses on perceptual processes and in particular on the relations between sensory modalities, both in their correspondences and in their interactions. They were conducted thanks to national collaborations (ISIR, UMR 7222; IJN, UMR 8129; ISM, UMR 7287; LPP, UMR 8158; LNP, UMR 8119; IRCAM, UMR 9912; Impact, U864; Costech-UTC; LIMSI, UPR 3251; Institut de Recherche Paul Bocuse), international collaborations (Department of Experimental Psychology, Oxford University; Institute of Philosophy, London University; Università degli Studi di Milano-Bicocca, Milan; Department of Cognitive Neuroscience, University of Bielefeld; Antwerp University; York University), and collaborations with the industry (Caylar, Ile de France; Pernod-Ricard, Paris; Roger McMahon & Associates, Glasgow; Oculus Research).

The conducted research is based on an intrinsic multidisciplinary approach. The tested hypotheses come from a joint consideration of theoretical models, experimental and clinical approaches, with a preoccupation of applied outputs in the field of perceptual rehabilitation. My scientific work gave rise to 34 journal articles, 12 book chapters, 68 communications in international conferences with proceedings, and 60 communications without proceedings (1921 quotes, according to Google Scholar, for an h-index of 19). My research on sensory substitution allowed me to participate to TV and radio documentaries, as well as general public articles.
## II. Sensory substitution

The first and main axis of my research concerns sensory substitution devices. These devices aim at compensating one or several functions of a deficient sensory modality by converting stimuli that are normally accessed through this deficient sensory modality into stimuli accessible by another sensory modality. For instance, these devices can convert visual information into sounds or tactile stimuli. Sensory substitution devices have broad applications such as sensory rehabilitation and perceptual augmentation. For instance, visual-to-tactile (e.g., Bach-y-Rita et al., 1969; White et al., 1970) and visual-to-auditory (e.g., Hanneton et al., 2010; Meijer, 1992) conversion systems were designed in order to assist blind and visually impaired people. In most visual-to-tactile devices, the translation code is analogical. For instance, a visual circle can be translated into a circular pattern of tactile stimuli. Non-analogical codes have also been used, for instance by converting distance into vibrations (e.g., Farcy et al., 2006; Maidenbaum et al., 2014). The code used in visual-to-auditory devices translates several dimensions of the visual signal into dimensions of the auditory signal. For instance, the vOICe (Meijer, 1992) translates vertical position into frequency, horizontal position into time scanning, and visual brightness into auditory loudness. Other visual-to-auditory devices convert colour information into musical notes (e.g., the EyeMusic, Levy-Tzedek et al., 2014) or use different orchestral instruments (the See ColOr; Bologna et al., 2009). Numerous devices were designed to compensate other deficits, such as proprioceptive deficits (e.g. see Danilov et al., 2007; Diot et al., 2014). In the field of perceptual augmentation, these devices were developed in order to gain access to visual information, either under degraded conditions of perception (such as fire fighting or military operations, e.g. see Jones et al., 2009) or when the amount of visual information is thought to be too important (such as in car driving; e.g. Ho & Spence, 2008). My research focused mainly on visual-to-tactile and visual-to-auditory sensory substitution devices that were designed to compensate for visual impairments.

Studies conducted on sensory substitution devices revealed an important structural and functional plasticity of the central nervous system. Regarding structural plasticity, studies have shown that the use and practice of visual-to-tactile (Ptito et al., 2005; see also Kupers et al., 2006) and visual-to-auditory (e.g., Striem-Amit & Amedi, 2014) devices result in increased activation in blind people's visual cortex (for reviews, see Proulx et al., 2014; Stiles & Shimojo, 2015; see also Ptito et al., in press, for a review and discussion of the functional correlates of increased activation in the visual cortex). Regarding functional plasticity, a variety of devices have been tested across a variety of tasks. In particular, studies have revealed that these devices allow their users to perform localisation tasks (e.g., Janson, 1983; Levy-Tzedek et al., 2012; Proulx et al., 2008) and simple form, as well as complex shapes recognition (e.g., Arno et al., 2001; Auvray, Hanneton, & O'Regan, 2007). In addition, users of sensory substitution devices can build, to some extent, a perceptual
space that possesses visual characteristics although the sensory inputs come from another sensory modality (e.g., Renier et al., 2005).

Since their inception in the sixties, sensory substitution devices proved to be fruitful tools to investigate questions in the fields of philosophy of perception, experimental psychology, and studies of brain plasticity. However, in spite of their potential and the perspectives they open, sensory substitution devices remain under-used to date. Thus, one of the major challenges for scientists working in the field of rehabilitation is to favour the use of aid devices by impaired people. The limited success of sensory substitution devices can be explained by insufficiencies in the analyses of what is sensory substitution, insufficiencies in the understanding of impaired people's perceptual specificities, and deficiencies in the design of the devices. To overcome these insufficiencies my research lies both upstream and downstream the design of sensory substitution devices. In particular, my research involves a conceptual understanding of the nature and possibilities offered by sensory substitution devices, studies of the central factors underlying their learning, their adequacy to the target population, taking into account their individual specificities, and the possibility of improving their design by the use of crossmodal correspondences. I have also collaborated to the design and tests of three sensory substitution devices, two visual-to-auditory conversion systems and a visual-to-tactile one.

## II.1. Conceptual understanding of sensory substitution

I have investigated conceptually the question of the nature of the processes involved in the use of sensory substitution devices in collaboration with Ophelia Deroy (University of London), Erik Myin (University of Antwerp), Maurice Ptito (McGill University, Montreal), Ron Kuppers (University of Copenhagen), Laurence Harris (York University), Mirko Farina (University of Edinbourg), and Gabriel Arnold (post-doctoral researcher).

From a theoretical point of view, one of the main goals is to understand what underlies the acquisition of new recognition and localization skills that are usually characteristic of a given sensory modality by means of another. In particular, one important question raised is to which sensory modality does the perception with a sensory substitution device belong? Two opposite theses were, at first, put forward: The dominance thesis (e.g., Block, 2003; Humphrey, 1992; Prinz, 2006) according to which perception with a sensory substitution device remains in the substituting modality (touch or audition) and the deference thesis (e.g., Hurley & Noë, 2003; Noë, 2004; O'Regan, 2011) according to which perception switches to the substituted modality (vision). The deference thesis has opened the door to over-optimistic claims involving the idea that users of visual-to-tactile substitution devices would become able to “see with the skin” (White et al., 1970) or to “see with the brain” (Bach-y-Rita et al., 2003)”. These optimistic claims have been echoed to the wider audience
including potential users of these devices: sensory substitution devices have been advertised as “rewiring brains to see with sound” and “restoring a form of sight to the blind” (Trivedi, 2010).

With my collaborators (Arnold, Pesnot-Lerousseau, & Auvray, 2017; Auvray & Farina, 2017; Auvray & Myin, 2008; Deroy & Auvray, 2012, 2014), we suggested that it is time to go beyond the dominance versus deference debate, and that perception with a sensory substitution device goes beyond assimilation to either the substituting or the substituted modality. According to this view, sensory substitution should be understood as being vertically integrated on pre-existing capacities that involve the substituting and the substituted sensory modalities, as well as cognitive processes (see figure 1).

<figure><img src="image_3.png" /><figcaption>Figure 1. Three alternative views on the processes involved in sensory substitution, with their specificities at the behavioural, neural, and phenomenological levels. Illustrative examples are given into brackets (figure from Arnold, Pesnot-Lerousseau, & Auvray, 2017).</figcaption></figure>

To go deeper into the defended view, the dominance versus deference debate remains based on an implicit perceptual equivalence which we suggest to identify as a perceptual assumption. The influence of this perceptual assumption is visible in the fact that researchers accept or target equivalences between using a sensory substitution device and the exercise of a sensory modality. In other words, the perceptual assumption considers that sensory substitution follows what occurs with canonical cases of perception through one of the
typical sensory modalities that is as specialized channels for transducing external information. To us, this perceptual assumption has led to a confirmation bias in the interpretation of the results. The results of existing studies have been filtered out of the negative evidence, or data fitting less well with this assumption, while the remaining evidence has been seen as confirming the equivalence between using a sensory substitution device and perceiving through one of the canonical senses. Furthermore, the experimental protocols themselves are built with the perceptual assumption in mind which, in turn, constrains or limits the kind of data that can be gathered. In the series of articles summarized below, we sought to stress the limits of the perceptual models and the perspectives that can be opened by alternative models.

First, in collaboration with Erik Myin (Auvray & Myin, 2009), we discussed the dominance versus deference debate. We had recourse to the criteria that have been used for the taxonomy of our sensory modalities to determine which of them are fulfilled by the use of sensory substitution devices. These criteria involve sensory organ, stimuli, properties, qualitative experience, behavioural equivalence, dedication, and sensorimotor equivalence. The reviewed criteria did not allow favouring the dominance or the deference theses, and most of them appeared as a matter of degree or a matter of interpretation. We subsequently argued that the evidence leads to an alternative view, according to which the experience after sensory substitution is a transformation, extension, or augmentation of our capacities rather than being equivalent or reducible to an already existing sensory modality. We developed this view by comparing sensory substitution devices to other “mind enhancing tools” such as pen and paper, sketchpads or calculators. In particular we built on the transformative view of mind enhancing tools (e.g., Clark, 2003; Menary, 2006, 2007) according to which they do transform cognition in a qualitative way. In this sense, these tools not only facilitate established cognitive processes, they can also allow for the appearance of novel cognitive operations, which simply would have been impossible without them. For example, without the proper means and tools to write down, calculate, or draw diagrams, human cognitive abilities would not have evolved to their current state (Van Leeuwen, Verstijnen, & Hekkert, 1999). To us, an analysis of sensory substitution in terms of mind enhancing tools unveils it as thoroughly transforming sensory experience and as giving rise to a novel form of interaction with the environment.

In a second step, we refined this view with Ophelia Deroy (see Deroy & Auvray, 2012, 2014), proposing a vertical integration model. We argued that learning to use a sensory substitution device should not be thought of as occurring horizontally. Indeed, the use of sensory substitution devices does not have to fit within the concept of a sense even if it apparently serves similar functions (e.g., identification and localization). The corresponding experience does not fit among the sensory modalities and it requires the existence of (some
of) these modalities. Experience with sensory substitution devices is built up from existing
sensory modalities both in terms of the used receptors and of the invented code the devices
rely on. In addition, this relation is of interdependence or crafting. In this sense, the new skill
starts from an existing one, and although it becomes progressively more independent, it does
not become totally detached from the initial elements. We used an analogy with dual-route
models of reading (e.g., Coltheart & Rastle, 1994), according to which, learning to use a
sensory substitution device is no longer to be thought of as being merely a matter of
perceptual learning or adaptation, but as the building of a parallel access to cognitive and
spatial representations that get grafted onto some pre-existing perceptual-cognitive route
(e.g., sounds to objects and spatial representations in the case of visual-to-auditory
conversion systems). What sensory substitution reveals in this case, is not strictly sensory
plasticity nor perceptual emergence, but mostly culturally driven multisensory plasticity, that
is the margin left for exploiting and redirecting the existing rules of multisensory and
crossmodal interactions to build new cognitive routes between existing components. To
summarize, here sensory substitution is treated as involving both perceptual and cognitive
processes, which take into account integration of the novel information with the existing
perceptual-semantic route.

In addition, with Mirko Farina (see Auvray & Farina, 2017), we discussed the claims made
that long-term use of sensory substitution devices may induce, in practiced users, the
emergence of forms of synaesthesia (e.g., Proulx, 2010; Ward & Meijer, 2010; Ward &
Wright, 2014). To clarify this position, we proposed to dissociate the substituted information
from the associated phenomenology and to verify for each, if they fulfil the criteria used to
define congenital synaesthesia (see Table 1). Our analysis highlighted that, when the
concurrent is considered to be the associated phenomenology there is a crucial lack of
empirical data, as the hypothesis arises from verbal reports of only two users. These
preliminary reports suggest that there might be a consciously perceived concurrent. There
can be idiosyncrasy, although narrower than in congenital synesthesia, and there are no
substantial data to know if there can be automaticity and consistency; although these cannot
be ruled out either. When the concurrent is considered to be the substituted information, the
existence of an inducer concurrent pairing can be acknowledged. However, the extent to
which both the concurrent and the inducer are consciously perceived awaits further
empirical data. In addition the concurrent is linked to the inducer more in a relative way (for
instance the louder, the brighter) than in an absolute way (which is closer to cross-modal
correspondences than to synesthesia; see Deroy & Spence, 2013) and there is no
idiosyncrasy. In summary, sensory substitution can hardly be claimed to match the criteria
established to single out genuine forms of synesthesia. As a consequence, anyone defending
SSD-use as being synesthetic will have to give an account of why one or several criteria to
define synesthesia can be loosened in some cases without weakening the understanding and definitions given to congenital synesthesia.

<table><thead><tr><th>Cases</th><th>Criteria</th><th>Inducer-concurrent pairing</th><th>Idiosyncrasy</th><th>Automaticity of the process</th><th>Consistency over time</th></tr></thead><tbody><tr><td></td><td>Congenital synesthesia</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td></tr><tr><td></td><td>SSD: associated phenomenology</td><td>Debated</td><td>Yes (narrow set)</td><td>Lack of Data</td><td>Lack of Data</td></tr><tr><td></td><td>SSD: substituted information</td><td>Lack of Data</td><td>No</td><td>Lack of Data</td><td>Yes</td></tr><tr><td></td><td>Crossmodal correspondences</td><td>No</td><td>No</td><td>Yes</td><td>Yes</td></tr></tbody></table>

Table 1. Summary of the criteria used to define synesthesia, which are fulfilled by the use of sensory substitution devices. In the table we also added congenital synesthesia and crossmodal correspondences. The terms yes and no are used when the claim is not controversial, debated is used when there are existing data, but their interpretation is subject to controversy, lack of data is used when more empirical data are needed.

Finally, with Gabriel Arnold (post-doctoral researcher) and Jacques Pesnot-Lerousseau (master student), we built on the vertical integration model in order to investigate the predictions that can be made from this model (Arnold, Pesnot-Lerousseau, & Auvray, 2017). In particular, the vertical integration model involves the idea that, at the perceptual level, performance and experience with the device will depend on each person's abilities. Second, if the use of sensory substitution devices is both perceptual and cognitive, then this leaves room for perceptual strategies that will differ as a function of people's specificities and cognitive style. In other words, performance and experience with the device will then be done with different weights attributed to each of the sensory modalities. These weights will depend on people's individual capacities, perceptual preferences, and cognitive styles. Note that these weights might subsequently vary as a function of expertise, mastery of the device, and type of task. To give an example, it might be the case that a musician will be able to perform auditory analyses that non-musicians would not, consequently giving more weights to auditory processes, and possibly having a predominant auditory experience when using the device.

## II.2. Seeing the thunder while still hearing it

In order to start and investigate behaviourally the vertical integration hypothesis, we conducted an experiment in collaboration with Gabriel Arnold (post-doctoral researcher) and Jacques Pesnot-Lerousseau (master student). **Note that this study is included as the first of the selected articles in the Annex (Pesnot-Lerousseau, Arnold, & Auvray, submitted).**
In this study, blindfolded sighted participants were trained with the visual-to-auditory conversion device The vOICe (Meijer, 1992), which translates visual images into soundscapes (see Figure 2A). After training, with the blindfold removed, participants were tested to see whether auditory stimuli would spontaneously evoke visual images. To evaluate such potential involvement of visual processes, we took advantage of a Stroop-like paradigm (Stroop, 1935). In the original Stroop task, people are requested to name the colour of coloured words. When the meaning of the word is also a colour (e.g. the word “red” written in blue), it interferes with the participant’s recognition of the word’s colour. For instance, it takes longer to say that the colour of the presented word is blue when the written word reads “red” rather than “blue”. Thus, even if the task consists in naming the colour of a word, reading processes and the ensuing access to the words’ meaning are automatically triggered.

<figure><img src="image_4.png" /><figcaption>Figure 2. Illustration of the methods. (A) The visual-to-auditory conversion device (The vOICe). The images recorded by the video camera embedded in the glasses are converted in real-time into soundscapes that are presented to the participants via headphones. The device converts the vertical dimension into auditory frequency, the horizontal dimension into time scanning, and visual brightness into auditory loudness. (B) Description of the crossmodal interference task. (C) The auditory targets (spectrogram) and visual distractors that are used in the crossmodal interference task.</figcaption></figure>

The same rationale was used in our study. The participants completed a crossmodal interference task both before and after a 3-hour training session with The vOICe, which involved object identification, object localization, and orientation discrimination tasks. The crossmodal interference task consisted of a sound recognition task with a simultaneous presentation of visual images (see figure 2B). The sounds corresponded to the auditory conversion of visual lines with The vOICe. Visual distractors, presented simultaneously,
consisted in these abovementioned visual lines; some had the same orientation as the sound, whereas others presented a different one (see figure 2C). After training with the visual-to-auditory conversion device, if visual processes are automatically triggered upon hearing the sounds, i.e. if auditory stimuli are mentally converted into visual images, then the presentation of visual distractors should interfere with the participants' performance in the sound recognition task. Should this be the case, their performance ought to be facilitated when the auditory target and visual distractors have the same orientation, and disturbed when auditory target and visual distractors have different orientations. Conversely, if auditory stimuli are processed solely as sounds, i.e. without any involvement of visual processes, then visual distractors should not affect the participants' performance. In addition, in order to investigate the possible influence of auditory processes, the participants completed low-level auditory tests before and after training. Furthermore, prior training, and after training in each of the tasks, they were questioned about their subjective experience by means of Lickert scales (see figure 3, for the different steps of the procedure).

<figure><img src="image_5.png" /><figcaption>Figure 3. The different steps of the experimental procedure.</figcaption></figure>

The first and main result from our study, obtained in the crossmodal interference task, showed that, before training, the visual images did not influence the participants' responses. After training, however, they interfered with the auditory recognition task. In particular, they disturbed the participants' responses when the auditory soundscape did not correspond to the conversion of the visual image. This visual interference effect reveals a rapid functional plasticity, as users, once trained, can associate visual imagery to auditory stimuli (see figure 4).

Second, the participants' performance during training for localisation and recognition tasks, depended on their auditory abilities. The correlations between the participants' auditory scores and their performance with the device revealed that those participants with higher auditory abilities performed better during the training tasks with the device than those with lower abilities. Third, the participants' associated phenomenology differed as a function of
the task, as in Auvray et al.'s (2007) study. The overall phenomenology remained auditory, which is likely due to the fact that the stimuli consisted in sounds. However, in the object recognition task, the participants had additionally an important visual experience; whereas in the object localization task they had an additional sonar-like experience, that is, an experience mainly based on spatial information of localization or distance from the body, without necessarily forming visual images of the object to localize. Another interesting result is that such difference in the phenomenology as a function of the task – which probably results from the fact that several different processes are involved – is stronger in participants with good auditory capacities than in participants with low ones. It is likely that good auditory capacities allow the participants to identify those dimensions of the auditory signal that are the most relevant to the task while discarding the less relevant ones. Participants with low auditory capacities have a phenomenology which did not differ with the task, which suggests a lower plasticity in the involved processes. As their auditory inputs are poorer, it could be the case that they are less able to disentangle the different auditory dimensions and consequently to analyse them independently.

<figure><img src="image_6.png" /><figcaption>figure 4. Mean accuracy (percentage of correct recognition) in the crossmodal interference task across participants of the experimental and control groups, as a function of the Session (pre-training, post-training) and the Type of visual distractor (same, different, none). In the control group, pre-training and post-training sessions corresponded to the first and second sessions respectively, however without training with the device in between. Error bars represent standard error of the mean.

**p < .01.</figcaption></figure>

To summarize, the results of this study reveal that using a visual-to-auditory sensory substitution device induces plasticity at the functional and phenomenological levels. The association of the visual interference effect with the role of individual auditory abilities underlines the fact that functional plasticity is complex, and based on a multisensory architecture involving both visual and auditory processes. Our results show that, after
training, people visualize auditory stimuli, while still processing it auditorily. These results
are in line with the vertical integration thesis, which posits that using a conversion device
involves flexible behavioural processes and phenomenologies, with different weights
attributed to the sensory modalities as a function of task, context, expertise, and individual
differences (Arnold, Pesnot-Lerousseau, & Auvray, 2017). Learning to use a conversion
device thus relies on pre-existing perceptual capacities, and it can involve different
perceptual strategies, as a function of these pre-existing capacities. Note that these results
are also in line with the metamodal inverse hierarchy model (Proulx et al., 2014, 2016)
which proposes that the functional plasticity at stake occurs at higher levels, involving
supramodal representations of shapes with a spatial integration of visual and auditory
stimuli. More generally, our results reveal that sensory plasticity in humans is a complex
phenomenon which depends both on the kind of processes that are involved and on
individual specificities. To put it in William James' (1890) words, when using the visual-to-
auditory sensory substitution system, people become able to see the thunder while keeping
on hearing it.

Several questions remain open to scientific investigation. First, the question of whether the
interference appears at the visual or at a supramodal level remains to be investigated.
Second, we still need data on the questions of the influence of visual capacities on
performance with the visual-to-auditory conversion system and the role of visual imagery in
the processes (i.e., if we can obtain a crossmodal interference effect the other way round).
We are currently running the experiments corresponding to this second question.

II.3. Sensory substitution and learning

An important feature of the mastery of sensory substitution devices is their users' facility to
learn them and the amount of training they require to do so. For instance, with the TVSS, an
extensive training is required. Most users are able, in one or two hours, to explore their
environment, approximate objects' positions, and describe their raw shape. However, the
learning required to reach a more complete level of performance is estimated at around 8
hours with visual-to-tactile devices (Kaczmarek & Haase, 2003) and 10 to 15 with visual-to-
auditory devices (Auvray et al., 2007), and an even a longer time to perform fine grained
tasks, such as recognizing body postures (Striem-Amit & Amedi, 2014). It is thus crucial to
investigate users' learning abilities, to identify the most efficient learning methods and those
that are adapted to different users' profile. A crucial characteristic of learning lies in the
ability to generalize, that is, the ability to extend the acquired perceptual abilities to both
new stimuli and new perceptual conditions. With Gabriel Arnold (post-doctoral researcher),
we focused for this part on visual-to-tactile sensory substitution devices. We have conducted
behavioural research to investigate the extent to which tactile learning transfers across body
surfaces and the extent to which learning transfers to novel stimuli and to novel perceptual conditions.

Perceptual learning 1. Tactile letter recognition transfers across body surfaces

In order to investigate the extent to which tactile learning transfers across body surfaces, we conducted an experiment in collaboration with Gabriel Arnold (post-doctoral researcher).

**Note that this study is included as the second of the selected articles in the Annex (Arnold, & Auvray, 2014).**

One important claim of Bach-y-Rita is that, once trained, users of visual-to-tactile sensory substitution devices no longer feel the stimulation on their skin, where it occurs, but they directly attribute the stimulation as resulting from an external object, i.e. located at a distance (Bach-y-Rita & Kercel, 2003). Bach-y-Rita then asserted that, consequent to this externalization process, the tactile stimulator array can be moved from one body surface to another, without loss in spatial localization abilities or other perceptual capacity. This claim, although recurrent, was only based on users’ verbal reports and it has never been investigated with objective methods. Moreover, studies in the field of tactile perceptual learning suggest that performance improvement transfers only to body surfaces that are closely represented in the somatosensory cortex. These studies have however mainly used discrimination tasks of stimuli varying along only one feature (e.g., orientation of gratings) whereas, in sensory substitution, tactile information consists in more complex stimuli.

Our study aimed at investigating the extent to which there is a transfer of tactile learning in the recognition of high-level symbols (i.e., made of the combinations of several features rather than consisting of single features). To do so, a tactile letter recognition task was used. Letters were drawn on the participants’ body by means of sequential vibrotactile stimulations. The participants first completed a baseline session in which they had to recognize tactile letters drawn on three different body surfaces: the belly, the front of the right thigh, and the right shin (see figure 5). They then underwent a training session in which the letters had to be recognized in only one of the three abovementioned body surfaces. Finally, the participants performed a post-training session on all three possible body surfaces. The amount of tactile learning was evaluated by computing performance improvement (both accuracy and response times) between the baseline and the post-training sessions. If there is a transfer of tactile learning from a trained to an untrained body surface, then the amount of performance improvement should be similar for trained and untrained surfaces. On the other hand, if tactile learning is specific to a given body surface, then performance improvement should be greater for trained than for untrained body surfaces. In addition, a control group of participants performed the baseline and post-training sessions, but without the training session in between. For these participants, performance is expected
to be either similar between the two sessions, or to improve in the second one but less than
for the participants in the trained group.

<figure><img src="image_7.png" /><figcaption>Figure 5 (A) Example of the letter F drawn on the three body surfaces (belly, thigh, and shin). The letters' left-right and top-bottom axes were always congruent to the participants' left-right and top-bottom axes. (B) Schematic figure illustrating the 3 × 3 array of rectangular vibrators. (C) The 8 letters used as tactile stimuli.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Figure 6. Participants' (A) accuracy and (B) response times for the trained and untrained body surfaces obtained during the baseline session (in white) and the post-training session (in black). Error bars represent the standard errors of the means.</figcaption></figure>

The results of our study revealed that training improved both participants' accuracy and response latency (see figure 6). In addition, participants performing intensive training with trial-by-trial feedback (trained group) improved performance to a greater extent than the participants merely repeating the task twice on each body surface (control group). Second, performance improvement was not restricted to the trained body surface but it transferred to the untrained ones. Third, the obtained transfer of tactile learning was similar for surfaces represented in adjacent (e.g., belly and thigh) and in non-adjacent (e.g., belly and shin) areas
of the somatosensory cortex (see Merzenich et al., 1978; Nakamura et al., 1998, for descriptions of the somatosensory cortex's topographic organization). Taken together, these results reveal that there is a transfer of learning in tactile letter recognition, which occurs independently of the distance between body surfaces. These three results provide support for Bach-y-Rita's claim that training in sensory substitution improves perceptual abilities, and that these perceptual abilities transfer from one body surface to another, resulting in a relative independence from the stimulated body surface (Bach-y-Rita & Kercel, 2003).

These results have implications for the use of sensory substitution devices and vibrotactile systems by visually impaired people. First, they suggest that training on one body surface is beneficial to the entire body. As a consequence, users do not need to undergo an extensive training on all body surfaces. This study also provides better understanding of the learning process, which is a crucial feature for efficient use of tactile devices. It reveals that transfer of learning for tactile letters occurs within a relatively short time frame, i.e., the 90 min of our experiment. The longer learning reported for recognition of more complex objects with sensory substitution devices (Kaczmarek & Haase, 2003) indicates that the amount of training required to reach accurate object recognition with tactile devices depends on the complexity of the information that is provided.

## Perceptual learning 2. Generalization to novel stimuli and novel perceptual conditions, specificity versus generalization of tactile learning

In order to investigate the extent to which tactile learning transfers to novel stimuli and to novel perceptual conditions, we conducted an experiment in collaboration with Gabriel Arnold (post-doctoral researcher). **Note that this study is included as the third of the selected articles in the Annex (Arnold, & Auvray, 2018).**

In this study, we more specifically investigated the perceptual learning of tactile alphanumerical stimuli. The learning protocol consisted in alternating a repeated list of symbols with lists of new symbols. The first experiment investigated the role of stimulus variability in the ability to generalize to new symbols during the learning of sets of four symbols. The results showed that recognition performance improved over time only for the repeated list. This result suggests that learning a small set of stimuli involves stimulus-specific learning strategies, preventing generalization. A second experiment revealed that increasing to six the set of learned stimuli results in higher generalization abilities. This result can be explained by greater difficulties in using stimulus-specific strategies in this case, thereby favouring the use of generalization strategies. Feature variability also appeared to be important to achieve generalization. Thus, as in visual perceptual learning, the involvement of stimulus-specific versus general strategies depends on task difficulty and
feature variability. A third experiment highlighted that tactile perceptual learning generalizes
to changes in orientation.

These different results for small and larger sets of stimuli can be accounted for by the
reverse hierarchy theory (RHT) of perceptual learning (Ahissar & Hochstein, 2004).
According to this theory, the degree of specificity depends on the difficulty of the perceptual
task and on the level of cortical processing that is required to perform the task. For a difficult
task, involving low-level cortical processes that are highly specific to the characteristics of
the task (e.g., fine discrimination of visual orientations), perceptual learning is specific to the
characteristics of the task (e.g., the visual orientations that have been discriminated) and it
does not generalize to new characteristics (e.g., new orientations). On the other hand, for an
easier task, involving higher-level processing, perceptual learning is less specific to the low-
level characteristics of the task or of the stimuli. Task difficulty, either during the learning
phase or during the evaluation of generalization, has indeed been reported to influence both
the specificity and generalization of learning (Jeter et al., 2009; Liu & Weinshall, 2000;
Wang, Zhou, & Liu, 2013).

This study has applied implications for the design of learning protocols for sensory
substitution. In particular, the results suggest that short training sessions with high
variability and diversity will facilitate generalization and will allow for a more optimal use
of the device in real-life conditions. However, one remaining question is how to achieve
complete generalization, that is, how to rapidly gain optimal level of performance for
entirely new objects. One of the possible cues is obviously the amount of training. Daily use
of the device would allow users to become experts. Learning would also benefit from a
highly diversified use of the device, with perception of different categories of complex
objects and of different objects of the same category.

II.4. Crossmodal correspondences for the optimization of the devices

In this series of experiments, our aim was to use crossmodal correspondences in order to
determine the most optimal codes for sensory substitution devices. This research was
conducted in collaboration with Ophelia Deroy (University of London), Irène Fasiello, and
Vincent Hayward (ISIR, UMR 7222). **Note that this study is included as the fourth of the
selected articles in the Annex (Deroy, Fasiello, Hayward, & Auvray, 2016).**

One crucial question when designing sensory substitution devices is how to code at best
visual dimensions into tactile or auditory dimensions. With visual-to-tactile generalist
devices (i.e., excluding those coding only for distance, e.g., O'Brien, Auvray, Hayward,
2015), most of the time an analogical code is used, for instance, a visual circle gives rise to a
circular pattern of vibrations (e.g., the TVSS, see White et al., 1970). With visual-to-
auditory devices, several dimensions of the visual information (most of the time horizontal and vertical dimensions, depth, brightness, and sometimes colour) are coded using several auditory dimensions (most of the time pitch, time scanning, inter-aural differences, loudness, and timber). The question arises as to how to determine the pairings between auditory and visual dimensions? The four main visual-to-auditory devices (e.g., the vOICe, Meijer, 1992; The PSVA, Capelle et al., 1998; Vibe, Hanneton, Auvray, & Durette, 2010; the EyeMusic, Abboud et al., 2014) have been designed by using the correspondence between direction of pitch and direction of movement (higher-higher; lower-lower). However they haven't tested whether this correspondence, intuitive for the designers and well-documented in audiovisual contexts for sighted individuals (see Parise & Spence, 2012; Spence, 2011) was also relevant for blind people.

In order to investigate the relevance of the pitch-elevation correspondence for blind persons, we had to test the audio-tactile analogue of this correspondence. Across four experiments, we tested the interactions between the direction of tactile movement (inward vs. outward movement on the fingertip) and changes in auditory frequency (increasing vs. decreasing pitch), see figure 7 for the description of the device used.

<figure><img src="image_9.png" /><figcaption>Figure 7. Tactile stimulator device and representation of the tactile stimulus.</figcaption></figure>

A first experiment investigated this correspondence using a selective attention method, requiring participants to focus on tactile or auditory signals while ignoring the other one presented simultaneously. The results with this method did not reveal any significant compatibility effect, suggesting that the effect does not occur at an explicit level. In order to investigate if this effect occurs at the implicit level, we used a variant of the implicit association task (IAT, e.g., Parise & Spence, 2012). In this variant of the IAT, blindfolded sighted participants were required to focus on one stimulus at a time, while the association was only at the level of response buttons. Four stimuli were sorted using only two response buttons, each of which refers to two of the four stimuli. The compatibility effects between intuitively congruent pairs of stimuli (i.e., outward tactile movement, going from the inside of the finger toward the fingertip and increasing pitch, or inward tactile movement and
decreasing pitch) and incongruent pairs stimuli (i.e., the reverse associations) were measured. The results revealed a significant congruency effect between changes in pitch and changes in direction of tactile movement. Participants were for instance faster in responding to a tone increasing in pitch when the response button was shared with the outward tactile movement than when it was shared with the inward tactile movement.

A third experiment showed that this effect was similar in the conditions where the arm was placed vertically and horizontally, that is whether or not the inward-onward tactile movement corresponded to the free movement of an object subjected to gravity. Thus, a rising pitch is associated to a tactile movement going toward the fingertip, and a descending pitch to a tactile movement going inward toward the palm, independently of the hand being oriented upward or horizontally. This result is consistent with the correspondence occurring in a hand-centred frame of reference. Finally, the fourth experiment investigated the correspondence in blind persons. The results revealed that neither early nor late blind persons were sensitive to it. In addition, there was no effect related to the age of onset. Thus, vision seems to be necessary to mediate the correspondence between direction of pitch and direction of tactile movement (see figure 8).

<figure><img src="image_10.png" /><figcaption>Figure 8. Participants' mean reaction times in Experiments 3 and 4, for the two conditions of stimulus presentation: tactile and auditory, and for the three congruency conditions: congruent, unimodal, and incongruent. Error bars represent the standard errors of the means. Note that, as there was no effect of arm position, in the graph, horizontal and vertical arm positions were collapsed.</figcaption></figure>

The results of this study reveals that the audio-tactile correspondence between direction of pitch and direction of movement exists in sighted participants along both the horizontal and the vertical plane, but it is absent in early and late blind people. These results have methodological implications for the testing of crossmodal correspondences and for the design of sensory substitution devices. They indeed suggest that the IAT might be an interesting tool to pre-test the relevance of certain codings or combinations of auditory and tactile cues, to tailor them more specifically to their users. Our aim now is to investigate more systematically the possible crossmodal correspondences in audition, vision, and touch, focusing on the dimensions of the signals that are relevant for sensory substitution.
## II.5. Prototypes of sensory substitution devices

From an applied point of view, the challenges that are faced involve finding the best way to provide users with information usually allowed by vision as well as the best way to ergonomically adapt the devices to their users. I collaborated to the design and testing of three prototypes of sensory substitution devices, with the aim to target specific perceptual functions (localization, recognition, navigation) that can be impaired in blind people. I indeed believe that one of the reasons of the under-use of sensory substitution devices lies on the unrealistic promises of a generalist device that would restore the entirety of a deficient sensory modality. By contrast, blind persons more easily use technical devices devoted to a specific function (such as auditory or tactile software to navigate the internet).

**The device Vibe.** In collaboration with Sylvain Hanneton (LNP, UMR 8119), we have designed and tested, through a pointing task, a sensory substitution device that allows the real-time conversion of visual images into an audio stream (Hanneton, Auvray, & Durette, 2010). This device converts a video stream into a continuous stereophonic audio signal that conveys information coded from the video stream. The conversion from the video stream to the audio stream uses a kind of retina with receptive fields. Each receptive field drives a sound source and users listen to a sound that is a mixture of all these sound sources. Compared to other existing visual-to-auditory sensory substitution devices, Vibe is highly versatile in particular because it uses a set of configurable units working in parallel, for instance, the receptive fields of receptors can be large or not, identical or not, overlapping or not. In addition, The Vibe offers the possibility to enhance the binaural perception and differentiation by the listener of the sound sources in the plane of the picture by adding interaural disparity cues to the sounds like ITD (interaural time difference) or ILD (interaural level differences).

**The NAVIG project** (navigation aided by artificial vision and GNSS), funded by the French Agency ANR aimed at designing a navigation aid device for blind persons, using an auditory feedback. It involved GPS and artificial vision systems. The project was conducted in collaboration with three research laboratories (Irit, Cerco, and LIMSI), two companies (Spikenet Technology and Navocap), and an education center for blind persons (Institut des Jeunes Aveugles). At the LIMSI, in collaboration with Michel Denis (UPR 3251), Mathieu Gallay (post-doctoral researcher), and Lucie Brunet (master student) we focused more precisely on two research axes. First studying the processes underlying the representation of space that are specific to blind persons using navigation aid devices and a study specifying blind people's needs when navigating as well as whether those needs are satisfied by the prototypes designed in the project (Brunet, Darses, & Auvray, 2018; Gallay, Denis, & Auvray, 2013; see also Katz et al. 2012a&b, for the results obtained by the consortium).
**Visuo-tactile conversion system.** With the company Caylar and Gabriel Arnold (post-doctoral researcher), we received a funding from the CFI (Centre Francilien de l'Innovation), to develop a sensory substitution device involving 80 tactile vibrators, based on the tactile technology developed by the company. The aim was to design an interface highly configurable, so that each user can specify different options of the presentation of information, with the aim of gaining at best mastery of the device. For this device we had the chance to have three second year master students whose project was to evaluate the existing commercialized devices, the possibilities of intellectual property for these types of devices, and the industries involved in the project. After testing the first prototype, we agreed with the company on the adjustments to make and the final device is now developed and ready to be tested.

## II.6. Conclusion on sensory substitution

Although sensory substitution appears as a promising technology to compensate visual impairments, so far the developed devices remain barely used by visually impaired people, either because their design is not optimal, or because their use involves too many constraints and cognitive effort. Taking into account individual differences, as suggested by the vertical integration hypothesis, will allow an optimal appropriation of these devices. The learning protocols will also benefit from taking into account specific and generalized conditions of learning in order to favour the transfer of learning to novel stimuli and novel perceptual conditions.

In addition, identifying different perceptual strategies associated to different phenomenologies, with some people having more of a visual use of the device and other people having more of an auditory or tactile use, can have implications on the design of learning protocols. One possibility would be to respect each user's tendency and to customize the learning protocols as a function of the different types of use. For instance, for people having a visual use only, learning could be oriented toward integrating a substitution device so that the experience becomes increasingly visual. This can be done by favouring tasks designed to better apprehend depth, to learn how to form global images of the objects. For people having more tactile or auditory experiences, learning could be oriented toward favouring the processes specific to these modalities. For instance, learning to better discriminate auditory frequencies and auditory intensities will allow a better discrimination of the complex auditory stimuli coming from a visual-to-auditory device. Finally, the choice of the conversion codes will benefit from the identification of the crossmodal correspondences that may exist in the target population.

Regarding the conceptual aspects, the collected data helps making advances in the understanding of the processes involved when using a sensory substitution device. The
results suggest that performance with the device rely on users' individual sensory abilities and that experience takes roots both on the substituting and the substituted sensory modalities. The data collected on the subjective experiences, even if they remain too sparse to make strong claims, revealed that they are subject to inter- and intra- individual variability. The fact that there is dependence on both the task and individual abilities suggests that there is room for processes that are more cognitive, and thus more subject to individual variability. These results are in line with the view that sensory substitution devices are vertically integrated on pre-existing capacities that involve cognitive and perceptual processes belonging both to the substituting and to the substituted sensory modalities.

### III- Multisensory spatial reference frames

The second axis of my work investigates the multisensory groundings of our spatial representations, which vary as a function of the sensory modalities that are involved. This research axis was funded by a grant from the French agency ANR (FRESCO) and a post-doctoral funding grant (Fondation des Aveugles de Guerre). The research was conducted in collaboration with Gabriel Arnold, Jessica Hartcher-O'Brien (post-doctoral researchers), Charles Spence (Oxford University), Vincent Hayward (ISIR, UMR 7722), Jérôme Dokic, and Frédérique de Vignemont (IJN, UMR 8129).

#### III.1. The variability in our reference frames

For the story, at the beginning, this research axis was initiated to investigate the processes of distal attribution in sensory substitution, which will be further described below, in section III.8. The idea was to investigate how people, through training with a visual-to-tactile sensory substitution device, transitions from sensing the tactile stimuli on their body to perceiving the object as being located in front of them, in the 3-D space. One way to do so was to study the changes in the reference frames that they used before and after training. However, the results we started to obtain appeared far more complex than what we anticipated, and the more we dug into it the more it appeared fascinating, until becoming a research axis per se.

The impression of perceiving the external world from a single spatial origin, i.e., the self, requires a spatial unification of the multisensory information that comes to our body through the different sensory modalities (see Denis, 2018, for a review of the core aspects of spatial cognition). For instance, a fly is often perceived in a multisensory way: we can see it, we can hear the sound of its wings if its flying, and we can feel it when it lands on our body. However, we perceive not only three distinct stimuli but also one single unified object, the fly. Multisensory information is thus integrated in space and time, in order to perceive an external world that is spatially organized (Boyer et al., 2013; Meredith & Stein, 1986). Such
perception of a unified external world is a complex process because spatial reference frames
are very different from one sensory modality to another. Visual information is initially coded
in retinotopic coordinates<fnref n="1" />, whereas tactile information is initially coded in somatotopic
coordinates, and auditory information is initially coded according to a head-centred
reference frame. Spatial diversity is also present within one sensory modality. For instance,
the spatial coordinates of tactile stimulation can be defined according to the stimulated body
part, to the entire body, or to the external world. These different reference frames can
conflict, as revealed by the longer time needed to report which hand, left or right, has been
stimulated first when the arms are crossed as compared to uncrossed (Shore, Spry, &
Spence, 2002). The additional time taken for localizing touch in the crossed-hand condition
has been interpreted as reflecting a conflict between a body-centred and an environment-
centred reference frame.

Regarding these spatial perspectives, the apparent primacy of self-centred perspectives has
been challenged by studies revealing the extent to which people can adopt spontaneously
self-centred versus other-centred perspectives (Arnold, Spence, & Auvray, 2016; Viaud-
Delmon & Warusfel, 2014). On the one hand, as interoception is mainly body-centred,
adopting a self-centred perspective on external stimuli is crucial for self-consciousness, as it
allows for the integration of both interoceptive and exteroceptive information into a common
egocentric body-centred reference frame (Blanke, 2012; Petkova, Khoshnevis, & Ehrsson,
2011). On the other hand, the adoption of a perspective that is decentred from the body can
be advantageous in the case of perspective-taking, when it comes to interacting and
communicating with others. As a consequence, it is important to understand what underlies
the adoption of self-centred versus decentred perspectives and how they are integrated to
give the impression of perceiving the external world from a single spatial origin.

In the experiments and theoretical work we conducted, we focused on investigating the
spatial perspectives that are adopted in tactile perception. This is of particular interest to the
study of bodily self-consciousness, especially when targeting the processes that integrate
bodily stimulation with external stimuli (e.g., Aspell et al., 2013). Another specific
characteristic of tactile perception is the spatial coincidence between the perceiver and the
object of perception. Unlike visual stimuli, which are presented at some distance from the
perceiver, tactile stimuli occur at the same location in space as the perceiver, i.e., the spatial
location of the body. As a consequence, when interpreting tactile stimulation received on the

<footnote n="1">¹ More precisely, after this initial coding, visual information is then either maintained in retinotopic coordinates or it can be transformed into external, craniotopic, or motor coordinates depending on the task and goal of the observer, whereas for touch several coordinate systems already compete at early stages of somatosensory processing (Heed et al., 2015).</footnote>
body surface, self-centred and decentred perspectives conflict with one another and there is a choice to be made between perceiving tactile stimulations from the body location (i.e. self-centred perspective) and from an external location (i.e. decentred perspective). It should be mentioned that these different self-centred perspectives can correspond to a mental localization of the tactile stimulation either on the body (proximal attribution) or on the external space (distal attribution, see figure 9).

<figure><img src="image_11.png" /></figure>

Figure 9: Illustration of the different spatial perspectives that can be adopted in the graphesthesia task, with the example of the tactile letter “b” drawn by the experimenter. (a) When the letter is drawn on the participant’s forehead, some participants perceive the letter “b”, assigning the top-bottom axis of the letter in the same direction as their own top-bottom axis, but the left-right axis in the direction opposite to their own left-right axis. These assignments may have resulted from a decentred perspective whose origin is located in front of the participant’s head. Other participants will perceive the mirror-reversed letter “d” instead, assigning the top-bottom and left-right axes in the same direction as their own body axes. These assignments may result from a self-centred perspective whose origin is located inside the head. (b) When the tactile letter is drawn below the head, here on the stomach, a third perspective centred on the head can be adopted, as if the head was bending forward to “see” the letter on the stomach. As a consequence of this bending-forward projection, the top-bottom axis of the symbol is assigned in the direction opposite to the participant’s own top-bottom axis and the participant perceives the 180°-rotated letter “q”.

In a review article (Arnold, Spence, & Auvray, 2017), we focused on the role of tactile spatial perspectives in self-consciousness. We review those studies that have used the graphesthesia task, that is, the task of recognizing ambiguous symbols (e.g., the letters b, d, p, and q) drawn on the body surface. The graphesthesia task provides an optimal paradigm with which to study the perspectives – self-centred and decentred – that are spontaneously adopted when interpreting bodily stimulations. Take the example of the lowercase letter “b” drawn on the forehead by an experimenter facing the participant (see figure 9). Recognizing
the letter “b” as drawn by the experimenter requires the participant to take a decentred perspective (i.e., decentred from the participant’s position and centred on that of the experimenter). However, because the lowercase letter “b” is ambiguous, it can also be recognized as the mirror-reversed letter “d”, from the participant’s self-centred perspective, as if the letter were to be projected forward and “seen” from the participant’s position. The crucial advantage of using these ambiguous symbols is that an intrinsic orientation of the symbol is impossible to determine and thus one specific perspective has to be taken in order to interpret the symbols. Consequently, when participants try to recognize these symbols, the pattern of their responses can be used to infer the way in which they have assigned the different axes to the stimulus and the perspective they have adopted.

Thus, the graphesthesia task can be used as a measure of self-localization in the body. For instance, in Ferrè et al.’s (2014) study, the participants had to recognize the letters b, d, p, and q, drawn on their forehead by the experimenter, while receiving galvanic stimulation of the vestibular system. The authors hypothesized that the vestibular system plays an important role in binding together multisensory information into an egocentric body-centred reference frame. When receiving galvanic stimulation of the vestibular system, participants adopted more of a self-centred perspective than a decentred one, compared to a sham stimulation in which a galvanic stimulation was provided on the neck’s skin, which provoked light tactile sensations without vestibular stimulation. This bias toward adopting a self-centred perspective with galvanic stimulation was interpreted as a reinforcement of the processes of the vestibular system, consisting in anchoring the self to the body (see also Deroualle, 2015; Pavlidou, Ferrè, & Lopez, 2018, on the influence of vestibular stimulation on the adoption of the egocentric reference frame). The graphesthesia task, as a measure of which perspectives are spontaneously adopted, can thus provide a reliable indicator of the potential disruption of the link between the self and the body, as people experiencing out-of-body experiences have been reported to adopt more spontaneously decentred perspectives than others (Blackmore, 1987).

The results from the literature also reveal that the adopted perspectives vary significantly with spatial factors (such as the location and orientation of the stimulated body parts), personal factors (such as gender, cognitive style, self-focused attention, spatial abilities), and on interpersonal factors (such as conflict or dominance in relationships, see Arnold, Spence, & Auvray, 2017, for the details of these variations). Such results suggest that the self can adopt a multiplicity of spatial locations. However, the unity of the self can partly be explained by the predominance of a head-centred perspective. Indeed, spatial information coming from the different senses can be integrated into one common and unified reference frame, usually an eye-centred reference frame, in the goal of performing actions (Cohen & Andersen, 2002). Spatial information coming from vision (Boussaoud & Bremmer, 1999),
audition (Zwiers, Van Opstal, & Paige, 2003), and touch (Harrar & Harris, 2010; Ho & Spence, 2007; Pritchett & Harris, 2011) has indeed been reported to be coded according to the direction of the eyes. The spatial transformation of multisensory information into a common eye-centred reference frame may contribute to the perception of a unified external world rather than multiple worlds. As a consequence, it can also induce the feeling of having one single unified self rather than multiple selves because the world is perceived from one single origin, i.e., from a self-centred perspective.

On the other hand, perspective-taking abilities contribute to the distinction between self and others, reinforcing self-consciousness. Indeed, if self-centred perspectives have primacy in our perception of the world, one can also adopt a perspective that is decentred from that of the self. A decentred perspective can correspond to the spatial viewpoint of another person (i.e., alter-centred, second-person, or third-person perspective), or simply centred at a different location that is not necessarily occupied by another person. Adopting different perspectives therefore facilitates imagining how the environment would be perceived from another point in space, and, importantly, understanding how the environment is perceived by others. The graphesthesia task thus appears as a promising tool to investigate the reference frames that can be adopted. From an experimental point of view, many questions remain to-be-investigated. We particularly focused on the questions of (III.2) whether the adopted perspective is natural or cognitive choice, (III.3) the influence of vision on the reference frames that are adopted, (III.4) the question of its embodied nature, (III.5) the role of proprioceptive information on spatial perspective taking, (III.6) the question of the transfer of a learned reference frame, (III.7) the specificities of the reference frames adopted on the hand, and (III.8), the processes of distal attribution.

### III.2. Adopting a reference frame, a natural process or a mere cognitive choice?

In order to investigate the extent to which adopting a given reference frame is a natural process or a mere cognitive choice, we conducted an experiment in collaboration with Gabriel Arnold (post-doctoral researcher) and Charles Spence (Oxford University). **Note that this study is included as the fifth of the selected articles in the Annex (Arnold, Spence, & Auvray, 2016).**

In this study, the participants were presented with the lowercases letters b, d, p, and q. These letters were presented by means of a matrix of 9 vibrators placed on the participants' stomach. At this location, three different perspectives can be adopted (see figure 10), a decentred perspective oriented toward the participant's stomach (response b); a trunk-centred perspective oriented forward the participant (response d); a head-centred perspective, as if the head was bending forward to "see" the tactile stimulation (response q).
The participants were requested to respond, for each stimulus, which of the four letters they perceived.

<figure><img src="image_12.png" /><figcaption>Figure 10. On the left, illustration of the three possible perspectives that participants can adopt when interpreting ambiguous symbols displayed on the body surface. On the right, proportion of the participants adopting each of the perspectives.</figcaption></figure>

The experiment was composed of three sessions. In Session 1, the participants were free to adopt any perspective that they wanted to recognize letters. In Sessions 2 and 3, they were instructed to adopt a specific perspective. In one of these two sessions, the imposed perspective was the same as in Session 1. In the other session, the imposed perspective was different. Half of the participants performed Session 2 with the same perspective as in Session 1 and performed Session 3 with the different one. The other half did the opposite. These imposed perspectives allowed for the evaluation of any cost associated with adopting an unnatural perspective. If the perspective that is adopted freely is natural, then imposing a different perspective should produce a cost in terms of recognition performance. In addition, in order to evaluate whether the cost of adopting an unnatural perspective is simply explained by changes in perspectival instructions or by the difficulty that is associated with disengaging from a perspectival choice, we evaluated whether performance would improve when the participants returned to the natural perspective after adopting an unnatural one.

We first computed the proportions of the participants that adopted each of the perspectives. To do so, we included the 80 participants who performed the present experiment together with a further 170 participants who performed the same first session and were then included in other studies. On these 250 participants, 49.6% adopted the trunk-centred, 29.2% the
head-centred, and 21.2% the decentred perspectives. Thus, around 4 out of 5 participants adopted a self-centred perspective (centred either on their head or on their trunk), whereas only 1 out of 5 adopted a decentred perspective (see figure 10).

We then computed the participants' mean accuracy and response times in each of the three blocks (i.e., on the 80 participants of this study). Results from the ANOVA revealed that recognition performance decreases with an unnatural perspective and returns to its previous high level with a natural one (see figure 11). This result supports the view that the perspective adopted is natural. In addition, when a different perspective was imposed, the participants made more errors corresponding to the adoption of their natural perspective than other errors. Some of the observers perceive spatial relations better from a self-centred perspective whereas others perceive better from a decentred perspective. Interestingly, the results also revealed a greater cost for self-centred participants to adopt a decentred perspective than for decentred participants to adopt a self-centred perspective.

<table><thead><tr><th></th><th>Session 1<br>free</th><th>Session 2<br>imposed</th><th>Session 3<br>imposed</th></tr></thead><tbody><tr><th>same-different</th><td>free</td><td>same</td><td>different</td></tr><tr><th>different-same</th><td>free</td><td>different</td><td>same</td></tr></tbody></table>

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /><figcaption>Figure 11. Participants' accuracy (percentage correct) and response times as a function of Block and Order of imposed perspectives (same-different, different-same). For both accuracy and response times, the performance decreased significantly when imposing a different perspective and increased significantly when returning to the same perspective as in Session 1.</figcaption></figure>
To summarize the results of this study, perceivers do not adopt the same perspective when interpreting ambiguous spatial information. Some persons prefer to perceive space from their own centred point of view whereas others prefer the point of view of another person (20% of the cases in our study). In addition, this perspective corresponds to a natural reference frame rather than a mere cognitive choice.

### III.3. The influence of vision on the reference frames that are adopted

In a second study, conducted in collaboration with Gabriel Arnold (post-doctoral researcher), our aim was to investigate the influence of vision on the reference frames that are adopted (Arnold & Auvray, in preparation). To do so, early blind, late blind, and blindfolded sighted participants completed an experiment with same protocol as in Arnold, Spence, & Auvray's 2016 study (described in section III.2), with however some minor adaptations so that it can be completed without sight.

The results on the proportions of responses revealed that blind participants adopted less frequently decentred perspectives than blindfolded sighted control participants ($\chi^2$ (2) = 7.33; *p* = .026). There was no significant difference between early and late blind participants ($\chi^2$ (2) = 1.60; *p* > .448). We then computed the proportions from the three groups of participants in this study, which were labelled as “without vision” condition (early blind, late blind, and blindfolded sighted participants) and we compared them to the results obtained in our previous study, a group which were labelled “with vision”. The participants without vision adopted more frequently an egocentric perspective corresponding to localizing the tactile stimulus on the body. On the other hand, the participants with vision adopted more frequently an egocentric perspective corresponding to a projection of the tactile stimuli out of the body ($\chi^2$ (2) = 24.15; *p* < .001), see figure 12.

<figure><img src="image_15.png" /><figcaption>Figure 12. On the left, proportions of the early blind, late blind, and blindfolded sighted participants adopting each of the three perspectives in this study. On the right, proportions of the participants with vision and without vision (early, late blind, and blindfolded sighted) adopting each of the three perspectives.</figcaption></figure>
It should be underlined here that the different self-centred perspectives can correspond to a
mental localization of the tactile stimulation either on the body (proximal attribution) or on
the external space (distal attribution). Here, the head-centred perspective corresponds to the
proximal attribution (the stimuli are considered by the participants to be on the skin) and the
trunk centred perspective corresponds to the distal one (the stimuli are considered by the
participants to be projected in front of them; see figure 9 for a schematic illustration of the
different axes of perception and projections).

We finally computed the cost in accuracy when the participants changed from their preferred
perspective to an imposed different one. Early-blind participants showed a greater cost in
accuracy when adopting an unnatural perspective than sighted and late-blind participants
(t(133) = 3,00; p = .003), there was no significant difference between sighted and late blind
participants (t(109) < 1).

To summarize, the results of this study revealed that vision and prior visual experience
influence the perspectives that are adopted to interpret tactile stimulation received on the
body surface. First, blind people adopt less frequently the perspectives that are not centred
on their body (i.e., decentred perspectives). This highlights the important role of vision for
spatial cognition and perspective-taking (see Afonso et al., 2014; Pasqualotto et al., 2013,
for similar claims with different methodologies). Second, sighted people are more prone to
adopt an egocentric perspective which corresponds to projecting the tactile stimulation in
front of their body when they can see during the task. However, when they are temporarily
deprived of vision by a blindfold, they are biased toward an egocentric perspective which
corresponds to localizing the tactile stimulation on the body. The spatial perspectives that
are adopted in touch are thus influenced both by the lack of vision, be it from birth or from
several years, and by a temporary deprivation of vision.

Note that this study has implications for the design of tactile devices conceived to
compensate visual impairments, as the results reflect the importance to take into account
individual specificities. These individual specificities are particularly crucial for early-blind
people, as they show greater difficulties to adopt an unnatural perspective. One possible
solution to overcome this consists in adapting the tactile device to their preferences in
orienting left-right and top-bottom of tactile stimulation. In addition, our hypothesis, which
is planned to-be-tested, is that blind persons using a visual-to-tactile sensory substitution
device might favour a trunk-centred perspective as compared with blind persons not using
one, who will remain using preferentially the head-centred perspective.
### III.4. The embodied nature of spatial perspective taking

In the third study, conducted in collaboration with Gabriel Arnold (post-doctoral researcher), our aim was to investigate whether adopting a head-centred perspective on the body is an embodied process (Arnold & Auvray, 2017). The head-centred perspective, which consists of mentally rotating the head toward the stimulated surface, is particularly interesting for investigating embodied perspective-taking as it involves mentally changing the body posture. Crucially, such a head-centred perspective has been described to be constrained by real body movements (Sekiyama, 1991).

In this study, the participants were asked to adopt a head-centred perspective to recognize the letters b, d, p, and q, presented on eight body surfaces (see Figure 13). They were instructed to do not look at the stimulated surface and to keep a standing position with the head oriented forward. They were asked to indicate which letter was recognized. The results of the analyses conducted on accuracy (percentage of correct responses) and response times (RTs) showed that adopting a head-centred perspective on the body is easier for surfaces that can be easily looked at with real head movements. The head-centred perspective was harder to adopt for surfaces that were either far away from the head (e.g., the shin), physically impossible to be directly looked at (e.g., the back), or necessitating a greater quantity of movements (e.g., the sides compared with the front and the legs compared with the trunk).

<figure><img src="image_16.png" /><figcaption>Figure 13. Mean accuracy and RTs for the eight stimulated surfaces.</figcaption></figure>

Overall, the results of this study support the embodied nature of perspective-taking. They show that mentally taking a spatial perspective on our own body involves a mental change in
body posture that is influenced by physical body parameters. More generally, they highlight
the important role the body plays in perception.

### III.5. The role of proprioceptive information on spatial perspective taking

In the fourth study, conducted in collaboration with Gabriel Arnold, Laura Fernandez (post-doctoral researchers), and Fabrice Sarlegna (UMR 7287) our aim was to investigate the influence of proprioception on the adopted reference frame (Arnold et al., submitted).

Previous studies highlighted that modifications of people's information relative to their body modify the reference frames they adopt. For instance, as was mentioned in section II.1, Ferrè et al.'s (2014) study using the graphesthesia task showed that when receiving galvanic stimulation of the vestibular system, participants adopted more of a self-centred perspective than a decentred one, compared to a control condition. The control condition consisted in galvanic stimulation provided on the neck's skin, which provoked light tactile sensations without vestibular stimulation. The authors suggested that the vestibular system plays an important role in binding together multisensory information into an egocentric body-centred reference frame (see also Lopez, 2016, for a review of the role of vestibular information on spatial cognition). On the other hand, according to the authors, perturbation of the vestibular system may disrupt the unity of the self and the body, giving rise to more decentred perspectives. We made the hypothesis that similarly, proprioceptive deficits might bias toward adopting a decentred perspective.

We used the graphesthesia task (this time with the letters drawn manually) in two deafferented patients (IW and GL) that have neither tactile nor proprioceptive perception below the head and a group of matched control participants that have no perceptual deficits. The ambiguous letters were drawn on the participants' forehead, left, and right surfaces of the head. This was done when their head was oriented forward, leftward, and rightward. We used these surfaces as, when it comes to adopt a spatial perspective, such as deciding whether an object is located to the body's left or right, the global orientation of the self has been shown to correspond to a combination of trunk and head orientations (Alsmith & Longo, 2014). The results obtained in control participants (N=20) revealed that, when the forehead was stimulated, a great majority of responses corresponded to adopting a self-centred perspective. For side surfaces, when the head was oriented forward, there was no preference for adopting a self-centred perspective or a decentred one. However, when the head was oriented leftward or rightward, the side surfaces of the head were aligned with the front and back surfaces of the trunk and there was a significant preference for the self-centred perspective. This confirms that in healthy participants the head and the trunk both play a role in orienting the self. Different results were obtained with deafferented patients, who did not show any effect of the stimulated surface nor of the head orientation on the perspective they took. This suggests that
their physical body does not influence the orientation of their self. Rather, these patients rely on external stimulation and they adopt strategies that are more cognitive and more prone to inter-individual differences. One of the patients consistently adopted a self-centred perspective in all conditions whereas the other consistently adopted a decentred perspective. These results are in line with the inter-individual differences we previously obtained with the same deafferented patients completing a spatial navigation task (Renault, Auvray, Parseihian, Miall, Cole, & Sarlegna, 2018). This study highlights the crucial role of proprioceptive information in defining the location and orientation of the experienced self. Thereby it allows shedding light on the contribution of the different sensory modalities on how the self is experienced.

### III.6. The transfer of a learned reference frame

In this fifth study, conducted in collaboration with Jess Hartcher-O'Brien (post-doctoral researcher), we investigated the perspectives taken on the hand as a function of its orientation, whether the perspective is malleable and can be updated to a novel perspective via adaptation, and finally whether adaptation on one surface transfers to non-adapted surfaces. Note that this study is included as the sixth of the selected articles in the Annex (Hartcher-O'Brien & Auvray, 2016).

<figure><img src="image_17.png" /><figcaption>(b) Tested orientations of the hand relative to the observers head</figcaption></figure>

<figure><img src="image_18.png" /><figcaption>(c) Adaptation task to mirror reverse perspective</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>(d) Post-test generalisation across fingers and body midline</figcaption></figure>

<figure><img src="image_20.png" /></figure>

Figure 14. Cutaneous patterns, potential perspectives, and fingertip orientations adopted during the experiment. (a) The device pin activation pattern for the letter b. The interpretation of the pattern as a b required assigning external coordinates to the activation pattern. An interpretation for a d would involve assigning the reverse, gaze-centred, coordinates. (b) The four surface orientations tested during task 1: orientation manipulation. The orientations are given with reference to the head of the
observer. (c) Illustration of the adaptation phase used in task 2. (d) Fingers tested during the post-adaptation phase. Orientation of the fingertip was held constant.

We investigated this specific body surface, because given the ability of the hand to explore objects from different vantage points, independent of the posture of the body, it may well exhibit unique reference frame selection relative to other body surfaces. The participants in our study were presented with the letters b, d, p, and q on their fingertips, by means of a tactile interface (Tactos, see Gapenne et al., 2003). They had to respond which of the letters they recognized. The experiment was divided into 3 tasks. In the first one, the participants were presented with the tactile letters on the index fingertip of their dominant hand and four surface orientations were tested. In the second task, the participants were required to adopt the reference frame corresponding to the device (i.e. mirror reversed as compared to a head-centred perspective) and they were trained with this perspective. In a third task, we measured whether this adapted perspective would generalize across non-adapted fingers and across the body midline (see figure 14).

Our results first showed that the orientation of the hand in the vertical plane determines the perspective taken: an external perspective is adopted when the hand faces the observer and a gaze-centred perspective is selected when the hand faces away. Second, the participants adapt to a mirror-reversed perspective through training. Third, this adapted perspective holds for the adapted surface and generalises to non-adapted surfaces, including across the body midline (see figure 15). This transfer suggests that the process measured here is a higher-level perspective taking, rather than an automatic coordinate assignment process linked to the receptive fields of the stimulated surfaces (Harrar et al., 2014).

<figure><img src="image_21.png" /><figcaption>Figure 15. Proportion of responses consistent with the natural and adapted perspectives, before, during, and after the adaptation task. (a) The proportion of responses consistent with the predicted, gaze-centred perspective, and the three other non-predicted ones, averaged across observers. The error bars represent the standard error of the mean. (b) The change in the response pattern during adaptation (averaged across observers and across the four letters). That is, adaptation phase trial 1 corresponds to four trials involving each of the four letters b, d, p, q. The variance (shaded error bar) represents the standard error across observers. The polynomial fit to the proportion of non-adapted responses, represented by the continuous grey line. (c) The proportion of responses in the post-test (averaged across observers and letters) consistent with the adapted perspective for the four surfaces:</figcaption></figure>
the adapted fingertip, the non-adapted middle finger of the adapted hand, the homologous finger of the non-adapted hand, and the middle finger of the non-adapted hand. The error bars represent the standard error of the mean.

Note that the transfer results have implications for training with novel haptic interfaces, suggesting that it is possible to remap, with a limited amount of training, the local cutaneous pattern of stimulation into different coordinates for the purposes of experiencing a three-dimensional world and a novel perspective from your own gaze-/body-centred point of view.

### III.7. The bi-stable palm

A sixth study was conducted in collaboration with Jess Hartcher-O'Brien (post-doctoral researcher) and Vincent Hayward (ISIR, UMR 7722). Its aim was to investigate further the perspectives that are taken on the hand (Hartcher-O'Brien, Hayward, & Auvray, in preparation). This study used a variation of the graphesthesia task. Circular shapes were drawn on the body surface by means of a tactile interface (Tactos, see Gapenne et al., 2003). These stimuli could be drawn clockwise along the device or counter-clockwise. The participants' task was to answer if they perceived each stimulus as being presented clockwise versus counter-clockwise and we identified the reference frame they adopted from their responses. In the first experiment of this study, the stimuli were presented on the index and palm of the participants' dominant hand, which each time varied across 5 orientations (see figure 16).

<figure><img src="image_22.png" /><figcaption>Figure 16. Viewpoint selection, stimuli and procedure. (A) The clockwise counter-clockwise pattern as displayed on the surface of the device. (B) Experiment 1. Observers (n = 16) placed their dominant hand index finger or palm against the active surface of the device and reported whether the pattern moved clockwise or counter-clockwise in a 2AFC procedure. (C) The five orientations tested.</figcaption></figure>
The obtained results revealed that when the stimulation is received on the index, the reference frame that is adopted varies with the orientation of the hand. As in Hartcher-O'Brien & Auvray (2016), an external perspective is adopted when the hand faces the observer and a gaze-centred perspective is selected when the hand faces away. A less straightforward response pattern was observed for stimulation on the palm, suggesting a potential change in processing across the skin's surfaces for orientation (see figure 17). This finding is in contrast to what has previously been assumed, i.e. that perspective taking across the hand is homogeneous (Parsons & Shimojo, 1989).

<figure><img src="image_23.png" /><figcaption>Figure 17. Viewpoint results for hand orientation. Proportion of gaze-centred responses for the five orientations tested for the fingertip (A) and the palm (B) averaged across participants. The error bars represent the standard deviation.</figcaption></figure>

Our aim in the next experiment was to verify if this response pattern identified for the palm could potentially be the result of poorer sensory acuity, i.e. lower resolution within this neighbourhood than for the fingertip (Mancini et al., 2014; Stevens & Choo, 1996) or if, instead it reflects a different process. To do so, the fingertip and then the palm were again stimulated with a circular pattern of motion, this time containing gaps in one of four possible locations. The participants' task of was to identify the position of the gap in the pattern of stimulation. Only clockwise patterns were used in this experiment and a control, 'no gap' stimulus allowed us to calculate d'prime. The results revealed that although the sensitivity was higher on average on the palm than on the fingertip, such sensitivity could not predict the coordinate assignment pattern on the fingertip nor on the palm (ps>5, see figure 18). This result suggests that the difference across skin surfaces cannot be accounted for by differences in spatial acuity of the two regions but is more likely linked to the functional role of the surface in everyday interactions.
<figure><img src="image_24.png" /></figure>

Figure 18. Correlation between change in viewpoint responses and the sensitivity of each surface as measured via the gap detection procedure for the fingertip (A) and the palm (B). The correlation between sensitivity, d'prime, and proportion change in perspective is not significant for any of the surfaces tested.

The third experiment investigated whether viewpoint selection across the entire surface of the hand was continuous or discretely separable into functionally defined neighbourhoods. The same procedure as in Experiment 1 was used, this time on eight neighbouring locations on the volar surface of the hand. If the response pattern changes gradually from fingertip to palm this would suggest continuous, non-functional organisation of the skin boundary. If, on the other hand, the response pattern changes discreetly across regions, from finger to palm, this would support a functional organisation of the surface. Figure 19 shows the discrete changes from the finger region to that of the palm. With respect to viewpoint adopted, three clusters emerged. The three groups are distinguishable and suggest a function-based viewpoint selection process on the hand.

<figure><img src="image_25.png" /></figure>

Figure 19. Role of surface function via structural regions of the hand. The clockwise-counter-clockwise stimuli were applied to distinct regions systematically across the hand. The proportions of body centred responses are given as a function of the stimulated locations.

### III.8. The processes of distal attribution

One of the key mechanisms of spatial cognition is distal attribution, also known as exteriorization. When tactile stimuli are used, distal attribution corresponds to the processes
allowing attributing the stimuli received on the body surface to an object located in the three-dimensional space. The research described in this section was conducted in collaboration with Vincent Hayward (ISIR, UMR 7722), Jessica Hartcher O'Brien, and Alexander Terekhov O'Brien (post-doctoral researchers).

It first should be mentioned that distal attribution is of important interest in sensory substitution. Indeed, one interesting result obtained with these devices suggests a change in users' subjective experience. After training, they no longer report feeling the stimulation on their skin, where it occurs, but directly attribute the cause of the stimulation to a distant object (Bach-y-Rita et al., 1969; Bach-y-Rita and Kercel, 2003; White et al., 1970;). **Note that a review on distal attribution in sensory substitution is included as the seventh of the selected articles in the Annex (Hartcher-O'Brien & Auvray, 2014).**

In addition, it should be mentioned that the graphesthesia task was one of the first paradigms we thought of to investigate distal attribution. What can be said for now about the graphesthesia task is that adopting a head-centred perspective for interpreting tactile stimuli does not necessarily imply the conversion of the stimuli into a visual format. However, both visual and tactile perspective taking seem to involve a common externalization process: the origin of the perspective must be located at a different position than the object of perception. In vision, the object is external to the perceiver, located in front of the head. In touch, the externalization process involves projecting the tactile stimulation forward, centring the perspective on another body part than the one being stimulated (i.e., bending the head forward to see the stimulation), or adopting a decentred perspective. The adoption of a consistent spatial perspective on tactile stimulation may thus characterize the transition from experiencing the tactile stimulation on the skin (proximal attribution) to becoming able to gain access to the distant object represented by the tactile stimulation (distal attribution, see Arnold, Spence, & Auvray, 2017). This distal attribution process, also named referral of touch (e.g., Petkova & Ehrsson, 2009), is crucial for the ability to distinguish internal from external stimulation, and more generally, the self from the external world. We plan to investigate in the coming year changes in this process with the graphesthesia task, before and after training with a visual-to-tactile sensory substitution device.

A study was also conducted on haptic shape constancy across distance, in collaboration with Jessica Hartcher O'Brien, Alexander Terekhov (post-doctoral researchers), and Vincent Hayward (ISIR, UMR 7722, see O'Brien, Terekhov, Auvray, & Hayward, 2014). Perceptual constancy is central to the process of distal attribution as it allows the nervous system to maintain a coherent experience of the world despite being able to move its sensors independently to the environment. In vision, constancy refers to the stability of the perceived properties of an object over space and time which can be achieved by converting retinal
signals into spatiotopic coordinates, allowing the observers to perceive space independently of their eye movements (Holway & Boring, 1941; Wexler & Held, 2005). In haptic perception, the problem is tricky as well, given that the sensory surfaces undergo even more complex movements in space. To give examples, an object's perceived haptic size is influenced by the extension of the arm, as well as by local deformation of the probe (e.g., Dostmohamed & Hayward, 2005). Thus, any sensory system with the ability to move and explore the environment faces the problem of accounting for changes in proximal patterns of stimulation that are due to self-generated movement, and those that are not. Moreover, in order to experience objects in external space, these proximal patterns need to be converted into an external / spatiotopic frame of reference such that the object can be represented independently of the observer generated perturbations.

The aim of this study was to investigate whether perceived curvature changes as a function of object displacement from the observer. To do so, blindfolded sighted participants had to scan consecutively the surface of two shapes and to report which of the two objects had higher curvature. One shape was close to the observer and the other was positioned further away. This spatial displacement changes the available proprioceptive information about the object shape, and therefore the combined proprio-tactile information may signal different objects at the two distances. The stimuli consisted in 13 three-dimensional rectangular forms with a circular hole. The curvature of one stimulus, the standard, was always maintained constant and the curvature of the others varied in steps of 2 m-1 (see figure 20).

<figure><img src="image_26.png" /><figcaption>Figure 20. Schematic representation of a) the standard stimulus and b) a subset of the comparison stimuli with changes in curvature by a step of 2 m-1. b) Single trial setup for the 2AFC procedure in which participants compared the curvature of two objects: one located at D1 and the second positioned at D2. The exploration was done consecutively with one hand, observers followed a line of tape on the tabletop to guide them to the second object.</figcaption></figure>

The results reveal a perceptual compensation for the change in proprioceptive information. However, two distinct patterns of distance compensation emerged. For one group, haptic shape estimates across distance were consistent with visual-like compensation, in other words, closer shapes needed to have lower curvature to be perceived as the same shape as
the standard at D2. The other group showed a reversed pattern of response such that objects further away need to have lower curvature to be perceived having equal curvature. In this case, the participants performed a reweighting of the proprio-tactile information to account for the displacement (see figure 21).

These results suggest that perceived haptic curvature across distance depends on observers’ differential weighting of the multiple available cues. Therefore unlike in visual processing there are multiple sources of information that can weigh in to account for the displacement. One hypothesis for the difference observed is that one group relies more on local touch information, while the second group weights proprioceptive cues more highly.

<figure><img src="image_27.png" /><figcaption>Figure 21. Psychometric functions for estimated curvature as a function of distance: when the standard is close at D1 (grey) and then further from the observer at D2 (black). a) An example observer from group 1 for whom the function for the close object shifts towards high curvature while distant objects are more likely to be perceived as low curvature. b) An example observer from the subset showing the reverse pattern for near and far objects. Data points in the fitting procedure have been weighted according to the number of trials used to obtain them.</figcaption></figure>

### III.9. Conclusions on multisensory reference frames

Katz (1925) suggested that touch has many of the capabilities of a distal or ‘far’ sense, such as vision and audition. However, spatial properties of objects, such as their distance from the observer, are readily available through the ‘far senses’ but not to the ‘proximal’ ones and in this way our experience of space depends on the sensory receptors stimulated. For tactile perception, attribution to a distal source or to the body surface can differ depending on the circumstances. For instance, active exploration via touch tends to promote distal attribution (Gibson, 1962; Katz, 1925). Characterizing spatial perception in touch thus requires to understand how the proximal stimulation (i.e., the stimulation received at the receptor
surface) spatially relates to its cause. In the experiments described in this section, we first approached this question by using the graphesthesia task to interrogate the coordinate systems that are used to perceive tactile stimuli. To start (III.2), our results revealed that there is an individual variability with some persons preferring to perceive space from their own centred point of view and others preferring a decentred perspective (20% of the cases in our study). This reference frame is natural and does not reduce to a mere cognitive choice. Second (III.3), our studies revealed an influence of vision on the reference frames that are adopted with blind people adopting less frequently the decentred perspectives and the (temporary or permanent) lack of vision favouring the egocentric perspective. (III.4) We also showed that the head-centred perspective has an embodied nature, as it is easier to adopt for surfaces that can be easily looked at with real head movements. (III.5) In addition, deafferented patients rely on external stimulation and they adopt strategies that are more cognitive and more prone to inter-individual differences. Regarding the hand, our studies showed (III.6) that when the participants are trained to adopt a novel reference frame on one body surface, there is a transfer of learning to other body surfaces. In addition, an external perspective is adopted when the hand faces the observer and a gaze-centred perspective is selected when the hand faces away. Far more variability was found when stimuli are presented on the palms than on the fingertips. (III.7) This variability in the responses was found to changes discreetly across regions, from finger to palm, which supports a functional organisation of the surface. Finally (III-8), using a different paradigm of curvature estimates, we found an inter-individual variability with some persons relying more on local touch information and others more on proprioceptive information.

These studies have implications for the design of tactile devices conceived to compensate for visual impairments, as they reflect the importance to take into account individual specificities. The use of these devices can be eased by implementing flexible options as a function of the differences in the reference frames adopted by people (egocentric versus allocentric; trunk-centred versus head-centred). These individual specificities are particularly crucial for early-blind people, as they show greater difficulties to adopt unnatural perspective. One possible solution to overcome this consists in adapting the tactile device to their preferences in orienting left-right and top-bottom of tactile stimulation. The transfer results have implications for training with novel haptic interfaces, suggesting that it is possible to remap, with a limited amount of training, the local cutaneous pattern of stimulation into different coordinates for the purposes of experiencing a three-dimensional world and a novel perspective from a gaze-/body-centred point of view.

The graphesthesia paradigm can also have implications for understanding the social and personal factors that influence the perspective that people adopt on their own body. This aspect will be detailed in the section perspective of research. Finally, further research with the
graphesthesia task and other paradigms investigating distal attribution might allow shading
light on the distinction between sensation and perception. Indeed, distal attribution occurs
when we experience a pattern of stimulation and assign the cause of this stimulation to an
object, stimulus, or event located in external three-dimensional space. When proximal
stimulation is not assigned an external cause, it gives rise to a mere sensation, not the
perception of an object located at a distance. Therefore understanding how distal attribution
emerges is central to understanding the distinction between bodily and external space, as well
as the plasticity of this border.

## IV. Perspectives for future research

The scientific research I plan to conduct the coming years aim at investigating the plasticity of our perceptual and bodily representations during tool use (involving the specific tools that are sensory substitution devices) and the role of multisensory information on this plasticity. This research schedule will be presented divided into three themes: 1) Exteriorisation and embodiment; 2) Sensory substitution of vision and proprioception; 3) Interactions between spatial and mental perspectives.

### IV.1. Exteriorisation and embodiment

Bodily and distant spaces seem well defined and delineated. There is the space of my body, the place where I feel, where I receive sensations, which is marked by the boundaries of my skin and the external space made up of objects that I perceive as being out there. However, several situations reveal that such distinction is not straightforward. For instance, when using a virtual reality or a tele-presence device I feel as being at a different place than where my body actually is. These examples underline the possibility to modify the boundaries of the body and hence its plasticity. In order to understand in full such plasticity, two sides of the same question have to be considered jointly: the attribution of distant stimuli to the body space and the attribution of proximal stimuli to a distant space. The research I have conducted so far focused on the second side of the question. My aim is to both prolong this line of research and to investigate its counterpart, the phenomena of bodily extensions and modifications.

Regarding the mechanisms of distal attribution or exteriorisation, we are currently extending the investigation of how tactile stimuli received on the body surface are projected to a distant space to the study of the effect of the spatial coincidence between eye fixations and tactile stimuli. This work started in July 2018 thanks to a post-doctoral funding from Oculus research (PI: V. Hayward). If the results are encouraging, we will then extend this work to the broader study of tactile sensation projections in link with oculomotor signals.
Regarding the investigation of bodily extensions and modifications, first, in collaboration with Vincent Hayward (ISIR) and Gabriel Arnold (company Caylar, previously post-doctoral researcher at ISIR), we started to investigate the numbness illusion (e.g., see Dieguez et al., 2009, see also figure 22). We believe that the effects of this illusion are as promising as those obtained with the rubber hand illusion while allowing for a diversity of new objective measures (such as the use of accelerometers, the measure of precision grip). A first investigation, during the internship of the master student Nathan Rish (2017-2018), interestingly showed that people touch themselves differently before and after the illusion. This promising line of research also validated the use of accelerometers (to measure forces of pressure) to investigate this question. Many control conditions remain to be conducted and, once this is done, we plan to extend this line of work to the shared factors that underlie the illusion as well as the individual differences at stake. The shared factors can include for instance hand configuration (i.e., if my hand is considered to be in my space or in the other's one, see figure 22). Regarding the individual factors, the illusion can induce two different feelings; some persons feel that their finger becomes numb, as if they were giving it to the partner whereas other persons feel an extension of their finger as if they incorporated that of the partner. The question then arises as to what account for these individual differences (e.g., in terms of spatial abilities, interpersonal skills, cognitive style). More broadly, our aim with this illusion is to investigate the link between body perception and body schema, through questions such as: If the body image is modified by the numbness illusion, does it change the way we touch ourselves? Does it change the way we touch someone else? If our body image determines our body schema, is there a difference between touching ourselves and touching someone else?

<figure><img src="image_28.png" /><figcaption>Figure 22. Illustration of the numbness illusion. The participant rubs his index finger and that of the experimenter (around 10 times). The illusion consists in feeling either a larger or a numb finger. Here are presented two possible configurations: the classic one and the reverse one.</figcaption></figure>

The mechanisms underlying body plasticity will also be conducted within the project “developmental tool mastery” (funded by the ANR, PI: Alessandro Farnè). The project aims at offering a comprehensive multidisciplinary framework for unveiling the rules of body representations plasticity that allow humans to master tool-use. More precisely, we aim at
investigating how young humans acquire control of tools to reach and grasp objects and to
sense and explore their environment. Then we will turn to the question of how the adult
human brain does code for these abilities. Finally, we will address the question of how do
multiple sensory modalities contribute to acquiring and regulating these abilities. In co-
supervision with Vincent Hayward, we have hired a post-doctoral researcher for two years in
November 20018 at ISIR to work on these question. We have hired a second post-doctoral
researcher for the same period to work on closely related topics. In particular, the aim will
be to use novel techniques (virtual and tactile interfaces) to empirically investigate how tool-
use sensing is encoded in the brain, by disentangling the sources of information arising from
physical invariants. This will be done by creating paradoxical stimuli that clearly violate
these invariants, and by selectively suppressing sources of information or sensory cues
arising from one modality (haptics, vision, audition) while maintaining and even enhancing
the other sources.

IV.2. Sensory substitution of vision and proprioception

The second theme of the research planned prolongs my previous work on sensory
substitution devices and extends it to devices designed to compensate for proprioceptive
impairments.

Sensory substitution of vision

First, I plan to carry on the study of the crossmodal correspondences in sighted and blind
persons (described in section II-4). The goal is now to investigate more systematically the
possible crossmodal correspondences in audition, vision, and touch, focusing on the
dimensions of the signals that are relevant for sensory substitution, and in addition, to
investigate the ways in which these correspondences are combined together.

Second, an intrinsic problem with systems encoding visual information into a single sensory
modality is that a maximum of 3 dimensions of the visual signal can be encoded
simultaneously. Hence, whereas some devices only code for depth (z-axis), others code for
the horizontal and vertical axes (x & y axes) together with brightness. None of them can
provide a code for all three axes and brightness simultaneously. This can be achieved by
developing new multisensory substitution devices that use the input of more than one
sensory channel. In collaboration with Maurice Ptito (McGill, Montreal), Ron Kuppers
(University of Copenhaguen), Ophelia Deroy (University of Munich), and the company
Roger McMahon & Associates (Glasgow), our aim is to develop the I-VEST (Innovation
through Vision Enhancement by Sound and Touch), a multisensory augmentation device
consisting of a tight fitting shirt with integrated tactile stimulators (actuators) and a new
generation of headphones. A mini-camera, mounted on a pair of glasses will capture visual
information that will be converted into tactile and auditory code, and wirelessly transmitted
to the ears and skin of the upper torso. Thus, here, visual information will be translated into both tactile and auditory code which will allow us to convert all three spatial axes and brightness. To do so in an optimal way, we will first investigate the audio-tactile interactions in blind and sighted individuals, to measure the neurobehavioral impact of the absence of vision on these interactions. Then we will integrate our findings into our portable and integrated technologies based upon audio-tactile information processing. Our aim is to build this device for blind individuals, for those suffering from other visual impairments, and for sighted people in conditions of low visibility. One key aspect is also to borrow methodologies from the video game and the entertainment sector to develop novel approaches to sensory augmentation. We plan to apply to European grants to fund this project.

## Sensory substitution of proprioception

Two projects are starting regarding the compensation for proprioception. The first one, I-Gait (PI: Wael Batcha, ISIR, funded by ANR) aims at better preventing falls in both elderly people and people suffering from proprioceptive deficits by means of a device assisting them when they walk. We will develop a tactile handle, which delivers computer-generated tactile feedback able to initiate the necessary postural and force adjustments, mounted on the top of a robotic cane. Our aim is to investigate the best conversion of proprioceptive signals into tactile ones such that users can initiate automatic and precise adjustments of the postural balance and of the force applied to the cane by mechanically stimulating the skin of the hand palm. This will be built on previous work (Terekhov & Hayward, 2015; Vérité et al., 2014) which showed that people's centre of pressure can be displaced precisely, without any conscious cooperation, through an appropriate control of the motion of a surface, which users touch lightly with their forefinger.

The second project is conducted in collaboration with Nathanaël Jarasse (ISIR) and Gabriel Arnold (company Caylar). Our aim is to ease the learning of prostheses involving myoelectric control in upper limb amputees. To do so, our aim is to investigate the best way to code for proprioceptive cues into tactile ones that will be transmitted to users by means of a vibrotactile bracelet. We have co-supervised a second year maser student (Yann Kechabia) in 2017-2018 to study the coding of wrist rotation into vibrations. The results are promising and our aim now is to extend this work to the other proprioceptive cues that are involved in the use of the prosthesis.

## IV.3. Interactions between spatial and mental perspectives

The aim of this theme is to prolong the research on multisensory spatial perspectives (described in section II) in order to investigate the reciprocal influence of mental and spatial perspectives.
Spatial cognition is subject to important inter-individual variability (e.g., Berthoz & Viaud-Delmon). It is thus important to understand the personal factors underlying this difference. First, during the internship of a master student in clinical psychology (Sandra Inard, in 2016-2017) and in collaboration with Gabriel Arnold (company Caylar), we used the graphesthesia task in participants and correlated their results to those they obtained in personality and attachment questionnaires. Promising results started to emerge, in particular in the link between spatial perspectives, introversion, and attachment. Our aim is now to investigate these links by means of more fine grained questionnaires.

Second, the ability to adopt someone else's perspective develops in infancy. However, this ability is impaired in some persons, such as those in the autistic spectrum. In collaboration with David Cohen and Jean Xavier (MD, G.H.U. Pitié-Salpêtrière) our aim is to use the graphesthesia task in autistic and neurotypical children (aged between 3 and 18 years old). This will allow us to investigate more in depth the link between mental and spatial perspectives. The next step will be to investigate if the reciprocal influence of spatial and mental perspectives can be modulated. In other words, the aim will be to determine the extent to which it is possible to change of mental perspectives by changing spatial perspectives and vice versa.

V. References

Abboud, S., Hanassy, S., Levy-Tzedek, S., Maidenbaum, S., & Amedi, A. (2014). EyeMusic: Introducing a “visual” colorful experience for the blind using auditory sensory substitution. *Restorative Neurology and Neuroscience*, 32, 247-257.

Afonso, A., Blum, A., Katz, B. F. G., Tarroux, P., Borst, G., & Denis, M. (2010). Structural properties of spatial representations in blind people: Scanning images constructed from haptic exploration or from locomotion in a 3-D audio virtual environment. *Memory and Cognition*, 38, 591-604.

Ahissar, M., & Hochstein, S. (2004). The reverse hierarchy theory of visual perceptual learning. *Trends in Cognitive Science*, 8, 457-464.

Alsmith, A. J., & Longo, M. R. (2014). Where exactly am I? Self-location judgements distribute between head and torso. *Consciousness and Cognition*, 24, 70-74.

Arno, P., De Volder, A. G., Vanlierde, A., Wanet-Defalque, M. C., Streel, E., Robert, A., et al. (2001). Occipital activation by pattern recognition in the early blind using auditory substitution for vision. *Neuroimage* 13, 632-645.

Arnold, G., & Auvray, M. (2018/to appear). Tactile recognition of visual stimuli: Specificity versus generalization of perceptual learning. *Vision Research*.

Arnold, G., & Auvray, M. (2014). Perceptual learning: Tactile letter recognition transfers across body surfaces. *Multisensory Research*, 27, 71-90.

Arnold, G. & Auvray, M. (in preparation). Blind and sighted people differ in the way they mentally represent space from different perspectives.
Arnold, G., Sarlegna, F., Fernandez, L., & Auvray, M. (submitted). The physical body plays a different role in locating and orienting the experienced self in deafferented patients and in controls.

Arnold, G., Spence, C., & Auvray, M. (2016). Taking someone else's spatial perspective: Natural stance or effortful decentring? *Cognition, 148*, 27-33.

Arnold, G. & Auvray, M. (2017). The graphesthesia paradigm: drawing letters on the body to investigate the embodied nature of spatial perspective taking. *I-Perception, 8*, 1-5.

Arnold, G., Pesnot-Lerousseau, J. & Auvray, M. (2017). Individual differences in sensory substitution. *Multisensory Research, 6*, 579-600.

Arnold, G., Spence, C., & Auvray, M. (2017). A unity of the self or a multiplicity of locations? How the graphesthesia task sheds light on the role of spatial perspectives in bodily self-consciousness. *Consciousness & Cognition, 56*, 100-114.

Aspell, J. E., Heydrich, L., Marillier, G., Lavanchy, T., Herbelin, B., & Blanke, O. (2013). Turning body and self inside out: Visualized heartbeats alter bodily self-consciousness and tactile perception. *Psychological Science, 24*, 2445-2453.

Auvray, M., & Farina, M. (2017). Patrolling the boundaries of synaesthesia: A critical appraisal of transient and artificially-acquired forms of synaesthetic experiences. In O. Deroy (Ed.). *Synaesthesia: Philosophical & Psychological Challenges* (pp. 248-274), Oxford: Oxford University Press.

Auvray, M., Hanneton, S., & O'Regan, J. K. (2007). Learning to perceive with a visuo-auditory substitution system: Localization and object recognition with The Voice. *Perception, 36*, 416-430.

Auvray, M., & Myin, E. (2009). Perception with compensatory devices. From sensory substitution to sensorimotor extension. *Cognitive Science, 33*, 1036-1058.

Bach-y-Rita, P., Collins, C. C., Saunders, F. A., White, B., & Scadden, L. (1969). Vision substitution by tactile image projection. *Nature, 221*, 963-964.

Bach-y-Rita, P., & Kercel, S. W. (2003). Sensory substitution and the human-machine interface. *Trends in Cognitive Sciences, 17*, 541-546.

Bach-y-Rita, P., Tyler, M. E., & Kaczmarek, K. A. (2003). Seeing with the brain. *International Journal of Human-Computer Interaction, 2*, 285-295.

Berthoz, A., & Viaud-Delmon, I. (1999). Multisensory integration in spatial orientation. *Current Opinion in Neurobiology, 9*, 708-712.

Blackmore, S. (1987). Where am I? Perspectives in imagery and the out-of-body experience. *Journal of Mental Imagery, 11*, 53-56.

Blanke, O. (2012). Multisensory brain mechanisms of bodily self-consciousness. *Nature Reviews Neuroscience, 13*, 556-571.

Blanke, O., Ortigue, S., Landis, T., & Seeck, M. (2002). Neuropsychology: Stimulating illusory own-body perceptions. *Nature, 419*, 269-270.

Block, N. (2003). Tactile sensation via spatial perception. *Trends in Cognitive Sciences, 7*, 285-286.

Bologna, G., Deville, B., & Pun, T. (2009). On the use of the auditory pathway to represent image scenes in real-time. *Neurocomputing, 72*, 839-849.
Boussaoud, D., & Bremmer, F. (1999). Gaze effects in the cerebral cortex: Reference frames for space coding and action. *Experimental Brain Research, 128*, 170-180.

Boyer, E. O., Babayan, B. M., Bevilacqua, F., Noisternig, M., Warusfel, O., Roby-Brami, A., Hanneton, S., & Viaud-Delmon, I. (2013). From ear to hand: the role of the auditory-motor loop in pointing to an auditory source. *Frontiers in Computational Neurosciences, 7*, 26.

Brunet, L. Darses, F., & Auvray, M. (2018). Strategies and needs of blind pedestrians during urban navigation. *Le Travail Humain, 81*, 141-171.

Clark, A. (2003). *Natural-born cyborgs: Minds, technologies, and the future of human intelligence*. New York: Oxford University Press.

Cohen, Y. E., & Andersen, R. A. (2002). A common reference frame for movement plans in the posterior parietal cortex. *Nature Reviews Neuroscience, 3*, 553-562.

Coltheart, M., & Rastle, K. (1994). Serial processing in reading aloud: evidence for dual-route models of reading. *Journal of Experimental Psychology Human Perception and Performance, 20*, 1197-1211.

Danilov, Y., Tyler, M., Skinner, K., Hogle, R., & Bach-y-Rita, P. (2007). Efficacy of electrotactile vestibular substitution in patients with peripheral and central vestibular loss, *Journal of Vestibular Research, 17*, 119-130.

Denis, M. (2018). *Space and spatial cognition: A multidisciplinary perspective*. Abingdon, UK: Routledge.

Deroualle, D., Borel, L., Devèze, A., & Lopez, C. (2015). Changing perspective: The role of vestibular signals, *Neuropsychologia, 79*, 175-185.

Deroy, O., & Auvray, M. (2012). Reading the world through sensory substitution devices. *Frontiers in Theoretical and Philosophical Psychology, 3*, 457.

Deroy, O., & Auvray, M. (2014). A crossmodal perspective on sensory substitution. In S. Biggs, M. Matthen, & D. Stokes (Eds.), *Perception and Its Modalities* (pp. 327-349), Oxford: Oxford University Press.

Deroy, O., Fasiello, I., Hayward, V., & Auvray, M. (2016). Differentiated audio-tactile correspondences in sighted and blind individuals, *Journal of Experimental Psychology: Human Perception and Performance, 42*, 1204-1214.

Deroy, O., & Spence, C. (2013). Why we are not all synesthetes (not even weakly so). *Psychonomic Bulletin & Review, 20*, 643-664.

Dieguez, S., Mercier, M. R., Newby, N., & Blanke, O. (2009). Feeling numbness for someone else's finger. *Current Biology, 19*, R1108-R1109.

Diot, B., Halavackova, P., Demongeot, J., & Vuillerme, N. (2014). Sensory substitution for balance control using a vestibular-to-tactile device, *Multisensory Research, 27*, 313-336.

Dostmohamed, H., & Hayward, V (2005). Trajectory of contact region on the fingerpad gives the illusion of haptic shape. *Experimental Brain Research, 164*, 387-394.

Farcy, R., Leroux, R., Jucha, A., Damaschini, R., Grégoire, C., & Zogaghi, A. (2006). Electronic travel aids and electronic orientation aids for blind people: Technical, rehabilitation and everyday life points of view. In M. A. Hersh (Ed.), *Proceedings of the Conference & Workshop on Assistive Technology for Vision and Hearing Impairment*.
Ferrè, E. R., Lopez, C., & Haggard, P. (2014). Anchoring the self to the body: Vestibular contribution to the sense of self. *Psychological Science*, 25, 2106-2108.

Gallay, M., Denis, M., & Auvray, M. (2013). Navigation assistance for blind pedestrians: Guidelines for the design of devices and implications for spatial cognition. In T. Tenbrink, J. Wiener, & C. Claramunt (Eds.), *Representing space in cognition: Interrelations of behaviour, language, and formal models*, Oxford: Oxford University Press.

Gapenne, O., Rovira, K., Ali Ammar, A., & Lenay C (2003). Tactos: special computer interface for the reading and writing of 2D forms in blind people. *Universal Access in HCI, Inclusive Design in the Information Society*, 10, 1270-1274

Gibson, J. J. (1962). Observations on active touch. *Psychological Review*, 69, 477-490.

Hanneton, S., Auvray, M., & Durette, B. (2010). The Vibe: A versatile vision-to-audition sensory substitution device. *Applied Bionics and Biomechanics*, 7, 269-276.

Harrar, V., & Harris, L. R. (2010). Touch used to guide action is partially coded in a visual reference frame. *Experimental Brain Research*, 203, 615-620.

Harrar, V., Spence, C., Makin, T. (2014) Topographic generalization of tactile perceptual learning. *Journal of Experimental Psychology: Human Perception and Performance*, 40, 15-23.

Heed, T., Buchholz, V. N., Engel, A. K., & Röder, B. (2015). Tactile remapping: from coordinate transformation to integration in sensorimotor processing. *Trends in Cognitive Sciences*, 19, 251-258.

Ho, C., & Spence, C. (2007). Head orientation biases tactile localization. *Brain Research*, 1144, 136-141.

Ho, C. & Spence, C. (2008) *The Multisensory Driver: Implications for ergonomic car interface design*. Aldershot: Ashgate Publishing.

Holway, A.H., & Boring, E.G. (1941). Determinants of apparent visual size with distance variant. *American Journal of Psychology*, 54, 21-37.

Humphrey, N. (1992). *A History of the Mind*. New York: Simon & Schuster.

Hurley, S., & Noë, A. (2003). Neural plasticity and consciousness. *Biology and Philosophy*, 18, 131-168.

James, W. (1890). *Principles of Psychology*. Holt, New York.

Jansson, G. (1983). Tactile guidance of movement. *International Journal of Neurosciences*, 19, 37-46.

Jeter, P. E., Dosher, B. A., Liu, S. H., & Lu, Z. L. (2010). Specificity of perceptual learning increases with increased training. *Vision Research*, 50, 1928-1940.

Jones, L. A., Kunkel, J., & Piateski, E. (2009). Vibrotactile pattern recognition on the arm and back. *Perception*, 38, 52-68.

Kaczmarek, K. A., & Haase, S. J. (2003). Pattern identification and perceived stimulus quality as a function of stimulation current on a fingertip-scanned electrotactile display. *IEEE Transaction on Neural System Rehabilitation Engineering*, 11, 9-16.

Katz, D. (1925). *The World of Touch*. Erlbaum, Hillsdale, NJ, USA (translation published 1989, translated by L. E. Kmeger).
Katz, B., Dramas, F., Parseihian, G., Gutierrez, O., Kammoun, S., Brilhault, A., Brunet, L., Gallay, M., Oriola, B., *Auvray*, M., Truillet, P., Thorpe, S., & Jouffrais, C. (2012-a). NAVIG guidance system for the visually impaired using virtual augmented reality. *Technology and Disability, 24*, 163-178.

Katz, B., Kammoun, S., Parseihian, G., Gutierrez, O., Brilhault, A., Auvray, M., Truillet, P., Thorpe, S., & Jouffrais, C. (2012-b). NAVIG: Augmented reality guidance system for the visually impaired. *Virtual Reality Journal, 16*, 253-269.

Kupers, R., Fumal, A., de Noordhout, A. M., Gjedde, A., Schoenen, J., & Ptito, M. (2006) Transcranial magnetic stimulation of the visual cortex induces somatotopically organized qualia in blind subjects. *Proceedings of the National Academy of Science of the USA, 103*, 13256-13260.

Levy-Tzedek, S., Novick, I., Arbel, R., Abboud, S., Maidenbaum, S., Vaadia, E., & Amedi, A. (2012). Cross-sensory transfer of sensory-motor information: visuomotor learning affects performance on an audiomotor task, using sensory-substitution. *Scientific Reports, 2*, 949.

Levy-Tzedek, S., Riemer, D., & Amedi, A. (2014). Color improves “visual” acuity via sound. *Frontiers in Neuroscience, 8*, 358.

Liu, Z., & Weinshall, D. (2000). Mechanisms of generalization in perceptual learning. *Vision Research, 40*, 97-109.

Lopez, C. (2016). The vestibular system: balancing more than just the body. *Current Opinion in Neurology, 29*, 74-83

Maidenbaum, S., Hanassy, S., Abboud, S., Buchs, G., Chebat, D. R., Levy-Tzedek, S., & Amedi, A. (2014). The “EyeCane”, a new electronic travel aid for the blind: Technology, behavior & swift learning. *Restorative Neurology and Neuroscience, 32*, 813-824.

Mancini, F., Bauleo, A., Cole, J., Lui, F., Porro, C. A., Haggard, P., & Iannetti, G. D. (2014) Whole-body mapping of spatial acuity for pain and touch. *Annals of Neurology, 75*, 917-924.

Meijer, P. B. L. (1992). An experimental system for auditory image representations. *IEEE Transactions on Biomedical Engineering, 39*, 112-121.

Menary, R. (2006). Attacking the bounds of cognition. *Philosophical Psychology, 19*, 329-324.

Menary, R. (2007). *Cognitive integration. Mind and cognition unbounded*. Basingstoke: Palgrave.

Meredith, M. A., & Stein, B. E. (1986). Spatial factors determine the activity of multisensory neurons in cat superior colliculus. *Brain Research, 365*, 350-354.

Merzenich, M. M., Kaas, J. H., Sur, M., & Lin, C.-S. (1978). Double representation of the body surface within cytoarchitectonic area 3b and 1 in “SI” in the owl monkey (*Aotus trivirgatus*), *Journal of Computational Neurology, 181*, 41-74.

Nakamura, A., Yamada, T., Goto, A., Kato, T., Ito, K., Abe, Y., Kachib, T. & Kakigic, R. (1998). Somatosensory homunculus as drawn by MEG, *NeuroImage, 74*, 377-386.

Noë, A. (2004). *Action in Perception*. Cambridge, MA: MIT Press.

O’Brien, J., & Auvray, M. (2016). Cognition overrides orientation dependence in tactile viewpoint selection. *Experimental Brain Research, 234*, 1885-1892.
O'Brien, J., & Auvray, M. (2014). The process of distal attribution illuminated through studies of sensory substitution. *Multisensory Research*, 27, 421-441.

O'Brien, J., Auvray, M., & Hayward, V. (2015). Perception of distance-to-obstacle through time-delayed tactile feedback. *Proceedings of the 2015 IEEE World Haptics Conference* (pp. 7-12).

O'Brien, J., Hayward, V., & Auvray, M. (in preparation). Frames of reference at the tip of your finger and in the palm of your hand.

O'Brien, J., Terekhov, A., Auvray, M., & Hayward, V. (2014). Haptic shape constancy across distance. *Haptics: Neuroscience, Devices, Modeling, and Applications Lecture Notes in Computer Science* (pp 77-84). Springer: Heidelberg, New-York, Dordrecht.

O'Regan, J. K. (2011). *Why Red Doesn't Sound Like a Bell. Understanding the Feel of Consciousness*. Oxford: Oxford University Press.

Parsons, L., & Shimojo, S. (1989). Perceived spatial organization of cutaneous patterns on surfaces of the human body in various positions. *Journal of Experimental Psychology: Human Perception and Performance*, 13, 488-504.

Parise, C. V., & Spence, C. (2012). Audiovisual crossmodal correspondences and sound symbolism: A study using the implicit association test. *Experimental Brain Research*, 220, 319-333.

Pavlidou, A., Ferrè, E., & Lopez, C. (2018). Vestibular stimulation makes people more egocentric, *Cortex*, 101, 302-305,

Pasqualotto, A., Spiller, M. J., Jansari, A. S., & Proulx, M. J. (2013). Visual experience facilitates allocentric spatial representation. *Behavioural Brain Research*, 236, 175-179.

Petkova, V. I., Khoshnevis, M., & Ehrsson, H. H. (2011). The perspective matters! Multisensory integration in ego-centric reference frames determines full-body ownership. *Frontiers in Psychology*, 2:35.

Pesnot-Lerousseau, J., Arnold, G., & Auvray, M. (submitted). Visualizing sounds: training-induced plasticity with a visual-to-auditory conversion device.

Prinz, J. (2006). Putting the brakes on enactive perception. *Psyche*, 12, 1-19.

Pritchett, L. M., & Harris, L. R. (2011). Perceived touch location is coded using a gaze signal. *Experimental Brain Research*, 213, 229-234.

Proulx, M. J. (2010). Synthetic synaesthesia and sensory substitution. *Consciousness and Cognition*, 19,501-503.

Proulx, M.J., Brown, D. J., Pasqualotto, A, & Meijer, P. (2014). Multisensory perceptual learning and sensory substitution. *Neuroscience and Biobehavioural Reviews*, 41, 16-25.

Proulx, M. J., Gwinnutt, J., Dell'Erba, S., Levy-Tzedek, S., de Sousa, A. A., & Brown, D. J. (2016). Other ways of seeing: From behavior to neural mechanisms in the online “visual” control of action with sensory substitution. *Restorative Neurology and Neuroscience*, 34, 29-44.

Proulx, M. J., Stoerig, P., Ludowig, E., & Knoll, I. (2008). Seeing ‘where’ through the ears: effects of learning-by-doing and long-term sensory deprivation on localization based on image-to-sound substitution. *PloS one*, 3, e1840.

Ptito, M., Iversen, K., Auvray, M., Deroy, O., Kupers, R. (forthcoming). The pluripotent visual cortex in congenital blindness. In F. Macpherson (Ed.). *Sensory Substitution and*
*Augmentation*, London: Proceedings of the British Academy.

Ptito, M., Moesgaard, S. M., Gjedde, A., & Kupers, R. (2005). Cross-modal plasticity revealed by electro- tactile stimulation of the tongue in the congenitally blind. *Brain* **128**, 606-614.

Renault, A. G., Auvray, M., Parseihian, G., Miall, R. C., Cole, J., & Sarlegna, F. (2018). Does proprioception influence human spatial cognition? A study on individuals with massive deafferentation. *Frontiers in Psychology*. doi.org/10.3389/fpsyg.2018.01322

Renier, L., Laloyaux, C., Collignon, O., Tranduy, D., Vanlierde, A., Bruyer, R., & De Volder, A. G. (2005). The Ponzo illusion using auditory substitution of vision in sighted and early blind subjects. *Perception*, **34**, 857-867.

Sekiyama, K. (1991). Importance of head axes in perception of cutaneous patterns drawn on vertical body surfaces. *Perception & Psychophysics*, **49**, 481-492.

Shore, D. I., Spry, E., & Spence, C. (2002). Confusing the mind by crossing the hands. *Cognitive Brain Research*, **14**, 153-163.

Spence, C. (2011). Crossmodal correspondences: A tutorial review. *Attention, Perception & Psychophysics*, **73**, 971-995.

Stevens, J., & Choo, K. K. (1996). Spatial acuity of the body surface over the life span. *Somatosensory and Motor Research*, **13**, 153-166.

Stiles, N. R., & Shimojo, S. (2015). Sensory substitution. In *The Oxford Handbook of Perceptual Organization*. Oxford: Oxford University Press.

Striem-Amit, E., & Amedi, A. (2014). Visual cortex extrastriate body-selective area activation in congenitally blind people ‘seeing’ by using sounds. *Current Biology*, **6**, 687-692.

Stroop, J. R. (1935). Studies of interference in serial verbal reactions. *Journal of Experimental Psychology*, **18**, 643-662.

Terekhov, A.V., & Hayward, V. (2015). The brain uses extra-somatic information to estimate limb displacement. *Proceedings of the Royal Society B*, **282**, 20151661.

Trivedi, B. (2010). Sensory hijack: rewiring brains to see with sound. *Science*, **2773**, 42-45.

van Leeuwen, C., Verstijnen, I. M., & Hekkert, P. (1999). Common unconscious dynamics underlie uncommon conscious effect: A case study in the iterative nature of perception and creation. In J. S. Jordan (Ed.), *Modeling consciousness across the disciplines* (pp. 179-218). Lanham MD: University Press of America.

Vérité, F., Bachta, W. and Morel, G. (2014). Closed Loop Kinesthetic Feedback for Postural Control Rehabilitation. *IEEE Transactions on Haptics*, **7**, 150-160.

Viaud-Delmon, I., & Warusfel, O. (2014). From ear to body: the auditory-motor loop in spatial cognition. *Frontiers in Neurosciences*, **8**, 283.

Wang, X., Zhou, Y., & Liu, Z. (2013). Transfer in motion perceptual learning depends on the difficulty of the training task. *Journal of Vision*, **13**, 1-9.

Ward, J, & Meijer, P. (2010). Visual experiences in the blind induced by an auditory sensory substitution device. *Consciousness and Cognition*, **19**, 492-500.

Ward, J., & Wright, T. (2014). Sensory substitution as an artificially acquired synaesthesia. *Neuroscience and Biobehavioral Review*, **41**, 26-35.
Wexler, M., & Held, R.M. (2005). Anticipating the three-dimensional consequences of eye movements. *Proceedings of the National Academy of Science of the USA, 102*, 1246-1251.

White, B. W., Saunders, F. A., Scadden, L., Bach-y-Rita, P., & Collins, C.C. (1970). Seeing with the skin. *Perception & Psychophysics*, 7, 23-27.

Zwiers, M. P., Van Opstal, A. J., & Paige, G. D. (2003). Plasticity in human sound localization induced by compressed spatial vision. *Nature Neuroscience*, *6*, 175-181.
CURRICULUM VITAE - MALIKA AUVRAY

CNRS - CR1

ISIR - Institut des Systèmes Intelligents et de Robotique

Mail: auvray@isir.upmc.fr

Web: www.malika-auvray.com

Tel: 0660481444

ACADEMIC EXPERIENCE

1015-Present: CNRS Researcher (CR1), Institut des Systèmes Intelligents et de Robotique, CNRS UMR 7222.

2008-2014: CNRS Researcher (CR2), Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur, CNRS UPR 3251.

2007-2008: Post-doctoral researcher, Laboratoire Neurophysique et Physiologie, CNRS UMR 8119, supervised by Agnès Roby-Brami and Sylvain Hanneton.

2006-2007: Post-doctoral researcher, Centre for Philosophical Psychology, University of Antwerp, supervised by Prof. Erik Myin.

& Visiting Scholar (5 months), Department of Experimental Psychology, University of Oxford, UK, supervised by Prof. Charles Spence.

2005-2006: Post-doctoral researcher, Department of Experimental Psychology, University of Oxford, UK, supervised by Prof. Charles Spence.

2004-2005: A.T.E.R. (Associate Lecturer) in Psychology, University of Compiègne, France. Member of the Perceptual Supplementation Group, supervised by Pr Charles Lenay.

02/2004-06/2004: Research contract for the European Network of Excellence “Enactives Interfaces”, hired by the CNRS, UMR 8581.

2001-2004: Ph.D. in Psychology, in collaboration between the Laboratory of Experimental Psychology, Paris, supervised by J. Kevin O'Regan (CNRS, UMR 8581) and the company BaBeL@StaL, supervised by Olivier Mevel.

EDUCATION

2001-2004: Ph.D. in Psychology, with honours, Ecole des Hautes Etudes en Sciences Sociales, Paris. Defended on the 20/12/2004.

Title: Immersion et perception spatiale. L'exemple des dispositifs de substitution sensorielle [Immersion and spatial perception. The example of sensory substitution devices] (297 pages). Ph.D. Thesis, EHESS, Paris.

Members of the committee: J. Kevin O'Regan (PhD supervisor), Pr Charles Lenay (President of the committee), Pr Yvette Hatwell (referee), Jérôme Dokic (referee), Pr Philippe Fuchs (committee member), Pr Michel Imbert (committee member).
**1999-2000:** M2 in Cognitive Sciences, Specialization in Psychology, with honours, Ecole des Hautes Etudes en Sciences Sociales, Paris.

**Title:** La cécité aux changements progressifs [Blindness to progressive changes] (95 pages). DEA, EHESS, Paris.

**Supervised by:** Dr J. Kevin O'Regan and Dr Pierre Jacob.

**1998-1999:** Master in Philosophy and Logic, with honours, University of Nantes.

**Supervised by:** Dr Nelly Robinet.

**Research report:** Auvray, M. (1999). Bergson et les neurosciences [Bergson and the neuroscience] (133 pages). Master, University of Nantes.

## OTHER QUALIFICATIONS

**2007.** Qualification to the profession of lecturer in Psychology by the French National University Council (qualification given as a function of the teaching experience and publications in the relevant discipline, necessary in order to apply for lectureships).

**2005.** Test of English for international communication (Toeic). Total: 840 (Listening: 430, reading: 410). Corresponds to the level 2+: Advanced Working Proficiency.

## GRANTS AND FELLOWSHIPS

**2018-2019:** One year post-doctoral fellowship funded by the company Oculus (PI. V. Hayward, ISIR & London University). Partners: M. Auvray (ISIR), O. Deroy, and Barry Smith (London University).

**2018:** Program Auton from the CNRS (MI, Mission for Interdisciplinarity). Renewal for the consortium Subilma (PI: F. Sarlegna, 24 K€).

**2017-2018:** Grant from the Labex SMART, project SAHMI (PI: Gilles Bailly, 186,5 K€). Partners: Y. Jansen, M. Auvray, S. Haliyo (ISIR), C. Lenay (Compiègne).

**2017:** Program Auton from the CNRS (MI). Renewal for the consortium Subilma (PI: N. Jarasse, 35 K€).

**2016:** Post-doctoral grant from the Labex SMART, in co-supervision with Nathanaël Jarasse (ISIR, UMR 7222) (beneficiary, G. Arnold, 12 months).

**2016-2019:** Grant from the French national agency for research (ANR), project: Developmental Tool Mastery (PI: A. Farne, 412, 3 K€). Partners: A. Farne, A. Roy (ImpAct, U864), F. de Vignemont (IJN, UMR 8129), M. Auvray, V. Hayward (ISIR).

**2016-2019:** ANR JCJC, I-gait (PI: W. Batcha, 235 K€). Partners: M. Auvray, N. Jarasse (ISIR), R. Reynolds (University of Birmingham).

**2016:** Program Auton from the CNRS (MI). Consortium Subilma (PI: N. Jarasse, 35 K€). Partners similar to the consortium Supplé-Sens 2014 & 2015.

**2015:** Program Auton from the CNRS (MI). Consortium Supplé-Sens (PI: M. Auvray, 35 K€). Partners similar to the consortium Supplé-Sens 2014 funded by DEFI-SENS.

**2014:** Program DEFI-SENS from the CNRS (MI). Perceptual deficiencies and multimodal supplementation. (PI: M. Auvray, 35 K€). Partners: F. Sarlegna (UMR 7287), K. O'Regan (UMR 8242), I. Fasiello (UMR 7222), P. Barone (UMR 5549), F. Matonti (UMR 7289), G.
Parseihian (UPR 7051), I. Viaud-Delmon (UMR 9912), O. Deroy (University of London), J. Cole (Poole Hospital), C. Miall (University of Birmingham).

**2013-2015:** Grant from the "Centre Francilien de l'Innovation". Project: Elaboration and evaluation of a visuo-tactile supplementation device for blind persons (PI: M. Auvray, 23 K€). Partners: Company Caylar, Ile de France (08/2013-12/2015).

**2013:** Program DEFI-SENS from the CNRS (MI). Optimisation of sensory substitution devices using crossmodal correspondences (PI: M. Auvray, 30 K€). Partners: O. Deroy (University of London), S. Hanneton (UMR 8119), K. O'Regan (UMR 8242), I. Fasiello (UMR 7222), F. de Vignemont (UMR 8129).

**2012:** Program DEFI-SENS from the CNRS (MI). Fundamental challenges for perceptual supplementation (PI: M. Auvray, 20 K€). Partners: O. Deroy (University of London), S. Hanneton (UMR 8119), K. O'Regan (UMR 8242), V. Hayward (UMR 7222).

**2011-2014:** Grant from the French national agency for research (ANR-JCJC), project FRESCO (ANR-11-JSH2-003-1). Frames of Reference in Spatial Cognition (PI: M. Auvray, 169,94 K€, 75%). Partners: Jérôme Dokic, Frédérique de Vignemont (IJN, UMR 8129), Mehdi Ammi, Sylvain Caillou (LIMSI, UPR 3251), Charles Spence (Oxford University).

**2009-2011:** Grant from the French national agency for research (ANR-08-TECS-0011). Project NAVIG. Navigation aided by artificial vision and GNSS (PI: Jouffrais, IRIT, UMR 5505, 109 K€). Partners: S. Thorpe (CerCo, UMR 5549), M. Auvray, M. Denis, B. Katz (LIMSI, UPR 3251), Navocap, Institut des Jeunes Aveugles.

**2008-2010:** Post-doctoral Fellowship Marie Curie (Intra European Fellowship).

**2006-2007:** Post-doctoral Fellowship FWO (Belgium).

**2005-2006:** Post-doctoral Fellowship Lavoisier, French Ministry of Foreign Affairs.

**2004:** Fellowship from the Fondation de France (to develop a project based on the technologies of sensory substitution).

**2001-2004:** Ph.D. Fellowship from the A.N.R.T. (French National Association for Technical Research).

**2000-2001:** Master 2 fellowship, Ecole des Hautes Etudes en Sciences Sociales.

## Grants obtained by PhD and post-doctoral researchers working under my supervision

**2015-2018:** PhD grant CIFRE, in co-supervision with Jérémy Lafraire (Institut Paul Bocuse) in collaboration with the company Pernod Ricard, for Jérémy Roque (36 months).

**2015-2017:** Post-doctoral fellowship CONICET (National Scientific and Technical Research Council, Argentine), in collaboration with Claudia Arias (Cordoba, Argentine), for Laura Fernandez (24 months).

**2014-2015:** Post-doctoral fellowship from "Fondation des Aveugles de Guerre », for Gabriel Arnold (12 months).

**2013-2015:** Post-doctoral fellowship from the foundation Fyssen, for Jessica Hartcher O'Brien (24 months).

**2010-2013:** PhD grant from the university Paris 11, in collaboration with Mehdi Ammi (LIMSI, UPR 3251), for Adrien Girard (36 months).
Editorial activities and organisations

2016. Organization of the Symposium *Sensory Substitution* at *International Congress of Psychology*, 24-29 July, Yokohama, Japan.

2014. Editor of the special issue *Sensory Substitution* in the journal *Multisensory Research*.

2014. Co-editor of *Haptics: Neuroscience, Devices, Modeling, and Applications*, Springer.

Co-presidency of the program for the conference *Eurohaptics* 2014, 24-26 June, Versailles, France. The conference gave rise to the abovementioned journal special issue *Haptics*.

2014. Member of the Copil of the experimental platform at the DEC.

2002. Organization of the *Third Workshop on Multimodal Interaction in Perception*, 28-29 October, Paris, France.

Selection committee

2016. Selection committee for the lecturer positions « 16 MCF0065, cognition et action» and « 16 MCF0125, mémoire et conscience » University Paris 5.

2015. Selection committee for the lecturer positions « 16 MCF0120, perception et action » and « 16 MCF1826, mémoire humaine » University Paris 5.

PARTICIPATION TO THE PRESS, TV, AND RADIO FOR THE GENERAL PUBLIC

Participation to journals for the general public

Auvray, M., & Dauphin, B. (2015). Les sens cachés du vin. *Revue des Œnologues*, 157, p. 21.

Dauphin, B., & Auvray, M. (2014). Le vin, une expérience multisensorielle. *Revue des Œnologues*, 41, 17-18.

Auvray, M., & O'Regan, J. K. (2003). Voir avec les oreilles: Enjeux de la substitution sensorielle. *Pour la Science*, 39, 30-35.

TV, radio, and public events

Participation to the documentary « Les forces du toucher » in the series « Nos cinq sens », broadcasted on Canal+, Planete+, TV5Monde, TV5Québec & RTBF in January 2018.
Realisation: Philippe Allante. Filming the 8 & 28 March 2017.

Participation to the audio documentary « Il faut croire pour voir » broadcasted on France Culture the 8 Mars 2017, Realisation: Tony Hayere, recorded the 16 January 2017.

Auvray, M. (2017). Les dispositifs de suppléance perceptive à destination des personnes non voyantes. *Conférence – Débat « Sciences et Cécités »*, Voir avec les oreilles, voir avec les doigts, 8 April, Blois, France.

Participation to the show ‘On n’est pas que des cobayes’ on sensory substitution, filming in November 2013, broadcasted the 31 January 2014.

Bloomsburry Festival, 19th October 2013, Centre for the Study of the Censes, Institute of Philosophy, London.

Presentation of posters during “Forum des Laboratoires en Sciences Cognitives d’Ile de France” (2002 and 2003)
## STUDENTS, PHD, AND POST-DOC SUPERVISION

2018-2020. Shared supervision (50%) with Vincent Hayward (ISIR) of a post-doctoral researcher (Xavier Job), ANR project: « Developmental tool mastery ».

2018-2020. Shared supervision (50%) with Vincent Hayward (ISIR) of a post-doctoral researcher (Louise Kirsch). Title of the project: «The processes of exteriorization and embodiment in tool use ».

2018-2019. Shared supervision (1/3) with Vincent Hayward (ISIR) and Ophelia Deroy (London) of a post-doctoral researcher (Antonio Cataldo), funded by Oculus VR. Title of the project: « Distal projection of tactile sensations from oculomotor signals »

2017-2018. Supervision of a student in Master 2 in Psychology at the university Paris 5 (Yann Kechabia). Title of the research: « Retour de proprioception artificiel par stimulation vibrotactile ».

2017-2018. Supervision of a student in Master 1 of the Cogmaster (Nathan Risch). Title of the research: « Existe-t-il une différence de toucher entre soi et autrui ? »

2017-2018. Supervision of a student in Master 1 of the Cogmaster (Daphne Parmentier). Title of the research: « Différences individuelles en substitution sensorielle visuo-auditive ».

2017-2018. Supervision of 6 students in third year of Polytechnic School. Title of the research: « Substitution sensorielle: voir par le toucher ».

2015-2018: Shared supervision (50%) with Jérémy Lafraire (Institut Paul Bocuse) of a PhD student (Jérémy Roque) CIFRE project in collaboration with Pernod Ricard. Title of the PhD: « Perception multimodale des boissons alcoolisées: le cas particulier de la perception de fraîcheur ».

2016-2017. Supervision of a student in M1 of the Cogmaster (Sacha Yesislatay). Title of the research : « Cognition spatiale chez les patients atteints d'autisme ».

2015-2017: Shared supervision (50%) with Claudia Arias (Cordoba, Argentina) of a post-doctoral researcher (Laura Fernandez) with a post-doctoral Grant CONICET (National Scientific and Technical Research Council).

2016-2017. Supervision of a student in M2 in Psychology at the University Paris 5 (Chenyang Wang). Title of the research: « Conscience corporelle et cognition spatiale ».

2016-2017. Supervision of a student in M2 in Psychology at the University Paris 5 (Irène Rabotin). Title of the research: « Le rôle de l'expérience visuelle pour l'appropriation d'un dispositif de substitution sensorielle ».

2016-2017. Supervision of a student in M1 in Psychology at the University Paris 8 (Sandra Inard). Title of the research: « Etude des facteurs psychologiques déterminant les cadres de références utilisés ».

2015-2016. Supervision of a student in M2 of the Cogmaster (Jacques Pesnot-Lerousseau). Title of the research: « Etude des processus visuels et auditifs impliqués dans l'utilisation du dispositif de substitution sensorielle The vOICe ».

2012-2016: Supervision of a post-doctoral researcher (Gabriel Arnold) with grants from the ANR FRESCO (2 years), post-doctoral fellowship from "Fondation des Aveugles de
Guerre” (1 year), fellowship from the Labex SMART (1 year in co-supervision with Nathanael Jarasse, 50%).

2014-2015. Shared supervision with Fabrice Sarlegna (CNRS UMR 7287), of a student (Alix Renault) in M2 STAPS. Title of the research: « Influence de la proprioception sur la représentation de l'espace ».

2013-2015. Supervision of a post-doctoral researcher (Jessica Hartcher O'Brien), with a grant from Fyssen.

2013-2014: Shared supervision with Frédérique de Vignemont (CNRS UMR 8129) of a student in M2 of the PhilMaster (Kahina Ouibrahim). Title of the research: « Le sens du self ».

2013-2014: Supervision of a student in M2, CNA (Systèmes complexes: Cognitions naturelle et artificielle) (Fatma Elleuch). Title of the research: « Inversion ou renforcement de la correspondance intermodale entre la fréquence auditive et l'élévation visuelle ».

2013-2014. Supervision of three students (Erika Kaiser, Marie Lorenz, Frédéric Albert) in Master 2 Ingénierie de l'innovation, Valorisation de la recherche et transfert de compétences. Title of the research: « Matrice de suppléance perceptive visuo-tactile pour personnes non-voyantes ».

2012-2013: Supervision of a student in M2, Cogmaster (Kahina Ouibrahim). Title of the research: « L'effet des facteurs de dominance sur la prise de perspective spatiale ».

2011-2012: Shared supervision with Sylvain Hanneton (CNRS, UMR 8119) of a student in M2, VHMA, Paris 5 (Philippe Herquel). Title of the research: « L'apprentissage de la pratique d'un jeu vidéo purement sonore ».

2010-2014: Shared supervision (50%) with Mehdi Ammi (CNRS, UPR 3251) of a PhD student from Paris 11 (Adrien Girard). Title of the PhD: « Interactions visio-haptiques pour la coordination et la conscience situationnelle partagée en environnement virtuel collaboratif ».

2009-2011: Shared supervision with Michel Denis (CNRS, UPR 3251) of a post-doctoral researcher (Mathieu Gallay), grant from the ANR Navig.

2010-2011: Shared supervision with Sylvain Hanneton (CNRS, UMR 8119) of a student in M2 VHMA, Paris 5 (Vincent Forma). Title of the research: « Les oreilles sur la main: l'étude de l'apprentissage d'un nouveau couplage audio-moteur lors d'une tache de pointage en environnement sonore 3D ».

2009-2010: Supervision of a student in M2 Ergonomie et Facteurs Humains, Paris 11 (Lucie Brunet). Title of the research: « Etude des stratégies et des besoins des personnes non-voyantes lors de la navigation pour la conception d'un dispositif d'aide performant et accepté ».

2009-2010: Supervision of a student in M2 ATIAM, Paris 6 (Lourdes Garcia). Title of the research: « Étude des représentations spatiales en environnement sonore ».

2008-2009: Shared supervision with Mark Wexler (CNRS, UMR 8158) of a student in M2 of the Cogmaster. Title of the research: « Le rôle de l'activité motrice dans le raisonnement spatial ».

02/2005-06/2005: Shared supervision with Charles Lenay (Université de Technologie de Compiègne) of two students in 4^th^ year of engineering school. Title of the research: « Les interactions perceptives en environnement minimaliste ».
02/2004-06/2004: Shared supervision with Sylvain Hanneton (CNRS, UMR 8119) of a student in M1 STAPS, Paris 5.

10/2003-01/2004: Shared supervision with Charles Lenay (Université de Technologie de Compiègne) of four students in 3ème year of engineering school.

04/2003-07/2003: Shared supervision, with J. Kevin O'Regan of a student in 3ème year of Polytechnic school.

06/2003: Supervision of three students in 1st year of Psychology, Université Paris 5.

## Participation to PhD committees

- Lise Obeika, "Interplay between multisensory integration and social interaction in auditory space". Defended the 29 November 2017. Isabelle Viaud-Delmon (PhD supervisor), Ana Tajadura-Jiménez (referee), Stefan Glasauer (referee), Guenther Knoblich (examiner), Malika Auvray (examiner), Giandomenico Iannetti (examiner).

- Julien Laroche, « Le processus d'interaction comme attracteur dans la coordination temporelle entre les personnes ». Defended the 24 June 2013. Pr Eric Brangier (directeur de thèse), Anna-Maria Berardi (PhD supervisor), Pr Olivier Gapenne (referee), Pr Maya Gratier (referee), Malika Auvray (examiner).

- Adrien Girard, « Interactions visio-haptiques pour la coordination et la conscience situationnelle partagée en environnement virtuel collaboratif ». Defended the 12 June 2014. M. Ammi (PhD supervisor), M. Auvray (PhD co-supervisor), J.-M. Burkhardt (referee), G. Moreau (referee), A. Lécuyer (examiner), A. Vilnat (examiner), G. Zachmann (examiner), Y. Bellik (examiner).

## V. Teaching

Total: 480 hours. Including 54h CM, 380h TD, 85h tutoring

<table><thead><tr><th>Institution</th><th>Title</th><th>Public</th><th>Years</th></tr></thead><tbody><tr><td>Département d'Etudes cognitives (DEC - ENS)</td><td>Cognition incarnée</td><td>Master (année 2)</td><td>2017-2018 (12h)<br>2016-2017 (12 h)<br>2015-2016 (12 h)<br>2014-2015 (12 h)<br>2013-2014 (12 h)<br>2012-2013 (12 h)</td></tr><tr><td>UPMC</td><td>Perception multisensorielle appliquées aux technologies de réhabilitation</td><td>Master (année 2)<br>Filière MSR</td><td>2017-2018 (8h)</td></tr><tr><td rowspan="3">Université Paris 5</td><td>Substitution sensorielle</td><td>Master (année 2)<br>Filière PCFA</td><td>2017-2018 (6h)<br>2016-2017 (4 h)<br>2015-2016 (2 h)</td></tr><tr><td>Psychologie Expérimentale</td><td>Licence (année 1) de psychologie</td><td>2003-2004 (48 h)<br>2002-2003 (48 h)</td></tr><tr><td>Méthodologies en psychologie (tutorat)</td><td>Licence (année 1) de psychologie</td><td>2002-2003 (45 h)</td></tr></tbody></table>
<table><tr><td>ENSCI - les ateliers</td><td>Perception multisensorielle et synesthesia</td><td>1<sup>ère</sup> à 5<sup>ème</sup> année de l'école de design</td><td>2017-2018 (10h)</td></tr><tr><td>Université Paris 11</td><td>Approfondissements en Psychologie Cognitive</td><td>Master (année 2)</td><td>2009-2010 (9 h)<br>2008-2009 (9 h)</td></tr><tr><td rowspan="3">Université de Technologie de Compègne</td><td>Sciences cognitives et naturalisation de l'esprit</td><td>1<sup>ère</sup> à 5<sup>ème</sup> année de l'école d'ingénieur</td><td>2004-2005 (68 h)<br>1/2 ATER Psychologie</td></tr><tr><td>Apprentissage et perception</td><td>1<sup>ère</sup> à 5<sup>ème</sup> année de l'école d'ingénieur</td><td>2002-2003 (8 h)<br>2001-2002 (8 h)</td></tr><tr><td>Psychoacoustique</td><td>5<sup>ème</sup> année de l'école d'ingénieur</td><td>2002-2003 (4 h)<br>2001-2002 (4 h)</td></tr><tr><td>Université Paris 13</td><td>Perception et cognition</td><td>Licence (année 1) de psychologie</td><td>2002-2003 (48 h)<br>2001-2002 (35 h)</td></tr><tr><td>ESIGELEC de Rouen</td><td>Perception et action</td><td>5<sup>ème</sup> année de l'ESIGELEC</td><td>2002-2003 (2 h)</td></tr><tr><td>Ecole Normale Supérieure</td><td>La substitution sensorielle</td><td>1<sup>ère</sup> année de l'ENS</td><td>2002-2003 (2 h)</td></tr><tr><td>Université de Nantes</td><td>Méthodologie en philosophie (tutorat)</td><td>Licence (année 1) de philosophie</td><td>1998-1999 (20 h)<br>1997-1998 (20 h)</td></tr></table>

## PUBLICATIONS

h-index: 19 / Citations: 1921 (Google Scholar)

The names of the PhD and master students I have supervised are underlined and in bold. The names of the post-doctoral researchers I have supervised are underlined.

## ARTICLES

1. Roque, J., Lafraire, J., Spence, C., & Auvray, M. (in press). The influence of audiovisual perceptual features on the categorization of freshness in beverages. *Journal of Sensory Studies*.

2. Renault, A. G., Auvray, M., Parseihian, G., Miall, R. C., Cole, J., & Sarlegna, F. (2018). Does proprioception influence human spatial cognition? A study on individuals with massive deafferentation. *Frontiers in Psychology*. doi.org/10.3389/fpsyg.2018.01322

3. Arnold, G., & Auvray, M. (2018). Tactile recognition of visual stimuli: Specificity versus generalization of perceptual learning. *Vision Research*. doi.org/10.1016/j.visres.2017.11.007

4. Brunet, L. Darses, F., & Auvray, M. (2018). Strategies and needs of blind pedestrians during urban navigation, *Le Travail Humain*. 81, 141-171.

5. Roque, J., Auvray, M., & Lafraire, J. (2018). Investigating the cognitive mechanisms underlying freshness perception: a research review. *Frontiers in Psychology*, 8, 2360. doi.org/10.3389/fpsyg.2017.02360

6. Arnold, G., Spence, C., & Auvray, M. (2017). A unity of the self or a multiplicity of locations? How the graphesthesia task sheds light on the role of spatial perspectives in
bodily self-consciousness. *Consciousness & Cognition*, 56, 100-114. doi:
10.1016/j.concog.2017.06.012

7. **Arnold, G.**, **Pesnot-Lerousseau, J.** & Auvray, M. (2017). Individual differences in sensory substitution. *Multisensory Research*, 6, 579-600. doi: 10.1163/22134808-00002561

8. **Arnold, G.** & Auvray, M. (2017). The graphesthesia paradigm: drawing letters on the body to investigate the embodied nature of spatial perspective taking. *I-Perception*, 8, 1-5. doi: 10.1177/2041669517690163

9. Deroy, O., Fasiello, I., Hayward, V., & Auvray, M. (2016). Differentiated audio-tactile correspondences in sighted and blind individuals. *Journal of Experimental Psychology: Human Perception and Performance*, 42, 1204-1214. doi:10.1037/xhp000015

10. O'Brien, J., & Auvray, M. (2016). Cognition overrides orientation dependence in tactile viewpoint selection. *Experimental Brain Research*, 234, 1885-1892. doi: 10.1007/s00221-016-4596-6

11. **Arnold, G.**, Spence, C., & Auvray, M. (2016). Taking someone else's spatial perspective: Natural stance or effortful decentring? *Cognition*, 148, 27-33. doi:10.1016/j.cognition.2015.12.006

12. Hanneton, S., **Herquel, P.**, & Auvray, M. (2015). Intermodal recoding of a video game: Learning to process signals for motion perception in a pure auditory environment. *International Journal of Adaptive Control and Signal Processing*, 29, 1475-1483. doi: 10.1002/acs.2549

13. O'Brien, J., & Auvray, M. (2014). The process of distal attribution illuminated through studies of sensory substitution. *Multisensory Research*, 27, 421-441. doi: 10.1163/22134808-00002456

14. Auvray, M., & Harris, L.R. (2014). The state of the art of sensory substitution. *Multisensory Research*, 27, 265-269. doi: 10.1163/22134808-00002464

15. **Arnold, G.**, & Auvray, M. (2014). Perceptual learning: Tactile letter recognition transfers across body surfaces. *Multisensory Research*, 27, 71-90. doi: 10.1163/22134808-00002443

16. Deroy, O., & Auvray, M. (2012). Reading the world through sensory substitution devices. *Frontiers in Theoretical and Philosophical Psychology*, 3, 457. doi: 10.3389/fpsyg.2012.00457

17. Auvray, M., & Rohde, M. (2012). Perceptual crossing: The simplest online paradigm. *Frontiers in Human Neuroscience*, 6, 181. doi: 10.3389/fnhum.2012.00181

18. Simard, J., Ammi, M., & Auvray, M. (2012). Collaborative strategies for the search of 3D targets in molecular environments. *IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews*, 6, 1555-1565. doi: 10.1155/2012/546521

19. Auvray, M., Hoellinger, T., Hanneton, S., & Roby-Brami, A. (2011). Perceptual weight judgments when viewing one's own and other's movements under minimalist conditions of visual presentation. *Perception*, 40, 1081-1103. doi:10.1068/p6879

20. Auvray, M., Gallace, A., & Spence, C. (2011). Tactile short term memory for stimuli presented on the fingertips and across the rest of the body surface. *Attention, Perception, & Psychophysics*, 73, 1227-1241. doi: 10.3758/s13414-011-0098-6
21. Hanneton, S., Auvray, M., & Durette, B. (2010). The Vibe: A versatile vision-to-audition sensory substitution device. *Applied Bionics and Biomechanics, 7*, 269-276. doi: 10.1080/11762322.2010.512734

22. Auvray, M., Myin, E., & Spence, C. (2010). The sensory-discriminative and affective-motivational aspects of pain. In F. McGlone (Ed.) *Touch, Temperature, Pleasure, and Pain*, special issue of *Neuroscience and Biobehavioral Reviews, 34*, 214-223. doi:10.1016/j.neubiorev.2008.07.008

23. Auvray, M., & Myin, E. (2009). Perception with compensatory devices. From sensory substitution to sensorimotor extension. *Cognitive Science, 33*, 1036-1058. doi: 10.1111/j.1551-6709.2009.01040.x

24. Auvray, M., Lenay, C., & Stewart, J. (2009). Perceptual interactions in a minimalist environment. *New Ideas in Psychology, 27*, 79-97. doi: 10.1016/j.newideapsych.2007.12.002

25. Auvray, M., Gallace, A., Hartcher-O’Brien, J., Tan, H. Z., & Spence, C. (2008). Tactile and visual distractors induce change blindness for tactile stimuli presented on the fingertips. *Brain Research, 1213*, 111-119. doi:10.1016/j.brainres.2008.03.015

26. Auvray, M., & Spence, C. (2008). The multisensory perception of flavour. *Consciousness & Cognition, 17*, 1016-1031. doi:10.1016/j.concog.2007.06.005

27. Auvray, M., Gallace, A., Tan, H. Z., & Spence, C. (2007). Crossmodal change blindness between vision and touch. *Acta Psychologica, 126*, 79-97. doi:10.1016/j.actpsy.2006.10.005

28. Auvray, M., Philipona, D., O’Regan, J. K., & Spence, C. (2007). The perception of space and form recognition in a simulated environment: The case of minimalist sensory-substitution devices. *Perception, 36*, 1736-1751. doi:10.1068/p5800

29. Auvray, M., Hanneton, S., & O’Regan, J. K. (2007). Learning to perceive with a visuo-auditory substitution system: Localization and object recognition with The Voice. *Perception, 36*, 416-430. doi:10.1068/p5631

30. Gallace, A., Auvray, M., & Spence, C. (2007). The modulation of haptic line bisection using a visual illusion and optokinetic stimulation. *Perception, 36*, 1003-1018. doi:10.1068/p5457

31. Gallace, A., Auvray, M., Tan, H. Z., & Spence, C. (2006). When visual transients impair tactile change detection: A novel case of crossmodal change blindness? *Neuroscience Letters, 398*, 280-285. doi:10.1016/j.neulet.2006.01.009

32. Auvray, M., Hanneton, S., Lenay, C., & O’Regan, J. K. (2005). There is something out there: Distal attribution in sensory substitution, twenty years later. *Journal of Integrative Neuroscience, 4*, 505-521. doi: 10.1142/S0219635205001002

33. Auvray, M. (2003). Bergson, une théorie sensorimotrice de la perception. *Psychologie et Histoire, 4*, 61-100.

34. Auvray, M., & O’Regan, J. K. (2003). L’influence des facteurs sémantiques sur la cécité aux changements progressifs dans les scènes visuelles. *L’Année Psychologique, 103*, 9-32. doi: 10.3406/psy.2003.29621
BOOK CHAPTERS

1. Ptito, M., Iversen, K., Auvray, M., Deroy, O., Kupers, R. (forthcoming). The pluripotent visual cortex in congenital blindness. In F. Macpherson (Ed.). *Sensory Substitution and Augmentation*, London: Proceedings of the British Academy.

2. Auvray, M., & Farina, M. (2017). Patrolling the boundaries of synaesthesia: A critical appraisal of transient and artificially-acquired forms of synaesthetic experiences. In O. Deroy (Ed.). *Synaesthesia: Philosophical & Psychological Challenges* (pp. 248-274), Oxford: Oxford University Press. doi: 10.1093/oso/9780199688289.003.0013

3. O'Brien, J., Auvray, M., & Hayward, V. (2015). Perception of distance-to-obstacle through time-delayed tactile feedback. *Proceedings of the 2015 IEEE World Haptics Conference* (pp. 7-12). doi: 10.1109/WHC.2015.7177683

4. Auvray, M., & Deroy, O. (2015). How do synesthetes experience the world? In M. Matthen (Ed.), *Oxford Handbook of Philosophy of Perception* (pp. 640-658), Oxford: Oxford University Press. doi: 10.1093/oxfordhb/9780199600472.013.027

5. O'Brien, J., Terekhov, A., Auvray, M., & Hayward, V. (2014). Haptic shape constancy across distance. *Haptics: Neuroscience, Devices, Modeling, and Applications Lecture Notes in Computer Science* (pp. 77-84). Springer: Heidelberg, New-York, Dordrecht. *Proceedings of Eurohaptics*, 24-26 June, Versailles, France. doi: 10.1007/978-3-662-44193-0_11.

6. Deroy, O., & Auvray, M. (2014). A crossmodal perspective on sensory substitution. In S. Biggs, M. Matthen, & D. Stokes (Eds.), *Perception and Its Modalities* (pp. 327-349), Oxford: Oxford University Press. doi: 10.1093/acprof:oso/9780199832798.003.0014

7. Spence, C., Auvray, M., & Smith, B. (2014). Confusing tastes with flavours. In S. Biggs, M. Matthen, & D. Stokes (Eds.), *Perception and Its Modalities* (pp. 247-276), Oxford: Oxford University Press. doi: 10.1093/acprof:oso/9780199832798.003.0011

8. Gallay, M., Denis, M., & Auvray, M. (2013). Navigation assistance for blind pedestrians: Guidelines for the design of devices and implications for spatial cognition. In T. Tenbrink, J. Wiener, & C. Claramunt (Eds.), *Representing space in cognition: Interrelations of behaviour, language, and formal models*, Oxford: Oxford University Press. doi: 10.1093/acprof:oso/9780199679911.003.0011

9. Tarroux, P., & Auvray, M. (2012). La perception visuelle. In M. Denis (Ed.), *La Psychologie Cognitive* (pp. 39-69), Paris: Editions de la Maison des Sciences de l'Homme.

10. Auvray, M., & Fuchs, P. (2007). Perception, immersion et interactions sensorimotrices en environnement virtuel. In A. Grumbach & E. Klinger (Eds.), *Réalité Virtuelle et Cognition* (pp. 23-36), numéro spécial de *Intellectica*.

11. Auvray, M. (2006). Remplacer un sens par un autre: La suppléance perceptive. In P. Fuchs (Ed.), *Le Traité de la Réalité Virtuelle III*, Vol.1, L'Homme et l'Environnement Virtuel (pp. 173-188). Paris: Les Presses de l'Ecole des Mines.

12. Auvray, M. (2006). Théories de la perception: Le monde comme mémoire externe. In P. Fuchs (Ed.), *Le Traité de la Réalité Virtuelle III*, Vol.1, L'Homme et l'Environnement Virtuel (pp. 209-228). Paris: Les Presses de l'Ecole des Mines.
## OTHER PUBLICATIONS

1. Katz, B., Kammoun, S., Parseihian, G., Gutierrez, O., Brilhault, A., *Auvray*, M., Truillet, P., Thorpe, S., & Jouffrais, C. (2012). NAVIG: Augmented reality guidance system for the visually impaired. *Virtual Reality Journal*, 16, 253-269. doi: 10.1007/s10055-012-0213-6.

2. Katz, B., Dramas, F., Parseihian, G., Gutierrez, O., Kammoun, S., Brilhault, A., **Brunet**, L., **Gallay**, M., Oriola, B., *Auvray*, M., Truillet, P., Thorpe, S., & Jouffrais, C. (2012). NAVIG guidance system for the visually impaired using virtual augmented reality. *Technology and Disability*, 24, 163-178. doi: 10.3233/TAD-2012-0344

3. Kammoun, S., Parseihian, G., Gutierrez, O., Brilhault, A., Serpa, A., Raynal, M., Oriola, B., Macé, M., *Auvray*, M., Denis, M., Thorpe, S., Truillet, P., Katz B., Jouffrais, C., (2012). Navigation and space perception assistance for the visually impaired: The NAVIG project. *IRBM, Ingénierie et Recherche Biomédicale*, 33, 182-189. doi: 10.1016/j.irbm.2012.01.009

4. Gauchou, H., & Auvray, M. (2007). L'inhibition comme absence de représentation: L'exemple de la cécité aux changements. *Le Journal des Psychologues*, 244, 29-34. doi: 10.3917/jdp.244.0029

5. Auvray, M., Lenay, C., O'Regan, J. K., & Lefèvre, J. (2005). Suppléance perceptive, immersion et informations proprioceptives. *Arobase*, 1, 94-114.

6. Khatchatourov, A., & Auvray, M. (2005). L'outil modifie-t'il la perception ou la rend-il possible ? *Arobase*, 1, 65-70.

7. Auvray, M. (2005). La suppléance perceptive: Implications sur les théories de la perception. *VOIR*, n° 30-31. Les aspects culturels de la vision et les autres modalités perceptives III. L'ouïe, 2-13.

## MANUSCRIPTS SUBMITTED AND IN PREPARATION

**Arnold**, G., Sarlegna, F., Fernandez, L., & Auvray, M. (submitted). The physical body plays a different role in locating and orienting the experienced self in deafferented patients and in controls. *Frontiers in Neurorobotics*.

**Pesnot-Lerousseau**, J., **Arnold**, G., & Auvray, M. (submitted). Visualizing sounds: training-induced plasticity with a visual-to-auditory conversion device.

**O'Brien**, J., Hayward, V., & Auvray, M. (in preparation). Frames of reference at the tip of your finger and in the palm of your hand.

**Arnold**, G. & Auvray, M. (in preparation). Blind and sighted people differ in the way they mentally represent space from different perspectives.

**Roque**, J., Lafraire, J., & Auvray, M., (in preparation). Audiovisual crossmodal correspondences: the case of carbonation in beverages.

**Risch**, N., **Arnold**, C., & Auvray, M. (in preparation). The numbness illusion influences how you touch yourself.
COMMUNICATIONS

* *Communications in international conferences with proceedings are indicated with an asterisk.*

* Auvray, M., & Arnold, G., Pesnot-Lerousseau, J. (2018). Training-induced plasticity with a visual-to-auditory conversion system. Seeing the thunder while still hearing it. *International Multisensory Research Forum*. 14-17 June, Toronto, Canada.

* **Roque**, J., Lafraire, J., & Auvray, M. (2018). Audiovisual crossmodal correspondences between bubbles’ size and pouring sounds’ pitch in carbonated beverages. *International Multisensory Research Forum*. 14-17 June, Toronto, Canada (Poster).

Auvray, M. (2018). *Taxonomizing the sense modalities without circularity*. Workshop: The Philosophy of Multisensory Perception, 14 June, Toronto, Canada.

* Auvray, M., & Arnold, G., & Pesnot-Lerousseau, J. (2018). Seeing the thunder while still hearing it. Functional plasticity with sensory substitution devices. *European Workshop on Imagery and Cognition*. 7-9 June, Padova, Italy.

* Auvray, M., & Arnold, G. (2017). Differences in the way sighted, early, and late-blind people mentally represent tactile stimuli from inside or from outside the body. *4th International Symposium Low Vision and the Brain*, 24-26 November, Berlin, Germany.

Auvray, M. (2017). Les processus spatiaux impliqués dans la substitution sensorielle, *Séminaire du Neuroscience Research Centre of Lyon, INSERM U1028, CNRS UMR529*, 19 June, Lyon (Bron).

Auvray, M. (2017). Multisensory and spatial processes in sensory substitution. *Séminaire du Laboratoire de Neurosciences Intégratives et Adaptatives*, UMR 7260, 24 October, Marseille

Auvray, M. (2017). Multisensory and spatial processes in sensory substitution, Séminaire du Laboratoire de Psychologie, 19 June, Boulogne-Billancourt, France.

* **Roque**, J., Lafraire, J., & Auvray, M. (2017). Investigating the multisensory perception of freshness in beverages: the case of audio-visual interactions. *International Multisensory Research Forum*, 19-22 May, Nashville, USA.

* Auvray, M., & Arnold, G. (2016). Taking into account individual differences in multisensory processing between sighted and visually impaired people to improve sensory substitution. *Israelian Society for Neuroscience*, 4-6 December, Eilat, Israel.

* Arnold, G., & Auvray, M. (2016). The influence of blindness and short-term visual deprivation on spatial cognition. *Israelian Society for Neuroscience*, 4-6 December, Eilat, Israel (poster).

Auvray, M. (2016). Multisensory perception and spatial cognition: Taking into account individual differences to improve sensory substitution. Workshop Computational Touch, 8-9 September, Paris, France.

* Arnold, G., & Auvray, M. (2016). Design and use of sensory substitution devices: differences between sighted and visually impaired people. *International Congress of Psychology*, 24-29 July, Yokohama, Japan.

* Auvray, M., & Arnold, G. (2016). Perceiving tactile objects from inside or from outside the body: sighted and visually impaired people differ in the way they mentally represent space
from different perspectives. *Association of the Scientific Study of Consciousness*, 13-18 June, Buenos Aires, Argentina (poster).

* Auvray, M., & *Arnold, G.* (2016). Learning to use sensory substitution devices: individual specificities and differences between sighted and visually impaired people. *European Workshop on Imagery and Cognition*, 6-8 June, Paris, France.

* *Arnold, G.*, & Auvray, M. (2016). Blind and sighted people differ in the way they mentally represent space from different perspectives. *European Workshop on Imagery and Cognition*, 6-8 June, Paris, France (poster).

* **Roque, J.**, Auvray, M., Garrel, C., & Lafraire, J. (2016). The multisensory perceptions of flavour and freshness: a new perspective. *British Feeding and Drinking Group*, 08 April, London, UK.

Auvray, M. (2016). Replacing one modality by another: Learning to use sensory substitution devices. *Séminaire de l'Institut des Sciences du Mouvement*, 03 March, Marseille, France.

Auvray, M., & Deroy, O. (2015). Audio-tactile crossmodal correspondences in sighted, early, and late blind people. *Seminar of the Department of Philosophy*, 18 December, PUC: Rio de Janeiro, Brasil.

Auvray, M. (2015). Where I am? Studies on spatial perspective in the tactile sensory modality. *Seminar of the Department of Philosophy*, 14 December, PUC: Rio de Janeiro, Brasil.

Auvray, M. (2015). Multisensory perception. *Séminaire de Recherche de l'Institut Paul Bocuse*, 13 November, Lyon, France.

* *Hartcher-O'Brien, J.*, Auvray, M., & Hayward, V. (2015). Perception of distance-to-obstacle through time-delayed tactile feedback. *World Haptics Conference*, 22-26 June, Evanston, IL, USA.

* *Arnold, G.*, *Hartcher-O'Brien, J.*, Hayward, V., Spence, C., & Auvray, M. (2015). Where am I? Investigating which perspectives are taken on ambiguous tactile symbols. *16th International Multisensory Research Forum*, 13-16 June, Pisa, Italy.

* *Hartcher-O'Brien, J.*, Auvray, M., & Hayward, V. (2015). Access to visual distance via a time-delayed tactile feedback device. *16th Multisensory Research Forum (IMRF)*, 13-16 June, Pisa, Italy.

* Beaudry-Richard, A., Harrar, V., Auvray, M., Spence, C., Kupers, R., & Ptito, M. (2015). The multisensory substitution device: replacing vision with multisensory perception *16th Multisensory Research Forum (IMRF)*, 13-16 June, Pisa, Italy (poster).

Auvray, M. (2015). Sensory substitution and augmentation in visual rehabilitation. *Seminar of the Department of Philosophy*, 20 April, PUC: Rio de Janeiro, Brasil.

Auvray, M. (2015). Changing spatial and mental perspective: Studies in touch. *Seminar of the Department of Philosophy*, 17 April, PUC: Rio de Janeiro, Brasil.

*Arnold, G., & Auvray, M. (2014). Perceptual learning, crossmodal interactions, and spatial reference frames: Implications for sensory substitution. *Journée Robotique et Neuroscience*, 17 December, Paris, France.

Sarlegna, F., & Auvray, M. (2014). Déficience perceptive et suppléance multimodale. *Technologies appliquées au handicap: de la compensation à la participation*, DVS Defisens, 26 September, Paris, France.
* Auvray, M., Deroy, O., Fasiello, I., & Hayward, V. (2014). Crossmodal correspondences between pitch and tactile movement in sighted, early, and late blind people. *European Society for Philosophy and Psychology*, 16-19 September, Noto, Sicily.

Arnold, G., & Auvray, M. (2014). The left-side bias in tactile recognition is modulated by the adopted reference frame. *4th North Sea Meeting on Brain Asymmetries*, 10-14 September, Durham, UK (poster).

* **Girard, A.**, Auvray, M., Ammi, M. (2014). Collaborative metaphor for haptic designation in complex 3D environments. *ACM Symposium on Applied Perception* (proceedings, pp. 31-37, doi: 10.1145/2628257.2628262), 08-09 August, Vancouver, Canada.

* Hartcher-O'Brien, J., Terekhov, A., Auvray, M., & Hayward, V. (2014). Haptic shape constancy across distance. Eurohaptics, 24-26 June, Versailles, France (poster).

* Arnold, G., & Auvray, M. (2014). Tactile symbol recognition: Specificity versus generalization of learning. *15th Multisensory Research Forum (IMRF)*, 11-14 June, Amsterdam, Netherlands (poster).

* Hartcher-O’Brien, J., & Auvray, M. (2014). The bi-stable palm: clockwise versus counterclockwise tactile motion perception. *15th Multisensory Research Forum (IMRF)*, 11-14 June, Amsterdam, Netherlands (poster).

Auvray, M. (2014). Questioning the use of sensory substitution devices at the behavioral, neurophysiological and phenomenal levels. *Seminar of the Paris Consciousness/Self-consciousness [PaCS]*, 31 March, Institut Jean Nicod, Paris, France.

Auvray, M. (2014). Learning to use sensory substitution devices. *Seminar Paris Ouest*, 19 March, Nanterre, France.

* Auvray, M. (2014). Replacing one sense by another: Learning to use sensory substitution devices. *Symposium on Cerebral visual injury: dysfunction, plasticity, restoration and rehabilitation*; held during the 4th International Scientific Conference Restauración Neurológica, 5-7 March, La Habana, Cuba.

* Auvray, M. (2014). Replacing one sense by another: learning to use a sensory substitution device. *Neuroeng2014*, 2-3 February, Adelaide, Australia.

* Auvray, M., & Arnold, G. (2014). Perceptual learning of tactile letters transfers across body surfaces. *Australasian Neuroscience Society (ANS)*, 28-31 January, Adelaide, Australia (poster).

Auvray, M. (2014). L’apprentissage des dispositifs de suppléance perceptive. *Colloque Perception Augmentée et Multimodale*, 15 January, Paris, France.

Arnold, G., & Auvray, M. (2013). Adopting and changing reference frames when interpreting tactile stimulation received on the body surface. *Symposium: vision, action and concepts: Behavioural and neural basis of embodied perception and cognition*, 28-30 October, Lille (poster).

Auvray, M. (2013). Learning to use sensory substitution devices. *2nd HCC Summer School*, 17-19 July, Ircam, Paris.

Auvray, M. (2013). Spatial cognition and sensory substitution. *Research Conference on Sensory Substitution, Brain Plasticity and Visual Rehabilitation*, 6-9 June, Israel Institute for Advanced Studies, The Hebrew University of Jerusalem, Israel.
* **Arnold, G., O'Brien, J., Spence, C., & Auvray, M.** (2013). The cost of adopting and adapting tactile frames of reference. *14<sup>th</sup> Multisensory Research Forum (IMRF)*, 3-6 June, The Hebrew University of Jerusalem, Israel.

* Deroy, O., Fasiello, I., Hayward, V., & Auvray, M. (2013). Audio-tactile crossmodal correspondances. *14<sup>th</sup> Multisensory Research Forum (IMRF)*, 3-6 June, The Hebrew University of Jerusalem, Israel.

* **O'Brien, J., Spence, C., & Auvray, M.** (2013). Frames of reference at the tip of your fingers and in the palm of your hand. *14<sup>th</sup> Multisensory Research Forum (IMRF)*, 3-6 June, The Hebrew University of Jerusalem, Israel.

* **Girard, A., Bellik, Y., Auvray, M., & Ammi, M.** (2013). Visuo-haptic tool for collaborative adjustment of selections, *8<sup>th</sup> International Workshop on Haptic and Audio Interaction Design (HAID 2013)*, 18-19 April, Daejeon, Korea (proceedings pp.40-49).

* Deroy, O., & Auvray, M. (2013). A new Molyneux's problem: Sounds, shapes and arbitrary crossmodal correspondences. *Shapes 2.3: The shape of things*. 3-4 April, Rio de Janeiro, Brasil.

Auvray, M. & Deroy, O. (2013). Audio-visual substitution. *Sensory Substitution and Augmentation Conference*, 26-28 March, British Academy, London.

* **Girard, A., Bellik, Y., Auvray, M., & Ammi, M.** (2013). Collaborative adjustment of selection areas for polygonal modelling, *IEEE 3D User Interface (3DUI'13)*, 16-17 March, Orlando, USA (proceedings pp. 135-136).

Auvray, M. (2013). Distal attribution in sensory substitution. *Seminar From Perceptual Interaction to Extended Cognition*, 22-25 January, Compiègne, France.

* **Girard, A., Bellik, Y., Auvray, M., & Ammi, M.** (2012). Collaborative approach for dynamic adjustment of selection areas in polygonal modelling, *18<sup>th</sup> ACM symposium on Virtual Reality Software and Technology (VRST)*, 10-12 December, Toronto, Canada (proceedings pp. 205).

Auvray, M. (2012). Substitution sensorielle: verrous et prospective. *Defisens: Audiovis*, 15-17 November, Toulouse, France.

* **Girard, A., Ammi, M., & Auvray, M.** (2012). Haptic designation strategy for collaborative molecular modeling. *IEEE International Symposium on Haptic Audio Visual Environments and Games (HAVE 2012)*, 08-09 October, Munich, Germany (proceedings, pp.13-18).

* Auvray, M., & Deroy, O. (2012). The vertical integration of sensory substitution devices. *European Society for Philosophy and Psychology*, 28-31 August, London, UK.

* Auvray, M., Gallace, A., & Spence, C. (2012). Crossmodal change blindness between touch, vision, and audition. *30<sup>th</sup> International Congress of Psychology (ICP)*, 22-27 July, Capetown, South Africa.

* Auvray, M. (2012). Beyond vision: Integrating sensory substitution devices. *30<sup>th</sup> International Congress of Psychology (ICP)*, 22-27 July, Capetown, South Africa.

* Auvray, M., Gallace, A., & Spence, C. (2012). Crossmodal change blindness between vision, touch, and audition: the influence of the task. *Association for the Scientific Study of Consciousness (ASSC 16)*, 2-6 July, Brighton, UK (poster).
* Auvray, M., & Deroy, O. (2012). Interpreting sensory substitution beyond the perceptual assumption: An analogy with reading. *13<sup>th</sup> International Multisensory Research Forum (IMRF)*, 19-22 June, Oxford, UK.

Auvray, M. (2012). Spatial cognition and sensory substitution. *Mind and Perception Research Seminar*, 23 April, Glasgow, Scotland.

Auvray, M. (2012). Interactions crossmodales. *Séminaire du Laboratoire Vision, Action, Cognition, Université Paris Descartes*, 26 March, Boulogne Billancourt, France.

Auvray, M. (2012). A commentary of Randy Flanagan ‘on sensorimotor learning’. *Workshop Perceptual Learning and Perceptual Recognition I*, 19-20 March, York University, U.K.

Auvray, M. (2012). Interactions crossmodales: L’exemple de la cécité aux changements. *Séminaire de l’ETIS, Université de Cergy-Pontoise*, 16 March, Cergy-Pontoise, France.

* **Girard, A.**, Ammi, M., Simard, J., & Auvray, M. (2012). Improvement of collaborative selection in 3D complex environments. *IEEE Haptics Symposium*, 4-7 March, Vancouver, Canada (proceedings, pp. 281-288).

Auvray, M. (2012). Cognition spatiale et substitution sensorielle. *Seminaire Interactions Situées*, University Paris IV, 24 February, Paris, France.

* Forma, V., Hoellinger, T., Auvray, M., Roby-Brami, A., & Hanneton, S. (2011). Ears on the hand: reaching 3D audio targets. *SKILLS 2011*, 15-16 December, Montpellier, France (poster).

Auvray, M. & Deroy, O. (2011). Understanding sensory substitution devices beyond the perceptual assumption. *Neuroscience Seminar*, 17 November, Trinity College, Hartford, USA.

Deroy, O., & Auvray, M. (2011). Synesthesia and parasitic qualia. *Cognitive Science Seminar, CUNY*, 11 November, New York, USA.

* Auvray, M., Gallace, A., Yoshida, T., & Spence, C. (2011). Tactile change blindness induced by tactile and visual distractors. *12<sup>th</sup> International Multisensory Research Forum (IMRF)*, 17-20 October, Fukuoka, Japan (poster).

Auvray, M., & Deroy, O. (2011). Understanding sensory substitution devices beyond the perceptual assumption. *Workshop Feelings, Perception, and Action*, 6-7 October, ENS, Paris.

Auvray, M., & Deroy, O. (2011). Integrating sensory substitution devices: An analogy with reading. *Seminar of the Sensory Research Forum*, Institute of Philosophy, 6 October, London, UK.

* Simard, J., Ammi, M., & Auvray, M. (2011). How to improve group performances on collocated synchronous manipulation tasks. *In Proceedings of Joint Virtual Reality Conference (JVRC)*. 20-21 September, Nottingham, UK (pp. 91-94).

* Auvray, M. (2011). Perceiving with compensatory devices: Quasi-vision or new sensory modality? *American Philosophical Association Pacific Division*, 20-23 April, San Diego, USA.

* Simard, J., Ammi, M., & Auvray, M. (2010). Closely coupled collaboration for search tasks. *Virtual Reality, Software and Technology (VRST)*, 22-24 November, Hong-Kong, China (ISBN: 978-1-4503-0441-2; poster).