representation. In future research, one way to test for the influence of explicit representations on performance would be to compare the results of the first experiment with and without specific “leading” instructions. This can be done for instance by saying to participants that the sounds and/or tactile stimuli are rising versus descending, or by designing a test where they are told that the correspondence between sounds and direction of inward and outward movements is informative.

The results obtained in the IAT experiment extend the results obtained by Occelli et al. (2009) regarding the correspondence between pitch height and tactile elevation. They reveal the existence of a congruency effect between the dynamic changes in pitch and the direction of tactile movement. The results obtained in our study also go further in terms of explaining the origin of the spatial mapping of pitch which underlies a well-documented series of effects (e.g., Evans & Treisman, 2010; Patching & Quinlan, 2002; Rusconi et al., 2006). They challenge the idea of a general, amodal spatial representation of pitch shared across all modalities, along with other spatial representations of magnitudes (e.g., Lourenco & Longo, 2011). This does not mean that tasks involving linguistic labels might not show a general spatial representation of pitch (e.g., see Bottini, Crepaldi, Casasanto, Crollen, & Collignon (2015) for the role of language on the spatial representations of time in the blind). The present results strongly suggest the need to investigate whether other spatial mappings of magnitudes, documented with vision, also hold for touch.

The present results are difficult to reconcile with the semantic hypothesis developed by Martino and Marks (1999) and by Walker and Walker (2012) to explain the presence of crossmodal correspondences. According to this hypothesis, correspondences come from the fact that different stimuli evoke similar concepts, or fall within the same part of a conceptual dichotomy (e.g., passive/active; pleasant/unpleasant). Blind participants, who refer to changes in pitch in terms of “rising” and “descending,” have no reason not to fail to apply similar concepts to the two stimuli presented here and to map these directions of pitch and direction of tactile movement. Further evidence of the role of semantic mapping is needed, and could be obtained by testing this audio-tactile correspondence in sighted participants whose language does not describe pitch in spatial terms (see for instance Parkinson et al., 2012, for a test with the audiovisual correspondence). The difference between crossmodal correspondence and semantic associations suggested here adds to the results obtained by Sadaghiani et al. (2009) who found that crossmodal associations between pitch and height were mediated by different neurological processes than metaphorical descriptors such as “high pitch” and “low pitch.”

Importantly, the absence of an effect in blind participants suggests that visual mediation plays a necessary role in the acquisition and persistence of the pitch-elevation correspondence. Our results add evidence to the role of vision in space-magnitude interactions (on the absence of spatial-numerical association of response codes [SNARC] effect in early blind individuals, see Crollen, Dormal, Seron, Lepore, & Collignon, 2013). In line with the present suggestion, Alink, Euler, Kriegeskorte, Singer, and Kohler (2012) have also found that auditory directional information was transferred to high-level visual cortex in healthy adults. Making sense of the role of vision in the pitch-direction correspondence opens several hypotheses. As mentioned in the introduction, vision might be necessary to associate higher pitched sounds to the higher

locations in space in which they most regularly originate (Parise et al., 2014). In this sense, the extraction of information from auditory scene statistics would require a crossmodal collaboration. Alternatively, vision might be necessary to mediate a transitive association between objects getting higher and therefore smaller, and emitting higher pitch as they get smaller. It is only in vision, and not in touch, that moving objects are associated to a change in objects' size, and change in size is in turn associated to a change in pitch (see Eitan, Schupak, Gotler, & Marks, 2011 for a recent study). However, if the absence of association mediated by vision might then explain why congenitally blind participants did not acquire the correspondence between tactile direction and change in pitch, the absence of an association in late blind also suggests a role of visual exposure in the persistence of the association. Due to the presence of visual imagery in the late blind, it could be interesting to test the transitive hypothesis by training them with the two associations (direction of tactile movement – change in [imagined] visual size) and (change in [imagined] visual size – change in pitch) to investigate whether this might lead to different results. It could also be the case that the mental notion of verticality for blind people is determined by the direction of the gravity load due to hand-held objects whereas in sighted people, it is driven by the visual aspect of standing structures. As such they do not refer to identical notions.

Here the findings should be related to Mossbridge, Grabowewcky, and Suzuki (2011), who measured the influence of auditory frequency changes on visuo-spatial attention. The influence disappeared when participants' head axis was tilted by 90°, and no longer aligned with the body axis, which led the author to conclude that “because this cross-modal cueing is dependent on the alignment of head-centered and environmental axes, it may develop through associative learning during waking upright experience” (Mossbridge et al., 2011, p. 133). As was noted earlier, hands occupy many different positions in space. They are not constrained by vestibular inputs that enable us to orient our heads vertically when walking, looking, and listening (Pozzo, Berthoz, & Lefort, 1990; Berthoz, 1991). Hands and fingertips, instead, are completely mobile; they operate independently from gravity and are subject only to proprioceptive relationships. Whereas there is good evidence that, in sighted and in late blind individuals, touch is encoded in a visual frame of reference (e.g., Azañón, Longo, Soto-Faraco, & Haggard, 2010; Harrar & Harris, 2010), this fact seems insufficient to explain the presence of an effect in sighted individuals when the hand is horizontal, and the absence of an effect in both hand positions for late blinds. As such, the fact that sighted people seem to associate a movement going from the inner palm to the tip of the finger to an upward movement (and at least to a rising pitch) for two different hand positions opens an interesting question regarding the implicit directionality of hands in the body schema: Whereas vision modulates implicit hand maps when it comes to size and shape (Longo, 2014), and the crossing of two hands has well documented effects on crossmodal tasks (Shore, Spry, & Spence, 2002), little has been done to test whether a default position or direction of one single hand is used in speeded responses. A further study could be performed to test whether these correspondences are maintained when the hands point in various directions and are located in various relationships with the body. Alternatively, given that the weighing of skin-based, anatomical coordinates and external spatiotopic coordinates is task-
dependent (Badde, Röder, & Heed, 2015), it cannot be excluded that the reference frame observed here was triggered by the task, rather than a default one.

To conclude, the present results reveal differences in the audio-
visual congruency exhibited by blind and sighted people. Other
differences have recently been shown in the classical sound sym-
bolic Bouba-Kiki task (Fryer, Freeman, & Pring, 2014) as well as
in SNARC effects (Crollen et al., 2013). The present study should
help to develop user-centered methodologies, accounting for a
diversity of potential users (e.g., Deroy & Auvray, 2012). Several
devices dedicated to assisting blind individuals (e.g., the vOICe,
Meijer, 1992; Vibe, Hanneton, Auvray, & Durette, 2010) or nav-
igation in the absence of vision (e.g., the Flybar, see Pedley &
Harper, 1959) have been designed by using the correspondence
between direction of pitch and direction of movement (higher-
higher; lower-lower) but without testing whether this correspon-
dence, intuitive for the designers and well-documented in audio-
visual contexts for sighted individuals, was also relevant for blind
people. Here, the relevance of this coding is questioned given that
this correspondence does not show in blind people. A better
understanding of visual impairment shows that there is not a single
abstract profile, but rather a continuum of cases ranging across
individuals and situations: defects or accidents, total or partial,
stable or progressive, constant or variable depending on the situ-
ation, with or without other deficits. The IAT test of crossmodal
correspondences might be an interesting tool to pretest the rele-
vance of certain codings or combinations of auditory and tactile
cues, to tailor them more specifically to their users. Cross-Modal
Correspondences Enhance Performance on a Colour-to-Sound
Sensory Substitution Device, Multisensory Research). Our study
also raise interesting questions as to the multiplicity of spatial
frames in which changes of pitch sounds can be mapped.

References

Alink, A., Euler, F., Kriegeskorte, N., Singer, W., & Kohler, A. (2012). Auditory motion direction encoding in auditory cortex and high-level visual cortex. *Human Brain Mapping, 33*, 969-978. http://dx.doi.org/10.1002/hbm.21263

Azañón, E., Longo, M. R., Soto-Faraco, S., & Haggard, P. (2010). The posterior parietal cortex remaps touch into external space. *Current Biology, 20*, 1304-1309. http://dx.doi.org/10.1016/j.cub.2010.05.063

Badde, S., Röder, B., & Heed, T. (2015). Flexibly weighted integration of tactile reference frames. *Neuropsychologia, 70*, 367-374. http://dx.doi.org/10.1016/j.neuropsychologia.2014.10.001

Ben-Artzi, E., & Marks, L. E. (1995). Visual-auditory interaction in speeded classification: Role of stimulus difference. *Perception & Psychophysics, 57*, 1151-1162. http://dx.doi.org/10.3758/BF03208371

Bernstein, I. H., & Edelstein, B. A. (1971). Effects of some variations in auditory input upon visual choice reaction time. *Journal of Experimental Psychology, 87*, 241-247. http://dx.doi.org/10.1037/h0030524

Berthoz, A. (1991). Reference frames for the perception and control of movement. In J. Paillard (Ed.), *Brain and space* (pp. 81–111). New York, NY: Oxford University Press.

Bottini, R., Crepaldi, D., Casasanto, D., Crollen, V., & Collignon, O. (2015). Space and time in the sighted and blind. *Cognition, 141*, 67–72. http://dx.doi.org/10.1016/j.cognition.2015.04.004

Chiou, R., & Rich, A. N. (2012). Cross-modality correspondence between pitch and spatial location modulates attentional orienting. *Perception, 41*, 339–353. http://dx.doi.org/10.1068/p7161

Cohen Kadosh, R., Lammertyn, J., & Izard, V. (2008). Are numbers special? An overview of chronometric, neuroimaging, developmental

and comparative studies of magnitude representation. *Progress in Neurobiology, 84*, 132-147. http://dx.doi.org/10.1016/j.pneurobio.2007.11.001

Cowan, N., & Barron, A. (1987). Cross-modal, auditory-visual Stroop interference and possible implications for speech memory. *Perception & Psychophysics, 41*, 393-401. http://dx.doi.org/10.3758/BF03203031

Crisinel, A. S., & Spence, C. (2010). A sweet sound? Food names reveal implicit associations between taste and pitch. *Perception, 39*, 417-425. http://dx.doi.org/10.1068/p6574

Crollen, V., Dormal, G., Seron, X., Lepore, F., & Collignon, O. (2013). Embodied numbers: The role of vision in the development of number-space interactions. *Cortex, 49*, 276-283. http://dx.doi.org/10.1016/j.cortex.2011.11.006

Demattè, M. L., Sanabria, D., & Spence, C. (2007). Olfactory-tactile compatibility effects demonstrated using a variation of the Implicit Association Test. *Acta Psychologica, 124*, 332-343. http://dx.doi.org/10.1016/j.actpsy.2006.04.001

Deroy, O., & Auvray, M. (2012). Reading the world through the skin and ears: A new perspective on sensory substitution. *Frontiers in Psychology, 3*, 457. http://dx.doi.org/10.3389/fpsyg.2012.00457

Deroy, O., Crisinel, A. S., & Spence, C. (2013). Crossmodal correspondences between odors and contingent features: Odors, musical notes, and geometrical shapes. *Psychonomic Bulletin & Review, 20*, 878–896. http://dx.doi.org/10.3758/s13423-013-0397-0

Deroy, O., & Spence, C. (2013). Why we are not all synesthetes (not even weakly so). *Psychonomic Bulletin & Review, 20*, 643–664. http://dx.doi.org/10.3758/s13423-013-0387-2

Dolscheid, S., Hunnius, S., Casasanto, D., & Majid, A. (2012). The sound of thickness: Prelinguistic infants' associations of space and pitch. In *CogSci 2012: The 34th annual meeting of the Cognitive Science Society* (pp. 306–311). Austin, TX: Cognitive Science Society.

Eitan, Z., Schupak, A., Gotler, A., & Marks, L. E. (2011). Lower pitch is larger, yet falling pitches shrink: Interaction of pitch change and size change in speeded discrimination. *Proceedings of Fechner Day, 27*, 81–88.

Evans, K. K., & Treisman, A. (2010). Natural cross-modal mappings between visual and auditory features. *Journal of Vision, 10*, 1–12.

Fernández-Prieto, I., Navarra, J., & Pons, F. (2015). How big is this sound? Crossmodal association between pitch and size in infants. *Infant Behavior and Development, 38*, 77–81. http://dx.doi.org/10.1016/j.infbeh.2014.12.008

Fryer, L., Freeman, J., & Pring, L. (2014). Touching words is not enough: How visual experience influences haptic-auditory associations in the “Bouba-Kiki” effect. *Cognition, 132*, 164–173. http://dx.doi.org/10.1016/j.cognition.2014.03.015

Gevers, W., Reynvoet, B., & Fias, W. (2003). The mental representation of ordinal sequences is spatially organized. *Cognition, 87*, B87–B95. http://dx.doi.org/10.1016/S0010-0277(02)00234-2

Glashouwer, K. A., Smulders, F. T., de Jong, P. J., Roefs, A., & Wiers, R. W. (2013). Measuring automatic associations: Validation of algorithms for the Implicit Association Test (IAT) in a laboratory setting. *Journal of Behavior Therapy and Experimental Psychiatry, 44*, 105–113. http://dx.doi.org/10.1016/j.jbtep.2012.07.015

Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual differences in implicit cognition: The implicit association test. *Journal of Personality and Social Psychology, 74*, 1464–1480. http://dx.doi.org/10.1037/0022-3514.74.6.1464

Hanneton, S., Auvray, M., & Durette, B. (2010). The vibe: A versatile vision-to-audition sensory substitution device. *Applied Bionics and Bio-mechanics, 7*, 269–276. http://dx.doi.org/10.1155/2010/282341

Harrar, V., & Harris, L. R. (2010). Touch used to guide action is partially coded in a visual reference frame. *Experimental Brain Research, 203*, 615–620. http://dx.doi.org/10.1007/s00221-010-2252-0
Heller, M., & Gentaz, E. (2013). *The psychology of touch and blindness*. New York, NY: Taylor & Francis.

Hötting, K., & Röder, B. (2009). Auditory and auditory-tactile processing in congenitally blind humans. *Hearing Research*, **258**, 165–174. http://dx.doi.org/10.1016/j.heares.2009.07.012

Longo, M. R. (2014). The effects of immediate vision on implicit hand maps. *Experimental Brain Research*, **232**, 1241–1247. http://dx.doi.org/10.1007/s00221-014-3840-1

Lourenco, S. F., & Longo, M. R. (2011). Origins and development of generalized magnitude representation. In S. Dehaene & E. M. Brannon (Eds.), *Space, time and number in the brain: Searching for the foundations of mathematical thought* (pp. 225–244). London, United Kingdom: Academic. http://dx.doi.org/10.1016/B978-0-12-385948-8.00015-3

Ludwig, V. U., Adachi, I., & Matsuzawa, T. (2011). Visuoauditory mappings between high luminance and high pitch are shared by chimpanzees (*Pan troglodytes*) and humans. *Proceedings of the National Academy of Sciences of the United States of America*, **108**, 20661–20665. http://dx.doi.org/10.1073/pnas.1112605108

MacLeod, C. M. (1991). Half a century of research on the Stroop effect: An integrative review. *Psychological Bulletin*, **109**, 163–203. http://dx.doi.org/10.1037/0033-2909.109.2.163

Maeda, F., Kanai, R., & Shimojo, S. (2004). Changing pitch induced visual motion illusion. *Current Biology*, **14**, R990–R991. http://dx.doi.org/10.1016/j.cub.2004.11.018

Marks, L. E. (2004). Cross-modal interactions in speeded classification. In G. A. Calvert, C. Spence, & B. E. Stein (Eds.), *Handbook of multisensory processes* (pp. 85–105). Cambridge, MA: MIT Press.

Martino, G., & Marks, L. E. (1999). Perceptual and linguistic interactions in speeded classification: Tests of the semantic coding hypothesis. *Perception*, **28**, 903–923. http://dx.doi.org/10.1068/p2866

Meijer, P. B. L. (1992). An experimental system for auditory image representations. *IEEE Transactions on Bio-Medical Engineering*, **39**, 112–121. http://dx.doi.org/10.1109/10.121642

Melara, R. D., & O’Brien, T. P. (1987). Interaction between synthetically corresponding dimensions. *Journal of Experimental Psychology: General*, **116**, 323–336. http://dx.doi.org/10.1037/0096-3445.116.4.323

Mossbridge, J. A., Grabowewcky, M., & Suzuki, S. (2011). Changes in auditory frequency guide visual-spatial attention. *Cognition*, **121**, 133–139. http://dx.doi.org/10.1016/j.cognition.2011.06.003

Occelli, V., Spence, C., & Zampini, M. (2009). Compatibility effects between sound frequency and tactile elevation. *Neuroreport*, **20**, 793–797. http://dx.doi.org/10.1097/WNR.0b013e32832b8069

Parise, C. V., Knorre, K., & Ernst, M. O. (2014). Natural auditory scene statistics shapes human spatial hearing. *Proceedings of the National Academy of Sciences of the United States of America*, **111**, 6104–6108. http://dx.doi.org/10.1073/pnas.1322705111

Parise, C., & Spence, C. (2008). Synesthetic congruency modulates the temporal ventriloquism effect. *Neuroscience Letters*, **442**, 257–261. http://dx.doi.org/10.1016/j.neulet.2008.07.010

Parise, C. V., & Spence, C. (2009). “When birds of a feather flock together”: Synesthetic correspondences modulate audiovisual integration in non-synesthetes. *PLoS ONE*, **4**, e5664. http://dx.doi.org/10.1371/journal.pone.0005664

Parise, C. V., & Spence, C. (2012). Audiovisual crossmodal correspondences and sound symbolism: A study using the implicit association test. *Experimental Brain Research*, **220**, 319–333. http://dx.doi.org/10.1007/s00221-012-3140-6

Parkinson, C., Kohler, P. J., Sievers, B., & Wheatley, T. (2012). Associations between auditory pitch and visual elevation do not depend on language: Evidence from a remote population. *Perception-London*, **41**, 854–861. http://dx.doi.org/10.1068/p7225

Patching, G. R., & Quinlan, P. T. (2002). Garner and congruence effects in the speeded classification of bimodal signals. *Journal of Experimental Psychology: Human Perception and Performance*, **28**, 755–775.

Pedley, P. E., & Harper, R. S. (1959). Pitch and the vertical localization of sound. *The American Journal of Psychology*, **72**, 447–449. http://dx.doi.org/10.2307/1420051

Pozzo, T., Berthoz, A., & Lefort, L. (1990). Head stabilization during various locomotor tasks in humans. I. Normal subjects. *Experimental Brain Research*, **82**, 97–106. http://dx.doi.org/10.1007/BF00230842

Pratt, C. C. (1930). The spatial character of high and low tones. *Journal of Experimental Psychology*, **13**, 278–285. http://dx.doi.org/10.1037/h0072651

Roffler, S. K., & Butler, R. A. (1968). Localization of tonal stimuli in the vertical plane. *The Journal of the Acoustical Society of America*, **43**, 1260–1266. http://dx.doi.org/10.1121/1.1910977

Rusconi, E., Kwan, B., Giordano, B. L., Umiltà, C., & Butterworth, B. (2006). Spatial representation of pitch height: The SMARC effect. *Cognition*, **99**, 113–129. http://dx.doi.org/10.1016/j.cognition.2005.01.004

Sadaghiani, S., Maier, J. X., & Noppeney, U. (2009). Natural, metaphoric, and linguistic auditory direction signals have distinct influences on visual motion processing. *The Journal of Neuroscience*, **29**, 6490–6499. http://dx.doi.org/10.1523/JNEUROSCI.5437-08.2009

Shore, D. I., Spry, E., & Spence, C. (2002). Confusing the mind by crossing the hands. *Cognitive Brain Research*, **14**, 153–163. http://dx.doi.org/10.1016/S0926-6410(02)00070-8

Spence, C. (2011). Crossmodal correspondences: A tutorial review. *Attention, Perception & Psychophysics*, **73**, 971–995. http://dx.doi.org/10.3758/s13414-010-0073-7

Spence, C., & Deroy, O. (2012). Crossmodal correspondences: Innate or learned? *I-Perception*, **3**, 316–318. http://dx.doi.org/10.1068/i0526ic

Spence, C., & Deroy, O. (2013). How automatic are crossmodal correspondences? *Consciousness and Cognition*, **22**, 245–260. http://dx.doi.org/10.1016/j.concog.2012.12.006

Teige-Mocigemba, S., Klauer, K. C., & Sherman, J. W. (2010). A practical guide to Implicit Association Tests and related tasks. In B. Gawronski & B. K. Payne (Eds.), *Handbook of implicit social cognition: Measurement, theory, and applications* (pp. 117–139). New York, NY: Guilford Press.

Walker, P., & Smith, S. (1984). Stroop interference based on the synaesthetic qualities of auditory pitch. *Perception*, **13**, 75–81. http://dx.doi.org/10.1068/p130075

Walker, P., & Walker, L. (2012). Size-brightness correspondence: Cross-talk and congruity among dimensions of connotative meaning. *Attention, Perception & Psychophysics*, **74**, 1226–1240. http://dx.doi.org/10.3758/s13414-012-0297-9

Wang, Q., & Hayward, V. (2010). Biomechanically optimized distributed tactile transducer based on lateral skin deformation. *The International Journal of Robotics Research*, **29**, 323–335. http://dx.doi.org/10.1177/0278364909345289

Received August 14, 2014

Revision received August 26, 2015

Accepted August 31, 2015
<figure><img src="image_42.png" /></figure>

ELSEVIER

Contents lists available at ScienceDirect

Cognition

journal homepage: www.elsevier.com/locate/COGNIT

<figure><img src="image_43.png" /></figure>

COGNITION

Short Communication

# Taking someone else's spatial perspective: Natural stance or effortful decentring?

<figure><img src="image_44.png" /></figure>

CrossMark

Gabriel Arnold<sup>a,b,*</sup>, Charles Spence<sup>c</sup>, Malika Auvray<sup>a</sup>

## ARTICLE INFO

**Article history:**

Received 9 February 2015

Revised 7 December 2015

Accepted 12 December 2015

Available online 23 December 2015

**Keywords:**

Perspective taking

Sense of self

Tactile perception

Visuo-spatial abilities

Individual differences

## ABSTRACT

When perceiving stimuli, self-centred and decentred perspectives can be adopted. In the present study, we investigate whether perceivers have a natural perspective that constrains their spatial perception, with some people perceiving better with self-centred than decentred perspectives and vice versa for other people. We used a recognition task of tactile ambiguous letters (b, d, p, and q) presented on the stomach, for which three perspectives can be adopted (trunk-centred, head-centred, and decentred). At first, the participants were free to adopt any perspective they wanted. Then, either the same or a different perspective was imposed on them. Without constraints, 80% of the participants adopted a self-centred perspective (50% trunk-centred, 30% head-centred) and 20% a decentred one. The perspective adopted freely appears to be natural as recognition performance decreases with a different perspective and returns to its previous high level with the same perspective. Thus, to perceive space, some perceivers adopt naturally a perspective centred on themselves whereas others take naturally others' perspective.

© 2015 Elsevier B.V. All rights reserved.

## 1. Introduction

Perceivers can adopt different spatial perspectives that are either centred on their own location (i.e., self-centred) or else on a different location (i.e., decentred). On the one hand, self-centred perspectives underlie self-consciousness by binding together the multisensory experiences and the physical body (Ferrè, Lopez, & Haggard, 2014; Vogeley & Fink, 2003). Consequently, self-centred perspectives are often seen as having some sort of primacy in terms of spatial cognition (e.g., Epley, Morewedge, & Keysar, 2004). On the other hand, the ability to adopt the perspective of others is crucial when it comes to communicating and interacting with them (Schober, 1993). Decentred perspectives can be spontaneously adopted in collaborative situations (Duran, Dale, & Kreuz, 2011) and even in situations where communication is not required (Thirioux, Jorland, Bret, Tramus, & Berthoz, 2009; Tversky & Hard, 2009). In addition, as a consequence of certain personality traits – for instance, being dominated or dominant (see Galinsky, Magee, Inesi, & Gruenfeld, 2006) – some people adopt the perspective of others whereas other people

adopt their own perspective. One important question is whether people have a natural stance to adopt either self-centred or decentred perspectives. In the present study, we target this question using the tactile ambiguous symbol recognition task.

Tactile perception is interesting for the investigation of spatial perspectives because self-centred (e.g., perception from our body) and decentred perspectives (e.g., perception from outside the body) conflict with each other. Moreover, more than one self-centred perspective exists: the perspective can be centred either on the stimulated surface or on a central body part (e.g., the head; Harrar & Harris, 2010; Ho & Spence, 2007). In this sense, the recognition of ambiguous tactile symbols displayed on the body surface such as the letters b, d, p, and q (Ferrè et al., 2014; Natsoulas & Dubanovski, 1964; Parsons & Shimojo, 1987; Sekiyama, 1991; for a review, see Arnold, Spence, & Auvray, submitted for publication) provides an excellent paradigm with which to investigate the perspectives that are naturally adopted by perceivers. The same perceived stimulation can be interpreted as corresponding to different symbols, as a function of the perspective that is taken when interpreting the stimulation. For example, when the letter b is drawn on a participant's stomach (from the viewpoint of the experimenter located in front of them), three different perspectives can be adopted (see Fig. 1): a decentred perspective oriented toward the participant's stomach (response b); a trunk-centred perspective oriented forward the participant (response d); a

<footnote><sup>a</sup>Institut des Systèmes Intelligents et de Robotique, CNRS UMR 7722, Université Pierre et Marie Curie, Paris, France</footnote>

<footnote><sup>b</sup>Institut Jean Nicod (ENS – EHESS – CNRS), Département d'Etudes Cognitives, Ecole Normale Supérieure, PSL Research University, Paris, France</footnote>

<footnote><sup>c</sup>Department of Experimental Psychology, Crossmodal Research Laboratory, University of Oxford, Oxford, United Kingdom</footnote>

<footnote>* Corresponding author at: Institut des Systèmes Intelligents et de Robotique, Université Pierre et Marie Curie, 4 place Jussieu, 75005 Paris, France.
E-mail address: arnold@isir.upmc.fr (G. Arnold).</footnote>
<figure><img src="image_45.png" /><figcaption>**Fig. 1.** Illustration of the three possible perspectives that participants can adopt when interpreting ambiguous symbols displayed on the body surface. In this figure, the lowercase letter “b” is drawn on the participant’s stomach from the experimenter’s perspective. Top row: the spatial perspective that can be inferred from the participant’s responses. Bottom row: the different responses reported by participants. (A) Perception of the letter “b”, resulting from the adoption of a decentred perspective whose origin is located in front of the participant. (B) Perception of the mirror-reversed letter “d”, resulting from a trunk-centred perspective. The horizontal and vertical axes of the letter are assigned congruently to the participant’s trunk. (C) Perception of the 180°-rotated letter “q”, resulting from a bending-forward head-centred perspective.</figcaption></figure>

head-centred perspective, as if the head was bending forward to “see” the tactile stimulation (response q).

Important inter-individual differences have been observed in the recognition of ambiguous tactile symbols with preferences to adopt one of the three possible perspectives (Sekiyama, 1991). However, one important question that has not been directly addressed is whether perceivers have a natural perspective that constrains their spatial perception. Do some perceivers perceive better from a self-centred perspective whereas other perceivers perceive better from a decentred perspective? The aim of the present study was therefore to investigate whether individual preferences for self-centred vs. decentred perspectives reflect the natural perspectives that people adopt. Tactile symbols were presented on the stomach, for which three different perspectives can be adopted (see Fig. 1). In order not to risk biasing the participants toward the experimenter’s perspective, symbols were not drawn manually by the experimenter (which was the case in all previous studies) but by means of a matrix of vibrators.

To test the *natural perspective* hypothesis, the instructions given to the participants were varied in terms of the perspective that was to be adopted. First, the participants were free to adopt any perspective that they wanted, thus allowing us to evaluate their baseline preferences. Second, different perspectives were imposed on the participants. These corresponded either to the same or to a different perspective than the one that they had adopted freely. These imposed perspectives allowed for the evaluation of any cost associated with adopting an unnatural perspective. If the perspective that is adopted freely is natural, then imposing a different perspective should produce a cost in terms of recognition performance. Some perceivers should perform better with self-centred than decentred perspectives and vice versa for other perceivers. On the other hand, if participants are not constrained by a natural perspective, one possibility is that imposing a different perspective should not induce a cost. However, as decentred perspectives are more demanding than self-centred perspectives (Epley et al., 2004; Natsoulas, 1966), another

possibility is that all of the perceivers would perform better with a self-centred than with a decentred perspective, independently of the perspective adopted freely. Finally, in order to evaluate whether the cost of adopting an unnatural perspective is simply explained by changes in perspectival instructions or by the difficulty that is associated with disengaging from a perspectival choice, we evaluated whether performance would improve when the participants returned to the natural perspective after adopting an unnatural one. If the cost of adopting a different perspective is explained simply by changes in instruction or the difficulty that is associated with disengaging from a perspectival choice, returning to the natural perspective adopted freely should not increase performance.

In addition, we evaluated whether the ability to adopt an unnatural perspective would be influenced by visuo-spatial abilities and by the natural perspective. We thus compared the cost of adopting an unnatural perspective in those participants who adopted the trunk-centred, head-centred, and decentred perspective. However, only the two perspectives for which the vertical axis is not reversed (i.e., the trunk-centred and decentred) were imposed. The decentred perspective was imposed on participants who freely adopted the trunk-centred perspective and vice versa for the decentred participants. For the head-centred participants, the trunk-centred perspective was imposed for one half and the decentred for the other half. The head-centred perspective was not imposed because the top-bottom axis is less prone to confusion than the left-right axis. Left-right confusion occurs when a self-centred or a decentred perspective is imposed on participants (Natsoulas, 1966). However, vertical confusion is less frequent because the vertical assignment is influenced, on the one hand, by both the external environment (i.e., gravity) and the orientation of the egocentric top-bottom axis (i.e., the head-foot axis; Oldfield & Phillips, 1983), and, on the other, by the orientation of the head with a head-centred perspective (Sekiyama, 1991). Moreover, when the vertical axis is reversed, consequently to the adoption of a head-centred perspective, there is only one possible left-right assignment.
## 2. Methods

### 2.1. Participants

Eighty participants completed the experiment (44 females; mean age = 26.6 years, range = 19–47), including participants who adopted freely the trunk-centred (N = 20), the decentred (N = 20), and the head-centred (N = 40) perspectives. The head-centred participants were divided into two groups (N = 20), one adopting the trunk-centred perspective and the other the decentred perspective. In addition to the 80 participants who performed the present experiment, a further 170 participants performed the first session and were then included in other studies (see Participants in the Supplementary Materials for details concerning the classification of the participants in the different groups). The participants provided informed consent and received payment for taking part in the study. The experiment was performed in accordance with the ethical standards laid down in the Declaration of Helsinki (1991).

### 2.2. Apparatus

The tactile stimuli were presented by means of 9 rectangular vibrators (Haptuator Mark II, Tactile Labs, Montreal, Canada) arranged in a 3-by-3 array with a centre-to-centre spacing of 5 cm (see Fig. 2a). A nine-channel amplifier drove each vibrator independently at a frequency of 250-Hz. The vibrator array was placed on the participant's stomach symmetrically to their body mid-sagittal line. Only one layer of clothing was allowed between the skin and the vibrators and the participants individually selected the intensity of each vibrator by means of a method of adjustment. The participants wore noise-reducing headphones with a noise reduction rating of 30 dB, in order to mask any sounds made by the vibrators.

### 2.3. Stimuli

The lower-case letters b, d, p, and q were presented. The tracing of these letters consisted of a sequence of 8 vibrotactile stimuli mapping the trajectory of vibrations as if the letters were traced beginning from the stem (see Fig. 2b). The same order of strokes was used for each letter, instead of respecting the conventions of normal manual writing, because the order in which the various strokes are made in normal manual drawing can itself provide a cue to letter recognition (Parkinson & Khurana, 2007). Note that it could be argued that tracing the letters from the stem end could have biased the participants' responses toward the letter “b”, given that only this letter is written in this way. However, each letter was reported equally often by participants (24.8% of trials for the b,

24.9% for the d, 25.5% for the p, and 24.8% for the q; $F(3, 237) < 1, ns$). The duration of each vibration was 250 ms with no interval between consecutive vibrations, resulting in a total duration of 2-s for the presentation of each letter.

### 2.4. Procedure

The experiment was composed of three sessions of the letter recognition task, followed by the completion of the Mental Rotation Test (MRT; Vandenberg & Kuse, 1978) and the Object Perspective Taking Test (OPTT; Hegarty & Waller, 2004). In Session 1, the participants were free to adopt any perspective that they wanted to recognize letters. In Sessions 2 and 3, they were instructed to adopt a specific perspective. In one of these two sessions, the imposed perspective was the same as in Session 1. In the other session, the imposed perspective was different. Half of participants performed Session 2 with the same perspective as in Session 1 and performed Session 3 with the different one. The other half did the opposite.

Participants gave their responses by pressing the corresponding key on the computer keyboard with the index finger of their preferred hand. The participants were instructed to keep their head oriented upright during the letter tracing, so that they do not see their stomach. The participants were able to give their responses at any time from the onset of the first vibration and up to 3000 ms after the end of the last vibration. At the end of each trial, there was an interval of 3000 ms before the start of the next trial.

Each of the three sessions was composed of 3 blocks of 16 trials (4 presentations of each of the 4 letters). At the beginning of Session 1, the participants performed a practice block with one presentation of each letter. At the beginning of Sessions 2 and 3, the practice block was composed of 3 presentations of each letter. During the practice blocks, feedback was presented indicating that the participant has given a response that was either correct or incorrect. Feedback was not presented during the test blocks but the participants were informed of their percentages of correct responses and mean response times at the end of each block.

## 3. Results

### 3.1. Proportions of each perspective adopted freely and consistency in the perspective adopted

Across all participants (N = 250; 150 females; mean age = 25.2 - years, range = 18–47) who completed the task under free instructions, 49.6% adopted the trunk-centred, 29.2% the head-centred, and 21.2% the decentred perspectives. Around 4 out of 5 participants thus adopted a self-centred perspective (centred either on

<figure><img src="image_46.png" /><figcaption>**Fig. 2.** (A) Schematic figure illustrating the 3 × 3 array of rectangular vibrators. (B) The sequence of 8 vibrations for drawing the letter b. The duration of each vibration was 250 ms, without intervals between each vibration.</figcaption></figure>
their head or on their trunk), whereas only 1 out of 5 adopted a decentred perspective.

For the 80 participants who performed the entire experiment, the consistency in the perspective adopted was high: 82% in Block 1, 92% in Block 2, and 96% in Block 3 (see the Supplementary Materials for more details on consistency). The consistency was not significantly different for those participants adopting the trunk-centred (88.4%), the head-centred (93.3%), and the decentred (88.8%) perspectives ($F(2,77) = 1.00$; $p > .250$). There was no significant difference between the 3 groups with respect to response times (RTs) during Session 1 ($F(2,77 < 1; ns)$). The three perspectives were thus not more or less demanding when participants were free to adopt their natural perspective.

## 3.2. Cost of perspective change

In order to evaluate the cost of adopting an unnatural perspective, ANOVAs were conducted on accuracy and RTs with Block (1-1, 1-2, 1-3, 2-1, 2-2, 2-3, 3-1, 3-2, 3-3; the first number indicates the Session and the second the Block) as a within-participant factors and the Order of imposed perspectives (same-different, different-same) as a between-participants factor. With accuracy, there was a significant main effect of Block ($F(8,624) = 9.45$; $p < .001$; $\eta^2 = .108$) and a significant interaction between Block and Order of imposed perspectives ($F(8,624) = 7.77$; $p < .001$; $\eta^2 = .091$). The cost of perspective change was evaluated by comparing recognition performance in blocks 1-3 and 2-1 for those participants who adopted the different perspective in Session 2 (different-same order), and in blocks 2-3 and 3-1 for those participants adopting the different perspective in Session 3 (same-different order). There was a significant cost associated with participants adopting the different perspective in Session 2 (a decrease of 9.3 percentage points in accuracy; $F(1,78) = 25.44$; $p < .001$; $\eta^2 = .246$) and for participants adopting the different perspective in Session 3 (a decrease of 7.2 percentage points; $F(1,78) = 11.65$; $p = .001$; $\eta^2 = .130$) (see Fig. 3a).

There were also training effects within sessions, with the participants making fewer errors after consecutive blocks with the same perspective (significant for both groups in Session 1, all ps < .001; approaching significance for the different perspective in Session 2, p = .054, and the different perspective in Session 3, p = .080). In order to make sure that the cost of adopting a different perspective was not merely explained by a lack of training, we compared recognition performance in Blocks 1-1 and 2-1, in Blocks 1-2 and 2-2, and in Blocks 1-3 and 2-3, for those participants who adopted the different perspective in Session 2. There was a significant cost in Block 2-2 (a decrease of 5.9 percentage points in accuracy; $F(1,78) = 6.80$; $p < .05$; $\eta^2 = .075$) and 2-3 (a decrease of 5.6 percentage points; $F(1,78) = 10.65$; $p < .01$; $\eta^2 = .130$), but not in Block 1 2-1 (an increase of 3.4 percentage points; $F(1,78) < 1$; ns). However, performance in Block 1-1 was low, probably because the participants were not yet familiar with the task. Except for the first block, performance was thus better for the natural than for the unnatural perspective, even when the participants received the same amount of training in each perspective. Note that we did not make these comparisons for those participants adopting the different perspective in Session 3 because they kept their natural perspective in Session 2 and received thus more training with the natural than the unnatural perspective. However, the cost was not significantly different for participants adopting the unnatural perspective in Session 2 (decrease of 9.4 percentage points in accuracy) and in Session 3 (decrease of 7.2 percentage points; $t(78) < 1$; ns).

Importantly, a significant improvement in accuracy was also observed for participants returning to their perspective in Session 3 (an increase of 5.3 percentage points in accuracy; $F(1,78) = 6.36$; $p = .014$; $\eta^2 = .075$). This improvement indicates that the

decrease in recognition performance with the different perspective was neither produced by a change in perspectival instructions, nor by the difficulty associated with disengaging from a perspectival choice, but by the difficulty that the participants experienced when trying to adapt to an unnatural perspective.

With respect to the RT data, there was a significant main effect of Block ($F(8,624) = 46.89$; $p < .001$; $\eta^2 = .375$) and a significant interaction between Block and Order of imposed perspectives ($F(8,624) = 43.78$; $p < .001$; $\eta^2 = .359$) (Fig. 3b). A significant cost of adopting an unnatural perspective was observed in Session 2 (an increase of 436 ms in RT; $F(1,78) = 45.64$; $p < .001$; $\eta^2 = .369$) and in Session 3 (an increase of 768 ms; $F(1,78) = 70.48$; $p < .001$; $\eta^2 = .475$). RTs were also significantly shorter for participants returning to their natural perspective in Session 3 (a decrease of 441 ms; $F(1,78) = 23.22$; $p < .001$; $\eta^2 = .229$).

## 3.3. Cost as a function of the natural perspective

We evaluated whether the cost of adopting an unnatural perspective varied with the natural perspective of the participant (see Fig. 4). With respect to the accuracy data, the cost was significantly different from zero for the trunk-centred participants when adopting the decentred perspective (a decrease of 10.6 percentage points in accuracy; $Z = 5.23$; $p < .001$), the head-centred participants when adopting the trunk-centred perspective (a decrease of 7.2 percentage points; $Z = 2.33$; $p = .010$), and the head-centred participants when adopting the decentred perspective (a decrease of 12.8 percentage points; $Z = 3.15$; $p = .001$). For the decentred participants adopting the trunk-centred perspective, the difference approached significance (a decrease of 2.5 percentage points; $Z = 1.41$; $p = .080$). A one-way ANOVA revealed that the cost was significantly smaller for the decentred than for the self-centred participants ($F(1,76) = 5.34$; $p = .024$; $\eta^2 = .066$), without there being any significant differences between the 3 groups of self-centred participants (all ps > .172).

With respect to the RT data, the cost was significantly different from zero for all groups of participants (all ps < .001). A one-way ANOVA showed that the cost was significantly smaller for the decentred than for the self-centred participants ($F(1,76) = 6.75$; $p = .011$; $\eta^2 = .082$), without there being significant differences between self-centred participants (all ps > .116).

## 3.4. Influence of visuospatial abilities

With respect to mental rotation, there was a significant negative correlation ($r = -.34$; $t(78) = 3.10$; $p = .01$) between the score in the MRT and the cost of perspective change in accuracy: The greater the mental-rotation abilities, the smaller the cost. In addition, the trunk-centred (mean score = 26.7, SD = 7.9) and decentred participants (mean score = 28.5, SD = 7.3) showed greater mental-rotation abilities than the head-centred participants (mean score = 22.8, SD = 9.3; $F(2,77) = 3.38$; $p < .05$). With respect to visuo-spatial perspective taking, there was neither a significant correlation between the mean error in the OPTT and the cost of perspective change ($r = -.07$; $t(78) < 1$; ns) nor any significant differences between the groups ($F(2,77) < 1$; ns).

# 4. Discussion

The study reported here was designed to evaluate whether perceivers have a natural perspective that constrains their perception of stimuli presented on their body surface. When perceivers are free to adopt any perspective that they want, 80% consistently adopt a self-centred perspective (50% trunk-centred, 30% head-centred) while 20% adopt a decentred one. The fact that
<table><thead><tr><th></th><th>Session 1<br>free</th><th>Session 2<br>imposed</th><th>Session 3<br>imposed</th></tr></thead><tbody><tr><th>same-different</th><td>free</td><td>same</td><td>different</td></tr><tr><th>different-same</th><td>free</td><td>different</td><td>same</td></tr></tbody></table>

<figure><img src="image_47.png" /></figure>

<figure><img src="image_48.png" /><figcaption>**Fig. 3.** (A) Illustration of the perspective imposed in each session for the two groups of participants. “Same-different” means that the imposed perspective was the same as in Session 1 for Session 2 and was different for Session 3. “Different-same” means that the imposed perspective was different for Session 2 and the same as in Session 1 for Session 3. Participants’ (A) accuracy (percentage correct) and (B) response times as a function of Block and Order of imposed perspectives (same-different, different-same). For both accuracy and response times, the performance decreased significantly when imposing a different perspective and increased significantly when returning to the same perspective as in Session 1. \*p < .05, \*\*p < .01, \*\*\*p < .001.</figcaption></figure>

recognition performance decreases with an unnatural perspective and returns to its previous high level with a natural one supports the view that the perspective adopted is natural. In addition, when a different perspective was imposed, the participants made more errors corresponding to the adoption of their natural perspective than other errors (see Quality of responses in the Supplementary Materials). Some of the observers perceive spatial relations better from a self-centred perspective whereas others perceive better from a decentred perspective. However, the greater cost for self-centred than decentred participants shows that decentred perceivers adopt more easily an unnatural perspective than self-centred perceivers.

The fact that the cost of adopting an unnatural perspective was correlated with mental rotation but not with visuo-spatial perspective taking also supports the existence of a natural perspective. When an unnatural perspective was imposed, the participants may have kept their natural perspective and then have mentally rotated the perceived letter rather than to really adopt the unnatural perspective (see Surtees, Apperly, & Samson, 2013, for the role of mental rotation in perspective taking), possibly explaining the slowing of RTs. The possible involvement of mental rotation when an unnatural perspective is imposed raises the question of what adopting a spatial perspective really means. It could just involve specifying the spatial coordinates that result from the perspective that is being taken (i.e., the left-right orientation of someone else’s body space) without imagining the self being located at the decentred location.

In the present study, only 20% of the participants adopted a decentred perspective, which is less than reported in previous studies presenting ambiguous symbols on the stomach (50% in Sekiyama, 1991; 71% in Parsons & Shimojo, 1987). This difference can be explained by the fact that the symbols were presented by means of a matrix of tactile vibrators, thus avoiding the major bias that may have been present in previous studies toward adopting the experimenter’s perspective (since they drew the stimuli on the participant’s skin). Nonetheless, even though the experimenter was not present in our experimental setup, some people appear to prefer adopting a decentred perspective, thus suggesting that this perspective may be their default perspective. In future work, it would be interesting to quantify such natural decentring by directly comparing the probability of decentring for tactile devices and for experimenter-drawn stimuli.

Self-centred perspectives can be centred on the stimulated surface (the trunk) or on a central body part (the head). Sekiyama (1991) has demonstrated that this head-centred perspective is adopted only when the corresponding bending-forward movement of the head toward the stimulated surface is possible. This head-centred perspective may thus involve a kind of visual strategy. The fact that some perceivers naturally adopt such a visual perspective may be explained by their lower spatial abilities. It would also be interesting to evaluate whether blind individuals adopt this head-centred perspective less frequently than do sighted individuals. Similar proportions of self-centred and decentred perspectives have been reported in blind and sighted people (Shimojo, Sasaki,
<figure><img src="image_49.png" /><figcaption>**Fig. 4.** Cost in (A) accuracy (percentage correct) and (B) response times when the different perspective was imposed on participants, as a function of the natural and the imposed perspectives. For example, 'trunk-centred to decentred' means that the cost was computed for the natural trunk-centred participants when adopting the decentred perspective. Error bars represent the standard errors of the means. The presence of asterisks indicates that the cost was significantly different from zero. \*\*p < .01, \*\*\*p < .001. (C) Schematic illustration of the natural and the imposed perspectives for the 4 different groups.</figcaption></figure>

Parsons, & Torii, 1989) but the vertical inversion corresponding to the adoption of a head-centred perspective was not taken into account. Finally, the fact that vertical inversion was observed only when the head-centred perspective was adopted (see Quality of responses in the *Supplementary Materials*) reinforces previous observations that the top-bottom axis is less prone to confusion that the left-right axis (Oldfield & Phillips, 1983; Parsons & Shimojo, 1987; see also Farrell, 1979; Takano, 1998; Uehara, 2013). The vertical axis may thus be assigned before the horizontal axis when interpreting tactile symbols.

To conclude, the present study reveals that perceivers do not adopt the same perspective when interpreting ambiguous spatial information. Some perceivers prefer to perceive space from their own centred point of view whereas others prefer the point of view of another person. Even though decentred perspectives were adopted less frequently than self-centred ones, the natural adoption of a perspective decentred from the location of the body was observed (in 20% of the cases) in our study. Such spatial decentring also characterizes out-of-body experiences, where the self and the body are temporarily disconnected (Blanke, Landis, Spinelli, & Seeck, 2004; Ehrsson, 2007). Interestingly, a reinforcement of the processes anchoring the self to the body by galvanic stimulation of the vestibular system has recently been reported to bias participants toward self-centred perspectives in the ambiguous tactile symbol recognition task (Ferrè et al., 2014). The results of the present study, however, reveal that the adoption of a decentred perspective can reflect other processes than a sole distortion in the relation between the self and the body. Not only does it occur quite frequently, but it can also be considered as a stance crucial to know that other persons perceive the world in a differently way than we do and to understand how they perceive it.

## Acknowledgements

We thank Gaëtan Parseihan and Mathieu Dubois for their help in designing the apparatus. We also thank Eric Siéroff for giving

us the opportunity to test participants in his laboratory (Neuropsychologie du Vieillissement, EA 4468, Université Paris Descartes). This research was supported by the Agence Nationale de la Recherche (ANR-11-JSH2-003-1) and by the Mission pour l'Interdis ciplinarité (Defisens, Supplé-Sens grant). GA is funded by grants from the Agence Nationale de la Recherche (ANR-11-JSH2-003-1) and from the Fondation des Aveugles de Guerre. CS is funded by the AHRC grant entitled Rethinking the Senses (AH/L007053/1). The Institut Jean Nicod laboratory finds support in ANR-10-LABX-0087 IEC and ANR-10-IDEX-0001-02 PSL.

## Appendix A. Supplementary material

Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.cognition.2015.12.006.

## References

Arnold, G., Spence, C., & Auvray, M. (2015). What perspectives do observers adopt when interpreting tactile stimuli? Reviewing the literature on the tactile ambiguous symbol recognition paradigm. *Psychonomic Bulletin and Review* (submitted for publication).

Blanke, O., Landis, T., Spinelli, L., & Seeck, M. (2004). Out-of-body experience and autoscopy of neurological origin. *Brain*, 127, 243–258.

Duran, N. D., Dale, R., & Kreuz, R. J. (2011). Listeners invest in an assumed other's perspective despite cognitive cost. *Cognition*, 121, 22–40.

Ehrsson, H. H. (2007). The experimental induction of out-of-body experiences. *Science*, 317, 1048.

Epley, N., Morewedge, C. K., & Keysar, B. (2004). Perspective taking in children and adults: Equivalent egocentrism but differential correction. *Journal of Experimental Social Psychology*, 40, 760–768.

Farrell, W. S. (1979). Coding left and right. *Journal of Experimental Psychology: Human Perception and Performance*, 5, 42–51.

Ferrè, E. R., Lopez, C., & Haggard, P. (2014). Anchoring the self to the body: Vestibular contribution to the sense of self. *Psychological Science*, 25, 2106–2108.

Galinsky, A. D., Magee, J. C., Inesi, M. E., & Gruenfeld, D. H. (2006). Power and perspectives not taken. *Psychological Science*, 17, 1068–1074.

Harrar, V., & Harris, L. R. (2010). Touch used to guide action is partially coded in a visual reference frame. *Experimental Brain Research*, 203, 615–620.
Hegarty, M., & Waller, D. (2004). A dissociation between mental rotation and perspective-taking spatial abilities. *Intelligence, 32*, 175–191.

Ho, C., & Spence, C. (2007). Head orientation biases tactile localization. *Brain Research, 1144*, 136–141.

Natsoulas, T. (1966). Locus and orientation of the perceiver (ego) under variable, constant, and no perspective instructions. *Journal of Personality and Social Psychology, 3*, 190–196.

Natsoulas, T., & Dubanovski, R. A. (1964). Inferring the locus and orientation of the perceiver from responses to stimulation of the skin. *The American Journal of Psychology, 77*, 281–285.

Oldfield, S. R., & Phillips, J. R. (1983). The spatial characteristics of tactile form perception. *Perception, 12*, 615–626.

Parkinson, J., & Khurana, B. (2007). Temporal order of strokes primes letter recognition. *Quarterly Journal of Experimental Psychology, 60*, 1265–1274.

Parsons, L. M., & Shimojo, S. (1987). Perceived spatial organization of cutaneous patterns on surfaces of the human body in various positions. *Journal of Experimental Psychology: Human Perception and Performance, 13*, 488–504.

Schober, M. F. (1993). Spatial perspective-taking in conversation. *Cognition, 47*, 1–24.

Sekiyama, K. (1991). Importance of head axes in perception of cutaneous patterns drawn on vertical body surfaces. *Perception & Psychophysics, 49*, 481–492.

Shimojo, S., Sasaki, M., Parsons, L. M., & Torii, S. (1989). Mirror reversal by blind subjects in cutaneous perception and motor production of letters and numbers. *Perception & Psychophysics, 45*, 145–152.

Surtees, A., Apperly, I., & Samson, D. (2013). Similarities and differences in visual and spatial perspective-taking processes. *Cognition, 129*, 426–438.

Takano, Y. (1998). Why does a mirror image look left-right reversed? A hypothesis of multiple processes. *Psychonomic Bulletin and Review, 5*, 37–55.

Thirioux, B., Jorland, G., Bret, M., Tramus, M. H., & Berthoz, A. (2009). Walking on a line: A motor paradigm using rotation and reflection symmetry to study mental body transformations. *Brain and Cognition, 70*, 191–200.

Tversky, B., & Hard, B. M. (2009). Embodied and disembodied cognition: Spatial perspective-taking. *Cognition, 110*, 124–129.

Uehara, I. (2013). Left-right and up-down mirror image confusion in 4-, 5- and 6-year-olds. *Psychology, 4*, 736–740.

Vandenberg, S. G., & Kuse, A. R. (1978). Mental rotations, a group test of three-dimensional spatial visualization. *Perceptual and Motor Skills, 47*, 599–604.

Vogeley, K., & Fink, G. R. (2003). Neural correlates of first-person-perspective. *Trends in Cognitive Sciences, 7*, 38–42.

World Medical Association (1991). Declaration of Helsinki. *Law, medicine & health care: A Publication of the American Society of Law & Medicine, 19*, 264–265.
<figure><img src="image_50.png" /></figure>

# Cognition overrides orientation dependence in tactile viewpoint selection

Jessica Hartcher-O'Brien<fnref n="1" />,<fnref n="2" /> · Malika Auvray<fnref n="1" />

Received: 14 December 2015 / Accepted: 8 February 2016

© Springer-Verlag Berlin Heidelberg 2016

**Abstract** Humans are capable of extracting spatial information through their sense of touch: when someone strokes their hand, they can easily determine stroke direction without visual information. However, when it comes to the coordinate system used to assign the spatial relations to the stimulation, it remains poorly understood how the brain selects the appropriate system for passive touch. In the study reported here, we investigated whether hand orientation can determine coordinate assignment to ambiguous tactile patterns, whether observers can cognitively override any orientation-driven perspectives on touch, and whether the adaptation transfers across body surfaces. Our results demonstrated that the orientation of the hand in the vertical plane determines the perspective taken: an external perspective is adopted when the hand faces the observer and a gaze-centred perspective is selected when the hand faces away. Participants were then adapted to a mirror-reversed perspective through training, and the results revealed that this adapted perspective holds for the adapted surface and generalises to non-adapted surfaces, including across the body midline. These results reveal plasticity in perspective taking which relies on low-level postural cues (hand orientation) but also on higher-order somatosensory processing that can override the low-level cues.

**Keywords** Touch · Viewpoint selection · Ambiguous pattern of stimulation · Coordinate assignment · Frame of reference

## Introduction

Spatial information about the objects in our environment is accessible not only through vision, but also through our senses of audition and touch (Vallbo and Johansson 1984). For instance, people can interpret the direction of stimulation when a visual or an auditory stimulus is displayed moving from left to right. This also holds true in touch when a body part is stroked from left to right. The question of how left and right are assigned, and therefore of how spatial information is taken from the environment, depends on the sensor that is used (Newell et al. 2001), with the mechanisms appearing to be more straightforward and established in vision than in touch.

In the case of visual viewpoint selection, when you watch your hand being stroked, the stroking pattern is initially coded in retinotopic coordinates. Depending on the task, either visual information is then either maintained in retinotopic coordinates for immediate action or it is transformed into external coordinates so that the observer can maintain a stable perceptual experience despite eye movements, changes in the position of the observer relative to the object, and object perturbations (Holway and Boring 1941; Wexler and Held 2005).

In the case of touch, the brain's viewpoint on stimuli is similarly not arbitrary; that is, when somebody strokes your hand, it is easy to distinguish whether they are stroking it from left to right or from right to left. However, unlike spatial processing in vision, at any given time there are potential contributions from multiple coordinate systems,

**Electronic supplementary material** The online version of this article (doi:10.1007/s00221-016-4596-6) contains supplementary material, which is available to authorized users.

<figure><img src="image_51.png" /></figure>

Jessica Hartcher-O'Brien
jesshartcher@gmail.com; hartcher@isir.upmc.fr

<footnote n="1">¹ UPMC Univ. Paris 06, UMR 7222, ISIR, Sorbonne Universités, 75005 Paris, France</footnote>

<footnote n="2">² Institut Jean Nicod, CNRS, EHESS, Ecole Normale Supérieure, 29 Rue d'Ulm, 75005 Paris, France</footnote>
even when the task and posture do not change (Parsons and Shimojo 1987; Volcic et al. 2009). For example, determining the stroke direction across the surface of the hand can initially be done by coding the information in hand-centred coordinates, where left and right are assigned according to the spatial layout of the hand. However, the hand is not an independent system, but is always relative to the person's trunk, head, or eyes. In this case, the body-centred, craniotopic, retinotopic, or gaze-centred (cranio- and retinotopic combined) coordinates can also be used to code and interpret the incoming tactile information. To summarise, concerning early stages of processing, visual information is initially coded in retinotopic coordinates and can then be transformed into external, craniotopic, or motor coordinates depending on the task and goal of the observer, whereas for touch several coordinate systems already compete at early stages of somatosensory processing (Heed et al. 2015).

Theoretically, for an optimal processing of passive touch, i.e. when no explicit action is required, the most parsimonious interpretation of the stroking of the hand is to not remap the information into another coordinate system. For the brain to do so would be unnecessarily costly; thus, left and right would be assigned according to the hand's spatial layout. On the other hand, if one is actively engaged in the exploration of an object, then the incoming information may be most efficiently coded by transforming the incoming somatosensory information into external coordinates. By transforming the stimulation into external coordinates, the brain can maintain a stable representation of the object, despite distortions due to movement, and postural cues. However, the theoretical parsimonious approach of coding information in hand-centred coordinates is not necessarily what has been observed (Parsons and Shimojo 1987; Volcic et al. 2009). The question then becomes what are the factors that determine this coordinate assignment process for passive touch?

Given the ability of the hand to explore objects from different vantage points, independent of the posture of the body, this surface may well exhibit unique reference frame selection relative to other body surfaces. Several studies approached the question of what perspectives may be taken on somatosensory stimuli on the hand (Azañón and Soto-Faraco 2008; Holmes 2014; Prather and Sathian 2002; Volcic et al. 2009). In these studies, coordinate selection is based on tests that give changes in reaction time between the stimulus/hand placed in a baseline coronal position and when it changes to a different position (Prather and Sathian 2002). The increase in RT is an indication of the cost of changing reference frame (Volcic et al. 2009). Note that the studies that behaviourally assess reference frame selection in passive touch (without motor activity or movement) on other body parts also used indirect methods such as RTs (Farrell and Robertson 1998; Schicke and Röder 2006; Torralbo et al. 2006).

The exception, which provides a promising method to directly investigate viewpoint selection, was to use an ambiguous tactile symbol paradigm, which consists of grapheme stimuli (letters and numbers) drawn on the body surface. These graphemes potentially have several interpretations until the observer selects one coordinate system with which to interpret them. For instance, the letters ‘b’, ‘d’, ‘p’, and ‘q’ displayed on the skin are ambiguous until the observer takes a perspective and assigns coordinates to the cutaneous input, and consequently perceive one of the letters. Parsons and Shimojo (1987), for instance, used this paradigm and traced the graphemes on multiple body surfaces when the relative position and orientation of the surface was varied. When the experimenter traced the tactile patterns on the palm of the hand, stimuli were perceived relative to the position and orientation of the stimulated surface with respect to the body torso. Parson and Shimojo’s results suggest the use of external frames of reference that are independent of the hand or head and that are associated with stimuli presented on the hand, that is, for Parsons and Shimojo, according to the experimenter-defined coordinates. However, in Parsons and Shimojo’s study (and in other studies using this paradigm on other body surfaces that the hand, e.g. Corcoran 1977) the graphemes were always drawn manually by the experimenter. As a consequence, the transformation into experimenter-defined coordinates could be solely due to the influence of the experimenter himself biasing the participants’ responses.

In our study, any experimenter-induced bias was removed from the equation by using a small Braille pin display device to present the directional cutaneous patterns (see Fig. 1). Our study was divided into three tasks. The first task investigated a similar question to Parsons and Shimojo’s (1987) study: that is, whether the orientation of the hand could determine the mind’s perspective on touch—however with our device and standardised procedure. However, in the case of our study, we were curious about interpretation of touch to the fingertip, not the palm (as investigated by Parsons and Shimojo 1987). We thus explored, for the first time, whether viewpoint selection could be overridden by adapting to the mirror-reversed perspective (in the second task). In a third task, we then measured whether this adapted perspective would generalise across non-adapted fingers and across the body midline.

## Methods

### Participants

Fourteen naïve participants (nine males) took part in the study. The average age was 26.8 years. The participants provided their written consent prior to the commencement

<figure><img src="image_52.png" /></figure>
<figure><img src="image_53.png" /><figcaption>**Fig. 1** Cutaneous fingertip stimulation pattern and two potential interpretations depending on the reference frame selected. When the stimulation pattern *b* is presented on the device surface (start point and direction of stimulation indicated by the arrow), this pattern is interpreted as a *b* if the spatial coordinates are assigned according to the external object's coordinates. However, this example pattern is ambiguous around the vertical axis with a mirror-reversed perspective giving an interpretation of the pattern as the *letter d*. Here, we define the object-centred coordinates as 'external', i.e. according to the spatial layout of the device surface. Given the position of the device in front of the observer, the mirror reverse of this assignment is consistent with craniotopic as well as retinoptopic coordinates. Because our task does not provide the means to disentangle the two potential coordinate systems, we have adopted the description used by *Harrar and Harris (2009)* where they define a merged coordinate system of head and eye as 'gaze-centred'</figcaption></figure>

of the experiment and were given €8 for their participation. The experiment took approximately 1 h to complete and was performed in accordance with the ethical standards laid down in the 1991 Declaration of Helsinki.

## Apparatus and stimuli

Stimuli were produced using TACTOS (Gapenne et al. 2003), a custom-built device that generates tactile stimuli within a 4 × 4 matrix design. The 4 × 4 pin-matrix display measures 2 cm in width and 2.5 cm in height. When activated, each pin extends 2 mm from the display surface. Observers sat with their head approximately 60 cm from the device, which was Velcro-taped to a stable frame. During the experiment, observers wore an eye mask to stop visual information about the device influencing their judgments. Tactile signals were on/off activations of the Braille pins. The letter stimuli were formed by sequential activations of the pins as shown in Fig. 1. The rate of activation was 20 mm/s. The stimulus lasted approximately 1000 ms in total with each pin activation corresponding to 100 ms. Each letter was defined according to the pattern of stimulation on the device surface.

## Procedure

The experiment was divided into three tasks: first, an orientation manipulation, followed by a verification and an adaptation phase, and finally a post-test generalisation in

perspective taking task (see Fig. 2d). The first task was divided into four blocks, one for each of the four hand orientations. The participants sat and first fixated a central fixation point on the wall in front of them. They placed the volar surface of their index fingertip, in the orientation indicated by the experimenter, against the device display surface. The fingertip of the dominant hand of the observer was stimulated on the volar surface. The blocked, postural manipulation did not alter the surface stimulated but rather the orientation of the surface relative to the head of the observer and the front/back surface of the device. The participants were then blindfolded to ensure that visual information did not bias their responses. The device was attached to a stable frame so that the participants could press their fingertip against the active surface of the device. Each trial consisted of a single sequential tactile pattern, randomly selected from the stimulus list: ‘b’, ‘d’, ‘p’, or ‘q’. Each letter began from the stem and consisted of a continuous motion ending with the body of the letter. We used a four-alternative forced-choice procedure in which the participants indicated which of the four letters they perceived, by pressing the corresponding key on the keyboard. The inter-trial interval varied randomly between 1000 and 2000 ms. The participants were able to respond during the stimulus presentation and up to 3000 ms after stimulus offset. They received no feedback as to the correctness of their responses. Each stimulus was repeated ten times in random order for each blocked hand orientation (i.e. 160 trials in total).

Task 2 consisted of an initial verification of the orientation dependent perspective and then an adaptation phase in which participants adopted orientation 1, as represented in Fig. 2b, and were asked to reverse the assignment of left/right relative to their responses in the first experiment. That is, they were required to respond as though they were looking at the cutaneous pattern from the perspective of the device. Correct/incorrect feedback was provided over the 80 adaptation trials, with 20 repetitions of each of the four cutaneous letter patterns.

The third task consisted of four post-test blocks, one for each of the generalisation surfaces tested (see Fig. 2d). In this phase, the strength and generalizability of the adapted perspective was verified. In this post-test phase, the task was identical to that of the first part of the experiment. However, the hand orientation did not change across blocks: the hand was kept in the ‘facing-away’ orientation, but we changed the surface that was stimulated: the adapted index finger surface and three non-adapted finger surfaces (see Fig. 2d).

The dominant hand of the observer was used during the pretest and adaptation phases of the experiment. In the post-test phase, both hands were stimulated. The stimulated region was always the volar surface of the
<figure><img src="image_54.png" /><figcaption>(b) Tested orientations of the hand relative to the observers head</figcaption></figure>

<figure><img src="image_55.png" /><figcaption>(c) Adaptation task to mirror reverse perspective</figcaption></figure>

<figure><img src="image_56.png" /><figcaption>(d) Post-test generalisation across fingers and body midline</figcaption></figure>

<figure><img src="image_57.png" /></figure>

**Fig. 2** Cutaneous patterns, potential perspectives, and fingertip orientations adopted during the experiment. **a** The device pin activation pattern for the letter *b*. The interpretation of the pattern as a *b* required assigning external coordinates to the activation pattern. An interpretation for a *d* would involve assigning the reverse, gaze-centred, coordinates. **b** The four surface orientations tested during task 1: orientation manipulation. Despite changes in the orientation of the

surface relative to the head and the device front/back surface, it is always the volar surfaces of the fingertip that were stimulated. The orientations are given with reference to the head of the observer. **c** Illustration of the adaptation phase used in task 2. **d** Fingers tested during the post-adaptation phase. Orientation of the fingertip was held constant

fingertip. During the first two phases of the experiment (i.e. the pretest and the adaptation phase), the index finger of the dominant hand was used. In the generalisation post-test phase, however, the volar surface of the two index fingers and middle fingers was stimulated, as shown in Fig. 2d.

## Results

The participants' responses were transformed from letter identification estimates to proportions of external or gaze-centred (i.e. mirror-reversed) coordinates. External coordinates were defined as imposing spatial relations onto the pattern of stimulation according to the external device coordinates. In this case, the proportion of responses corresponding to this perspective was calculated as the number of 'b'/'d'/'p'/'q' responses consistent with a 'b'/'d'/'p'/'q' pattern on the device surface. Gaze-centred coordinates were defined as the mirror reverse of this assignment, with left/right, up/down being assigned according to the eyes/head of the participant. Thus, the response observed

indicates the perspective adopted by the participant during the tactile stimulation.

For the first task (orientation manipulation), we analysed the proportion of responses consistent with the participants adopting each of the four possible perspectives: external, gaze-centred, external but inverted, and gaze-centred but inverted. A one-way repeated-measures ANOVA was conducted on the different letter stimuli (four levels) which failed to reveal any significant difference in the perspective taken across the four letters [$F(3,13) < 1, p > 0.05$]. Consequently, the data were averaged across the different cutaneous letter patterns.

A repeated-measures ANOVA, conducted on the participants' responses with the four-level orientation factor, was significant [$F(3, 33) = 53.22, p < 0.001$]. Post hoc pairwise comparisons with Bonferroni's correction showed significant differences between orientations 1 and 3 (see Fig. 2b), [$t(26) = 4.37, p = 0.001$] and between orientations 1 and 4 [$t(26) = 4.34, p = 0.002$]. There was no difference between orientations 1 and 2 [$t(26) = 1.15, p = 0.25$], nor between 3 and 4 [$t(26) = 0.09, p = 0.92$]. These results demonstrate that the participants'
**Fig. 3** Proportion of responses for each orientation, averaged across the four cutaneous patterns *b*, *d*, *p*, and *q* consistent with assigning **a** external, **b** gaze-centred, **c** external but horizontally inverted, **d** gaze-centred but horizontally inverted coordinates to the cutaneous pattern. The proportion of responses sums to one across the four possible perspectives. The error bars represent the standard error of the mean across participants

<figure><img src="image_58.png" /></figure>

perspective depends on the orientation of the hand relative to the device with the differences being driven by the facing-away versus facing-towards positions, the upright versus perpendicular position having no influence on the results (Fig. 3a). In other words, the participants' perspective depends on the orientation of their hand along the vertical axis. With respect to the adopted perspective, when the hand is facing towards the participant, when both upright and perpendicular to the body midline, an external perspective is adopted. When the hand is oriented facing away from the participant, a gaze-centred perspective is taken on the tactile patterns of stimulation. The inverted perspectives were adopted significantly less than chance [$t(13) > 1, p < 0.012$], see Fig. 3c, d.

With respect to task 2, we analysed what happens during the pretest, the adaptation phase, and the post-test of the experiment, respectively. During the pretest, the proportion of responses was consistent with the gaze-centred perspective predicted from the results obtained during the pre-adaptation task (see Fig. 4a). The gaze-centred perspective was mainly adopted (82 % of the trials), and the three other perspectives occurred significantly less often than chance [$t(11) = -2.49, p = 0.95, t(11) = -20.28, p = 1, t(11) = -16.69, p = 0.9$], see Fig. 4a.

During the adaptation task, the perspective is hypothesised to shift from the initial perspective and converge towards the adapted perspective as a function of time (see Fig. 4b). Figure 4b represents the proportion of responses consistent with the non-adapted perspective across trials, averaged across the 14 participants and across the four letters. To understand the adaptation rate for novel perspective taking in touch, we fit a power function to the adaptation data. The function has a slope of 0.8489 (95 % CI 0.6912, 1.007) and an intercept of -0.409 (95 % CI -0.5315, -0.2864), with $R^2 = 0.7807, p < 0.01$. From the fit, we can predict that saturation in the amount of adaptation is expected after about 100 adaptation trials where the function reaches a local minimum. The change in response over time during adaptation demonstrates that our feedback (correct/incorrect response) modified the responses as to which letter was felt on the fingertip.

With respect to the generalisation post-test, a repeated-measures ANOVA was conducted on the participants' responses with one factor: finger stimulated (four levels). The analysis did not reveal significant differences in the perspective taken for any of the surfaces tested, the adapted one or the three non-adapted [$F(3,33) = 1.938, p = 0.14$]. Moreover, single-sample *t* tests, with Bonferroni's
<figure><img src="image_59.png" /><figcaption>**Fig. 4** Proportion of responses consistent with the natural and adapted perspectives, before, during, and after the adaptation task. **a** The proportion of responses consistent with the predicted, gaze-centred perspective, and the three other non-predicted ones, averaged across observers. The *error bars* represent the standard error of the mean. **b** The change in the response pattern during adaptation (averaged across observers and across the *four letters*). That is, adaptation phase trial 1 corresponds to four trials involving each of the *four letters b, d, p, q*. The variance (*shaded error bar*) represents</figcaption></figure>

the standard error across observers. The polynomial fit to the proportion non-adapted responses, represented by the continuous *grey line*. **c** The proportion of responses in the post-test (averaged across *observers* and *letters*) consistent with the adapted perspective for the four surfaces: the adapted fingertip, the non-adapted middle finger of the adapted hand, the homologous finger of the non-adapted hand, and the middle finger of the non-adapted hand. The *error bars* represent the standard error of the mean

correction applied, revealed that the transfer effect is significantly higher than chance [$t(11) > 2.5, p < 0.002$] for all surfaces tested. Thus, the learnt perspective, consistent with mapping the stimulation into external coordinates, remains active for the adapted finger and generalises across the non-adapted, middle finger of the same hand and across the body midline to the homologous and middle finger of the non-adapted hand (Fig. 4c). These results show that the orientation cues can be overridden by training to the reverse perspective: causing participants to map the cutaneous information into external coordinates.

## Discussion

Our study explored whether postural cues can account for the mind's perspective on touch, whether the perspective is malleable and can be updated to a novel perspective via adaptation, and finally whether adaptation of one surface transfers to non-adapted surfaces.

### Orientation

The first main result to emerge from our study is that the perspective taken on ambiguous tactile patterns presented to the fingertip is dependent on the posture of the hand, i.e. its orientation. The results show a dependence on orientation around the vertical axis, and a rotation of 90° from the vertical to the horizontal axis did not influence the results.

Parsons and Shimojo (1987) previously explored whether the perspective taken on stimulation of the palm

and dorsal regions of the hand is affected by orientation of the hand. Observers in their study adopted seven different hand orientations. For orientations in which the hand was in front of them, the observers interpreted the cutaneous pattern primarily according to external coordinates for both surfaces of the hand. These results are consistent with interpreting the cutaneous pattern according to the experimenter's viewpoint. However, as was outlined in the introduction, in Parsons and Shimojo's experiment, the fact that the experimenter manually traced the letter on the participant's body surface might have biased their responses towards the experimenter's point of view. This would appear to be the case, given that in our study we have removed this potential bias by using an automatised presentation of the stimuli, and our results reveal that when the hand is in front of the observer, tactile information is transformed both into external coordinates and into gaze-centred coordinates.

With respect to the two coordinate systems that we observed, it should be noted that it is not unusual that touch on the hand is coded into coordinates other than hand-centred. Indeed, viewpoint selection in touch is a non-straightforward process because not only are there multiple different body surfaces, but the position and orientation of the surface relative to the rest of the body, the eyes, and the object itself strongly influences the reference frame adopted (Parsons and Shimojo 1987; Volcic et al. 2009). This leads to the interdependence of several different reference frames, namely those based on somatosensory/body, and external world coordinates (Harrar and Harris 2009). Several studies have suggested that tactile information on different surfaces can also be remapped into a gaze-centred,

<figure><img src="image_60.png" /></figure>
or a motor coordinate system (Harrar and Harris 2009; Mancini and Haggard 2014). In addition to the multiple potential coordinate systems available for coding touch, we assumed that the stimulation on the fingertip may be influenced further by the fact that the fingers can move independently to the rest of the hand (functionally driven) and may therefore recruit additional coordinate systems for decoding stimulation, to that used for interpreting touch to the rest of the hand.

## Adaptation

The second result to emerge from our study concerns the plasticity in the perspective taken as a result of adaptation. When the participants held the orientation of their hand constant and were adapted to a mirror-reversed perspective, as compared with their initial choice for that orientation in task 1, the interpretation of the ambiguous cutaneous patterns was updated to the adapted (external) perspective. The results of this adaptation phase demonstrate that the mind's perspective on touch is malleable and plastic in the face of feedback. Note that the adaptation to the reverse perspective occurred rapidly, during the 80 trials of adaptation (20 for each cutaneous pattern). The fact that simple correct-incorrect feedback can override the low-level postural cue of orientation supports the idea that there is already an ambiguity in terms of which coordinate system might be applied to the pattern of stimulation on the skin, and given that the brain must resolve an existing conflict between multiple possible coordinate assignments for touch, a higher-order learning factor can therefore easily override low-level cues (Corcoran 1977; Parsons and Shimojo 1987; Volcic et al. 2009).

## Transfer of learning

The results from the third task of our study demonstrate that the adapted perspective transfers across non-adapted fingers and across the body midline (i.e. the other hand). This, in conjunction with the fact that ‘correct’/‘incorrect’ feedback can mediate the adopted perspective (overriding orientation cues), suggests that the process we measured here is a higher-level perspective taking, rather than an automatic coordinate assignment process linked to the receptive fields of the stimulated surfaces (Harrar et al. 2014). The strength of the transfer did not decrease significantly when tested on non-adapted surfaces, which is interesting because it suggests that the adapted perspective relies on something more than perceptual factors (Spengler et al. 1997). Moreover, RT data (see supplementary material) suggest that the process we observe is not solely the result of mental rotation, given that the difference between RT data in the pre- and post-test phases is only

approximately 8 ms. While significant, this is a different order of magnitude to that observed in mental rotation tasks (e.g. Just and Carpenter 1985).

For learning of tactile patterns on different body surfaces, i.e. the trunk and leg of the observer, Arnold and Auvray (2014) have also found a transfer of learning to non-adapted surfaces. The authors suggest that their effect is driven by the organisation of the somatosensory cortex. However, in our case where the transfer is about reference frame rather than patterns of stimuli, the fact that the learning transfers across the body midline suggests that the transfer at stake occurs at later stages in the processing pathway, given the lateralisation of early somatosensory processing (Boven et al. 2005). The ability to train a person to adopt the mirror-reversed perspective may be due to the fact that the stimuli themselves are high level and therefore recruit higher-order processing regions.

The transfer results have implications for training with novel haptic interfaces, suggesting that it is possible to remap, with a limited amount of training, the local cutaneous pattern of stimulation into different coordinates for the purposes of experiencing a three-dimensional world and a novel perspective from your own gaze-/body-centred point of view.

## Implications of the results

For the results obtained across the three tasks of our study: the perspective on touch adopted as a function of hand orientation, adaptation, and transfer, we could speculate that a similar level of processing is actually involved, given that the cutaneous patterns were not found to be coded in hand-centred coordinates for any orientations. That is, our results can be explained by higher-order processes probably involving brain areas not organised topographically (see also Harrar et al. 2014). It is clear that the observer's interpretation of patterns of stimulation on the skin is influenced by a number of factors, including competing coordinates systems (Parsons and Shimojo 1987), object identity (Arnold and Auvray 2014), the goal of the observer (Oldfield and Phillips 1983), and the observer's natural preferences (Arnold et al. 2016). What we observe with passive stimulation of tactile letters is that the competition between these different factors is resolved by stable high-level factors. We can quantify the perspective at the fingertip as being driven by postural cues but informed ultimately by learning, mental rotation, and feedback.

This can be accounted for by the fact that in order to maintain a stable experience of our environment, despite our ability to move our sensors relative to the environment, we need to be able to take into account the change in our own sensor positions (Hartcher-O'Brien and Auvray 2014). Thus, the orientation dependence observed in the first task of the current
study highlights one way in which the brain tries to account for changes in sensor position, i.e. to map information into gaze-centred and external coordinates. For visual processing, the visual system remaps information into external coordinates in order to maintain a stable representation across eye and stimulus movements (Epstein 1977). Here, our study shows how the brain updates its perspective on cutaneous stimulation during changes in hand orientation, transforming the patterns into non-hand-based coordinates. That is, in order to maintain a stable representation of the environment, the brain selects predominantly gaze-centred and external coordinates during the processing of passive tactile information.

To conclude, the coordinate system chosen to interpret tactile objects is a complex interplay between the object position in space, on the skin, and the orientation of the body part receiving the stimulation relative to other body surfaces (Parsons and Shimojo 1987). In our study, in no case was the hand-centred perspective adopted over the external and gaze-centred perspectives. The importance of hand orientation in perspective taking in everyday life, as demonstrated in the first part of the experiment, is highlighted in the example of how your brain uses the orientation of your hand to infer the surface (top or bottom) of a table: when you rest your hand on the top of a table, your perspective on the table's surface is unambiguous and primarily coded from the orientation of your hand. You know that your hand is in contact with the tabletop. If you rotate your hand 180° and establish contact with the table again, it is clear that you are in contact with the lower surface of the table. In any construction of external objects and space, it is important to provide coordinates that can be used to determine the spatial relations between objects and our senses. When we touch objects or when we are touched, both physical and physiological constraints influence our interpretation of the objects we perceive. Nevertheless, cognitive factors can override physiological constraints as seen in the adaptation of the mind's perspective on touch.

**Acknowledgments** JHO was funded by the Fyssen Foundation. MA was funded by a grant from the Agence Nationale de la Recherche (ANR-11-JSH2-003-1). The Institut Jean Nicod laboratory receives financial support via the ANR-10-LABX-0087 IEC and ANR-10-IDEX-0001-02 PSL.

## References

Arnold G, Auvray M (2014) Perceptual learning: tactile letter recognition transfers across body surfaces. Multisens Res 27(1):71-90. doi:10.1163/22134808-00002443

Arnold G, Spence C, Auvray M (2016) Taking someone else's spatial perspective: natural stance or effortful decentring? Cognition 148:27-33. doi:10.1016/j.cognition.2015.12.006

Azañón E, Soto-Faraco S (2008) Changing reference frames during the encoding of tactile events. Curr Biol 18(14):1044-1049

Boven RWV, Ingeholm JE, Beauchamp MS, Bikle PC, Ungerleider LG (2005) Tactile form and location processing in the human brain. Proc Natl Acad Sci USA 102(35):12601-12605. doi:10.1073/pnas.0505907102

Corcoran DWJ (1977) The phenomena of the disembodied eye or is it a matter of personal geography? Perception 6(3):247-253. doi:10.1068/p060247

Epstein W (1977) Stability and constancy in visual perception: Mechanisms and processes. Wiley, New York

Farrell MJ, Robertson IH (1998) Mental rotation and automatic updating of body-centered spatial relationships. J Exp Psychol Learn Mem Cogn 24(1):227-233. doi:10.1037/0278-7393.24.1.227

Gapenne O, Rovira K, Ali Ammar A, Lenay C (2003) Tactos: special computer interface for the reading and writing of 2D forms in blind people. Univers Access HCI Incl Des Inf Soc 10:1270-1274

Harrar V, Harris L (2009) Eye position affects the perceived location of touch. Exp Brain Res 198(2-3):403-410. doi:10.1007/s00221-009-1884-4

Harrar V, Spence CJ, Makin TR (2014) Topographic generalization of tactile perceptual learning. J Exp Psychol Hum Percept Perform 40(1):15-23. doi:10.1037/a0033200

Hartcher-O'Brien J, Auvray M (2014) The process of distal attribution illuminated through studies of sensory substitution. Multisens Res 27(5-6):421-441

Heed T, Buchholz VN, Engel AK, Röder B (2015) Tactile remapping: from coordinate transformation to integration in sensorimotor processing. Trends Cogn Sci 19(5):251-258. doi:10.1016/j.tics.2015.03.001

Holmes NP (2014) Hand-centred space, hand-centred attention, and the control. Cortex 25:189-220

Holway AH, Boring EG (1941) Determinants of apparent visual size with distance variant. Am J Psychol 54(1):21-37

Just MA, Carpenter PA (1985) Cognitive coordinate systems: accounts of mental rotation and individual differences in spatial ability. Psychol Rev 92(2):137-172. doi:10.1037/0033-295X.92.2.137

Mancini F, Haggard P (2014) Perception: a motion after-effect for voluntary actions. Curr Biol 24(2):R70-R72

Newell FN, Ernst MO, Tjan BS, Bülthoff HH (2001) Viewpoint dependence in visual and haptic object recognition. Psychol Sci 12(1):37-42

Oldfield SR, Phillips JR (1983) The spatial characteristics of tactile form perception. Perception 12(5):615-626. doi:10.1068/p120615

Parsons L, Shimojo S (1987) Perceived spatial organization of cutaneous patterns on surfaces of the human body in various positions. J Exp Psychol Hum Percept Perform 13:488-504

Prather SC, Sathian K (2002) Mental rotation of tactile stimuli. Cogn Brain Res 14(1):91-98

Schicke T, Röder B (2006) Spatial remapping of touch: confusion of perceived stimulus order across hand and foot. Proc Natl Acad Sci 103(31):11808-11813. doi:10.1073/pnas.0601486103

Spengler F, Roberts TPL, Poeppel D, Byl N, Wang X, Rowley HA, Merzenich MM (1997) Learning transfer and neuronal plasticity in humans trained in tactile discrimination. Neurosci Lett 232(3):151-154. doi:10.1016/S0304-3940(97)00602-2

Torralbo A, Santiago J, Lupiáñez J (2006) Flexible conceptual projection of time onto spatial frames of reference. Cogn Sci 30(4):745-757. doi:10.1207/s15516709cog0000_67

Vallbo AB, Johansson RS (1984) Properties of cutaneous mechanoreceptors in the human hand related to touch sensation. Hum Neurobiol 3(1):3-14

Volcic R, Wijntjes MWA, Kappers AML (2009) Haptic mental rotation revisited: multiple reference frame dependence. Acta Psychol 130(3):251-259

Wexler M, Held RM (2005) Anticipating the three-dimensional consequences of eye movements. Proc Natl Acad Sci USA 102(4):1246-1251
# The Process of Distal Attribution Illuminated Through Studies of Sensory Substitution

Jess Hartcher-O’Brien<fnref n="1" />,<fnref n="2" />,* and Malika Auvray<fnref n="1" />,<fnref n="2" />

Received 31 December 2013; accepted 30 June 2014

**Abstract**

When we interact with objects in our environment, as a general rule we are not aware of the proximal stimulation they provide, but we directly experience the external object. This process of assigning an external cause is known as distal attribution. It is extremely difficult to measure how distal attribution emerges because it arises so early in life and appears to be automatic. Sensory substitution systems give us the possibility to measure the process as it occurs online. With these devices, objects in our environment produce novel proximal stimulation patterns and individuals have to establish the link between the proximal stimulation and the distal object. This review disentangles the contributing factors that allow the nervous system to assign a distal cause, thereby creating the experience of an external world. In particular, it highlights the role of the assumption of a stable world, the role of movement, and finally that of calibration. From the existing sensory substitution literature it appears that distal attribution breaks down when one of these principles is violated and as such the review provides an important piece to the puzzle of distal attribution.

**Keywords**

Sensory substitution, distal attribution, prior of a stable world, movement, calibration, externalisation

## 1. Distal Attribution

As we move through the world, we are directly aware of external objects not the proximal stimulation they provide at our receptors (Crick and Koch, 1995; O'Regan, 1992). For instance, changes in air pressure or vibrations arriving at our cochlea are perceived as arising from an object in the environment (such as a moving car) rather than from the cochlea itself. In this instance, proxi-

<footnote n="1">¹ Institut Jean Nicod, CNRS UMR 8129, Département d'Etudes Cognitives, Institut d'Etude de la Cognition, Ecole Normale Supérieure, Paris, France</footnote>

<footnote n="2">² LIMSI CNRS, B.P. 133, Orsay-Cedex, 91400 France</footnote>

<footnote>* To whom correspondence should be addressed. E-mail: jesshartcher@gmail.com</footnote>
mal patterns of stimulation (e.g., vibrations in the cochlea) are perceived as
having an external cause (e.g., the car). What are the cues necessary for this
assignment to emerge?

1.1. *Definition*

Distal attribution, also known as externalisation, occurs when we experience
a pattern of stimulation and assign the cause of this stimulation to an object,
stimulus, or event located in external three-dimensional space. This process
allows us to make sense of the world around us. When proximal stimulation
(i.e., stimulation received at the receptor surface) is not assigned an external
cause, it gives rise to a mere sensation, not the perception of an object located
at a distance. Therefore understanding how distal attribution emerges is key to
understanding the distinction between sensation and perception.

1.2. *The Problems in Investigating Distal Attribution*

The process of distal attribution is so automatic and present so early in life that
it is extremely difficult to investigate it objectively or even to understand how
it emerges (Bach-y-Rita, 2002; Loomis, 1992). Studies attempting to under-
stand the phenomenon of distal attribution have therefore relied upon instances
where distal objects give rise to novel proximal stimulation patterns, as hap-
pens with the use of sensory substitution devices (SSDs from now on) (Auvray
et al., 2005; Bach-y-Rita, 1972; Bach-y-Rita et al., 1969; Epstein et al., 1986;
Loomis, 1992; von Békésy, 1955; White et al., 1970).

To introduce them, SSDs were initially designed to assist or replace specific
functions of a deficient sensory modality by providing corresponding informa-
tion to an alternative sensory modality. This can be achieved by converting the
stimuli normally sensed through the deficient modality (e.g., light for SSDs
compensating for vision) into stimuli accessible to another sensory modality
(e.g., tactile vibrations or sounds). Since their inception in the sixties various
kinds of devices have been developed, tested, and shown to allow their users
to behave to some degree as if they possessed the substituted sensory organ
(see Auvray and Myin, 2009; Bubic et al., 2010; Deroy and Auvray, 2012, for
reviews). For instance, thanks to visual-to-auditory and visual-to-tactile con-
version systems, blind individuals report being able to localize and recognize
objects in three-dimensional space (e.g., Auvray et al., 2007; Bach-y-Rita et
al., 1969; Levy-Tzedek et al., 2012). Sensory substitution offers a novel way
to investigate how distal attribution emerges because with sensory substitution
devices the boundary between proximal- and distal-attribution is blurred. Such
devices provide the unusual circumstance in which the perceiver initially ex-
periences a proximal pattern of stimulation and, after training, perceives this
same pattern as a distal event (Bach-y-Rita, 2002).
### 1.3. Review Objectives

With SSDs, it is possible to observe how users transform the proximal stimulation rendered by the device into an experience of external objects. In such mediated perception (similar to the non-mediated kind), if we fail to assign a distal cause, we would no longer have a perceptual space corresponding to physical space, but we would be left with the sensation only. Therefore, as was mentioned above, understanding how distal attribution emerges, in both instances, is crucial to understanding perception itself. When distal attribution fails to emerge with SSDs, the users do not establish the corresponding hypothesis of a distal cause of the sensation (e.g., Bach-y-Rita, 1972; Epstein et al., 1986). The literature to date has not directly addressed the link between prior beliefs of an external stable world and the process of distal attribution. Here we will explore the role of such assumptions in the emergence of distal attribution using SSDs. Furthermore we will identify three components essential for distal attribution to emerge: movement with its consequent feedback about the world, calibration of space, and the prior of a stable world. In turn this will allow us to reconcile how the use of distance perception as synonymous with distal attribution has changed our approach to testing distal attribution with SSDs. We would suggest that distance perception is a bi-product of the distal attribution process and that both phenomena depend on a prior of a stable world (e.g., Glennerster et al., 2006; Knill, 2007).

## 2. Establishing External Space

### 2.1. Problems of Establishing External Space with SSDs

As was noted above, SSDs allow us to sense distal objects by providing novel proximal stimulation that corresponds to the external object at an alternative receptor surface. As such, the novel perceptual space that emerges is not necessarily a function of the same assumptions, regularities or spatial maps as those underpinning non-mediated perception. Through the SSD interface, observers have to re-learn the relationships between the experienced pattern of stimulation and the properties of the external objects. When distal attribution of proximal stimulation does not occur in SSD space, it is possible to see how prior beliefs fail to transfer and how the process of distal attribution breaks down. For example, if a proximal pattern of stimulation is experienced on the skin and is derived from interacting with a teacup 0.5 meters away, then the observer needs to re-establish the relationship that links the received proximal patterns to the teacup. If only the proximal pattern is experienced, i.e., without its link to the external object, then the perceiver does not experience external space and as a consequence no distance estimate can be derived.
On the other hand, if the observer has no reason to assume that the experi-
enced pattern represents an object 0.5 meters away, then the mere experience
of tactile vibrations, i.e., the lack of distal attribution, is the most parsimonious
interpretation of the incoming stimulation. In a recent study, Siegle and War-
ren (2010) investigated the perceived distance of objects experienced through
a minimalistic SSD. Participants were divided into two groups and performed
the task under two different conditions: observers were asked either to focus
their attention on the proximal stimulation on the body surface or, in the sec-
ond condition, they were asked to focus their attention on the distal source.
Blindfolded participants observed a target light using a device consisting of
a single finger-mounted photodiode that drives tactile vibration on the back
whenever the finger points at the target. After active exploration of their en-
vironment, participants had to move a reference object (experienced without
the SSD) to match the perceived egocentric distance of the target. The partici-
pant group who were instructed to focus their attention on the distal object had
higher performance than the group who focused their attention on the proxi-
mal stimulation during the task. However, while Siegle and Warren's study
clearly showed a difference in performance between the two groups, this dif-
ference could be the result of attention rather than distal attribution per se. In
non-mediated perception both externalisation and distance estimation depend
upon the prior of an external world. Although the two can be conflated, as in
Siegle and Warren's study, it is important to understand their relative contri-
butions to our construction of an external space. In the following sections we
will discuss what is known about the emergence of the concept of space in
non-mediated perception and then turn to the role of such processes in SSD
space.

2.1.1. *The Emergence of Space During Non-Mediated Perceptual Processing*
We do not have direct access to the physical world. When we move our ner-
vous system is provided with feedback about its current environment. Thus
what we have access to are the relative changes in position of our sensory
organs (e.g., our eyes or hand), and the consequent changes in sensory in-
put. The consequence of movement is that our senses acquire a sequence of
stimulation patterns. The information from this sequence is sufficient to deter-
mine the structure of 3D space (e.g., Glennerster et al., 2006; Graham, 1989).
Therefore we can experience space through the simple relationship between,
for example, the distance the eyes have moved and concurrent changes in sen-
sory stimulation. Thus our experience of an external world depends upon our
ability to account for the effects of self-generated movement on changes in
proximal stimulation. When the two are correlated in time, the sensory input
can be experienced as objects in external space. In a Bayesian sense, the ner-
vous system has a remarkable ability to detect statistical correlations between
sensory inputs and receptor states and it can use this information to form per-
ceptual representations of the external world.

To minimise the cost of keeping track of the changes, the nervous system
can establish the likelihood that a given sensorimotor relationship indicates
a particular physical object and thus predict the influence of other explo-
ration patterns. Previous research suggests that the cost of keeping track of the
changes can also be minimised by integrating prior knowledge we have about
the world and the physical laws that constrain objects therein (e.g., Glenner-
ster et al., 2006; Hayward, 2011; Loomis, 1992). To map proximal stimulation
to objects in external space, the nervous system has to have an assumption that
there is an external world. Inherent in this is the belief that the world is stable
and that any physical object can only change its position (according physical
laws) or its state (according to the laws of nature).

When space emerges from our interactions with the environment, the num-
ber of spatial dimensions that we perceive depends upon the sensor we use to
detect the incoming information: perception of objects in depth is most eas-
ily accessed through vision, audition and haptics/kinaesthetic. In his seminal
work, "The World of Touch", Katz (1925) suggested that touch has many of
the capabilities of a distal or 'far' sense, such as vision and audition (see also
Krueger, 1970 for a summary of Katz's work). However, spatial properties of
objects, such as their distance from the observer, are readily available through
the 'far senses' but not to the other senses and in this way our experience of
space depend on the sensory receptor stimulated. For tactile perception, attri-
bution to a distal source or to the body surface can differ depending on the
circumstances: Active exploration via touch tends to promote distal attribu-
tion, whereas passively received tactile input to the skin are more likely to be
experienced as coming from the body itself (Gibson, 1962; Katz, 1925). If vi-
sual information is converted into a tactile code then the ability to passively
experience depth depends on remapping this property into a property acces-
sible to the skin. Therefore, it is important to consider how the perception of
external objects changes when we use a SSD. One issue is that in non-mediated
perception, experience consolidates the mapping between the proximal stim-
ulation and its distal object over time. In SSD space novel mappings emerge
and need to be consolidated over time. Moreover, the information experienced
via the device needs to be aligned with information coming from other sensory
modalities.

2.1.2. Maintaining Spatial Relations when Going from Physical to Perceptual Space

In order to survive, organisms need to correctly locate objects in space, orient
to them, or flee from them. To aid in this response optimisation, humans and
other animals have evolved spatial maps or fields that organise the incoming
information according to the spatial relations among physical objects. Each sensory modality has its own map and set of coordinates. The nervous system establishes spatial maps at low-level processing sites and all perceptual information is segregated according to these maps throughout the processing hierarchy. To be beneficial to behaviour, information from the different maps needs to be aligned such that a coherent object localisation emerges despite different sensory co-ordinates. Animal research indicates that neurons at multisensory sites such as the SC and the VPM cortex code the object location relative to the receptor used and furthermore that different sensory maps are aligned (e.g., Graziano et al., 1999; Lee and Groh, 2009). When an auditory signal is moved in depth, neurons in the monkey's ventral premotor cortex are selectively tuned to respond maximally to specific locations in physical space (Graziano et al., 1999). Through the selective responses a neural spatial map is established in the premotor cortex. Critically, these neurons, unlike those in A1 or visual areas, also respond to visual stimuli located in the same external co-ordinates. Animal and human research alike suggests that these maps are critical to our perceptual representation of space and spatial relations (Graziano et al., 1999; King et al., 2001; Moore and King, 1999; Poggio and Fisher, 1977).

We acquire information about the world from all of our senses. Our ears detect air pressure changes, our eyes monitor light patterns, our skin reports the pressure of objects impinging on it, and our nose and mouth measure the chemical content of our current environment. Despite the variety of routes that sensory information can take to get to our brains, we tend to experience the world as a unified entity. We are also capable of recruiting different types of sensory information for the same purpose, irrespective of which sensor was initially responsible for detecting the information. Given this, proximal stimulation through a SSD should potentially correspond to the experience of an object located in external space. Although distal attribution through SSDs has been the subject of many studies, most research has focused on the behavioural changes and therefore there is no information relating spatial map organisation to perception of spatial properties of the world during SSD use. It remains to be seen which maps will be recruited for coding the incoming information and how the interactions with the objects in 3D space are altered. One possibility is that perception mediated by a SSD, produces similar neural changes as those occurring in sensory deprivation (see Knudsen, 1999) and the consequent recruitment of a different sensory pathway to process incoming information. That is, the maps recruited to organise the proximal stimulation would correspond to those of the functional modality. In any case, for distal attribution to emerge the nervous system needs to adapt to the novel sensorimotor mappings and thus re-establish external space. Whether and how this changes the neural processing is not yet completely understood. The interesting aspect for the
current review is how the possible changes in sensory information would be
organised to maintain spatial relations.

3. The Process of Distal Attribution

The distinction between proximal stimulation and attribution of this stimula-
tion to an external object requires the assumption of an external world (e.g.,
Epstein *et al.*, 1986). This may seem an obvious statement. Indeed it is. How-
ever, not addressing such assumptions alters how we define the key principles
or mechanisms underlying distal attribution. Many studies have explored the
question of how distal attribution arises during SSD-use. Early studies were
based on qualitative reports (e.g., Bach-y-Rita *et al.*, 1969) whereas later stud-
ies used different measurement techniques, differing not only in their objec-
tivity, but also in whether participants had direct knowledge of external space
or only the sensory substitution space (e.g., Epstein *et al.*, 1986; White *et al.*,
1970). Each measurement technique has advantages and disadvantages. But
here we focus on how embedding the assumption of an external world in the
task itself, as opposed to defining space as purely an internal entity, alters the
attribution process. We will address how valid the SSD data are in informing
us about the process of distal attribution itself. We will then define the essential
components influencing distal attribution. The factors involved in this process
are: (1) the prior of a stable world; (2) the link between self-generated move-
ment and the resulting sensory stimulation patterns and the additional laws
of movement necessary for spatialisation (i.e., reversibility and interposition);
and (3) the role of calibration in establishing and maintaining an experience of
external space.

3.1. *The Problematic Aspects of SSD Measures for Understanding the Process of Distal Attribution*

There are numerous verbal reports from participants in SSD studies that detail
the emergence of distal attribution after prolonged SSD-mediated experience
(Bach-y-Rita et al., 1969). That is, SSD-users report experiencing objects in
the environment and this tends to dominate the proximal stimulation afforded
by the device (Bach-y-Rita, 1972; Lenay et al., 2003; Segond et al., 2005).
More particularly, SSD-users have indicated that, as a consequence of train-
ing, they no longer feel the tactile stimuli on their skin, where they occur, but
interpret them directly as an object located in external space (see Bach-y-Rita,
2002). One well-cited verbal report comes from Guarniero, a blind philosophy
student, after training with the TVSS: “... very soon after I had learned how
to scan, the sensations no longer felt as if they were on my back, and I became
less and less aware that vibrating pins were making contact with my skin. By
this time objects had come to have a top and a bottom; a right side and a left;
but no depth — they existed in an ordered two dimensional space.” (Guarniero, 1974, p. 104).

One issue with verbal reports is that they cannot be verified in the way an objective measure can (e.g., a left-right judgment). Although these verbal reports indicated that distal attribution emerges in SSD space after training, there was no additional measure to specify how it emerged. That is, it remains unclear whether these reports reflect a genuine perceptual experience of distal objects or a cognitive inference based on perceived patterns in the proximal stimulation. The techniques used to address the question of distal attribution via SSDs have improved since the inception of the devices. In the following sections we will highlight the assumptions and cues to distal attribution that have emerged from the existing studies.

## 3.2. *The Role of the Prior of a Stable External World*

To externalize proximal stimulation we need to assume the existence of an external world. Without this prior belief (or assumption) that the external world is a stable entity, neither distal attribution nor the explicit distance estimates of the distal object would be possible. As mentioned above, when we explore the world, we do not have direct access to the objects in space. What we have access to are the proximal patterns of activation and our own movements (Merleau-Ponty, 1945). The nervous system compensates for the distortions in incoming stimulation, due to movement, and manages to maintain the experience of a stable world and of the objects therein. The prior of an external, stable world is essential to this process. This prior posits that objects are stable. Inherent in this is that if an object changes it can only change its state (according to the laws of nature) or its position (according to the laws of physics). When a change is inconsistent with either positional or state changes in the object itself, the parsimonious assumption is that observer-generated movement must have caused the change in incoming information. These kind of changes can easily be correlated with the efference copy of the observer's movement and therefore compensated for. The compensation phenomenon is known as perceptual constancy (e.g., Gregory, 1963). This mechanism suggests how the distal object is maintained, not how it emerges. Below we consider the relative contribution of this prior compared to that of the signal-receptor interaction to shaping the experience of an external world.

One way to explore this is to look at the modality specific constraints to distal attribution and measure their influence. If modality specific constraints prevent attribution to a distal source then it is reasonable to assume that the process is not amodal in nature but depends on cues that are modality specific. For example, visual-to-tactile SSDs may provide inputs that obey visual laws or principles. Yet, it is also possible that such constraints do not apply to the tactilely sensed information. There is a vibrant discussion addressing this is-
sue of whether the novel emergent perceptual space is defined by either the substituting or substituted modality, or something else altogether (Auvray and Myin, 2009; Deroy and Auvray, 2012: Loomis et al., 2012).

Feedback about the world (the result of an action/perception loop) makes it possible to establish the mapping between self-generated movement and changes in the proximal stimulation. One question is what is altered in our representation of space when information is remapped from one sensory modality to another with SSDs? Using a visual-to-tactile SSD, Epstein et al. (1986) delivered patterns of vibrotactile stimulation to the participants’ index fingertip. They assessed how the participants attributed the transformations of vibrotactile stimulation as a consequence of self-generated movement. Attribution was measured through asking participants to rate how well a variety of scenarios matched what they experienced. Analysis of the ratings from these scenarios revealed that although the participants became aware of the relationship between self-generated movement and stimulation transformations, they never chose the scenario that corresponded to the real set-up, i.e., the SSD set up. The authors concluded that their participants had not developed the hypothesis of distal sources, that is, the hypothesis that the ultimate cause of their vibrotactile experience was an encounter with an object in an external world. Here we see an example in which no distal cause is assigned probably because participants do not have an assumption that the stimulation is part of an external world.

However, a second study by Auvray *et al.* (2005), using a visual-to-auditory sensory substitution device (the Vibe, see Hanneton *et al.*, 2010) and a similar method to that of Epstein *et al.* (1986) demonstrated that people appeared to be able, under certain conditions, to attribute stimulation to a distal object. In this study, the scenarios were re-written in order to determine which component of distal attribution the participants reached: the existence of a correlation between their movements and the resulting sensory stimulation (coupling), the existence of an object that caused their sensations (object), and the existence of external space produced by this coupling (space). In the conditions in which there were no additional cues to that provided in Epstein *et al.*'s study, participants in Auvray *et al.*'s study similarly failed to reach the hypothesis of a distal cause although participants indicated that they experienced a sensorimotor coupling between their own actions and the sensed information.

In this case, the device rendered the objects in such a way that they could plausibly exist independently of the existence of external space. However, in a second condition Auvray *et al.* gave their participants a cardboard sheet that they could use as an occluder. The cardboard served as an occluder because by moving the sheet vertically participants could interrupt the source of stimulation. As such intermittent calibration of external space occurred because, if the cardboard could interrupt the source of stimulation, it must therefore be
located *behind* the cardboard, i.e., in external space. These two studies demonstrate the importance of (a) the assumption of an external stable world and (b) the ability to interact with that world and account for the perturbations caused by the interactions.

Other studies used both 2D and 3D spatial estimates (White *et al.*, 1970) and egocentric distance estimates (Siegle and Warren, 2010) to specify the external world. As such they invoked the stable world prior, which informed the way users made sense of the SSD input. White *et al.* (1970) observed that SSD users were able to estimate the slant of a tactually sensed visual object. That is, they had access to the third dimension of external space in their novel perceptual space. Siegle and Warren tested whether observers were more veridical in distance estimates when they focused on the proximal patterns of stimulation or on the distal object. Both the study by White *et al.* (1970) and Siegle and Warren (2010) demonstrated improvements in performance when observers assumed an external 3D world. From the work mentioned so far it is possible to see that SSDs offer a unique opportunity to study distal attribution as it emerges. However, as discussed here, the SSD studies also have their limitations.

### 3.3. *The Role of Movement*

What are the cues necessary for the nervous system to assign a distal cause to the proximal stimulation at a receptor surface? Many studies note the importance of movement (e.g., Bach-y-Rita *et al.*, 1969; Haynes *et al.*, 1965; White, 1970): In all SSD studies, active exploration of the environment is necessary to establish the existence of external space. During exploration the organism can determine the mapping between changes in the incoming stimulation and its self-generated movements (Auvray *et al.*, 2005; Bach-y-Rita *et al.*, 1969; Hurley and Noë, 2003; Loomis, 1992; O'Regan and Noë, 2001; von Békésy, 1955; White *et al.*, 1970). Movement also allows us to account for deformations that are derived from self-generated movement or physical displacement. For example, spatial position of physical objects is mapped onto retinal location when objects are sensed visually. If an external object caused the stimulation pattern then eye movements should alter the retinal location of the object; whereas if the input is not linked to a position in external space, then eye-movements should have no impact on the perceived location of the stimulation. For proximal attribution on the other hand, incoming sensory information and your efference information should not be correlated. For example, the locus of pain in your fingertip remains at your fingertip even if you move your finger relative to the rest of your body. Therefore being able to account for changes in feedback about the environment as a consequence of exploring it is essential for externalisation of the proximal stimulation.
### 3.3.1. *Space from Eye Movements*

If we take the example of visual processing, the retinotopic coding of the eye might suggest that the location can be read off from the receptor array, yet the retinal image is not the only source of spatial information. Important contributions to the updating of spatial representations come from efferent oculomotor signals (von Helmholtz, 1925), as revealed by experiments in which stimuli are displayed immediately before or during saccades (Hallett and Lighthstone, 1976; Wurtz, 2008). The role of local movement patterns coupled to afferent information could also be derived from the extra-ocular muscle proprioception as was suggested by Sherrington (1918). However, experimental evidence on the function of eye proprioception has remained controversial (Donaldson, 2000), leading to the idea that this signal is primarily used for oculomotor calibration and learning rather than for spatial representation (Lewis and Maler, 2001; Poletti et al., 2013). However, in a first instance the correlation can be used to define stimulation in external co-ordinates. Extra-retinal signals can consequently calibrate this space. Whether the correlation between the extra-ocular muscle movement and the retinal image combined is sufficient to experience 3D space without early calibration from touch is still a contentious issue (e.g., Gori et al., 2010).

When we consider space perception, we usually refer to the process through which humans and other organisms become aware of the relative positions of their own bodies and objects around them. Space perception provides cues, such as depth and distance, which are important for movement and orientation in the environment. At a functional level if we consider that space emerges in the interaction between the organism and its environment (e.g., Terekhov and O'Regan, 2013) then spatial perception is derived from the organism establishing the relationship between action and the resulting stimulation patterns. Furthermore, the ability to perceive different spatial dimensions depends upon the constraints of each sensory modality to move within two or three dimensions. In the case of the latter, what assumptions constrain the space that emerges through SSD-information? How do these assumptions differ from those derived by exploration and perception during unmediated circumstances? To tentatively address these questions let's consider the experience of three-dimensional space during normal development. In a series of intriguing studies into spatial constancy, Bower (1964, 1965, 1966a, b) explored whether infants perceive only the retinal projection or the external object. Using generalization of operant responding to assess discrimination, Bower demonstrated that two-month-old infants see the constant real size and shape of an unfamiliar object despite variation in its projective size and shape produced by altering distance and slant. In addition, consistent with processing of visual objects in adult humans, these infants detect variation in the object's physical size and shape when its angular projection remains constant. This evidence for object
representation in such a young developing nervous system suggests that attri-
bution of the proximal stimulation to a coherent external object is present in
pre-crawl infants. Thus, simple eye and head movement appear to be sufficient
to establish an external cause of the proximal stimulation patterns.

Similar findings (e.g., Caron et al., 1978; Shuwairi et al., 2007) also high-
light some of the behavioural changes (and the corresponding evoked poten-
tial changes) that allow somatosensory externalization (Rigato et al., 2014).
Rigato et al. (2014) provided recent insights into how the developing nervous
system establishes the external location of visual and somatosensory objects
and re-maps the proximal stimulation (in receptor coordinates) into external
coordinates. These authors demonstrated that making the transformation from
proximal stimulation to an external source occurs slowly during the first year
of life, and involves not only neural response changes but the ability to update
for postural changes in the infant's own body representation. Therefore infants
require knowledge about effector position changes in order for them to be able
to externalise incoming patterns of stimulation.

The knowledge derived from movement, both for distance perception and
distal attribution is the corner stone of the sensorimotor tradition (e.g., O'Re-
gan and Noë, 2001). From a more traditional viewpoint, von Helmholtz (1909)
also observed its importance: "It is only by voluntarily bringing our organs of
sense in various relations to the objects that we learn to be sure as to our judg-
ments of the causes of our sensations. We explain the table as having existence
independent of our observation because, at any moment we like, simply by as-
suming the proper position with respect to it, we can observe it." It should be
mentioned that this necessity of a structured correlation between actions and
sensations in order to allow distal attribution was nicely anticipated by Condil-
lac (1754). If we were only able to passively receive sensations, we would not
understand that these sensations refer to objects that exist in an external world.
Indeed, if all our knowledge about the world came from our sensations, and if
sensations were just passive modifications of our minds, how could we infer
the existence of an external world? For Condillac, our exploratory movements
allow us to extract the spatial organisation present in our sensations; it is this
spatial organisation of the objects in the world that allows us to consider them
as external. In other words, spatialisation and object-hood emerge simultane-
ously.

It should be noted that the movement necessary to reach distal attribution
goes beyond a mere coupling between sensation and actions; it must also re-
flect a group structure. In other words, the experience of external space cannot
be reduced to correctly extracting the existence of a coupling between ac-
tion and sensation. Indeed, understanding a correlation between self-generated
movements and their resulting stimulation involves understanding that differ-
ent actions give rise to different sensations. There is a space of displacements
when the *same* actions can give rise to *different* sensations, and when *different* actions can give rise to the *same* sensation. For instance, von Helmholtz (1909) highlighted that actions that can cancel each other from the viewpoint of sensory inputs determine a mathematical group structure, and properties of such a group characterize displacements within a geometrical space. The same sensation obtained by different actions can then be understood as "position" of an object. This was concisely explained by Poincaré (1905) according to who reversibility is the only way to differentiate a change in state from a change in position.

Movement along multiple axes is useful for establishing the existence of external objects. With SSD study results, it is tempting to try to disentangle which movement trajectories are most useful in this process, e.g., translation (changing the position of an object), or rotation (circular movement around a centre point). Comparing the movement type (translation and rotation) across studies demonstrating either distal attribution or a lack thereof, however does not lead to a definitive picture. Both Siegle and Warren (2010) and Epstein et al. (1986) allowed observers to translate the sensor through space. In one case the consequent vibrotactile pattern was attributed to an external object (Siegle and Warren, 2010) while in the other, no distal attribution emerged (Epstein et al., 1986). As such we can simply conclude that correlated changes between self-generated movement (even with a camera end-point) and sensory feedback are necessary, without being able to classify the optimal motion type. However, the essential difference between the two studies for this review is that Siegle and Warren's study involved the prior of an external world whereas Epstein et al.'s study did not, suggesting this prior to be an important factor for distal attribution.

### 3.4. Calibrating and Maintaining External Space

We have discussed the importance of a prior of a stable world and the role of movement in the emergence of distal attribution. In the following sections we address how our experience of external space is maintained via sensory calibration and spatial maps in non-mediated perception. Although a lot is known about the role of calibration of space for information that is conveyed by functional sensory modalities (e.g., King et al., 1988; Knudsen, 1999; Lee and Groh, 2009), the role that calibration plays in maintaining an experience of external space in SSD environments has been rather neglected. In the following two sections we discuss how calibration operates for perception in non-mediated conditions and then postulate its possible role in SSD-mediated external space.

#### 3.4.1. *The Potential Role of Calibration*

All measurement systems require calibration to become and remain accurate. Our perceptual system is no different. Calibration, in the case of our senses,
usually refers to the process of adaptation as a way of producing environmentally geared behaviour (Mon-Williams and Bingham, 2007). While this is particularly critical in vision given that our visual system has no direct access to attributes such as distance, solidity, and size (Ho et al., 2009), cross-sensory calibration occurs for most senses at one time or another: Visual object attributes must be calibrated, that is, verified through touch; auditory position must be calibrated by vision (e.g., King et al., 2001). In each instance the sense with the least error calibrates its co-occurring counterpart. Indeed, since vision is often distorted (see Gori et al., 2010) haptic feedback may be fundamental in improving visual perception through calibration. Calibration has been recognized as essential for maintaining a coherent representation across the senses (e.g., Wallace et al., 1998). In calibration it is important that the system receives feedback (traditionally an error signal) in order to be able to adapt to the conditions it currently experiences: changes in object information occur as we move through the world, calibration is necessary to link the consequent changes in stimulation patterns to the experienced objects (e.g., Ernst, 2008).

However, calibration may also be fundamental to more gradual processes occurring during development, in which a sensory modality calibrates (or teaches) the others about some properties of the world. For instance, recent studies reinforce the suggestion that the haptic system has a role in calibration of the visual system in judgments of position and size. Both animal and human studies have highlighted the role of cross-sensory calibration in the development of spatial processing (Bergan and Knudsen, 2009; Gori et al., 2011; Zwiers et al., 2001, 2003).

If the nervous system is to make full use of the spatial relations among objects and use these relationships to interact with the environment, then online (re)calibration is essential. Experiments in which animals have been reared with modified auditory or visual inputs have revealed substantial plasticity in the animal's spatial representations (e.g., Knudsen, 1999). If animals are raised with distorted binaural cues (used for sound localisation) locating auditory objects in external space produces compensatory changes that can be induced in the auditory space map (e.g., in ferrets, see King and Parsons, 1999; and barn owls, Gold and Knudsen, 2000). These adjustments in auditory spatial tuning tend to preserve the alignment with the visual representation. In owls, they are brought about by frequency-specific shifts in neuronal tuning to interaural time differences (ITDs) and interaural level differences (ILDs) (Gold and Knudsen, 2000), although, as discussed below, the basis for this adaptive plasticity in humans may be different. It is also important that information coming in from completely different co-ordinate systems can be integrated if co-occurring and segregated if not. Spatial maps are one way to achieve this (see Lee and Groh, 2009).
The human (and other animal) nervous system clearly establishes spatial maps to enable efficient processing of incoming information and allow the stimulation to be mapped from receptor to external space. As such there is a need for calibration across the sensors to verify object positions in external coordinates (for example, see Lee and Groh, 2009; Mancini and Haggard, 2014). From the studies exploring space perception via SSDs, there is almost no consideration given to the role of such calibration in learning to perceive the proximal stimulation as distal and maintain this experience over time (e.g., Auvray et al., 2005; Bach-y-Rita, 1972; Bubic et al., 2010; Epstein et al., 1986; Hanneton et al., 2010; Siegle and Warren, 2010). One could speculate that the SSD experiments require such a long training phase not only due to learning the translation code between substituted and substituting modality but also as they may involve a calibration phase. In such an interpretation the nervous system would calibrate the space by feedback derived from self-generated movement.

When we experience visual or tactile objects, they are generally coded within a specific set of coordinates that allows the nervous system to extract the relative position of the object in space. The nervous system establishes spatial maps at low-level processing sites and the perceptual information is segregated according to these maps throughout the processing hierarchy. Sensorimotor acts involve multiple transformations: When we point to an object, a chain of nested coordinate transformations occurs. These include the retinal map itself (target-to-eye), eye-to-head, head-to-body, and body-to-arm mappings. The complexity of coordinating such simple actions is aided by spatial map alignment. However, in exploring the relationship between distance perception and distal attribution, the role of spatial maps and calibration for updating the perceptual space need to be considered separately.

### 3.4.2. Spatial Maps and the Calibration of External Space

Before crossmodal spatial calibration can effectively guide interactions in the world the incoming information needs to be organised in a spatial field in which coordinate systems can be compared and integrated. Not only are signals in the environment organised in a spatial pattern but this pattern is mirrored at a neural level for vision, audition, and touch (see King and Moore, 1991). In fact the amount of plasticity evident at the level of the superior colliculus in many animals is a clear indicator that the sensory maps are used to coordinate action and integrate information across the different coordinate systems. The assumed functional use of spatial maps in the superior colliculus is that they are for optimising orientation responses to incoming information. Detecting and orienting to objects in external space is already one step up from the simple process of using online feedback (from movement) to establish the existence of an external space. The question posed here is whether
the documented training required for SSD-users to experience proximal stimulation as distal is required because training and movement allow the user to ‘re-establish’ or recalibrate existing maps. These maps definitely play a role in non-mediated perception with neurophysiological evidence for the existence of spatial maps both in the superior colliculus and ascending to higher cortical regions like A1 (King et al., 1988; Zwiers et al., 2003). Moreover, for estimates of distance there is evidence that calibration is essential (see Smets et al., 1987) for the percept to emerge. We therefore suggest that whether the spatial maps emerge within the interaction or are established at a neural level, they appear to play a role in maintaining the experience of an external 3D world.

### 3.4.3. Calibration and Distance

As was discussed in Section 3.4.1, calibration is important in maintaining the veridical perception of external space and it therefore plays an important role in distal attribution. The process is also important for distance estimates using 2D surfaces. Smets et al. (1987) suggested that distance perception (on a 2D surface) relies upon a focus point; i.e., a reference against which to calibrate the estimate. Calibration of space increases the veridicality of the judgment. This is also true for the auditory space where relative distance estimates are more accurate than those derived from absolute cues (Shinn-Cunningham, 2001). In distal attribution, a focus point (also achieved through an interposition of two objects) is important for calibrating external space. Establishing spatial relations among objects and perceivers allows us to extract the different reference frames that can be used to describe external space (see, e.g., Knapen et al., 2010; Poletti et al., 2013).

If we take the example of visual perception, the distance of a visual object cannot be directly sensed (Koenderink et al., 2001). However, perception does not only depend on currently sensed information and sensor position. Experience is also shaped by our previous interactions with the world through which we move, our assumptions about the physical world and underlying principles. For example, the emergence of depth is possible due to the priors we have about the world and the physical laws that constrain the (visually) sensed objects and events therein. We assume the world is stable (Glennerster et al., 2006). One piece of evidence for the strength of our stable world prior is the ability to compensate for changes in an object's size and shape. Perceptual constancies of shape and size are used to compensate for distance in vision are effective because of our prior of a stable world (Glennerster et al., 2006). If we assumed that objects could change their shape or form, then these cues to distance would no longer be valid. There is no origin for depth per se in physical terms although a metric of length is valid. However, a ruler for visual depth, for example, is an internal scale, a compromise between priors about object
constancy and a stable world, likelihood given the retinal projection and the compensation mechanisms provided by our nervous system.

The existence of a distal visual object, as opposed to the proximal pattern of stimulation it affords, can be verified via touch. As such it could be suggested that crossmodal calibration gives the nervous system a direct access to the distal form of objects. To reiterate, distance is an estimate that defines an external world that exists independently of the perceptual processing of the information. Organisms perceive objects in space. Such perception represents the response of the organism to a complex pattern of stimulation, integrated with the remaining effects of past experience with similar objects.

**4. Summary**

To highlight what has been learnt about distal attribution through the prism of SSDs, let's explore the factors that appear to be essential to its emergence. First distal attribution depends upon the assumption (or prior) of an external world. Non-separable from this belief of an external world, is the coupling between self-generated movements and the resulting changes in stimulation. This factor is crucial given that if there is no correlation between changes in motor output and changes in proximal patterns of stimulation, then the stimulation will be experienced solely as sensation, not perception (as was made evident in relation to the example of pain). The coupling emerges due to an organism's ability to move and explore the physical world. From the movement patterns the organism can learn about the constraints on external objects such that they can change only in two ways: their position according to the laws of physics or their state according to the laws of nature. In Bayesian terms this second level is the likelihood or currently available stimulation. Using both prior and likelihood the nervous system can then assign a cause, distal or receptor based, given the proximal pattern and its coupling to sensor movement. In this context, distance perception can be seen as a bi-product of the distal attribution process with both phenomena depending on the prior of a stable world (e.g., Glennerster et al., 2006; Knill, 2007). Finally, calibration also plays an important role in maintaining the experience of a stable external world. The ability to assign co-ordinates to the object position in space is possible due to sensory maps which help translate and maintain the input from external world co-ordinates into perceptual co-ordinates.

To summarize, distal attribution of incoming sensory information is influenced by three main parameters: the prior of an external world, the coupling between movements and the resulting changes in stimulation, and calibration. Furthermore, distal attribution has proved crucial for distance perception, in that estimating the co-ordinates of an object in external space requires the object to be perceived as distal, rather than as a proximal pattern of stimulation.
Thus, without the process of distal attribution, perceived distance would not be possible.

## Acknowledgements

We would like to thank Irene Fasiello for the invigorating discussions regarding the nature of distal attribution. We would also like to thank Verena Conrad for her input on earlier drafts of this manuscript. Jess Hartcher-O’Brien is supported by a two-year post-doctoral fellowship from the Foundation Fyssen. Malika Auvray was funded by a grant from the Agence Nationale de la Recherche (ANR-11-JSH2-003-1). The DEC supported via ANR-10-LABX-0087 IEC and ANR-10-IDEX-0001-02 PSL.

## References

- Auvray, M., Hanneton, S., Lenay, C. and O’Regan, J. K. (2005). There is something out there: distal attribution in sensory substitution twenty years later, *J. Integr. Neurosci.* **4**, 505–521.

- Auvray, M., Hanneton, S. and O’Regan, J. K. (2007). Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with ‘The vOICe’, *Perception* **36**, 416–430.

- Auvray, M. and Myin, E. (2009). Perception with compensatory devices: from sensory substitution to sensory motor extension, *Cogn. Sci.* **33**, 1036–1958.

- Bach-y-Rita, P. (1972). *Brain Mechanisms in Sensory Substitution*. Academic Press, London, UK, and New York, NY, USA.

- Bach-y-Rita, P. (2002). Sensory substitution and qualia, in: *Vision and Mind*, A. Noë and E. Thompson (Eds), pp. 497–514. MIT Press, Cambridge, MA, USA.

- Bach-y-Rita, P., Collins, C. C., Saunders, F., White, B. and Scadden, L. (1969). Vision substitution by tactile image projection, *Nature* **221**, 963–964.

- Bergan, J. F. and Knudsen, E. I. (2009). Visual modulation of auditory responses in the owl inferior colliculus, *J. Neurophysiol.* **101**, 2924–2933.

- Bower, T. G. R. (1964). Discrimination of depth in premotor infants, *Psychonom. Sci.* **1**, 368.

- Bower, T. G. R. (1965). Stimulus variables determining space perception in infants, *Science* **149**, 88–89.

- Bower, T. G. R. (1966a). Slant perception and shape constancy in infants, *Science* **151**, 832–834.

- Bower, T. G. R. (1966b). The visual world of infants, *Sci. Am.* **215**, 80–92.

- Cubic, A., Striem-Amit, E. and Amedi, A. (2010). Large-scale brain plasticity following blindness and the use of sensory substitution devices, in: *Multisensory Object Perception in the Primate Brain*, M. J. Naumer and J. Kaiser (Eds), pp. 351–380. Springer, New York, NY, USA.

- Caron, A. J., Caron, R. F. and Carlson, V. R. (1978). Do infants see objects or retinal images? Shape constancy revisited, *Infant Behavior and Development* **1**, 229–243.

- Condillac, E. B. (1754). *Traité des Sensations [Treatise of Sensations]*. Reissued in the series *Corpus des Œuvres de Philosophie en Langue Francaise*. Fayard, Paris, France, 1984.
Crick, F. and Koch, C. (1995). Are we aware of neural activity in primary visual cortex? *Nature* **375**, 121–123.

Deroy, O. and Auvray, M. (2012). Reading the world through the skin and ears: a new perspective on sensory substitution, *Front. Theor. Phil. Psychol.* **3**, 457.

Donaldson, I. (2000). The functions of the proprioceptors of the eye muscles, *Phil. Trans. R. Soc. B Biol. Sci.* **335**, 1685–1754.

Epstein, W., Hughes, B., Schneider, S. and Bach-y-Rita, P. (1986). Is there anything out there? A study of distal attribution in response to vibrotactile stimulation, *Perception* **15**, 275–284.

Ernst, M. O. (2008). Multisensory integration: a late bloomer, *Curr. Biol.* **18**, R519–R521.

Gibson, J. J. (1962). Observations on active touch, *Psychol. Rev.* **69**, 477–490.

Glennerster, A., Tcheang, L., Gilson, S. J., Fitzgibbon, A. W. and Parker, A. J. (2006). Humans ignore motion and stereo cues in favor of a fictional stable world, *Curr. Biol.* **16**, 428–432.

Gold, J. I. and Knudsen, E. I. (2000). Abnormal auditory experience induces frequency-specific adjustments in unit tuning for binaural localization cues in the optic tectum of juvenile owls, *J. Neurosci.* **20**, 862–877.

Gori, M., Sandini, G., Martinoli, C. and Burr, D. (2010). Poor haptic orientation discrimination in nonsighted children may reflect disruption of cross-sensory calibration, *Curr. Biol.* **20**, 223–225.

Gori, M., Sciutti, A., Burr, D. and Sandini, G. (2011). Direct and indirect haptic calibration of visual size judgments, *PLoS ONE* **6**, e25599.

Graham, N. (1989). *Visual Pattern Analyzers*. Oxford University Press, New York, NY, USA.

Graziano, M. S., Reiss, L. A. and Gross, C. G. (1999). A neuronal representation of the location of nearby sounds, *Nature* **397**, 428–430.

Gregory, R. L. (1963). Distortions of visual space as inappropriate constancy scaling, *Nature* **199**, 678–680.

Guarniero, G. (1974). Experience of tactile vision, *Perception* **3**, 101–104.

Hallett, P. E. and Lightstone, A. D. (1976). Saccadic eye movements towards stimuli triggered by prior saccades, *Vis. Res.* **16**, 99–106.

Hanneton, S., Auvray, M. and Durette, B. (2010). The Vibe: a versatile vision-to-audition sensory substitution device, *Appl. Bionics Biomech.* **7**, 269–276.

Haynes, H., White, B. L. and Held, R. (1965). Visual accommodation in human infants, *Science* **148**, 528–530.

Hayward, V. (2011). Is there a 'plenhaptic' function? *Phil. Trans. R. Soc. B Biol. Sci.* **366**, 3115–3122.

Ho, Y. X., Serwe, S., Trommershauser, J., Maloney, L. T. and Landy, M. S. (2009). The role of visuohaptic experience in visually perceived depth, *J. Neurophysiol.* **101**, 2789–2801.

Hurley, S. and Noë, A. (2003). Neural plasticity and consciousness, *Biol. Philos.* **18**, 131–168.

Katz, D. (1925). *The World of Touch*. Erlbaum, Hillsdale, NJ, USA (translation published 1989, translated by L. E. Kmeger).

King, A. J., Hutchings, M. E., Moore, D. R. and Blakemore, C. (1988). Developmental plasticity in the visual and auditory representations in the mammalian superior colliculus, *Nature* **332**, 73–76.

King, A. J. and Moore, D. R. (1991). Plasticity of auditory maps in the brain, *Trends Neurosci.* **14**, 31–37.

King, A. J. and Parsons, C. H. (1999). Improved auditory spatial acuity in visually deprived ferrets, *Eur. J. Neurosci.* **11**, 3945–3956.
King, A. J., Schnupp, J. W. and Doubell, T. P. (2001). The shape of ears to come: dynamic coding of auditory space, *Trends Cogn. Sci.* **5**, 261–270.

Knapen, T., Rolfs, M., Wexler, M. and Cavanagh, P. (2010). The reference frame of the tilt aftereffect, *J. Vis.* **10**, 1–13.

Knill, D. C. (2007). Learning Bayesian priors for depth perception, *J. Vis.* **7**, 1–20.

Knudsen, E. I. (1999). Mechanisms of experience-dependent plasticity in the auditory localization pathway of the barn owl, *J. Comp. Physiol.* **185**, 305–321.

Koenderink, J. J., van Doorn, A. J., Kappers, A. M. L. and Todd, J. T. (2001). Ambiguity and the “mental eye” in pictorial relief, *Perception* **30**, 431–448.

Krueger, L. E. (1970). David Katz's Der Aufbau der Tastwelt (The World of Touch): a synopsis, *Atten. Percept. Psychophys.* **7**, 337–341.

Lee, J. and Groh, J. M. (2009). Auditory signals evolve from hybrid- to eye-centered coordinates in the primate superior colliculus, *J. Neurophysiol.* **108**, 227–242.

Lenay, C., Gapenne, O., Hanneton, S., Marque, C. and Genouëlle, C. (2003). Sensory substitution: limits and perspectives, in: *Touching for Knowing*, Y. Hatwell, A. Streri and E. Gentaz (Eds), pp. 275–292. John Benjamins, Amsterdam, The Nertherlands.

Levy-Tzedek, S., Hanassy, S., Abboud, S., Maidenbaum, S. and Amedi, A. (2012). Fast, accurate reaching movements with a visual-to-auditory sensory substitution device, *Restor. Neurol. Neurosci.* **30**, 313–323.

Lewis, J. E. and Maler, L. (2001). Neuronal population codes and the perception of object distance in weakly electric fish, *J. Neurosci.* **21**, 2842–2850.

Loomis, J. M. (1992). Distal attribution and presence, *Presence* **1**, 113–119.

Loomis, J. M., Klatzky, R. L. and Giudice, N. A. (2012). Sensory substitution of vision: importance of perceptual and cognitive processing, in: *Assistive Technology for Blindness and Low Vision*, R. Manduchi and S. Kurniawan (Eds), pp. 162–191. CRC Press, Boca Raton, FL, USA.

Mancini, F. and Haggard, P. (2014). Perception: a motion after-effect for voluntary actions, *Curr. Biol.* **24**, R70–R72.

Merleau-Ponty, M. (1945). *Phénoménologie de la Perception*. Gallimard, Paris, France.

Mon-Williams, M. and Bingham, G. P. (2007). Calibrating reach distance to visual targets, *J. Exp. Psychol. Hum. Percept. Perform.* **33**, 645–656.

Moore, D. R. and King, A. J. (1999). Auditory perception: the near and far of sound localization, *Curr. Biol.* **9**, R361–R363.

O'Regan, J. K. (1992). Solving the “real” mysteries of visual perception: the world as an outside memory, *Can. J. Psychol.* **46**, 461–488.

O'Regan, J. K. and Noë, A. (2001). A sensorimotor account of vision and visual consciousness, *Behav. Brain Sci.* **24**, 939–973.

Poggio, G. F. and Fisher, B. (1977). Binocular interaction and depth sensitivity in striate and prestriate cortex of behaving rhesus monkeys, *J. Neurophysiol.* **40**, 1392–1405.

Poincaré, H. (1905). *La Valeur de la Science [The Value of Science]*. Flammarion, Paris, France.

Poletti, M., Burr, D. C. and Rucci, M. (2013). Optimal multimodal integration in spatial localization, *J. Neurosci.* **33**, 14259–14268.

Rigato, S., Begum Ali, J., van Velzen, J. and Bremner, A. J. (2014). The neural basis of somatosensory remapping develops in human infancy, *Curr. Biol.* **24**, 1222–1226.

Segond, H., Weiss, D. and Sampaio, E. (2005). Human spatial navigation via a visuo-tactile sensory substitution system, *Perception* **34**, 1231–1249.
Sherrington, C. S. (1918). Observation on the sensual role of the proprioceptive nerve supply of the extrinsic ocular muscles, *Brain* **41**, 332–343.

Shinn-Cunningham, B. G. (2001). Localizing sound in rooms, in: *Proceedings of the ACM/SIGGRAPH and Eurographics Campfire: Acoustic Rendering for Virtual Environments*. Snowbird, UT, USA.

Shuwairi, S. M., Albert, M. K. and Johnson, S. P. (2007). Discrimination of possible and impossible objects in infancy, *Psychol. Sci.* **18**, 303–307.

Siegle, J. H. and Warren, W. (2010). Distal attribution and distance perception in sensory substitution, *Perception* **39**, 208–223.

Smets, G. J. F., Overbeeke, C. J. and Strathman, M. H. (1987). Depth on a flat screen, *Percept. Mot. Skills* **64**, 1023–1034.

Terekhov, A. and O'Regan, J. K. (2013). Space as an invention of biological organisms. http://arxiv.org/abs/1308.2124

von Békésy, G. (1955). Human skin perception of traveling waves similar to those on the cochlea, *J. Acoust. Soc. Am.* **27**, 830–841. [Reprinted in: G. von Békésy (1960). *Experiments in Hearing*, E. G. Wever (Ed. and Trans.), pp. 547–634. McGraw-Hill, New York, NY, USA.]

von Helmholtz, H. (1909). *Physiological Optics*. Optical Society of America, Rochester, NY, USA.

von Helmholtz, H. (1925). *Treatise on Physiological Optics*. Dover, New York, NY, USA.

Wallace, M. T., Meredith, M. A. and Stein, B. E. (1998). Multisensory integration in the superior colliculus of the alert cat, *J. Neurophysiol.* **80**, 1006–1010.

White, B. W. (1970). Perceptual findings with the vision-substitution system, *IEEE Trans. Man-Mach. Syst.* **11**, 54–58.

White, B. W., Saunders, F. A., Scadden, L., Bach-y-Rita, P. and Collins, C. C. (1970). Seeing with the skin, *Percept. Psychophys.* **7**, 23–27.

Wurtz, R. H. (2008). Neuronal mechanisms of visual stability, *Vis. Res.* **48**, 2070–2089.

Zwiers, M. P., Van Opstal, A. J. and Cruysberg, J. R. M. (2001). Two-dimensional sound-localization behavior of early-blind humans, *Exp. Brain Res.* **140**, 206–222.

Zwiers, M. P., Van Opstal, A. J. and Paige, G. D. (2003). Plasticity in human sound localization induced by compressed spatial vision, *Nat. Neurosci.* **6**, 175–181.