Development and experience-dependence of multisensory spatial processing
Authors: Patrick Bruns, Brigitte Röder
Journal: Trends in Cognitive Sciences (2023)

Multisensory spatial perception involves two distinct processes: integration of crossmodal cues to maximize localization precision, and recalibration of unisensory maps to maximize localization accuracy.The ability to detect crossmodal synchrony and enhanced crossmodal associative learning allow infants to infer a coarse causal structure of the environment.Multisensory integration emerges prior to crossmodal recalibration during development, suggesting that the multisensory percept contains a crucial signal for the calibration of spatial representations across sensory modalities.Experience and evolutionary predispositions define crossmodal recalibration rules during development.Multisensory spatial processes are strongly dependent on bottom-up stimulus characteristics during early development, but become more specific and flexibly incorporate top-down priors towards adulthood. Multisensory spatial processes are fundamental for efficient interaction with the world. They include not only the integration of spatial cues across sensory modalities, but also the adjustment or recalibration of spatial representations to changing cue reliabilities, crossmodal correspondences, and causal structures. Yet how multisensory spatial functions emerge during ontogeny is poorly understood. New results suggest that temporal synchrony and enhanced multisensory associative learning capabilities first guide causal inference and initiate early coarse multisensory integration capabilities. These multisensory percepts are crucial for the alignment of spatial maps across sensory systems, and are used to derive more stable biases for adult crossmodal recalibration. The refinement of multisensory spatial integration with increasing age is further promoted by the inclusion of higher-order knowledge. Multisensory spatial processes are fundamental for efficient interaction with the world. They include not only the integration of spatial cues across sensory modalities, but also the adjustment or recalibration of spatial representations to changing cue reliabilities, crossmodal correspondences, and causal structures. Yet how multisensory spatial functions emerge during ontogeny is poorly understood. New results suggest that temporal synchrony and enhanced multisensory associative learning capabilities first guide causal inference and initiate early coarse multisensory integration capabilities. These multisensory percepts are crucial for the alignment of spatial maps across sensory systems, and are used to derive more stable biases for adult crossmodal recalibration. The refinement of multisensory spatial integration with increasing age is further promoted by the inclusion of higher-order knowledge. From hunting prey and evading predators in the forests and savannahs to crossing busy city streets, the ability to precisely and accurately locate objects and events in the environment has remained one of the most fundamental challenges that our cognitive system needs to solve. This is so because spatial cues (see Glossary) derived from the neural signals originating in our peripheral sensory systems are inherently noisy [1.Ernst M.O. Bülthoff H.H. Merging the senses into a robust percept.Trends Cogn. Sci. 2004; 8: 162-169Abstract Full Text Full Text PDF PubMed Scopus (1226) Google Scholar] and constantly changing [2.King A.J. Visual influences on auditory spatial learning.Phil. Trans. R. Soc. B. 2009; 364: 331-339Crossref PubMed Scopus (0) Google Scholar]. To increase localization precision and accuracy, the brain engages in multisensory integration, a process that takes advantage of the fact that the sensory systems provide partially redundant information about the world. Spatial perception is better when it is based on congruent input across multiple sensory modalities rather than on input from a single sense [3.Stein B.E. et al.Neurons and behavior: the same rules of multisensory integration apply.Brain Res. 1988; 448: 355-358Crossref PubMed Scopus (195) Google Scholar], and the behavioral benefits of multisensory integration likely outweigh the enormous neural processing cost of linking inputs in inherently different spatial reference frames across distant sensory brain areas [4.Driver J. Noesselt T. Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments.Neuron. 2008; 57: 11-23Abstract Full Text Full Text PDF PubMed Scopus (692) Google Scholar]. A pitfall of multisensory integration, however, is that only sensory inputs which belong to the same event should be integrated. Thus, using a process of causal inference, the brain needs to estimate (e.g., based on the spatial and temporal arrangement) how likely it is that two inputs (e.g., an auditory and a visual stimulus) originated from the same event [5.Körding K.P. et al.Causal inference in multisensory perception.PLoS One. 2007; 2e943Crossref PubMed Scopus (646) Google Scholar,6.Rohe T. Noppeney U. Cortical hierarchies perform Bayesian causal inference in multisensory perception.PLoS Biol. 2015; 13e1002073Crossref PubMed Scopus (177) Google Scholar]. In addition, linking spatial information across sensory modalities requires continuous recalibration of spatial representations to account both for long-term changes in the correspondence between spatial cues such as those occurring during development and for transient changes in the environment [7.Kramer A. et al.Feedback modulates audio-visual spatial recalibration.Front. Integr. Neurosci. 2020; 13: 74Crossref PubMed Scopus (6) Google Scholar, 8.Wozny D.R. Shams L. Recalibration of auditory space following milliseconds of cross-modal discrepancy.J. Neurosci. 2011; 31: 4607-4612Crossref PubMed Scopus (0) Google Scholar, 9.Zaidel A. et al.Multisensory calibration is independent of cue reliability.J. Neurosci. 2011; 31: 13949-13962Crossref PubMed Scopus (55) Google Scholar]. For example, when entering a room from outside or when the ear canal becomes occluded, auditory spatial cues (and thus their relation to visual spatial cues) change dramatically [2.King A.J. Visual influences on auditory spatial learning.Phil. Trans. R. Soc. B. 2009; 364: 331-339Crossref PubMed Scopus (0) Google Scholar]. It is conceivable that multisensory integration depends on the currently existing calibration values of the individual spatial cues while at the same time providing a crucial signal for subsequent recalibration [10.Magosso E. et al.A neural network model of ventriloquism effect and aftereffect.PLoS One. 2012; 7e42503Crossref Scopus (29) Google Scholar]. However, only recently has research tried to relate multisensory integration and crossmodal recalibration and found that these two processes involve astonishingly distinct underlying mechanisms [11.Bruns P. et al.Cross-modal training induces changes in spatial representations early in the auditory processing pathway.Psychol. Sci. 2011; 22: 1120-1126Crossref PubMed Scopus (30) Google Scholar, 12.Bruns P. Röder B. Sensory recalibration integrates information from the immediate and the cumulative past.Sci. Rep. 2015; 5: 12739Crossref PubMed Scopus (40) Google Scholar, 13.Park H. Kayser C. The neurophysiological basis of the trial-wise and cumulative ventriloquism aftereffects.J. Neurosci. 2021; 41: 1068-1079Crossref PubMed Scopus (11) Google Scholar]. After defining multisensory integration and recalibration and discussing current views of how they are related in more detail, we will focus here on two open questions in multisensory research. First, how do multisensory integration and recalibration capabilities emerge during ontogenetic development, and how are they shaped by early sensory experience? Second, how does the brain learn to integrate different types of potentially conflicting information about the causal structure that can stem from the current sensory evidence (e.g., the spatial and temporal arrangement of the stimuli) as well as from prior knowledge (e.g., learned associations or context) acquired at various time scales? The emerging view is that multisensory integration in spatial tasks seems to precede rather than follow crossmodal recalibration during development, likely because it provides a crucial signal for the unsupervised calibration of unisensory spatial representations [14.Rohlf S. et al.Multisensory integration develops prior to crossmodal recalibration.Curr. Biol. 2020; 30: 1726-1732Abstract Full Text Full Text PDF PubMed Scopus (21) Google Scholar]. However, whereas adults flexibly include top-down knowledge in the process of multisensory spatial integration [15.Tong J. et al.Crossmodal associations modulate multisensory spatial integration.Atten. Percept. Psychophys. 2020; 82: 3490-3506Crossref PubMed Scopus (8) Google Scholar,16.Zierul B. et al.Reduced multisensory integration of self-initiated stimuli.Cognition. 2019; 182: 349-359Crossref PubMed Scopus (7) Google Scholar], children's multisensory integration capabilities are more coarse and more strongly dependent on bottom-up sensory evidence [17.Lewkowicz D.J. et al.The multisensory cocktail party problem in children: synchrony-based segregation of multiple talking faces improves in early childhood.Cognition. 2022; 228105226Crossref PubMed Scopus (0) Google Scholar]. In the laboratory, the principles of multisensory integration and crossmodal recalibration have been extensively studied with the ventriloquist effect [18.Bruns P. The ventriloquist illusion as a tool to study multisensory processing: an update.Front. Integr. Neurosci. 2019; 13: 51Crossref PubMed Scopus (15) Google Scholar,19.Chen L. Vroomen J. Intersensory binding across space and time: a tutorial review.Atten. Percept. Psychophys. 2013; 75: 790-811Crossref PubMed Scopus (153) Google Scholar] (Figure 1, Key figure). If a sound is presented together with a spatially discrepant light, the perceived sound location is typically shifted towards the visual stimulus (multisensory integration). After repeated exposure to a constant audiovisual spatial discrepancy, unimodal auditory spatial representations are adjusted to correct for the spatial mismatch (crossmodal recalibration), indicative of a crossmodal learning process (Box 1). Similar crossmodal spatial integration and recalibration phenomena have been described for visual–tactile [20.Badde S. et al.Modality-specific attention attenuates visual–tactile integration and recalibration effects by reducing prior expectations of a common source for vision and touch.Cognition. 2020; 197104170Crossref PubMed Scopus (24) Google Scholar], visual–vestibular [9.Zaidel A. et al.Multisensory calibration is independent of cue reliability.J. Neurosci. 2011; 31: 13949-13962Crossref PubMed Scopus (55) Google Scholar], and audio–tactile [21.Bruns P. Röder B. Tactile capture of auditory localization: an event-related potential study.Eur. J. Neurosci. 2010; 31: 1844-1857Crossref PubMed Scopus (30) Google Scholar,22.Bruns P. et al.Tactile recalibration of auditory spatial representations.Exp. Brain Res. 2011; 209: 333-344Crossref PubMed Scopus (15) Google Scholar] stimulus combinations.Box 1Crossmodal recalibration as a type of perceptual learningOutside laboratory situations in which a highly artificial conflict between auditory and visual spatial cues is introduced, calibration of the perceptual systems through crossmodal experience is thought to improve the accuracy with which stimuli in the environment can be localized. The resulting increase in perceptual performance is a feature that crossmodal recalibration shares with other types of perceptual learning, such as enhanced detection or feature discrimination thresholds after unimodal training on a perceptual task [114.Watanabe T. Sasaki Y. Perceptual learning: toward a comprehensive theory.Annu. Rev. Psychol. 2015; 66: 197-221Crossref PubMed Scopus (192) Google Scholar]. Under certain conditions, perceptual learning is induced even by mere passive exposure to a stimulus feature [115.Beste C. et al.Improvement and impairment of visually guided behavior through LTP- and LTD-like exposure-based visual learning.Curr. Biol. 2011; 21: 876-882Abstract Full Text Full Text PDF PubMed Scopus (78) Google Scholar]. However, research on visual perceptual learning has identified attention and reward as important drivers of learning. These studies found that learning is determined not only by stimulus presentation but also by the task: learning of attended stimulus features was enhanced, whereas task-irrelevant stimulus features were typically suppressed unless they escaped top-down inhibitory control because they were presented subliminally [116.Tsushima Y. et al.Task-irrelevant learning occurs only when the irrelevant feature is weak.Curr. Biol. 2008; 18: R516-R517Abstract Full Text Full Text PDF PubMed Scopus (86) Google Scholar]. Unsuppressed subliminal stimulus features were particularly likely to induce learning when they were temporally paired with a reward signal [117.Seitz A.R. et al.Rewards evoke learning of unconsciously processed visual stimuli in adult humans.Neuron. 2009; 61: 700-707Abstract Full Text Full Text PDF PubMed Scopus (252) Google Scholar].Recent evidence suggests that the interplay between attention and reward in perceptual learning undergoes dramatic changes during ontogenetic development. Compared to young adults, both elementary school-age children [118.Frank S.M. et al.Fundamental differences in visual perceptual learning between children and adults.Curr. Biol. 2021; 31: 427-432Abstract Full Text Full Text PDF PubMed Scopus (7) Google Scholar] and older adults (>65 years of age) [119.Chang L.-H. Age-related declines of stability in visual perceptual learning.Curr. Biol. 2014; 24: 2926-2929Abstract Full Text Full Text PDF PubMed Google Scholar] showed enhanced perceptual learning of (and thus a reduced ability to suppress) suprathreshold task-irrelevant features. It is currently unclear whether these less effective cognitive control processes in children and older adults translate into a heightened sensitivity for crossmodal recalibration, particularly for larger crossmodal discrepancies that might usually be detected and suppressed in young adults [94.Slutsky D.A. Recanzone G.H. Temporal and spatial dependency of the ventriloquism effect.Neuroreport. 2001; 12: 7-10Crossref PubMed Google Scholar]. However, as in unimodal perceptual learning studies, reward and task-relevance of the individual modality cues have recently been shown to modulate the strength of multisensory spatial integration and recalibration in adults [7.Kramer A. et al.Feedback modulates audio-visual spatial recalibration.Front. Integr. Neurosci. 2020; 13: 74Crossref PubMed Scopus (6) Google Scholar,20.Badde S. et al.Modality-specific attention attenuates visual–tactile integration and recalibration effects by reducing prior expectations of a common source for vision and touch.Cognition. 2020; 197104170Crossref PubMed Scopus (24) Google Scholar,120.Bruns P. et al.Reward expectation influences audiovisual spatial integration.Atten. Percept. Psychophys. 2014; 76: 1815-1827Crossref PubMed Scopus (19) Google Scholar], although the role of these top-down factors during ontogenetic development of spatial functions has remained elusive. These similarities might suggest that crossmodal recalibration is not a special type of learning but rather linked to more general learning mechanisms that are relevant across different domains. Outside laboratory situations in which a highly artificial conflict between auditory and visual spatial cues is introduced, calibration of the perceptual systems through crossmodal experience is thought to improve the accuracy with which stimuli in the environment can be localized. The resulting increase in perceptual performance is a feature that crossmodal recalibration shares with other types of perceptual learning, such as enhanced detection or feature discrimination thresholds after unimodal training on a perceptual task [114.Watanabe T. Sasaki Y. Perceptual learning: toward a comprehensive theory.Annu. Rev. Psychol. 2015; 66: 197-221Crossref PubMed Scopus (192) Google Scholar]. Under certain conditions, perceptual learning is induced even by mere passive exposure to a stimulus feature [115.Beste C. et al.Improvement and impairment of visually guided behavior through LTP- and LTD-like exposure-based visual learning.Curr. Biol. 2011; 21: 876-882Abstract Full Text Full Text PDF PubMed Scopus (78) Google Scholar]. However, research on visual perceptual learning has identified attention and reward as important drivers of learning. These studies found that learning is determined not only by stimulus presentation but also by the task: learning of attended stimulus features was enhanced, whereas task-irrelevant stimulus features were typically suppressed unless they escaped top-down inhibitory control because they were presented subliminally [116.Tsushima Y. et al.Task-irrelevant learning occurs only when the irrelevant feature is weak.Curr. Biol. 2008; 18: R516-R517Abstract Full Text Full Text PDF PubMed Scopus (86) Google Scholar]. Unsuppressed subliminal stimulus features were particularly likely to induce learning when they were temporally paired with a reward signal [117.Seitz A.R. et al.Rewards evoke learning of unconsciously processed visual stimuli in adult humans.Neuron. 2009; 61: 700-707Abstract Full Text Full Text PDF PubMed Scopus (252) Google Scholar]. Recent evidence suggests that the interplay between attention and reward in perceptual learning undergoes dramatic changes during ontogenetic development. Compared to young adults, both elementary school-age children [118.Frank S.M. et al.Fundamental differences in visual perceptual learning between children and adults.Curr. Biol. 2021; 31: 427-432Abstract Full Text Full Text PDF PubMed Scopus (7) Google Scholar] and older adults (>65 years of age) [119.Chang L.-H. Age-related declines of stability in visual perceptual learning.Curr. Biol. 2014; 24: 2926-2929Abstract Full Text Full Text PDF PubMed Google Scholar] showed enhanced perceptual learning of (and thus a reduced ability to suppress) suprathreshold task-irrelevant features. It is currently unclear whether these less effective cognitive control processes in children and older adults translate into a heightened sensitivity for crossmodal recalibration, particularly for larger crossmodal discrepancies that might usually be detected and suppressed in young adults [94.Slutsky D.A. Recanzone G.H. Temporal and spatial dependency of the ventriloquism effect.Neuroreport. 2001; 12: 7-10Crossref PubMed Google Scholar]. However, as in unimodal perceptual learning studies, reward and task-relevance of the individual modality cues have recently been shown to modulate the strength of multisensory spatial integration and recalibration in adults [7.Kramer A. et al.Feedback modulates audio-visual spatial recalibration.Front. Integr. Neurosci. 2020; 13: 74Crossref PubMed Scopus (6) Google Scholar,20.Badde S. et al.Modality-specific attention attenuates visual–tactile integration and recalibration effects by reducing prior expectations of a common source for vision and touch.Cognition. 2020; 197104170Crossref PubMed Scopus (24) Google Scholar,120.Bruns P. et al.Reward expectation influences audiovisual spatial integration.Atten. Percept. Psychophys. 2014; 76: 1815-1827Crossref PubMed Scopus (19) Google Scholar], although the role of these top-down factors during ontogenetic development of spatial functions has remained elusive. These similarities might suggest that crossmodal recalibration is not a special type of learning but rather linked to more general learning mechanisms that are relevant across different domains. Multisensory spatial integration is well described by hierarchical causal inference models, which take the relative reliability of the individual spatial cues and their estimated causal structure into account [5.Körding K.P. et al.Causal inference in multisensory perception.PLoS One. 2007; 2e943Crossref PubMed Scopus (646) Google Scholar,6.Rohe T. Noppeney U. Cortical hierarchies perform Bayesian causal inference in multisensory perception.PLoS Biol. 2015; 13e1002073Crossref PubMed Scopus (177) Google Scholar]. These models suggest that the brain computes two spatial estimates: one in which the individual spatial cues are integrated according to their relative reliability (assumption of a common cause), and one in which they are segregated (assumption of separate causes). These two spatial estimates are then combined depending on their respective probability, which is derived from the sensory evidence and the prior expectation of a common cause. More recently, it has been noticed that in multisensory integration observers tend to overestimate the reliability of the visual signal [23.Arnold D.H. et al.Suboptimal human multisensory cue combination.Sci. Rep. 2019; 9: 5155Crossref PubMed Scopus (10) Google Scholar,24.Meijer D. et al.Integration of audiovisual spatial signals is not consistent with maximum likelihood estimation.Cortex. 2019; 119: 74-88Crossref PubMed Scopus (25) Google Scholar], likely because reliability is estimated at least partly from past experience [25.Beierholm U. et al.Using the past to estimate sensory uncertainty.eLife. 2020; 9e54172Crossref PubMed Google Scholar] and typically is higher for visual than for auditory spatial input, resulting in a visual bias in spatial tasks [26.Petrini K. et al.Hearing where the eyes see: children use an irrelevant visual cue when localizing sounds.Child Dev. 2015; 86: 1449-1457Crossref PubMed Scopus (19) Google Scholar]. It has been demonstrated that immediately after encountering a spatially discrepant audiovisual stimulus, unimodal auditory localization remains partially shifted in the direction of the visual location in the preceding audiovisual trial [8.Wozny D.R. Shams L. Recalibration of auditory space following milliseconds of cross-modal discrepancy.J. Neurosci. 2011; 31: 4607-4612Crossref PubMed Scopus (0) Google Scholar,12.Bruns P. Röder B. Sensory recalibration integrates information from the immediate and the cumulative past.Sci. Rep. 2015; 5: 12739Crossref PubMed Scopus (40) Google Scholar,27.Mendonça C. et al.Predicting auditory space calibration from recent multisensory experience.Exp. Brain Res. 2015; 233: 1983-1991Crossref PubMed Scopus (15) Google Scholar]. Immediate recalibration after a single audiovisual discrepancy trial has been dissociated from cumulative recalibration (also known as the ventriloquist aftereffect) resulting from prolonged exposure to a consistent crossmodal spatial discrepancy for tens to a few hundred trials [12.Bruns P. Röder B. Sensory recalibration integrates information from the immediate and the cumulative past.Sci. Rep. 2015; 5: 12739Crossref PubMed Scopus (40) Google Scholar,28.Bosen A.K. et al.Multiple time scales of the ventriloquism aftereffect.PLoS One. 2018; 13e0200930Crossref PubMed Scopus (17) Google Scholar,29.Watson D.M. et al.Distinct mechanisms govern recalibration to audio-visual discrepancies in remote and recent history.Sci. Rep. 2019; 9: 8513Crossref PubMed Scopus (13) Google Scholar]. Additional evidence has suggested that cumulative recalibration can be long-lasting and further increase with repeated exposure over multiple sessions [30.Bruns P. Röder B. Repeated but not incremental training enhances cross-modal recalibration.J. Exp. Psychol. Hum. Percept. Perform. 2019; 45: 435-440Crossref PubMed Scopus (6) Google Scholar]. Recently, the role of spatial cue reliability for crossmodal recalibration has been debated. In multisensory integration, individual spatial cues are weighted according to their relative reliability, which maximizes the precision with which a crossmodal event can be localized [1.Ernst M.O. Bülthoff H.H. Merging the senses into a robust percept.Trends Cogn. Sci. 2004; 8: 162-169Abstract Full Text Full Text PDF PubMed Scopus (1226) Google Scholar,24.Meijer D. et al.Integration of audiovisual spatial signals is not consistent with maximum likelihood estimation.Cortex. 2019; 119: 74-88Crossref PubMed Scopus (25) Google Scholar]. While some studies have suggested a similar dependence of crossmodal recalibration on relative reliability [31.Burge J. et al.Visual-haptic adaptation is determined by relative reliability.J. Neurosci. 2010; 30: 7714-7721Crossref PubMed Scopus (57) Google Scholar,32.Hong F. et al.Causal inference regulates audiovisual spatial recalibration via its influence on audiovisual perception.PLoS Comput. Biol. 2021; 17e1008877Crossref Scopus (4) Google Scholar], others have failed to observe an effect of spatial cue reliability, and have suggested that cue weighting in cumulative recalibration instead follows a fixed ratio [9.Zaidel A. et al.Multisensory calibration is independent of cue reliability.J. Neurosci. 2011; 31: 13949-13962Crossref PubMed Scopus (55) Google Scholar,33.Rohlf S. et al.The effects of cue reliability on crossmodal recalibration in adults and children.Multisens. Res. 2021; 34: 743-761Crossref Scopus (5) Google Scholar]. A fixed ratio weighting – in which, for example, the visual spatial cue always dominates regardless of its current reliability – would be compatible with a primary aim of crossmodal recalibration to maximize accuracy rather than precision. When no feedback about the veridical stimulus location is available, observers might put a fixed high weight on the usually more accurate visual input. By contrast, when feedback is available and indicates that the visual rather than the auditory cue is inaccurate, cumulative recalibration is reduced [7.Kramer A. et al.Feedback modulates audio-visual spatial recalibration.Front. Integr. Neurosci. 2020; 13: 74Crossref PubMed Scopus (6) Google Scholar]. Electrophysiological and neuroimaging studies in humans have revealed that the neural computations underlying the causal inference process in multisensory integration are subserved by parietal [6.Rohe T. Noppeney U. Cortical hierarchies perform Bayesian causal inference in multisensory perception.PLoS Biol. 2015; 13e1002073Crossref PubMed Scopus (177) Google Scholar] and prefrontal [34.Cao Y. et al.Causal inference in the multisensory brain.Neuron. 2019; 102: 1076-1087Abstract Full Text Full Text PDF PubMed Scopus (79) Google Scholar,35.Mihalik A. Noppeney U. Causal inference in audiovisual perception.J. Neurosci. 2020; 40: 6600-6612Crossref PubMed Scopus (11) Google Scholar] brain regions and require around 200 ms from stimulus onset [36.Aller M. Noppeney U. To integrate or not to integrate: temporal dynamics of hierarchical Bayesian causal inference.PLoS Biol. 2019; 17e3000210Crossref PubMed Scopus (41) Google Scholar]. The output of this process is then fed back to the sensory cortices where it results in an adjustment of the perceived stimulus location [21.Bruns P. Röder B. Tactile capture of auditory localization: an event-related potential study.Eur. J. Neurosci. 2010; 31: 1844-1857Crossref PubMed Scopus (30) Google Scholar,37.Bonath B. et al.Neural basis of the ventriloquist illusion.Curr. Biol. 2007; 17: 1697-1703Abstract Full Text Full Text PDF PubMed Scopus (117) Google Scholar]. The relatively long latency of these computations suggests that goal-directed actions performed under time pressure may not fully utilize top-down knowledge about the causal structure, whereas unspeeded verbal judgments might incorporate decisional processes to a larger degree [35.Mihalik A. Noppeney U. Causal inference in audiovisual perception.J. Neurosci. 2020; 40: 6600-6612Crossref PubMed Scopus (11) Google Scholar]. However, while multisensory spatial integration can be modulated by top-down cognitive processes – such as semantic congruence (e.g., combining the sight of a dog with a barking sound) [38.Delong P. Noppeney U. Semantic and spatial congruency mould audiovisual integration depending on perceptual awareness.Sci. Rep. 2021; 11: 10832Crossref PubMed Scopus (5) Google Scholar] and availability of meta-information about the spatial layout [39.Smyre S.A. et al.Multisensory enhancement of overt behavior requires multisensory experience.Eur. J. Neurosci. 2021; 54: 4514-4527Crossref PubMed Scopus (5) Google Scholar] – it primarily involves sensory processes that are independent of response strategies [40.Vroomen J. Stekelenburg J.J. A bias-free two-alternative forced choice procedure to examine intersensory illusions applied to the ventriloquist effect by flashes and averted eye-gazes.Eur. J. Neurosci. 2014; 39: 1491-1498Crossref PubMed Scopus (6) Google Scholar] and even awareness of the visual stimulus component [38.Delong P. Noppeney U. Semantic and spatial congruency mould audiovisual integration depending on perceptual awareness.Sci. Rep. 2021; 11: 10832Crossref PubMed Scopus (5) Google Scholar]. The processes underlying multisensory spatial integration largely overlap with immediate recalibration but are more distinct from cumulative recalibration. For example, the amount of immediate crossmodal recalibration is correlated with the size of the ventriloquist illusion in the preceding audiovisual trial [33.Rohlf S. et al.The effects of cue reliability on crossmodal recalibration in adults and children.Multisens. Res. 2021; 34: 743-761Crossref Scopus (5) Google Scholar,41.Bruns P. et al.Audiovisual spatial recalibration but not integration is shaped by early sensory experience.iScience. 2022; 25104439Abstract Full Text Full Text PDF PubMed Scopus (2) Google Scholar], in line with neural findings suggesting a common neural substrate of multisensory integration and immediate recalibration in parietal brain regions [42.Park H. Kayser C. Shared neural underpinnings of multisensory integration and trial-by-trial perceptual recalibration in humans.eLife. 2019; 8e47001Crossref Google Scholar]. Moreover, event-related potential studies have observed that bo