PROCEEDINGS of the23rd International Congress on Acoustics
9 to 13 September 2019 in Aachen, Germany

Relatively Speaking: Spatial Discrimination Tasks As Toolsfor Studying Multisensory Integration
Ross K MADDOX1
1

University of Rochester, USAABSTRACT
Both the visual and auditory systems provide spatial information. Because many objects can be both seen andheard, the brain combines information from each sensory modality to form a single estimate of an object'slocation. When there is conflict between the cues from a single object, those cues are weighed by theirreliability, which usually results in a visually-driven bias of auditory localization, namely the “ventriloquisteffect” (1,2). Numerous studies of crossmodal integration have described the phenomenon behaviorally,modeled it (3,4), and explored its neural underpinnings (5). The overwhelming majority of these studies haveused absolute localization tasks, for which localization bias is the principal outcome, rather thandiscrimination tasks (i.e., relative localization), which measure spatial acuity. Studies from our lab suggest
this represents a missed opportunity. We have used spatial discrimination to show effects of eye gaze on
auditory spatial coding unrevealed by absolute localization tasks (6). This study showed that while auditory
localization biases induced by short-term gaze shifts are small and highly dependent on the specificlocalization task, the same gaze shifts lead to substantial changes in listeners’ ability to discriminate theazimuths of two nearby auditory stimuli. We have also used discrimination in tasks where auditory and visual
stimuli are collocated, so no localization bias is expected (7). This study goes beyond the typical paradigm ofa single pair of audio-visual stimuli with conflicting locations. Instead, it employs a much more naturalistic
paradigm that involves more than one stimulus pair, with congruent locations between auditory and visualstimuli. This paradigm makes bias unusable as an outcome (since no bias is expected for congruent locations)but uses spatial discrimination thresholds to learn about audio-visual integration when there is no location
conflict. Finally, preliminary studies in our lab have used spatial discrimination tasks to study auditory spatial
processing in people with hemianopia (one-sided blindness due to a visual cortical lesion), where studies ofauditory localization have suggested normal processing. In summary, using relative localization tasks instudies of audio-visual integration can reveal complementary information to tasks that rely on absolutelocalization (and thus cue conflicts), potentially even revealing substantial effects where absolute localization
may not.
Keywords: Audio-visual integration, psychophysics

ACKNOWLEDGEMENTS
This work was supported by the National Institute on Deafness and Other Communication
Disorders [R00DC014288].

REFERENCES
1. Jack CE, Thurlow WR. Effects of degree of visual association and angle of displacement on the“ventriloquism” effect. Percept Mot Skills. 1973 Dec;37(3):967–79.
2. Bertelson P, Radeau M. Cross-modal bias and perceptual fusion with auditory-visual spatial discordance.
Percept Psychophys. 1981;29(6):578–84.
3. Alais D, Burr D. The Ventriloquist Effect Results from Near-Optimal Bimodal Integration. Curr Biol.
2004 Feb 3;14(3):257–62.
4. Körding KP, Beierholm U, Ma WJ, Quartz S, Tenenbaum JB, Shams L. Causal Inference inMultisensory Perception. PLoS ONE. 2007 Sep 26;2(9):e943.
5. Bonath B, Noesselt T, Martinez A, Mishra J, Schwiecker K, Heinze H-J, et al. Neural Basis of theVentriloquist Illusion. Curr Biol. 2007 Oct 9;17(19):1697–703.
1ross.maddox@rochester.edu3913

6. Maddox RK, Pospisil DA, Stecker GC, Lee AKC. Directing Eye Gaze Enhances Auditory Spatial Cue
Discrimination. Curr Biol. 2014 Mar 31;24(7):748–52.
7. Cappelloni MS, Shivkumar S, Haefner RM, Maddox RK. Improvement in auditory spatialdiscrimination from ambiguous visual stimuli is not explained by ideal observer causal inference.
bioRxiv. 2019 Apr 3;598425.

3914