PROCEEDINGS of the23rd International Congress on Acoustics
9 to 13 September 2019 in Aachen, Germany

The role of audiovisual temporal coherence in auditory sceneanalysisJennifer Kim BIZLEY1; Huriye ATILGAN1,2; Ana Isabel SANCHEZ JIMENEZ3
12

University College London, United Kingdom

University College London, United Kingdom & Yale University, United States of America
3

University College London, United Kingdom

ABSTRACT
Perception is seamlessly multisensory, and interpreting an object through multiple modalities offers
advantages over unisensory processing. We have recently demonstrated that audiovisual temporal coherence
can modulate the performance of human listeners in an auditory selective attention task (1) (Maddox et al.,2015). When the radius of a visual stimulus changed in time with the intensity fluctuations of a target stream,
listeners were better able to detect brief pitch or timbre perturbations than when the visual stimulus was
temporally coherent with the to-be-ignored stream. Importantly, the timing or presence of the pitch / timbre
perturbations were not predicted by the changes in the size of the visual stimulus.
Recording neural activity in the auditory cortex of ferrets in response to similar stimuli revealed a single
neuron correlate of this effect, with the visual stimulus dictating which sound neurons represented within a
mixture (2). Therefore by using stimuli that consisted of independently amplitude modulated acoustic
streams and a radius (for human psychophysics) or intensity (for recordings in ferrets) modulated visual
stimulus we were able to demonstrate preferential processing for sounds that were temporally coherent withthe visual stimulus. In these studies the coherent changes in visual size/luminance and auditory amplitude
bound information across modalities. However, the perceptual / neural benefits we observed extended tonon-binding features of the sound – auditory pitch and timbre- consistent with the formation of an
audiovisual object (3). Ongoing work is exploring the parameters over which these effects extend by
manipulating onset asynchronies across modalities and the extent of the temporal coherence.
Human listeners varied in their ability to benefit from audiovisual temporal coherence (1). We therefore
conducted a training study in which participants performed 5 short training sessions learning to either detect
audiovisual temporal coherence, or were exposed to temporally coherent audiovisual stimuli while
performing a rate discrimination task, or only completed the pre and post-test tasks. Both pre and post-tests
included the auditory selective attention task used in (1) and an audiovisual temporal coherence detection
threshold task. Both trained groups improved their performance on the auditory selective attention task frompre-test to post-test. However, only listeners who improved their ability to discriminate audiovisual temporal
coherence altered the way in which they were able to benefit from audiovisual temporal coherence, such that
temporal coherence between both target and masker sounds was advantageous when compared toperformance with an independently modulated visual stream (4).
Keywords: Audiovisual, Multisensory, Auditory Cortex

REFERENCES
1.

Maddox RK, Atilgan H, Bizley JK, Lee AK. Auditory selective attention is enhanced by a task-irrelevant

temporally coherent visual stimulus in human listeners. eLife. 2015;4:e04995.
13930

J.bizley@ucl.ac.uk

2.

Atilgan H, Town SM, Wood KC, Jones GP, Maddox RK, Lee AKC, et al. Integration of VisualInformation in Auditory Cortex Promotes Auditory Scene Analysis through Multisensory Binding. Neuron.
2018;97(3):640-55 e4.
3.

Bizley JK, Maddox RK, Lee AK. Defining Auditory-Visual Objects: Behavioral Tests and Physiological

Mechanisms. Trends Neurosci. 2016;39(2):74-85.
4.

Atilgan H, Bizley J. Training listeners to detect auditory-visual temporal coherence enhances their ability

to exploit visual information for auditory scene analysis. BioRxiv. 2018.

3931