PROCEEDINGS of the23rd International Congress on Acoustics
9 to 13 September 2019 in Aachen, Germany

Perception of auditory-visual simultaneity changes by ambientillumination
Masaaki TEZUKA1; Mariko TSURUTA-HAMAMURA2; Hiroshi HASEGAWA3
1

Utsunomiya University, Japan

2

Utsunomiya University, Japan3

Utsunomiya University, Japan

ABSTRACTIn this study, we investigated changes in the point of subjective simultaneity (PSS) between auditory andvisual stimuli by changes in ambient illumination. The test stimuli were a pure tone sound of 1000 Hz at 80
dB SPL and white light of LED at 21.9 cd/m². The illuminance levels at around the light source and at thesubject’s eyes were adjusted at the following 4 conditions;(100lx, 100 lx), (100 lx, 1000 lx), (1000 lx, 100lx), and (1000 lx, 1000 lx). Nine people in their early 20’s participated in the experiments. Synchronizedsound-light stimuli, which were the same as the test stimuli, were presented as the preceding stimuli,
followed by the test stimuli at an interval 1000 ms. The test stimuli had a stimulus onset asynchrony (SOA) of
0, ±20, ±40, ±80, or ±160 ms. After each presentation of the test stimuli, we asked subjects to answer whichof the stimuli ––– sound or light ––– they perceived first. Then we evaluated the PSS of the test stimuli andconsidered the influence of ambient illuminance on the PSS. As a result, the PSS shifted toward positive
direction. We showed that the PSS shift depended by changes in ambient illumination.
Keywords: PSS (point of subjective simultaneity), ambient illuminance, SOA (stimulus onset asynchrony)

1. INTRODUCTION
There are many researches have been conducted on the simultane ity between auditory and visualinformation (1, 10). We also have been studying the simultaneity of a paired auditory and visualstimulus (11, 16), and we showed that the point of subjective simultaneity (PSS) between an auditory
and visual stimulus shifted depending on the illuminance level at the subject’s eyes in the previous
paper (16).
The present paper investigates changes in the PSS between an auditory and visual stimulus
depending on the ambient illuminance, where the illuminance level changes not only at the subject ’seyes but also at around the light source.

2. Experiment
We investigated the perception of simultaneity between an auditory and visual stimulus when
ambient illumination changed.
2.1 Stimuli
We used a pure tone of 1000 Hz at 80 dB SPL (sound pressure level) and a while LED light with aluminance of 21.9 cd/m². Both stimuli had a duration of 10 ms. The auditory stimulus had a liner
fade-in and fade-out of 2.5 ms to reduce transient responses.
2.2 Apparatus
We carried out the experiments in a soundproof room (D-30; Fig. 1). We generated sound and lightstimuli using a stimulus generator (S-15137, Takei Scientific Instruments) controlled by a computer.
1

mt186524@cc.utsunomiya-u.ac.jp
mariko@is.utsunomiya-u.ac.jp
3
hasegawa@is.utsunomiya-u.ac.jp
26751

The sound stimuli with amplification through an amplifier (CR-D2, Onkyo) were presented to thesubject via headphones (SRH840, Shure), and the light stimuli were presented with a LED(LP-10HW3B). The illuminance levels around the light source and at the subject’s eyes were adjusted
at the following 4 patterns; (100 lx, 100 lx), (100 lx, 1000 lx), (1000 lx, 100 lx), and (1000 lx, 1000 lx)
using spotlights with dimmers located above the subject’s head and the light source as shown in Fig. 2.
The subject sat on a chair 2 m in front of the light source. The testing space was closed off with ablackout curtain to eliminate all other light.

Figure 1 – Experimental apparatus. The sound stimuli were presented via headphone to the subject.
The light stimuli were presented with a LED.

Figure 2 – Schematic figure of the experiment. Illuminance levels at around the LED and at thesubject’s eyes were adjusted using spotlights with dimmers located above the subject’s head and thelight source.

6752

2.3 Procedure
To control the subject’s attention to the sound and light stimuli, we presented the synchronizedcontinual trains of sound and light stimuli before a paired test stimulus as shown in Fig. 3. Twenty
synchronized sound-light stimuli were presented every 50 ms. Each stimulus was the same as the teststimulus. At 1000 ms after the onset of the last preceding stimulus, the sound and light test stimul iwere then presented at offset (stimulus onset asynchronies: SOAs) of 0, ±20, ±40, ±80, or ±160 ms,
where positive and negative values indicate that the light and sound stimuli were presented first,respectively. We presented the test stimuli with the nine SOAs in random order. After eachpresentation, the subject answered: “The sound was perceived first,” or “The light was perceivedfirst.”

Figure 3 – Presentation timing of the paired test stimulus. Twenty preceding synchronized stimuli
were presented every 50 ms before the test stimuli. At 1000 ms after the last preceding stimulus, thesound and light test stimuli were then presented at a SOA of 0, ±20, ±40, ±80, or ±160 ms.
We performed the above test in each ambient illumination condition Ⅰ to Ⅳ(table 1) separately.
Table 1 – Ambient illumination conditions.
Illumination condition

ⅠⅡⅢ

Ⅳ

Around light source (lx)

100

100

1000

1000

Subject’s eyes (lx)

100

1000

100

1000

2.4 Subjects and trials
Seven men and two women in their early 20’s participated in the experiments. All subjects had
normal hearing acuity and normal or corrected-to-normal vison. We conducted 45 trials (9 SOAs ×5 iterations) per session and performed 20 sessions in each illumination condition Ⅰ to Ⅳ, in which thepresentation order was different in every session. In total, we performed 32,400 trials (45 trials × 20sessions × 4 illumination condition × 9 subjects).

3. Results
Figures 4 (a) ~ (d) show the experimental results in the illumination conditions Ⅰ to Ⅳ, respectively.
The vertical and horizontal axes denote the answer rate and the stimulus onset asynchrony (SOA)
between the sound and light, respectively. The while and dark bars indicate the answer rates “soundwas perceived first” and “light was perceived first,” respectively.
In all the figures, the perception of sound precedence decreased and that of light precedence
increased as the SOA increased. At a SOA of 0 ms, the perception of the sound precedence wassmaller than that of light precedence in all the conditions. These results mean that the point ofsubjective simultaneity (PSS) shifted to sound precedence.

6753

(a) Illumination condition Ⅰ (around light: 100 lx, subject’s eyes: 100 lx)(b) Illumination condition Ⅱ (around light: 100 lx, subject’s eyes: 1000 lx)(c) Illumination condition Ⅲ (around light: 1000 lx, subject’s eyes: 100 lx)(d) Illumination condition Ⅳ (around light: 1000 lx, subject’s eyes: 1000 lx)
Figure 4 – Experimental results in the conditions Ⅰ to Ⅳ. The vertical and horizontal axes
denote the answer rate and the stimulus onset asynchrony (SOA) between the sound and light,respectively. The white and dark bars indicate the perception of sound and light precedence,respectively.

6754

4. Discussion
In order to quantitative analyze the experimental results, we fitted a psychometric function (11) tothe result of each subject in each condition, and obtained the PSS as the 50 % point of the answer ratewith respect to the SOA.
Table 2 show the average values of PSS of all subjects in each condition.
Table 2 – Averaged PSS values in each illumination condition.
Illumination condition

ⅠⅡⅢ

Ⅳ

Around light source (lx)

100

100

1000

1000

Subject’s eyes (lx)

100

1000

100

1000

PSS (ms)

െ71.40െ55.45

െ66.70െ43.60In table 2, all the PSS values were negative, i.e., the PSS shifted toward the perception of soundprecedence. In comparing the PSS values between the conditions Ⅰ and Ⅱ, where the illuminance levelat around the light source was 100 lx, the PSS shifted toward positive direction by 15.95 ms, i.e., thedegree of the perception of light precedence increased as the illuminance level at the subject’s eyes
increased from 100 lx to 1000 lx. This tendency agreed with our previous reports (14, 16), that is, by
raising illuminance level at the eyes, relative attention to the LED light decreased, and thus attentio n to
the sound relatively increased. Paying attention to the sound increased the speed of transmission ofauditory information, so the sound and light and light were perceived as simultaneous when the light
was presented earlier. The PSS then shifted toward the perception of light precedence. The sametendency was shown in comparing the PSS values between the conditions Ⅲ and Ⅳ.
On the other hand, in comparing the PSS values between the conditions Ⅰ and Ⅲ, where theilluminance level at the subject’s eyes was 100 lx, the PSS shifted toward positive direction by 4.70 ms,i.e., the degree of the perception of light precedence increased as the illuminance level at around theLED increased from 100 lx to 1000 lx. We here evaluated the Michelson luminance -contrast of theLED lights, then the contrast of the LED light decreased as the illuminance level at around the lightsource increased from 100 lx to 1000 lx as shown in Table 3. We considered that by decreasing thecontrast of the LED light, the visibility of the LED light decreased. The deterioration of th e visibilityof the LED light caused a decrease of relative attention to the LED, i.e., attention to the soundrelatively increased. Then the PSS shifted toward the perception of the light precedence. The sametendency was shown in comparing the PSS values between the conditions Ⅱ and Ⅳ.
From the above, we found that the PSS changed not only by changing in the illumination at the
subject’s eyes, but also by changing in the contrast of the light source.
Table 3 – Michelson luminance-contrast of the LED light.
Illuminance at around

Luminance ofLuminance ofMichelson ofthe light source (lx)the LED Light (cd/m2)

the peripheral area (cd/m2)

luminance-contrast

100

21.9

0.33

0.97

1000

25.0

4.22

0.71

5. CONCLUSIONSThe point of subjective simultaneity (PSS) between the auditory and visual stimuli shifted toward
the perception of light precedence as the ambient illumination increased. The PSS shift depended not
only by changes in the illumination at the subject’s eyes, but also by changes in the contract of the lightsource.

6755

REFERENCES
1. Hershenson, M., Reaction time as a measure of intersensory facilitation, Journal of Experimental
Psychology, 63 (3), 289-293, (1962).
2. Aschersleben, G. and Musseler, J., Dissociations in the timing of stationary and moving stimuli, Jounalof Experimental Psychology, 25 (6), 1709-1720, (1999)
3. Stone, J. V., Hunkin, N. M., Porriill, J., Wood, R., Keeler, V., beanl, M., Port, M., and Porter, N.R.,
When is now? Perception of simultaneity, Proceedings of the Royal Society B Biological Science, 268,
31-38, (2001)
4. Sugita, Y. and Suzuki, Y., Implicit estimation of sound-arrival time, Nature, 421, No.27, 911, (2003).
5. Neumann O. and Noepel M., Timing of ͆perception͇and perception of ͆time͇, Psychophysics beyond
sensation: laws and invariants of human cognition, 245-269, (2003)
6. Fujisaki, W., Shimojo, S., Kashino, M., and Nishida, S., Recalibration of audiovisual simultaneity ,Nature Neuroscience, 7, 773-778, (2004).
7. Lewald, J. and Guski, R., Auditory-visual temporal integration as a function of distance: nocompensation for sound-transmission time in human perception, Neuroscience Letters, 357 (2), 119-122,(2004)
8. Arnold, D. H., Johnston, A., and Nishida, S., Timing sight and sound, Vision Research, 45 (10),1275-1284, (2005)
9. Zampini, M., Shore, D. I., and Spence, C., Audiovisual prior entry, Neuroscience Letters, 381 (3),217-222, (2005)
10. Zampini, M., Guest,S., D. I., and Spence, C., Audio-visual simultaneity judgment, Perception &Psychophysics, 67 (3), 531-544, (2005)
11. Hasegawa, H., Yoshida, J., and Kasuga, M., Effect of attention to preceding stimuli on the simultaneity
perception between an auditory and visual stimulus, Proceedings of 20th International Congress onAcoustics, 236, (2010)
12. Hasegawa, H., Kasuga, M., and Yoshida, J., Influence of light luminance on auditory-visual
simultaneity perception, Proceedings of Forum Acusticum 2011, 000204, (2011)13. Hasegawa, H., Yamamoto, Y., Saito, W., and Yoshida, J., Learning changes the perception of theequivalence of strengths between auditory and visual stimuli, Proceedings of 20th International
Congress on Sound and Vibration, 341, (2013)
14. Hasegawa, H., Ishiwatari, K., and Yoshida, J., Ambient illuminance alters perception of auditory-visual
simultaneity, Proceedings of Forum Acusticum 2014, R17_12, (2014).
15. Hasegawa, H., and Saito, W., Learning of the equality between an auditory and a visual stimulus alters
the perception of auditory-visual simultaneity, Proceedings of 22th International Congress on Sound andVibration 2015, 676, (2015).
16. Hasegawa, H., and Hatakeyama, S., Perception of auditory-visual simultaneity changes by illuminanceat the eyes, Proceedings of 23rd International Congress on Sound and Vibration 2016, 636, (2016).

6756