PROCEEDINGS of the23rd International Congress on Acoustics
9 to 13 September 2019 in Aachen, Germany

A study on the relationship between speed of sound image andevaluation value of moving sound image
Mika KURABAYASHI 1; Kenji MUTO2
1, 2

Shibaura Institute of Technology, JapanABSTRACT
The overall aim of this research is to develop a high-presence sound system for an emergency drill
simulator using a virtual reality display system that runs on a personal computer. Previous studies
have reported systems that provide high-presence sound, but these systems needed powerfulcomputers to perform the required processing. With our proposed method, the processing is reduced
by using a discontinuous moving sound image given by discrete coordinates. We evaluated thesynchronization of the moving sound image by conducting an experiment in which pairs of stimuli
were compared. In the experiment, moving sound images with different speeds were compared inorder to obtain a threshold for the number of discrete coordinates of the moving sound image to realize
high presence. We analyzed the relationship between the evaluations and the speed using a paired
comparison method. The results showed that the evaluation of the realism depended on the speed ofthe sound image.
Keywords: Moving sound image, Discontinuous moving sound image, Subjective evaluation, Paircomparison method

1. INTRODUCTION

A number of systems have been developed to reproduce highly realistic sound. For example, Iseet al. developed a high-presence interactive sound field (1) based on the boundary-surface control(BoSC) principle (2). Poletti defined the theory of three-dimensional sound systems on the basis ofspherical harmonics (3). Highly realistic sound systems are necessary to produce higher immersion,
such as virtual reality content. For example, Hamasaki et al. developed a 22.2 multichannel soundsystem for ultra-high-definition television systems (4). However, such systems are very expensive
because they require a powerful computer to perform a huge amount of calculations.
Our aim is to develop a highly realistic sound system for an emergency drill simulator and to havethe system run on a personal computer. Reducing the frame rate enables the size of transmitted video
to be reduced (5). We have proposed a method for reducing the amount of required processing bygiving discrete coordinates to a sound image and moving it discontinuously. In many cases, soundimage was given by the difference of sound pressures and time delay as in stereo system. In ourexperiments, sound image was given by an acoustical panning technique; the difference of soundpressures, and it was obtained good localization (6).
The moving sound image was shown with a continuous movie. When visual capture occurred, themoving sound image was perceived as being natural (7). Some studies have clarified the conditions
under which visual capture occurs for a continuous moving image (8, 9). However, the conditions
under which it occurs when a discontinuous moving sound image is displayed remain unclear.
Moreover, the effect of the speed of the sound image on the conditions has not been investigated.
In this paper, we propose a moving sound image technique that uses a visual effect to simplify
sound images. In this study, we presented moving sounds in conjunction with moving sound icons
at fast and slow speeds with stereo speakers. We conducted an experiment to evaluate thesynchronization between a sound image moving discontinuously and a movie of icon moving
continuously.

1ma18038@shibaura-it.ac.jp, 2 k-muto@shibaura-it.ac.jp4827

2. TECHNIQUE FOR MOVING SOUND IMAGE

We proposed a technique to reduce the amount of processing by using a sound image that moves
discontinuously (6). The sound image was moved the same distance and at the same speed by using thedifference in sound pressure between left and right channels. Figure 1 shows the concept of the moving
sound and movie and figure 2 shows the definition of the parameters for sound image localization. Let D
be the length of an arc between two loudspeakers which the center is the observer. The velocity of the soundimage 𝑣𝑣 is given by the following equation:v = D/(tmax – t0 ).
(1)When the sound image is given different levels in stereo speakers by using an acoustical panningtechnique. It was easy for reducing the amount of required processing (10). Our previous research showsthat accuracy of the sound image localization was high, and the location identification was around thetheoretical value (6).
The amplitudes of the left channel 𝐴𝐴𝐿𝐿 and the right channel 𝐴𝐴𝑅𝑅 are different. Here, S(t) is the input
signal, 𝑃𝑃𝐿𝐿 is the sound pressure output from the left, and 𝑃𝑃𝑅𝑅 is the sound pressure output from the rightchannel, with 𝑃𝑃𝐿𝐿 and 𝑃𝑃𝑅𝑅 thus given by the following equations,
𝑃𝑃𝐿𝐿 = 𝐴𝐴𝐿𝐿 𝑆𝑆(𝑡𝑡),(2)
𝑃𝑃𝑅𝑅 = 𝐴𝐴𝑅𝑅 𝑆𝑆(𝑡𝑡).
(3)

The sound pressure at the center of the observer’s head is given by the following equation:
𝑃𝑃 = (𝐴𝐴𝐿𝐿 + 𝐴𝐴𝑅𝑅 )𝑆𝑆(𝑡𝑡).
(4)
In Figure 2, x gives the location of the sound image on the arc between loudspeakers centered at theobserver. The arc is divided in the number of 𝑥𝑥𝑚𝑚𝑚𝑚𝑚𝑚 . The amplitudes of the left channel and theright channel at x are given by
𝐴𝐴𝐿𝐿 = 𝐶𝐶1 (𝑥𝑥𝑚𝑚𝑚𝑚𝑚𝑚 − 𝑥𝑥)(5)
𝐴𝐴𝑅𝑅 = 𝐶𝐶1 𝑥𝑥(6)
𝐴𝐴𝐿𝐿 + 𝐴𝐴𝑅𝑅 = 𝐶𝐶2(7)

where 𝐶𝐶1 and 𝐶𝐶2 are constants, and P is always constant because the distance between the soundimage and observer is also constant. Figure 3 shows the time waveform of 𝑃𝑃. The time waveformof 𝑃𝑃𝐿𝐿 and 𝑃𝑃𝑅𝑅 in the case where 𝑥𝑥𝑚𝑚𝑚𝑚𝑚𝑚 = 4 is shown in Figure 4 and the waveform in the case where𝑥𝑥𝑚𝑚𝑚𝑚𝑚𝑚 = 150 is shown in Figure 5. The icon for movie moved same direction as the sound image at30fps. The icon size is constant because it moved on the arc.
We presented sound and movie stimuli and evaluate whether they were perceived as being
synchronized by using a paired comparison method.

Figure 1 - The concept of the moving soundand movie
Figure 2 – Location examples of sound image
Each location 𝑥𝑥𝑖𝑖 is of a moving sound image

4828

Figure 3 – Time waveform for mixed right channel and left channel

Figure 4 – Time waveforms of sound stimuli inleft and right channels, where 𝑥𝑥𝑚𝑚𝑚𝑚𝑚𝑚 = 4Figure 5 – Time waveforms of sound stimuli inleft and right channels, where 𝑥𝑥𝑚𝑚𝑚𝑚𝑚𝑚 = 150

3. EXPERIMENTS TO EVALUATE SYNCHRONIZATION

We measured the synchronization between a moving sound image and a moving icon. The typeof sound used for moving sound is the siren of a Japanese ambulance in both experiment. TheDoppler effect didn’t include in the experiment because the sound image was moved the same distanceand speed from the observer.
3.1 Stimuli Patterns
The length of the stimuli had four patterns: 1.3 s, 2.5 s, 5.0 s, and 10 s; therefore, the speeds of thesound image were 36.8 deg/s, 18.4 deg/s, 9.2 deg/s, and 4.6 deg/s, respectively. The total number of
discrete coordinates for the sound images was 150. We used an ambulance as the icon for the moving
sound. The movie stimuli were produced at 30 fps. Table 1 shows the four patterns of stimuli usedin the subjective experiment.
Table 1 Categories of stimuli4829

3.2 Experimental environment and procedure
The stimuli were played back using 24-inch PC monitors (DELL P2416) and USB loudspeakers(Towa Electronics, Olasonic TW-S7). The distance between the monitor and the observer was 60 cm
and the viewing angle 46 degrees. The 11 observers were college students in their twenties.
We presented a pair of stimuli to the observer and asked to him or her to indicate on a questionnaire
whether the moving sound was synchronized with the moving icon. Figure 6 shows a conceptual
representation of the experiment. Four patterns of stimuli were arranged as six different pairs andwe presented these pairs to each observer in a random order. For each pair of stimuli, the stimulus
presented first was named A and the stimulus presented second was named B. The interval between
A and B was 2 s. After all the pairs were presented, we showed the stimuli in reverse order for eachpair. In the questionnaire, the observers entered their evaluations of the synchronization on a 7-point
scale. The evaluations were analyzed using Ura’s method of paired comparison (11).

Figure 6 – Experiment composition

4. ANALYSIS OF ACCURACY AND SYNCHRONIZATION MEASUREMENTS

Figure 7 shows the analysis of these evaluations. As shown in the figure, speed affected theevaluation of the synchronization. The arrow in the figure shows the confidence interval used toassess the significance. The case where the speed was 18.4 deg/s is considerably different from thecase where the difference was 36.8 deg/s. It indicates that when the speed is faster, the evaluation islower. Figure 8 shows the analysis results for the relationship between the number of sound images
𝑥𝑥𝑚𝑚𝑚𝑚𝑚𝑚 and the evaluation of the synchronization and Table 2 shows the ten patterns of stimuli used in
the subjective experiment (12). A number next to a letter in the figure indicates the value of 𝑥𝑥𝑚𝑚𝑚𝑚𝑚𝑚 .
In the case where the speed of the sound source is 9.2 deg/s, there is no great difference in theevaluations between S150 to S10, in the result for this speed means that 𝑥𝑥𝑚𝑚𝑚𝑚𝑚𝑚 = 10 is not over thethreshold for being proposed as the number of locations of a continuous moving sound. In contrast,in the case where the speed of the sound source is 18.4 deg/s, there is a considerable difference in theevaluations between F150 to F10, in the result for this speed means that 𝑥𝑥𝑚𝑚𝑚𝑚𝑚𝑚 = 10 is over thethreshold for being proposed as the number of locations of a continuous moving sound. In summary,when the number of sound images was less, the evaluation was lower.
These experiments demonstrated that speed affects the evaluation of synchronization, as shown inFigures 7 and 8. When the speed was faster, the length of video was shorter. We think that thedecreasing synchronization was due to the length of time available for presentation of the sound image
icon being too short when the speed was fast.
To construct a system meeting the objective of this research, we need to obtain a threshold for thenumber of discrete coordinates for a moving sound image when the observer moves the head to search
for the sound source.

4830

Figure 7 – Scale diagram showing confidence of synchronizing at different speeds of moving
sound images and moving icons

Figure 8 – Scale diagram showing confidence interval of synchronizing for five patterns of 𝑥𝑥𝑚𝑚𝑚𝑚𝑚𝑚 andtwo patterns of speed
Table 2 Categories of stimuli used in Figure 8

5. CONCLUSIONS

Our aim was to reduce amount of processing used to generate visual effects for developing a highpresence sound system that runs on a personal computer. In this study, we measured the effect of thespeed of a sound image on the synchronization between sound and movie. The analysis of theevaluations of the synchronization shows that the evaluation of realism depends on the speed of thesound image. In future work, we will investigate sound images moving at various speeds.

ACKNOWLEDGEMENTS

This work was supported by JSPS KAKENHI Grant-in-Aid for Scientific Research (C) Numbers
21500922.

REFERENCES

1. Ikeda Y, Ise S. Sound Cask: Music and Voice Communications System with Three-dimensional Sound
Reproduction Based on Boundary Surface Control Principle. Proc ICA 19; 2-7 June 2013; Montrial,Canada 2013. pp. 1-8
2. Ise S. A Principle of Active Control of Sound Based on the Kirchhoff-Helmholtz Integral Equation andthe inverse (in Japanese). J. Acoust. Soc. Jpn. 1997; 53(9): 706-7134831

3. Poletti M. A. Three-Dimensional Surround Sound Systems Based on Spherical Harmonics. J. Audio
Eng. Soc. 2005; 53(11): 1004-1025
4. Hamasaki K, Nishiguchi T, Okumura R, Nakayama Y, Ando A. 22.2 Multichannel Sound System forUltra High-Definition TV. SMPRW Motion Imaging Journal. 2007; 117(3): 40-49
5. Takishima Y, Wada M, Murakami H. A Study of Optimal Frame Rate in Low Bit Rate Video Coding.
ITEJ technical report 1991; 15(49): 45-52
6. Kurabayashi M, Muto K. A Study on the Impression of Moving Sound Image Influenced by the Speedof the Image of Sound Source (in Japanese). Acoust. Soc. Conf. 2019; 5-7 March 2019; Tokyo Japan
2019. pp.
7. Slutsky D.A., Tecanzon G. H. Temporal and spatial dependency of the ventriloquism effect. NeuroReport.
2001; 12(1): 7-10

8. Kobayashi M, Fujii S, Iwaya Y, Sakamoto S, Suzuki Y. Temporal Characteristics of the Ventrioquism
Effect (in Japanese). TVRSJ 2011; 16(1):93-97
9. Kimura M, Kajii H, Takahashi M, Yamamoto K. The Ventriloquism Effect in Peripheral Vision (inJapanese). TVRSJ 1999; 14(1): 253-260
10 Hirahara T, Ashihara K, Ozawa K, Miyasaka E. Sound and humans (in Japanese), Tokyo, Japan: Corona
publishing; 2013. p. 183.
11. Amasaka K, Nagasawa S. Basics and Applocations of Sensory Evaluation – For the Kansei Engineeringin Automobile Development – (in Japanese), Tokyo, Japan: Japan Standards Association; 2000. pp. 182193.
12. Kurabayashi M, Muto K. A Study on the Impression of Moving Sound Image Influenced by Presence
Order of stimuli (in Japanese). IEICE Gen. Conf. 2019; 19-22 March 2019; Tokyo, Japan 2019. p. 464832