Measurement of Auditory Alley in Virtual Environment and Its Mathematical Model
Michiko Ohbra*, Yasuyuki Yanagida**, Taro Maeda**, & Susumu Tachi***Faculty of Engineering, Shibaura Institute of Technology,Omiya, Saitama 330-8570, Japanohkura@sic.shibaura-it.ac.jp** Faculty of Engineering, The University of Tokyo,
Bunkyo-ku, Tokyo 113-8656, Japan{yanagida, maeda, tachi}@star.t.u-tokyo.ac.jp
ABSTRACTIn space perception, the straight lines runningphenomenally parallel to the median plane and straightlines running phenomenally equi-distant from the medianplane are called parallel alley and equi-distance alley,
respectively. It is known that these lines are concave
bending towards the median plane in binocular visualspace. Using virtual environmental display system, it
became possible to construct the experimental system tomeasure the parallel alley and the eqi-distance alley inbinaural auditory space. The experiments were conducted
and the results were obtained as follows:X

i - R - - yFig. 1Parallel alley and equi-distance alley(1) Parallel alley and eqi-distance alley in auditory spaceof the same horizontal plane and the frontoparallel planeincluding two eyes of the observer. A pair of lines p shows

are not always straight in physical sense, and theparallel alley and the other pair of lines d shows equi-forms of them depend on the distance from thedistance alley. Parallel alley is a pair of perceived straightlines obtained when two rows of luminous points on thehorizontal plane of the eye level are arranged
symmetrically to appear straight and parallel to the medianplane. Physically, the two lines should be straight andparallel to each other, but they are actually not and theirform depends on the distance fiom the median plane. Equi-median plane.

(2) Parallel alley lies inside eqi-distance alley.
Employing sound intensity and binaural time differences asparameters, mathematical models were constructed toexplain the auditory alleys. By simulation, it wasconfirmed that the models successfully explain theexperimental results obtained in virtual environmentalsystem.

distance alley is a pair of perceived straight lines obtained
when two rows of luminous points on horizontal plane of

1. INTRODUCTION
For binocular visual space, it is well-known that horizontallines which appear straight on a subjective fiontoparallel
plane are not always straight in the physical sense, and thatthe form of the lines depends on the distance from theobserver. This phenomenon is known as Helmholtz's
horopter[l]. On the other hand, parallel alley and equidistance alley represent the concept of subjectively parallelto the median plane. In Fig.l, suppose that x axis is theintersection of the horizontal plane of the eye level and themedian plane of the observer, and y axis is the intersection

the eye level are arranged symmetrically to appear straightand laterally equi-distant from the median plane.
Physically, the two lines should be straight and equi-distant,
but they are actually not and their form depends on thedistance from the median plane[2]. Moreover, the twokinds of alleys should be physically the same, but they areperceptually not and usually the latter lies outside theformer in physical space as shown in Fig.l[3]. A similar
phenomenon is also known for tactile space[4][5].
A basic experiment for sound localization with distancewas conducted by the authors, and a similar phenomenon

II -436to Helmholz's horopter was found in binocular auditory
space. That is, it was confirmed that the forms of the lines

HMD - HEADPHONE

subjectively parallel to frontoparallel plane are not always
straight in the physical sense and they depend on thedistance between the sound source and the subject[6]. In
this experiment, sound intensity and binaural timedifferences are considered to be very important cues for thesubjects to judge the position of the sound sources. Then,these two cues were employed as parameters to formulate amathematical model for space perception to explain theauditory horopter, and the formulated model wasconfirmed by simulation to be able to explain the distanceFig.2 Virtual environmental display systemdependency of the forms of auditory horopter[7].
From the similarity of sensory modalities and theand served as the visual signals for the subject through aphenomenon of auditory horopter, phenomenon of parallel
alley and equi-distance alley in auditory similar to visual

head-mounted display attached to the subject. As for headmounted
display,an
see-throughhead-mounted

space and tactile space is expected. There has been nostudy, however. In order to measure the form of theperceptual parallel lines to the median plane in binaural
auditory space, several sound sources should be locatedsymmetrically with the median plane, and the sound fromeach sound source should be outputted in order. Assuming
such an experimental system setting up actually, the space
for the system must be large, and the physical existence ofnearer sound sources must occlude the sound from farther
sound sources to reach the ears of the subjects. Thesedifficulties may prevent to carry the experiments forauditory alley. Then, the authors has tried to realize thepsychophysical experimental system to measure the formsof parallel alley and equi-distance alley in auditory spaceby using virtual environmental display system which wasdisplay(STHMD), which superimposes the virtualenvironment on the real environment by beam splitter, wasemployed. A calibration method was developed andapplied to the visual parameters of the STHMD toeliminate errors caused by mechanical misalignments in
the HMD and by individual differences between actual anddesigned location of pupils of eyes, which ensured thecoincidence of the apparent distance in virtual environment
with that in real environment[9]. By using one of thecomputers, any auditory signals were synthesized,
outputted through DSP, and displayed through a headphone(Senheiser HD25SP) to both ears of the subject.

2.2 Experimental Method

The diagram of the virtual environmental display systemused in this study is shown in Fig.2. This system is theWe designed the experiments to measure parallel alley andequi-distance alley in auditory space using the method ofconstant stimuli[lO]. The diagrams of the experimentalsystems are shown in Fig.3.
A pair of two sound sources in the symmetrical position to
the median plane of the subject is called "pair of soundsources" hereinafter. Four virtual sound sources are
assumed, which consist of a standard pair of sound sources
and a comparing pair of sound sources, and the latter pairsame as the one used in the study to clarify the role ofsome cues for the sound distance localization by using
virtual environment[S].
In order to construct the virtual environment, twocomputers(IR1S Indigo) were used. By the computers,
computer graphics images for both eyes were generated

is located nearer to the subject than the former pair withvarious inner distances. As visual signals, images of two
sound sources which are in the symmetrical position of themedian plane on the horizontal plane of ear level of thesubject are displayed as the images of the standard pair of
the sound sources. The auditory signal given to the bothcomposed for study of sound distance perception[S].

2. THE MEASUREMENT OF PARALLELAND
EQUI-DISTANCEALLEYS IN AUDITORY SPACE
2.1 The System Setup

II -4437

standardpairofsoundsources

1

1i!

sound source
direction

II'comparing
pair ofsound(a) parallel alley

g-9I

standard
pairofsound1sourcesfji

comparingpair ofsound(b) equidistancealleyFig.3 Experimental system of parallel alley
and equi-distance alleyFig.4 Approximation of the difference
between sound source and both ears

ears of the subject is pseudo random noise, and its intensityand binaural time differences are controlled as follows:
Consider the sound stimuli which reaches both ears of thesubject from a sound source located at the left side of themedian plane. The following notation will be used:
P, : The intensity of the sound stimuli which reaches theleft ear directly from the objective sound sourced : The distance between the objective sound source andthe subject
Po : The standard sound intensity
d o : The distance between the subject and the sound sourcefrom which the sound stimuli reached directly the left
ear has the standard sound intensity.
As P, is anti-proportional to the distance from the soundsource[ll], the sound stimuli reaches the left ear of thesubject is

4 =-dod Po

right ear ( P , ) is executed as follows:

P, =-d0
d o +Ad " '(4)Also, the time difference between the sound stimuli
reached the right ear and the left ear ( A t ) is executed asfollows, where r is 8.75 cm and the sound velocity ( v ) is340dsec:
At = Ad / v = 257(6 +sin 0) [ p s e c ] .
(5)Assuming the sound stimuli from the left sound source, theauditory signal given to the right ear of the subject is
controlled to have a ratio and a binaural time differencewith the signal given to the left ear using the aboveequations(4),(5). Assuming the sound stimuli from theright sound source, the auditory signals are given to bothears oppositely.
The head of the subject is fixed and visual parameters of
STHMD are calibrated individually before the experiments.

On the other hand, the sound stimuli reaches the right ear
via the circumference of the head of the subject as shownin Fig. 4. Therefore, the difference between the distance

Each trial is composed as follows:(1) By assuming that the standard pair of the virtual soundfrom the sound source to the right ear and the distance tothe left ear ( A d ) is approximated as follows, where theradius of the head of the subject is r [12].

Ad = r e + r s i n B .

(2)

The angle 6 is executed by using the distance from thesound source ( y ) and the distance between the soundsource and the median plane ( x ).
0 = arcsin(x / y ) .
(3)
Therefore, the intensity of the sound stimuli reaches theIT -438

sources are located at a certain distance from both earsof the subject, parallel to the physical median plane,and symmetric with regard to the plane, the soundstimuli supposed from the right source of the standard
pair of the sound sources and the stimuli supposed fromthe left one are given to the subject in order repeatedly
with 500msec-duration and 500msec-interval. A subject
is instructed to move a pair of visual markers of thevirtual sound sources to the position where they areperceived by using a mouse cursor in order to fix theI'7II

Q

4IIIQI

Fig.5 Order of display and relations between two pairs ofvirtual sound sources in experiments of parallel alley't0parallel alley
eqi-distance alley0ob1020
Distance from frontoparallel plane [cm]Fig.7 Results of analysis
distance alley, the order of the virtual sound sources ofsound stimuli is assumed as right standard sound source,
left standard sound source, right comparing soundFig.6 Order of display and relations between two pairs ofvirtual sound sources in experiments of equi-distance alley
subjective position of the standard pair of the soundsources. After that, the position of the pair of visual
markers are consistent so that the subject can confirm
the position visually.
(2) Next, the sound stimuli assuming from the standard
pair and the comparing pair of the sound sources are
given with 500msec-duration and 2500rnsec-interval
repeatedly in order, and the subject is indicated to reply
the result of histher judgement by using two mousebuttons. In case of experiments to measure the form ofparallel alley, the order of the virtual sound sources ofsound stimuli is assumed as right standard sound source,
right comparing sound source, left standard soundsource, and left comparing sound source as shown in
Fig.S(a). The subject judges the relation between theline connecting the former two sound sources and theline connecting the latter two sound sources is whethertype@) or type(c) shown in Fig.5. On the other hand, incase of experiments to measure the form of the equi-II -439

source, and left comparing sound source as shown in
Fig.6(a). The subject judges whether the inner distanceof the latter two sound sources is more narrow or wider
than that of the former two sound sources as shown inFig.6@) and (c).
The assuming distance and inner distance of the standard
pair of the sound sources are 4&m, and 0.4 m
respectively. The assuming distance of the comparing pairof the sound sources is set as 2 f i or 4 m, and theirassuming inner distance is set as 0.0, 0.2,0.4,0.6, or 0.8 m.

2.3 The Experimental Results and Their Analysis
In one trial of the experiments to measure the form of theparallel alley or the equi-distance alley, the sound stimuli
were presented to a subject with randomly selected innerdistance of the comparing pair of the sound sources, where
every inner distance was assumed four times each. As theresults of each trial, psychometric functions were obtained,and probit analysis[l3] was applied to estimate theequivalence of the inner distance of the comparing pair of
the sound sources, which was perceived to be parallel to orequi-distant with the standard pair of the sound sources.
One example of the results of the analysis is shown in
Fig.7, where white dots show the experimental results of

the measurement of the parallel alley, and black dots show

the distance from the origin to both ears[ 151. Also, as forthe experimental results of the measurement of the equi-binaural time differences A t , the following equation isdistance alley.
From the results of the analysis, followings are obtained as

obtained from the equation of phase difference described in

the general tendencies:(1) Both parallel alley and equi-distance alley in auditory
space have different form with that of physicallyparallel line to the median plane.
(2) The auditory parallel alley lies inside the auditory equidistance alley in physical space.

At =[ 151:JW-J-O)2+y2V(8)where v is the sound velocity.
As the model for auditory parallel alley, the following
equation is assumed on the analogy of the model for
auditory horopter:

These tendencies are similar to the relations between
parallel alley and equi-distance alley in visual space or

H ( k ,At) = At + H , ( k ) ,(9)tactile space obtained in real environment.

3. THE MATHEMATICAL MODEL FOR SPACEPERCEFTION TO EXPLAIN AUDITORY ALLEYS
3.1 The Formulation of the Mathematical Model forSpace Perception
We, the authors, have constructed a neural network model
called ISLES model (Independent Scalar Learning Element
Summation Model) using biological information andconstraints, and the model have succeeded to explain thedistance dependency of the forms of horopter and alleys invisual space and in tactile space uniformly[l4]. Moreover,it was confirmed that the distance dependency of theauditory horopter can be successfully explained by theISLES model constructed by using sound intensity andbinaural time difference as parameters[7]. Therefore, it isattempted to construct the ISLES model which explains
auditory alleys by using sound intensity and binaural timedifference as parameters.
Based on [15], sound intensity k is defined as the productof the sound intensity reached to the right ear I, and thesound intensity reached to the left ear I, ,where binaural time differences ( A t ) is compensated by
the function of sound intensity( k ).
Assuming the learning area for auditory space perception isto the direction of distance from thefrom xmin to x,
median plane ( x ), and the distribution of learning points is
uniform concerning with x , the second term of the above
equation(9) is represented as follows:

prW , k &%ax

-Xmin

H,(k) = -(10)On the other hand, the ISLES model for auditory equidistance alley is assumed as follows:

H, (k, At) = H,,( k ) + H , (At) .

Assuming the learning area is from ym, to y,
to thedirection of distance from the frontoparallel plane
including both ears ( y ) and the distribution of learning
points is uniform concerning with y , the following
equations are obtained similar to the equation (10):S””.(Y7H,, (k)= Ymin

W Y
7k =I, ’I,.

(11)

(6)

(12)

Ymax -Ymin

Then, the origin of a coordinate system in the physicalspace is set to be the midpoint of both ears, x axis is set tobe the line including both ears, and y axis is set to be thehorizontal line perpendicular to the x axis. The soundintensity k of a certain position ( x , y ) in the coordinate
system is represented as follows:

3.2 The Examination of the Experimental Results
By executing equations (lo), (12), and (13) for various
learning areas, the forms of the equations (9) ( parallelarray ) and (11) (equi-distance alley) are examined. As thewhere I, is the intensity of the sound source, and a isresults of these simulations, it is confirmed that these forms

II -440

REFERENCES

6I

eqi-distancealleyparallel alley

0.1 0.2 0.3 0.4 0.5
Distance from frontoparallel plane [cm]Fig.8 Examples of the results of simulation

have similar tendencies with the experimental results ofauditory alleys by using virtual environment.
Examples are shown in Fig.8 ( x - =O[mm], x, = 300[mm] for parallel alley, and y,,=lOOO[rnm],18000[mm] for equi-distance alley ).

y,=4. THE CONCLUSION
By using virtual environmental display system,
experimental system to measure parallel alley and equidistance alley in auditory space is constructed.

As the results of the experiments using the system, thefollowings are obtained:(1) The forms of parallel alley and eqi-distance alley inauditory space are not always straight in physicalsense, and depend on the distance from thefrontoparallel plane.

(2) Parallel alley lies inside the eqi-distance alley.
These results are in good agreements with the phenomena
in binocular visual space and tactile space.
Employing sound intensity and binaural time differences asparameters, mathematical models were constructed toexplain the auditory alleys. The models were based on themathematical model using scalar learning rule proposed byT. Maeda et a1.[14]. By simulation, it was confirmed thatthe models successfully explain the experimental results
obtained by using virtual environmental system.

II -441[l] H. V. Helmholtz, “Treatise on Psychological Optics,” OpticalSociety of America, Vo1.3, 1925, pp.482.
[2] F. Hillebrand, “Theorie der scheinbaren &&e bei binocularenSehen,” Denkschriften der Wiener Akademie, MathematischNaturwissenschaft Klasse, Vo1.72, 1902, pp.255-307.
[3] T. Indow, “An Approach to Geometry of Visual Space withNo A Priori Mapping Functions: Multidimensional Mapping
According to Riemannian Metria,” Journal of Mathematical
Psychology, V01.26,1982, pp.204-236.
[4] E. Rubin, “Haptische Untersuchungen,” Acta Psychologica,
Vol.1, 1936, pp.285-380.
[5] W. Blumenfeld, “The relationship between the optical andhaptic construction of space,” Acta Psychologica 11, V01.2, 1937,
pp. 125-174.
[6] M. Ohkura, S. Tachi, “Sound localization in Distance andHoropter in Auditory Space”(in Japanese), Trans. of the Societyof Instrument and Control Engineers, Vo1.30, No.11, 1992,
pp. 1287-1292.
171 M. Ohkura, T. Maeda, S. Tachi, ‘%lathematical Model forSpace Perception to Explain Auditory Horopter Curves”(inJapanese), Trans. of the Society of Instrument and ControlEngineers, Vo1.34, No.10, 1998, pp.1472-1477.
[8] M. Ohkura, Y. Yanagida, S. Tachi, ‘I Sound Distance
Localization using Virtual Environment,” HCI International’95 inTokyo, Yokohama, Japan, July 1995, pp.485-490.
[9] M. Ohkura, T. Oishi, S . Tachi, “Calibration Method of Visual
Parameters of See-Through Head-Mounted Display to Adjust
Apparent Distance”(in Japanese), Journal of the Robotics Societyof Japan, Vo1.13, No.4, 1995, pp. 580-582.
[lo] J. P. Guilford, Psychometric Methods, New York, McGrawHill, 1954.
[ l l ] Y. Blauert, Spatial Hearing, MIT Press, 1983.
[12] P. H. Lindsay, D. A. Norman, Human Information
Processing -An Introduction to Psychology-, Academic Press,
1977.
[13] D. J. Finney, Probit Analysis, Cambridge University Press,London, 1971.
[14] T. Maeda, S. Tachi, E. Ohyama, “Sensory integration forspace perception based on scalar learning rule,” Proc. 1993 Int’l
Joint Conf. on Neural Networks, Nagoya, Japan, 1993, pp.13171320.
[15] S. Tachi, “Separation of a Periodic Signal in Noise byBispectrum Analysis”(in Japanese), Trans. of the Society ofInstrument and Control Engineers, Vo1.9, No.6,1973, pp.105-114.