Proceedings of the 14th International Conference on Auditory Display, Paris, France June 24 - 27, 2008

DEPTHROW: A PHYSICS-BASED AUDIO GAME
Stefano Papetti, Delphine Devallez, and Federico Fontana
University of Verona
Department of Computer Science,
Strada Le Grazie 15, 37134 Verona, Italy{stefano.papetti, delphine.devallez, federico.fontana}@univr.it

ABSTRACT
We present an interactive audio game designed around the auditory perception of distance and the use of physics-based models
for simulations of the dynamics, the sound source, and the acoustical environment. The game consists in throwing a virtual sounding object inside a virtual open-ended tube which is inclined. Thetask is to keep the object inside the tube, in other words the user
should adjust the initial velocity applied to the object such that thelatter does not fall out at the far end of the tube. The position of
the object inside the tube is provided by continuous audio feedback. User performance is closely related to its ability to perceive
the dynamic distance of the object in the virtual tube. Therefore,
this game represents a potential tool for exploring the usability ofauditory distance information in interaction design.
1. ARCHITECTURE OF THE INTERFACE
The game prototype has been developed as a complex Max/MSP1
patch where three interacting physics-based models enable to simulate, respectively, the dynamics of the environment (an inclinedplane), the sound source (a rolling ball), and to render the acoustical properties of the environment (an open-ended tube).
The action of throwing is performed using a Wii Remote2
which provides, among other things, a 3D accelerometer, a vibration actuator, several buttons, and Bluetooth connectivity.
The three physics-based models are connected in a top-downchain structure. Figure 1 shows the main Max/MSP patch. The
throwing force applied to the virtual ball is captured (gesture mapping) by means of the accelerometer and provides the input to thecomputation (integration) of the initial velocity with which the ballis getting thrown inside the inclined tube. Then the initial velocity
provides the input to the simulation of the inclined plane. The resulting velocity of the ball moving along the inclined plane drives
the real-time synthesis of a rolling sound, which constitutes theaudio input to a spatialization process (simulation of a tube), while
the distance currently covered controls the distance parameter ofthe spatialization model. The latter conveys auditory information
about the relative position of the sound source inside the virtualtube.
The vibration actuator is exploited for providing tactile feedback: for example when the object comes back to the user’s hand,or when the ball falls over.
1 http://www.cycling74.com/products/maxmsp

Figure 1: The main Max/MSP patch

2. PHYSICS-BASED MODELS
2.1. Inclined plane
The first (topmost) model is a simplified simulation of the dynamics of an inclined plane. It is currently possible to set the length,tilt-angle and friction of the plane. The model takes as input theinitial velocity of an object moving upward along the plane, andreturns its current displacement and velocity. As for the implementation, we wanted these quantities to be returned as signals (incontrast to controls) in order to allow high- (in fact, audio-) rate
control of the following models. For this reason, we discarded premade dynamics libraries such as pmpd 3 , and decided to develop
our own dynamics simulation as a Max/MSP external object.
2.2. Rolling sound model
The sound source is represented by a rolling sound model [1] which
was originally developed and implemented during the EU-funded
project SOb [2] (The Sounding Object), and further developed for
the EU-funded project CLOSED4 (Closing the Loop Of Sound
Evaluation and Design).
The rolling model combines different underlying components:
the heart of the model is a physics-based impact model [3], which

2 The handed wireless controller for Nintendo’s Wii console.

Thanks to Masayuki Akamatsu for his [aka.wiiremote] Max object (available at http://www.iamas.ac.jp/∼aka/max/).

ICAD08-1

3 http://drpichon.free.fr/pmpd/
4 http://closed.ircam.fr

Proceedings of the 14th International Conference on Auditory Display, Paris, France June 24 - 27, 2008

is driven by two pseudo-physical models that simulate respectively,
the geometry of the contacting surfaces (which controls the temporal pattern of micro-impacts), and the ball’s asymmetries (whichsuperimpose a modulation of the impact force).
As for the contact interaction between the rolling ball and theplane surface, only normal impacts are taken into account, discarding friction components (which are comparatively small). The
non-linear impact model adopted complies with the modular structure “resonator-interactor-resonator”, hence representing the impact interaction between two resonating objects. In this case theresonators are a ball (object1 ) and a tube (object2 ). They are modeled through the modal synthesis [4] technique, which describes
a vibrating object by means of its resonating modes (i.e., the frequency, decay time and modal weights of each mode). For simplicity, in our case, only object2 resonates while object1 is considered
as a point mass, and therefore does not have any resonating mode.
The shape of the plane surface is simulated by a band-pass filtered noise signal. This serves as input for a specifically designed
filter, which in turn calculates the trajectory of (the center of) a ballrolling on such surface. Given a surface, the trajectory depends on
the ball’s diameter. Again, for simplicity, the possible bouncing of
the ball is not taken into account.

source from its physical distance, and on a high response variability across listeners. In particular, distance perception is very poorwithout reverberation. However, a recent study [8] has shown that
distance accuracy improves with experience in reverberant environments. This result suggests that user performance may improvewith training. User testing should be carried out to validate this assumption.
Of interest for the present work are the studies by Yao andHayward [9] and Rath and Rocchesso [10]. In [9] the authors
studied the cues exploited by users to locate the position of a virtual rolling or sliding ball inside a tube, and showed the relatively
good ability of users to guess the length of the tube just by tilting it and perceiving (from auditory and/or haptic cues) the virtual
object’s dynamics. A physics-based model of rolling sound wasalso used in [10] as a continuous auditory feedback for balancing
a ball along a tiltable track. To evaluate the interface, subjects had
to move a virtual ball until a specific position on the track. Even
without any visual display, all subjects managed to solve the task,
which means that people were able to locate the virtual ball withthe only auditory feedback.

2.3. Sound spatialization[1] M. Rath, “An expressive real-time sound model of rolling,”
in Proc. of the Int. Conference on Digital Audio Effects(DAFx-03), London, UK, September 2003.

In recent years, research on the Digital Waveguide Mesh (DWM)
has enabled to use discrete-time simulations of acoustic propagation as a model for real-world acoustical spaces [5]. Fontana
and Rocchesso [6] have recently demonstrated the effectivenessof a DWM modeling a rectangular parallelepiped to provide auditory distance cues. Our sound spatialization model consists ofa two-dimensional rectangular DWM. Each internal junction isconnected to four other junctions via waveguides, thus providing acoustic wave transmission. Besides, reflections at the meshboundaries are modeled by Digital Waveguide Filters, whose coefficients have been tuned to model specific reflective properties
of surfaces. This structure, modeling a tubular environment in 2D,
allows to render dynamic variations of the sound source position,and consequently provides a tool for interactive manipulation ofsound sources along the depth dimension.
3. GOALS AND ISSUES
Besides investigating whether the game is entertaining, user performance allows to evaluate the playability of the game which is
the result of a perception-compliant mapping of user gesture (theact of throwing the virtual object), i.e. the mapping of the acceleration signals sensed by the Wii Remote onto the control parameters of the dynamics environment (inclined plane) and the rollingmodel. This adequate mapping is achieved through the iterative
development and informal evaluation of the prototype.
Another issue of this gaming application is the perception of
the far end of the virtual tube to avoid the object falling over.
Human ability to perceive distance is relatively weak in comparison with the perception of direction [7]. The most problematic
issue during listening experiments on distance perception is thedifficulty of humans to separate the distance cues from the spectral characteristics of the sound source itself. Therefore, users
may need some a priori information about the sound source, ora preliminary phase of training. Moreover, many studies agree
on the systematic deviation of the perceived distance of the sound

4. REFERENCES[2] D. Rocchesso, R. Bresin, and M. Fernström, “Sounding objects,” IEEE MultiMedia, vol. 10, no. 2, pp. 42–52, 2003.
[3] D. W. Marhefka and D. E. Orin, “A compliant contact modelwith nonlinear damping for simulation of robotic systems,”
IEEE trans. on systems, man, and cybernetics - Part A: systems and humans, vol. 29, no. 6, pp. 566–572, November
1999.
[4] J. M. Adrien, “The missing link: Modal synthesis,” in Representations of Musical Signals, G. De Poli, A. Piccialli, andC. Roads, Eds., pp. 269–297. MIT Press, Cambridge, MA,
1991.
[5] D. T. Murphy and M. Beeson, “The KW-boundary hybrid
digital waveguide mesh for room acoustics applications,”
IEEE transactions on audio, speech and language processing, vol. 15(2), pp. 552–564, February 2007.
[6] F. Fontana and D. Rocchesso, “A physics-based approach tothe presentation of acoustic depth,” in Proc. of the Int. Conference on Auditory Display, Boston, MA, USA, July 2003,
pp. 79–82.
[7] P. Zahorik, D.S. Brungart, and A.W. Bronkhorst, “Auditory distance perception in humans: A summary of past andpresent research,” Acta Acustica united with Acustica, vol.
91, pp. 409–420, 2005.
[8] B.G. Shinn-Cunningham, “Learning reverberation: Considerations for spatial auditory displays,” in Proc. of the 2000
Int. Conference on Auditory Display, Atlanta, GA, 2000.
[9] H.-Y. Yao and V. Hayward, “An experiment on length perception with a virtual rolling stone,” in Proc. of the EuroHaptics Int. Conf., Paris, France, 2006, pp. 275–278.
[10] M. Rath and D. Rocchesso, “Continuous sonic feedback
from a rolling ball,” IEEE MultiMedia, vol. 12, no. 2, pp.
60–69, 2005.

ICAD08-2