Sonoliards: Rendering Audible Sound Spots
by Reflecting the Ultrasound Beams
Naoya Muramatsu*
sh.mn.nat@gmail.com
Ooi Chun Wei*
made3102@gmail.com

Kazuki Ohshima*s1613108@u.tsukuba.ac.jp
Yuta Sato*s.yutassie@gmail.com

Ryota Kawamura *s1721663@s.tsukuba.ac.jp
Yoichi Ochiai *wizard@slis.tsukuba.ac.jp*University of Tsukuba (1-1-1 Tennodai, Tsukuba, Ibaraki, Japan)

Figure 1: (a) Our system set up. (b) Rendering of simulation result. (c) Parametric speaker with two-axis servo motors.
AUTHOR KEYWORDS

Ultrasound beams; Parametric Speaker; Acoustic Field
Generation; Acoustic Simulation; Ray Tracing Method.
ACM Classification Keywords

H.5.1. Information interfaces and presentation (e.g., HCI):
Multimedia Information Systems.
INTRODUCTION

We propose a real-time indoor dynamic acoustic field
generation system of a spot audio towards a particularperson.
Personalized sound propagation is becoming important in
the ubiquitous computing context. Many audio interaction
systems that are appearing in mobile interactions andpersonal recognition by audio recognition techniques arenowadays increasing its accuracy. In these fields,
continuous sound propagation for individuals is highlydesired. However, sound waves have low directivity thus
widely received by unintentional hearers. To solve thisPermission to make digital or hard copies of part or all of this work for personalor classroom use is granted without fee provided that copies are not made or
distributed for profit or commercial advantage and that copies bear this noticeand the full citation on the first page. Copyrights for third-party components ofthis work must be honored. For all other uses, contact the owner/author(s).
Copyright is held by the author/owner(s).
UIST’17 Adjunct, October 22–25, 2017, Québec City, QC, Canada.

© 2017 Copyright is held by the owner/author(s).
ACM ISBN 978-1-4503-5419-6/17/10.
DOI: http://dx.doi.org/10.1145/3131785.3131807

problem, continuous efforts have been made, such as
audible-sound directions loudspeakers and ultrasound-based
super directional loudspeakers.
Studies using sound propagation are found in various
research fields. Tung et al. estimates the force applied to thesmartphone via structure-borne sound propagation using itsbuilt-in sensor instead of specialized sensors used inconventional methods [1]. This enables the application of
expressive interfaces. Also, Ochiai et al. proposed a methodof spatial audio rendering using ultrasound and realized ahigher audience selectivity [2]. Conventional 3D soundtechnology based on head related transfer function (HRTF)
could not reproduce realistic sounds in the virtual space, but
the problem can be solved by sound rendering. The soundrendering pipeline shown in the study consists of soundsynthesis, sound propagation, and sound generation. Thismakes it possible to reproduce realistic sounds such assounds occurred when people drop something. Nowadays
various types of ultrasound-based super directional
loudspeakers are available on the market and is used inmany places such as museum, platforms, galleries, andthese places need to avoid sound contaminations.
Hence, spot audio techniques have been explored bygenerating the ultrasound beams toward the target person ina certain area. However, everyone in the area can hear thesound. To solve this problem, our system generates anacoustic field dynamically which can only be heard by thetarget person using motion capture system and parametricspeaker with two-axis servo motors. Furthermore, with theproposed system, we accomplished real-time tracking andsound rendering up to 15 fps, allowing practicalapplications which rely heavily on continuous soundpropagation to be deployed.
SYSTEM OVERVIEW

Figure 2. System overview of Sonoliards.

The program uses the ray tracing method to calculate thedirection of the loudspeaker so that the sound can only beheard by the target person. The algorithm is based on theassumption that the sound as a form of light particles in thecalculation of ray tracing method, the number of soundparticles is determined by assuming at least one soundparticles is present in the volume of a person headregardless of his position in the simulated space (Figure1(b)) after the sound particles are being emitted. In
simulation, the propagation of the emitted sound beam tothe surroundings from the source is calculated, and thecollision of sound beam on the target person head isdetected. At this time, we determine the direction of theloudspeaker to the initial direction of the sound beam which
has the least reflection number and the shortest traveling
distance to the target or the least likely to collide withpeople other than the target person. Real-time tracking andrendering of acoustic field is realized by continuously
repeating the initialization of sound beam, target head
position tracking, determination of the direction of soundbeam in simulation, and shifts in direction of theloudspeaker accordingly.
When we run the program using an indoor 3D model where
the number of wall surfaces is about 20, the operation speedof this system is about 15 fps. In the experiments, weallowed the subjects to freely roam through the room andgot the same impression that most of them felt as if thesound followed them in a real time.
APPLICATION

Figure 3. Structure of two axis servo direction controller
we used.
The purpose of this system is to create a highly directionalsound reproduced with a parametric-speaker and togenerate an acoustic field to reach only a specific person inreal time by using the reflection indoors. In order to realize
this, it is crucial to acquire the position of the person
relative to the loudspeaker in the room then calculation isdone to determine the direction of the loudspeaker. Thissystem consists of three parts: motion capture, two-axis
loudspeakers with servo control, and algorithm for thecalculation of the loudspeaker direction (see Figure 1(a) andFigure 2).
Motion capture is used to obtain the position of the head ofa person indoors. This system uses six motion capture
cameras. In order to acquire the position of the head, theuser of this system needs to attach a marker of motion
capture on his head via a hat or the like.
The direction of the loudspeaker is controlled by two axis
servo motors, and its movable range is the upper half of thesphere (see Figure 1(c) and Figure 3). It moves according tothe angle sent from the program.

The main feature of this system is that you can use thereflection to avoid obstacles and deliver the sound withhigh accuracy, so you can play the sounds that only reachspecific people.
For example, this can be applied to raise individual
attention during the examination. Also, it is possible togenerate an acoustic field reaching a position where soundcannot be delivered by a conventional speaker system, suchas a specific position on the opposite side of the wall. Thiseliminates the dependency relation between the spacedesign and the speaker position, and enables interior spacedesign with higher degree of freedom. Such a high degreeof freedom in space design is thought to be very useful forthe design of amusement facilities and so on.
Also, by combining this system with multiple speakers, it ispossible to deliver different sounds to different people atthe same time. We expect to realize the system thatimplements different guidance and warnings for eachperson.
It is not implemented yet now, but when you deliver thesound to the target person, it can be used for personalized
guidance or even in horror amusement facilities where it
can designate the direction of where you deliver the sound.

REFERENCES

1.Tung, Y.-C., and Shin, K. G. Expansion of human-phoneinterface by sensing structure-borne sound propagation. In
Proceedings of the 14th Annual International Conference
on Mobile Systems, Applications, and Services, MobiSys’16, ACM (New York, NY, USA, 2016), 277–289.

2.Ochiai, Y., Hoshi, T., and Suzuki, I. Holographic
whisper: Rendering audible sound spots in threedimensional space by focusing ultrasonic waves. In
Proceedings of the 2017 CHI Conference on HumanFactors in Computing Systems, CHI ’17, ACM (NewYork, NY, USA, 2017), 4314–4325.