PROCEEDINGS of the23rd International Congress on Acoustics
9 to 13 September 2019 in Aachen, Germany

A real-time virtual reality building acoustic auralization frameworkfor psychoacoustic experiments with contextual and interactivefeatures
Anne Heimes; Imran Muhammad; Michael Vorländer
Institute of Technical Acoustics, RWTH Aachen University, Germany

ABSTRACTThe auralization of sound insulation between adjoining rooms in virtual reality (VR) environments describes
the sound transmission between enclosures, where sound insulation prediction methods are applied for ahigh-quality auditory stimulus. In this context, a real-time building acoustic auralization framework in 3D
audio-visual technology is developed to introduce more realism and contextual features into psychoacousticexperiments. This paper describes the design and implementation of this framework that is a vital part of 3Dimmersive sound rendering VR-Systems. This framework relies on up-to-date knowledge of building
acoustic prediction models and enables a perceptually accurate airborne sound insulation auralization insimulated environments, including important effects such as sound transmission, source directivities, andsource / receiver room acoustics. Despite this realistic sound insulation, rendering between adjoining rooms,
not only the indoor sources are supported at runtime, but also outdoor moving sources such as vehicles. Allfeatures are evaluated by investigating both, the overall accuracy of the building acoustics ﬁlters and theperformance of implemented algorithms. This framework was used in psychoacoustic experiments bypresenting a virtual building scenario, for an office to office situation, to the user with Head-Mounted-Display.
Keywords: Building Acoustic, Auralization, Psychoacoustic, Perception

1. INTRODUCTION
Sound is crucial for creating a convincing virtual reality (VR) experience, because auditory cues
are relevant for the sensation of being present in an actual, physical space. It contributes to the user’s
sense of immersion. Building acoustics deals with sound transmission between the adjacent rooms
connected through structural elements of the buildings (1). This paper deals with the creation of an
acoustic virtual reality (AVR) experience of building acoustic auralization from a technical point ofview. Creating this virtual world and all the objects in it (i.e. audio and visual) is implemented in areal-time VR framework, which can be used for audio-visual demonstrations and, furthermore, forpsychoacoustic experiments with contextual and interactive features (2). We introduce and discuss thebasics of 3D graphics rendering tools which are vital for our building acoustics implementation tocreate an environment and the techniques such as audio rendering, real time convolution, filter design
and building acoustic plugin that make our objects appear realistic. Above all, the focus is to pay more
attention to the requirements of virtual reality, including performance issues, making sure that theaudio filters run fast enough in VR. We make use of the professional game and VR engine, Unity 3D.
Unity is one of the most demanding game engines and is a relatively easy to use, however, fullyfeatured.
The recent up-to-date work in acoustic virtual reality is available for room acoustical simulations
and auralization, implemented in VR environments. Up to our knowledge, building acousticauralization has not been implemented in VR environments. To achieve this, three levels ofimplementation are associated with a universal platform for such kind of advanced VR environments(3). First, the sound insulation prediction models and building acoustic filter design strategie s. Second
the filters rendering techniques: such as sound insulation, convolution, binaural 3D real-time
interactive audio-visual VR technology. The general flow diagram of our technical framework i sshown in Figure 1. The focus in this paper is on the third part; the implementation of building acoustics
in VR and creating interactive scenes, the performance of rendering engine, and the interactive
psychoacoustic experimental environment by creating an example of work performance in a short-6417

term memory experiment.

Figure 1 - Building acoustic auralization process and VR implementation design

2. VIRTUAL REALITY FRAMEWORKIn building acoustics, the sound insulation prediction was first introduced by Gerretsen (4), basedon classical sound insulation theory (e.g. Vigran (5)). Subsequently, the first application ofauralization of airborne sound insulation was introduced by Vorländer (6) and Thaden (7). They
proposed sound insulation model based on ISO 12354-1 and designed an auralization framework bydeveloping sound insulation filters. These filters were used to calculate airborne sound transmission
paths from source to receiver placed in simple adjacent rooms of a building and made use of binauraltechnology for reproduction. Later on, certain improvements were made by (8-9) in these approaches,such as; source position, its directivity and incident sound intensity on the source room elements (i.e.
source room walls), which was calculated based on room acoustics of the source room (reverberationof the source room from ISO standards), on contrast to common practice of assuming a purely diffusesound field in the source room walls for all transmission paths from source to receiver. Further,improvements were proposed by (3), such as including the synthesized room impulse response basedon one-third octave band values of the reverberation time and representing each radiating element (i.e.
receiver room walls) by a set of evenly distributed point sources (i.e. secondary sources) on its
surfaces.
The latest available auralization framework is now integrated into virtual reality. This auralization
framework might be used in listening experiments and would be able to allow the test subjects toperform any task of daily life’s work or learning under conditions of usual behaviour and movement.
Therefore, it is intended to create more ecologically valid noise perception tests in real-time virtual
reality environment. To achieve this, we make use of the professional game and VR en gine software
Unity (11) as our virtual environment renderer. Unity is one of the most demanding game engine andvisual rendering software, which is relatively easy to use, compared to other available game engines.
However, it contains all necessary tools that are needed for our purpose. It also supports thearchitectural constructions importing feature, directly from other third-party commercial software
such as SketchUp-2017 (12), AutoCAD etc. The building of Institute of Technical Acoustics (ITA) ofRWTH Aachen University is selected as a working site, as shown in Figure 2.
The choice of a suitable 3D architectural modelling software is necessary for creating a virtual
building scene. SketchUp-2017 it is used to design the ITA building in a CAD model, which isimported into Unity, where the rest of the graphical features were integrated into the scene, such as6418

the audio source (i.e. noise source), the listener, a camera view (i.e. to visualize the scene) and theinteractive scene (i.e. a scene for listening tests and psychoacoustic evaluation). Once the building
model is designed, the rendering process can start. For graphical rendering there exist many thirdparty plugins such as Oculus Rift Unity integration plugin, Steam VR plugin etc. with basic audio
rendering features such as reverberation and simple room acoustic mixers e tc. These auralization
features, however, are supporting only free field conditions or generic reverberation, which are notappropriate for specific auralization purposes. Therefore, the sound insulation auralization model
described above is implemented for audio rendering and auralization into Unity by using its scripting
features (e.g. Unity supports java and C# languages for scripting and customizing the scenes).

Figure 2 – ITA building: indoor/outdoor scenes in Unity
2.1 Implementation of Building Acoustic Filters
A software framework is described which includes the selection of materials to room surfaces,
detection of source and receivers’ positions, selection of structural properties of the building elements,
the input audio files and sound card channels for auralization. The complete processing chain is shownin Figure 3. Some of the important plugin and features for the framework are listed below.
••••

Building acoustic model (sound insulation prediction model)
Building acoustic filters (sound transmission)Architecture geometry manipulation
Ray casting6419

••
•
•
•
•
•

Real-time source receiver detections
HRTF database (ITA dummy head recordings)
FFT and iFFT libraries (open source from MIT USA)Interpolation
Convolution
Audio filter rendering
Visual rendering

Figure 3 - Auralization Framework processing chain in Unity
2.2 Virtual Geometry Manipulation
A plugin was developed that lists building elements with the geometric dimension and material
properties, by using ray casting approach. The material properties can be pre-defined or interactively
assigned to elements through graphical user interface in Unity as show in Figure 4. This processprovides the geometric data for calculation of sound insulation metrics. This process is performed
before the auralization process starts and, hence, is required only for initialization of the mainframework.

Figure 4 - Implementation: Assigning material parameters to building elements and calculations forgeometric input data6420

2.3 Sound Insulation Metrics and Transfer Functions
The second process detects sound source and receiver positions, orientations in the dwelling (i.e.
source and receiving rooms). The sound insulation metrics (i.e. the input data for filters) for eachelement of source and receiving rooms are calculated, and subsequently both the rooms and building
acoustic transfer functions for each path (direct as well as flanking) are computed in terms oftransmission coefficients or the normalized level differences (‫ܦ‬௡் ) spectra in one-third octave bands
in the frequency range of 50 Hz to 5000 Hz. These spectra are interpolated by using a 4097 points
cubic spline interpolation to get suitable audio filter with an appropriate frequency resolution for theselected adjacent rooms. Afterwards, these spectra are used to calculate the final transfer functions
from the sound source to the secondary sources (patches, for reference (3)) by adding the different
flanking transmission paths.
2.4 Real-Time Rendering and Auralization
Once the sound insulation filters and HRTFs data are available, the real -time convolution process
for these filters is performed. Inside the receiving room, each secondary source (SS) of each element(i.e. wall) are considered as independent radiating elements.

Figure 5 - Real-time auralization and listening test scene
The directional cues for binaural signals from these radiating elements relative to the listener are
considered by applying the HRTF regarding the incident angles to the listener’s head. An inverse
Fourier transform (iFFT) and summation is processed to get final binaural signal in time domain forauralization. The secondary sources are located at different distances relative to the listener. Therefore,
they are amplitude-weighted and delayed according to the dimensions of the receiving room and theposition of the listener. The final step is to convolve the binaural signal with sound or speech signal
for auralization. Normally, the sound file has 24 bit and 44.1 kHz sampling rate. This operation isperformed by FFT/iFFT (including overlap add method) by dividing the source signal into frames of
256 samples, transforming to frequency domain, multiplying with binaural filters and converting backto time domain. In this way, the whole rendering chain works in real time and with this, the interactive
scenarios can be created in which the listener can freely move (turn the head or walk around) in thereceiving room.

3. PERFORMANCE
The performance of the framework is verified for initialization and the real-time processes.
Initialization include geometry manipulation, filter construction and insulation metric calculation
along with room impulse response synthesis for source and receiving rooms. Real-time processing
involves convolution by FFT and iFFT, source and receiver position updates (including headmovements) and binaural filters. Figure 6 shows the time consumed by each major process. Allcomputations are performed on desktop personal computer featuring an Intel Core i7 -7700 CPU @3.60 GHz multi-core with 16 GB RAM, Windows 10 (64-bit) operating system.

6421

Figure 6 - Performance of the building acoustics VR plugin

4. AUDITORY-VISUAL ENVIRONMENTS
In this section, it is discussed the possibility to use this framework as a platform for psychologicaland psychoacoustic test environments such as cognitive performance tests under different building
acoustic conditions. With this, novel experiments on noise effects, on evaluation of noise and speech
effects on humans in built-up environments can be performed which go beyond the traditional
listening experiments. As an example, the design of a test paradigm is explained for real-time
interactive insulation auralization and for perceptual evaluation of noise stimuli with more realism
and more contextual features.
4.1 Example VR Scene
To conduct a listening experiment, a test paradigm is required which explains the tasks for theparticipants. The scene was designed using the VR framework with a research objective of studying
the cognitive performance at presence of background noise in a building. The specific question was if
frequency-specific sound insulation and speech signals transmitted through the wall creates impact on
the cognitive performance, in particular in comparison of intelligible and non-intelligible speech. In
the scene, the participants were presented a virtual reality environment, where they are sitting in an
office-like receiving room, performing a cognitive task of verbal serial recall on the screen and hence
evaluating the verbal short-term memory capacity. Their answers were recorded under different
background stimuli of “irrelevant speech”, originating from the neighbouring office (source room)(10). The test is designed to study the impact of meaning of background sound (intelligible speech vs.
non-intelligible speech) on the cognitive performance, the sound insulating components of thebuilding are selected in the way that the final binaural signal at the listener ’s ear presents bad andgood sound insulation. To present the scene to the participants for this in VR, the head-mounted
display HTC Vive is used in Virtual Reality Lab. The headset uses "room scale" tracking technology,
allowing the user to move in 3D space and use motion-tracked handheld controllers to interact withthe environment. The results are going to be published soon.
4.2 Software Features
In order to present different background building acoustical condition s to the participants, severalfeatures can be switched on or off in a particulate test design. Activation and deactivation of room
acoustics (i.e. reverberation etc.), source/receiver characteristics (i.e. position and/or orientation anddirectivity etc.) and most importantly the choice of the methods for sound insulation filter construction.
These options can be selected during the configuration and initialization of the VR environment.

5. SUMMARY
A real-time VR auralization framework is introduced that relies on present up-to-date knowledgeof building acoustic simulation techniques. It is implemented in Unity graphics rendering and game6422

engine software. This framework enables several important features along with sound insulation
auralization for adjacent rooms. It is intended to use this platform for listening experiments where it
allows the test subjects to perform any task of daily life’s work or learning under conditions of usual
behaviour and movement, creating more realistic noise perception tests in real-time virtual reality
environments. Furthermore, in the next version of this framework, sound insulation auralization
against outdoor sound source will be incorporated.

ACKNOWLEDGEMENTS
This work was funded by HEAD-Genuit Foundation under the Project ID: <P-17/4-W>.

REFERENCES
1. Vorländer M, Schröder D, Pelzer S, Wefers F. Virtual reality for architectural acoustics. Journal ofBuilding Performance Simulation. (2014) 8(1): 15-25.
2. Vorländer M. Auralization: fundamentals of acoustics, modelling, simulation, algorithms and acoustic
virtual reality. Springer Science & Business Media; (2007).
3. Imran, M., Heimes, A. and Vorländer, M., Real-time building acoustics noise auralization and evaluationof human cognitive performance in virtual reality, In proceeding of DAGA-2019, Rostock, Germany,(2019): 551-554.
4. Gerretsen, E., Calculation of the sound transmission between dwellings by partitions and flanking
structures. Applied Acoustics 12(6) (1979): 413-433.
5. Vigran, T. E., Predicting the sound reduction index of finite size specimen by a simplified spatial
windowing technique. J. Sound Vib. 325(3) (2009): 507-512.
6. Vorländer, M., Thaden, R., Auralization of airborne sound insulation in buildings. Acta Acustica unitedwith Acustica 86 (1) (2000): 70-76.
7. Thaden R. Auralisation in building acoustics. Ph.D. Dissertation, Institute of Technical Acoustics, RWTH
Aachen University (2005).
8. Vorländer, M. and Imran, M., Real-Time Auralization of Sound Insulation. In Proc. Inter Noise (2018).
9. Imran, M., Heimes A., Vorländer, M., Auralization of airborne sound transmission and framework forsound insulation filter rendering. In Proc. Euronoise 2018, Crete Greece, (2018): 283-288.
10. Schlittmeier, S.J., Hellbrück J., Thaden R. and Vorländer M. The impact of background speech varying
in intelligibility: Effects on cognitive performance and perceived disturbance. Ergonomics. 2008 (5):719-736.
11. https://unity.com
12. https://www.sketchup.com

6423