Spatial release from masking assessment in virtual reality for bilateralcochlear implants users
Lorenzo Picinali(1) , Marina Salorio Corbetto(2) , Deborah Vickers(3)(1) Imperial College London, UK, l.picinali@imperial.ac.uk(2) University of Cambridge, UK, ms878@cam.ac.uk(3) University of Cambridge, UK, dav1000@cam.ac.uk

Abstract
In addition to enhanced sound localisation abilities, one of the potential benefits of having bilateral cochlear
implants is the improvement of speech-in-noise perception, as bilateral auditory inputs potentially allow thespatial separation of speech and noise sources. This process is known as spatial release from masking (SRM).
When assisting bilateral cochlear implant users, it is essential to assess binaural hearing performance in a timeefficient manner, trying to maintain some of the real-world complexities, e.g. multiple sources and reverberation.
Traditional tests of SRM and/or localization are often time consuming, and typically assess these in unrealistic settings, with only one controlled stimulus to attend to. With this in mind, Bizley and colleagues (2015)
developed the Spatial Speech in Noise test (SSiN), which allows for the simultaneous evaluation of speech discrimination for various locations, SRM, and relative localisation measures using speech in a background babble.
Within the BEARs (Both EARs) project, we have recently developed a Virtual Reality-based binaural training
applications suite aimed at improving spatial hearing, speech-in-noise perception and ease of listening for bilateral cochlear implantees. In order to assess the impact of training on SRM and localisation without requiring
custom spaces and expensive loudspeakers arrays, we have developed a 3D headphones-based implementationof the SSiN test. This article includes details of this implementation.
Keywords: Cochlear implants, speech, intelligibility, localisation

1INTRODUCTION

Prior to 2009, unilateral cochlear implants (CIs) were the recommended intervention for children meeting CI
criteria in the UK. However, as the benefits of two implants as opposed to one were widely documented, thisled to the National Institute of Health and Care Excellence (NICE [1]) recommending bilateral CIs for all
children within criteria. Approximately 450 children receive bilateral cochlear implants annually in the UK, andthere are currently 3230 bilateral cochlear implants users in total [2]. Combining the information from bilateralCIs across ears is expected to reduce listening effort in real-world situations, improve localisation abilities,
and improve incidental listening (over-hearing conversation) to maximise speech and language acquisition andlearning. Even more importantly, one of the potential benefits of having bilateral CIs is the improvement ofspeech-in-noise perception, specifically the advantages gained by being able to spatially separate the speech andnoise sources. This process is known as spatial release from masking (SRM).
Although provision of bilateral CIs is now routine in developed countries [3], and evidence suggests that language development, spatial listening skills and speech-in-noise understanding are better than for unilateral implantees, these skills still remain far below those of normally-hearing children [4] [5] [6]. Furthermore, there
are no accepted protocols for the fitting and rehabilitation of bilateral cochlear-implant users. This is likely tocontribute to the performance gap between children with bilateral CIs and their normal-hearing peers in terms ofbinaural listening skills. Even when some of the barriers for binaural performance arise from device limitations,the evidence suggests that abilities such as sound localisation can develop further if children are provided withappropriate input and training [5].
The BEARs (Both EARs) project uses a participatory design approach, therefore engaging with the various

7647

stakeholders from the early-design stages [7], to develop a series of VR-based applications for training various
aspects of binaural listening, including speech understanding, sound source localisation and music listening.
Early prototypes of the BEARs applications suite are currently being evaluated, and a video containing a brief
description of the various applications has been released [8].
BEARs specifically targets teenage bilateral cochlear implantees. A series of tests and evaluations will be carriedout on a regular basis while using the BEARs applications in order to monitor the effects of training. Oneof such evaluations is the Spatial Speech in Noise (SSiN) test [9], which allows to assess both SRM andsound sources localisation within a single procedure. In order to be able to regularly perform this test, but
without the need of custom spaces and expensive loudspeakers arrays, we have developed a 3D headphonesbased implementation.

2IMPLEMENTATION

2.1 The Spatial Speech in Noise test (SSiN)The SSiN test was designed to simultaneously assess speech discrimination and relative localisation. The test
uses a dual-task paradigm to increase the cognitive load of the speech discrimination task, and better reflect thedifficulties faced when communication in noisy environments in everyday life. In the current implementation
participants are assessed in a sound-attenuated booth containing an array of loudspeakers positioned on thehorizontal plane between 90◦ and -90◦ of Azimuth with respect to the participant. For normal-hearing participants
the spacing between loudspeakers is normally set to 15◦ while for hearing impaired listeners a 30◦ spacing ismost appropriate. The spacing is to ensure that the task is challenging but not too difficult. In each trial, theparticipant is presented with a reference word from a loudspeaker selected at random, followed immediately
by a target word coming from an adjacent loudspeaker. The target stimuli are words from the Chear Auditory
Perception Test (CAPT [10]), spoken by a female talker. The words are arranged into confusion groups madeup of 4 words each, and have been selected to assess a specific type of discrimination (complex vowel, simple
vowel, initial consonant, or final consonant). Sixteen words are used (four groups of four words), presenting thewords in a four-option closed format paradigm. The participant is asked to select from a screen the two words
in the right temporal order and the location of the target word with respect to the reference, where the options
are right or left. The words are presented in a background noise of uncorrelated 16-talker babble coming fromfour locations in either extremes of the loudspeaker array. The signal-to-noise ratio (SNR) at which the test is
carried out is optimised for each participant in order to avoid ceiling and floor effects. The outcomes of thetest are a Word Accuracy Score, which is calculated for both reference and target words across all locations,and a Localization Performance Score, which is defined as the number of correct localization judgements (rightor left).
2.2 Binaural spatialisation and the 3D Tune-In Toolkit
Binaural spatialisation is a technique which allows the creation of three-dimensional soundscapes through asimple pair of headphones. This technique is widely used in interactive audio and Virtual Reality (VR) applications. Considering its simple requirements in terms of playback device (i.e. a standard pair of headphones),
and the virtually unlimited capabilities in terms of 3D soundscape complexity (i.e. number of sources, positionand distance of each of them, etc.) and environmental simulation (e.g. large halls, small studios, etc.), binauralspatialisation could have a major impact when applied to audiological testing.
The 3D Tune-In (3DTI) Toolkit is an open-source standard C++ library which includes, among other functionalities, a binaural spatialiser. It was developed within the 3D Tune-In [11] EU-funded project [12], and a
description of its functionalities can be found in [13]. A journal publication with a detailed description of thebinaural spatialisation implementation is available open access [14]. The source code of the 3DTI Toolkit ispublicly availabkle [15], and various releases currently exist, including a Test Application with a user-friendly
graphical interface, which has been used for the headphones-based implementation of the SSiN test.

7648

2.3 Integration of the 3DTI Toolkit and the SSiN test
Using the visual programming language MaxMSP [16] and the 3DTI Toolkit Test Application, the SSiN test
procedures have been implemented within an acoustic VR environment which, thanks to binaural spatialisation,
can be rendered with a simple pair of headphones. The user interface, which integrates all the procedures andis implemented in MaxMSP, communicates with the 3DTI Toolkit via Open Sound Control (OSC [17]). Finally,a smartphone-based head tracker sends information about the orientation of listener’s head to the 3DTI Toolkit
using, also in this case, the OSC protocol.

3FUTURE WORKS

The binaural implementation of the SSiN test is currently being evaluated with normal hearing and hearing impaired subjects, and will soon be assessed also on cochlear implantees. Future works will include the implementation of the whole testing platform in one single easy-to-use application (desktop, mobile and/or web-based),
which will include both the user interface and the sound processing modules.

References[1] https://www.nice.org.uk/ - last accessed in May 2019[2] https://www.bcig.org.uk/annual-uk-update/ - last accessed in May 2019[3] Peters, B.R.; Wyss, J.; Manrique, M. Worldwide trends in bilateral cochlear implantation. The laryngoscope.
2010 May;120(S2):S17-44.
[4] Grieco-Calub, T.M.; Litovsky, R.Y. Sound localization skills in children who use bilateral cochlear implants
and in children with normal acoustic hearing. Ear and hearing. 2010 Oct;31(5):645.
[5] Zheng, Y.; Godar, S.P.; Litovsky, R.Y. Development of sound localization strategies in children with bilateralcochlear implants. PloS one. 2015 Aug 19;10(8):e0135790.
[6] Boons, T.; Brokx, J.P.; Frijns, J.H.; Peeraer, L.; Philips, B.; Vermeulen, A.; Wouters, J.; Van Wieringen,A. Effect of pediatric bilateral cochlear implantation on language development. Archives of pediatrics andadolescent medicine. 2012 Jan 2;166(1):28-34.
[7] https://www.youtube.com/watch?v=UjWuMBrUOHE - last accessed in May 2019[8] https://www.youtube.com/watch?v=cvPReMtBGlk&t=100s - last accessed in May 2019[9] Bizley, J. K.; Elliott, N.; Wood, K. C.; Vickers, D. A. Simultaneous assessment of speech identification andspatial discrimination: A potential testing approach for bilateral cochlear implant users? Trends in Hearing,19, 2015, 2331216515619573.
[10] Vickers, D. A.; Moore, B. C.; Majeed, A.; Stephenson, N.; Alferaih, H.; Baer, T.; Marriage, J.E. ClosedSet Speech Discrimination Tests for Assessing Young Children. Ear and hearing. 2018 Jan 1, 39(1), pp32-41.
[11] http://3d-tune-in.eu/ - last accessed in May 2019[12] Eastgate, R.; Picinali, L.; Patel, H.; D’Cruz, M. 3D games for tuning and learning about hearing aids. TheHearing Journal. 2016 Apr 1;69(4):30-2.
[13] Cuevas-Rodriguez, M.; Gonzalez-Toledo, D.; de La Rubia-Cuestas, E.; Garre, C.; Molina-Tanco, L.; ReyesLecuona, A.; Poirier-Quinot, D.; Picinali, L. An open-source audio renderer for 3D audio with hearing loss
and hearing aid simulations. In Audio Engineering Society Convention 142, 2017 May 11.

7649

[14] Cuevas-Rodriguez, M.; Picinali, L.; Gonzalez-Toledo, D.; Garre, C.; de la Rubia-Cuestas, E.; MolinaTanco, L.; Reyes-Lecuona. A. 3D Tune-In Toolkit: An open-source library for real-time binaural spatialisation. PloS one. 2019 Mar 11;14(3):e0211899.
[15] https://github.com/3DTune-In/3dti_AudioToolkit - last accessed in May 2019[16] https://cycling74.com/ - last accessed in May 2019[17] Wright, M.; Freed, A. Open SoundControl: A New Protocol for Communicating with Sound Synthesizers.
InICMC 1997 Sep 25.

7650