Influence of asymmetric processing delays on the localization abilityof bimodal CI/HA users
Stefan ZIRN(1) , Julian ANGERMEIER(1) , Thomas WESARG(2)(1) University of Applied Sciences Offenburg, Germany(2) University ENT Clinic Freiburg, Germany

Abstract
Under optimal hearing conditions the normal-hearing auditory system can perceive interaural time differences(ITD) as little as 10 µs. Due to physiological limitations, the largest occurring ITD is about 700 µs. Differencesin processing latencies of digital hearing aids (HA) and cochlear implants (CI) can be as high as 9 ms in bimodal
users. These differences result in an interaural stimulation timing mismatch that superimposes physiologicallyoccurring ITDs. This interaural stimulation timing mismatch could account for the reduced sound localization
performance in bimodal users, which has been reported by other authors. To investigate this hypothesis, weconducted localization tests in bimodal CI/HA users with and without a compendsated stimulation timing mismatch. For this, we delayed the CI stimulation according to the individual HA processing latency by a portable,
programmable delay line based on a microcontroller. First, an initial sound localization test without compensation was conducted. Then, the delay line was activated. After a familiarization period to the delay line thetest was repeated. The results show a highly significant improvement of the localization ability in the horizontal
plane after compensation.
Keywords: Localization, bimodal stimulation, stimulation timing2243

1INTRODUCTION

In bimodal users of Cochlear Implants (CI) and hearing aids (HA) the processing delays between both devices
are currently not synchronized. In a previous study it could be shown, that the processing delay of MED-EL CIsystems τCI is relatively similar to the physiological delay in normal hearing ears. In contrast, the processing
delay of HA τHA is mostly frequency independent and several milliseconds large. Because HA processing isdone in front of the physiological processing in the ear, a device delay mismatch in the order of τHA occurs inbimodal MED-EL CI/HA users [1]. The measured values of τHA are in accordance with other studies reporting
HA processing delays between 3 and 11 ms [2]. For the identification of syllables Stone & Moore have shownthat delays below 15 ms are relatively uncritical [3], this robustness can not be assumed for other mechanismsof binaural hearing. Especially in sound localization, experimental results suggest that bimodal users perform
poorer than e.g. bilateral HA users or bilateral CI users [4]. A reason for this could be the device delay
mismatch between CI and HA, since the normal hearing auditory system uses interaural time differences (ITD)in the temporal fine structure (TFS) and in the amplitude envelope (ENV) for sound localization. These cues,already diminished by the signal processing of the CI, are likely to be further deteriorated by the superimposed
temporal asymmetry between both devices. Even though the auditory system can compensate for static timing
asymmetries to some extent, as shown by Trapeau and Schönwiesner [5], plasticity of the auditory system
towards asymmetries in the order of several milliseconds has not been investigated. The hypothesis of this study
is, that sound localization accuracy of bimodal users will increase, if the device delay mismatch is reduced.

2METHODS

2.1 Test subjects
Nine adult bimodal test subjects participated in the localization experiments (age: 51.8 ± 13.0 years). Thesubjects had mild to severe sensorineural or combined hearing loss on the ear provided with the HA and nomeasurable residual acoustic hearing on the ear provided with the CI. The criteria for inclusion in the localization tests were the following:
1. everyday combined CI and HA use
2. a minimum 50% correct score on the Freiburg monosyllabic test at 65 dB SPL on CI and HA
The work described was conducted in accordance with the Declaration of Helsinki. The approval by the ethics
committee of the University of Freiburg was obtained before testing (89/17). Informed consent was obtained by
all participants.
2.2 Delay line
To partly compensate for the timing asymmetry between CI and HA stimulation timing, a programmable,portable delay line (DL) was designed based on a microprocessor plattform(Ardunio DUE). With this DL theinput to the CI was delayed according to the HA processing latency τHA measured with the ACAM 5 HAanalyzer (Acousticon GmbH, Reinheim, Germany). The delay was realized by implementing a circular buffer
on the microprocessor which allowed to delay the incoming signals by integer multiples of the sampling time
T . With a sampling rate of Fs = 48 kHz, T is 20.8 µs. Since the analog-digital converters and digital-analog
converters also need additional clock cycles for their conversions an offset of 50 µs has to be added to theprogrammed delay. The complete setup for delaying the input to the CI consisted of four components. At
first the acoustic signal was captured by an OPUS2 CI audio processor from MED-EL (Innsbruck, Austria)
without additional signal processing and therefore no additional delay. The audio processor was connected to amicrophone tester, usually used to test the microphones of the audio processors by CI fitting specialists. This
microphone tester is an analog amplifier which introduces no remarkable delay to the processing chain. The
output of the microphone tester was connected to the DL where the signal was delayed according to the programmed delay. The DL output was fed into the auxiliary input of a second OPUS2 audio processor with the2244

internal microphones deactivated to avoid non delayed input. This second, silent OPUS2 was programmed withthe subjects everyday programming. For subjects HA, settings were not changed. A schematic of the processing
chain can be seen in figure 1.

Figure 1. Schematic setup for delaying CI stimulation
2.3 Stimuli and Test environment
The subjects were seated in an audiometric booth with seven loudspeakers (Genelec 8030A) arranged in thefrontal horizontal hemisphere between -90° and 90° at intervals of 30° with a distance of 1 m from the subjects
and at the height of the subjects ears. The speakers were labeled with numbers 1 to 7 according to 2.

Figure 2. Loudspeaker arrangement for localization tests
The stimuli consisted of 200 ms bursts of white noise bandpass filtered between 125 and 6000 Hz with eightpole butterworth filters and on- and offset ramps of 20 ms as described by [6]. The stimuli were presented

2245

at a level of 65 dB(A). The subjects were instructed to center their head on loudspeaker number 4 and torefrain from head movements throughout testing. Subjects entered their response to the stimuli with a wireless
bluetooth keypad containing the numbers corresponding to the speaker numbers.
2.4 Experimental procedures
Each subject underwent two localization tests, each consisting of 140 stimulus presentations resulting in 20
presentations per speaker per test. The first localization test was conducted with the DLs delay set to 0 ms.
Although this test could have been conducted without the DL, it was applied to ensure similar testing conditions
between tests. As described above a programmed delay of 0 ms still includes the DLs processing delay of 50µs, which is further seen as negligible. After finishing the first test the DL was programmed to the subject
specific τHA . To allow some familiarization to the change in stimulation timing, the subjects were instructed togo for a walk outside the lab for 45 minutes while paying attention to the perceived everyday sound sources(e.g. traffic or conversations). After this period the subjects returned to the lab and were presented with a 15
minute segment of TV news from the night before so that similar training for all subjects could be ensured.
After this familiarization period with a total duration of one hour, the second localization test was conducted.
2.5 Data analysis
Root mean square (rms) errors were calculated by the formula proposed by Rakerd and Hartmann [7].
s1 M
D=A∑ (ri − k)2
M i=1

(1)

In equation 1, A is the angle between loudspeakers (i.e. 30°) and M is the total number of stimuli presented.
The parameter ri corresponds to the response in terms of a speaker number and k is the target speaker, also interms of the respective speaker number. Rms errors were calculated for every loudspeaker to further investigate
the effect of a compensation of timing mismatch. For this purpose equation 1 was used with a fixed k for every
loudspeaker over a total of 20 presentation per speaker. Before the analysis, the data was normalized so thatthe side, where the subjects used their CI was always the left side (i.e. -90°) for better comparability. Becauseof the small number of data points per subject and speaker no statistic tests were applied.

3RESULTS

The results for rms errors across all seven speakers show a highly significant improvement in localization ability
after activation of the delay line. Details can be found in [8]. In Figure 3 the rms errors for every loudspeaker
are shown as boxplots without active use of the DL (blue) and after using the DL for an hour (green). The
red lines represent the median and the lower and upper edges of the boxes represent the first and third quartile.
The r axis corresponds to the rms error and the θ axis is the respective loudspeaker position.

2246

Speaker azimuth [°]0-30

30-6060
without delay linewith delay line

-9090
01530

45RMS-Error [°]6075

Figure 3. Speaker dependent rms error [°] before and after application of DL (CI always at -90°)
The median rms error for all loudspeakers except for -60° and -90° show an improvement in localization accuracy after employment of the DL, with the biggest improvement of about 30° being at 90°. Further there is adecrease of the interquartile range in almost all of the loudspeakers where localization improved with applicationof the DL.
For the loudspeaker at the CI side (thus at -90°) of the bimodal listeners a decrease in localization accuracy
after application of the DL can be seen. This is hypothesized to be due to the impact of the delayed CI
stimulation in contrast with the subjects everyday condition without the applied delay. Longer familiarization
periods could possibly reduce this negative effect.

4CONCLUSIONSIn this refinement of the previously published data [8] it could be shown that analysis of localization ability
on a finer scale yields a better understanding of the improvement in localization ability through a compensationof a device delay mismatch in bimodal listeners. The results also reinforce the conclusion that an temporalalignment of processing delays is mostly beneficial to bimodal users. To investigate the reasons for the decreasein localization accuracy at some loudspeaker positions further research needs to be conducted with longer familiarization periods applied. Further investigations on the influence of asymmetric processing delays on otherbinaural mechanisms like spatial release from masking could reinforce the hypothesis that temporal synchronization between CI and HA may be necessary for optimal treatment.

ACKNOWLEDGEMENTS
The authors thank the participants of this study for their time and effort. Further the authors thank MED-EL,
Innsbruck, Austria and MED-EL Germany GmbH for their support.

2247

REFERENCES[1] Stefan Zirn, Susan Arndt, Antje Aschendorff, and Thomas Wesarg. Interaural stimulation timing in single
sided deaf cochlear implant users. Hearing Research, 328:148–156, October 2015. ISSN 1878-5891. doi:
10.1016/j.heares.2015.08.010.
[2] Harvey Dillon, Gitte Keidser, Anna O’Brien, and Heidi Silberstein. Sound quality comparisons of advanced
hearing aids:. The Hearing Journal, 56(4):30, April 2003. ISSN 0745-7472. doi: 10.1097/01.HJ.0000293908.
50552.34.
[3] Michael A. Stone and Brian C. J. Moore. Tolerable hearing aid delays. III. Effects on speech production
and perception of across-frequency variation in delay. Ear and Hearing, 24(2):175–183, April 2003. ISSN
0196-0202. doi: 10.1097/01.AUD.0000058106.68049.9C.
[4] Michael F. Dorman, Louise H. Loiselle, Sarah J. Cook, William A. Yost, and René H. Gifford. Sound
Source Localization by Normal-Hearing Listeners, Hearing-Impaired Listeners and Cochlear Implant Listeners. Audiology and Neurotology, 21(3):127–131, 2016. ISSN 1420-3030, 1421-9700. doi: 10.1159/000444740.
[5] Régis Trapeau and Marc Schönwiesner. Adaptation to shifted interaural time differences changes encodingof sound location in human auditory cortex. NeuroImage, 118:26–38, September 2015. ISSN 10538119.
doi: 10.1016/j.neuroimage.2015.06.006.
[6] William A. Yost, Louise Loiselle, Michael Dorman, Jason Burns, and Christopher A. Brown. Sound source
localization of filtered noises by listeners with normal hearing: A statistical analysis. The Journal of theAcoustical Society of America, 133(5):2876–2882, May 2013. ISSN 0001-4966. doi: 10.1121/1.4799803.
[7] Brad Rakerd and W. M. Hartmann. Localization of sound in rooms. III: Onset and duration effects. The
Journal of the Acoustical Society of America, 80(6):1695–1706, December 1986.
[8] Stefan Zirn, Julian Angermeier, Susan Arndt, Antje Aschendorff, and Thomas Wesarg. Reducing the Device Delay Mismatch Can Improve Sound Localization in Bimodal Cochlear Implant/Hearing-Aid Users.
Trends in Hearing, 23:233121651984387, January 2019. ISSN 2331-2165, 2331-2165. doi: 10.1177/
2331216519843876.

2248