Speaker Distance Estimation using Binaural Hearing Aidsand Deep Neural Networks*
Mehdi Zohourian, Jakob Stinner, and Rainer Martin
Ruhr-Universität Bochum, Institute of Communication Acoustics, Bochum, Germany

Abstract
In this paper, we address the problem of speaker distance estimation using binaural hearing aid microphones. The
proposed approach is based on deep neural networks which aim to classify discrete distances at different levels of
precision. We investigate two types of networks, namely, feedforward and convolutional neural networks which aretrained on the binaurally estimated direct-to-reverberant energy ratio. The performance of the proposed algorithm
is assessed on speech signals convolved with both real and synthetic binaural room impulse responses for several
distances and in different acoustical scenarios. On the one hand the proposed approach offers a reliable classificationof coarse distances classes which is also robust against variations of the reverberation time. On the other hand, whena more accurate distance estimation is required, the proposed approach is robust only against small variations of thereverberation time.
Keywords: speaker distance estimation, hearing aid, deep neural networks

1

INTRODUCTION

Acoustic source localization refers to the task of identifying the position of different sources, most commonly interms of direction and distance. While the estimation of the direction-of-arrival (DOA) has been widely studied
in the last decades, estimation of sound source distance has received less attention. The knowledge of the soundsource distance is desirable for a wide range of applications, e.g. assistive devices for visually-impaired people,
human-machine interfaces, and hearing aids (HA).
A well-known acoustic parameter which is widely employed to indicate the source distance is the direct-toreverberant energy ratio (DRR). The DRR is a function of room characteristics, distance to a sound source,
and the directivities of both sound source and microphones. In a reverberant room, the DRR is proportional tothe inverse of the source distance, i.e., every doubling of distance decreases the DRR by 6 dB [1]. In otherwords, if the sound source distance increases the energy of the direct sound decreases whereas the energy of
the reverberant sound remains unchanged. The DRR can be computed directly from the measured room impulse
response (RIR) which practically requires an intrusive measurement within the room. Alternatively, the DRR
is estimated blindly from the incoming signals assuming that the reverberation is modeled as a diffuse soundfield. Moreover, in multi-microphone configurations it is assumed that the energy of the direct component isfully represented by the energy of the coherent source.
State-of-the-art distance estimation algorithms can be classified into learning-based approaches and approaches
based on prior knowledge of the room properties [2]. Both groups implicitly require a calibration w.r.t. one
or multiple points within a room. Learning-based approaches utilize different features for the training phase.
The authors in [3] use the magnitude-squared coherence (MSC) of binaural signals to train a Gaussian mixture
model (GMM). The estimated DRR is a common feature used for instance in [4, 5]. Georganti et al. [6]propose to use the spectral power of binaural signals with which two variants of learning algorithms are trained,
namely, a support vector machine (SVM) and a GMM. Head-related transfer functions (HRTFs) are anotherfeature proposed in [7] for source distance estimation of binaural signals in three dimensional space. In almost
all these evaluations the reverberation time (T60 ) of the training and test sets are similar as otherwise large* This work has received funding from the European Fund for Regional Development, grant no. EFRE-0800372 (RaVis-3D).

3297

errors are to be expected.
In this paper we aim for the estimation of speaker distance using 2x2 binaural HA microphones. Our approachis based on the use of deep feedforward and convolutional neural networks trained on the binaurally estimated
DRR. Our algorithm is able to classify discrete distances in simulated and real acoustic environments.
The remainder of this paper is organized as follows. Section 2 describes the estimation of binaural DRR using
a coherence based technique. Section 3 explains the topology of the two neural networks used in this paper.
The experimental setup and validation results are presented in Section 4. Section 5 concludes this paper.

2

Direct-to-reverberant energy ratio

In our scenario we consider a noiseless acoustic environment with a point source signal s received by 2x2
microphones distributed on a pair of HAs. Using the convolution operator ∗, the microphone signal x may be
expressed asxm (n) = hm (n, θs ) ∗ s(n), m ∈ {1, . . . , 4},(1)
where n denotes the sampling index and θs determines the DOA of the source. In this equation, hm indicatesthe binaural RIR (BRIR) for the corresponding source/receiver positions which is composed of hm (n, θs ) =
(d)
(er)
(lr)
(d)
(er)
(lr)
hm (n, θs ) + hm (n) + hm (n), where hm (n, θs ), hm (n), and hm (n) denote the direct path, the early reflections,and the late reverberation, respectively. Then, the DRR for each microphone can be directly computed from the
RIR asTm +N∑ |hm (n, θs )|2

ηtrue =

n=1

|hm (n, θs )|2

∑

,

(2)

n=Tm +N+1

where Tm indicates the index of the first peak in the BRIR for each microphone. Moreover, N denotes the rangeof samples in the neighborhood of the direct component and early reflections used to compute the energy ofthe direct path. Practically, we consider N/ fS = 2 ms in the neighborhood of the highest peak in the RIR forthe computation of the energy of the direct path.
In practice, however, the RIR is not available and thus the DRR is blindly estimated from the multi-microphonesignal. For this purpose, the effect of early reflections is commonly not considered and thus the signal modelin (1) may be written asym (n) = xm (n, θs ) + νm (n),

(3)

(d)

(lr)

where x(n, θs ) = hm (n, θs ) ∗ s(n) indicates the non-reverberated microphone signal and νm (n) = hm (n) ∗ s(n)denotes the reverberated part of the microphone signal which is modeled as an ideal diffuse sound field.
Applying the short-time Fourier transform (STFT) to the signals leads to
Ym (k, µ) = Xm (k, µ, θs ) +Vm (k, µ),(4)where k and µ indicate frequency and time frame indices, respectively.
One solution to estimate DRR is to compute the coherence ΓYmYm0 between two microphone signals defined as
ΓYmYm0 (k) = q

ΦYmYm0 (k),(5)ΦYmYm (k)ΦYm0 Ym0 (k)where ΦYmYm0 (k) = E Ym (k)Ym∗0 (k) determines the cross power spectrum of the microphone signals which isestimated by using the first-order recursive temporal smoothing filter as
ΦYmYm0 (k, µ) = αΦYmYm0 (k, µ − 1) + (1 − α)Ym (k, µ)Ym∗0 (k, µ),

3298

(6)

where α = 0.92 denotes the smoothing factor.
Based on the signal model (3) and the assumption that the non-reverberated and reverberated components of the
microphone signals are mutually uncorrelated, we may express
ΦYmYm0 (k, µ) = ΦXm Xm0 (k, µ, θs ) + ΦVV (k, µ)ΓVmVm0 (k).

(7)

In this equation, ΦVV indicates the power of the reverberated components of the microphone signals and ΓVmVm0
denotes its coherence which is estimated assuming the spherically isotropic (diffuse) sound field asΓVmVm0 (k) =sin(2Ωk fs ac−1 ),2Ωk fs ac−1(8)with a = 8.5 cm indicating the radius of the head and c = 343 m/s denoting the speed of sound. Here, Ωk =2πk/M where M is the number of discrete Fourier transform (DFT) bins.
Moreover, taking the microphone m as a reference we may write
ΦXm Xm0 (k, µ, θs ) =Dm0 (k, θs )ΦXX (k, µ),Dm (k, θs )(9)where ΦXX denotes the PSD of the non-reverberated component of microphone m (the reference microphone)(d)and Dm (k, θs ) indicates the Fourier transform of hm (n, θs ). In this work, we only consider the coherence
between binaural microphones and thus m ∈ {1, 3} and m0 ∈ {2, 4}. Therefore, we may writeΦXm Xm0 (k, µ, θs ) = G(k, θs ) exp jΩk fs ∆τ(θs ) ΦXX (k, µ)(10)
D (k,θ )where G(k, θs ) = Dmm0(k,θss) and ∆τ(θs ) = τm0 (θs ) − τm (θs ) are the interaural level difference (ILD) and interauraltime difference (ITD) for the source angle θs , respectively.
XX (k,µ)By inserting (7) in (5) and introducing η(k, µ) = ΦΦVV (k,µ) as the narrowband DRR of microphone m we mayachieve

G(k, θs )e jΩk fs ∆τ(θs ) η(k, µ) + ΓVmVm0 (k).
ΓYmYm0 (k, µ) = p(η(k, µ) + 1)(G2 (k, θs )η(k, µ) + 1)

(11)

In principle, ΓYmYm0 is a complex value whereas η is a positive real-valued quantity. We thus need a meaningful
mapping of the complex coherence to the real-valued DRR. In our previous work [8] we propose to take the
MSC to estimate the distance of speech signals coming from different azimuths. In this work we employ the
MSC approach to estimate the speaker’s distance at θs = 0. Therefore, G(k, 0) = 1, ∆τ(0) = 0, and (11) is
simplified toΓYmYm0 (k, µ) =η(k, µ) + ΓVmVm0 (k).
η(k, µ) + 1(12)

Taking the magnitude square of (12) and rearranging the equation leads to the following MSC based DRR
estimation approachη̂(k, µ) =

ΓVmVm0 (k) − |ΓYmYm0 (k, µ)|.
|ΓYmYm0 (k, µ)| − 1(13)

Furthermore, inspired by the human auditory system, we apply 40 mel-scaled triangular bandpass filters to theestimated narrowband DRR and average across time frames asη̂mel (c) =1 B−1 C−1
∑ ∑ L(c, µ)η̂(k, µ),
B µ=0
c=0

(14)

where L(c, µ) indicates the filter response of auditory channel c.

3299

3

Distance estimation using classification techniques

In order to estimate the distance of a sound source we propose to use a classification technique using two
neural network architectures, namely, feedforward and convolutional neural network that aim to classify discrete
distances. The networks are trained on estimated DRR and will be evaluated on several acoustic scenarios.
3.1 Feedforward neural network
The first classifier is based on a feedforward neural network (FNN) which is also known as a multilayer perceptron (MLP). The structure of the network consists of an input layer, an output layer, and multiple fully
connected hidden layers. The network weights are computed by minimizing a loss function using the gradientdescent approach. In this work we design a FNN comprising 2 hidden layers with 44 hidden neurons, each ofwhich compute a non-linear activation function of the weighted sum of the output of the previous layer. The
final layer uses the softmax rule to estimate the posterior probability for each distance class using the crossentropy loss function. The network is trained in full batch mode with scaled conjugate gradient backpropagation(SCG) [9] and early stopping.
3.2 Convolutional neural network
The second classifier is based on a convolutional neural network (CNN) which typically consists of convolutional and pooling layers followed by a fully connected layer. The convolutional layer applies the convolution
operation to the input and each neuron receives input from only a restricted subarea (receptive field) of the previous layer. The pooling operation reduces the size of the input data and the fully connected layer aggregates
local information to provide class discrimination. This topology is widely used in practice since it significantly
reduces the computational complexity for large sizes of input data.
In this work, we propose a six-layer CNN including five convolutional and one fully-connected layers. The sizeof kernels (filters) in the convolutional layers is set to 3 × 3. The number of filters (depth) in the convolutional
layers is set to 16 in the first layer and increases by a factor of 2 after each pooling up to 128 in the final layer.
The convolutional stride is fixed to 1 and a padding of 1 is designed to maintain the resolution after convolution.
We use max-pooling with a 3×3 kernel, stride of 2 to bisect the input dimensions. Furthermore we use rectified
linear unit (ReLU) activation functions and batch normalization. Global pooling over time followed by a dropout
of 50% prior to the final fully connected layer enforces approximate time-translation invariance. The final layer
uses the softmax rule to estimate the posterior probability for each distance class using the cross-entropy loss
function. Moreover, for optimizing the learning process we employ the Adam algorithm [10].

4Experimental results

In our experiment we use speech signals convolved with synthetic and real BRIRs at eight discrete distances
dtrue ∈ [0.5 : 0.5 : 4] m.
In order to generate synthetic BRIRs we integrate the head-related impulse responses (HRIRs) in [11] into
the auditory virtual environment (AVE) software [12]. The HRIRs [11] are recorded for 6 microphones of apair of behind-the-ear (BTE) HAs from which we only use 4 channels corresponding to the front and rear
HA microphones. The AVE software uses a modified image source model [13] and an improved statistical
reverberation model [14] to synthesize acoustics of multiple sound sources in different rooms. The algorithm
also provides a smooth transition between the early reflections and late reverberation. With this software we
simulate a room with dimensions of 4 m × 5.5 m × 3 m and three reverberation times of T60 = 0.4, 0.5, and
0.9 s.
We also measure BRIRs using 2x2 microphones of a pair of BTE HAs attached to a Head and Torso Simulator(HATS) [15]. BRIRs are recorded in a reverberant room with dimensions of 7.5 m × 6.3 m × 3.3 m, threereverberation times of T60 = 0.4, 0.7, and 0.9 s and for the various distances mentioned earlier.
We convolve the BRIRs with speech signals taken from the TIMIT database [16]. For the training set we

3300

Table 1. Accuracy (in percent) of the proposed FNN-based distance estimation algorithm when the network is
trained and tested on signals generated in the same specific reverberation time T60 = {0.4, 0.5, 0.7, 0.9}. Results
are shown for both simulated (Synth.) and measured (Real) BRIRs.
50

100

150

Distance (cm)
200 250 300

Synth.

0.4s
0.5s
0.9s

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

Real

0.4s
0.7s
0.9s

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

T60

350

400

Overalluse 462 speakers including 326 males and 136 females each of which speak 10 sentences, forming a total setof 4620 training sentences. Each sentence has a duration of approximately 2 s. For the test set we use 168
speakers including 112 males and 56 females each of which speak 10 sentences forming a total set of 1680 test
samples. The training and test sets are created such that they contain different speakers.
For the estimation of DRR features which are processed in the STFT domain, we segment signals into overlapping frames of 32 ms using a Hamming window with a frame advance of 8 ms. The DFT size is equal to
M = 512.
The DRR is estimated for front and rear binaural microphones resulting in 80 = 2 × 40 values for each timeframe. Since FNN requires only one-dimensional inputs, we take the mean of the estimated narrowband DRRs
across all time frames leading to the feature vector of dimension 80 × 1. For CNN, however, we compute theestimated narrowband DRRs across 40 consecutive time frames and for both front and rear microphones which
result in a feature matrix of dimension 40 × 40 × 2. We train both networks on a set of 4620 feature vectors foreach distance randomly split into 85% training and 15% validation sets. Each input vector is normalized to therange of [−1, 1].
The performance of the proposed algorithm is measured in terms of accuracy of the classification which isdefined as the percentage of examples for which the model predicts the correct outputs. It is mathematically
expressed as
A=

TP + TN,TP + TN + FP + FN

(15)

where TP and TN indicate the sum of true positive and true negative predictions and FP and FN denote thesum of false positive and false negative predictions, respectively.
We investigate the performance of our approach in different scenarios. The first scenario considers signals
generated in a room with the same T60 for training and testing. Results for both FNN and CNN and for bothsynthetic and real BRIRs are presented in Tables 1 and 2. It can be observed from these tables that when
the networks are trained and tested on signals generated with the very same reverberation time both networks
achieve reliable distance estimation accuracies.
In the second scenario we assess the generalization capability of the proposed learning-based approach. Hence,
we train the networks on signals generated with a particular T60 and test on signals generated with different
T60 s. Here, we train the networks on signals generated with T60 = 0.5 s for synthetic and with T60 = 0.7 s formeasured BRIRs and test on signals generated using all the available T60 s. Results for both FNN and CNN
classifiers and for both synthetic and real BRIRs are illustrated in Tables 3 and 4, respectively. According tothese tables when the reverberation time between the training and the test signals varies slightly, the accuracyof both networks is fairly robust. However, when the network has not been properly trained on the acoustical

3301

Table 2. Accuracy (in percent) of the proposed CNN-based distance estimation algorithm when the network is
trained and tested on signals generated in the same reverberation time T60 of the room. Results are shown forboth simulated (Synth.) and measured (Real) BRIRs.
50

100

150

Distance (cm)
200 250 300

Synth.

0.4s
0.5s
0.9s

100
100
100

100
100
100

99.7
99.6
99.4

100
99.8
99.2

99.9
99.5
98.9

100
99.4
99.6

100
99.9
99.4

99.9
99.5
99.7

99.9
99.7
98.4

Real

0.4s
0.7s
0.9s

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

100
100
100

T60

350

400

Overall

Table 3. Accuracy (in percent) of the proposed FNN-based distance estimation algorithm when the network is
trained on signals generated with T60 = 0.5 s for synthetic (Synth.) and with T60 = 0.7 s for measured (Real)
BRIRs and is tested on signals generated using all the available T60 s.
50

100

150

Distance (cm)
200 250 300

Synth.

0.4s
0.5s
0.9s

100
100
0.1

90.4
100
77.7

24.3
100
43.7

37.6
100
97.0

0.1
100
7.3

94.8
100
10.8

45.1
100
0.1

95.3
100
11.8

61
100
31.1

Real

0.4s
0.7s
0.9s

100
100
100

93.6
100
100

1.1
100
100

91.4
100
94.4

32.8
100
100

1.3
100
97.6

88.6
100
99.6

96.3
100
100

63.1
100
99

T60

350

400

Overallproperties of the room, e.g., the reverberation time the deep learning approaches can not estimate the source
microphone distances reliably. The outliers for some particular distances shown in these tables may result fromeither the low sensitivity of these distance classes to the estimated DRR or from the small size of the network.
Overall, the DNN-based speaker distance estimation technique will be a reliable candidate provided that the
network is trained on signals from different acoustic situations.
In the third scenario we investigate the performance of the proposed DNN-based distance estimation algorithms
on the classification of coarse distance classes. For this experiment we train the networks on signals generated
for distances in the range of dtrue ∈ {1 m, 2 m, 3 m, 4 m} which implicitly reduces the resolution of the distanceestimation. Similar to the second scenario, we train the networks on signals generated with T60 = 0.5 s forsynthetic and with T60 = 0.7 s for measured BRIRs and test on signals generated using all the available T60 s.
Results for both synthetic and real BRIRs and for FNN and CNN are presented in Table 5. Comparing theoverall results of Tables 3 and 4 with the overall results of Table 5 shows that the proposed learning based
distance estimation approaches can be more reliable for the estimation of coarse distance classes, e.g., in stepsof 1 m. Higher accuracies of acoustic distance estimation indeed require more advanced training schemes or
employing other modalities provided by, for instance, a radar sensor which is beyond the scope of this paper.

3302

Table 4. Accuracy (in percent) of the proposed CNN-based distance estimation algorithm when the network is
trained on signals generated with T60 = 0.5 s for synthetic (Synth.) and with T60 = 0.7 s for measured (Real)
BRIRs and is tested on signals generated using all the available T60 s.
50

100

150

Distance (cm)
200 250 300

Synth.

0.4s
0.5s
0.9s

93
100
0

85
100
4.2

96.2
99.6
100

98.5
99.8
0

16.5
99.5
4.4

40.5
99.4
14.3

13.6
99.9
0

55.8
99.5
7.9

62.4
99.7
16.4

Real

0.4s
0.7s
0.9s

28.9
100
100

0
100
100

0.1
100
99.1

73.8
100
99

93.7
100
97.3

4
100
82.1

32.6
100
21.6

1.3
100
54.8

29.3
100
81.7

T60

350

400

Overall

Table 5. Accuracy (in percent) of the proposed FNN-based and CNN-based distance estimation algorithm when
the network is trained on signals generated with T60 = 0.5 s for synthetic (Synth.) and with T60 = 0.7 s formeasured (Real) BRIRs and is tested on signals generated using all the available T60 s. Here a coarse level ofprecision for distance classification is tested.
FNN
T60

5

CNN

100

200

300

400

Distance (cm)Overall
100 200

300

400

Overall

Synth.

0.4s
0.5s
0.9s

98.1
100
91.2

69.1
100
83.3

99.5
100
3.7

57.1
100
24.9

81
100
50.8

100
100
85.7

90.0
99.9
47.2

56.3
99.3
9.4

75.2
99.3
38.8

80.4
99.7
45.3

Real

0.4s
0.7s
0.9s

100
100
100

76.7
100
100

18.2
100
97.0

95.2
100
100

72.5
100
99.2

89.7
100
100

62.5
100
99.3

59.8
100
100

100
100
79.9

75.5
100
94.8

CONCLUSIONSIn this work, we presented a learning-based algorithm for the estimation of a speaker’s distance using HA
microphones. Our algorithm aims to classify discrete sound source distances using deep neural networks. The
networks are trained on estimated DRRs which are computed using the magnitude squared coherence taking thebinaural configuration into account. We tested the accuracy of our algorithm on speech signals convolved withsynthetic and measured BRIRs at fine and coarse distance classes and under different reverberation times.
Our results show that the speaker’s distances can be reliably estimated if the network is trained properly on
the acoustical properties of the room. The method achieves a reliable accuracy when the reverberation timesof the signals in the training and test sets varies only slightly. For large variations in the reverberation time ofthe signals under test the estimation is sufficiently accurate only for the classification of coarse distance classes.
For the classification of fine distance classes, however, apparent mismatch of training and test data occurs inspecific conditions which requires an additional regularization.

3303

References[1] J. Blauert, Communication Acoustics, vol. 2, Springer, 2005.
[2] Y. Hioka, K. Niwa, S. Sakauchi, K. Furuya, and Y. Haneda, “Estimating direct-to-reverberant energy ratiousing D/R spatial correlation matrix model,” IEEE Transactions on Audio, Speech, and Language Processing,vol. 19, no. 8, pp. 2374–2384, 2011.
[3] S. Vesa, “Sound source distance learning based on binaural signals,” in IEEE Workshop on Applications of
Signal Processing to Audio and Acoustics, Oct 2007, pp. 271–274.
[4] Y. C. Lu and M. Cooke, “Binaural estimation of sound source distance via the direct-to-reverberant energy
ratio for static and moving sources,” IEEE Transactions on Audio, Speech, and Language Processing, vol.
18, no. 7, pp. 1793–1805, 2010.
[5] A. Brendel and W. Kellermann, “Learning-based acoustic source-microphone distance estimation using
the coherent-to-diffuse power ratio,” in IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP), April 2018, pp. 61–65.
[6] E. Georganti, T. May, S. Van De Par, and J. Mourjopoulos, “Sound source distance estimation in rooms
based on statistical properties of binaural signals,” IEEE Transactions on Audio, Speech, and Language
Processing, vol. 21, no. 8, pp. 1727–1741, 2013.
[7] F. Keyrouz, “Binaural range estimation using head related transfer functions,” in IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Sept 2015, pp. 89–94.
[8] M. Zohourian and R. Martin, “Direct-to-reverberant energy ratio estimation based on interaural coherence
and a joint ITD/ILD model,” in IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), 2019.
[9] M. F. Møller, “A scaled conjugate gradient algorithm for fast supervised learning,” Neural networks, vol.
6, no. 4, pp. 525–533, 1993.
[10] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980,
2014.
[11] H. Kayser, S. D. Ewert, J. Anemüller, T. Rohdenburg, V. Hohmann, and B. Kollmeier, “Database of multichannel in-ear and behind-the-ear head-related and binaural room impulse responses,” EURASIP Journal
on Advances in Signal Processing, vol. 2009, pp. 6, 2009.
[12] Christian Borss, An Improved Parametric Model for the Design of Virtual Acoustics and its Applications,
Verlag Dr. Hut, 2011, PhD. Thesis.
[13] J. B. Allen and D. A. Berkley, “Image method for efficiently simulating small-room acoustics,” The
Journal of the Acoustical Society of America, vol. 65, no. 4, pp. 943–950, 1979.
[14] J. A. Moorer, “About this reverberation business,” Computer Music Journal, pp. 13–28, 1979.
[15] ITU-T Recommendation P.58, Head and Torso Simulator for telephonometry, HEAD acoustics, 2018.
[16] V. Zue, S. Seneff, and J. Glass, “Speech database development at MIT: TIMIT and beyond,” Speechcommunication, vol. 9, no. 4, pp. 351–356, 1990.

3304