# Multiple Sound Source Localization and Counting Using One Pair of Microphones in Noisy and Reverberant Environments

Authors: Yuzhuo Fang; Zhiyong Xu
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

A multiple sound source localization and counting method based on an angular spectrum is proposed in this paper. Local signal-to-noise ratio tracking, onset detection, and a coherence test are introduced to filter the generalized cross-correlation angular spectrum in the time-frequency domain for multiple sound source localization and counting in noisy and reverberant environments. Then, dual-width matching pursuit is introduced to replace peak search as the method of localization and counting. A comprehensive comparison of two statistical indicators, mean precision and mean absolute estimated error, indicates that the proposed localization and counting algorithm using both the filtered angular spectrum and dual-width matching pursuit method is more robust and accurate than the classic counterpart, especially in environments with low signal-to-noise ratio, strong reverberation, and abundant sound sources.


---

## Body

## 1. Introduction

In the field of array signal processsing, multiple sound source localization and counting is a critical issue for applications such as indoor conferences, building security, and virtual reality [1]. Multiple sound source localization is usually subject to ambient noise, reverberation caused by confined space and obstacles, and mutual interference between multiple sound signals [2]. Many heuristic algorithms have been studied over the past two decades to suppress the effects of these negative factors and achieve robust and accurate performance. The main branch of research is based on time-frequency (TF) processing, which can be divided into three categories: clustering [3–6], histogram [7–10], and angular spectrum [6, 11–14].The first category clusters the TF bins associated with each sound source on the basis of a criterion, such as interphase difference (IPD) [3] and direction estimation of mixing matrix (DEMIX) [4]. This method which directly achieves localization and counting results by iterative processing is sensitive to the initial clustering parameter [6]. In the second category, the weighted histogram using one pair of microphones is computed in [7] to solve the underdetermined problem. By using a relatively more complex topology, such as a unit circular array [9] and a soundfield microphone [10], the redundancy information of the circular integrated cross spectrum (CICS) [9] and the smoothed histogram [10] can improve the localization and counting accuracy, while expanding the estimated range of direction of arrival (DOA) from 0,180° to 0,360°. These histogram-based algorithms are sensitive to the spatial aliasing ambiguity in some widely spaced microphone arrays because of the local TF computation [5]. In this paper, we focus on the third category, angular spectrum, which consists of two steps: (i) angular spectrum construction and (ii) sound source localization and counting.First, an enumerated function related to all possible DOAs in each TF bin is accumulated to construct the angular spectrum [6], such as classical generalized cross-correlation phase transform (GCC-PHAT, simplified as GCC in the following) [11, 12] and the kernel density estimator (KDE) [13, 14]. GCC which achieves a certain degree of antinoise performance through the phase weighting of the cross-correlation function is robust to the spatial aliasing problem [6, 15]. However, in environments with the coexistence of noise, reverberation, and mutual interference between multiple sound sources, the performance deteriorates substantially because of the limitations of the ideal single-source propagation model [16]. KDE has better antireverberation performance than GCC when the number of sound sources is relatively small (e.g., 2) [17]. The spatial aliasing ambiguity can be suppressed by the embedded frequency-dependent weighting factors in the kernel function, but the suppression is sensitive to the choice of kernel bandwidth [18].Second, the DOAs and the corresponding number of multiple sound sources are obtained through the exhaustive search of the optimal parameter value from the angular spectrum. Traditional peak search (PS) [19], which is based on single-point peak amplitude, implements source localization and counting by comparing the peak amplitude with the cut-off threshold. The cut-off threshold becomes adaptive by using the previous peak [9]. Because the angular spectrum is seriously distorted in adverse environments, the estimated DOA when using PS may have a large offset, resulting in instability. Matching pursuit (MP) [20] can be used to improve the performance of PS by calculating the maximum inner product, but the choice of matching structure and atom width needs careful consideration. On the basis of source contributions, iterative contribution removal (ICR) [15] filters out the TF bins associated with the current estimated sound source during each iteration and then reconstructs the angular spectrum from the remainder to the next source localization. The reconstruction can restrain the distortion of the angular spectrum, but the search of the TF bins is computationally expensive, and incorrect previous source localization may exert considerable influence on the following iterative process.In this paper, we improve each step of the angular spectrum-based algorithm. GCC is used because of its applicability to any microphone spacing [6, 15]. The main innovations are as follows. In the angular spectrum construction step, three TF filtering modules, local signal-to-noise ratio (SNR) tracking [15, 21], onset detection [22, 23], and coherence test [24, 25], are introduced to extract the TF bins that are less disturbed by noise, reverberation, and mutual interference between sound sources, respectively. The filtered GCC is termed GCCTF. In the localization and counting step, PS with the single-point amplitude is replaced by MP [8, 20] with the inner product of the atom from the perspective of contribution removal. The dual-width structure [9] and the source merging module are used to improve the iteration efficiency and to remove the repeatedly estimated results, respectively.The remainder of this paper is organized as follows. In Section2, the classic GCC is constructed based on the signal propagation model of multiple sound sources; then, three TF filtering modules are introduced to produce the GCCTF spectrum. In Section 3, the dual-width structure and the source merging module are presented during the description of MP. Numerical comparisons between the proposed GCCTF-MP algorithm and the classical ones are given in Section 4. Finally, Section 5 concludes the paper.


## 2. GCC Angular Spectrum and TF Filtering

### 2.1. Signal Propagation Model of Multiple Sound Sources

The propagation model of multiple sound sources is shown in Figure1. In the sound field space composed of several independent sound sources s1,…,sN, there is a pair of omnidirectional microphones m1andm2 with spacing dmic. The DOA of the sound source, θ1,…,θN∈0,180°, is defined in an anticlockwise manner, with 90° being the direction perpendicular to the line connecting the pair of microphones.Figure 1
Propagation model of multiple sound sources.Ωθ is indicated as the set of DOAs. The elements of Ωθ and the corresponding number N=cardΩθ are unknown, where card⋅ is the operator used to measure the number of elements in the set. In the approximately far field, the signal propagation model of multiple sound sources can be expressed as(1)xmi=∑n=1Nhm,nTsni+wmi,where xmi,m=1,2, denotes the observed signal of the m-th microphone, hm,n=hm,n0,…,hm,nLh−1T denotes the impulse response between the n-th sound source and the m-th microphone, sni=sni,…,sni−Lh+1T denotes the discrete time signal vector of the n-th sound source with sampling rate fs, and wmi is additive white Gaussian noise independent of the sources and the impulse responses.WhenLFFT points short-time Fourier transform (STFT), the expression in the discrete TF domain can be obtained as(2)Xmr,k=∑n=1NHm,nTr,kSnr,k+Wmr,k,where Xmr,k and Snr,k denote the STFT coefficients of observed signal xmi and sound source signal sni corresponding to the r-th frame and the k-th discrete frequency, respectively; Wmr,k denotes the additive complex noise; and Hm,nk is the transfer function between the n-th sound source and the m-th microphone. Under the assumption of diffuse reverberation [26, 27], Hm,nk can be decomposed as the direct wave component Hm,nDk and the reverberation component Hm,nRk, that is,(3)Hm,nk=Hm,nDk+Hm,nRk,=αm,ne−j2πfkTm,n+Hm,nRk,where αm,n and Tm,n denote the propagation attenuation and arrival time of the direct wave from the n-th sound source to the m-th microphone, respectively, and fk=kfs/LFFT denotes the frequency in the k-th frequency bin.By inserting equation (3), equation (2) is expanded as(4)Xmr,k=∑n=1Nαm,nSnr,ke−j2πfkTm,n︸XmDr,k+∑n=1NSnr,kHm,nRk︸XmRr,k+Wmr,k,where XmDr,k and XmRr,k denote the direct wave component and the reverberation component of the observed signal Xmr,k, respectively.


## 2.1. Signal Propagation Model of Multiple Sound Sources

The propagation model of multiple sound sources is shown in Figure1. In the sound field space composed of several independent sound sources s1,…,sN, there is a pair of omnidirectional microphones m1andm2 with spacing dmic. The DOA of the sound source, θ1,…,θN∈0,180°, is defined in an anticlockwise manner, with 90° being the direction perpendicular to the line connecting the pair of microphones.Figure 1
Propagation model of multiple sound sources.Ωθ is indicated as the set of DOAs. The elements of Ωθ and the corresponding number N=cardΩθ are unknown, where card⋅ is the operator used to measure the number of elements in the set. In the approximately far field, the signal propagation model of multiple sound sources can be expressed as(1)xmi=∑n=1Nhm,nTsni+wmi,where xmi,m=1,2, denotes the observed signal of the m-th microphone, hm,n=hm,n0,…,hm,nLh−1T denotes the impulse response between the n-th sound source and the m-th microphone, sni=sni,…,sni−Lh+1T denotes the discrete time signal vector of the n-th sound source with sampling rate fs, and wmi is additive white Gaussian noise independent of the sources and the impulse responses.WhenLFFT points short-time Fourier transform (STFT), the expression in the discrete TF domain can be obtained as(2)Xmr,k=∑n=1NHm,nTr,kSnr,k+Wmr,k,where Xmr,k and Snr,k denote the STFT coefficients of observed signal xmi and sound source signal sni corresponding to the r-th frame and the k-th discrete frequency, respectively; Wmr,k denotes the additive complex noise; and Hm,nk is the transfer function between the n-th sound source and the m-th microphone. Under the assumption of diffuse reverberation [26, 27], Hm,nk can be decomposed as the direct wave component Hm,nDk and the reverberation component Hm,nRk, that is,(3)Hm,nk=Hm,nDk+Hm,nRk,=αm,ne−j2πfkTm,n+Hm,nRk,where αm,n and Tm,n denote the propagation attenuation and arrival time of the direct wave from the n-th sound source to the m-th microphone, respectively, and fk=kfs/LFFT denotes the frequency in the k-th frequency bin.By inserting equation (3), equation (2) is expanded as(4)Xmr,k=∑n=1Nαm,nSnr,ke−j2πfkTm,n︸XmDr,k+∑n=1NSnr,kHm,nRk︸XmRr,k+Wmr,k,where XmDr,k and XmRr,k denote the direct wave component and the reverberation component of the observed signal Xmr,k, respectively.


## 3. GCC Angular Spectrum Construction

In an ideal environment, noise and reverberation do not exist, and the assumption of W-disjoint orthogonality (W-DO) is satisfied [28]. Both XmRr,k and Wmr,k in equation (4) are 0, and at most one sound source dominates the power in each TF bin. In this case, equation (4) can be simplified as(5)Xmr,k=αm,ηr,kSηr,kr,ke−j2πfkTm,ηr,k,where ηr,k∈1,…,N denotes the index of the dominant sound source in each TF bin. Then, the IPD [3] which indicates the phase difference between the observed signals of the pair of microphones can be obtained as(6)λr,k=∠X2r,kX1r,k=2πfkτηr,k+ξr,k,where ∠ is the operator to find the phase of a complex number, τηr,k=T2,ηr,k−T1,ηr,k is the time difference of arrival (TDOA) between the pair of microphones in each TF bin, and(7)ξr,k=2πfkτηr,k2π,denotes the wrapping factor, where ⋅|2π is the operator for obtaining the retained integer after mod2π. The simulation results showed that wider microphone spacing can bring better resolution for localization [17]. However, the widening of dmic is bound to break the limit of half the minimum wavelength λmin, thus making ξr,k≠0, resulting in spatial aliasing ambiguity.On the basis of the IPD obtained in equation (6), the local GCC related to the unknown DOA θ in each TF bin can be expressed as(8)ϕGCCr,k,θ=ℜeλr,k⋅e−j2πfkτθ,=ℜe2πfkτηr,k−τθ⋅ej2πξr,k︸1,where τθ=−dmiccosθ/c (c denotes the atmospheric sound velocity) and ℜ⋅ denotes the real part of the complex argument. Because ej2πξr,k≡1, as shown in equation (8), the wrapping factor can be eliminated. By accumulating the local GCC in equation (8) across all TF bins, the GCC angular spectrum [6] can be obtained as(9)ΦGCCθ=∑r,kϕGCCr,k,θ,θ∈Ωθtotal,where Ωθtotal denotes the linear space of 0,180° with angle grid θmin and ΩTFtotal denotes the set of all TF bins.


## 4. TF Filtered GCC Angular Spectrum

In practice, noise and reverberation are inevitable, and sources are more likely to overlap in the TF domain when the number of simultaneously occurring sound sources increases [9, 10]. Therefore, three modules, local SNR tracking, onset detection, and coherence test, are used to extract the TF bins that are less affected by the above problem in the GCC angular spectrum construction step. A block diagram of GCC and its filtered variant, GCCTF, is shown in Figure 2.Figure 2
A block diagram of GCC and GCCTF.Local SNR tracking: the stronger noise contained in the observed signal will generate a higher angular spectrum floor with a lower recognition degree. So it is necessary to consider the enhancement of the observed signal which is often realized by tracking the noise floor [21]. In this paper, a simple SNR tracking method is used to obtain the SNR in each TF bin, namely, the local SNR [15], which can be expressed as(10)γSTr,k=minlog10Xmr,k2PWr,k−1m=1,2,where PWr,k denotes the local noise power and min⋅ denotes the operator used to obtain the minimum of the set.Assume an ideal case that the whole observed signal starts with a section of pure noise. Then, the firstLW frames are used to measure the initial local noise power. LW should not be set too long since the local noise power changes slowly over time. It is usually set to 2 or 3 empirically. The initial local noise power can be expressed as(11)PWr,k=1LW∑1LWXmr,k2,r=1,…,LW.Then, the local noise power increases slowly during signal frames and decreases slowly during noise frames. It can be expressed as(12)PWr+LW,k=1−βSTPWr+LW−1,k,noise frame,1+βSTPWr+LW−1,k,signal frame,where βST is the updating factor.The TF bins with local SNR above a user-defined thresholdΓST are extracted. The set of TF bins that satisfy the local SNR tracking module can be expressed as(13)ΩTFST=r,kγSTr,k>ΓST.Onset detection: in real environments, strong reverberation will bring serious angular spectrum distortion, which causes incorrect localization and counting results. Regardless of the special case that a new sound is very weak in each frequency band, the onset of a new sound, which is related to the direct wave componentXmDr,k, is often accompanied by a sudden rise in signal amplitude (energy) within some frequency bands [22]. To detect this rise, the parameter of onset detection in each TF bin is set as follows [23]:(14)γOD,mr,k=gXmr,k−Xmr−1,k,where g⋅=⋅+|⋅/2 is the half-wave rectifier function.For them-th microphone, once the peak of γOD,mr,k is detected in each frequency band, the Xmr,k corresponding to the same TF bin is considered as the onset. The threshold is set as Xmr,k and is then gradually attenuated as the time frame r moves forward:(15)ΓOD,mr,k=Xmr,k,onset,βOD⋅ΓOD,mr−1,k,otherwise,where βOD is the decaying factor empirically decided according to the experimental environment βOD<1. The set of TF bins that satisfy the onset detection module can be expressed as(16)ΩTFOD=r,kX1r,k≥ΓOD,1r,k,∩r,kX2r,k≥ΓOD,2r,k.Coherence test: in practice, when the number of simultaneously occurring sound sources increases, the probability of sources with comparable power overlapping in the same TF bin increases. The W-DO assumption cannot be strictly met. So the assumption is appropriately relaxed: when accumulating the angular spectrum, there are TF bins with only one dominant sound source. Then, the coherence test module can be used to extract these TF bins effectively, which mitigates the effect of simultaneously occurring sources. The coherence test parameter is set as follows [25]:(17)γCTr,k=EX1r,kX2∗r,kEX1r,kX1∗r,kEX2r,kX2∗r,k,where ⋅∗ denotes the complex conjugate operator, E⋅ denotes the average expectation of the 2C+1 consecutive time frames, and(18)EXmr,kXm′∗r,k=12C+1∑r′=r−Cr+CXmr′,kXm′∗r′,k,m,m′∈1,2.TF bins withγCTr,k above the user-defined threshold ΓCT are considered to contain only one dominant sound source. Then, the corresponding set can be expressed as(19)ΩTFCT=r,kγCTr,k>ΓCT.On the basis of the three TF filtering modules (local SNR tracking, onset detection, and coherence test), combined with the derivation of GCC, the GCCTF angular spectrum can be obtained as(20)ΦGCCTFθ=∑r,k∈ΩTFϕGCCr,k,θ,where ΩTF=ΩST∩ΩOD∩ΩCT is the set of TF bins after TF filtering.


## 5. Dual-Width Matching Pursuit Method

After the angular spectrum construction in the previous section, the angular spectrum vector with the lengthcardΩθtotal is used to realize the multiple sound source localization and counting. It can be expressed as(21)Φstr=Φstr0,…,Φstrzθmin,…,Φstr180°,z∈ℕ,where the subscript “str” represents the string “GCC” or “GCCTF.” Without loss of generality, Φstr is simplified to Φ for convenience.PS realizes localization and counting through the extraction of the spectrum peak above the cut-off thresholdΓPS, while MP uses the inner-product comparison and the iterative source contribution removal. Consider an atom with one signal pulse, which can be approximately seen as the basic unit of the angular spectrum vector. The set of all the atoms can be defined as(22)Ωu=u⟶q0≤q≤cardΩθtotal−1,q∈ℤ,where ⟶ denotes the operator of a circular shift to the right, u⟶q denotes the row vector of u shifted to the right by q bits, and u can be expressed as(23)u=u⟶0=v⟶−Qv⟶−Q,where ⋅ denotes the ℓ2-norm operator of a vector, v⟶−Q denotes the row vector of v shifted to the left by Q bits, and v can be expressed as(24)v=v⟶0=b,0,…,0,…,0︸cardΩθtotal−2Q+1T,where b is a Blackman window with width 2Q+1 and Q denotes one half of the window width Q∈ℕ+.The choice ofQ must consider a compromise: in the same noisy and reverberant environment, an excessively wide width may incorrectly estimate the DOAs of sound sources with small angular intervals, and an excessively narrow width may reduce the iteration efficiency.Therefore, a dual-width structure is proposed, where a narrower width is used to localize and a wider width is used to process the iterative contribution removal. To differentiate, subscripts “1” and “2” are added to parametersQ, u, v, and b to indicate the atoms with narrower and wider window widths, respectively. A block diagram of the proposed MP method is shown in Figure 3, where the initial value Φ1 is set as Φ. The corresponding steps are as follows:(1)
DOA estimation: the inner-product of each atom with a narrower window widthu1⟶q in set Ωu1 and the i-th angular spectrum vector Φi can be expressed as(25)pq,i=u1⟶q,Φi,where⋅ denotes the inner-product operator. Then, through the search of the maximum inner product, the estimated DOA in the i-th iteration can be obtained as(26)θ^i=argmaxqpq,i⋅θmin,=q˙i⋅θmin,whereq˙i is the shifted bit of u1 when the maximum inner product in the i-th iteration is obtained.(2)
Contribution measurement: on the basis of the maximum inner product in equation (25) and the corresponding atom with wider window width, the contribution vector of the estimated sound source in the i-th iteration can be measured as(27)κi=pq˙i,i⋅u2⟶q˙i.(3)
Stop judgement: give two conditional expressions:(28)κsumiκsum1<ΓMP,(29)i>Imax,whereΓMP is the user-defined threshold, Imax denotes the maximum number of iterations, and κsumi=sumκi denotes the contribution corresponding to the contribution vector κi in equation (27), where sum⋅ denotes the operator for the summation of the vector elements. If either equation (28) or equation (29) is satisfied, the loop stops.(4)
Residual calculation: after removing the contribution vectorκi from the angular spectrum vector Φi in the i-th iteration, the residual used in the i+1-th iteration can be calculated as(30)Φi+1=Φi−κi.Set the number of iterations when the loop stops asIloop; then, the set of the estimated DOAs after the loop body can be represented as(31)Ωθ^I=θ^i1≤i≤Iloop−1,i∈ℤ.Due to the limited window width, the contribution of a certain sound source may not be completely removed from the angular spectrum in each iteration, resulting in closely located sound sources. Extra counts generated by these sources will cause some deviations in counting results. Thus, the postprocessing step called source merging is used to merge the closely located sources into only one source in case that redundantly estimated sound sources are counted.(5)
Source merging: setAmin as the minimum angular interval. Any two estimated DOAs whose angular interval is smaller than Amin should be merged into one sound source according to their corresponding initial inner products. If θ^i′−θ^i<Amin, where θ^i and θ^i′ are two estimated DOAs, the source merging process can be expressed as(32)θ^i=θ^i′,pq˙i,1>pq˙i′,1,θ^i′=θ^i,otherwise.Figure 3
A block diagram of the MP method.Then, the closely located sources are merged, and the final localization and counting results can be obtained as(33)Ωθ^=θ^nn=1,…,N^,N^=cardΩθ^,where Ωθ^ is the set of the final estimated DOAs, θ^n is the n-th element of Ωθ^, and N^ is the number of the estimated DOAs.


## 6. Numerical Analysis

To verify the performance of the proposed GCCTF angular spectrum and dual-width MP method, the image-source model [27, 29] is used to generate the observed data, where the room size is 8.5m×7.5m×3m and the sound velocity c is 344 m/s. The plane schematic diagram of the room is shown in Figure 4 where the heights of the microphones and sources are all set to 1.3 m. A pair of omnidirectional microphones m1andm2 parallel to the x-axis is located at the center of room o with spacing dmic=0.8m. N sound sources s1,…,sn are distributed on a semicircle with o as the centroid and microphone-source distance dms as the radius.Figure 4
Plane schematic diagram of all the microphones and sources in the room.The DOA distribution when the true number of sourcesN varies from 2 to 6 is presented in Table 1. 8 male and 8 female voices taken from the TIMIT dataset are used as sound sources [30], with a sampling rate fs=16kHz. The total number of simulations Isim is set to 200. In each simulation, N segments of length 1.024 s from different voices are randomly extracted and then preprocessed to have the same average power.Table 1
Number of sourcesN versus DOA.
NDOA (°)245, 135315, 45, 135415, 45, 135, 165515, 45, 75, 135, 165615, 45, 75, 105, 135, 165From equation (1), the observed signal is produced through convolution of the source signal with the impulse response generated by the image-source model and then added with white Gaussian noise with(34)SNR=10log10P¯1+P¯22P¯W,where P¯1, P¯2, and P¯W denote the average power of the two microphones m1 and m2 and the additive noise, respectively. We discuss the performance in three scenarios with different reverberation times RT60 and dms: (i) RT60=0.2s, dms=2m; (ii) RT60=0.5s, dms=2m; and (iii) RT60=0.5s, dms=3.5m. Direct to reverberation ratio (DRR) which indicates the reverberation level is defined as follows [31]:(35)DRRm,n=10log10∑ihm,nDi2∑ihm,nRi2,where the numerator and denominator of the logarithmic function represent the total power of the direct wave component and the reverberation component of the impulse response between the n-th sound source and the m-th microphone, respectively. Then, the average DRRs of all the sources when N=6 in scenario (i), scenario (ii), and scenario (iii) are 3.19dB, −1.87dB, and −6.60dB, respectively.


## 7. Comparison of Angular Spectrum Recognition Degree

The parameter configuration in the angular spectrum construction step is presented in Table2, where γST is 5 dB, as suggested in [15], and other parameters are set empirically through the previous experiment. Figure 5 shows the normalized angular spectra of GCC and GCCTF when N=2 and SNR=10dB from scenario (i) to scenario (iii) in a single simulation, where the arrows in each subfigure indicate the true DOAs. The peak amplitudes of the two angular spectra are almost the same as the true DOAs in scenario (i) and scenario (ii). The local SNR tracking module in GCCTF can efficiently reduce the angular spectrum floor so that the peak amplitude deviations from the true DOAs are lower than those of GCC. In scenario iii, the growth of dms makes the distance between the direct wave peak and the first reverberant peak shorter, which results in the aggravation over the reverberation level. The spectrum floor is higher than scenario (ii), and the false peak marked by “x” has a high amplitude compared to the true DOA in GCC, resulting in an incorrectly estimated DOA. The onset detection module in GCCTF can prevent this phenomenon so that GCCTF can retain the correct estimation.Table 2
Parameter configuration in the angular spectrum construction step.
Parameter className or notationValueSTFTFrame length512Frame shift256LFFT512Angular spectrumθmin0.5°Local SNR trackingLW3ΓST5 dBβST0.01Onset detectionβOD0.45Coherence testC2ΓCT0.8Figure 5
Angular spectra of GCC and GCCTF whenN=2 and SNR=10dB in three scenarios: (a) scenario (i); (b) scenario (ii); (c) scenario (iii).
(a)(b)(c)GCCTF forms a more recognizable angular spectrum than GCC from scenario (i) to scenario (iii) in Figure5. To quantitatively indicate the recognition degree from a statistical perspective, we introduce the following mean precision (MPRE), which can be expressed as follows [17, 18]:(36)MPRE=1Isim∑i=1IsimNcoiN0.2i,where Ncoi and N0.2i denote the number of correctly estimated DOAs and the spectrum peaks whose normalized amplitudes are greater than 0.2 in the i-th simulation, respectively. The criterion for judging whether the DOA is estimated correctly is that the interval between the true value and the estimated value is less than Amin, which is 10°, as suggested in [9, 15].Figure6 shows the MPREs of GCC and GCCTF versus SNR from scenario (i) to scenario (iii). In each subfigure of Figure 6, MPREs improve with the growth of SNR, which is mainly due to the fact that the reduction of noise floor means the recognition degree improvement of angular spectrum. Horizontal comparison of all the subfigures in Figure 6 shows that MPREs tends to decrease from scenario (i) to scenario (iii), which is mainly due to the fact that the deterioration of reverberation will strengthen the effect of false peaks and thus decrease the recognition degree. GCCTF performs better than GCC under the same noisy and reverberant environment, which shows that TF filtering can indeed bring better recognition degree and then bring a positive impact on the subsequent localization and counting performance.Figure 6
Angular spectra of GCC and GCCTF whenN=2 and SNR=10dB in three scenarios: (a) scenario (i); (b) scenario (ii); (c) scenario (iii).
(a)(b)(c)


## 8. Comparison of the Localization and Counting Performances

The mean absolute estimated error (MAEE) is used to measure the localization and counting performance [9, 10]:(37)MAEE=1Isim∑i=1Isim1Nmaxi∑n=1Nmaxiθ^ni−θni,where θni and θ^ni denote the true and estimated DOA of the n-th sound source in the i-th simulation, respectively; and Nmaxi=maxNi,N^i, where Ni and N^i denote the numbers of true and estimated sound sources in the i-th simulation, respectively. Since Ni is not greater than Nmaxi, the excessively estimated DOAs are set to meet the following expression:(38)θ^ni−θni=Amin.


### 8.1. Choice of the Window Width in the MP Method

When using the MP method, the final localization and counting performance are affected by the choice of the dual-width. Through large numbers of simulations, we find that whenQ1, one half of the narrower window width, is chosen from 4 to 6, the maximum inner product can accurately determine the DOAs. In this case, the choice of Q2, the wider counterpart, exhibits a regular change with N. To illustrate this regularity, without loss of generality, we present the MAEE with Q1 fixed at 4 while Q2 varies from 4 to 8 (in intervals of 2), where the GCCTF angular spectrum is used. Based on the comprehensive consideration of algorithm efficiency and accuracy, the maximum number of loops Imax is set to 10, and the cut-off thresholds ΓMP in scenario (i), scenario (ii), and scenario (iii) are 0.51, 0.53, and 0.60, respectively.Figures7 and 8 show the MAEEs of GCCTF-MP in three scenarios when SNR=15dB and SNR=5dB, respectively. Q2=4 (single-width) performs the best when N=3 from scenario (i) to scenario (iii); then, the performance declines as the width increases.Figure 7
MAEEs of GCCTF-MP versusN when SNR=15dB in three scenarios: (a) scenario (i); (b) scenario (ii); (c) scenario (iii).
(a)(b)(c)Figure 8
MAEEs of GCCTF-MP versusN when SNR=5dB in three scenarios: (a) scenario (i); (b) scenario (ii); (c) scenario (iii).
(a)(b)(c)In scenario (i), whenN is 4, 5, and 6, the best performance is obtained with Q2 set to 6, 8, and 10, respectively. Thus, in a low reverberant environment (e.g., scenario (i)), a wider window width improves the performance of MP when there are more sound sources.The trends in scenario (ii) are similar, except that whenSNR=5dB and N=4, Q2=4 performs the best. In scenario (iii), when SNR=15dB and N=5, Q2=6 performs the best; when SNR=5dB and N is 4, 5, and 6, the best performance is obtained with Q2 set to 4, 4, and 6, respectively. Thus, increasing the window width results in sensitivity to the adverse environment with relatively strong reverberation and low SNR. However, the wider width Q2 shows an improvement when there are more sound sources in the same noisy and reverberant environment.


### 8.2. Comparison of the Localization and Counting Performances

To provide a comprehensive comparison of the localization and counting performance, three algorithms, GCC-PS, GCCTF-PS, and GCC-MP, are obtained similarly to GCCTF-MP by combining the angular spectrum with the localization and counting method. Based on the previous analysis, whereQ1 is set to 4, Table 3 presents the parameter configuration when using the PS and MP methods from scenario (i) to scenario (iii).Table 3
Parameter configuration when using PS and MP.
NotationConditionValueQ2Scenarios (i) and (ii);N=34Scenario (iii);N=3,4Scenario (iii);N=5;SNR≤5dBScenarios (i) and (ii);N=46Scenario (iii);N=5;SNR>5dBScenarios (i) and (ii);N=58Scenarios (i)–(iii),N=610ΓPSScenario (i)0.57Scenario (ii)0.63Scenario (iii)0.67ΓMPScenario (i)0.51Scenario (ii)0.53Scenario (iii)0.60Figures9–11 show the MAEEs versus SNR when N varies from 3 to 6 in scenario (i), scenario (ii), and scenario (iii), respectively.Figure 9
MAEEs of GCC-PS, GCCTF-PS, GCC-MP, and GCCTF-MP versus SNR whenN varies from 3 to 6 in scenario (i).Figure 10
MAEEs of GCC-PS, GCCTF-PS, GCC-MP, and GCCTF-MP versus SNR whenN varies from 3 to 6 in scenario (ii).Figure 11
MAEEs of GCC-PS, GCCTF-PS, GCC-MP, and GCCTF-MP versus SNR whenN varies from 3 to 6 in scenario (iii).In Figure9, the MAEEs ranges from 0.5° to 5.5°. As N increases, the MAEE increases due to aggravation of the mutual interference between sound sources. In each subfigure, the MAEE gradually decreases with increasing SNR. GCC-PS and GCCTF-MP perform the worst and best among the four algorithms. When SNR>0dB, the two algorithms using the MP method have lower MAEE than those using PS, and the two algorithms using the GCCTF angular spectrum have lower MAEE than those using GCC. When SNR=0dB, the MAEE of GCCTF-PS is lower than that of GCC-MP, which shows that the SNR tracking module can effectively reduce the TF bins with relatively low local SNR.In Figure10, medium reverberation degrades the performance of the four algorithms, with MAEE between 1° and 7°. The trend is similar to scenario (i), except when SNR≤10dB, GCCTF-PS performs better than GCC-MP. Compared to the turning point of 0 dB in scenario (i), in a medium reverberant environment (e.g., scenario (ii)), the onset detection module may have a positive impact on the final localization and counting.In Figure11, strong reverberation results in the worst performance for the four algorithms, with MAEE between 2° and 8°. The trend is similar to scenario (ii), except when N>4 and SNR varies from 0 to 15 dB, GCCTF-PS performs better than GCC-MP. Compared to the turning point of 10 dB in scenario (ii), in a relatively strong reverberant environment (e.g., scenario (iii)), the coherence test module may have a positive impact on the final localization and counting.Based on the results from Figures9 to 11, GCCTF can provide a more robust angular spectrum than GCC especially under the acoustically adverse environment. This is because that the TF bins seriously affected by low SNR, strong reverberation, and abundant sound sources have been removed by the three filtering modules. So the distortion of the angular spectrum can be effectively alleviated, and the increase in the false peak amplitude can be suppressed simultaneously. MP performs better than PS because PS may produce instable performance with severely distorted angular spectrum. GCCTF-MP, which uses both the GCCTF angular spectrum and MP method, is the most robust and accurate multiple sound source localization and counting algorithm among the four combined algorithms.


## 8.1. Choice of the Window Width in the MP Method

When using the MP method, the final localization and counting performance are affected by the choice of the dual-width. Through large numbers of simulations, we find that whenQ1, one half of the narrower window width, is chosen from 4 to 6, the maximum inner product can accurately determine the DOAs. In this case, the choice of Q2, the wider counterpart, exhibits a regular change with N. To illustrate this regularity, without loss of generality, we present the MAEE with Q1 fixed at 4 while Q2 varies from 4 to 8 (in intervals of 2), where the GCCTF angular spectrum is used. Based on the comprehensive consideration of algorithm efficiency and accuracy, the maximum number of loops Imax is set to 10, and the cut-off thresholds ΓMP in scenario (i), scenario (ii), and scenario (iii) are 0.51, 0.53, and 0.60, respectively.Figures7 and 8 show the MAEEs of GCCTF-MP in three scenarios when SNR=15dB and SNR=5dB, respectively. Q2=4 (single-width) performs the best when N=3 from scenario (i) to scenario (iii); then, the performance declines as the width increases.Figure 7
MAEEs of GCCTF-MP versusN when SNR=15dB in three scenarios: (a) scenario (i); (b) scenario (ii); (c) scenario (iii).
(a)(b)(c)Figure 8
MAEEs of GCCTF-MP versusN when SNR=5dB in three scenarios: (a) scenario (i); (b) scenario (ii); (c) scenario (iii).
(a)(b)(c)In scenario (i), whenN is 4, 5, and 6, the best performance is obtained with Q2 set to 6, 8, and 10, respectively. Thus, in a low reverberant environment (e.g., scenario (i)), a wider window width improves the performance of MP when there are more sound sources.The trends in scenario (ii) are similar, except that whenSNR=5dB and N=4, Q2=4 performs the best. In scenario (iii), when SNR=15dB and N=5, Q2=6 performs the best; when SNR=5dB and N is 4, 5, and 6, the best performance is obtained with Q2 set to 4, 4, and 6, respectively. Thus, increasing the window width results in sensitivity to the adverse environment with relatively strong reverberation and low SNR. However, the wider width Q2 shows an improvement when there are more sound sources in the same noisy and reverberant environment.


## 8.2. Comparison of the Localization and Counting Performances

To provide a comprehensive comparison of the localization and counting performance, three algorithms, GCC-PS, GCCTF-PS, and GCC-MP, are obtained similarly to GCCTF-MP by combining the angular spectrum with the localization and counting method. Based on the previous analysis, whereQ1 is set to 4, Table 3 presents the parameter configuration when using the PS and MP methods from scenario (i) to scenario (iii).Table 3
Parameter configuration when using PS and MP.
NotationConditionValueQ2Scenarios (i) and (ii);N=34Scenario (iii);N=3,4Scenario (iii);N=5;SNR≤5dBScenarios (i) and (ii);N=46Scenario (iii);N=5;SNR>5dBScenarios (i) and (ii);N=58Scenarios (i)–(iii),N=610ΓPSScenario (i)0.57Scenario (ii)0.63Scenario (iii)0.67ΓMPScenario (i)0.51Scenario (ii)0.53Scenario (iii)0.60Figures9–11 show the MAEEs versus SNR when N varies from 3 to 6 in scenario (i), scenario (ii), and scenario (iii), respectively.Figure 9
MAEEs of GCC-PS, GCCTF-PS, GCC-MP, and GCCTF-MP versus SNR whenN varies from 3 to 6 in scenario (i).Figure 10
MAEEs of GCC-PS, GCCTF-PS, GCC-MP, and GCCTF-MP versus SNR whenN varies from 3 to 6 in scenario (ii).Figure 11
MAEEs of GCC-PS, GCCTF-PS, GCC-MP, and GCCTF-MP versus SNR whenN varies from 3 to 6 in scenario (iii).In Figure9, the MAEEs ranges from 0.5° to 5.5°. As N increases, the MAEE increases due to aggravation of the mutual interference between sound sources. In each subfigure, the MAEE gradually decreases with increasing SNR. GCC-PS and GCCTF-MP perform the worst and best among the four algorithms. When SNR>0dB, the two algorithms using the MP method have lower MAEE than those using PS, and the two algorithms using the GCCTF angular spectrum have lower MAEE than those using GCC. When SNR=0dB, the MAEE of GCCTF-PS is lower than that of GCC-MP, which shows that the SNR tracking module can effectively reduce the TF bins with relatively low local SNR.In Figure10, medium reverberation degrades the performance of the four algorithms, with MAEE between 1° and 7°. The trend is similar to scenario (i), except when SNR≤10dB, GCCTF-PS performs better than GCC-MP. Compared to the turning point of 0 dB in scenario (i), in a medium reverberant environment (e.g., scenario (ii)), the onset detection module may have a positive impact on the final localization and counting.In Figure11, strong reverberation results in the worst performance for the four algorithms, with MAEE between 2° and 8°. The trend is similar to scenario (ii), except when N>4 and SNR varies from 0 to 15 dB, GCCTF-PS performs better than GCC-MP. Compared to the turning point of 10 dB in scenario (ii), in a relatively strong reverberant environment (e.g., scenario (iii)), the coherence test module may have a positive impact on the final localization and counting.Based on the results from Figures9 to 11, GCCTF can provide a more robust angular spectrum than GCC especially under the acoustically adverse environment. This is because that the TF bins seriously affected by low SNR, strong reverberation, and abundant sound sources have been removed by the three filtering modules. So the distortion of the angular spectrum can be effectively alleviated, and the increase in the false peak amplitude can be suppressed simultaneously. MP performs better than PS because PS may produce instable performance with severely distorted angular spectrum. GCCTF-MP, which uses both the GCCTF angular spectrum and MP method, is the most robust and accurate multiple sound source localization and counting algorithm among the four combined algorithms.


## 9. Conclusion

In this paper, GCCTF-MP, an algorithm for multiple sound source localization and counting, is proposed for noisy and reverberant environments. Three modules, local SNR tracking, onset detection, and a coherence test, are used to filter the GCC angular spectrum; the dual-width MP method is used to replace the amplitude comparison with the inner product and contribution removal. On the basis of the statistical indicators MPRE and MAEE, MP is shown to be a more accurate method than PS, and GCCTF is shown to be a more recognizable and robust angular spectrum, especially in environments with low SNR, strong reverberation, and abundant sound sources. The proposed GCCTF-MP, which uses both the GCCTF angular spectrum and MP method, is thus a robust and accurate multiple sound source localization and counting algorithm. Furthermore, we find that the final localization and counting performance when using the MP method is affected by the choice of the dual-width. A brief comparison when using a fixed narrower width and a different wider counterpart is presented. In practice, the environmental parameters are difficult to determine, and the number of sound sources is unknown. How to implement the width in an adaptive manner is a challenging problem that warrants further study.



---

*Source: 8937829-2020-09-07.xml*