➠➡
NEAR-FIELD FREQUENCY DOMAIN BLIND SOURCE SEPARATIONFOR CONVOLUTIVE MIXTURES
Ryo Mukai

Hiroshi Sawada

Shoko Araki

Shoji Makino

NTT Communication Science Laboratories, NTT Corporation
2–4 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619–0237, Japan{ryo,sawada,shoko,maki}@cslab.kecl.ntt.co.jp

ABSTRACTThis paper presents a method for solving the permutation
problem of frequency domain blind source separation (BSS)
when source signals come from the same or similar directions. Geometric information such as the direction of arrival (DOA) is helpful for solving the permutation problem, and a combination of the DOA based and correlation
based methods provides a robust and precise solution. However when signals come from similar directions, the DOA
based approach fails, and we have to use only the correlation based method whose performance is unstable. In thispaper, we show that an interpretation of the ICA solution
by a near-field model yields information about spheres on
which source signals exist, which can be used as an alternative to the DOA. Experimental results show that the proposed method can robustly separate a mixture of signals arriving from the same direction.
1. INTRODUCTION
Independent Component Analysis (ICA) [1] in the frequency domain is one of the most important and practicalmethods for Blind Source Separation (BSS) of convolutive
mixtures. A convolutive mixture in the time domain is converted into multiple instantaneous mixtures in the frequency
domain, and a complex-valued ICA algorithm for instantaneous mixtures can be applied for each frequency. Thisapproach has the advantage of fast convergence, which enables realtime processing [2]. However, a permutation ambiguity in the ICA solutions is a serious problem. We needto align separated signals so that a separated signal in thetime domain contains frequency components from the same
source signal. This problem is known as the permutation
problem of frequency domain BSS.
The other approach for separating convolutive mixtures
is time domain BSS, which does not suffer the permutation
problem, however it takes much more computational timethan frequency domain BSS [3]. Therefore we adopt thefrequency domain approach.

0-7803-8484-9/04/$20.00 ©2004 IEEE

Many methods have been proposed for solving the permutation problem. The use of geometric information, suchas beam patterns [4, 5, 6], direction of arrival (DOA) andsource locations [7], is one of the effective approaches. Another approach is based on the inter-frequency correlations[8, 9], which uses the inter-frequency correlations of output signal envelopes to align the permutations. However thecorrelation based method is not robust since a misalignment
at one frequency bin causes consecutive misalignments.
We have proposed a robust and precise method by combining the DOA based method and the correlation based
method, which almost completely solves the permutation
problem for two sources that come from different directions [10]. However the DOA based method fails in thefirst stage when the signals come from the same or similar
directions. In such cases we have to rely on the correlation
based method that is unstable.
In this paper, we show that the interpretation of the
ICA solution by a near-field model yields information about
spheres on which source signals exist. This information can
substitute for the DOA when signals come from the same
direction.
2. FREQUENCY DOMAIN BSS
When the source signals are s i (t)(i = 1, ..., N ), the signals
observed by microphone j are x j (t)(j = 1, ..., M ), and theseparated signals are y k (t)(k = 1, ..., N ), the BSS model
can be described as:
N
xj (t) = i=1 (hji ∗ si )(t),
(1)
M
(2)
yk (t) = j=1 (wkj ∗ xj )(t),where hji is the impulse response from source i to microphone j, wkj are the separating filters, and ∗ denotes the
convolution operator.
Figure 1 shows the BSS flow in the frequency domain.
A convolutive mixture in the time domain is converted into
multiple instantaneous mixtures in the frequency domain.
Therefore, we can apply an ordinary ICA algorithm [1] inIV - 49ICASSP 2004

➡

➡
Sourcesignals
s1

DFT

ICA ω

Permutation
problemtimes2

ri,jj  = 1.2
ri,jj  = 1.3
ri,jj  = 1.5 r

Scalingproblem IDFT

ri,jj  = 0.83
ri,jj  = 0.77
i,jj  = 0.67

time

freq

freq

W(ω)

P(ω)

1

D(ω)

Permutation
misalignment
Multiple instantaneous mixtures

Convolutive mixtures

0.5

Fig. 1. Flow of frequency domain blind source separation

z(m)
0

−0.5

the frequency domain to solve a BSS problem in a reverberant environment. Using a short-time discrete Fourier transform, the model is approximated as:X(ω, n) = H(ω)S(ω, n),(3)where ω is the angular frequency, n represents the frame index, H(ω) is the mixing system in the frequency domain,S(ω, n) = [S1 (ω, n), ..., SN (ω, n)]T is the source signal,
and X(ω, n) = [X1 (ω, n), ..., XM (ω, n)]T denotes the observed signals. The separating process can be formulated ineach frequency bin as:Y(ω, n) = W(ω)X(ω, n),(4)where Y(ω, n) = [Y1 (ω, n), ..., YN (ω, n)]T is the estimated source signal, and W(ω) represents the separating
matrix. W(ω) is determined so that Y i (ω, n) and Yj (ω, n)(i = j) become mutually independent.
The ICA solution suffers permutation and scaling ambiguities. This is due to the fact that if W(ω) is a solution, then D(ω)P(ω)W(ω) is also a solution, where D(ω)
is a diagonal complex valued scaling matrix, and P(ω) is
an arbitrary permutation matrix. We thus have to solve thepermutation and scaling problems to reconstruct separated
signals in the time domain.
There is a simple and reasonable solution for the scaling problem: D(ω) = diag{[P(ω)W(ω)] −1 }, which is obtained by the minimal distortion principle (MDP) [11], andwe can use it. In contrast, the permutation problem is complicated, and many solutions have been proposed.
3. PROPOSED METHOD
3.1. Invariant in ICA solution
If a separating matrix W(ω) is calculated successfully andit extracts source signals with scaling ambiguity, there is adiagonal matrix D(ω), and D(ω)W(ω)H(ω) = I holds.
Because the scaling matrix determined by the MDP does
not satisfy this equation, and D(ω) cannot be determinedin general, we cannot obtain H(ω) simply from the ICA
solution. However, the ratio of elements in the same column
Hji (ω)/Hj  i (ω) is invariable in relation to D(ω), and given1

pj pj 0.5
qi = [x, y, z]
−1
2

1.5

1

0.5




−1 

∆  [W ]ji ri,jj  = [W −1 ]  
ji

0

y(m)

−0.5

−1

−1.5

−2

0

x(m)

−0.5
−1

Fig. 2. Example of spheres determined by eq.(8) (p j =[0, 0.15, 0], pj  = [0, −0.15, 0])byHji (ω)[W−1 (ω)D−1 (ω)]ji[W−1 (ω)]ji
=
=
, (5)
−1
−1
Hj  i (ω)[W (ω)D (ω)]j  i[W−1 (ω)]j  iwhere [·]ji denotes the ji-th element of the matrix. We canestimate several types of geometric information related tosource signals by using this invariant. The estimated information can be used for solving the permutation problem.
Previously we have shown that the DOA of source signals can be estimated by comparing (5) with a far-fieldmodel [10, 12]. The interpretation of the ICA solution by anear-field model yields other geometric information as described below.
3.2. Estimation of sphere with ICA solution
When we adopt the near-field model, including the attenuation of the wave, H ji (ω) is formulated as:
Hji (ω) =−11ejωc (qi −pj ) ,
qi − pj 

(6)where pi represents the location of microphone j, q i is thelocation of source i, and c is the speed of wave propagation.
By taking the ratio of (6) for a pair of microphones j and j 
we obtain:qi − pj   jωc−1 (qi −pj −qi −pj )Hji (ω)=e. (7)Hj  i (ω)
qi − pj 
By using the modulus of (7) and (5), we have:qi − pj    [W−1 (ω)]ji =.
qi − pj 
[W−1 (ω)]j  i (8)

By solving (8) for q i , we have a sphere whose center O i,jj
and radius Ri,jj are given by:

IV - 50

Oi,jj = pj −1
(pj  − pj ),
2
ri,jj
 − 1

(9)

➡

➡
445 cm14

Reverberation time: 130 ms@500 Hz
Room height: 250 cm

SIR (dB)

180 cm12355 cm

30 cm

Mic.1

Mic.2

30 deg
S1 60 cm
S2

10
8Each of 12 source pairs

6

Average

150 cm4

225 cmMicrophones (omnidirectional, height: 135 cm)
Loudspeakers (height: 135 cm)∞

16
Threshold αCorrelation only

Fig. 4. Experimental results. SIRs are evaluated for 12 combinations of source signals with various values for thresholdparameter α.

Table 1. Experimental conditions

8 kHz
2s
Hanning1024 point (128 ms)
256 point (32 ms)
Infomax (complex valued)ri,jj
(pj  − pj ),Ri,jj =  2
ri,jj − 1

8Geometric information(estimated spheres) only

Fig. 3. Room layout
Sampling rate
Data length
Window
Frame length
Frame shift
ICA algorithm

12 4

(10)

where ri,jj = |[W−1 (ω)]ji /[W−1 (ω)]j  i |. Therefore, we
can estimate a sphere (Ôi,jj , R̂i,jj ) on which qi exists byusing the separation matrix W(ω) obtained by ICA and thelocations of the microphones p j and pj  . Figure 2 shows
an example of spheres determined by (8) for various ratios
ri,jj .
3.3. Solving permutation problem
The estimated spheres can be utilized for sorting or classifying the output signals of ICA, thus we can solve the permutation problem even when signals come from the sameor similar directions. However, geometric information suchas the DOA or the sphere tends to have a large error in a reverberant environment, especially for lower frequency bins.
Accordingly, we use the geometric information based solution only for frequency bins whose geometric information is highly reliable. Then we apply the correlation based
method to the rest of frequency bins. The correlation based
method solves the permutation problem by maximizing theinter-frequency correlation for neighboring or harmonic frequency bins [10].

in Fig. 3. The sources are located in the same direction forthe microphone pair. The reverberation time of the roomwas 130 ms at 500 Hz. Other conditions are summarized inTable 1. The experimental procedure is as follows.
First, we apply ICA to observed signals x j (t)(j = 1, 2),
and calculate separating matrix W(ω) for each frequency
bin. Then we estimate spheres by using W −1 (ω). We use
the radiuses of two spheres R̂1,12 and R̂2,12 , and the permutation is aligned so that R̂2,12 ≥ R̂1,12 . In order to evaluate the reliability of the solution provided by the estimatedspheres, we introduce a threshold parameter α ≥ 1, andwe accept solutions only for frequency bins that satisfy thecondition R̂2,12 /R̂1,12 ≥ α. We then apply the correlation
based method to the rest of frequency bins. The permutation
problem is solved simply by using the geometric information when α = 1, and simply by using the correlation when
α = ∞.
The performance is measured by the signal-to-inference
ratio (SIR). The portion of y k (t) that comes from si (t) is2
calculated by y ki (t) =j=1 (wkj ∗ hji ∗ si )(t), and theoutput SIR for y k (t) is defined as: 
SIROk = 10 log[ t ykk (t)2 / t ( i=k yki (t))2 ] (dB).
We define the SIR as an average of SIR O1 and SIRO2 in order to cancel out the effect of input SIR. We measured SIRs
for 12 combinations of source signals using two male andtwo female speakers and varying the threshold parameter α.
5. RESULTS AND DISCUSSION

4. EXPERIMENTS
We carried out experiments with 2 sources and 2 microphones using speech signals convolved with impulse responses measured in a room. The room layout is shown

Figure 4 shows the experimental results. When we solve
the permutation problem using only the estimated spheres(α = 1), the performance is insufficient. In contrast, theperformance we obtain using only the correlation (α = ∞)IV - 51➡

➠
1.51.5Filter for Y1 (the 1st row of W)

ing from the same direction. Some sound examples can befound on our web site [13].
The proposed method is valid for a microphone pairwith a large spacing. When we can use many microphonepairs and the DOAs are available through the use of microphone pairs with a small spacing, the source locations canbe estimated in more detail. We have succeeded in separating a mixture of six speech signals arriving from various directions, even when two of them come from the same
direction, by using microphone pairs with small and large
spacings [14, 15].

Gain (dB)
10

10

s

2
(jammer)
1.0 1

5

5

0

0
-5

−5

y(m)

−10
-10
−15
-15

0.50.5

s1

−20

-20

(target)

−25
-25
−30

-30
0
0−1.5
-1.5

1.51.5

−1

-1.0

−0.5

x(m) -0.5

Filter for Y2 (the 2nd row of W)

0

−35
-35

7. REFERENCES
Gain (dB)
10

10

s

2
(target)
1.0 1

5

5

0

0
-5

−5

y(m)

−10
-10
−15
-15

0.5

s1

0.5

−20

-20

(jammer)

−25
-25
−30

-30
0
0−1.5
-1.5

−1

-1.0

−0.5

x(m) -0.5

0

−35
-35

Fig. 5. Example of spatial gain patterns of separating filters(f = 1000 Hz)
is unstable. The combination of both methods yields goodand stable performance. These tendencies are similar to theresults we obtain when we use DOAs as geometric information [10].
We obtained good performance when the threshold parameter α was relatively large. When α was 8 to 16, the portion of frequency bins whose permutation was determinedby the geometric information was about 1/5 to 1/10.
Finally, we show the gain patterns of the separating system. Figure 5 shows the spatial gain patterns of separating
filters in one frequency bin (f = 1000 Hz). Microphone 1 isused as a reference, i.e., when a signal observed by microphone 1 is directly outputted, the gain is 0 dB. We can seethat the separating filter forms a spot null beam focusing on
the jammer signal.
6. CONCLUSION
The interpretation of the ICA solution by a near-field modelyields information about spheres on which source signals
exist. This information can be used as an alternative to
the DOA when signals come from the same or similar directions. Experimental results showed that the proposed
method can robustly separate a mixture of signals originat-[1] A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis, John Wiley & Sons, 2001.
[2] R. Mukai, H. Sawada, S. Araki, and S. Makino, “Robust
real-time blind source separation for moving speakers in aroom,” in Proc. of ICASSP’03, 2003, vol. 5, pp. 469–472.
[3] K. Matsuoka, Y. Ohba, Y. Toyota, and S. Nakashima, “Blindseparation for convolutive mixture of many voices,” in Proc.
IWAENC 2003, 2003, pp. 279–282.
[4] S. Kurita, H. Saruwatari, S. Kajita, K. Takeda, and F. Itakura,“Evaluation of blind signal separation method using directivity pattern under reverberant conditions,” in Proc. of
ICASSP’00, 2000, pp. 3140–3143.
[5] M. Z. Ikram and D. R. Morgan, “A beamforming approach topermutation alignment for multichannel frequency-domain
blind speech separation,” in Proc. of ICASSP’02, 2002,
vol. 1, pp. 881–884.
[6] L. C. Parra and C. V. Alvino, “Geometric source separation: Merging convolutive source separation with geometricbeamforming,” IEEE Trans. on Speech and Audio Processing, vol. 10, no. 6, pp. 352–362, Sep. 2002.
[7] V. C. Soon, L. Tong, Y. F. Huang, and R. Liu, “A robust
method for wideband signal separation,” in Proc. of ISCAS’93, 1993, pp. 703–706.
[8] J. Anemüller and B. Kollmeier, “Amplitude modulation
decorrelation for convolutive blind source separation,” in
ICA 2000, 2000, pp. 215–220.
[9] N. Murata, S. Ikeda, and A. Ziehe, “An approach to blindsource separation based on temporal structure of speech signals,” Neurocomputing, vol. 41, no. 1-4, pp. 1–24, 2001.
[10] H. Sawada, R. Mukai, S. Araki, and S. Makino, “A robust
and precise method for solving the permutation problem offrequency-domain blind source separation,” in Proc. Intl.
Symp. on Independent Component Analysis and Blind Signal
Separation (ICA2003), 2003, pp. 505–510.
[11] K. Matsuoka and S. Nakashima, “Minimal distortion principle for blind source separation,” in Proc. of Intl. Workshop
on Independent Component Analysis and Blind Signal Separation (ICA’01), 2001, pp. 722–727.
[12] H. Sawada, R. Mukai, and S. Makino, “Direction of arrival estimation for multiple source signals using independent component analysis,” in Proc. of ISSPA’03, 2003,
vol. 2, pp. 411–414.
[13] http://www.kecl.ntt.co.jp/icl/signal/mukai/demo/icassp2004/[14] R. Mukai, H. Sawada, S. de la Kethulle de Ryhove, S. Araki,
and S. Makino, “Array geometry arrangement for frequency
domain blind source separation,” in Proc. IWAENC 2003,
2003, pp. 219–222.
[15] http://www.kecl.ntt.co.jp/icl/signal/mukai/demo/iwaenc2003/

IV - 52