➠➡
BLIND EXTRACTION OF A DOMINANT SOURCE SIGNALFROM MIXTURES OF MANY SOURCES
Hiroshi Sawada

Shoko Araki

Ryo Mukai

Shoji Makino

NTT Communication Science Laboratories, NTT Corporation
2-4 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0237, Japan{sawada,shoko,ryo,maki}@cslab.kecl.ntt.co.jp

ABSTRACTThis paper presents a method for enhancing a dominant target
source that is close to sensors, and suppressing other interferences.
The enhancement is performed blindly, i.e. without knowing thenumber of total sources or information about each source, suchas position and active time. We consider a general case where thenumber of sources is larger than the number of sensors. We employ
a two-stage processing technique where a spatial filter is first employed in each frequency bin and time-frequency masking is thenused to improve the performance further. To obtain the spatial filter we employ independent component analysis and then select thecomponent of the target source. Time-frequency masks in the second stage are obtained by calculating the angle between the basis
vector corresponding to the target source and a sample vector. The
experimental results for a simulated cocktail party situation werevery encouraging.
1. INTRODUCTION
The technique for estimating individual source components fromtheir mixtures at sensors is known as blind source separation (BSS)[1]. In some applications such as brain imaging or wireless communications, it makes sense to extract as many source components
as possible, because many sources are equally important. However, in audio applications such as speech enhancement, the significance of each source is not necessarily equal. We often want
to extract only one source that is close to sensors, has a dominant
power, and/or has interesting features.
This paper presents a method for extracting a source signal ofinterest and suppressing other interferences blindly. Let us formulate the task. Suppose that a target source s1 and other interference
sources sk , k = 2, . . . , N are convolutively mixed and observed at
M sensors
xj (t) =

N 
k=1

l

hjk (l) sk (t − l), j = 1, . . . , M ,(1)where hjk (l) represents the impulse response from source k tosensor j. The goal is to have an output signal
y1 (t) =lhJ1 (l) s1 (t − l)(2)which is the component of s1 measured at sensor J. The task
should be performed only with the M observed signals. The number of sources N is unknown and may be larger than M .
The first problem is how to extract a target source s1 blindly.
Even if N could be larger than M , independent component analysis (ICA) [2] with an N = M assumption produces M components
which maximize an ICA criterion such as non-Gaussianity. We assume that a target source s1 is non-Gaussian, close to sensors, and

0-7803-8874-7/05/$20.00 ©2005 IEEE

dominant in the mixtures. Therefore, it is expected that one of the
M components corresponds to s1 whose ICA criterion is high.
We apply ICA in the time-frequency domain. The reason isthat it is efficient and also it fits time-frequency masking, which
will be discussed in the next paragraph. An additional operationis the selection of the s1 component in every frequency bin. It has
been reported that the selection of a component with maximum
kurtosis works well when the target is speech and interferences arebabble source [3]. However, we consider a case where interferences are also speech. Thus, we exploit the information of basis
vectors produced by ICA. In Sec. 2.3, we propose a new idea that
improves our previously reported methods [4, 5].
The next issue is the fact that some interference still remains
in the target component in a case where N > M . Post filtering[3, 6] can be used to reduce such residuals. However, it needs additional adaptation where the step size should be controlled basedon the short-term power of the target. Another approach is timefrequency masking [7–11], which is effective for sources having
sparseness in the time-frequency domain, such as speech. The performance of time-frequency masking depends on how well we canspecify the time-frequency slots where a target source is active.
A simple way to specify such slots is to calculate the phase and/or
amplitude difference of sensor observations [7, 8]. In [10], the output of a beamformer is also used to improve mask accuracy. We
propose a new criterion for specifying masks in Sec. 2.5. It is basedon how close a sample vector is to the basis vector corresponding
to a target. The closeness is calculated in a whitened space where
the basis vectors corresponding to interferences are expected to be
far from that of the target.
The next section describes our proposed method. Section 3
shows experimental results, and Section 4 concludes this paper.
2. THE PROPOSED METHOD
2.1. Frequency domain operations
Figure 1 shows the flow of the method proposed here. First, timedomain signals xj (t) sampled at frequency fs are converted intofrequency-domain time-series signals xj (f, τ ) with an L-point shorttime Fourier transform (STFT):
xj (f, τ ) =L/2−1r=−L/2xj (τ + r) win(r) e−j2πf r ,(3)

where f = 0, L1 fs , . . . , L−1
fs is a frequency, win(r) is a winL
dow that tapers smoothly to zero at each end, such as a Hanningwindow 21 (1 + cos 2πr), and τ is a new index representing time.
L
The remaining operations are performed in the frequency domain. There are two advantages to this. First, the convolutive

III - 61ICASSP 2005

➡

➡
STFT

ICA

Scaling &Spatial filter

Permutation

Time-frequency
masking

ISTFT

Fig. 1. Flow of the proposed method

mixtures in (1) can be approximated as instantaneous mixtures ineach frequency bin:xj (f, τ ) =

N

k=1

hjk (f )sk (f, τ ),(4)

where hjk (f ) is the frequency response from source k to sensor j,and sk (f, τ ) is a frequency-domain time-series signal of sk (t) obtained by the same operation as (3). The frequency-domain counterpart of (2) is
y1 (f, τ ) = hJ1 (f )s1 (f, τ ),(5)

where J should be the same for all frequency bins f . The second
advantage is that the sparseness of a source signal becomes prominent in the time-frequency domain if the source is colored such as
speech. The possibility of sk (f, τ ) being close to zero is muchhigher than that of sk (t).
Through several operations, which will be discussed in the following subsections, we have an output ỹ1 (f, τ ), which should beclose to (5) in each frequency bin. At the end of the flow, we havean output y1 (t) by an inverse STFT (ISTFT) :
L−1
l
l
1
ỹ1 ( fs , τ ) exp(j 2π fs r).
y1 (τ + r) =L·win(r) l=0
L
L

2.3. Permutation
The next operation is to find i for each frequency f such that ai (f )corresponds to h1 (f ). This is considered to be the permutation
problem of frequency-domain BSS. Integrating the basis vectorai (f ) and signal envelope |yi (f, τ )| information solves the problem effectively [4]. We also employ this approach here.
In the rest of this subsection, we discuss how to exploit thebasis vector ai (f ). The former methods estimate geometric information (direction [4, 5] and distance [5]) about the sources byselecting two elements from ai (f ), and then cluster the directionsor the distances to solve the permutation problem. However, thedirections and distances are clustered separately, and such estimations depend on the selection of two elements. Consequently, different kinds of estimations by different pairs of elements need tobe reconciled. To solve this issue, we here propose a new methodthat considers all the elements of ai (f ) simultaneously.
The new idea is to normalize all basis vectors ai (f ), i =1, . . . , M , for all frequency bins f = 0, L1 fs , . . . , L−1
fs suchLthat they form clusters, each of which corresponds to each source.
The normalization is performed by selecting a reference sensor J
and calculatingāji (f ) ← |aji (f )| exp j
2.2. Independent component analysis (ICA)
Let us have a vector notation of the mixing model (4) :x(f, τ ) =

N

k=1

hk (f )sk (f, τ ),(6)y(f, τ ) = W(f )x(f, τ ),where y = [y1 , . . . , yM ]T is a vector of separated signals andW = [w1 , . . . , wM ]H is an M × M separation matrix. In the experiments shown in Sec. 3, we calculated W by using a complexvalued version of FastICA [2], and improved it further by using
InfoMax [12] combined with the natural gradient [13] whose nonlinear function is based on the polar coordinate [14].
By calculating the inverse of W
[a1 , · · · , aM ] = W−1 , ai = [a1i , . . . , aM i ]T ,(7)

we have the decomposition of xx(f, τ ) =M
i=1

ai (f )yi (f, τ ),(8)

where ai is a basis vector. Since s1 is assumed to be a dominant
non-Gaussian source, it is expected that one of y1 , . . . , yM corresponds to s1 and therefore one of a1 , . . . , aM corresponds to h1 .

(9)where c is the propagation velocity and d is the maximum distance
between the reference sensor J and a sensor ∀ j ∈ {1, . . . , M }.
The rationale of this operation will be explained afterwards. Then,
we apply unit-norm normalizationT

where x = [x1 , . . . , xM ] is a sample vector and hk = [h1k , . . . ,hM k ]T is the vector of frequency responses from source sk to all
sensors. Independent component analysis (ICA) is used as a firststep to identify the vector h1 of a dominant source s1 .
Even though the number of independent components N may
be larger than the number of sensors M , we employ ICA by assuming that N is equal to M :arg[aji (f )/aJi (f )]4f c−1 dāi (f ) ← āi (f ) / ||āi (f )||(10)

for āi (f ) = [ā1i (f ), . . . , āM i (f )]T . The next step is to find M
clusters C1 , . . . , CM formed by normalized vectors āi (f ). The
centroid ck of a cluster Ck is calculated by
ck ←



ā∈Ck

ā/|Ck |, ck ← ck /||ck ||,where |Ck | is the number of vectors in Ck . The criterion of clustering is to minimize the total sum J of the squared distances between cluster members and their centroid
J =

M

k=1

Jk , Jk =



ā∈Ck

||ā − ck ||2 .

(11)This minimization can be performed efficiently with the k-meansclustering algorithm [15].
This paragraph explains the reason why normalized basis vectors āi (f ) form a cluster for a source. Let us approximate themulti-path mixing model (1) by using a direct-path mixing model


q(f )
hjk (f ) =exp j 2πf c−1 (djk − dJk ) ,(12)
djk
where djk > 0 is the distance between source k and sensor j. We
assume that the attenuation q(f )/djk depends on both the distanceand a frequency-dependent constant q(f ) > 0, and that the delay(djk −dJk )/c depends on the distance normalized with the reference sensor J. By considering the permutation and scaling ambi-

III - 62➡

➡
445 cmguity of ICA, a basis vector and its elements are represented as
a i = α i hk ,aji = αi hjk ,

(13)

70 cm

where αi is a scalar representing the scaling ambiguity, and index
k that may be different from index i represents the permutation
ambiguity. Substituting (12) and (13) into (9) and (10) yields



1
djk D

exp j



π (djk − dJk )
, D=
2
d



M
i=1

70 cm

50 cm

a

dik

25 cm

100 cmc

1
355 cmāji (f ) =Target source
candidatesb2which is independent of frequency, and dependent only on the positions of the sources and sensors. From the fact that maxj,k |djk −dJk | ≤ d, an inequality holds:

Mic. 2
Mic. 1

4 cm
4 cm

Interference
sources

4 cm
Mic. 3−π/2 ≤ arg[āji (f )] ≤ π/2.
This property is important for the distance measure (11), since |ā−ā | increases monotonically as | arg(ā) − arg(ā )| increases.
After we have found M clusters C1 , . . . , CM , we need to
identify a cluster that corresponds to a dominant source s1 . Since
we assume that s1 is close to the sensors, the mixing model (12)is more valid for s1 than for the other sources. Therefore, we decide that a cluster that has the minimum variance σk2 = Jk /|Ck |corresponds to s1 .

120 cm100 cm
25 cm

100 cm

25 cm• Reverberation time: RT60 = 130 ms• Room height: 250 cm
Loudspeaker (135 cm height)

Microphone (135 cm height, omni-directional)

Fig. 2. Experimental conditions

2.4. Scaling and Spatial filter
Here the columns of (7) have been permuted such that a1 (f ) corresponds to h1 (f ). Then, we solve the scaling ambiguity in (8) :ai yi (τ ) = (αi ai )(yi (τ )/αi ), for any complex scalar αi .
This is easily solved byai ← ai /aJi or equivalently wi ← [W−1 ]Ji wi
to make aJi = 1 for a selected sensor J. The reason can be seen
by comparing the h1 term in (6) and the a1 term in (8), and taking(5) into consideration

Fig. 3. Masking functions: parameters (θT , g) = (π/5, 20) and(π/3, 16) were used for 2- and 3-microphone cases, respectively

h1 s1 (τ ) ≈ a1 y1 (τ ) = a1 hJ1 s1 (τ ) ⇔ h1 ≈ a1 hJ1 .
Now the permutation and the scaling ambiguities are solved. The
extraction of s1 by a spatial filter is performed byy1 (τ ) = w1H x(τ )= w1H h1 s1 (τ ) +

N
k=2

(14)
w1H hk sk (τ ).

(15)

If N ≤ M , w1 satisfies w1H hk = 0, ∀ k ∈ {2, . . . , N } and makes

the second term zero. However, we assume that the number ofsources N is generally larger than M . In this case, there exists a
set K ⊆ {2, . . . , N } such that 
w1H hk = 0, ∀ k ∈ K. Thus, y1 (τ )contains an unwanted residual k∈K w1H hk sk (τ ).
2.5. Time-frequency masking
The purpose here is to have another output ỹ1 (τ ) that contains
less power of the residual k∈K w1H hk sk (τ ) than y1 (τ ). This isperformed by time-frequency maskingỹ1 (f, τ ) = M(f, τ ) · y1 (f, τ ),(16)

where 0 ≤ M(f, τ ) ≤ 1 is a mask specified for each timefrequency slot (f, τ ). We specify masks based on the angle θ1 (f, τ )between a1 (f ) and x(f, τ ) calculated in the space transformed
by a whitening matrix V(f ) = R−1/2 , R = x(τ ) x(τ )H τ .
Let z(f, τ ) = V(f )x(f, τ ) be whitened samples and b1 (f ) =V(f )a1 (f ) be the basis vector of y1 (τ ) in the whitened space.
The angle is calculated byθ1 (f, τ ) = cos−1|bH
1 (f ) · z(f, τ )|||b1 (f )|| · ||z(f, τ )||(17)

for each time-frequency slot. Then, we calculate a mask by using
a logistic function
M(θ1 (f, τ )) = 1/(1 + eg(θ1 −θT ) ),(18)

where θT and g are parameters specifying the transition point andits steepness, respectively. The smaller θT , the less power of theresidual appears in ỹ1 but the more musical noises sound in y1 .
The effectiveness of the above operation depends on the sparseness of sources. If we assume that the possibility of sk (f, τ ) being
close to zero is very high, (6) can be approximated asx(f, τ ) ≈ hk (f )sk (f, τ ), k ∈ {1, . . . , N },(19)

where k depends on each time-frequency slot (f, τ ). Let us consider the whitened-space counterpart of (19), with distinguishing
the cases where s1 is the only active source and other cases:z(τ ) ≈ Vh1 s1 (τ ) ≈ Va1 y1 (τ )(20)z(τ ) ≈

(21)

N

k=2

Vhk sk (τ ).

If the number of sources N is equal to or less than the numberof sensors M , vectors Vhk in whitened space are orthogonal toeach other. Even if N > M , the vector b1 = Va1 ≈ Vh1of a dominant source s1 tends to have large angles with the othervectors Vhk , k = 2, . . . , N . Therefore, calculating the angle (17)provides information about whether or not s1 is the only active
source at a time-frequency slot (f, τ ).

III - 63

➡

➠
Target position: a

SIR improvement (dB)22

Target position: b

22

20

20

20

18

18

18

16

16

16

14

14

14

12

12

12

10

10

10

8

8

8

6

6

6

4

−6

−4

−2

0

2

Input SIR (dB)

4

4

−6

−4

−2

0

2

Target position: c

22

4

4

Only spatial filter by ICA, 2 mics
Only spatial filter by ICA, 3 mics
With T−F masking, 2 mics
With T−F masking, 3 mics

−6

Input SIR (dB)

−4

−2

0

2

4

Input SIR (dB)

Fig. 4. SIR improvements for various input SIRs

3. EXPERIMENTS
We performed experiments to examine the effectiveness of the proposed method. We measured impulse responses under the conditions shown in Fig. 2. The speaker positions simulated a cocktail party situation. Mixtures at microphones were made by convolving the impulse responses and 6-second speeches sampled at
8 kHz. For each setup, we selected one of the three speakers (a,b, c) as a dominant target source, and the remaining two speakers were kept silent. The six speakers away from the microphones
were used as interferences for every setup. We set parameters of(18) as shown in Fig. 3, which were empirically proved to be good.
Figure 4 shows experimental results for various input SIRs,
which were controlled by multiplying a positive constant for all interferences equally. The vertical axis shows the SIR improvements
obtained solely by using a spatial filter made by ICA, and by using the combination of the spatial filter and time-frequency (T-F)
masking. The frame size L of STFT (3) was 1024 (128 ms). SIR
improvements depend heavily on the position of the target source.
Position a was good for enhancement, whereas c was a hard position as a lot of interferences came from the similar directions.
The improvement realized by using a spatial filter was almost thesame independent of input SIRs. This means that ICA and thefollowing permutation alignment worked similarly well for various input SIRs. In contrast, the further improvement achieved bytime-frequency masking depends on the input SIRs. This is why
the vectors Vhk of interferences in the whitened space approaches
that Vh1 of a target source, as the interference power increases.
With the masking functions shown in Fig. 3, musical noises hardly
sound in the output signals, especially 3-microphone cases.
4. CONCLUSION
We have presented a method for extracting a dominant target sourceand suppressing interferences. The process of ICA and following permutation alignment extracts the target source by a spatial
filter, and estimates the basis vector corresponding to the target
source. The performance depends on the accuracy of the permutation alignment, which has been improved by the new idea shownin subsection 2.3. Time-frequency masking in the second stage reduces the power of the residual caused by the limitations of a spatial filter. It exploits the sparseness of sources. The angle between
the basis vector corresponding to the target source and a sample
vector gives information about whether or not the target is active.

The experiments showed good results for extracting a dominantsource out from six interferences only with 2 or 3 microphones, ina room whose reverberation time was 130 ms.
5. REFERENCES[1] S. Haykin, Ed., Unsupervised Adaptive Filtering (Volume I: Blind
Source Separation), John Wiley & Sons, 2000.
[2] A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component
Analysis, John Wiley & Sons, 2001.
[3] S. Y. Low, R. Togneri, and S. Nordholm, “Spatio-temporal processing for distant speech recognition,” in Proc. ICASSP 2004, May
2004, vol. I, pp. 1001–1004.
[4] H. Sawada, R. Mukai, S. Araki, and S. Makino, “A robust and precise
method for solving the permutation problem of frequency-domain
blind source separation,” IEEE Trans. Speech Audio Processing, vol.
12, pp. 530–538, Sept. 2004.
[5] R. Mukai, H. Sawada, S. Araki, and S. Makino, “Frequency domain
blind source separation using small and large spacing sensor pairs,”
in Proc. ISCAS 2004, May 2004, vol. V, pp. 1–4.
[6] R. Mukai, S. Araki, H. Sawada, and S. Makino, “Removal of residualcrosstalk components in blind source separation using LMS filters,”
in Proc. NNSP 2002, Sept. 2002, pp. 435–444.
[7] M. Aoki, M. Okamoto, S. Aoki, H. Matsui, T. Sakurai, and
Y. Kaneda, “Sound source segregation based on estimating incident
angle of each frequency component of input signals acquired by multiple microphones,” Acoustical Science and Technology, vol. 22, no.
2, pp. 149–157, 2001.
[8] S. Rickard, R. Balan, and J. Rosca, “Real-time time-frequency based
blind source separation,” in Proc. ICA2001, Dec. 2001, pp. 651–656.
[9] S. Araki, S. Makino, A. Blin, R. Mukai, and H. Sawada, “Underdetermined blind separation for speech in real environments withsparseness and ICA,” in Proc. ICASSP 2004, May 2004, vol. III,
pp. 881–884.
[10] N. Roman and D. Wang, “Binaural sound segregation for multisource
reverberant environments,” in Proc. ICASSP 2004, May 2004, vol. II,
pp. 373–376.
[11] D. Wang, “On ideal binary mask as the computational goal of auditory scene analysis,” in Speech Separation by Humans and Machines,
P. Divenyi, Ed., pp. 181–197. Kluwer Academic Publishers, 2004.
[12] A. Bell and T. Sejnowski, “An information-maximization approachto blind separation and blind deconvolution,” Neural Computation,
vol. 7, no. 6, pp. 1129–1159, 1995.
[13] S. Amari, “Natural gradient works efficiently in learning,” NeuralComputation, vol. 10, no. 2, pp. 251–276, 1998.
[14] H. Sawada, R. Mukai, S. Araki, and S. Makino, “Polar coordinate
based nonlinear function for frequency domain blind source separation,” IEICE Trans. Fundamentals, vol. E86-A, no. 3, pp. 590–596,
Mar. 2003.
[15] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, Wiley
Interscience, 2nd edition, 2000.

III - 64