Eurospeech 2001 - Scandinavia

Separation and Dereverberation Performance of Frequency Domain
Blind Source Separation for Speech in a Reverberant Environment
Ryo Mukai, Shoko Araki, Shoji Makino
NTT Communication Science Laboratories
2-4 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0237, Japan{ryo,shoko,maki}@cslab.kecl.ntt.co.jp

Abstract
In this paper, we investigate the performance of an
unmixing system obtained by frequency domain BlindSource Separation (BSS) based on Independent Component Analysis (ICA). Since ICA is based on statistics, i.e.,
it only attempts to make outputs independent, it is noteasy to predict what is going on in a BSS system. We
therefore investigate the detailed components in the processed signals of a whole BSS system by measuring four
impulse responses of the system. In particular, we focuse on the direct sound and reverberation in the target
and jammer signals. As a result, we reveal that the direct
sound and reverberation of the jammer can be reduced
compared to a null beamformer (NBF), while the reverberation of the target cannot be reduced.

sourcesignalss1

h11

observedsignals

x1+

h21

2. Frequency domain BSS ofconvolutive mixtures
When the source signals are si (1 ≤ i ≤ N ), the signals
observed by microphone j are xj (1 ≤ j ≤ M ), and they1+y2

w12x2+

h22

mixing systemw22

unmixing system

Figure 1: Model of the BSS system
unmixed signals are yi (1 ≤ i ≤ N ), the model can be
described by the following equations:

1. Introduction
Blind Source Separation (BSS) is a technique that separates and extracts target signals only from mixture signals
observed without using information on the characteristicsof the source signals and the acoustic system [1, 2]. There
are several BSS algorithms such as Independent Component Analysis (ICA), which assumes the independency
of source signals. Most BSS algorithms are considerably effective for instantaneous (non-convolutive) mixtures of signals, and some attempts have been made toapply BSS to signals mixed in convolutive environments[3, 4]. However, it has also been pointed out that a sufficient performance cannot be obtained in environmentswith a lot of reverberation [5, 6].
In this paper, we examine the performance of a separation system obtained by frequency domain BSS basedon ICA. We focus our attention on the power of (1) the direct sound of the target signal, (2) the reverberation of thetarget signal, (3) the direct sound of the jammer signal,and (4) the reverberation of the jammer signal, and evaluate each power separately. As a result, it is shown thatfrequency domain BSS based on ICA can reduce the direct sound and reverberation of the jammer signal, while
it does not remove the reverberation of the target signal.

+w21

h12s2

w11

unmixedsignals

xj (t) =N
hji ∗ si (t)(1)

wij ∗ xj (t),

(2)

i=1

yi (t) =M

j=1

where hji is the impulse response from source i to microphone j, wij is the coefficient when the unmixing system
W is assumed as an FIR filter, and the ∗ is the convolution
operator.
In this paper, we consider a two-input, two-output
convolutive BSS problem, i.e., N = M = 2 (Fig. 1). In
addition, it is assumed that s 1 is separated to y1 , and s2
is separated to y2 .
Because it is possible to convert a convolutive mixture
in the time domain into an instantaneous mixture in thefrequency domain, frequency domain BSS is effective forseparating signals mixed in a reverberant environment.
Using a T -point short time Fourier transform for (1),
we obtainX(ω, m) = H(ω)S(ω, m).

(3)

We assume that the following separation has been
completed in a frequency bin ω:Y (ω, m) = W (ω)X(ω, m),(4)

where X(ω) = [X1 (ω), X2 (ω)]T is the observed signal in frequency bin ω, Y (ω) = [Y 1 (ω), Y2 (ω)]T is theEurospeech 2001 - Scandinavia

W i+1 = W i +(5)

η[diag(Φ(Y )Y H ) − Φ(Y )Y H ]W i
where · denotes the averaging operator, i is used to express the value of the i-th step in the iterations, and η isthe step size parameter. In addition, we define the nonlinear function Φ(·) asΦ(Y ) =1

1 + e−Re(Y )+j1

1 + e−Im(Y )(6)where Re(Y ) and Im(Y ) are the real and imaginary partsof Y , respectively.
In general, it is necessary to solve the permutation
problem and scaling problem when ICA is used. In ourexperiment, the effect of the permutation problem wasnegligible and so we did not coordinate the permutation. The problem of scaling was solved by adjusting thepower of the target signal in the output signal to 0 dB.

3. Evaluation method
The performance of BSS is usually evaluated by the ratioof a target-originated signal to a jammer-originated signal. This measure is reasonable for evaluating the separation performance, but is unsuitable for evaluating thedereverberation performance because of its inability todistinguish the direct sound and reverberation. Since wewant to know the detailed components in separated signals, i.e., the direct sound and reverberation of the targetand jammer, we take the following procedure,(1) estimate unmixing matrix W (ω) for each frequency.
(2) by using IFFT, transform frequency domain unmixing matrix W (ω) to time domain unmixing filter w ij (t).
(3) driving with the impulse as a source signal, measure four impulse responses, from s1 to y1 , s1 toy2 , s2 to y1 , and s2 to y2 .
(4) investigate the four impulse responses in detail andcompare them to the responses of a null beamformer (NBF).
3.1. Definitions of performance measurement factors
We consider a separated signal y1 , target signal s1 , andjammer signal s2 . When the target s1 is an impulse δ(t)and the jammer s2 = 0, we call the observed signal x1
as x1s1 [Fig. 2(a)], and y1 as y1s1 [Fig. 2(b)]. Similarly,
when s1 = 0 and s2 = δ(t), we call x1 as x1s2 , andy1 as y1s2 [Fig. 2(c)]. x1s1 is an impulse response from(a)s1 h11 x1
0.8(b)(c)h11w
11w11 y1s1y1h21 w12h12 w12s2 h22

x1s1

y1s1

y1s2

0-0.8
0-10-20-30-40(dB)estimated source signal, and W (ω) represents the unmixing matrix. W (ω) is determined so that Y 1 (ω, m)
and Y2 (ω, m) become mutually independent. The above
calculations are carried out for each frequency independently.
For the calculation of unmixing matrix W , we usean optimization algorithm based on the minimization of
the Kullback-Leibler divergence [7, 8]. The optimal W is
obtained by using the following iterative equation:

PIR

PTR
0125(ms)

PJ
0125(ms)

0125(ms)

Figure 2: Definitions of factors.
s1 to x1 by the mixing system H, and y 1s1 is an impulse
response from s1 to y1 by the whole system W·H. These
are calculated by using hij and wij as follows.
x1s1
x1s2
y1s1
y1s2

=
=
=
=

h11
h12w11 ∗ h11 + w12 ∗ h21
w11 ∗ h12 + w12 ∗ h22

(7)
(8)
(9)
(10)

From the viewpoint of source separation, we can consider
y1s1 as the direct and reverberant sound of target s 1 , andy1s2 as the remaining sound of jammer s2 .
To simplify the evaluation, we normalize h ji so thatthe power of the observed signals x 1s1 and x1s2 is equal
to 0 dB, and make the following definitions (Fig. 2).
• PIR : the power of the reverberant sound in x 1s1 ,• PT R : the power of the reverberant sound in y 1s1 ,• PJ : the power of y1s2 .
We also define the reduction of the reverberation of target signal R T and the reduction of jammer signal R J as
followsRTRJ==−(PT R − PIR )
−PJ .

(11)
(12)

4. Experiments
In order to examine what is separated by an unmixing
system based on ICA, and what remains as noise, we investigated impulse responses of the system. In frequency
domain BSS, it has been confirmed that the separation
performance changes according to the length of the frame[6], so we chose the frame length and the frame shift asparameters.
4.1. Conditions for the experimentsThe layout of the room we used to measure the impulse
responses of the mixing system H is shown in Fig. 3.
The reverberation time of the room was 300 ms. We used
a two-element array with inter-element spacing of 4 cm.

Eurospeech 2001 - Scandinavia

20
reduction ratio (dB)

5.73 m1.56 m3.12 m1.15 m2.15 mloudspeakers(height : 1.35 m)

30
404 cmmicrophones 1.15 m(height : 1.35 m)

y1s1

ICA(b)

y1s1

0-0.6
0.6(c)

y1s2(d)

y1s2

0-0.6

0ms

125ms

0ms

125ms

Figure 4: Target and jammer impulse responses of NBFand ICA◦

The speech signals arrived from two directions, i.e., −30and 40◦. The contribution of the direct sound of h 11 andh21 was 6.6 dB, that of h12 and h22 was 5.7 dB.
Two sentences spoken by two male speakers selectedfrom the ASJ continuous speech corpus for research wereused as the source signals. The lengths of these mixed
speech signals were about eight seconds each. We usedthe entire eight seconds of the mixed data for learning
according to (5).
In these experiments, we changed the frame length
T from 32 to 4096 and investigated the performance foreach condition. The sampling rate was 8 kHz, and theanalysis window was a Hamming window. The frame
shift S was T /2 and T /32, which correspond to double
and 32 times oversampling.
The number of iterations for (5) was 100, except
when S = T /2 and T = 1024, 2048, and 4096, where
the iteration was stopped at 70, 30, and 20, respectively,
because deterioration of the performance was observed.
4.2. Experimental results
Figures 4(a) and (c) show examples of impulse responses
y1s1 and y1s2 of the unmixing system obtained by a null
beamformer (NBF) that forms a steep null directivity pattern towards a jammer under the assumption of the jam-reduction ratio (dB)

NBF(a)

reduction ratio (dB)Figure 3: Layout of the room used in experiments. Reverberation time = 300 ms.
0.6

15RJ110RJ25020room height : 2.70 m(a) ICA, S=T/2

RT1RT2(b) ICA, S=T/32

RJ115RJ2
105015

RT1RT2(c) NBF10RJ1RJ250RT1RT23264128256 512 1024 2048 4096

frame length TFigure 5: Relationship between frame length and reduction ratio.

mer’s direction being known. Figures 4(b) and (d) areresults obtained by ICA.
For the target signal, we can see that the reverberation passes the system in both cases (NBF and ICA) inFigs. 4(a) and (b). Figure 4(c) shows that the direct soundof the jammer is removed, but the reverberation is not removed by NBF. This was expected. On the other hand,Fig. 4(d) indicates that ICA not only removes the direct
sound, but also reduces the reverberation of the jammer.
Figure 5 shows the relationship between the frame
length T and the reduction ratios R T and RJ defined by(11) and (12). R T 1 and RJ 1 are RT and RJ when thetarget signal is s1 . RT 2 and RJ 2 are results when thetarget signal is s2 . Figures 5(a) and (b) show results by
ICA when S = T /2 and S = T /32, respectively. For thesake of comparison, the performance of NBF is shown inFig. 5(c).
Note that these results are measured by the power of
impulse responses, and different from the noise reduction
rate (NRR) [6] measured by using a speech signal having
a highly colored spectrum. Our results indicate seemingly better values than the NRR of the speech signal.

Eurospeech 2001 - Scandinavia

0.6

observed
signalx1s1(≅x2s1)0

the reverberation, but because each tap of the filter haserrors that derive from the statistical method of ICA, theamount of errors increases. Moreover, the pre-echo noise
also increases, and this causes the poor performance.

0.06

06. Conclusion

-0.06

-0.6
0ms

0ms

250ms

0.6y1s1

T=51200.06

250ms

We investigated the performance of an unmixing system
obtained by frequency domain BSS based on ICA using
the impulse responses of target and jammer signals.
As a result, we revealed that ICA not only removes
the direct sound of the jammer signal, but also reduces
the reverberation, while the reverberation of the target is
not reduced.
The jammer reduction performance increases as theframe length becomes longer. However, an overly long
frame length decreases the performance due to accumulating errors. The performance of the target reverberation
reduction does not depend on the frame length and is as
poor as that of NBF.

frame length

0-0.06

-0.6
0ms

250ms

0.6

T=20480ms

y1s1

0250ms0.06
frame length

0-0.06

-0.6
0ms

250ms

0.6

T=4096

y1s1

00ms
0.06

250ms

7. Acknowledgements
frame length

We would like to thank Dr. Hiroshi Saruwatari for his
valuable discussions. We also thank Dr. Shigeru Katagiri
for his continuous encouragement.

0-0.06

-0.6
0ms

250ms

pre-echo

0ms

250ms

Figure 6: Jammer impulse response of a BSS system

5. Discussion
First, we discuss the jammer reduction ratio R J . WhenT ≤ 128, the reduction performance of BSS is as poor asthat of NBF, and when 256 ≤ T ≤ 2048, the reduction
ratio increases. In the case of T = 2048, S = T /32,
RJ 1=19.5 dB, RJ 2=16.6 dB. This is far greater than thecontribution of the direct sound, i.e., 6.6 dB and 5.7 dB.
This means that the system of ICA can reduce not only
the direct sound of the jammer but also the reverberant
sound of the jammer. In addition, comparing the resultsof S=T /2 and S=T /32 [Figs. 5(a) and (b)], we can seethat oversampling improves the jammer reduction ratio.
On the other hand, the reduction ratio of the reverberation of target R T is low, and does not vary through theentire frame length T . This means that dereverberation
was not achieved for the target signal.
From these results, it should be concluded that W is
not the approximation of the inverse system of H, but afilter that can eliminate the jammer signal [9].
Finally, we show the reason why the reduction ratioof jammer signal RJ declines when T is too long. Figure 6 shows the jammer signal’s impulse response y1s2 ,when T = 512, 2048, and 4096. The best performanceis obtained when T = 2048. In the case of T = 512, thelength of the unmixing system is much shorter than thelength of the reverberation, accordingly, the reverberation longer than the frame cannot be reduced at all. On
the other hand, when T = 4096, which is longer than thereverberant time, the unmixing system can wholly cover

8. References[1] A. J. Bell and T. J. Sejnowski, “An informationmaximization approach to blind separation and blind
deconvolution,” Neural Computation, Vol. 7, No. 6,pp. 1129–1159, 1995.
[2] S. Haykin, ed., Unsupervised adaptive filtering, John
Wiley & Sons, 2000.
[3] T. W. Lee, A. J. Bell, and R. Orglmeister, “Blindsource separation of real world signals,” Neural Networks, Vol. 4, pp. 2129–2134, 1997.
[4] J. Xi and J. P. Reilly, “Blind separation and restoration of signals mixed in convolutive environment,” inProc. ICASSP97, pp. 1327–1330, 1997.
[5] M. Z. Ikram and D. R. Morgan, “Exploring permutation inconsistency in blind separation of speech
signals in a reverberant environment,” in Proc.
ICASSP2000, pp. 1041–1044, 2000.
[6] S. Araki, S. Makino, T. Nishikawa, andH. Saruwatari, “Fundamental limitation of frequency
domain blind source separation for convolutive
mixture of speech,” in Proc. ICASSP2001, 2001.
[7] S. Ikeda and N. Murata, “An informationmaximization approach to blind separation andblind deconvolution,” in Proc. ICA99, pp. 365–370,
1999.
[8] S. Kurita, H. Saruwatari, S. Kajita, K. Takeda, andF. Itakura, “Evaluation of blind signal separation
method using directivity pattern under reverberantconditions,” in Proc. ICASSP2000, pp. 3140–3143,
2000.
[9] S. Araki, S. Makino, R. Mukai, and H. Saruwatari,“Equivalence between frequency domain blindsource separation and frequency domain adaptive
null beamformers,” in Proc. Eurospeech2001, 2001,(submitted).