2011 Joint Workshop on Hands-free Speech Communication and Microphone Arrays

May 30 - June 1, 2011

BLIND SOURCE SEPARATION OF MIXED SPEECH
IN A HIGH REVERBERATION ENVIRONMENTKeiju Iso†‡ Shoko Araki‡ Shoji Makino†
Tomohiro Nakatani‡ Hiroshi Sawada‡ Takeshi Yamada† Atsushi Nakamura‡†University of Tsukuba
1-1-1 Tennoudai, Tsukuba, Ibaraki, 305-8573, Japan
Tel: +81-29-853-6564, email: iso@mmlab.cs.tsukuba.ac.jp‡NTT Communication Science Laboratories, NTT Corporation,
2-4 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0237, JapanIn [4], Duong et al. assume that a source image including
the spatial characteristics of reverberant source signals canbe modeled by a complex Gaussian random variable, andpropose considering full-rank spatial covariance matricesthat model diffused late reverberation. They suggest amethod of BSS by estimating the parameters of thecovariance of the complex Gaussian random variable, andseparating the individual sources using a Wiener filter
designed using these parameters. To estimate the model
parameters, they use the expectation maximization (EM)algorithm. However, its convergence depends heavily on its
initial values. To obtain high separation performance, wehave to find a good set of initial parameters. In this paper,for the initial value setting, we propose employing theresults of the source separation method using frequency binwise clustering reported by Sawada et al., which does notconsider reverberation [5].

ABSTRACT
Blind source separation (BSS) is a technique for
estimating and separating individual source signals from amixed signal using only information observed by eachsensor. BSS is still being developed for mixed signals thatare affected by reverberation. In this paper, we propose
combining the BSS method that considers reverberation
proposed by Duong et al. with the BSS method reported bySawada et al., which does not consider reverberation, for theinitial setting of the EM algorithm. This proposed method
assumes the underdetermined case. In the experiment, wecompare the proposed method with the conventional method
reported by Duong et al. and that reported by Sawada et al.,and demonstrate the effectiveness of the proposed method.
Index Terms— blind source separation, reverberation,model parameter estimation

2. PROBLEM SETTING
1. INTRODUCTION
2.1. Mixture
Blind source separation (BSS) is a technique forseparating a mixed signal using only information observedby individual sensors, and estimating each source signal.
BSS techniques for speech signals are expected to find many
applications, such as hands free TV conferencing systems.
Many methods such as independent component analysis[1], and sparse component analysis [2] [3], have alreadybeen developed and have performed well for mixed signals
under low reverberation conditions. However, BSS for
mixed signals with strong reverberation needs to be
developed because most of previous methods assume ananechoic environment. In this paper, first of all, we discuss asource separation method that uses model parameter
estimation considering the reverberation described byDuong et al. [4], which we call "the full-rank method". After
that, we propose a method that solves a problem in the fullrank method.

978-1-4577-0999-9/11/$26.00 ©2011 IEEE

In a real environment, source signals observed using
sensors can be modeled by convolutive mixtures as follows.
Jxi (t )   hij (t ) s j (t   ) (i  1,, I )(1)

j 1 x i is an signal observed by sensor i, hij is the impulse
response from source j to sensor i. s j ( j  1,, J ) aresource signals. I and J are the numbers of sensors and source
signals, respectively. With a vector notation, (1) can be
expressed as follows:
JJj 1 j 1x(t )   h j ( ) s j (t   )   c j (t )36(2)whereFrom (6), the covariance matrix R c j can be divided intox(t )  [ x1 (t ), x2 (t ),, xI (t )]T , h j  [h1 j , h2 j ,, hIj ]T ,time-dependent signal variancec j  c1 , c2 ,, cJ  .
Tcovariancec j is a source image which is a sound signal containing

matrixRjv j and time-invariant

representingthespatialcharacteristics of sounds. Here if the observed signal
consists only of the direct sound, the rank of a covariancematrix R j will be one, but considering reverberation, thespatial characteristics such as reverberation.
The goal of BSS is to obtain a separated signal c j using
only the observed signals x . In this paper, we discuss theunderdetermined problem (I < J), where I and J are assumed
to be known.
This study employs a time-frequency domain
representation of the signals. This is because a voice signal
in the time-frequency domain is sparser than in the time
domain [6], and convolution in the time domain can beapproximated into a product in each frequency. An observed
signal x in the time-frequency domain is modeled asfollows.
Jx(n, f )   h j  f s j n, f  ( j  1,  , J )(3)full-rank method assumes that R j is full rank [4].

We estimate the model parameters   v j , R j bymaximizing the likelihood function as follows.
Px, c j ,    Px | c j ;  Pc j |  nf(7)jWith this likelihood function, we iteratively update themodel parameters v j , R j and the source image c j using
the EM algorithm.
Each step of the EM algorithm is calculated as follows.

j 1J  c j n, f 

Initialization: In [4], the initial values of the model
parameters are as follows.

j 1

c j n, f   h j  f s j n, f (4)R initj f 

Where h j  f  represents the transfer function from source j
to each sensor,s j n, f  and xn, f  and c j n, f 

represent the source signal, the observed signal and thesource image with a short-time Fourier transform signal
respectively. n is the time frame index and f is thefrequency.

(9)frequency component of each separated source ĉ j . The fullrank method calculates C j by clustering xn, f  with thehierarchical clustering approach [4].
E-step: Update Wiener filter W j for separation and source
image estimation ĉ j .

h j  f  and the source signal s j n, f  , is a randomvariable that has a complex Gaussian distribution with acovariance matrix R c j . The assumption is expressed in the 

p c j ; R c j  N 0, R c jwhere,(10)cˆ j n, f   Wj n, f xn, f (11)R c j n, f   v j n, f R j  f (13)JR x n, f    v j n, f R j  f R c j n, f   c j n, f c Hj n, f (14)j 1

 s j n, f  h j  f h Hj  f 2

 v j n, f R j  f Wj n, f   Rc j n, f R x1 n, f ˆ n, f   cˆ n, f cˆ H n, f   I  W n, f R n, f  (12)
Rcjjjjcj(5)

From (4) the covariance matrix R c j can be expressed asfollows.

v initj n, f   1

C j , where each cluster member corresponds to the time-

In this section, first, we outline the full-rank method [4],which estimates the source image using the model parameter
estimation. This approach assumes that the source imagec j n, f  , which is a product of the transfer function

(8)where C j denotes the total number of samples in cluster

2.2. Separation process

following formula.

1
Hxn, f xn, f C j x n , f C jM-step: Update model parameters v j and R j .
(6)

v j n, f  371ˆ n, f 
trace R j1  f R
cjI(15)

In the experiments, we separated two-channel mixed audio
signals with three speakers including reverberation, andmeasured the objective evaluation scores of the obtained
separated signals. The objective evaluation scores include
four measures proposed in [8].

1 N
1ˆ n, f (16)R jf  RcN n 1 v j n, f  jWith the iteration of these algorithms, separated signals areobtained by formula (11). Obtained signals are transformed
back to the time domain by inverse short-time Fouriertransformation.



3. PROPOSED METHOD
Instead of using hierarchical clustering to initialize theEM algorithm as in [4], our proposed method applies thesource separation method which uses the bin-wiseclassification [5]. Figure 1 shows a block diagram of the
proposed algorithm. In our proposed approach, at theinitialization stage, we utilize the bin-wise classification
method. In [5], the cluster C j is estimated according toinformation on the vectors

They are expressed in dB, and higher numbers, indicate
better performance.
4.2. Experimental condition
A. Experiment 1: Comparison with the full-rank method
Using audio data from SiSEC*2010, we compared theresults obtained with the proposed method with thosereported by Duong which were published in SiSEC2010.
The reverberation time of the room was 250 ms, and thedistance between two microphones was 5 cm. The sampling
frequency was 16 kHz, the FFT size was 2048 and theoverlap was 256. The proposed method was iterated 20times.
B. Experiment 2: Comparison with the bin-wise clustering
methodWe separated eight seconds of audio signal mixed withthree speakers, and 8 kHz sampling. We set the FFT size
1024. We evaluated the separation results for 1, 5, 10, 20,30, 40, and 50 iterations of the proposed method, under six
reverberation conditions (130 ~ 450 ms). We averaged thecalculated values for six combinations of speakers.

h j estimated from thenormalized norm of xn, f  and determining the mostdominant source for each time n. Since this method is known
to have high separation performance under low
reverberation conditions*, it is expected to provide us withgood initial values of C j . The proposed method utilizes thisinit

cluster C j for the initial value calculation of R j
initial value v

SDR (Signal to Distortion Ratio)
ISR (source Image to Spatial distortion Ratio)
SIR (Source to Interference Ratio)
SAR (Sources to Artifacts Ratio)(8). Theinit
is set in the same way as (9). In Fig. 1, thejmodel parameter estimation and Wiener filter estimation areexecuted in the same way as (10)-(16). The permutation ofthe frequency components of the separated signals werealigned with the method described in [7].

4.3. Experimental results
The results of experiments 1 and 2 are shown in Tables 1, 2and Figure 2 respectively.
Table 1: Results of experiment 1Algorithm
Full-rank method

Fig. 1: The flow of algorithm
4. EXPERIMENT AND EVALUATION

Bin-wise clustering

4.1. Experiments: Comparison with two conventional
methods
To evaluate the effectiveness of the proposed method, wecompare its performance with that of the full-rank methodwith initialization (8) (9) [4], and with the frequency binwise clustering method [5].

*SDRISR

SIR

SAR

sim1

0.1

6.0

-0.5

7.8

sim2

5.2

7.6

9.1

11.6

sim3

3.0

4.8

5.7

8.0

sim1

7.0

10.9

13.2

8.9

sim2

5.4

12.5

8.7

7.9

sim3

8.5

12.7

15.0

10.8

SiSEC: Signal Separation Evaluation Campaign(http://sisec.wiki.irisa.fr/tiki-index.php)

38

Proposed

sim1

7.8

11.1

11.8

11.6

sim2

6.4

10.4

9.3

10.1

sim3

9.2

12.9

13.2

13.7

We discuss the way in which the performance wasimproved by the proposed method compared with the twoconventional methods.
From Table 1, the SDR was poor with the full-rank method
and we can obtain a significant improvement by setting theinitial value in the proposed method. Table 2 shows thatalthough The SIR with the proposed method is slightly lower
than that of [5], the overall SDR performance is improved
due to the SAR improvement. From this result, we can saythat the sound quality is improved with the proposed method
while maintaining separation performance, and that theoverall performance is improved compared with the bin-wiseclustering method. We also confirmed the quality of theseparated sounds provided by each method by using an
informal listening test and the results were consistent withthe objective results.
Figure 2 shows that the SDR improvement for longer
reverberation was small compared with the bin-wiseclustering method which does not consider reverberation.
Therefore, we may be able to improve the performance if wecan model the reverberant signals more appropriately.

Table 2: Results of experiment 2
Algorithm / Iteration

SDR

ISR

SIR

SAR

Bin-wise clustering

6.8

12.3

11.9

8.9

Proposed / 1

5.4

8.7

6.0

13.4

5

5.9

9.4

8.0

11.44

10

7.1

10.9

10.1

11.36

20

7.72

12.1

11.1

11.65

30

7.68

12.5

11.2

11.76

40

7.8

12.82

11.4

11.83

50

7.9

12.84

11.5

11.80

5. CONCLUSIONSIn this paper, we proposed a source separation method thatcombines a model parameter estimation method considering
reverberation, and an initial value setting based on results
obtained with a frequency bin-wise clustering approach. We
confirmed the effectiveness of the proposed method
experimentally.
6. REFERENCES[1] S. Makino, S. Araki, R. Mukai, and H. Sawada, "Audio source
separation based on independent component analysis," in ISCAS 2004,
pp. V-668 – V-671, May 2004.
[2] S. Araki, H. Sawada, R. Mukai, and S. Makino, "Underdeterminedblind sparse source separation for arbitrarily arranged multiple
sensors," in Signal Processing, vol. 87, pp. 1833-1847, 2007.
[3] O. Yilmaz and S. Rickard, "Blind separation of speech mixtures via
time-frequency masking", in Signal Processing, vol. 52, pp. 1830-1847,
2004.
[4] N. Q. K. Duong, E. Vincent and R. Gribonval, "Under-determined
reverberant audio source separation using a full-rank spatial covariancemodel", IEEE Trans. on ASLP, vol. 18, no. 7, pp. 1830-1840, Sept.
2010.
[5] H. Sawada, S. Araki, and S. Makino, "Underdetermined convolutive
blind source separation via frequency bin-wise clustering andpermutation alignment," IEEE Trans. on ASLP, vol.19, no.3, pp.516527, Mar. 2011.
[6] P. Bofill and M. Zibulevsky, "Blind separation of more sources thanmixtures using sparsity of their short-time-Fourier transform, " in ICA
2000, pp.87-92, Jun. 2000.
[7] H. Sawada, S. Araki, and S. Makino, "Measuring dependence of binwise separated signals for permutation alignment in Frequency-domain
BSS," in ISCAS 2007, pp. 3247-3250, May 2007.
[8] E. Vincent, H. Sawada, P. Bofill, S. Makino, and J. P. Rosca, "First
stereo audio source separation evaluation campaign: data, algorithms
and results, " in ICA 2007, pp. 552-559, Sept. 2007.

Fig. 2: SDR of the proposed method in Experiment 2
for six reverberation conditions
In Table 1, we also show the results of the bin-wiseclustering method under the same conditions for reference.
The reverberation time is 130 ms in Table 2. Figure 2 shows
the SDR for each number of iterations of the proposed
method in experiment 2, for six reverberation times.
Iteration 0 indicates the results obtained using the initialvalue with the bin-wise clustering method. In Table 1, "sim"denotes "source image." Tables 1 and 2 (Fig. 2) show that
the SDR representing overall distortion increases with theproposed method. This indicated the effectiveness of theproposed method.
4.4. Discussion

39