IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.12 DECEMBER 2005

3593

PAPER

Subband-Based Blind Separation for Convolutive Mixtures of
SpeechShoko ARAKI†a) , Shoji MAKINO† , Members, Robert AICHNER†† , Nonmember, Tsuyoki NISHIKAWA††† ,and Hiroshi SARUWATARI††† , Members

SUMMARY
We propose utilizing subband-based blind source separation (BSS) for convolutive mixtures of speech. This is motivated by the
drawback of frequency-domain BSS, i.e., when a long frame with a fixed
long frame-shift is used to cover reverberation, the number of samples ineach frequency decreases and the separation performance is degraded. In
subband BSS, (1) by using a moderate number of subbands, a suﬃcientnumber of samples can be held in each subband, and (2) by using FIR filters in each subband, we can manage long reverberation. We confirm thatsubband BSS achieves better performance than frequency-domain BSS.
Moreover, subband BSS allows us to select a separation method suited toeach subband. Using this advantage, we propose eﬃcient separation procedures that consider the frequency characteristics of room reverberation andspeech signals (3) by using longer unmixing filters in low frequency bandsand (4) by adopting an overlap-blockshift in BSS’s batch adaptation in low
frequency bands. Consequently, frequency-dependent subband processing
is successfully realized with the proposed subband BSS.
key words: blind source separation, speech separation, convolutive mixtures, subband processing, frequency dependent processing

1.

Introduction

Blind source separation (BSS) is an approach that estimatesoriginal source signals si (n) using only information on themixed signals x j (n) observed in each input channel. We consider the BSS of speech signals in a real environment, i.e.,the BSS of convolutive mixtures of speech. In a real environment, signals are filtered by the acoustic room channel.
To separate such complicated mixtures, we need to estimate
the unmixing filters of several thousand taps.
Several methods have been proposed for achieving the
BSS of convolutive mixtures [1], [2], most of which utilize
independent component analysis (ICA). To solve the convolutive BSS problem, algorithms in time and frequency domains have been proposed [3]–[11].
In time-domain BSS, ICA is directly applied to convolutive mixtures, and unmixing FIR filters are directly estimated (e.g. [3]–[5]). Therefore, the independence of outputs can be evaluated directly. However, the convergence
Manuscript received April 14, 2005.
Manuscript revised August 2, 2005.
Final manuscript received August 26, 2005.
†
The authors are with the NTT Communication Science Laboratories, NTT Corporation, Kyoto-fu, 619-0237 Japan.
††
The author is with the University Erlangen-Nuremberg,
Cauerstrasse 7, 91058 Erlangen, Germany.
†††
The authors are with the Graduate School of Information Science, Nara Institute of Science and Technology, Ikoma-shi, 6300192 Japan.
a) E-mail: shoko@cslab.kecl.ntt.co.jp
DOI: 10.1093/ietfec/e88–a.12.3593

of most time-domain BSS algorithms is generally slower
than that of frequency-domain methods because the adaptation of such long filters is very complex. Computational
complexity is also a problem. Moreover, most time-domain
BSS algorithms have another problem: the whitening eﬀect.
Since most time-domain BSS algorithms were designed fori.i.d. signals, such algorithms try to make output signals both
spatially and temporally independent [3], [7]. When applying such time-domain BSS algorithms to mixtures of speech
signals, the output speech signals are whitened and soundunnatural.
By contrast, in frequency-domain BSS, mixtures areconverted into the frequency domain, and ICA is applied
to instantaneous mixtures in each frequency (e.g. [8]–[11]).
Although we can greatly reduce computational complexity
by using frequency-domain BSS, frequency-domain BSS algorithms have inherent problems, namely, permutation andscaling problems, which result in the estimated source signal
being recovered with a diﬀerent permutation and gain in different frequency bins. Some solutions have been provided
for these problems [8], [11]–[14].
Furthermore, we have shown that performance becomes poor with frequency-domain BSS when using a long
frame to estimate a long unmixing filter that can coverrealistic reverberation [15], [16]. In a real environment,
since impulse response changes momentarily, it is therefore
preferable to estimate unmixing filters using adaptation datathat are as short as possible. However, when using a longer
frame for a few seconds of speech mixtures to convert signals into the frequency domain, the number of samples ineach frequency bin becomes small, and therefore, we cannot correctly estimate the statistics in each frequency bin.
This means that, in such a case, independence is not evaluated correctly. This is our strongest reason for employing
our subband-domain BSS method.
Motivated by these facts, we propose utilizing a BSS
method that employs subband processing, hereafter called
subband BSS. With subband BSS, observed signals are converted into the subband domain with a filterbank and thenseparated in each subband using a time-domain BSS algorithm. Then unmixed signals in each subband are synthesized to obtain fullband unmixed signals. With this method,since we can choose a moderate number of subbands, we
can maintain a suﬃcient number of samples in each subband. The subband system also allows us to estimate FIR
filters as unmixing filters in each subband. Moreover, as thec 2005 The Institute of Electronics, Information and Communication Engineers
Copyright IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.12 DECEMBER 2005

3594

unmixing filter length in each subband is shorter than thatfor time-domain BSS, it is easier to estimate unmixing filters than in time-domain BSS. Therefore, we can obtain unmixing filters long enough to cover reverberation. That is,
the subband BSS approach copes with both the frequencydomain approach’s diﬃculty in estimating statistics and thetime-domain technique’s diﬃculty in adapting many parameters. Confirming this point is one of our aims of this paper.
Subband BSS has other advantages. First, its permutation problem is less serious than in frequency-domain BSS.
This is because the permutation problem does not occur ineach subband and, therefore, there are fewer permutation
problems in subband BSS. Second, subband BSS can mitigate the whitening eﬀect, which is troublesome in a timedomain BSS algorithm usually designed for i.i.d. signals, bylimiting it in each subband.
Previous studies have used subband processing for BSS[17]–[20] to reduce computational complexity. By contrast,
our main aim is to maintain the number of samples in eachsubband so that independence is properly evaluated. Although some authors [19], [20] utilized a scalar coeﬃcientfor the unmixing system in each subband, in this paper we
use FIR filters for this purpose so as to estimate suﬃciently
long unmixing filters to cover reverberation.
Furthermore, subband BSS allows us to select a separation method suited to each subband. Using this advantage,
we propose an eﬃcient separation procedure that considers the frequency characteristics of room reverberation andspeech signals. Generally speaking, an impulse response
is usually longer in low frequency bands than in high frequency bands. This makes the separation in low frequency
bands diﬃcult. Moreover, because speech signals have highpower in low frequency bands, the separation performancein low frequency bands dominates the speech separation performance. Therefore, it is very important to improve separation performance in low frequency bands for speech separation. In this paper, we propose to utilize longer unmixing
filters and the overlap-blockshift technique in low frequency
bands.
The organization of this paper is as follows. Section 2
presents the framework for BSS of convolutive mixtures ofspeech. In Sect. 3, we describe the configuration of subband
BSS and mention implementation issues. Section 4 reports
experiments conducted to confirm the validity of subband
BSS. In Sect. 5, we propose ways of improving the low frequency subband performance in which the signal to interference ratio (SIR) is worse than at high frequencies by considering the frequency characteristics of room reverberationand speech signals. The final section concludes this paper.
2.

BSS of Convolutive Mixtures

2.1 Model Description
In real environments, signals are aﬀected by reverberation and observed by microphones. Therefore, N s signals
recorded by Nm microphones are modeled as

Fig. 1x j (n) =BSS system configuration (when N s = Nm = 2).
Ns 
P


h ji (k)si (n − k + 1) ( j = 1, · · · , Nm ), (1)i=1 k=1

where si is the source signal from source i, x j is the signal
observed by microphone j, and h ji is the P taps impulse
response from source i to microphone j.
To obtain unmixed signals, we estimate the unmixing
filters wi j (k) of Q taps, and the unmixed signals are obtained
as below:yi (n) =QNm 

wi j (k)x j (n − k + 1) (i = 1, · · · , N s ). (2)j=1 k=1

The unmixing filters are estimated so that the unmixed signals become mutually independent.
The BSS block diagram is shown in Fig. 1 for N s =Nm = 2. In this paper we consider the case of N s = Nm =N sm .
2.2 Frequency-Domain BSS and Related Problems
2.2.1 Frequency-Domain BSS
A frequency domain approach to convolutive mixturestransforms the problem into an instantaneous BSS problem in each frequency [8]–[11]. Using T -point short time
Fourier transformation for (1), we obtain the approximate
time-frequency representation of mixtures,X(ω, m) = H(ω)S(ω, m) (m = 0, · · · , Lm − 1),(3)

where ω denotes the frequency bin, m represents thetime dependence of the short time Fourier transformation(STFT), Lm is the number of data samples in each frequency
bin, S(ω, m) = [S 1 (ω, m), · · · , S Nsm (ω, m)]T is the source signal vector, and X(ω, m) = [X1 (ω, m), · · · , XNsm (ω, m)]T is theobserved signal vector. We assume that (N sm × N sm ) mixing
matrix H(ω) is invertible and that H ji (ω)  0. The STFT isusually executed by applying a window function of length
T . In this paper, we call this T the frame size for STFT.
The unmixing process can be formulated in a frequency
bin ω:Y(ω, m) = W(ω)X(ω, m) (m = 0, · · · , Lm − 1),(4)

where Y(ω, m) = [Y1 (ω, m), · · · , YNsm (ω, m)]T is the estimated source signal vector, and W(ω) represents an (N sm ×

ARAKI et al.: SUBBAND-BASED BLIND SEPARATION FOR CONVOLUTIVE MIXTURES OF SPEECH

3595

N sm ) unmixing matrix at frequency bin ω. Here, we assume
that the STFT frame size T is equal to the unmixing filter
length Q. The unmixing matrix W(ω) is determined so that
outputs Yi (ω, m) become mutually independent. This calculation is carried out independently at each frequency.
2.2.2 Problem of Frequency-Domain BSS
To handle long reverberation, we need to estimate a long unmixing filter wi j (k) of Q taps using learning data that are as
short as possible. If the filters are relatively short, we cannot reduce the reverberant components of interferences thatare longer than the filters and this has a detrimental eﬀect
on the separation performance [21]. On the other hand, withbatch adaptation, it is desirable to estimate unmixing filters
using adaptation data that are as short as possible, because
an impulse response changes momentarily in a real environment. We therefore have to estimate long unmixing filterswith speech data of short length.
We have however verified in [16] that when employing
a long frame with a fixed long frame shift for several seconds of data to prepare an unmixing filter long enough tocover reverberation, it becomes diﬃcult to maintain a sufficient number of data samples to estimate the statistics ineach frequency bin. This makes the estimation of statistics
diﬃcult. In particular, independence assumption between
source signals seems to collapse [16]. Therefore, we cannot
obtain suﬃcient separation performance with a long framewith frequency-domain BSS.
3.

Subband Based BSS

Motivated by the above frequency-domain BSS problem, we
propose utilizing subband BSS. With subband BSS, we canchoose a moderate number of subbands and therefore maintain a suﬃcient number of samples in each subband. Subband BSS also allows us to estimate short FIR filters as unmixing filters in each subband, due to the down-sampling
procedure at the subband analysis stage. Therefore, weshould be able to obtain an unmixing filter long enough tocover reverberation. Moreover, as the unmixing filter lengthin each subband is shorter than that for time-domain BSS,
it is easier to estimate unmixing filters than in time-domain
BSS. That is, the subband BSS approach oﬀers a compromise between a time-domain technique, which is usually
computationally complex and usually converges slower thana frequency-domain counterparts, and a frequency domain
technique, which has diﬃculty estimating statistics.
3.1 Configuration of Subband BSS
The subband BSS system is composed of three parts: a
subband analysis stage, a separation stage, and a subband
synthesis stage (Fig. 2) [22], [23]. For the subband analysis/synthesis system, we utilize a polyphase filterbank [24]with oversampling [25] and single side band (SSB) modulation, which is widely used in echo canceller area [26], [27].

Thanks to the oversampling we can reduce the aliasing distortion, and thanks to the SSB modulation we can utilize any
existing real-number BSS algorithm in each subband.
First, in the subband analysis stage, input signals x j (n)are divided into N subband signals X j (k, m) (k = 0 · · · , N −1), where k is the subband index, m is the time index,
and N is the number of subbands (0–2π). Here we useda polyphase filterbank [24], that has the form of a generalized discrete Fourier transform filterbank [28]. Furthermore,
to execute BSS on real-valued signals, we also used single
sideband (SSB) modulation/demodulation [28] in the analysis/synthesis stage. Since signals are band-limited in eachsubband, we can employ decimation at the down-sampling
rate R. To reduce the aliasing problem, we used a downsampling rate of R < N. In this paper, SSB-modulated
subband signals were not critically sampled, but two-times
oversampled. That is, the down-sampling rate R was given
by R = N4 . The low-pass filter used in the analysis filternπ) of length 6N. By using SSB
bank was f (n) = sinc( N/2
modulation, we obtain SSB modulated real-valued signals
X Sj S B (k, m) in each subband.
Then, time-domain BSS is executed on X Sj S B (k, m) ineach subband in the separation stage. As SSB modulation is
performed in the analysis stage, we can implement a timedomain BSS algorithm without expanding it into a complex
value version. Since we employ down-sampling, short FIR
filters of length Q/R are suﬃcient to separate the subband
signals in each subband. Thus SSB modulated unmixed signals YiS S B (k, m) are obtained in each subband.
Finally, in the subband synthesis stage, unmixed signals yi (n) are obtained by synthesizing each unmixed signal
YiS S B (k, m). The low-pass filter used in the synthesis filternπ) of length 6R.
bank was g(n) = sinc( R/2
3.2 Time-Domain BSS Implementation for a Separation
Stage
We can use any time-domain BSS algorithm for subband
BSS. Here, we describe the algorithm used in our experiment. In addition, this section describes how to design theinitial value of time-domain BSS for each subband and how
to solve the scaling and permutation problems.
3.2.1 Time-Domain BSS Algorithm
In this paper, we used an algorithm based on time-delayed
decorrelation for non-stationary signals [4], [29], [30]. Relying on the non-stationarity and non-whiteness of the source
signals, this algorithm simultaneously minimizes the crosscorrelation of output signals for some time lags for all analysis blocks. We estimate FIR filters as the separation filters
wkij (m) in each subband k. We write them in a matrix formwk (m) where its i j component is wkij (m) for convenience.
The adaptation rule of the i-th iteration isIEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.12 DECEMBER 2005

3596

Fig. 2 System configuration of subband BSS. TDBSS denotes time-domain BSS. LPF denotes low
pass filter. A 2 × 2 case is depicted.

wki+1 (m) = wki (m)α {(diagRby (0))−1 (diagRby (m))
BS b=0
BS −1+− (diagRby (0))−1 Rby (m)} ∗ wki (m),(5)

where Rby (τ) represents the covariance matrix of outSB
puts y(m) ≡ [Y1S S B (k, m), · · · , YNS sm(k, m)]T in the b-th(b=0, · · · , B− 1) analysis block with time delay τ, [i.e.,Ly(b SL +t)yT (b SL +t−τ)], α denotes a step-size
Rby (τ) = L1 t=1
parameter, ∗ denotes a convolution operator, L is the block
length, and S is the blockshift rate.
Note that the algorithm we used here is a batch algorithm, i.e., the algorithm runs by using all the data on each
iteration.
3.2.2 Initial Value Design of Unmixing Filters
The initial value of the unmixing filters is very important forthe convergence of time-domain BSS. Moreover, this initialization mitigates the permutation problem in frequency
and subband BSS. As the initial value of the unmixing filters w, we can use constraint null beamformers [31]. This isFig. 3

Setup of a null beamformer.

based on the fact that the BSS solution behaves as adaptive
beamformers, which form a spatial null towards a jammer
direction [32]. Based on this, we design null beamformers
towards possible sound directions and utilize them as our
initial values for BSS adaptation.
Here, we assume a linear microphone array with aknown microphone spacing. First, we assume that the mixing system H(ω) represents only the time diﬀerence ofsound arrival τ ji with respect to the midpoint between the
microphones (Fig. 3). This H(ω) is written in the frequency
domain as follows:ARAKI et al.: SUBBAND-BASED BLIND SEPARATION FOR CONVOLUTIVE MIXTURES OF SPEECH

3597

H(ω) = H(2π f ) exp ( 2π f τ11 )..
= 
.
exp ( 2π f τNsm 1 )
· · · exp ( 2π f τ1Nsm ) 

..
..
 ,
.
.

· · · exp ( 2π f τNsm Nsm )(6)dwhere τ ji = cj sin θi , d j is the position of the j-th microphone, θi is the direction of the i-th source as an initial value,and c is the speed of sound. Note that these d j values neednot be precise because this H(ω) is used only for the initialization of BSS. Note also that the precise directions ofsources, which are not given in a blind scenario, are not required for initialization. That is, θi values can be set at very
rough approximations, e.g., ±60◦ for the 2 × 2 case (i.e., leftor right position, for example).
Then we calculate the inverse of H(ω) at each frequency, W(ω) = H−1 (ω) and convert the elements Wi j (ω)
of this W(ω) into the time domain, wi j (n) = IFFT(Wi j (ω)).
This is the null beamformer that forms nulls towards θi , andthis is the initial value for time-domain BSS. We then obtain
the initial value in each subband by using subband analysis
on these wi j (n).
3.2.3 Solving Permutation and Scaling Problems
Thanks to the initial value mentioned in Sect. 3.2.2, we did
not encounter the permutation problem in our experiments.
If it arises, it can be solved by reordering the row of estimated unmixing filters wk (m) so that the null of the directivity pattern obtained by wk (m) is sorted and forms a null inalmost the same direction in all subbands [12], [33]. We canalso solve the permutation problem by sorting the row of theestimated unmixing filter wk (m) so that the cross-correlationof separated signals in adjacent subbands is maximized.
The scaling problem did occur in our experiments.
That is, the estimated source signal components had diﬀerent gain in diﬀerent subbands. To solve it, we can also use
the directivity pattern calculated with unmixing filters [34].
Our scaling method was as follows:i) Synthesize wk (m) to obtain w(n) in the time-domain
and then obtain W(ω) using a discrete Fourier transform (DFT).
ii) Draw the directivity gain pattern of W(ω) [34] and obtain the estimated signal directions θi (i = 1, · · · , N sm )from the minimum of each directivity pattern. WhenN sm ≥ 3, they can be easily estimated using the method
proposed in [35]. If the permutation problem is observed, solve it by reordering the W(ω) row so that theθi values are sorted.
iii) Make null beamformers by using (6) with the estimatedθi in step ii) and by calculating W(ω) = H−1 (ω). We
call this null beamformer W NBF (ω).
iv) Calculate the inverse DFT of W NBF (ω) and perform subband analysis to obtain wkNBF (m).
v) Rescale wk (m) so that ||wkij (m)|| = ||wkNBFi j (m)||, where||x(m)|| means mQk x2 (m) and Qk is the unmixing filter
length in the k-th subband.

Fig. 44.

Layout of room used in experiments.

Basic Experiments for Subband BSS

4.1 Experimental Setup
We undertook separation experiments using speech data
convolved with impulse responses measured in a real environment for the 2 × 2 case. The impulse responses were
measured in the room shown in Fig. 4. Reverberation time
T R was 300 ms. Since the sampling rate was 8 kHz, 300 ms
corresponds to 2400 taps. As original speech, we used two
sentences spoken by two male and two female speakers. Investigations were carried out for six combinations of speakers. The lengths of these mixed speech signals were about
eight-seconds each. We used the first three seconds of themixed data for learning, and we separated the entire eight
second data.
To evaluate the performance, we used the signal to interference ratio (SIR), defined as
SIRi = SIROi − SIRIi


(7)

2
n yisi (n)SIROi = 10 log  
2n ( ji yis j (n))
 2n xksi (n)SIRIi = 10 log  ,
2n ( ji xks j (n))
where yis j is the output of the whole system at yi when onlys j is active, and xksi = hki ∗si (∗ is a convolution operator, k =i in our experiments). SIR is the ratio of a target-originatedsignal to jammer-originated signals.
4.2 Subband System
For subband analysis and synthesis, we used a polyphasefilterbank [24] with single sideband (SSB) modulation/demodulation [28], which was mentioned in Sect. 3.1.
Here, the number of subbands N was 64 and the downsampling rate R was 16(= N/4). We chose this numberof subbands N so that the down-sampling rate of subband
BSS corresponded to that of the conventional frequencydomain BSS of frame size T = 32 with half frame shift(see Sect. 4.3).

IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.12 DECEMBER 2005

3598

For the time-domain algorithm used in subband BSS,
we estimated unmixing filters wi j of 64 and 128 taps in eachsubband. Step-size for adaptation α was 0.02, and the number of blocks B was fixed at 20 for three seconds of speech.
We adopted θi =±60◦ as the initial values of the unmixing
filters (see Sect. 3.2.2).
We measured the signal to distortion ratio (SDR) to
evaluate the subband analysis-synthesis system. SDR is defined as Lδ 2b (t − D)
SDR = 10 log Lδ t
[dB],
(8)
2
t {b(t − D) − a(t)}where system input b(t) = δ(t − L2δ ), Lδ is the length ofthe delta function, D is the delay caused by low pass filters (LPF) in the analysis and synthesis stages, and a(t)is the output (impulse response) of the subband analysissynthesis system. SDR was 59.2 dB. This distortion causedby subband analysis and synthesis can be ignored because
the separation performance SIR (7) is at most 15 dB (seeSect. 4.5.1) and thus masks this distortion.
4.3 Conventional Frequency-Domain BSS
The frequency-domain BSS iteration algorithm was a natural gradient based algorithm [3]∆W i (ω) = ηdiag Φ(Y)Y H  −Φ(Y)Y H  W i (ω),

where Y=Y(ω, m), superscript H denotes a conjugate transpose, and x(m)
Lm −1denotes the time-average with respect totime m: L1m m=0x(m). Subscript i expresses the value ofthe i-th step in the iterations, η is a step-size parameter,and Φ(·) is a nonlinear function. As the nonlinear function Φ(·), we used Φ(Y) = tanh(g · abs(Y))e jarg(Y ) [36],where g is a parameter to control nonlinearity and we utilized g = 100. As an initial value of the unmixing matrix,
we utilized W(ω) = H−1 (ω) with θi =±60◦ (see Sect. 3.2.2).
We fixed the frame shift at half the STFT frame size
T so that the number of samples in the time-frequency domain were the same. To solve the scaling and permutation
problems, we used the blind beamforming algorithm proposed by Kurita et al. [33]: first, from the directivity patternobtained by W(ω), we estimate the source directions and reorder the row of W(ω) so that the directivity pattern forms
a null toward the same direction in all frequency bins, andthen we normalize the row of W(ω) so that the gains of thetarget directions become 0 dB.
Note that we used a time-average of Y(ω, m) of threeseconds for adaptation, i.e., we used a batch algorithm. It
should also be noted that if we fix the data length and frame
shift at half the frame size, the number of samples Lm ofsequences Y(ω, m) in each frequency bin depends on frame
size T : roughly speaking, Lm ∝ (data length)/T .
4.4 Conventional Fullband Time-Domain BSS
We also examined fullband time-domain BSS. The algorithm was the same as that used in subband BSS, i.e., (5).

Fig. 5 Separation performance of frequency-domain BSS (white bars),
subband BSS (black bars) and fullband time-domain BSS (gray bars).
“CC”: average correlation coeﬃcient. T R = 300 ms.

In this case, output signal vector y(n) was the signal in time
domain [y1 (n), · · · , yNsm (n)]T . We used values of α = 0.002
and B = 20. To obtain the initial condition of the unmixing
filters, we also utilized W(ω) = H−1 (ω) with θi =±60◦ andconverted it into time domain (see Sect. 3.2.2).
In fullband time-domain BSS, output speech signals
are distorted and whitened (see [37] and Sect. 4.6). We evaluated the SIR values after compensating for this whitening
eﬀect [31].
4.5 Results
4.5.1 Separation Performance of Subband BSS
In order to confirm the superiority of subband BSS overfrequency- and time-domain BSS, we compared the separation performance of subband BSS with that of frequencydomain BSS.
Figure 5 shows the separation result SIR and the valueof the averageNcorrelation coeﬃcient between source signals|rk |, where N is the number of subbands for
CC(N) = N1 k=1
subband BSS or frame size for frequency-domain BSS andrk is the correlation coeﬃcient between source signals of ak-th frequency/subband.
For frequency-domain BSS, the parameter was the
STFT frame size T . In Fig. 5, T is shown by the x axis. For
subband BSS, we used unmixing filters wk (m) of 64 and 128
taps in each subband, corresponding to 1024 and 2048 taps
in a fullband, respectively. They are shown as “sub1024”
and “sub2048” in Fig. 5, respectively. N = 64 subbands
with decimation R = 16 correspond to T = 32 in frequencydomain BSS with regard to the down-sampling rate. The
number of learning data samples in the time-frequency domain was the same for subband and frequency-domain BSS.
With frequency-domain BSS, CC becomes large andthe independent assumption seems to collapse as frame size
T becomes large because the number of samples in eachfrequency bin becomes small. Therefore, the performancedegraded when we used unmixing filters of 2048 taps (i.e.,frame size T = 2048). With fullband time-domain BSS,

ARAKI et al.: SUBBAND-BASED BLIND SEPARATION FOR CONVOLUTIVE MIXTURES OF SPEECH

3599

on the other hand, the CC was very small, and we obtained
good results when the unmixing filter length was 1024 (see“full1024” in Fig. 5). However, when we utilized the unmixing filter length of 2048 (see “full2048” in Fig. 5), it became
diﬃcult to estimate unmixing filters, and performance wasdegraded.
In contrast, we achieved better separation performancein subband BSS even when we estimated unmixing filtersof 2048 taps. Moreover, in subband BSS, we were able toconfirm that the CC value was suﬃciently small. From theCC values, we argue that the independence assumption held
well in subband BSS. Another possible reason for the superior performance of subband BSS is that the permutation
problem does not arise in the subbands. This point will be
discussed in the next subsection.
4.6 Discussion
In the experiments, we saw that we can maintain the number of samples in each subband and obtain better separation
performance.
Moreover, using subband BSS, we obtained separated
signals with less distortion than when using fullband timedomain BSS. When using the usual time-domain BSS algorithm, the output signal spectrum is flattened [37] because
we are removing the time dependence of the speech signals. These whitened speech signals sound unnatural. In
contrast, because this whitening eﬀect is limited to each subband, it can be diminished by subband BSS. Figure 6 shows
an example of separated speech with time-domain BSS andsubband BSS. The separated signal is whitened using timedomain BSS, while the shape of the spectrum holds wellusing subband BSS.
Furthermore, in general, the permutation problem occurs in frequency-domain BSS and subband BSS; spectral
components of sources are recovered in a diﬀerent order at
diﬀerent frequencies, although we did not face such a problem in our experiments due to the initialization with null
beamformers. This makes the time domain reconstructionof separated signals diﬃcult. However, this problem is lessserious in subband BSS than in frequency-domain BSS because the permutation problem does not occur in each subband as the separation procedure is executed in each subband. Therefore, we face smaller number of permutation
problems than with frequency-domain BSS. In particular,
subband BSS encounters very few permutation problems in
low frequency bands, where it is diﬃcult to solve the problems with frequency-domain BSS [12]. Moreover, we canuse a wider band signal than frequency-domain BSS to solve
the permutation problem in between subbands. Therefore,
we can use more information on separated signals and unmixing filters and can solve the problem more easily than infrequency-domain BSS.
Finally, we discuss computational cost. Because thecalculation of convolution and correlation in time domain(5) is very expensive, we calculate them in the frequency
domain. As discussed in [17], [18], subband processing re-Fig. 6 Example of a spectrum of a separated signal with (a) time-domain
BSS and (b) subband BSS (broken lines). Solid lines show the spectrum of
the original speech.

duces computational cost. By considering the decimation
R, computational cost for N subband per time is reduced toabout (N/2 + 1)/(R × R) times that of fullband time-domain
BSS. As R = N/4 in our case, we can reduce computational
cost by about 2/R.
5.

Further Improvement with Frequency Appropriate
Processing

In subband BSS, we can use diﬀerent separation methods
to estimate unmixing filter for diﬀerent subbands. In thissection, we propose concentrating on low frequency bands.
The SIR is generally worse in low frequency bands
as shown in Fig. 7, which plots the SIR values of separated signals for each subband. One reason for poor performance at low frequencies is that the impulse response isusually longer (see Fig. 8), and therefore it is more diﬃcult
to separate signals in low frequency bands than in high frequency bands. Moreover, since speech signals have highpower in low frequency bands, the performance in thesebands dominates the overall speech signal separation performance. Therefore, it is important to improve separation
performance in low frequency bands to obtain better overall
separation performance.
From a beamforming point of view, the resolution of a
spatial cancellation is proportional to the frequency. Therefore, the small phase diﬀerence between the observations at
the microphones is another reason for poor performance inIEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.12 DECEMBER 2005

3600
Table 1 Separation performance of subband BSS. (A)–(F): the overlapblockshift was executed only for bands 0-5; and (G) and (H): the overlapblockshift was executed for all subbands.

Fig. 7 SIR of separated signals in each subband. SIR is poor in low
frequency bands for every speaker combination.

Fig. 9Fig. 8 Example of acoustic impulse response of a room. Black indicateshigh power and white indicates low power. Reverberation is longer at low
frequencies than at high frequencies.

low frequency bands. We may be able to use diﬀerent microphone pairs for each subband having appropriate spacing foreach subband. In this paper, however, we consider the caseof Nm microphones whose number and spacing are fixed andignore multiple spacing microphone case.

Eﬀect of the filter length for the low frequency bands.

mixing filters only in low frequency bands [Table 1 (B)],separation performance was greatly improved. However,when we used 128 taps in low frequency bands, separation
performance degraded [see Table 1 (C)]. Figure 9 shows SIR
for cases (A)–(C): the performance of (C) is worse than (B)because the number of samples in each subband is too small
to allow us to precisely estimate a 128 tap unmixing filter.
The proposal in the next section (Sect. 5.2) will overcome
this problem.
5.2 Overlap-Blockshift in Low Frequency Bands

5.1 Longer Unmixing Filters in Low Frequency Bands
One possible way to improve SIR in low frequency bands is
to estimate longer unmixing filters in these bands to cover
reverberation. We therefore propose using longer unmixing
filters for low frequency bands (bands 0-5). Figure 8 showsthat the reverberation is long below about 600 Hz. Therefore, we used long filters for these frequency bands. The
column labeled “no-overlap” in Table 1 shows the separation performance for each unmixing filter length condition.
In Table 1 (A)–(C), we used a 32 tap separation filter
for high frequency bands, and changed the filter length for
low frequency bands (bands 0-5). It is conceivable that a 32
tap long unmixing filter cannot cover reverberation in low
frequency bands [see Table 1 (A)]. When we used long un-

Another possible way to improve SIR in low frequency
bands is to utilize a fine overlap-blockshift in the timedomain BSS stage for low frequency bands. Using thefine overlap-blockshift, we can outwardly increase the number of samples in each subband and estimate the unmixing
filters more precisely. Since our time-domain BSS algorithm (5) divides signals into B blocks to utilize signal nonstationarity, we can divide signals into blocks with an overlap as long as non-stationarity is expressed among blocks.
Note that this overlap-blockshift is executed in the separation stage, i.e., after the decimation for subband analysis.
In Table 1 [(B)–(F)], the columns show SIR obtainedby the overlap-blockshift only for low frequency bands(bands 0-5). Overlap (×2) and overlap (×4) denotes theARAKI et al.: SUBBAND-BASED BLIND SEPARATION FOR CONVOLUTIVE MIXTURES OF SPEECH

3601

Fig. 10Eﬀect of the overlap-blockshift only in low frequency bands.

Fig. 11 Example of SIR in each subband using a long filter in all frequency bands.

block-shift rate S = 2 and 4 in (5), respectively. (D) and (E)in Table 1 show that when we used the overlap-blockshift
only for low frequency bands, we obtained better separation
performance. With a fourfold overlap-blockshift, we wereable to estimate the unmixing filters of 128 taps in low frequency bands, and we obtained the best separation performance (underlined in Table 1). Figure 10 shows the eﬀectof the fine overlap-blockshift in low frequency bands.
5.3 Discussion
Even when using 128 taps for all the frequency bands [(F)in Table 1], the performance is no better than when we used
128 taps only for the low frequency bands [(E) in Table 1].
Figure 11 shows SIR in each subband for cases (E) and (F).
The use of long unmixing filters is not so eﬀective in highfrequency bands. Sometimes, short filters achieve better
separation performance than long filters in high frequency
bands. It is a waste of eﬀort to use 128 taps in all subbands.
When the overlap-blockshift was used in all subbands[see (G) and (H) in Table 1], the increase in SIR was verysmall compared with the SIR for (D) and (F) in Table 1.
The improvement in separation performance provided by
the overlap-blockshift is shown in Fig. 12. The overlapblockshift is also eﬀective in high frequency bands. However, the contribution of the improvement in high frequency
bands to SIR is not dominant because the original powerof the high frequency components of the speech signal issmaller than that of the low frequency components. Therefore, we conclude that the use of a fine overlap-blockshift
only in low frequencies is suﬃcient to obtain improved performance.
6.

Conclusion

Subband processing was applied to BSS for convolutive
mixtures of speech. This was motivated by the fact that
separation performance is degraded when a long frame size
is used for several seconds of speech in frequency-domain
BSS. We showed that subband BSS can (1) maintain a sufficient number of samples to estimate the statistics in eachFig. 12 Example of SIR in each subband obtained with the overlapblockshift in all subbands.

subband and (2) estimate an unmixing filter long enough tocover reverberation. We confirmed in experiments that subband BSS is eﬀective.
Furthermore, by eﬃciently using subband processing,i.e., employing an appropriate separation method for eachfrequency band, we showed that (3) we can improve separation performance with long unmixing filters and (4) withthe overlap-blockshift technique in low frequency bands. By
using long unmixing filters and the fine overlap-blockshifttechnique only in low frequency bands, we can eﬃciently
separate convolutive mixtures of speech. Such frequencydependent processing is impossible with time-domain BSS
and complicated with frequency-domain BSS. Moreover,
we can save computation cost without degrading separation performance by limiting the use of long unmixing filters
and the fine overlap-blockshift only to low frequency bands.
Subband BSS is a powerful separation tool when source signals si or the impulse response of system h ji have diﬀerent
characteristics in diﬀerent frequency bands.
References[1] S. Haykin, Unsupervised Adaptive Filtering, John Wiley & Sons,
2000.

IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.12 DECEMBER 2005

3602

[2] A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component
Analysis, John Wiley & Sons, 2001.
[3] S. Amari, S.C. Douglas, A. Cichocki, and H.H. Yang, “Multichannel blind deconvolution and equalization using the natural gradient,”
Proc. IEEE Workshop on Signal Processing Advances in WirelessCommunications, pp.101–104, April 1997.
[4] M. Kawamoto, A.K. Barros, A. Mansour, K. Matsuoka, and
N. Ohnishi, “Real world blind separation of convolved nonstationary signals,” Proc. ICA’99, pp.347–352, Jan. 1999.
[5] H. Buchner, R. Aichner, and W. Kellermann, “A generalization ofblind source separation algorithms for convolutive mixtures basedon second order statistics,” IEEE Trans. Speech Audio Process.,
vol.13, no.1, pp.120–134, Jan. 2005.
[6] T.W. Lee, Independent component analysis -Theory and applications, Kluwer, 1998.
[7] S.C. Douglas, “Blind separation of acoustic signals,” Microphone
Arrays: Techniques and Applications, ed. M. Brandstein and D.B.
Ward, pp.355–380, Springer, Berlin, 2001.
[8] P. Smaragdis, “Blind separation of convolved mixtures in the frequency domain,” Neurocomputing, vol.22, pp.21–34, 1998.
[9] S. Ikeda and N. Murata, “A method of ICA in time-frequency domain,” Proc. ICA’99, pp.365–370, Jan. 1999.
[10] M.Z. Ikram and D.R. Morgan, “Exploring permutation inconsistency in blind separation of speech signals in a reverberant environment,” Proc. ICASSP2000, pp.1041–1044, June 2000.
[11] J. Anemüller and B. Kollmeier, “Amplitude modulation decorrelation for convolutive blind source separation,” Proc. ICA2000,
pp.215–220, June 2000.
[12] H. Sawada, R. Mukai, S. Araki, and S. Makino, “A robust approachto the permutation problem of frequency-domain blind source separation,” Proc. ICASSP2003, pp.381–384, April 2003.
[13] K. Rahbar and J.P. Reilly, “A new fast-converging method for BSS
of speech signals in acoustic environments,” WASPAA2003, pp.21–
24, 2003.
[14] K. Matsuoka and S. Nakashima, “Minimal distortion principle forblind source separation,” Proc. ICA2001, pp.722–727, Dec. 2001.
[15] S. Araki, S. Makino, T. Nishikawa, and H. Saruwatari, “Fundamental limitation of frequency domain blind source separation for convolutive mixture of speech,” Proc. ICASSP2001, pp.2737–2740, May
2001.
[16] S. Araki, R. Mukai, S. Makino, T. Nishikawa, and H. Saruwatari,“The fundamental limitation of frequency domain blind source separation for convolutive mixtures of speech,” IEEE Trans. Speech Audio Process., vol.11, no.2, pp.109–116, 2003.
[17] J. Huang, K.C. Yen, and Y. Zhao, “Subband-based adaptive decorrelation filtering for co-channel speech separation,” IEEE Trans.
Speech Audio Process., vol.8, no.4, pp.402–406, July 2000.
[18] F. D-Beaulieu and B. Champagne, “Fast convolutive blind speech
separation via subband adaptation,” Proc. ICASSP2003, pp.513–
516, 2003.
[19] N. Grbic, X.J. Tao, S.E. Nordholm, and I. Claesson, “Blind signal separation using overcomplete subband representation,” IEEE
Trans. Speech Audio Process., vol.9, no.5, pp.524–533, July 2001.
[20] Y. Qi, P.S. Krishnaprasad, and S. Shamma, “The subband-based independent component analysis,” Proc. ICA2000, pp.199–204, June
2000.
[21] R. Mukai, S. Araki, H. Sawada, and S. Makino, “Evaluation of separation and dereverberation performance in frequency domain blindsource separation,” Acoustical Science and Technology, vol.25,
no.2, pp.119–126, March 2004.
[22] S. Araki, S. Makino, R. Aichner, T. Nishikawa, and H. Saruwatari,“Blind source separation for convolutive mixtures of speech using
subband processing,” Proc. SMMSP2002 (International Workshop
on Spectral Methods and Multirate Signal Processing), pp.195–202,
Sept. 2002.
[23] S. Araki, S. Makino, R. Aichner, T. Nishikawa, and H. Saruwatari,“Subband based blind source separation for convolutive mixtures ofspeech,” Proc. ICASSP2003, pp.509–512, April 2003.
[24] M.R. Portnoﬀ, “Implementation of the digital phase vocoder using
the fast Fourier transform,” IEEE Trans. Acoust. Speech Signal Process., vol.24, no.3, pp.243–248, June 1976.
[25] S.L. Gay and R.J. Mammone, “Fast converging subband acoustic echo cancellation using RAP on the WE DSP16A,” Proc.
ICASSP90, pp.1141–1144, April 1990.
[26] P.L. Chu, “Weaver SSB subband acoustic echo canceller,” Proc.
IWAENC93 (International Workshop on Acoustic Echo Control),pp.173–176, Sept. 1993.
[27] S. Makino, J. Noebauer, Y. Haneda, and A. Nakagawa, “SSB subband echo canceller using low-order projection algorithm,” Proc.
ICASSP96, pp.945–948, May 1996.
[28] R. Crochiere and L. Rabiner, Multirate Digital Signal Processing,
Prentice-Hall, Englewood Cliﬀs, NJ, 1983.
[29] T. Nishikawa, H. Saruwatari, and K. Shikano, “Blind source separation of acoustic signals based on multistage ICA combining
frequency-domain ICA and time-domain ICA,” IEICE Trans. Fundamentals, vol.E86-A, no.4, pp.846–858, April 2003.
[30] M. Kawamoto, K. Matsuoka, and N. Ohnishi, “A method of blindseparation for convolved non-stationary signals,” Neurocomputing,vol.22, pp.157–171, 1998.
[31] R. Aichner, S. Araki, S. Makino, T. Nishikawa, and H. Saruwatari,“Time domain blind source separation of non-stationary convolved
signals by utilizing geometric beamforming,” NNSP2002, pp.445–
454, 2002.
[32] S. Araki, S. Makino, Y. Hinamoto, R. Mukai, T. Nishikawa,and H. Saruwatari, “Equivalence between frequency domain blindsource separation and frequency domain adaptive beamforming for
convolutive mixtures,” EURASIP Journal on Applied Signal Processing, vol.2003, no.11, pp.1157–1166, 2003.
[33] S. Kurita, H. Saruwatari, S. Kajita, K. Takeda, and F. Itakura, “Evaluation of blind signal separation method using directivity patternunder reverberant conditions,” Proc. ICASSP2000, pp.3140–3143,
June 2000.
[34] H. Saruwatari, S. Kurita, and K. Takeda, “Blind source separation combining frequency-domain ICA and beamforming,” Proc.
ICASSP2001, pp.2733–2736, May 2001.
[35] H. Sawada, R. Mukai, and S. Makino, “Direction of arrival estimation for multiple source signals using independent component analysis,” Seventh International Symposium on Signal Processing and its
Applications (ISSPA 2003), pp.411–414, 2003.
[36] H. Sawada, R. Mukai, S. Araki, and S. Makino, “Polar coordinate
based nonlinear function for frequency domain blind source separation,” Proc. ICASSP2002, pp.1001–1004, May 2002.
[37] X. Sun and S. Douglas, “A natural gradient convolutive blind source
separation algorithm for speech mixtures,” Proc. ICA2001, pp.59–
64, Dec. 2001.

Shoko Araki
received the B.E. and the M.E.
degrees in mathematical engineering and information physics from the University of Tokyo,
Japan, in 1998 and 2000, respectively. In 2000,
she joined NTT Communication Science Laboratories, Kyoto. Her research interests include
array signal processing, blind source separation
applied to speech signals. She received the
TELECOM System Technology Award from the
Telecommunications Advancement Foundationin 2004, the Best Paper Award of the IWAENC
in 2003 and the 19th Awaya Prize from Acoustical Society of Japan (ASJ)
in 2001. She is a member of the IEEE and the ASJ.

ARAKI et al.: SUBBAND-BASED BLIND SEPARATION FOR CONVOLUTIVE MIXTURES OF SPEECH

3603

Shoji Makino
received the B.E., M.E., andPh.D. degrees from Tohoku University, Japan, in
1979, 1981, and 1993, respectively. He joined
NTT in 1981. He is now an Executive Manager at the NTT Communication Science Laboratories. He is also a Guest Professor at the
Hokkaido University. His research interests include adaptive filtering technologies and realization of acoustic echo cancellation, blind source
separation of convolutive mixtures of speech.
He received the TELECOM System Technology
Award of the TAF in 2004, the Best Paper Award of the IWAENC in 2003,
the Paper Award of the IEICE in 2005 and 2002, the Paper Award of the
ASJ in 2005 and 2002, the Achievement Award of the IEICE in 1997, andthe Outstanding Technological Development Award of the ASJ in 1995. He
is the author or co-author of more than 200 articles in journals and conference proceedings and has been responsible for more than 150 patents. He
is a member of the Conference Board of the IEEE SP Society and an Associate Editor of the IEEE Transactions on Speech and Audio Processing.
He is also an Associate Editor of the EURASIP Journal on Applied Signal Processing. He is a member of the Technical Committee on Audio andElectroacoustics of the IEEE SP Society as well as the Technical Committee on Blind Signal Processing of the IEEE Circuits and Systems Society.
He is also a member of the International ICA Steering Committee and theOrganizing Chair of the ICA2003 in Nara. He is the General Chair of the
IWAENC2003 in Kyoto. He was a Vice Chair of the Technical Committee
on Engineering Acoustics of the IEICE and the ASJ. He is an IEEE Fellow,
a council member of the ASJ, and a member of the EURASIP.

Robert Aichnerreceived the Dipl.-Ing.
(FH) degree in electrical engineering from the
University of Applied Sciences, Regensburg,
Germany in 2002. From 2001 to 2002 he researched at the Speech Open Lab of the Research and Development division of the Nippon
Telegraph and Telephone Corporation (NTT) in
Kyoto, Japan. There he was working on timedomain blind source separation of audio signals.
Since 2002 he is a member of the research staﬀ
at the Chair of Multimedia Communications andSignal Processing at the University of Erlangen-Nuremberg, Germany.
His current research interests include multichannel adaptive algorithms forhands-free human-machine interfaces and its application to blind source
separation, noise reduction, source localization, adaptive beamforming andacoustic echo cancellation.

Tsuyoki Nishikawa
received the B.E. degree in electronic system and information engineering from Kinki University in 2000, theM.E. and Ph.D. degree in information and science from Nara Institute of Science and Technology (NAIST) in 2002, 2005, respectively. In
2005, he joined Audio Group, Matsushita Electric Industrial CO.,LTD. His research interests
include acoustics signal processing, sensor array processing, and blind source separation. He
received TELECOM System Technology Awardand TELECOM System Technology Student Award from the Telecommunications Advancement Foundation in 2004, C&C Young Best Paper
Award of the Foundation for C&C Promotion in 2005, and Funai Information Technology Award for Young Researchers of the Funai Foundation for
Information Technology in 2005. He is a member of the the AcousticalSociety of Japan.

Hiroshi Saruwatari
was born in Nagoya, Japan, on July 27, 1967. He received
the B.E., M.E. and Ph.D. degrees in electrical engineering from Nagoya University, Nagoya, Japan, in 1991, 1993 and 2000, respectively. He joined Intelligent Systems Laboratory, SECOM CO.,LTD., Mitaka, Tokyo, Japan,
in 1993, where he engaged in the research anddevelopment on the ultrasonic array system forthe acoustic imaging. He is currently an associate professor of Graduate School of Information Science, Nara Institute of Science and Technology. His research interests include array signal processing, blind source separation, and soundfield reproduction. He received the Paper Award from IEICE in 2001. He
is a member of IEEE, and the Acoustical Society of Japan.