JOINT SEPARATION AND DEREVERBERATION OF REVERBERANT MIXTURES
WITH MULTICHANNEL VARIATIONAL AUTOENCODER
Shota Inoue1 , Hirokazu Kameoka2 , Li Li1 , Shogo Seki3 , Shoji Makino1
1

2

University of Tsukuba, JapanNTT Communication Science Laboratories, NTT Corporation, Japan
3
University of Nagoya, Japans.inoue@mmlab.cs.tsukuba.ac.jp, kameoka.hirokazu@lab.ntt.co.jp

matrix factorization (DMNMF) [5], which was later named as
independent low-rank matrix analysis (ILRMA) [6, 7], adopts
the NMF concept to source spectrogram modeling that approximates each source power spectrogram as the linear combination of a limited set of spectral templates scaled by magnitudes varying with time.
Recently, to achieve further improvements, some attempts
have been made to combine deep neural networks (DNNs)
with the ICA-based multichannel source separation framework [8–10]. One of them is the multichannel variational
autoencoder (MVAE) [10]. MVAE trains a conditional VAE
using spectrograms of clean source signals and the corres
ponding attribute class labels so that the trained decoder distribution can be used as a generative model of the underlying
source signals in a mixture, which is called the CVAE source
model. MVAE has shown to significantly outperform conventional methods, which indicates that the CVAE source modelhas the capability to improve the source separation performance. However, one drawback is that the source separation
performance degrades in highly reverberant environments
since MVAE assumes an instantaneous mixture model.
To address this drawback, this study proposes an extension of MVAE that allows us to simultaneously perform source separation and dereverberation. Specifically,
we formulate the separation system of mixture signals as
a frequency-domain convolutive mixture model, which has
been shown to be effective for separating highly reverberant
mixtures in many previous studies [5, 11, 12].
The rest of this paper is structured as follows. In Section
2, we formulate a multichannel BSS problem with instantaneous mixture models and review MVAE. In Section 3, we
present the MVAE using frequency-domain convolutive mixture models and derive a convergence-guaranteed algorithm
for the optimization. The results of highly reverberant source
separation experiments are presented in Section 4.

ABSTRACTIn this paper, we deal with a multichannel source separation
problem under a highly reverberant condition. The multichannel variational autoencoder (MVAE) is a recently proposedsource separation method that employs the decoder distribution of a conditional VAE (CVAE) as the generative modelfor the complex spectrograms of the underlying source signals. Although MVAE is notable in that it can significantly
improve the source separation performance compared withconventional methods, its capability to separate highly reverberant mixtures is still limited since MVAE uses an instantaneous mixture model. To overcome this limitation, in thispaper we propose extending MVAE to simultaneously solve
source separation and dereverberation problems by formulating the separation system as a frequency-domain convolutive
mixture model. A convergence-guaranteed algorithm basedon the coordinate descent method is derived for the optimization. Experimental results revealed that the proposed method
outperformed the conventional methods in terms of all thesource separation criteria in highly reverberant environments.
Index Terms— Blind source separation, blind dereverberation, multichannel audio signal processing, multichannel
variational autoencoder (MVAE)
1. INTRODUCTION
Blind source separation (BSS) is a technique for separating
individual source signals from recorded microphone array inputs without any prior information about source signals andthe transfer characteristics between sources and microphones.
The most commonly used approach for determined BSS problems is independent component analysis (ICA) [1], which
achieves source separation by assuming the statistical independence between the sources. Among the ICA-based methods, those methods performing separation in the frequency
domain provide the flexibility of utilizing various models forthe time–frequency representations of source signals and array responses, which play critical roles in BSS. For example,
independent vector analysis (IVA) [2–4] solves frequencywise source separation and permutation alignment simultaneously by assuming that the magnitudes of the frequency
components originating from the same source tend to varycoherently over time. Determined multichannel non-negative

2. MULTICHANNEL VARIATIONAL
AUTOENCODER
We consider a determined situation where J source signals
are observed by I microphones (J = I). Let xi (f, n) andsj (f, n) denote the short-time Fourier transform (STFT) coefficients of the signal observed at the i-th microphone andthe j-th source signal, where f and n are the frequency andtime indices, respectively. Now, we use a separation system

This work was supported by JSPS KAKENHI Grant Number 17H01763
and 18J20059, and SECOM Science and Technology Foundation.

978-1-5386-4658-8/18/$31.00 ©2019 IEEE

96

ICASSP 2019

it has the same form as the LGM (3).
∏NC (s(f, n)|0, v(f, n)),pθ (S|z, c, g) =of the forms(f, n) = W H (f )x(f, n),
W (f ) = [w1 (f ), . . . , wI (f )] ∈ C

(1)
I×I

(2)

v(f, n) = g · σθ2 (f, n; z, c),to describe the relationship between the observed signalsx(f, n) = [x1 (f, n), . . . , xI (f, n)]T ∈ CI and sourcess(f, n) = [s1 (f, n), . . . , sI (f, n)]T ∈ CI , where W H (f )is called the separation matrix and (·)H denotes Hermitiantranspose.
Let us assume that sj (f, n) independently follows a zeromean complex Gaussian distribution with variance vj (f, n) =E[|sj (f, n)|2 ]sj (f, n) ∼ NC (sj (f, n)|0, vj (f, n)).

k

(3)

(4)

where V (f, n) is a diagonal matrix with diagonal entries
v1 (f, n), . . . , vI (f, n). From (1) and (4), we can show thatx(f, n) follows
x(f, n) ∼ NC (x(f, n)|0, (W H (f ))−1 V (f, n)W (f )−1 ). (5)Hence, the negative log-likelihood of the parameters V ={vj (f, n)}f,n,j and W = {W H (f )}f given the observed
mixture signals X = {xi (f, n)}i,f,n is given as

where z(k), µϕ (k; S, c) and σϕ2 (k; S, c) represent the k-th element of the latent space variable z and the encoder outputsµϕ (S, c) and σ 2ϕ (S, c), respectively. The trained decoder distribution can then be used as a generative model of the complex spectrogram of the j-th source pθ (Sj |zj , cj , gj ), wherezj , cj and gj are the unknown parameters of the model. This
generative model is called the CVAE source model. The optimization algorithm of MVAE consists of iteratively updating
the separation matrices W using the iterative projection (IP)method [4], the source model parameters Ψ = {zj , cj }j using backpropagation and the global scale G = {gj }j using thefollowing update rule:gj ←1 ∑ |wjH (f )x(f, n)|2.
FN
σθ2 (f, n; zj , cj )

(11)

f,n

MVAE is notable in that (i) it takes full advantage of thestrong representation power of DNNs for source power spectrogram modeling, and (ii) the convergence of the source separation algorithm is guaranteed. However, similar to the otherinstantaneous mixture model-based methods, the source separation capability of MVAE is limited in a highly reverberant
environment where the length of the room impulse responses(RIRs) can be large than the STFT frame length.

c

L(V, W|X ) = − 2N log det W H (f ))∑(|sj (f, n)|2+
log vj (f, n) +,vj (f, n)(9)where σθ2 (f, n; z, c) denotes the (f, n)-th element of the decoder output and g represents the global scale of the generated
spectrogram. Regarding the encoder distribution qϕ (z|S, c),a regular Gaussian distribution is adopted∏
qϕ (z|S, c) =N (z(k)|µϕ (k; S, c), σϕ2 (k; S, c)), (10)We call (3) the local Gaussian model (LGM). We further assume sj (f, n) to be independent from one source to the others. s(f, n) thus followss(f, n) ∼ NC (s(f, n)|0, V (f, n)),

(8)

f,n

(6)

f,n,j

where =c denotes equality up to constant terms. It is important to note that if we individually treat vj (f, n) as a free parameter indexed by frequency f , the negative log-likelihood
will be split into frequency-wise source separation problems.
This means that the permutation of the separated components
in each frequency is not uniquely determined. Permutationalignment is thus needed after W has been obtained.
To solve this problem, for MVAE [10], a conditional
VAE (CVAE) [13] is used to model and estimate the spectrograms of the sources sj (f, n). Let S = {s(f, n)}f,n be
the complex spectrogram of a particular sound source andc be the corresponding attribute class label, which is represented as a one hot vector. With a set of labeled training
samples {Sm , cm }M
m=1 , a CVAE network consisting of an encoder network qϕ (z|S, c) and a decoder network pθ (S|z, c)
is trained jointly by maximizing

3. PROPOSED METHOD
3.1. Formulation
In this section, we describe the proposed method that extends
MVAE to simultaneously solve source separation and dereverberation problems. Specifically, instead of the instantaneous mixture model (1), we formulate the separation system as a frequency-domain convolutive mixture model, which
has been shown to be effective for separating highly reverberant mixtures [5, 11, 12]. With the frequency-domain convolutive mixture model that has a multichannel finite-impulseresponse form, the relationship between the observed signalsx(f, n) and sources s(f, n) is written as

J (ϕ, θ) = E(S,c)∼pD (S,c) [Ez∼qϕ (z|S,c) [log pθ (S|z, c)]′− KL[qϕ (z|S, c)||p(z)]], (7)s(f, n) =where E(S,c)∼pD (S,c) [·] denotes the sample mean over the
training examples and KL[·||·] is the Kullback–Leibler divergence. Here, MVAE defines the decoder distribution asa zero-mean complex Gaussian distribution as follows so thatN∑

W H (f, n′ )x(f, n − n′ ).

(12)

n′ =0

Here, W H (f, n′ ), 0 ≤ n′ ≤ N ′ are the coefficient matrices
of size I × I and W H (f, 0) is equivalent to W H (f ) in (1).

97

By dropping the constant terms with respect to D from(15), we obtain

When W H (f, 0) is invertible, the dereverberated mixture signal y(f, n) = [yi (f, n), . . . , yI (f, n)]T ∈ CI and the source
signal s(f, n) can be written as

I(D) =′

N∑y(f, n) = x(f, n) −D H (f, n′ )x(f, n − n′ ),

∑

(13)

′

x(f, n)−N∑

D H (f, n′ )x(f, n−n′ ),Σw/v(f,n)

n′ =1

f,n

2

(20)

n′ =1
H

s(f, n) = W (f, 0)y(f, n),
′

√

(14)
−1

′

xH Σw/v(f,n) x with Σw/v(f,n) =where |x|Σw/v(f,n) =∑ wj (f )wjH (f )jvj (f,n) , which is assumed to be positive definite. To
obtain independent updating rules for each f , we vectorize{D(f, n′ )}n′ as′where D (f, n ) = −(W (f, 0)) W (f, n ), 1 ≤ n ≤N ′ . Note that (13) can be seen as a dereverberation processof the observed mixture signal x(f, n), whereas (14) can beseen as an instantaneous demixing processing of the dereverberated mixture signal y(f, n). Therefore, the negative loglikelihood of interest is a function of the dereverberation filter
D = {D H (f, n′ )}f,n′ , separation matrices W, spectral parameters Ψ and scale parameter G:HHHd(f ) = [dT1 (f, 1), . . . , dTI (f, 1), dT1 (f, 2), . . . , dTI (f, 2), . . . ,2∑(c
I(D,W, Ψ, G|X ) = −2N log det W H (f ) +log vj (f, n)+|wjH (f )(x(f, n)−

∑N ′

′

dT1 (f, N ′ ), . . . ., dTI (f, N ′ )]T ∈ CI N ,(21)

where di (f, n′ ) is the i-th column of D(f, n′ ). Thus, the term
∑N ′
H
′
′
n′ =1 D (f, n )x(f, n − n ) in (20) can be rewritten as

f,n,j

H
′
′
2)
n′ =1 D (f, n )x(f, n−n ))|.
vj (f, n)
(15)

′

N
∑

D H (f, n′ )x(f, n − n′ ) = X(f, n)d∗ (f ),

(22)

n′ =1

3.2. Related workwhere d∗ (f ) represents the complex conjugate of d(f ) andIn [12], the idea of formulating the separation system ofhighly reverberant mixture signals using a frequency-domain
convolutive mixture model has been adopted to ILRMA [6],which allows the method to solve source separation anddereverberation simultaneously. We refer to this method
as ILRMA+ hereafter. The proposed method is different
from ILRMA+ in the way of modeling the sources sj (f, n),where the proposed method uses the CVAE source model
whereas ILRMA+ employs a non-negative matrix factorization model. Specifically, ILRMA+ models the variance∑Kj
vj (f, n) = k=1
bj,k (f )hj,k (n), which amounts to assuming that the power spectrograms of source signals can beapproximated by the linear combination of a small numberof spectral templates bj,1 (f ), . . . , bj,Kj (f ) ≥ 0 scaled bymagnitudes varying with time hj,1 (n), . . . , hj,Kj (n) ≥ 0.
From this viewpoint, the proposed method can be seen as
an extension of ILRMA+ that replaces the NMF model with
the CVAE source model to achieve better source separation
performance by enhancing the representation power of thesource model.

X(f, n) = [I ⊗ xT (f, n − 1), I ⊗ xT (f, n − 2), . . . ,′I ⊗ xT (f, n − N ′ )] ∈ CI×I N .
2(23)Here, ⊗ stands for the Kronecker product. By substituting(22) into (20), we obtain∑()H
I(D) =
x(f, n) − X(f, n)d∗ (f )f,n()× Σw/v( f,n) x(f, n) − X(f, n)d∗ (f ) .

(24)

Since (24) is a quadratic equation with respect to d∗ (f ), thisfunction can be readily minimized by calculating the partial
derivative of I(D) to be zero. The update rules for each d∗ (f )is given as a closed form:()−1∑d∗ (f ) ←X H (f, n)Σw/v(f,n) X(f, n)n

3.3. Optimization process

×

(
∑

)
H

X (f, n)Σw/v(f,n) x(f, n) .

(25)

We describe the optimization algorithm in this subsection, in
which the objective function (15) is iteratively decreased using a coordinate descent method in which each iteration comprises the following four minimization steps:

We employ the following update rules derived on the basis of
the IP method [4] to update W:ˆD ← argmin I(D, W, Ψ, G|X ),(16)

wj (f ) ← (W H (f, 0)Σy/vj (f ) )−1 ej ,Ŵ ← argmin I(D, W, Ψ, G|X ),(17)

wj (f ) ← √Ψ̂ ← argmin I(D, W, Ψ, G|X ),

(18)

D

W

n

G

,

(27)

wjH (f )Σy/vj (f )wj (f )∑where Σy/vj (f ) = (1/N ) n y(f, n)y H (f, n)/vj (f, n) andej denotes the j-th column of the I × I identity matrix.

ΨĜ ← argmin I(D, W, Ψ, G|X ).

wj (f )

(26)

(19)

98

Table 1. The average SDR, SIR and SAR improvements
achieved by each method. The bold font shows the top scores.
Improvement (dB)
RIRs
Methods
SDR
SIR
SAR
ILRMA
2.57
7.60 -0.94
JR1
ILRMA+ 5.06 11.20 1.15
T60 = 0.60 (s)MVAE3.68 10.67 -0.42
MVAE+
6.66 14.74 2.22
ILRMA
2.43
7.48 -1.04
OFC
ILRMA+ 5.43 11.48 1.63
T60 = 0.78 (s)MVAE
3.53 10.43 -0.50
MVAE+
6.89 14.90 2.64

Fig. 1. Microphone and source positions, where ◦ and × represent the positions of microphones and sources, respectively.
To update Ψ and G, we apply backpropagation and (11).
The parameters of encoder and decoder network are trainedby maximizing (7) with labeled clean audio samples. These
training and optimization procedures are the same as those of
MVAE.
Therefore, the proposed algorithm is summarized as follows:

100 iterations for ILRMA and ILRMA+, and 60 iterations for
MVAE and the proposed method. To initialize W H (f ) andD H (f, n′ ) of the proposed method, we run ILRMA+ for 30
iterations. For the encoder and decoder networks, we employed the same architectures as those used in [10], i.e., athree-layer fully convolutional network with gated linear units(GLUs) [15] and a three-layer fully deconvolutional networkwith GLUs. Adam optimization [16] was used for training
CVAE and estimating Ψ during the source separation. Notethat we must take into account the sum-to-one constraints
when updating cj . This can be easily implemented by inserting an appropriately designed softmax layer

1. Train θ and ϕ using (7).
2. Initialize Ψ, G, W and D.
3. Repeat the following updates until convergence.
(a) Update wj (f ) for each j using (26) and (27).
(b) Update zj , cj for each j using backpropagation.
(c) Update gj for each j using (11).

cj = Softmax(uj ),∗(d) Update d (f ) for each f using (25).

(28)

and treat uj as the parameter to be estimated instead.
We took the average of the signal-to-distortion ratios(SDR), signal-to-interference ratios (SIR) and signal-toartifact ratios (SAR) [17] as the evaluation criteria. Table
1 shows the separation performances under the two reverberant conditions. The proposed method is shown to outperform
all the conventional methods in terms of SDR, SIR and SAR.
By comparing the results of ILRMA+ and the proposed
method with those of ILRMA and MVAE, it is confirmedthat the frequency-domain convolutive mixture models are
effective for improving the source separation performances
under highly reverberant conditions.

4. EXPERIMENTS
To evaluate the effectiveness of the proposed method inhighly reverberant environments, we conducted experiments
in which we compared the source separation performance ofthe proposed approach with those of ILRMA [6], MVAE [10]and ILRMA+ [12]. Specifically, we used RIRs measured
in a Japanese-style room (JR1) and an office room (OFC),
the reverberation times (T60 ) of which were 0.60 s and 0.78s, respectively. Fig. 1 shows the two configurations of the
microphones and sources we tested. We used utterances of
two female speakers “SF1” and “SF2”, and two male speakers “SM1” and “SM2” excerpted from the Voice Conversion
Challenge (VCC) 2018 dataset [14] for composing the training and evaluation sets. The audio files for each speaker weremanually segmented into 116 short sentences (about 7 min)where 81 and 35 sentences (about 5 and 2 min) were provided
as training and evaluation sets, respectively. We generated 10
speech combinations for each speaker pair, namely, SF1+SF2,
SF1+SM1, SF2+SM2 and SM1+SM2. Hence, there were in
total 80 test signal in each reverberant environment. The
length of each signal was about 4 to 7 s long. Speaker identities were considered as the only class category. Thus, theclass label c was a four-dimensional one hot vector.
All mixture signals were resampled at 16 kHz. The STFT
was computed using the Hamming window with 256 ms longand 64 ms window shift. For ILRMA and ILRMA+, the basis number K was set at 5. The dereverberation filter length
N ′ was set at 3 for JR1 and 4 for OFC, respectively. We run

5. CONCLUSIONSIn this paper, we proposed an extension of MVAE that is
capable of solving source separation and dereverberation
problems simultaneously by formulating the separation system as a frequency-domain convolutive mixture model. A
convergence-guaranteed optimization process was derived,
which consists of iteratively updating (i) the spectral parameters of each source by applying backpropagation using the
CVAE source model, (ii) the separation matrices using the IP
method and (iii) the dereverberation filters using multichannel
linear prediction. The experimental results showed that the
combination of the CVAE source model and the frequencydomain convolutive mixture model was able to improve thesource separation performances in highly reverberant environments.

99[1] A. Hyvärinen and E. Oja, “Independent component
analysis: algorithms and applications,” Neural networks, vol. 13, no. 4–5, pp. 411–430, 2000.

[14] J. Lorenzo-Trueba, J. Yamagishi, T. Toda, D. Saito,
F. Villavicencio, T. Kinnunen, and Z. Ling, “The voice
conversion challenge 2018: Promoting developmentof parallel and nonparallel methods,” arXiv preprint
arXiv:1804.04262, Apr. 2018.

[2] T. Kim, T. Eltoft, and T.-W. Lee, “Independent vector
analysis: An extension of ICA to multivariate components,” in Proc. ICA, 2006, pp. 165–172.

[15] Y. N. Dauphin, A. Fan, M. Auli, and D. Grangier, “Language modeling with gated convolutional networks,”
arXiv preprint arXiv:1612.08083, 2016.

[3] A. Hiroe, “Solution of permutation problem in frequency domain ICA, using multivariate probability density functions,” in Proc. ICA, 2006, pp. 601–608.

[16] D. P. Kingma and J. Ba, “Adam: A method for stochastic
optimization,” in Proc. ICLR, 2015.

6. REFERENCES[17] E. Vincent, R. Gribonval, and C. Févotte, “Performancemeasurement in blind audio source separation,” IEEE
Trans. ASLP, vol. 14, no. 4, pp. 1462–1469, 2006.

[4] N. Ono, “Stable and fast update rules for independentvector analysis based on auxiliary function technique,”in Proc. WASPAA, 2011, pp. 189–192.
[5] H. Kameoka, T. Yoshioka, M. Hamamura, J. Le Roux,and K. Kashino, “Statistical model of speech signals
based on composite autoregressive system with application to blind source separation,” in Proc. LVA/ICA,
2010, pp. 245–253.
[6] D. Kitamura, N. Ono, H. Sawada, H. Kameoka, and
H. Saruwatari, “Determined blind source separation unifying independent vector analysis and nonnegative matrix factorization,” IEEE/ACM Trans. ASLP, vol. 24, no.
9, pp. 1626–1641, 2016.
[7] D. Kitamura, N. Ono, H. Sawada, H. Kameoka, and
H. Saruwatari, “Determined blind source separation
with independent low-rank matrix analysis,” in Audio
Source Separation, S. Makino, Ed. Mar. 2018, pp. 125–
155, Springer.
[8] A. A. Nugraha, A. Liutkus, and E. Vincent, “Multichannel audio source separation with deep neural networks,”
IEEE/ACM Trans. ASLP, vol. 24, no. 9, pp. 1652–1664,
2016.
[9] S. Mogami, H. Sumino, D. Kitamura, N. Takamune,
S. Takamichi, H. Saruwatari, , and N. Ono, “Independent deeply learned matrix analysis for multichannel audio source separation,” in Proc. EUSIPCO, 2018.
[10] H. Kameoka, L. Li, S. Inoue, and S. Makino, “Semiblind source separation with multichannel variational
autoencoder,” arXiv preprint arXiv:1808.00892, Aug.
2018.
[11] T. Yoshioka, T. Nakatani, M. Miyoshi, and H. G. Okuno,“Blind separation and dereverberation of speech mixtures by joint optimization,” IEEE Trans. ASLP, vol.
19, no. 1, pp. 69–84, 2011.
[12] H. Kagami, H. Kameoka, and M. Yukawa, “Joint separation and dereverberation of reverberant mixtures withdetermined multichannel non-negative matrix factorization,” in Proc. ICASSP, 2018, pp. 31–35.
[13] D. P. Kingma, S. Mohamedy, D. J. Rezendey, and
M. Welling, “Semi-supervised learning with deep generative models,” in Adv. Neural Information Processing
Systems (NIPS), 2014, pp. 3581–3589.

100