Joint separation, dereverberation and classification of multiple sources using
multichannel variational autoencoder with auxiliary classifier
Shota INOUE(1) , Hirokazu KAMEOKA(2) , Li LI(3) , Shoji MAKINO(4)(1) University of Tsukuba, Japan, s.inoue@mmlab.cs.tsukuba.ac.jp(2) NTT Communication Science Laboratories, NTT Corporation, Japan, hirokazu.kameoka.uh@hco.ntt.co.jp(3) University of Tsukuba, Japan, lili@mmlab.cs.tsukuba.ac.jp(4) University of Tsukuba, Japan, maki@tara.tsukuba.ac.jp

Abstract
This paper proposes a unified approach to jointly solving the separation, dereverberation, and classificationof mixed sound sources from microphone array observations. The proposed method uses a frequency-wise
convolutive mixture model to express the mixing process under highly reverberant environments and the auxiliary classifier conditional variational autoencoder (ACVAE) to model the complex spectrograms of underlying
sources. Using an ACVAE as the source generative model allows us to estimate the latent vectors and the classindex of each source in a test mixture by computing the outputs of the pretrained approximate posterior inference
networks without using backpropagation. We experimentally confirmed that the proposed method outperformed
conventional methods in terms of both computation time and source classification.
Keywords: Source separation, blind dereverberation, multichannel audio signal processing, multichannel
variational autoencoder (MVAE)

1

INTRODUCTION

Source separation is a technique of separating individual source signals from observed mixture signals. In particular, blind source separation (BSS) achieves the separation of source signals without any prior information aboutsources and spatial transfer characteristics between microphones and sources. One of the most generally usedapproaches to solving determined BSS problems, with equal numbers of microphones and sources, is independent component analysis (ICA) [1], which achieves source separation by assuming the statistical independence
between sources.
A frequency-domain formulation of ICA provides the flexibility of utilizing various models for the time-frequency
representation of source signals. For example, determined multichannel non-negative matrix factorization (DMNMF) [2], which was later called “independent low-rank matrix analysis" (ILRMA) [3], adopts the NMF concept to source spectrogram modeling which approximates source power spectrograms as linear combinations ofspectral templates scaled by time-varying amplitudes. One drawback as regards ILRMA is that it can fail towork for sources with spectrograms that do not comply with the NMF model. In stead of the NMF model,
the multichannel variational autoencoder (MVAE) method [4] uses the pretrained decoder of a conditional VAE(CVAE) as a generative model of the complex spectrograms of each source. This method allows us to performsource separation and source class identification simultaneously by optimizing the decoder inputs, consisting of
the latent space variables and the source class index, so that the log-likelihood is maximized. Thanks to thestrong representation power of neural networks for source spectrogram modeling, MVAE was shown to produce
good source separation performances. However, MVAE still suffers from two main issues, namely, performancedegradations caused by long reverberation and expensive computational costs.
To address the first drawback, we have previously proposed MVAE+ [5], which employs a frequency-wise convolutive mixture model instead of the instantaneous mixture model so that it can handle long reverberation.
Under this model, we have derived a convergence-guaranteed algorithm for joint separation and dereverberation,
consisting of alternately updating the dereverberation filter, the demixing matrix and the source model parameters. To overcome the second drawback, we have recently introduced a fast algorithm called the “fast MVAE”(fMVAE) [6], where the idea is to replace the process of optimizing the decoder inputs using backpropagation
with the forward computations of the two pretrained approximate posterior inference networks.
To combine the advantages of MVAE+ and fMVAE, this paper proposes a fast algorithm for MVAE+ that6988

performs source separation, dereverberation and source classification in a joint manner. The rest of this paper
is structured as follows: Section 2 formulates a multichannel BSS problem with frequency-domain convolutive
mixture models and reviews MVAE+. Section 3 presents the proposed fast algorithm that not only reduces
the computational time but also improves source classification performance. The experimental results of speech
separation and source classification under highly reverberant environments are presented in Section 4.

2

UNIFIED APPROACH FOR SOURCE SEPARATION AND DEREVERBERATION

2.1 Problem formulation
We consider a determined situation where J source signals are observed by I microphones (J = I). Let xi ( f , n)and s j ( f , n) denote the short-time Fourier transform (STFT) coefficients of the signal observed at the i-th microphone and the j-th source signal, where f and n are the frequency and time indices, respectively.
Now, we formulate the separation system as a frequency-domain convolutive mixture model in order to handle
highly reverberant environments where the length of the room impulse responses (RIRs) can be longer than
STFT frame length [2, 5, 7, 8]. With this model, we can write the relationship between the observed signalsx ( f , n) = [x1 ( f , n), . . . , xI ( f , n)]T ∈ CI and sources s ( f , n) = [s1 ( f , n), . . . , sI ( f , n)]T ∈ CI in the following multichannel finite-impulse-response:N0s ( f , n) = ∑ W H ( f , n0 )xx( f , n − n0 ).

(1)

n0 =0

Here, W ( f , 0) corresponds to the separation matrix and (·)H denotes Hermitian transpose. When W H ( f , 0) is
invertible, (1) can be rewritten equivalently as follows:N0y ( f , n) = x ( f , n) − ∑ D H ( f , n0 )xx( f , n − n0 ),(2)s ( f , n) = W H ( f , 0)yy( f , n),

(3)

n0 =1

W H ( f , 0))−1W H ( f , n0 ), 1 ≤ n0 ≤ N 0 denotes the dereverberation filter, y ( f , n) = [y1 ( f , n), . . . ,where D H ( f , n0 ) = −(WTyI ( f , n)] denotes a dereverberated version of the mixture signals and s ( f , n) denotes the source signals. Notethat (2) can be seen as a dereverberation process applied to the observed mixture signal x ( f , n), whereas (3)can be seen as an instantaneous demixing process applied to the dereverberated version of the mixture signaly ( f , n).
Let us now assume the local Gaussian model (LGM) [9, 10] in which s j ( f , n) independently follows a zeromean complex Gaussian distribution with power spectral density v j ( f , n) = E[|s j ( f , n)|2 ],s j ( f , n) ∼ NC (s j ( f , n)|0, v j ( f , n)).

(4)

We further assume that s j ( f , n) are statistically independent to each other. s ( f , n) thus follows
V ( f , n)),s ( f , n) ∼ NC (ss( f , n)|0,V(5)

where V ( f , n) is a diagonal matrix with diagonal entries v1 ( f , n), . . . , vI ( f , n). From (1) and (5), we can showthat y ( f , n) follows
W H ( f ))−1V ( f , n)W
W ( f )−1 ).
y ( f , n) ∼ NC (yy( f , n)|0, (W(6)

Hence, the negative log-likelihood of the spectral parameters V = {v j ( f , n)} f ,n, j , the dereverberation filter D =6989

DH ( f , n0 )} f ,n0 , separation matrices W given the observed signal X is given as{D
|wHj y ( f , n)|2
W ( f ) + ∑ log v j ( f , n) +I (D, W , V |X ) = −2N log detW,v j ( f , n)f ,n, jc

H(7)where =c denotes equality up to constant terms. It is important to note that if we individually treat eachelement of v j ( f , n) as a free parameter to optimize, the negative log-likelihood will be split into frequency-wise
source separation and dereverberation problems. This results in the problem that permutations of the separated
components of each frequency cannot be uniquely determined. Thus, we typically need to apply permutationalignment to group the separated components of different frequency bins that originate from the same source
signal after we obtain W or apply some constraints to v j ( f , n) to eliminate the permutation ambiguity during
the estimation of W .
2.2 MVAE+: a unified approach using multichannel variational autoencoder
The MVAE method [4] uses a conditional VAE (CVAE) [11] to model and estimate the spectrograms of thesources s j ( f , n). A reverberation-aware extension of the MVAE method, which we call MVAE+, employs aseparation system given by (2) and (3).
Let S = {s( f , n)} f ,n be the complex spectrogram of a particular sound source and c be the corresponding attribute class label whose form is a one-hot vector. Given a set of labeled training samples {SS m , cm }M
m=1 , a
CVAE, consisting of an encoder qφ (z|SS , c) and a decoder pθ (SS |z, c), can be trained by maximizing
J (φ , θ ) = E(SS,c)∼pD (SS,c) [Ez∼qφ (z|SS,c) [log pθ (SS |z, c)] − KL[qφ (z|SS , c)||p(z)]],(8)where E(SS,c)∼pD (SS,c) [·] denotes the sample mean over the training examples and KL[·||·] is the Kullback-Leibler
divergence. Here, we define the decoder distribution as a zero-mean complex Gaussian distribution as follows
so that it has the same form as the LGM (4):pθ (SS |z, c, g) = ∏ NC (s( f , n)|0, v( f , n)),

(9)

f ,n

v( f , n) = g · σθ2 ( f , n; z, c),(10)where σθ2 ( f , n; z, c) denotes the ( f , n)-th element of the decoder output and g represents the global scale of thegenerated spectrogram. As regards the encoder distribution qφ (z|SS , c), we assume a regular Gaussian distribution
qφ (z|SS , c) = ∏ N (z(k)|µφ (k; S , c), σφ2 (k; S , c)),

(11)

k

where z(k), µφ (k; S , c) and σφ2 (k; S , c) represent the k-th element of the latent space variable z and the encoder
outputs µ φ (SS , c) and σ 2φ (SS , c), respectively. The trained decoder distribution can then be used as a generativemodel of the complex spectrogram of the j-th source pθ (SS j |z j , c j , g j ), where z j , c j and g j are the unknown
parameters of the model. This generative model is called the CVAE source model. Note that since c j denotes
the class index of source j, estimating c j corresponds to identifying the class of source j in a test mixture.
The optimization algorithm of MVAE+ consists of iteratively updating the separation matrices W using theiterative projection (IP) method [12], the source model parameters Ψ = {z j , c j } j using backpropagation, theDH ( f , n0 )} f ,n0 using the multichannel linear prediction method and the global scale
dereverberation filter D = {DG = {g j } j using the following update rule:gj ←

wHj ( f )xx( f , n)|2|w
1.
∑
FN f ,n σθ2 ( f , n; z j , c j )(12)

MVAE+ is notable in that (i) it takes full advantage of the strong representation power of DNNs for source
power spectrogram modeling, (ii) the convergence of the source separation algorithm is guaranteed, and (iii) it

6990

is capable of removing the reverberant component in the observed mixture signals. We experimentally confirmed
in [5] that MVAE+ successfully improved on both MVAE and ILRMA+ [8], namely a reverberation-aware extension of ILRMA, showing the effects of the CVAE source model and the frequency-wise convolutive mixture
model.
However, MVAE+ does not address the remaining two drawbacks of MVAE, namely the high computational
cost and the limited source classification accuracy.

3

PROPOSED METHOD

3.1 Auxiliary classifier VAE
The auxiliary classifier VAE (ACVAE) [13] is a variant of CVAE that incorporates an information-theoretic
regularization [14] to enhance the effect of the class label on the decoder output by maximizing the mutual
information between c and S ∼ pθ (S|z, c) conditioned on z. As shown in [6, 13], the regularization term that
we would like to maximize with respect to φ , θ , ψ becomes
L (φ , θ , ψ) = E(S,c)∼pD (S,c),qφ (z|S,c) [Ec∼p(c),S∼pθ (S|z,c) [log(c|S)]].
Since the labeled training samples can also be used to train the auxiliary classifier (c|S), ACVAE also includes
the cross-entropy
I (ψ) = E(S,c)∼pD (S,c) [log(c|S)](13)

in the training criterion. The entire training criterion is thus given by
J (φ , θ ) + λL L (φ , θ , ψ) + λI I (ψ),(14)

where λL ≥ 0 and λI ≥ 0 are weight parameters.
Note that the auxiliary classifier (c|S) both assists the encoder and decoder to learn a more disentangled representation.
3.2 optimization process for proposed method
In this subsection, we describe an optimization algorithm for obtaining D, W , and Ψ. The MVAE+ algorithm
consists of iteratively maximizing the log-likelihood (7) with respect to D, W , and Ψ in turn. One drawback
as regards MVAE+ is that the process of optimizing Ψ has particularly been computationally expensive. Notethat when D and W are fixed, maximizing the log-likelihood (7) with respect to Ψ is equivalent to finding
the maximum point of the distribution p(z j , c j |S j ) for each j. While the MVAE and MVAE+ method usedbackpropagation for finding z j and c j , fMVAE took a different approach to reduce the computational effort.
By using the fact that the trained encoder qφ (z j |S j , c j ) of the CVAE model and the trained auxiliary classifier(c j |S j ) are approximations to the exact posteriors p(z j |S j , c j ) and p(c j |S j ), the distribution p(z j , c j |S j ) can beapproximated as the product of qφ (z j |S j , c j ) and (c j |S j ). Thus, we can search for the points that approximately
maximize p(z j , c j |S j ) by computing the output of (c j |S j ) followed by computing the mean of qφ (z j |S j , c j ).
These values can be obtained simply via forward computations of the two networks. Here, we use this idea offMVAE to update Ψ.
The update rules for D can be derived as in [5, 8]. Let us vectorize {D( f , n0 )} by
2 0

T
T
T
T
T
0 H
I N
0
,
d ∗ ( f ) = [dd T
1 ( f , 1), . . . , d I ( f , 1), d 1 ( f , 2), . . . , d I ( f , 2), . . . , d 1 ( f , N ), . . . ., d I ( f , N )] ∈ C(15)where d i ( f , n0 ) is the i-th column of D ( f , n0 ) and (·)∗ represents the complex conjugate. Then, the update rulefor each d ∗ ( f ) can be derived as a closed form:!−1∗d (f) ←H

∑ X ( f , n)Σw/v( f ,n) X ( f , n)n!
H

∑ X ( f , n)Σw/v( f ,n) x ( f , n) ,

(16)

n

6991

Figure 1. Microphone and source positions, where ◦ and × represent the positions of microphones and sources
respectively.
with
2 0

X ( f , n) = [I ⊗ x T ( f , n − 1), I ⊗ x T ( f , n − 2), . . . , I ⊗ x T ( f , n − N 0 )] ∈ CI×I N ,(17)

wH ( f )w j ( f )wj, which is assumed to be positive definite. Here, ⊗ stands for the Kronecker prodand Σw/v( f ,n) = ∑ j v j ( f ,n)uct.
We employ the following update rules derived on the basis of the IP method [12] to update W :W H ( f , 0)Σy/v j ( f ) )−1 e j ,w j ( f ) ← (Ww j( f ) ← q

w j( f )wHj ( f )Σy/v j ( f )w
w j( f )

,

(18)
(19)

where Σy/v j ( f ) = (1/N) ∑n y ( f , n)yyH ( f , n)/v j ( f , n) and e j denotes the j-th column of the I × I identity matrix.
Therefore, the proposed algorithm is summarized as follows:
1. Train θ , φ and ψ using (14).
2. Initialize z j , c j , G , W and D.
3. Repeat the following update steps for each j.
(a) Update w j ( f ) using (18) and (19).
(b) Update c j ← argmaxc j ∈{1,2,...,C} (c j |S j )(c) Update z j ← µφ (SS j , c j )(d) Update g j using (12).
(e) Update d ∗ ( f ) using (16).

4EXPERIMENTAL RESULTS

To evaluate the effectiveness of the proposed method under highly reverberant environments, we conducted
multi-speaker separation experiments to compare the source separation performance, computational time andsource classification accuracy of the proposed approach with those of ILRMA+ [8] and MVAE+ [5].
4.1 Experimental conditions
For learning the parameters of CVAE for MVAE+, we used the same clean speech samples as in [4, 5]. For
learning the parameters of ACVAE, we used reverberant speech samples that were generated from the same
samples mentioned above and the measured RIRs. Specifically, we used six RIRs involved in the CENSREC-4
database for training networks of ACVAE and used two RIRs excerpted from the RWCP database to generate

6992

Table 1. Conditions of RIRs used for training CVAE source model of the proposed method and generating
observed mixture signals in tests.
Database

CENSREC-4(Used in training phase)

RWCP(Used in test phase)

Recording environment
In-carOfficeLoungeJapanese style bath
Living room
Meeting room
Japanese style room
Meeting room

Reverberation time (RT60 ) [ms]
55 ms
250
500
600
650
650
600 ms780

the multichannel mixture signals. Table 1 and Fig. 1 respectively show the conditions of RIRs and the two
configurations of the microphones and sources we tested. We used utterances of two female speakers “SF1”and “SF2”, and two male speakers “SM1” and “SM2” excerpted from the Voice Conversion Challenge (VCC)
2018 dataset [15] to compose the training and evaluation sets. The audio files for each speaker were manuallysegmented into 116 short sentences (about 7 minutes totally), where 81 and 35 sentences (about 5 and 2 minutes) were provided as training and evaluation sets, respectively. We generated 10 speech combinations for eachspeaker pair, namely SF1+SF2, SF1+SM1, SF2+SM2, and SM1+SM2. Hence there were 80 test signals in total
under each reverberant environment. The length of each signal was about 4 to 7 seconds long. The class labelc was a four-dimensional one-hot vector that indicates the speaker identity.
We resampled all mixture signals at 16 kHz. The STFT was computed using a 256 ms long Hamming window
with 64 ms window shift. For ILRMA+, the basis number K was set at 5. The dereverberation filter length
N’ was set at 3 for JR1 and 4 for OFC, respectively. We run 100 iterations for ILRMA+, 60 iterations for
MVAE+ and 40 iterations for the proposed method. To initialize W H ( f ) and D H ( f , n0 ) for MVAE+ and the
proposed method, we ran ILRMA+ for 30 iterations. We used the same network architectures for the encoderand decoder in this paper as those used in [4, 5], i.e., a three-layer fully-convolutional network with gated linear
units (GLUs) [16] and a three-layer fully-deconvolutional network with GLUs. We also used the same networkarchitectures for ACVAE as those used in [6]. Specifically, architectures of the encoder and decoder were thesame as those described above, and the classifier network consisted of a four-layer fully-convolutional networkwith GLUs and a softmax-layer as the output layer. Adam optimization [17] was used for training networks
and estimating Ψ during the source separation. Note that we must take account of the sum-to-one constraints in
MVAE+ when updating c j , which can be easily implemented by inserting appropriately designed softmax layer
outputs
c j = Softmax(u j ),(20)

and treat u j as the parameter to be estimated instead.
The optimization algorithms were run using an Intel(R) Core(TM) i7-7800X CPU@3.50 GHz and a TITAN V
GPU.
4.2 Results
We took the average of the signal-to-distortion ratios (SDRs), signal-to-interference ratios (SIRs) and signalto-artifact ratios (SARs) [18] over the 80 test signals under each condition to evaluate the source separation
performance, and we measured the computational times of ILRMA+, MVAE+ and the proposed method.
Tab. 2–3 respectively show the separation performances and computational times of ILRMA+, MVAE+ and the
proposed method under the two reverberant conditions. The proposed method was 7.5 times faster than MVAE+
each iteration and slightly faster than ILRMA+ in terms of total computational time when using GPU. We
confirmed that the proposed method achieved a comparable performance in terms of SIR improvement comparedwith MVAE+. The proposed method also outperformed ILRMA+ in terms of SDR and SIR improvements while
SAR was slightly worse than ILRMA+. Further investigation on the SAR degradation is one direction of our

6993

Table 2. Average SDR, SIR and SAR improvements of ILRMA+, MVAE+ and the proposed method. The
values in bold are the highest scores.
RIRs

Methods

JR1
T60 = 600 ms
OFC
T60 = 780 ms

ILRMA+
MVAE+
Proposed
ILRMA+
MVAE+
Proposed

Improvement [dB]
SDR
SIR
SAR
5.06 11.19 1.15
6.66 14.73 2.22
5.50 14.34 0.90
5.46 11.56 1.62
6.89 14.89 2.64
5.99 14.60 1.49

Table 3. Computational times of MVAE+, the proposed method and ILRMA+. MVAE+ and the proposed
method were initialized with by running the ILRMA+ algorithm for 30 iterations in CPU. MVAE+ ran 60
iterations of the optimization algorithm in GPU and the proposed method ran 40 iterations of the optimization
algorithm in CPU or GPU. ILRMA ran 100 iterations in CPU.
MethodMVAE+ (GPU)
Proposed (CPU)
Proposed (GPU)ILRMA+ (CPU)JR1 (RT60 = 600 ms)
Runtime / Iteration [sec] Total [sec]5.935058
389.018216
0.935716
63.547227
0.689945
54.147118
0.694537
70.446281

OFC (RT60 = 780 ms)
Runtime / Iteration [sec] Total [sec]6.156569
409.731895
1.189260
81.256035
0.935897
71.375679
0.951857
96.179699

Table 4. Accuracy rates of source classifications obtained with MVAE+ and the proposed method.
MVAE+
Proposed

all iterations [%]
50.51
78.98

final estimation [%]
74.38
80.00

future work.
To evaluate the performances of source classification, we computed the classification accuracy rates over all
the results estimated at each iteration and those estimated at the final iteration. Table 4 shows the obtainedclassification accuracy rates. We confirmed that the proposed method could estimate the attribute class of eachsource signal more accurately by utilizing the classifier in the optimization process for class label estimation.

5

CONCLUSIONSThis paper proposed a unified approach to simultaneously solving source separation, dereverberation, and sourceclassification problems. In the proposed method, we used the frequency-domain convolutive mixture model tomodel the separation system, and VAEs with an auxiliary classifier (ACVAE) to model and estimate the power
spectrograms and speaker identity of the source signals. The optimization process of the proposed method consists of iteratively updating (i) the spectral parameters of each source by the forward calculation of the auxiliary
classifier VAE, (ii) the separation matrices using the IP method and (iii) the dereverberation filters using multichannel linear prediction. The experimental results showed that the proposed method achieved comparable
performance to MVAE+ in terms of SIR improvement with a source classification rate of 80 % and a reductionof about 83 % in the computational time in each iteration.

6994

ACKNOWLEDGEMENTS
This work was supported by SECOM Science and Technology Foundation, JSPS KAKENHI Grant Number
19H04131, 17H01763 and 18J20059.

REFERENCES[1] Hyvärinen, A.; Oja, E. Independent component analysis: algorithms and applications, in Neural networks,
Vol 13, 2000, pp 411–430.
[2] Kameoka, H.; Yoshioka, T.; Hamamura, M.; Le Roux, J.; Kashino, K. Statistical model of speech signals
based on composite autoregressive system with application to blind source separation, in Proc. LVA/ICA,
2010, pp 245–253.
[3] Kitamura, D.; Ono, N.; Sawada, H.; Kameoka, H.; Saruwatari, H. Determined blind source separation withindependent low-rank matrix analysis, in Audio Source Separation, Springer, Mar. 2018, pp 125–155.
[4] Kameoka, H.; Li, L.; Inoue, S.; Makino, S. Semi-blind source separation with multichannel variational
autoencoder, arXiv preprint arXiv:1808.00892, Aug. 2018.
[5] Inoue, S.; Kameoka, H.; Li, L.; Seki, S.; Makino, S. Joint separation and dereverberation of reverberant
mixtures with multichannel variational autoencoder, in Proc. ICASSP, 2019, pp 96–100.
[6] Li, L.; Kameoka, H.; Makino, S. Fast MVAE: Joint separation and classification of mixed sources basedon multichannel variational autoencoder with auxiliary classifier, in Proc. ICASSP, 2019, pp 546–550.
[7] Yoshioka, T.; Nakatani, T.; Miyoshi, M.; Okuno, HG. Blind separation and dereverberation of speech
mixtures by joint optimization, IEEE Trans. ASLP, Vol 19 (1), 2011, pp 69–84.
[8] Kagami, H.; Kameoka, H.; Yukawa, M. Joint separation and dereverberation of reverberant mixtures withdetermined multichannel non-negative matrix factorization, in Proc. ICASSP, 2018, pp 31–35.
[9] Fev́otte, C.; Bertin, N.; Durrieu, J.-L. Maximum likelihood approach for blind audio source separation
using time-frequency Gaussian models, in Proc. WASPAA, 2005, pp 78–81.
[10] Vincent, E.; Arberet, S.; Gribonval, R. Underdetermined instantaneous audio source separation via local
Gaussian modeling, in Proc. ICA, 2009, pp 775–782.
[11] Kingma, DP.; Mohamedy, S.; Rezendey, DJ.; Welling, M. Semi-supervised learning with deep generativemodels, in Adv. Neural Information Processing Systems (NIPS), 2014, pp 3581–3589.
[12] Ono, N. Stable and fast update rules for independent vector analysis based on auxiliary function technique,in Proc. WASPAA, 2011, pp 189–192.
[13] Kameoka, H.; Kaneko, T.; Tanaka, K.; Hojo, N. ACVAE-VC: Non-parallel many-to-many voice conversion
with auxiliary classifier variational autoencoder, arXiv preprint arXiv:1808.05092, 2018.
[14] Chen, X.; Duan, Y.; Houthooft, R.; Schulman, J.; Sutskever, I.; Abbeel, P. Infogan: Interpretable representation learning by information maximizing generative adversarial nets, in Adv. NIPS, 2016, pp 2172–2180.
[15] Lorenzo-Trueba, J.; Yamagishi, J.; Toda, T.; Saito, D.; Villavicencio, F.; Kinnunen, T.; Ling, Z. The Voice
Conversion Challenge 2018: Promoting development of parallel and nonparallel methods, arXiv preprint
arXiv:1804.04262, Apr. 2018.
[16] Dauphin, YN.; Fan, A.; Auli, M.; Grangier, D. Language modeling with gated convolutional networks,
arXiv preprint arXiv:1612.08083, 2016.
[17] Kingma, DP.; Ba, J.; Adam: A method for stochastic optimization, in Proc. ICLR, 2015.
[18] Vincent E.; Gribonval, R.; Févotte, C. Performance measurement in blind audio source separation, IEEE
Trans. ASLP, Vol 14 (4), 2006, pp 1462–1469.

6995