Journal of Signal Processing, Vol.25, No.4, pp.145-149, July 2021

SELECTED PAPER AT NCSP'21

Single-Channel Multispeaker Separation with Variational Autoencoder Spectrogram Model
Naoya Murashima1 , Hirokazu Kameoka2 , Li Li1 , Shogo Seki2 and Shoji Makino1,3
1

Graduate School Science and Technology, University of Tsukuba, Ibaraki 305-8573, Japan
E-mail: naodrrb@gmail.com, {lili@mmlab.cs, maki@tara}.tsukuba.ac.jp
2
NTT Communication Science Laboratories, Nippon Telegraph and Telephone Corporation, Kanagawa 243-0124, Japan
E-mail: hirokazu.kameoka.uh@hco.ntt.co.jp
3
Graduate School of Information, Production and Systems, Waseda University, Fukuoka 808-0135, Japan
E-mail: s.makino@waseda.jp
1. Abstract
This paper deals with single-channel speaker-dependent
speech separation. While discriminative approaches using
deep neural networks (DNNs) have recently proved powerful, generative approaches, including methods based on
non-negative matrix factorization (NMF), are still attractive
because of their flexibility in handling the mismatch between training and test conditions. Although NMF-based
methods work reasonably well for particular sound sources,
one limitation is that they can fail to work for sources with
spectrograms that do not comply with the NMF model. To
address this problem, attempts have recently been madeto replace the NMF model with DNNs. With a similar
motivation to these attempts, we propose in this paper avariational autoencoder (VAE)-based monaural source separation (VASS) method using a conditional VAE (CVAE)
for source spectrogram modeling. We further propose an
extension of the VASS method, called the discriminative
VASS (DVASS) method, which uses a discriminative criterion for model training so that the separated signals directly become optimal. Experimental results revealed that
the VASS method performed better than an NMF-based
method, and the DVASS method performed better than the
VASS method.
2. Introduction
Speech separation is a technique of separating the signalof each speaker from a mixture signal of multiple speakers
and can be used to improve the accuracy of speech recognition and the quality of voice communication. A discriminative approach using a deep neural network (DNN) has recently proved powerful in single-channel source separation
tasks [1–4]. The general idea is to train a DNN that predicts Time-Frequency (TF) masks or TF embeddings froma given mixture signal based on spectro-temporal features.
Recently, methods of training a DNN that directly predicts
the waveform of each speaker have also been proposed. Although these methods can achieve reasonably good separation, they can fail to work if there is a large mismatch between training and test conditions caused by, for example,
reverberation.
Generative approaches, including non-negative matrix
factorization (NMF) [5], are attractive in the flexibility inaddressing the mismatch between training and test conditions. For example, this can be achieved by explicitly incorJournal of Signal Processing, Vol. 25, No. 4, July 2021

porating the generative process that causes the mismatch
into the generative model of observed signals, and simultaneously estimating the parameters of the entire model during the test time. The idea of NMF is to approximate thespectrum of a mixture signal observed in each short-term
frame as a linear sum of a limited number of basis spectra scaled by time-varying amplitudes. In supervised NMF(SNMF) [6], separation is achieved by fitting the basis spectra, pretrained on each source, to an observed mixture signal and then applying a Wiener filter. However, one problem with the SNMF method is that the training criterion forthe basis spectra is inconsistent with the objective function
at the test time. In other words, the basis spectra are nottrained so that the separated signals at the test time become
optimal. Discriminative NMF (DNMF) [7] was later proposed to resolve this inconsistency. Specifically, the idea is
to make the training scenario consistent with the test scenario, and train the basis spectra so that the separated signals (the outputs of Wiener filters) directly become optimal.
Although these NMF-based methods work reasonably wellfor particular sound sources, one limitation is that they canfail to work for sources with spectrograms that do not comply with the NMF model.
In recent years, with the aim of modeling source spectrograms more flexibly than by the NMF model, generative
approach-based methods using DNNs have been proposed[8–16]. For multichannel source separation under a determined condition, a method that uses the conditional variational autoencoder (CVAE) [17] for source spectrogram
modeling, called the multichannel VAE (MVAE) method,has been proposed. This method has been shown to significantly outperform independent low-rank matrix analysis [18, 19], which uses the NMF model for spectrogram
modeling. This indicates that the CVAE is better than the
NMF model at expressing the spectrogram of each source
and correctly discriminating the spectrogram of one sourcefrom that of another. With the same motivation, a monaural
speech enhancement method (VAE-NMF) [13, 14] and its
multichannel extension [15, 16] have also been proposed.
Motivated by the success of the MVAE method, we
propose in this paper a VAE-based monaural source separation (VASS) method using a CVAE for source spectrogram modeling. We further propose a discriminative
counterpart of the VASS method, called the discriminative
VASS (DVASS) method, namely, an extension to the VASS
method equivalent to the extension from SNMF to DNMF.
145

3. Conventional Methods

common way of training Wj is to solve′{Ŵj , Ĥj } = argmin D(S̃j |Wj Hj )3.1 Problem formulation(5)

Wj ,Hj′where S̃j is a concatenation of the power spectrograms of
We consider a situation where a mixture of signals of J all training utterances of speaker j. D is a cost function thatspeakers is observed. Let Y = {y(f, n)}f,n ∈ CF ×N and measures the dissimilarity of S̃′j and Wj Hj , such as the
Sj = {sj (f, n)}f,n ∈ CF ×N be the complex spectrograms Itakura-Saito (IS) divergence. At the test time, given theof the observed signal and the signal of the jth speaker, power spectrogram Ỹ of the mixture signal, we must solve
where f and n are the frequency and time indices, respecĤ = argmin D(Ỹ|ŴH)(6)
tively. Let us now assume that sj (f, n) independently folHlows a zero-mean complex Gaussian distribution with variwhere Ŵ = [Ŵ1 , . . . , ŴJ ] denotes the basis matrix conance vj (f, n) = E[|sj (f, n)|2 ].
(1) taining the pretrained basis spectra. Ŵj Ĥj correspondssj (f, n) ∼ NC (sj (f, n) | 0, vj (f, n))Equation (1) is called the local Gaussian model (LGM) [20, to the estimate of the power spectrogram associated with21]. When Sj and Sj ′ (j = j ′ ) are independent, from Y = speaker j. The complex spectrogram Ŝj of speaker j canthen be obtained as
j Sj y(f, n), we can show that y(f, n) follows
Ŵj Ĥj(2)
y(f, n) ∼ NC (y(f, n) | 0, v(f, n))⊙ Y.
(7)Ŝj = 
where v(f, n) = j vj (f, n). When Vj = {vj (f, n)}f,n ,j ′ Ŵj ′ Ĥj ′the negative log-likelihood − log p(Y|V) of V ={V1 , . . . , VJ } given Y is equivalent up to a constant term
to the IS divergence between ỹ(f, n) = |y(f, n)|2 and 3.3 DNMF
v(f, n),If we assume the use of the Wiener filter output, Eq. (7),  ỹ(f, n)ỹ(f, n)toobtainthe signal of each speaker, the training and test ob− log−1(3)
DIS (Ỹ|V) =jectives become inconsistent. Namely, the basis spectra arev(f, n)v(f, n)f,n
not necessarily trained in such a way that the separated sigwhere Ỹ = {ỹ(f, n)}f,n . Since Y and S1 , . . . , SJ are nals at test time will be optimal. DNMF has been developed
jointly Gaussian, the minimum mean square error estima- to address this inconsistency in SNMF, based on the idea of
tor of Sj given Y and V1 , . . . , VJ is given bytraining the basis spectra in such a way that the separated
signals become optimal at the test time.
Vj
E[Sj |Y] = ⊙Y(4)With SNMF, at the test time, the basis matrix Ŵ is used′j ′ Vj·notonly for estimating Ĥ from Ỹ in Eq. (6) but also forwhere · and ⊙ respectively denote elementwise division
constructing
the Wiener filter in Eq. (7). However, the basis
and multiplication. Note that the multiplicative factor of
Eq. (4) is called the Wiener mask. Equation (4) implies that matrices used in these steps do not have to be the same;once V1 , . . . , VJ are estimated, we can estimate the signal rather, it would be more advantageous at the test time toof each speaker. Thus, the single-channel speech separation treat them as different variables and train them separately.
problem can be formulated as the problem of estimating We thus use W and B to denote the basis matrices at thesebe used to train them.
V1 , . . . , VJ with Eq. (3) as the objective function, under steps, and discuss what criteria should′some constraint or assumption imposed on V1 , . . . , VJ .
By using the power spectrogram Ỹ = {|y ′ (f, n)|2 }f,n ofa random mixture of training utterances as the input andthat of each of the utterances as the regression target, we
can train W and B using the process that exactly mimics
3.2 SNMF
the test scenario. After solving Eq. (6) by using the basis
SNMF is a monaural speech separation method that uses matrix Ŵ obtained via Eq. (5), we can train B so that the
the NMF model to express Vj . Namely, Vj is represented output of Eq. (7) matches the regression target as closely as
possible. Therefore, in DNMF, the training objective for B
as the product of two non-negative matrices Wj Hj , i.e., can be defined as
vj (f, n) = k wj,k (f ). Here, the basis matrix Wj is as′  Bj Ĥj′sumed to be trained prior to separation using the training
D |Sj |(8)
⊙ |Y |
B̂ = argmin
 BĤ
utterances of each speaker. Representing the spectrogramBjas the product of two non-negative matrices (a low-rank′′′matrix) corresponds to representing the spectra observed in where |Y | and |S1 |, . . . , |SJ | denote the magnitude spectrogramsofthemixturesignaland the source signals, reeach frame as a non-negative combination of a finite numspectively.
Note
that
here
|
·
|
is
used
to denote the operation
ber of basis spectra. Therefore, we can expect to obtain basis spectra unique to each speaker through the pretraining of taking the elementwise absolute value of a matrix. At theof Wj . At the test time, after fitting WH to the spectro- test time, the separated signals can be obtained by the samegram of a test mixture signal Y with W = [W1 , . . . , WJ ] process as Eq. (6) and refeqWiener3 using Ŵ trained via
fixed at the pretrained basis spectra, V1 , . . . , VJ can be es- Eq. (5) and B̂ trained via Eq. (8).
T T
timated using the estimate of H = [HTBoth SNMF and DNMF assume that each speech spec1 , . . . , HJ ] . The
source signals can then be separated out using Eq. (4). A trogram can be represented by a low-rank matrix. However,146

Journal of Signal Processing, Vol. 25, No. 4, July 2021

this assumption is not always accurate and limits the sepa- KL divergence. Thus, minimizing Eq. (12) amounts to distribution fitting. Note that the second term in Eq. (12) isration performance of both methods.
equal up to a constant term to the expectation of the IS divergence between Ŝ = {|s(f, n)|2 }f,n and v(f, n) owing
4. Proposed Methods
to the decoder distribution defined in the same form as the
LGM.
4.1 CVAE source model
In the CVAE source model, the latent variable z can beinterpreted as context information corresponding to the linSince the matrix product representation Wh can be re- guistic content of S and the decoder NN parameter θ as thegarded as a single-layer linear fully connected neural net- quantity that governs the mapping from the context inforwork (NN) with h as the input, a deeper model with mul- mation to the spectrogram. In this respect, z and θ can betiple nonlinear layers can be a more powerful alternative to regarded as corresponding to the coefficient (activation) mathe NMF model. One idea would be to express the vari- trix H and basis matrix W in the NMF model, respectively.
ance vj (f, n) in the LGM, Eq. (1), as the output of a DNN.
As described below, this corresponds to a special case ofa VAE. The MVAE method, mentioned earlier, is a multi- 4.2 Proposed method 1: VASS methodchannel source separation method that uses a CVAE, condiThe VASS method corresponds to SNMF in which thetioned on a speaker code, as the source spectrogram model
based on this idea. This model is called the CVAE source NMF-type source model is replaced by the CVAE source
model. In this paper, a single-channel speech separation model. Similarly to SNMF, the VASS method consistsof pretraining the source model (training step), fitting themethod based on the CVAE source model is proposed.
source model to the spectrogram of an observed mixture
A CVAE is a type of autoencoder consisting of an en- signal (test step 1), and extracting source signals using thecoder and decoder. It is unique in that both the encoder and Wiener mask (test step 2). Owing to the conditional modeldecoder are modeled in the form of parametric probabil- ing, the CVAE source model with a single set of parameters
ity distributions, and both distributions are conditioned on can be made to represent the spectrograms of all speakers in
auxiliary variables. Let S be the complex spectrogram of the training set by training the parameters using Eq. (12) asa particular speaker’s utterance and c be the speaker code.
Here, we assume that the speaker code c is represented as a the objective. Let θ̂ be the parameters of the CVAE source
one-hot vector. Now, we condition the decoder distribution model obtained after the training step. The first step at theon c and further define it as a zero-mean complex Gaussian test time (test step 1) can be formulated as a maximum likedistribution so that it has the same form as the LGM (Eq. lihood estimation problem.
(1)):
(13)
{Ẑ, Ĉ, η̂} = argmax log p(Y|Z, C, η)Z,C,η
NC (s(f, n)|0, v(f, n))(9)pθ (S|z, c, η) =where the likelihood function p(Y|Z, C, η) can be derived
f,non the basis of the assumption that the complex spectrogram(10) y(f, n) of a mixture signal follows
v(f, n) = η · σθ2 (f, n; z, c)y(f, n) ∼NC (y(f, n)|0, v(f, n))(14)where σθ2 (f, n; z, c) denotes the (f, n)th element of the de
coder network output σ 2θ (z, c), z represents a latent variablev(f, n) =vj (f, n)(15)generated from the encoder distribution, and η is a parame  jter corresponding to the scale (total energy) of S. Next, weηj σ 2 (f,n;zj ,cj )θ̂define the encoder distribution as a Gaussian distribution As mentioned in sect. 3.1, − log p(Y|Z, C, η) is equal up
with diagonal covariance:
to a constant term to the IS divergence between |y(f, n)|2(11) and v(f, n). Hence, this problem is equivalent to findingqφ (z|S, c) = N (z|µφ (S, c), diag(σ 2φ (S, c))),and define the prior distribution p(z) as a standard Gaus- Z, C, and η that minimize DIS (Ỹ|V) with θ̂ fixed, where
sian distribution. Here, the mean µφ (S, c) and varianceỸ = {|y(f, n)|2 }f,n . Once Ẑ, Ĉ, and η̂ are estimated,
σ 2φ (S, c) are assumed to be the encoder network out- the signal of each speaker can be obtained using the Wiener
puts. Both the unknown network parameters θ and φ filter (test step 2)
are trained using a set of speaker-labeled training samples
ηj σ 2θ̂ (ẑj , ĉj ){Sm , cm }M
.
The
goal
is
to
train
θ
and
φ
so
that
the
en⊙Y
(16)
=
S
j
m=1
2
j ′ ηj ′ σ θ̂ (ẑj ′ , ĉj ′ )coder distribution qφ (z|S, c) becomes consistent with theNote that there are several possible ways of solving Eq.
posterior pθ (z|S, c) ∝ pθ (S|z, c)p(z). The decoder distribution with the resulting θ is expected to fit the true dis- (13). The first is to simply optimize Z, C, and η ustribution of the spectrograms of each speaker reasonably ing the gradient method (backpropagation for Z and C)
well. If we define the training objective as the Kullback- with log p(Y|Z, C, η) or DIS (Ỹ|V) as the criterion. The
Leibler (KL) divergence between qφ (z|S, c) and pθ (z|S, c), second method is to optimize them using the expectationthe training objective is equal up to a constant term tomaximization (EM) algorithm, treating the complex spectrogram sj (f, n) of each speaker as the latent variable.
J (φ, θ) = E(S,c)∼pD (S,c) KL[qφ (z|S, c)||p(z)]We can keep increasing the log-likelihood log p(Y|Z, C, η)− Ez∼qφ (z|S,c) [log pθ (S|z, c)](12) by iteratively increasing an auxiliary function defined aswhere E(S,c)∼pD (S,c) [·] denotes the sample mean over the ES∼p(S|Y,Z′ ,C′ ,η′ ) [log p(S|Z, C, η)] through iterative uptraining examples {Sm , cm }M
m=1 , and KL[·||·] denotes the dates called the E- and M-steps. The M-step is the process
Journal of Signal Processing, Vol. 25, No. 4, July 2021

147

of updating Z, C, and η to increase the auxiliary function.
Z and C can be updated by backpropagation. When Z and
C are fixed, η that maximizes the auxiliary function can bederived analytically. The E-step is the process of recomputing the auxiliary function each time Z, C, and η are updated
by substituting the updated Z, C, and η into Z′ , C′ , andlog p(S|Z, C, η) is split into J indiη ′ respectively. Since vidual terms, namely, j f,n log p(sj (f, n)|0, vj (f, n)),(z1 , c1 , η 1 ), . . . , (zJ , cJ , η J ) can be updated in parallel at the M-step. The third method is to increaselog p(Y|Z, C, η) iteratively by using an auxiliary function
with another form as in [12]. Owing to space limitations,the details and derivations of the algorithms for these threemethods are omitted. In the experiments described below,
we used the method based on the EM algorithm.
4.3 Proposed method 2: DVASS method
In the VASS method, as in SNMF, the training objective for the parameter θ of the CVAE source model does notmake the separated signals (Wiener filter outputs) optimal at
the test time. To address this mismatch between the training and test objectives, we further propose improving the
VASS method by following the idea of DNMF. Recall thatthe idea of DNMF is to treat the basis matrix responsible
for obtaining the coefficient matrix and that responsible forconstructing the Wiener filter as separate variables. In thesame manner, we treat the CVAE source model parameters
responsible for obtaining Eq. (13) and those responsible forconstructing the Wiener filter as separate variables, and denote them by θ and ϑ, respectively. As in DNMF, we cantrain these parameters by following the process that exactly
mimics the speech separation process at the test time. Let
Ẑ, Ĉ, and η̂ represent the values obtained using Eq. (13),with θ̂ fixed at the value obtained using Eq. (12). By using Ẑ, Ĉ, and η̂, we can train ϑ to match the output of Eq.
(16) to the target signal as closely as possible. The training
objective can be defined as η σ 2 (ẑ , ĉ )j ϑ j j′ ′
⊙ |Y |
D |Sj | ϑ̂ = argmin
 j ′ ηj ′ σ 2ϑ (ẑj ′ , ĉj ′ )
ϑ
j

(17)
At the test time, the separated signals can be obtained byevaluating Eq. (13) and Eq. (16) using the trained θ and
ϑ. An overview of the DVASS method is shown in Fig. 1,
where the criterion for Eq. (17) is denoted as Jre (ϑ).
5. Experimental Evaluations

The proposed method was evaluated on a single-channel
speech separation task of separating two speakers. We
chose the SNMF and DC [1] methods as baseline methods
for comparison. As the experimental data, we used speechsamples of the CMU ARCTIC database [22]. We used a set
of utterances of two female (‘clb’and ‘slt’) and two male(‘bdl’ and ‘rms’) speakers. For each speaker, we used 1000
utterances for training and 132 utterances for testing. We
generated 81 speech mixtures for three speaker combinations: bdl+clb, bdl+rms, and clb+slt. Each test mixture
signal was generated so that the energy of each speaker is148

equal. We generated 560 mixture signals Y′ used for training ϑ in the same manner. All the speech signals were resampled at 8 [kHz] and STFT analysis was conducted witha frame length of 512 [ms] and a hop length of 256 [ms]. In the VASS method, we used a three-layer fully convolutional network with gated linear units and a three-layer
fully deconvolutional network with gated linear units as theencoder and decoder networks, respectively, in the CVAE
model, as in [8]. In the DVASS method, we used the same
network architectures for the encoder and decoder. We used
Adam [23] for NN training and updating the model parameters z and c. In the VASS and DVASS methods, the initiallyseparated signals were obtained by SNMF run for 100 iterations, and z was initialized by feeding the initially separated
signals into the encoder. For each paired training sample(S, c), we fixed c at a one-hot vector corresponding to the
speaker of S. The VASS and DVASS methods were run for
two iterations. In SNMF, the number of bases was set to 10
for each source, and the KL divergence criterion was usedas D. As the evaluation metrics, we used the scale-invariant
signal-to-distortion ratio (SDR), scale-invariant signal-toinference ratio (SIR), and scale-invariant signal-to-artifact
ratio (SAR) [24] between the reference and separated signals.
The experimental results are shown in Figure 2. Compared with the baseline SNMF method, the high separation
performance of the proposed VASS method was confirmed.
The performance difference between the SNMF and VASS
methods may reflect the difference in the ability of eachsource model to achieve separation. The DVASS method
showed a higher separation performance than the VASS
method in all metrics. This confirms the effectiveness of
discriminative training. However, we also confirmed thatthe DVASS method still had room for improvement up to
the high separation performance of the DC method.
6. Conclusion
In this paper, we proposed the VASS method as a singlechannel speech separation method using the CVAE source
model and also proposed the DVASS method, which trains
the CVAE source model based on a discriminative criterion. The effectiveness of the proposed method was investigated through specific two-speaker separation experiments. The experimental evaluation showed that both the
VASS and DVASS methods performed better than SNMF,
and the DVASS method performed better than the VASS
method.
References[1] J. R. Hershey, Z.Chen, J. Le Roux and S. Watanabe: Deep clustering: Discriminative embeddings forsegmentation and separation, 2016 IEEE Int. Conf.
Acoust. Speech Signal Process., pp. 31–35, 2016.
[2] Y. Liu and D. Wang: Divide and conquer: A
deep CASA approach to talker-independent monaural
speaker separation, IEEE/ACM Trans. Audio SpeechLang. Process, Vol. 27, No. 12, pp. 2092–2102, 2019.
[3] J. Le Roux, G. Wichern, S. Watanabe, A.Sarroff and
J. R. Hershey: Phasebook and friends: Leveraging
discrete representations for source separation, IEEE J.
Journal of Signal Processing, Vol. 25, No. 4, July 2021

Figure 1: Schematic overview of DVASS
Sel. Top. Signal Process., Vol. 13, No. 2, pp. 370–382,
2019.
[4] D. Wang and J. Chen: Supervised speech separation
based on deep learning: An overview, IEEE/ACM
Trans. Audio Speech Lang. Process., Vol. 26, No. 10,
pp. 1702–1726, 2018.
[5] D. D. Lee and H. S. Seung: Algorithms for nonnegative matrix factorization, Adv. Neural Inf. Process. Syst., pp. 556–562, 2001.
[6] P. Smaragdis, B. Raj and M. Shashanka: Supervisedand semi-supervised separation of sounds from singlechannel mixtures, 2007 Int. Conf. Independent Compon. Anal. Signal Sep., pp. 414–421, 2007.
[7] F. Weninger, J. Le Roux, J. R. Hershey and S. Watanabe: Discriminative NMF and its application to singlechannel source separation, 2014 Annu. Conf. Int.
Speech Commun. Assoc., pp. 865–869, 2014.
[8] H. Kameoka, L. Li, S. Inoue and S. Makino: Supervised determined source separation with multichannel
variational autoencoder, Neural Comput., Vol. 31, No.
9, pp. 1891–1914, 2019.
[9] A. A. Nugraha, A. Liutkus and E. Vincent: Multichannel audio source separation with deep neural networks, IEEE/ACM Trans. Audio Speech Lang. Process., Vol. 24, No. 9, pp. 1652–1664, 2016.
[10] N. Makishima, S. Mogami, N. Takamune, D. Kitamura, H. Sumino, S. Takamichi, H. Saruwatari and
N. Ono: Independent deeply learned matrix analysis
for determined audio source separation, IEEE/ACM
Trans. Audio Speech Lang. Process, Vol. 27, No. 10,
pp. 1601–1615, 2019.
[11] L. Li, H. Kameoka and S. Makino: FastMVAE:
Joint separation and classification of mixed sources
based on multichannel variational autoencoder with
auxiliary classifier, IEEE Access, Vol. 8, No. 1, pp.
228740–228753, 2020.
[12] S. Seki, H. Kameoka, L. Li, T. Toda and K. Takeda:
Generalized multichannel variational autoencoder forunderdetermined source separation, IEEE Access,Vol. 7, No. 1, pp. 168104–168115, 2019.
[13] Y. Bando, M. Mimura, K. Itoyama, K. Yoshii and
T. Kawahara: Statistical speech enhancement basedon probabilistic integration of variational autoencoderand non-negative matrix factorization, 2018 IEEE Int.
Conf. Acoust. Speech Signal Process., pp. 716–720,
2018.
Journal of Signal Processing, Vol. 25, No. 4, July 2021

Figure 2: Average separation performance [dB][14] S. Leglaive, L. Girin and R. Horaud: A variance modeling framework based on variational autoencoders
for speech enhancement, 2018 Int. Workshop Mach.
Learn. Signal Process., 2018.
[15] K. Sekiguchi, Y. Bando, K. Yoshii and T. Kawahara:
Bayesian multichannel speech enhancement with adeep speech prior, 2018 Asia Pac. Signal Inf. Process.
Assoc. Annu. Summit Conf., pp. 1233–1239, 2018.
[16] S. Leglaive, L. Girin and R. Horaud: Semi-supervised
multichannel speech enhancement with variational
autoencoders and non-negative matrix factorization,
2019 IEEE Int. Conf. Acoust. Speech Signal Process.,
pp. 101–105, 2019.
[17] D. P. Kingma, D. J. Rezende, S. Mohamed and M.
Welling: Semi-supervised learning with deep generative models, Adv. Neural Inf. Process. Syst., 2014.
[18] H. Kameoka, T. Yoshioka, M. Hamamura, J. Le Rouxand K. Kashino: Statistical model of speech signals
based on composite autoregressive system with application to blind source separation, 2010 Int. Conf. Latent Var. Anal. Signal Sep., pp. 245–253, 2010.
[19] D. Kitamura, N. Ono, H. Sawada, H. Kameoka and H.
Saruwatari: Determined blind source separation unifying independent vector analysis and non-negative
matrix factorization, IEEE/ACM Trans. Audio SpeechLang. Process., Vol. 24, No. 9, pp. 1626–1641, 2016.
[20] C. Fevotte and J. F. Cardoso: Maximum likelihood approach for blind audio source separation using timefrequency Gaussian source models, 2005 IEEE Workshop Appl. Signal Process. Audio Acoust., pp. 78-–81,
2005.
[21] E. Vincent, S. Arberet and R. Gribonval: Underdetermined instantaneous audio source separation via local Gaussian modeling, 2009 Int. Conf. Independent
Compon. Anal. Signal Sep., pp. 775-–782, 2009.
[22] J. Kominek and A. W. Black: The CMU arctic speech
databases, 2004 ISCA Speech Synth. Workshop, pp.
223–224, 2004.
[23] D. P. Kingma and J. Ba: Adam: A method for stochastic optimization, 2015 Int. Conf. Learn. Represent.,
2015.
[24] J. Le Roux, S. Wisdom, H. Erdogan and J. R. Hershey: SDR – Half-baked or well done?, 2019 IEEE
Int. Conf. Acoust. Speech Signal Process., pp. 626–
630, 2019.

149