ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | 978-1-7281-7605-5/20/$31.00 ©2021 IEEE | DOI: 10.1109/ICASSP39728.2021.9414884

SEPNET: A DEEP SEPARATION MATRIX PREDICTION NETWORK FOR
MULTICHANNEL AUDIO SOURCE SEPARATION
Shota Inoue1 , Hirokazu Kameoka2 , Li Li1 , Shoji Makino1
1

2

University of Tsukuba, JapanNTT Communication Science Laboratories, NTT Corporation, Japans.inoue@mmlab.cs.tsukuba.ac.jp, hirokazu.kameoka.uh@hco.ntt.co.jp

array responses to ﬁnd clues for separation matrix estimation. For
example, independent vector analysis (IVA) [2–4] is a frequencydomain method that solves frequency-wise source separation andpermutation alignment simultaneously by assuming that the magnitudes of the frequency components originating from the same source
tend to vary coherently over time. Determined multichannel nonnegative matrix factorization [5], later called independent low-rankmatrix analysis (ILRMA) [6, 7], is another relatively recent example that adopts the non-negative matrix factorization (NMF) concept to model the power spectrogram of each source. Recently, several attempts have also been made to combine deep neural networks(DNNs) with the ICA-based methods or other methods for further
improvement [8–15]. Their aim is to improve the representation
power of the source spectrogram model employed in multichannel
NMF [5, 16, 17] or ILRMA so that better separation matrices can befound.

ABSTRACTIn this paper, we propose SepNet, a deep neural network (DNN)
designed to predict separation matrices from multichannel observations. One well-known approach to blind source separation (BSS)
involves independent component analysis (ICA). A recently developed method called independent low-rank matrix analysis (ILRMA)
is one of its powerful variants. These methods allow the estimationof separation matrices based on deterministic iterative algorithms.
Speciﬁcally, ILRMA is designed to update the separation matrix
according to an update rule derived based on the majorizationminimization principle. Although ILRMA performs reasonably
well under some conditions, there is still room for improvement interms of both separation accuracy and computation time, especiallyfor large-scale microphone arrays. The existence of a deterministic
iterative algorithm that can ﬁnd one of the stationary points of the
BSS problem implies that a DNN can also play that role if designed
and trained properly. Motivated by this, we propose introducing a
DNN that learns to convert a predeﬁned input (e.g., an identity matrix) into a true separation matrix in accordance with a multichannel
observation. To enable it to ﬁnd one of the multiple solutions corresponding to different permutations of the source indices, we further
propose adopting a permutation invariant training strategy to train
the network. By using a fully convolutional architecture, we candesign the network so that the forward propagation can be computed
efﬁciently. The experimental results revealed that SepNet was able
to ﬁnd separation matrices faster and with better separation accuracy
than ILRMA for mixtures of two sources.

All the methods above involve iterative procedures for estimating the separation matrix. A fast update rule for the separation matrix has been proposed in [4]. Although this update rule allows
the separation matrix to converge with a small number of iterations,the computational cost per iteration can be expensive especially forlarge-scale microphone arrays. This can be problematic, for example, when implementing real-time BSS systems. To address this,a faster update rule has recently been proposed [18]. While these
update rules are derived manually, in this paper, we take a learningbased approach with the aim of automatically obtaining even fasterand more accurate update rules using a DNN. The idea is similar in
spirit to the concept of deep unfolding [19] in that each layer of a
DNN is interpreted as a single update step in an iterative algorithm.

Index Terms— Blind source separation, microphone array,deep neural network, permutation invariant training

The existence of a deterministic iterative algorithm that leads toone of the stationary points of the BSS problem motivates us to expect that a DNN, if designed and trained properly, can also play thatrole. Namely, we can think of a DNN that converts a predeﬁned
input (corresponding to an initial point of an iterative algorithm)
into the true separation matrix in accordance with a multichannel
observation. However, implementing such a DNN is not necessarilystraightforward, since the BSS problem is an ill-posed one that canhave multiple solutions corresponding to different permutations of
the source indices. A deterministic BSS algorithm can ﬁnd one ofthese solutions, but in most cases it cannot predict in advance which
permutation it will end up with. The problem is that the resulting
permutation varies depending on an initial point of the iterative algorithm and observed signals. Thus, naively using the source signals arranged in some ﬁxed prespeciﬁed order as a regression target
for model training would result in a useless model, namely one that
produces a separation matrix that does not achieve separation at all.
To allow the DNN to ﬂexibly ﬁnd one of the desired solutions in

1. INTRODUCTION
Blind source separation (BSS) is a technique for extracting individual source signals from mixture signals recorded by a microphonearray without any prior information about the source signals and thetransfer characteristics between the sources and microphones. One
widely used approach for the determined BSS problem is independent component analysis (ICA) [1], which achieves source separation by ﬁnding a separation matrix that makes the separated signals
as statistically independent as possible. Many ICA-based methods
are formulated in the frequency domain. Frequency-domain methods enable fast implementations compared to methods formulated
in the time domain. In addition, they allow us to utilize various
models for the time-frequency representations of source signals andThis work was supported by JST CREST Grant Number JPMJCR19A3
and JSPS KAKENHI Grant Number 19H04131.

978-1-7281-7605-5/21/$31.00 ©2021 IEEE

191

ICASSP 2021

Authorized licensed use limited to: NTT. Downloaded on November 07,2023 at 09:02:46 UTC from IEEE Xplore. Restrictions apply.

accordance with an observation, we further propose adopting a permutation invariant training (PIT) strategy [20] to train the network.
On top of this, we use a fully convolutional architecture to design the
network so that the forward propagation can be computed efﬁciently.
We call this model SepNet, which stands for the Separation matrix
prediction Network.
The rest of this paper is structured as follows. Section 2 formulates the BSS problem and brieﬂy reviews ILRMA, Section 3 describes the details of the architecture of SepNet and discusses thetraining process, and Section 4 presents the experimental results.

predefined
input

observed
mixtureestimated
separation matrix
DNN
Block

DNN ・・・
Block

DNN
Block

separated
signals
Demix

Fig. 1: Entire architecture of proposed method for estimating separation matrix from predeﬁned input and observed mixture.
DNN Block++

2. PROBLEM FORMULATION
2.1. Determined BSS
We consider a determined situation where J source signals are observed by I microphones (J = I). Let xi (f, n) and sj (f, n) denotethe short-time Fourier transform (STFT) coefﬁcients of the signal
observed at the ith microphone and the jth source signal, where fand n are the frequency and time indices, respectively. Under a determined situation, we can use a separation system of the forms(f, n) = WH (f )x(f, n),W(f ) = [w1 (f ), . . . , wJ (f )] ∈ C

Fig. 2: Diagram of building blocks (DNN Blocks). Each building
block consists of two layers with no learnable parameters (Lpre ,Lpost ) and two layers with learnable parameters (Pθ , Qϕ ).
P
where Σx/vj (f ) = (1/N ) n x(f, n)xH (f, n)/vj (f, n) is aweighted spatial covariance matrix and ej denotes the j-th columnof the I × I identity matrix. The ILRMA algorithm is guaranteed
to converge to a stationary point of (3) and experimentally shown toconverge quickly with a small number of iterations. As (4) shows,each iteration of the ILRMA algorithm includes a matrix inversion,
which becomes computationally expensive especially when J islarge.

(1)
I×J

,

(2)

to describe the relationship between the observed signals x(f, n) =[x1 (f, n), . . . , xI (f, n)]T ∈ CI and sources s(f, n) = [s1 (f, n),. . . , sJ (f, n)]T ∈ CJ , where W(f ) is called the separation matrixand (·)H denotes Hermitian transpose. The aim here is to estimate
W = {wj (f )}j,f from the observations X = {xi (f, n)}i,f,n .

3. SEPNET
2.2. ILRMA
If we assume each source j to independently follow the local Gaussian model [21, 22] sj (f, n) ∼ NC (sj (f, n)|0, vj (f, n)), where
vj (f, n) denotes the power spectral density of the (f, n)th timefrequency element, the negative log-likelihood of V = {vj (f, n)}f,n,j
and W = {wj (f )}j,f given the observed mixture signals X ={xi (f, n)}i,f,n is given asc

L(V, W) = − 2N log det WH (f )X|wHj (f )x(f, n)|2+
log vj (f, n) +,vj (f, n)(3)

To obtain a fast and accurate separation matrix estimator, we propose the idea of using an appropriately sized DNN as an alternative
to an iterative algorithm for ﬁnding W. We call this DNN SepNet.
To this end, we consider a DNN that takes X = {x(f, n)}f,n anda set W (0) of identity matrices as the inputs, and produces the set
W (k) = {W(k) (f )}f of separation matrix estimates from eachblock k. The input W (0) and each block k of this DNN can bethought of as corresponding to the initial point and a single step of avirtual iterative algorithm, respectively.
3.1. Architecture

f,n,jFig. 1 and Fig. 2 show the entire architecture of SepNet and theblock diagram for each building block. As shown in Fig. 1, theproposed architecture consists of K blocks with the same structure,so the entire network can be viewed as a deep architecture consistingof multiple update processes.
Rather than designing the entire architecture from scratch, we
consider it reasonable to design each block by taking inspirationfrom the update equation shown in Subsec. 2.1. As shown in (4),the update equation in the IP method
P uses a weighted spatial covariance matrix Σx/vj (f ) = (1/N ) n x(f, n)xH (f, n)/vj (f, n) to
update each row of WH (f ), whose weight is determined according
to the estimate vj (f, n) of the power spectral density of the corresponding source. We thus conﬁgure each block to consist of four
differentiable layers Lpre , Pθ , Lpost , and Qϕ playing the following roles: The ﬁrst layer, Lpre , produces source signal estimates bysimply applying the separation matrix estimates produced from theprevious block to the observed signals. The second layer, Pθ , es-

where =c denotes equality up to constant terms.
ILRMA [5–7] is a BSS method that incorporates the NMF
model into (3) by expressing vj (f, n) as the linear sum of M
basis spectra B = {bj,m (f )}j,m,f ≥ 0 scaled by time-varying
magnitudes H = {hj,m (n)}j,m,n ≥ 0. Namely, vj (f, n) =PM
m=1 bj,m (f )hj,m (n). The optimization algorithm of ILRMA
consists of iteratively updating W, B and H so that (3) is ensured
to be nondecreasing at each iteration. To update B and H, we
can employ the expectation-maximization (EM) algorithm or the
majorization-minimization (MM) algorithm [5–7]. To update W,
we can use the natural gradient method or IP. The update rule derived
on the basis of the IP method [4] is given as
wj (f ) ← (WH (f )Σx/vj (f ) )−1 ej ,qwj (f ) ← wj (f )/ wHj (f )Σx/vj (f )wj (f ),

(4)
(5)

192
Authorized licensed use limited to: NTT. Downloaded on November 07,2023 at 09:02:46 UTC from IEEE Xplore. Restrictions apply.

the network output is assigned, the PIT objective can be written as

timates the power spectrogram corresponding to each source signal
estimate obtained from Lpre . The third layer, Lpost , computes theweighted spatial covariance matrix using each power spectrogram
estimate obtained from Pθ . The fourth layer, Qϕ , ﬁnally updates
the separation matrix estimates using the weighted spatial covariance matrices computed through Lpost . Here, Lpre and Lpost denote layers with no learnable parameters, whereas Pθ and Qϕ denote nonlinear layers parameterized by learnable parameters θ and(k−1)
ϕ, respectively. By using W (k−1) = {wj(f )}j,f to denote theE(θ(k) ) =hi
X X (k) H
EX ,S min|wj (f )x(f, n) − sπ(j) (f, n)| .
π

(k)

(k)

V (k) = {vj (f, n)}j,f,n , and U (k) = {Uj (f )}j,f to denote theoutputs of the intermediate layers in the kth block, the ﬁnal output(k)
W (k) = {wj (f )}j,f of the kth block is given asŶ (k) = Lpre (W (k−1) , X ),

(6)

V

(k)

U

(k)

(13)

f,n

Namely, we evaluate the separation errors for J! possible outputtarget assignments, from which we ﬁnd the best assignment π thatgives the minimum error for each training example. Our goal is tominimize the mean of the minimum errors computed in this way over
all the training examples. Note that only in the I = J = 2 case did
our method work well even without PIT, by simply sorting the source
indices in the target in ascending order of the direction-of-arrivals.

output of the (k − 1)th block and using Ŷ (k) = {ŷj (f, n)}j,f,n ,(k)j

4. EXPERIMENTS= Pθ(k) (Ŷ

(k)

),

(7)

4.1. Experimental settings= Lpost (V

(k)

),

(8)

W (k) = W (k−1) + Qϕ(k) (W (k−1) , U (k) ),(9)

We conducted a multichannel speech separation experiment to evaluate our method. ILRMA was chosen as a baseline method for comparison. For the experiment, we created two types of samples ofmultichannel reverberant mixtures, one being two-channel recordings of two speakers and the other being three-channel recordingsof three speakers. To simulate an open-set scenario, we used different speech databases consisting of recordings of completely different speakers to create the training and test sets. Speciﬁcally, weused utterances of 18 speakers excerpted from the CMU ARCTIC
database [23] for training, and those of four speakers excerpted fromthe Voice Conversion Challenge (VCC) 2018 dataset [24] for evaluation. The training and test sets consist of 3,000 and 100 samples,
respectively. The mixture signals were created from simulated twochannel recordings of two sources and three-channel recordings ofthree sources, where the room impulse responses were generated byusing the image method [25]. The depth, width, and height of theroom were set at 4.0, 5.0, and 3.0 m. We created each sample of the
training and test sets as follows. First, the reverberation time (T60)
was randomly set within the range of 55 to 160 ms. Second, eachmicrophone was randomly placed at least 0.5 m away from the wallsof the room. Third, once the positions of the microphone were determined, each speaker was randomly placed at a distance of 0.5 m
to 1.0 m from all the microphones so that the angle between eachspeaker pair with respect to the center of the microphones was atleast 20◦ . All the signals were sampled at 8 kHz. The network architectures for Pθ and Qϕ are detailed in Fig. 3. We conﬁgured Pθ
to consist of four sub-layers, each of which was designed using a
2D convolution layer (Conv) or a 2D transposed convolution layer(Deconv) with a gated linear unit (GLU) [26]. We conﬁgured Qϕ toconsist of ﬁve sub-layers, each of which was designed using a 3D
convolution layer (Conv) or a 3D transposed convolution layer (Deconv) with a GLU. In Qϕ , a complex-valued matrix is treated as athree-way array for convenience of implementation. Namely, wheninterpreted as an image, the row and column correspond to the height
and width, and the real and imaginary parts correspond to ﬁrst andsecond channels, respectively. Since the network is fully convolutional, X is allowed to have an arbitrary length. During training, theutterance was divided into 4.16-s-long segments (128 frames). The
tied and untied models were designed to have ten and four blocks,
respectively. The STFT was computed using a Hamming window
with a length of 64 ms and an overlap of 32 ms. For ILRMA, thebasis number M was set to 2, and the algorithm was run for 50 iterations. The initial value of the separation matrix was set at an identity
matrix for each frequency bin. All the algorithms were implemented

(k)

(k)

where each element ŷj (f, n) of Ŷ and each element Uj (f ) of
U (k) are given respectively as(k)(k−1) Hŷj (f, n) = |wj(f )x(f, n)|2 ,(10)

1 X x(f, n)xH (f, n)(k)Uj (f ) =.
(k)N n
vj (f, n)(11)(7) can be seen as a process of reﬁning the power spectrograms ofthe tentatively separated signals. Note that (9) is designed to havea residual connection, so Qϕ(k) is responsible for predicting the direction in which W (k−1) should be updated. As detailed later, we
design both Pθ and Qϕ using fully convolutional architectures.
(7) and (9) assume the use of different parameter sets θ(k) andϕ(k) for each block. Alternatively, we can think of tying the parameters in all the blocks together (by removing the superscript k fromθ(k) and ϕ(k) ) to reduce the parameters to be learned. To distinguishbetween these two versions, we call the former the untied model andthe latter the tied model. A comparison of these versions will bemade later.
3.2. Training process
The training process is the key to the success of SepNet. Givena training example of the pair {X , S}, one possible option for the
training objective would be to use a direct separation error, namely(k)the distance between wj H (f )x(f, n) and sj (f, n):
hXX
i
(k)
E(θ(k) ) = EX ,S|wj H (f )x(f, n) − sj (f, n)| , (12)jf,nwhere EX ,S [·] denotes the sample mean over all the training examples. However, model training using this objective comes withthe permutation problem mentioned earlier. That is, the order ofthe sources in the target S may not be the same as the order ofthe sources in the network output W. To solve this problem, we
adopt a PIT strategy [20], which was originally introduced to achieve
speaker-independent monaural speech separation. The idea is to ﬁrst
ﬁnd the best output-target assignment and then minimize the separation error given that assignment. By using π(j) ∈ {1, . . . , J} todenote the source index in the target to which the source index j in

193
Authorized licensed use limited to: NTT. Downloaded on November 07,2023 at 09:02:46 UTC from IEEE Xplore. Restrictions apply.

Output

Conv

Conv
BN
GLU

Deconv
BN
GLU

Conv
BN
GLU

Input

OutputDeconv

Deconv
BN
GLU

Conv
BN
GLU

Conv
BN
GLU

Input

Conv
BN
GLU

Layer

Layer

Mean SDR [dB]
10 1

100

10 1

100

Mean SIR [dB]

25
20
15
10
5
0
10 2
25
20
15
10
5
0
10 2
35
30
25
20
15
10 2
10

ILRMA (CPU)
Tied (GPU)
Tied (CPU)
Untied (GPU)
Untied (CPU)Mean SAR [dB]

Mean SAR [dB]

Mean SIR [dB]

Mean SDR [dB]Fig. 3: Detailed network architectures for layer Pθ and Qϕ . “h”, “w”, “c” and “k” denote the height, width, channel number, and kernel size,
respectively. “J” denotes the channel number of input mixture signals. “Conv”, “Deconv”, “BN”, and “GLU” denote 2D or 3D convolution,
2D or 3D deconvolution, batch normalization, and gated linear unit, respectively.

10 1Computational time [sec]100(a) Result for two-speaker separation.

20
15
10
5
0
10 2
20
15
10
5
0
10 2
30
25
20
15
10 2
10

ILRMA (CPU)
Tied (GPU)
Tied (CPU)
Untied (GPU)
Untied (CPU)

10 1

100

10 1

100

10 1
Computational time [sec]100(b) Result for three-speaker separation.

Fig. 4: Average scores of SDR, SIR, and SAR per computational time for ILRMA and both of the proposed methods. The tied model anduntied model run ten and four iterations respectively in a CPU or GPU. ILRMA runs 50 iterations in a CPU.

in PyTorch and run on Intel (R) Core i7-7800X CPU@3.50 GHz and
GeForce TITAN V GPU.

ﬂexible the network architecture of SepNet is, the better it performs.
5. CONCLUSION

4.2. Results and Discussion
In this paper, we proposed SepNet, a DNN-based separation matrix
predictor for determined frequency-domain BSS. The idea was to let
a DNN learn how to ﬁnd separation matrices from observed signals.
For model training, we adopted the PIT strategy to address the permutation mismatch between the order of the sources in the network
output and the order of the sources in the regression target. The experimental results revealed that SepNet was able to ﬁnd separation
matrices faster and with better separation accuracy than ILRMA for
mixtures of two and three sources.
Although SepNet is advantageous over ILRMA or the IP method
in terms of the computational efﬁciency of the inference process, thetraining process becomes more challenging as the source/channel
number increases. This is because the time complexity of PIT is factorial O(J!). Hence, developing methods for efﬁcient model training is important.

We computed the average of the signal-to-distortion ratios (SDRs),
signal-to-interference ratios (SIRs), and signal-to-artifact ratios(SARs) [27] over the 100 test samples to evaluate the source separation performance and measured the computational time per iteration.
Fig. 4 shows graphs of the speech separation accuracy versus run
time obtained with the separation matrices estimated at each iteration in ILRMA and at each block in SepNet. The results showthat both the tied and untied models could produce good separation
matrices even from the ﬁrst block. The separation matrices produced
from the ﬁnal block in the untied model yielded better separation
accuracy than the best accuracy achieved by ILRMA. Moreover, thetime it took to reach the ﬁnal block in the untied model was much
shorter than the time ILRMA needed to obtain the best accuracy.
Although the tied model performed slightly better than the untied
model in terms of the SAR, the untied model performed signiﬁcantly
better in terms of the SDR and SIR. This may indicate that the more

194
Authorized licensed use limited to: NTT. Downloaded on November 07,2023 at 09:02:46 UTC from IEEE Xplore. Restrictions apply.

[16] A. Ozerov and C. Févotte, “Multichannel nonnegative matrix
factorization inconvolutive mixtures for audio source separation,” IEEE Trans. ASLP, vol. 18, no. 3, pp. 550–563, 2010.

6. REFERENCES[1] A. Hyvärinen and E. Oja, “Independent component analysis:
algorithms and applications,” Neural networks, vol. 13, no.
4–5, pp. 411–430, 2000.

[17] H. Sawada, H. Kameoka, S. Araki, and N. Ueda, “Multichannel extensions of non-negative matrix factorization withcomplex-valued data,” IEEE Trans. ASLP, vol. 21, no. 5, pp.
971–982, 2013.

[2] T. Kim, T. Eltoft, and T.-W. Lee, “Independent vector analysis:An extension of ICA to multivariate components,” in Proc.
ICA, 2006, pp. 165–172.

[18] R. Scheibler and N. Ono, “Fast and stable blind source separation with rank-1 updates,” in In Proc. ICASSP, 2020, pp.
236–240.

[3] A. Hiroe, “Solution of permutation problem in frequency domain ICA, using multivariate probability density functions,” inProc. ICA, 2006, pp. 601–608.

[19] J. R. Hershey, J. L. Roux, and F. Weninger, “Deep unfolding:Model-based inspiration of novel deep architectures,” arXiv
preprint arXiv:1409.2574, 2014.

[4] N. Ono, “Stable and fast update rules for independent vector
analysis based on auxiliary function technique,” in Proc. WASPAA, 2011, pp. 189–192.

[20] Z.-H. Tan D. Yu, M. Kolbæk and J. Jensen, “Permutation invariant training of deep models for speaker-independent multitalker speech separation,” in Proc. ICASSP, 2017, pp. 241–245.

[5] H. Kameoka, T. Yoshioka, M. Hamamura, J. Le Roux, and
K. Kashino, “Statistical model of speech signals based on composite autoregressive system with application to blind source
separation,” in Proc. LVA/ICA, 2010, pp. 245–253.

[21] C. Fevotte and J. Cardoso, “Maximum likelihood approach forblind audio source separation using time-frequency gaussian
source models,” in In Proc. WASPAA, 2005, pp. 78–81.

[6] D. Kitamura, N. Ono, H. Sawada, H. Kameoka, and
H. Saruwatari, “Determined blind source separation unifying
independent vector analysis and nonnegative matrix factorization,” IEEE/ACM Trans. ASLP, vol. 24, no. 9, pp. 1626–1641,
2016.

[22] E. Vincent, S. Arberet, and R. Gribonval, “Underdetermined
instantaneous audio source separation via local Gaussian modeling,” in Proc. ICA, 2009, pp. 775–782.
[23] J. Kominek and A. W. Black, “The CMU Arctic speech
databases,” in Proc. SSW, 2004, pp. 223–224.

[7] D. Kitamura, N. Ono, H. Sawada, H. Kameoka, and
H. Saruwatari, “Determined blind source separation with independent low-rank matrix analysis,” in Audio Source Separation, S. Makino, Ed. Mar. 2018, pp. 125–155, Springer.

[24] J. Lorenzo-Trueba, J. Yamagishi, T. Toda, D. Saito, F. Villavicencio, T. Kinnunen, and Z. Ling, “The voice conversion challenge 2018: Promoting development of parallel and nonparallel methods,” arXiv preprint arXiv:1804.04262, Apr. 2018.

[8] A. A. Nugraha, A. Liutkus, and E. Vincent, “Multichannel audio source separation with deep neural networks,” IEEE/ACM
Trans. ASLP, vol. 24, no. 9, pp. 1652–1664, 2016.

[25] J. B. Allen and D. A. Berkley, “Image method for efﬁciently
simulating small-room acoustics,” The Journal of the Acoustical Society of America, vol. 65, no. 4, pp. 943–950, 1979.

[9] S. Mogami, H. Sumino, D. Kitamura, N. Takamune,
S. Takamichi, H. Saruwatari, , and N. Ono, “Independent
deeply learned matrix analysis for multichannel audio source
separation,” in Proc. EUSIPCO, 2018.

[26] Y. N. Dauphin, A. Fan, M. Auli, and D. Grangier, “Language
modeling with gated convolutional networks,” arXiv preprint
arXiv:1612.08083, 2016.

[10] H. Kameoka, L. Li, S. Inoue, and S. Makino, “Semi-blindsource separation with multichannel variational autoencoder,”arXiv preprint arXiv:1808.00892, Aug. 2018.

[27] E. Vincent, R. Gribonval, and C. Févotte, “Performance measurement in blind audio source separation,” IEEE Trans. ASLP,
vol. 14, no. 4, pp. 1462–1469, 2006.

[11] K. Sekiguchi, Y. Bando, K. Yoshii, and T. Kawahara,“Bayesian multichannel speech enhancement with a deep
speech prior,” in Proc. APSIPA ASC, 2018, p. 1233–1239.
[12] S. Leglaive, L. Girin, and R. Horaud, “Semi-supervised multichannel speech enhancement with variational autoencoders
and non-negative matrix factorization,” in Proc. ICASSP, 2019,
pp. 101–105.
[13] H. Kameoka, L. Li, S. Inoue, and S. Makino, “Supervised determined source separation with multichannel variational autoencoder,” Neural Computation, vol. 31, no. 9, pp. 1891–
1914, 2019.
[14] L. Li, H. Kameoka, and S. Makino, “Fast MVAE: Joint separation and classiﬁcation of mixed sources based on multichannel variational autoencoder with auxiliary classiﬁer,” in Proc.
ICASSP, 2019, pp. 546–550.
[15] S. Inoue, H. Kameoka, L. Li, S. Seki, and S. Makino, “Joint
separation and dereverberation of reverberant mixtures withmultichannel variational autoencoder,” in Proc. ICASSP, 2019,
pp. 96–100.

195
Authorized licensed use limited to: NTT. Downloaded on November 07,2023 at 09:02:46 UTC from IEEE Xplore. Restrictions apply.