Adaptive Processing and Learning for
Audio Source Separation
Jen-Tzung Chien∗ , Hiroshi Sawada† and Shoji Makino‡∗ Department of Electrical and Computer Engineering, National Chiao Tung University, Hsinchu, Taiwan† NTT Service Evolution Laboratories, NTT Corporation, Yokosuka-shi, Kanagawa, Japan‡ Graduate School of Systems and Information Engineering, University of Tsukuba, Ibaraki, Japanjtchien@nctu.edu.tw, sawada.hiroshi@lab.ntt.co.jp, maki@tara.tsukuba.ac.jp
Abstract— This paper overviews a series of recent advances in
adaptive processing and learning for audio source separation. In
real world, speech and audio signal mixtures are observed in reverberant environments. Sources are usually more than mixtures.
The mixing condition is occasionally changed due to the moving
sources or when the sources are changed or abruptly present orabsent. In this survey article, we investigate different issues in audio source separation including overdetermined/underdetermined
problems, permutation alignment, convolutive mixtures, contrast
functions, nonstationary conditions and system robustness. We
provide a systematic and comprehensive view for these issues
and address new approaches to overdetermined/underdetermined
convolutive separation, sparse learning, nonnegative matrix factorization, information-theoretic learning, online learning andBayesian approaches.

I. I NTRODUCTION
We are surrounded by sounds and noises in presence ofroom reverberation. The observed mixtures are usually lessthan source signals. The mixing condition is prone to be variedby the moving sources or in case of source replacement. It
becomes challenging to estimate the desired audio and speech
signals and develop a comfortable acoustic communication
channel between humans and machines. Audio source separation in realistic conditions has been a fascinating avenue forresearch which is crucial for broad extensions and applications
ranging from speech enhancement, speech recognition, music
retrieval, sound classiﬁcation, human-machine communicationand many others. How to extract and separate a target audio
or speech signal from noisy and nonstationary observations
is now impacting the communities of signal processing andmachine learning.
The traditional blind source separation (BSS) approaches
based on independent component analysis (ICA) were designed to resolve the instantaneous mixtures by optimizing
a contrast function or an independence measure. In previous
BSS methods, the frequency characteristics and location ofeach sources and how these sources were mixed were not sophisticatedly investigated. Solving the instantaneous mixtures
did not truly reﬂect the real reverberant environment which
structurally mixed the sources as the convolutive mixtures[25][31]. The underdetermined problem in presence of more
sources than sensors may not have been carefully treated [30].
The contrast functions may not ﬂexibly and honestly measure
the independence for an optimization with convergence [7].

The static mixing system could not catch the underlying
dynamics in source signals and sensor networks. The uncertainty of system parameters may not be precisely characterized
so that the robustness against adverse conditions was notguaranteed [5][9][15].
Generally, signal processing and machine learning provide
fundamental knowledge and algorithm to resolve different
issues in audio source separation. In the past years, there
have been a remarkable progress on development of cuttingedge adaptive processing and learning algorithms for source
separation and its applications. Machine learning has been oneof the most rapidly growing areas in international conferences
and journals. The goal of this paper is to overview a seriesof recent advances in adaptive processing and learning algorithms for BSS in presence of speech and music signals.
In this paper, we address general issues in an audio source
separation system including permutation problem, overdetermined/underdetermined convolutive mixtures, optimization ofcontrast function, nonstationary mixing condition and model
regularization. We provide a comprehensive and uniﬁed viewfor these issues and present a systematic survey over the
recent important solutions to overdetermined/underdetermined
convolutive separation [14][24][26], sparse source separation [3], reverberant source separation [31], nonnegative matrix factorization (NMF) [8][12][27], information-theoretic
learning [6][9], online learning [9] and Bayesian inference[4][18][19][28]. We address how these algorithms are connected and why they work for source separation particularly
in speech and music applications.
The remaining of this paper is organized into three sections.
Section 2 addresses key issues in BSS system and discusses
some challenging issues in real-world audio source separation.
The competing solutions to these components and issues are
generally categorized into two parts: the front-end processing
and the back-end learning, which are detailed in Second 3
and Section 4, respectively. Section 5 shall summarize thisoverview study and point out future directions on adaptive
processing and learning for audio source separation and applications.
II. C HALLENGES IN AUDIO S OURCE S EPARATION
Audio source separation is known as a challenging research topic and has had a big progress in recent years.

Adaptive Audio
Sourse Separation

Front-End
Processing

OverDetermined
Mixtures

UnderDetermined
Mixtures

Permutation
Alignment

Back-End
Learning
Information
TheoreticContrast
Function

NonStationary
Mixtures

Nonnegative
Matrix
Factorization

Bayesian
Separation

Fig. 1. Some issues in adaptive audio source separation.

In particular, the challenges in speech separation and recognition have been impacting the development of practical
automatic speech recognition systems and attracting the attention of many researchers in different communities, e.g.
machine learning, audio signal processing and spoken language processing as illustrated by the recent CHiME challenge(http://spandh.dcs.shef.ac.uk/chime challenge/).
In this study, we review some advances in adaptive audio
source separation which are generally classiﬁed into frontend processing and back-end learning as shown in Figure
1. A number of adaptive processing and learning algorithms
will be introduced to deal with different issues in audio
source separation. In front-end processing, we highlight on theadaptive signal processing to analyse the information on eachsource, such as its frequency characteristics and location, or
identifying how the sources are mixed. The input signals areprocessed through several processing components to obtainthe separated signals. In this section, we review a numberof high-impacting works on frequency-domain audio source
separation which could align the permutation ambiguities [26],separate the convolutive mixtures [28], identify the numberof sources [3], resolve the overdetermined/underdetermined
problem [2][30] and compensate for the room reverberation[31].
The back-end learning is devoted to recover the source
signals by using only the information about their mixtures
observed in each microphone without possessing frequency
and location information on each source. We build a statisticalmodel for the whole system and infer the model by using
the mixtures. Machine learning algorithms are developed foraudio source separation. The model-based speech separation
and recognition could be established [20]. In this section,
we present the estimation of demixing parameters throughconstruction and optimization of information-theoretic contrast
function [6][7]. The solutions to music source separation basedon NMF [12][27] and sparse learning [8] are introduced. Next,
we focus on the uncertainty modeling for the regularized signal
separation in accordance with Bayesian perspective [18]. The
nonstationary and temporally-correlated audio source separation [9] is presented.

III. F RONT-E ND P ROCESSING
Considering the issue of unknown number of sources, aGaussian mixture model with Dirichlet prior for mixture
weight parameter was proposed to identify the direction-ofarrival (DOA) of source speech signal from individual timefrequency units. This model was applied to estimate thenumber of sources and deal with the sparse source separation[3][4]. Moreover, it is popular to apply linear ﬁltering andspectrum enhancement methods for reverberant speech processing which is applied for speech recognition in reverberant
environment [31]. In this section, we mainly focus on thesolutions to frequency-domain blind source separation which
are applied to resolve the overdetermined/underdetermined
problems and the permutation alignment.
A. Overdetermined System
When the number N of microphones is enough for thenumber M of sources (in a determined M = N or overdetermined M < N case), we employ the complex-valued
ICA to separate the frequency bin-wise mixtures. Let xf t =[xf t1 , . . . , xf tN ]T ∈ CN be N dimensional complex vector
that represents N microphones’ observations at frequency
f and time t. ICA obtains the separated signals ŝf t =[ŝf t1 , . . . , ŝf tM ]T ∈ CM by a linear transformationŝf t = Wf xf t , t = 1, . . . , T(1)with a frequency-dependent separation matrix Wf of size
M ×N . The matrix Wf is optimized so that the distribution of
the vector elements ŝf tm , t = 1, . . . , T is far from a Gaussian
distribution. Various optimization learning rules have been
proposed, for example FastICA [17] or one based on natural
gradient [1][10]. In the learning rule, a non-linear function is
utilized to evaluate how similar or different the distribution ofeach element ŝf tm is compared with the Gaussian distribution.
For a complex-valued variable obtained as a result of shorttime Fourier transform (STFT), a polar coordinate based nonlinear function [21] is effective.
There is scaling ambiguity in an ICA solution. For an audio
source separation task, the scaling ambiguity is resolved bytrying to represent the observed signals at microphones withscaled separated signals. For this purpose, we calculate theor the Mooreinverse matrix Af = [af 1 , . . . , af M ] = W−1
f
Penrose pseudo inverse matrix Af = [af 1 , . . . , af M ] = W+fof the separated matrix. By multiplying Af on both sides of(1), we have
Af ŝf t =M

af m ŝf tm = xf t .

(2)

m=1
(m)

Then we obtain a vector s̆f t
resolved) separated signals asof the scaled (ambiguity-(m)s̆f t = af m ŝf tm .

(3)

B. Underdetermined System
When the number N of microphones is insufﬁcient for thenumber M of sources (in an underdetermined M > N case),
we typically employ the method based on time-frequency
masking, where we need to estimate which source has thelargest amplitude for each time frequency slot (f, t). For that
purpose, we apply a clustering method to observation vectors
xf t and to calculate the posterior probability p(Cm |xf t ) thata vector xf t belongs to a cluster Cm . Then, time frequency
masks are made by
1 p(Cm |xf t ) ≥ p(Cm |xf t ), ∀ m = m
Mf tm =
(4)0 otherwise,
and the separated signals are obtained by(m)ŝf t = Mf tm xf t .

(5)

In terms of clustering methods, one based on an anechoic
propagation model [2][24] is easy and simple, and works wellunder low reverberant conditions. However, to cope with more
complicated real-room sound propagation, frequency bin-wiseclustering has been proposed [26]. Speciﬁcally, a Gaussian
mixture model (GMM)p(xf t |Θ) =Mπf m p(xf t |af m , σf2 m )

(6)

m=1

with a complex Gaussian density function of the form [26]1p(x|af m , σf2 m ) =(2πσf2 m )N −1


(7)
2
||x − (aHf m x) · af m ||× exp −σf2 mis assumed for each frequency bin f , and we estimate aparameter set Θ = {πf 1 , af 1 , σf2 1 , . . . , πf M , af M , σf2 M } that
T
maximizes the likelihood p(Xf |Θ) = t=1 p(xf t |Θ). In (6),af m is the mean vector, σf2 m is the variance, and πf m is the
mixture ratio of the m-th cluster. In (7), (aHf m x) · af m denotes
the orthogonal projection of x onto the subspace spannedby af m . After the parameter set is estimated, the posterior
probabilities used in (4) is given by
πf m p(xf t |af m , σf2 m ).
p(Cm |xf t ) = M2
m=1 πf m p(xf t |af m , σf m )(8)

C. Permutation Alignment
The method based on ICA or GMM, described in previous
Subsections, performs a source separation task in a frequency
bin-wise manner. Therefore, we need to align the permutation
ambiguity of the ICA or GMM results in each frequency bin sothat a separated signal in the time domain contains frequency
components from the same source signal. This problem is wellknown as the permutation problem of frequency-domain BSS[22]. Although various approaches to the permutation problem
have been proposed [25], the following approach based on thedominance measures [23][26] performs very well.
When using ICA, we employ the power ratio(m)||s̆f t ||2(m)rf (t) = M(m) 2
m=1 ||s̆f t ||(9)of the scaled separated signals (3) as a dominance measure[23]. On the other hand, when using a GMM for timefrequency masking, we employ the posterior probability (8)(m)rf(t) = p(Cm |xf t )(10)

as a dominance measure [26]. After calculating the dominancemeasure, we basically interchange the indices m of the sep(m) (m)arated signals so that the correlation coefﬁcient ρ(rf , rf  )between the dominance measures at different frequency binsf and f  is maximized for the same source. The optimization
procedure is described in details in a reference [26].
IV. BACK -E ND L EARNING
In this section, we focus on the machine learning solutions
to audio source separation. We consider blind speech or music
separation as a learning problem without special treatment on
convolutive mixtures or extraction of frequency features andlocation information on each source signal. The informationtheoretic learning, online learning, dictionary learning, sparselearning and Bayesian learning are investigated.
A. Information-Theoretic Contrast Function
Information-theoretic learning is important to ﬁnd the
demixing solution to audio source separation. Let the observation vector xt = [xt1 , . . . , xtN ]T from N microphones at timeframe t be mixed by xt = Ast where A is an unknown N ×M
mixing matrix and st = [st1 , . . . , stM ]T denotes a vector of M
mutually-independent source signals. For the case of N = M ,BSS problem is resolved by ICA method which optimizes
a contrast function D(X, W) measuring the independence or
non-Gaussianity of the demixed signals ŝt based on a demixed
matrix or separation matrix W, i.e. ŝt = Wxt . The demixing
matrix can be estimated in accordance with the gradientdescent algorithm or the natural gradient algorithm [1] froma set of audio signals X = {x1 , . . . , xT }W(n+1) = W(n) − η∂D(X, W(n) )W(n)T W(n)
(11)
∂W(n)
where n is the iteration index and η is the learning rate. The
scaled natural gradient algorithm was proposed to improvelearning process by imposing a posteriori scalar gradient constraint [13]. The metrics of likelihood function, negentropy andkurtosis are popular to serve as ICA contrast functions. Moremeaningfully, the information-theoretical contrast function isadopted to measure the independence between the demixed
signals.
The statistical hypothesis test was recently proposed tocarry out an information measure of conﬁdence towards independence by investigating the null hypothesis H0 where
the demixed signals Ŝ = {ŝ1 , . . . , ŝT } are independent against
the alternative hypothesis H1 where the demixed signals are
dependent [6]. The contrast function was formed as a log
likelihood ratio given by
DNLR (X, W) = log p(Ŝ|H0 ) − log p(Ŝ|H1 ).

(12)

However, the parametric Gaussian distribution is not allowed
to represent demixed signals in p(Ŝ|H) by using ICA method.
The nonparametric distribution based on Parzen window
density function was applied to develop the nonparametric
likelihood ratio (NLR) contrast function for speech separation.
This method was extended to unsupervised learning of acoustic
hidden Markov models for speech recognition. More generally,
the measure of independence is calculated as a divergence
between the joint distribution of the demixed signals andthe product of marginal distributions of individual demixed
signals. This divergence measure equals to zero in case thatthe condition of independence is met. A general convex
divergence measure was derived
a general 1−αby substituting
41+α(1+α)/2
into
+
t
−
t
convex function f (t) = 1−α222Jensen’s inequality to construct a contrast function for ICA
optimization. This convex divergence DC (X, W, α) is written
by [7]T2 
1 − α2 t=1
⎡

1
2
2



 (1+α)/2M

p(Wxt ) +− ⎣p(Wxt )(1+α)/2) + Mp(wm xt )m=1

p(wm xt )(1+α)/2 ⎤ 
⎦

m=1

(13)
where W = [wT1 , . . . , wTM ]T and α is an adjustable convexity parameter. By substituting the nonparametric distribution p(·) and adjusting the convexity parameter α, weestimated different nonparametric solutions to the demixing
matrices Ŵ for audio source separation. In cases of α = 1
and α = −1, the general convex divergence is realized
to the convex-Shannon divergence and the convex-logarithm
divergence where the convex functions based on Shannon’s
entropy and negative logarithm are adopted, respectively. The
convergence in optimizing convex divergence for ICA was
improved by choosing the convexity parameter with sharp
learning curve [7]. Information-theoretic learning is illustrated
to work for adaptive source separation.

B. Nonnegative Matrix Factorization
The information-theoretic learning is further extended to
dictionary learning based on the nonnegative matrix factorization (NMF) which is recently hot issue in audio source
separation [11]. NMF attempts to decompose the nonnegative
mixed samples X ∈ RN ×T into a product of nonnegative
mixing matrix A ∈ RN ×M and nonnegative source signals
S ∈ RM ×T by minimizing a divergence measure D(X, A, S)
between X and AS. NMF is a parts-based representation which
only allows additive combination and can be directly applied todecompose the nonnegative mixed audio signals. The absolutevalues of short-time Fourier transform (STFT) are calculated
to form X. The standard NMF is fulﬁlled according to aregularized least square (RLS) criterion DRLS (X, A, S) withsparsity constraint|Xnt − [AS]nt |2 + γaf (Anm ) + γsf (Smt ) (14)
n,t

n,m

m,t

where γa ≥ 0 and γs ≥ 0 denote the regularization parameters.
The sparseness measure f (Anm ) = |Anm |, f (Anm ) = Anmor f (Anm ) = Anm log(Anm ) could be imposed in (14).
More recently, the α divergence [12], convex divergence [7]and Itakura-Saito (IS) divergence [27] were treated as objectivefunction to derive solution to NMF.
example, IS divergence For XXnt
nt
is written by DIS (X, A, S) =(n,t [AS]nt − log [AS]nt − 1)
Xnt
. This property is
which depends only on the ratio [AS]ntfavorable for analyzing music and speech signals where low
frequency components have much higher energy than highfrequency components. In [16][27], minimizing IS divergencewas shown to be equivalent to maximizing the log-likelihood
log p(X̃|A, S) based on the multivariate complex-valued Gaussian distributions where X̃ denotes a matrix of STFT complexvalued coefﬁcients. The multiplicative updating rule for NMF
was obtained by




Xnt Smt
Xnt Anm t ([AS] n ([AS]
2
2
nt )
nt )

, Smt ← Smt  .
Anm ← AnmSmtt [AS]nt

Anmn [AS]nt(15)
However, this solution is only designed for the case of an
M -dimensional observation frame xt at a single frequency
bin f . The multichannel time-frequency NMF was developed for music source separation where an M -dimensional
mixed signals xf t was extended to different frequency bins{(f, t), 1 ≤ f ≤ F }. A clustering procedure of NMF bases
was performed. The separate bases were used to recoverindividual source signals.
In [8], Bayesian NMF was proposed for monaural musicsource separation which decomposed a single-channel mixed
signal X into a rhythmic signal Xr and a harmonic signal
Xh . Let the nonnegative monaural matrix X ∈ RF ×T intime-frequency domain be chunked into L segments {X(l) }.
(l)
Each segment is represented by X(l) = X(l)=r + Xh(l) (l)(l)(l)
Ar S(l)
+A
S
where
{S
,
S
}
are
two
groupsofsegmentr
r
h h
h
(l)
dependent encoding coefﬁcients, Ah denotes the bases forharmonic source which are individual for different segments

l, and Ar denotes the bases for rhythmic source which
are shared across segments. Assuming the basis components are Gamma distributed and the encoding coefﬁcients
(l)
(l)
(l)
S(l) = {S(l)r , Sh } are Laplacian distributed by p([S ]|λ ) =1 (l)(l) (l)(l)exp{−λ [S ]} with hyperparameter λ , Bayesian2λgroup sparse learning for NMF was performed to resolve theunderdetermined problem in audio source separation througha Gibbs sampling procedure over different parameters andhyperparameters of Gamma distributions and Laplacian distributions. BSS was viewed as a probabilistic framework withlatent variables including source signals, mixing coefﬁcients,
modeling errors and their associated parameters.
C. Bayesian Learning for Nonstationary Source Separation
In real-world audio source separation, we face the challenges of changing sources or moving speakers, namely thesource signals may abruptly appear or disappear, the speakers
may be replaced by new ones or even moving from one
location to the other. The mixing conditions and source
signals are accordingly nonstationary and should be traced
to assure robustness in nonstationary source separation [9]. A
meaningful approach to deal with the robustness issue in audio
source separation is constructed from Bayesian perspective[5][18]. The Bayesian approaches to blind speech separation[9][18] are examined. Some prior information is introduced
for uncertainty modeling and knowledge integration. Model
regularization and system robustness could be improved. Let(l)X(l) = {xt } denote a set of mixed signals at segment l. The
signals are mixed by a linear combination of M unknown(l)source signals S(l) = {st } using a mixing matrix A(l) ,
(l)
(l)
(l)
i.e. considering a noisy ICA model xt = A(l) st + εt(l)(l)
where E = {εt } denotes the noise signals. We assume
that A(l) and S(l) are unchanged within a segment l but
varied across segments. To tackle the nonstationary source
separation, we attempt to incrementally characterize the variations of A(l) and S(l) from the observed segments X (l) ={X(1) , X(2) , . . . , X(l) }. Online learning is conducted to compensate for nonstationary conditions of mixing coefﬁcients andsource signals segment by segment. At each learning epochl, we ﬁrst accumulate to calculate the predictive distributionof current ICA parameters Θ(l) = {A(l) , S(l) , E(l) } based on(l)previous segment data X (l−1) . When new data X(l) = {xt }of segment l is enrolled, the posterior distribution is corrected by p(Θ(l) |X (l) ) ∝ p(X(l) |Θ(l) )p(Θ(l) |X (l−1) ) which
is proportional to the product of a likelihood function ofcurrent data segment X and a posteriori distribution given
the previous data segments X (l−1) . In this sense, the modesof posterior distributions are updated in an online manner
Θ(0) → Θ(1) → · · · → Θ(l) .
In particular, source signals are represented by a mixture of(l)Gaussian distributions p(st |Θ(l) ) given by
M
m=1

K

k=1

(l)

(l)

(l)

(l)

πmk N (sm,t |μmk , (γmk )−1 )(16)

with parameters Θ(l) consisting of mixture weights Π(l) =

(l)

(l)

{πmk }, means M(l) = {μmk } and precisions R(l) =(l)(l){γmk }. The noise vector εt is assumed to be Gaussian(l)N (εt |0, (B(l) )−1 ) with zero mean and diagonal precision(l)matrix B(l) = diag{βn }. The prior density of N × M(l)(l)
mixingmatrix A = {anm } is distributedby p(A(l) |α(l) ) =

M
N
(l)
(l) −1) . Importantly, the hypern=1
m=1 N (anm |0, (αm )(l)parameter αm is known as an automatic relevance determination (ARD) [29], which reveals the activity of a source signal(l)sm,t . The matrix A(l) is prone to be sparse with near zero(l)entries at the mth column of A(l) if the estimated ARD αmis large. The mth source is likely inactive at segment l. The
redundant sources are disregarded automatically.
To fulﬁll full Bayesian approach to audio source separation,
we consider the uncertainties of parameters and hyperparameters Θ(l) = {A(l) , S(l) , E(l) , Π(l) , M(l) , R(l) , B(l) , α(l) }which are latent variables and compensate these uncertainties by using the solution obtained by optimizing themarginal likelihood over all latent variables. Conjugate priorsp(Θ(l) |Φ(l−1) ) are introduced to characterize the uncertaintiesof {Π(l) , M(l) , R(l) , B(l) , α(l) }. The hyperparameters Φ(l−1)
are continuously updated from history data X (l−1) . With theseconjugate priors, the integral in marginal likelihood has closedform solution and is calculated efﬁciently. This nonstationary
Bayesian method was formulated based on the variational
Bayesian expectation maximization (VB-EM) algorithm [9]where the variational inference was solved by maximizing thelower bound of marginal likelihood.
More recently, we present the solution to nonstationary
and temporally correlated source separation where the mixing
condition is changed continuously and the temporal correlationin time-series signals, e.g. mixing coefﬁcients and source
signals, is taken into account. Online learning and Gaussian
process (GP) [19] are merged into a separation system which
compensates for the nonstationary and temporally correlated
mixing environments and source signals, respectively. Consid(l)ering a noisy ICA model with time-varying mixing matrix At
at segment l and time bin t, the temporally correlated mixing
coefﬁcients are generated by the distributions of nonparametriclatent functions. GP ﬂexibly explores the unknown temporal(l)structure of anm,t . A latent function f (·) is employed to(l)connect the relation between current coefﬁcient anm,t and its
(l)
(l)
(l)
past p coefﬁcients anm,t−1 = [anm,t−1 , . . . , anm,t−p ]T by
(l)

(l)

(l)

anm,t = f (anm,t−1 ) + εnm,t

(17)

(l)

where
εnm,t
denotesthewhitenoise.
This
function is generated from a zero-mean Gaussian
(l)
(l)
with
a  varianceκ(anm,t−1 , anm,τ −1
)
given
by
2 
(l−1) 
λa


(l)
(l)
(l−1)
which
ξanm exp − nm
anm,t−1 − anm,τ −1 2is an exponential-quadratic kernel function with(l−1) (l−1)
The
GP
prior
hyperparameters{λanm , ξanm }.
(l)
(l−1)
p(anm |μanm , R(l−1)
forthemixing
coefﬁcients
anm )
(l)
(l)
(l)
(l)
anm = [anm,1 , . . . , anm,L ]T written by N (anm |0, K(l−1)
anm )

(l)

(l)

where [K(l−1)
anm ]tτ = κ(anm,t−1 , anm,τ −1 ) + δtτ . With GPprior and its hyperparameters, VB-EM algorithm is again
applied for model inference of the proposed online GP. The
variational and sequential Bayesian inference is presented to
implement a dynamic audio source separation system [9].
V. C ONCLUSIONS
We have presented a series of adaptive methods whichwere developed for different issues in audio source separation.
These methods were systematically categories into front-end
processing and back-end learning. In front-end processing,
we addressed high-performance solutions to overdetermined
and underdetermined problems which are based on the processing of complex-valued time-frequency signals and thenoise-masking method using Gaussian mixture model. The
permutation problem was solved according to the correlation
coefﬁcient between dominance measures at different frequency
bins. In back-end learning, we addressed the importance of
information-theoretical learning for ICA optimization. The
recent methods of sparse learning and dictionary learning
based on nonnegative matrix factorization were presented
for speech/music source separation. The online learning andBayesian learning designed for nonstationary source separation were also presented for improving the robustness foraudio source separation. In the future, the directions of singlechannel and multi-channel dereverberation are still impacting
the communities of signal processing and machine learning.
The application of audio source separation in robust speech
recognition is continuing to be attractive for future studies.
R EFERENCES[1] S. Amari, “Natural gradient works efﬁciently in learning,” NeuralComputation, vol. 10, pp. 251-276, 1998.
[2] S. Araki, H. Sawada, R. Mukai, and S. Makino, “Underdetermined blindsparse source separation for arbitrarily arranged multiple sensors,” Signal
Processing, vol. 87, pp. 1833–1847, 2007.
[3] S. Araki, T. Nakatani, H. Sawada, and S. Makino, “Blind sparse source
separation for unknown number of sources using Gaussian mixture
model ﬁtting with Dirichlet prior,” in Proc. of International Conferenceon Acoustics, Speech, and Signal Processing (ICASSP), 2009, pp. 33–36.
[4] S. Araki, T. Nakatani, H. Sawada, and S. Makino, “Stereo source
separation and source counting with MAP estimation with Dirichletprior considering spatial aliasing problem,” in Proc. of Independent
Component Analysis and Blind Signal Separation, 2009, pp. 742–750.
[5] C. M. Bishop, Pattern Recognition and Machine Learning. Springer,
2006.
[6] J.-T. Chien and B.-C. Chen, “A new independent component analysis
for speech recognition and separation,” IEEE Transactions on Audio,
Speech and Language Processing, vol. 14, no. 4, pp. 1245–1254, 2006.
[7] J.-T. Chien and H.-L. Hsieh, “Convex divergence ICA for blind source
separation,” IEEE Transactions on Audio, Speech and Language Processing, vol. 20, no. 1, pp. 290–301, 2012.
[8] J.-T. Chien and H.-L. Hsieh, “Bayesian group sparse learning for
nonnegative matrix factorization,” in Proc. of INTERSPEECH, 2012,
pp. 1552–1555.
[9] J.-T. Chien and H.-L. Hsieh, “Nonstationary source separation using
sequential and variational Bayesian learning,” IEEE Transactions on
Neural Networks and Learning Systems, vol. 24, no. 5, pp. 681–694,
2013.
[10] A. Cichocki and S. Amari, Adaptive Blind Signal and Image Processing,John Wiley & Sons, 2002.
[11] A. Cichocki, R. Zdunek, and S. Amari, “New algorithms for nonnegative matrix factorization in applications to blind source separation”,
in Proc. of International Conference on Acoustics, Speech, and Signal
Processing (ICASSP), 2006, pp. 621-624.

[12] A. Cichocki, H. Lee, Y.-D. Kim, and S. Choi, “Non-negative matrix
factorization with α-divergence,” Pattern Recognition Letters, vol. 29,
pp. 1433–1440, 2008.
[13] S. C. Douglas and M. Gupta, “Scaled natural gradient algorithms for
instantaneous and convolutive blind source separation”, in Proc. ofInternational Conference on Acoustics, Speech, and Signal Processing(ICASSP), 2007, pp. 637-640.
[14] S. Douglas, M. Gupta, H. Sawada, and S. Makino, “Spatio-temporal
FastICA algorithms for the blind separation of convolutive mixtures,”IEEE Transactions on Audio, Speech and Language Processing, vol. 15,
no. 5, pp. 1511–1520, 2007.
[15] C. Fevotte, “Bayesian audio source separation,” in Blind Speech Separation, S. Makino, T.-W. Lee, and H. Sawada, Eds. Springer, 2007,
pp. 305–335.
[16] C. Fevotte, N. Bertin, and J.-L. Durrieu, “Nonnegative matrix factorization with the Itakura-Saito divergence: with application to music
analysis,” Neural Computation, vol. 21, no. 3, pp. 793–830, 2009.
[17] A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis , John Wiley & Sons, 2001.
[18] A. Mohammad-Djafari and K. Knuth, “Bayesian approaches,” in Handbook of Blind Source Separation: Independent Component Analysis andApplications, P. Common and C. Jutten, Eds. Elsevier, 2010, pp. 467–
513.
[19] S. Park and S. Choi, “Gaussian processes for source separation,” in Proc.
of International Conference on Acoustics, Speech, and Signal Processing(ICASSP), 2008, pp. 1909–1912.
[20] S. J. Rennie, J. R. Hershey, and P. A. Olsen,“Single-channel multitalker
speech recognition,” IEEE Signal Processing Magazine, vol. 27, no. 6,
pp. 66–80, 2010.
[21] H. Sawada, R. Mukai, S. Araki, and S. Makino, “Polar coordinate based
nonlinear function for frequency domain blind source separation,” IEICE
Trans. Fundamentals, vol. E86-A, pp. 590–596, 2003.
[22] H. Sawada, R. Mukai, S. Araki, and S. Makino, “A robust and precise
method for solving the permutation problem of frequency-domain blindsource separation,” IEEE Transactions on Speech and Audio Processing,vol. 12, pp. 530–538, 2004.
[23] H. Sawada, S. Araki, and S. Makino, “Measuring dependence of binwise separated signals for permutation alignment in frequency-domain
BSS,” in Proc. ISCAS , pp. 3247–3250, 2007.
[24] H. Sawada, S. Araki, R. Mukai, and S. Makino, “Grouping separated
frequency components by estimating propagation model parametersin frequency-domain blind source separation,” IEEE Transactions on
Audio, Speech and Language Processing, vol. 15, no. 5, pp. 1592–1604,
2007.
[25] H. Sawada, S. Araki, and S. Makino, “Frequency-domain blind source
separation,” in Blind Speech Separation, S. Makino, T.-W. Lee, and
H. Sawada, Eds. Springer, 2007, pp. 47–78.
[26] H. Sawada, S. Araki, and S. Makino, “Underdetermined convolutive
blind source separation via frequency bin-wise clustering and permutation alignment,” IEEE Transactions on Audio, Speech and Language
Processing, vol. 19, no. 3, pp. 516–527, 2011.
[27] H. Sawada, H. Kameoka, S. Araki, and N. Ueda, “Multichannel extensions of non-negative matrix factorization with complex-valued data,”
IEEE Transactions on Audio, Speech and Language Processing, vol. 21,
no. 5, pp. 971–982, 2013.
[28] H. K. Solvang, Y. Nagahara, S. Araki, H. Sawada, and S. Makino,“Frequency-domain Pearson distribution approach for independent component analysis (FD-Pearson-ICA) in blind source separation,” IEEE
Transactions on Audio, Speech and Language Processing, vol. 17, no. 4,
pp. 639–649, 2009.
[29] M. E. Tipping, “Sparse Bayesian learning and the relevance vector
machine”, Journal of Machine Learning Research, vol. 1, pp. 211-244,
2001.
[30] S. Winter, W. Kellermann, H. Sawada, and S. Makino, “MAP-based
underdetermined blind source separation of convolutive mixtures byhierarchical clustering and 1 -norm minimization,” EURASIP Journal
on Advances in Signal Processing, vol. 2007, pp. 1–12, 2007.
[31] T. Yoshioka, A. Sehr, M. Delcroix, K. Kinoshita, R. Maas, T. Nakatani,and W. Kellermann, “Making machines understand us in reverberant
rooms - robustness against reverberation for automatic speech recognition,” IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 114–126,
2012.