NAIST-IS-MT1251035

Master’s Thesis

Music Signal Separation Combining Directional
Clustering and Nonnegative Matrix
Factorization with Spectrogram Restoration

Daichi Kitamura

March 6, 2014

Department of Information Science
Graduate School of Information Science
Nara Institute of Science and Technology

A Master’s Thesis
submitted to Graduate School of Information Science,
Nara Institute of Science and Technology
in partial fulﬁllment of the requirements for the degree of
MASTER of ENGINEERING
Daichi Kitamura
Thesis Committee:
Professor Satoshi Nakamura(Supervisor)
Professor Kazushi Ikeda(Co-supervisor)
Associate Professor Hiroshi Saruwatari (Co-supervisor)

Music Signal Separation Combining Directional
Clustering and Nonnegative Matrix
Factorization with Spectrogram Restoration ∗Daichi Kitamura

Abstract
In this thesis, to address a music signal separation problem, I propose a new
hybrid method that concatenates directional clustering and supervised nonnegative matrix factorization (NMF) with spectrogram restoration for the purposeof the speciﬁc sound extraction from the multichannel music signal that consistsof multiple instrumental sounds. Recently, a main format for obtaining musicaltunes has become electronic data such as music ﬁles, which can be made available
over the Internet owing to progress in information technology. Hence, users caneasily obtain and edit music tunes, resulting in the active creation of new contents. According to this background, music signal separation technologies havemuch attention. Music signal separation is aimed to extract a speciﬁc target signal from music signals that contain multiple music instrumental sounds. Audio
remixing by the users, automatic music transcription, and musical instrument
education are one of the feasible music signal separation applications.
In the previous studies, music signal separation based on NMF has been a very
active area of the research. Various methods using NMF have been proposed, butthey remain many problems, e.g., poor convergence in update rules in NMF andlack of robustness. To solve these problems, I propose a new supervised NMF(SNMF) with spectrogram restoration and its hybrid method that concatenates
the proposed SNMF after directional clustering. Via extrapolation of supervised
spectral bases, this SNMF with spectrogram restoration attempts both target∗

Master’s Thesis, Department of Information Science, Graduate School of Information
Science, Nara Institute of Science and Technology, NAIST-IS-MT1251035, March 6, 2014.

i

signal separation and reconstruction of the lost target components, which are
generated by preceding binary masking performed in directional clustering.
Next, I provide a theoretical analysis of basis extrapolation ability and reveal
the mechanism of marked shift of optimal divergence in SNMF with spectrogram
restoration and trade-oﬀ between separation and extrapolation abilities. Evaluation experiment of the separation using artiﬁcial and real-recorded music signals
show the eﬀectiveness of the proposed hybrid method.
Finally, based on the above-mentioned ﬁndings, I propose a new scheme forframe-wise divergence selection in the proposed hybrid method to separate thetarget signal using optimal multi-divergence. The results of an evaluation experiment show that the proposed hybrid method with multi-divergence can always
achieve high performance under any spatial conditions, indicating the improvement in robustness of the proposed method.

Keywords:
Music signal separation, Directional clustering, Spectrogram restoration, Nonnegative matrix factorization, Supervised method.

ii

スペクトログラム修復機能付き非負値行列因子分解と方位クラスタリングを組み合わせた音楽信号分離 ∗
北村 大地

内容梗概
本論文では，複数の音源が多重に混合されたマルチチャネル音楽信号から頑健に目的の信号成分を分離する手法の確立を目指し，方位クラスタリングとスペク
トログラム修復機能付き教師あり非負値行列因子分解 (NMF) を組み合わせた新
しい音楽信号分離手法を提案する．近年，音楽メディアは電子ファイルとして供給されインターネットを通じて配信される機会が増加している．そのため，ユーザが既存の音楽メディアを自由に編集する等の能動的な創作活動が盛んになっている．このような背景から，音楽信号を対象とした信号分離手法が広く注目を集めており世界中で盛んに研究されている．この音楽信号分離技術は，複数の楽器が多重に混合された音楽信号の中から特定の楽器音を分離・抽出することを目的としており，オーディオリミックス，自動採譜，楽器演奏法のための音楽教育といった様々な応用先が考えられる．
これまでの研究において，NMF を用いた音楽信号分離技術が非常に高い注目を集めており，様々な改良手法が提案されている．しかしながら，いずれの手法においても多くの問題があり，頑健かつ高精度に目的音を分離する手法は未だ提案されていないのが現状である．そこで本研究では，新たにスペクトログラム修
復機能付き教師あり NMF (SNMF) を提案し，方位クラスタリングと組み合わせ
ることによって頑健かつ高精度に目的音を分離するマルチチャネル信号分離手法を提案する．新たに提案するスペクトログラム修復機能付き SNMF は，信号の分離と同時に，前段処理の方位クラスタリングによって失われた目的音成分の復元を教師スペクトル基底の外挿によって実現する．すなわち，前段処理によって傷∗

奈良先端科学技術大学院大学 情報科学研究科 情報科学専攻 修士論文, NAIST-IS-MT1251035,
2014 年 3 月 6 日.

iii

ついた混合信号のスペクトログラムを，後段の SNMF の教師スペクトル基底の外挿によって復元することができる．
次に，教師スペクトル基底の外挿能力について，信号の生成モデルに基づく理論解析を行い，基底外挿における最適なダイバージェンス規範を示し，スペク
トログラム修復機能付き SNMF 特有の最適ダイバージェンスシフトのメカニズ
ムを明らかにする．加えて，ダイバージェンス規範に対する分離能力と外挿能力のトレードオフを理論的に示し，人工及び実録音音楽信号を用いた音楽信号分離
実験において解析結果と同様の現象が現れることを確認する．
さらに，明らかにされた外挿能力理論に基づいて異なるダイバージェンス規範をフレームごとに切り替える多重ダイバージェンス型ハイブリッド手法を提案する．このダイバージェンスのダイバーシチを実装した提案ハイブリッド手法はいかなる空間的配置の入力信号に対しても常に最高の分離性能を達成し，提案手法の頑健性をさらに向上させることができる．本手法の有効性は，音楽信号分離
実験によって確認される．

キーワード
音楽信号分離，非負値行列因子分解，方位クラスタリング，スペクトログラム修復，教師あり手法．

iv

Contents
1. Introduction
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Prior works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Scope of thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4 Outline of thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1
1
1
3
5

2. Conventional Signal Separation Methods
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Conventional single-channel signal separation methods . . . . . .
2.2.1 Overview of NMF . . . . . . . . . . . . . . . . . . . . . . .
2.2.2 SNMF and PSNMF . . . . . . . . . . . . . . . . . . . . . .
2.3 Conventional multichannel signal separation methods . . . . . . .
2.3.1 Directional clustering . . . . . . . . . . . . . . . . . . . . .
2.3.2 Hybrid method of directional clustering and PSNMF . . .
2.3.3 Multichannel NMF . . . . . . . . . . . . . . . . . . . . . .
2.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7
7
7
7
12
15
15
16
16
19

3. SNMF with Spectrogram Restoration and Its Hybrid Method 20
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.2 SNMF with spectrogram restoration . . . . . . . . . . . . . . . . 20
3.2.1 Motivation and strategy . . . . . . . . . . . . . . . . . . . 20
3.2.2 Cost function and update rules . . . . . . . . . . . . . . . 23
3.3 Theoretical analysis of basis extrapolation based on generation model 27
3.3.1 Optimal divergence for basis extrapolation and generation
model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.3.2 Simulation conditions . . . . . . . . . . . . . . . . . . . . . 29
3.3.3 Simulation results and discussion . . . . . . . . . . . . . . 29
3.4 Comparison between proposed hybrid method and conventional
methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.4.1 Experimental conditions . . . . . . . . . . . . . . . . . . . 32
3.4.2 Experimental results . . . . . . . . . . . . . . . . . . . . . 36
3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44v

4. Optimal Divergence Diversity for SNMF with spectrogram restoration
45
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.2 SNMF with spectrogram restoration based on multi-divergence . . 45
4.2.1 Divergence dependency on local chasms condition . . . . . 45
4.2.2 Cost function and update rules . . . . . . . . . . . . . . . 46
4.3 Evaluation experiment . . . . . . . . . . . . . . . . . . . . . . . . 50
4.3.1 Experimental conditions . . . . . . . . . . . . . . . . . . . 50
4.3.2 Experimental results . . . . . . . . . . . . . . . . . . . . . 51
4.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
5. Conclusion
5.1 Summary of thesis . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

55
55
56

Acknowledgements

57

References

59

List of Publications

64

vi

List of Figures
1
2
3
4
5
6
7
8
9
10
11
12
13
14

15
16
17
18

Applications of music signal separation. . . . . . . . . . . . . . . .
Relation between conventional methods and proposed hybrid methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Decomposition model of simple NMF. . . . . . . . . . . . . . . .
Variation of β-divergence function when β = 0. . . . . . . . . . .
Variation of β-divergence function when β = 1. . . . . . . . . . .
Variation of β-divergence function when β = 2. . . . . . . . . . .
Variation of β-divergence function when β = 3. . . . . . . . . . .
Variation of β-divergence function when β = 4. . . . . . . . . . .
Decomposition model in each process of SNMF. . . . . . . . . . .
Directional source distribution of (a) observed stereo signal, (b)separated target components in the center cluster. . . . . . . . . .
Signal ﬂow of conventional hybrid method; PSNMFs are cascaded
after stereo output of directional clustering. . . . . . . . . . . . .
Example of spectrum of signal separated by directional clustering.
Signal ﬂow of proposed hybrid method; SNMF with spectrogram
restoration concatenates after directional clustering. . . . . . . . .
Directional source distribution of (a) observed stereo signal, (b)separated components in center cluster, and (c) component separated and extrapolated by spectrogram restoration. . . . . . . . .
Extrapolation abilities for (a) 75%-binary-masked data and (b)98%-binary-masked data. . . . . . . . . . . . . . . . . . . . . . .
Trade-oﬀ between separation and extrapolation abilities. Overall
performance is highest when βNMF > 1. . . . . . . . . . . . . . . .
Scores of each part. . . . . . . . . . . . . . . . . . . . . . . . . .
Panning of four sources with sine law used in artiﬁcial signal caseexperiment. Numbered black circles represent locations of instruments in stereo format. For example, if target is Ob., No.1 is set
to Ob. and Nos.2, 3, and 4 are combinations of Fl., Tb., and Pf. .

vii

2
4
8
9
9
10
10
11
12
17
18
19
21

22
31
32
33

34

19

20

21

22

23

24

25

26
27
28

Geometry of the loudspeaker and binaural microphone (dummyhead). Numbered black circles represent locations of loudspeakers. Target source and supervision sound is always located in No.1
position. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Average scores with various divergences and regularizations in artiﬁcial signal case when θ = 15◦ : (a) shows SDR, (b) shows SIR,
and (c) shows SAR for proposed methods. . . . . . . . . . . . . .
Average scores with various divergences and regularizations in artiﬁcial signal case when θ = 45◦ : (a) shows SDR, (b) shows SIR,
and (c) shows SAR for conventional and proposed methods. . . .
Average scores with various divergences and regularizations in realrecorded signal case: (a) shows SDR, (b) shows SIR, and (c) shows
SAR for conventional and proposed methods. . . . . . . . . . . .
Average scores in artiﬁcial signal case when θ = 15◦ : (a) shows
SDR, (b) shows SIR, and (c) shows SAR for conventional andproposed methods. . . . . . . . . . . . . . . . . . . . . . . . . . .
Average scores in artiﬁcial signal case when θ = 45◦ : (a) shows
SDR, (b) shows SIR, and (c) shows SAR for conventional andproposed methods. . . . . . . . . . . . . . . . . . . . . . . . . . .
Average scores in real-recorded signal case: (a) shows SDR, (b)shows SIR, and (c) shows SAR for conventional and proposed
methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Divergence diversity algorithm of proposed method. . . . . . . . .
Scores of each part. The observed signal consists of four measures.
Average scores of each method and each spatial condition.: (a)shows SDR, (b) shows SIR, and (c) shows SAR for conventional
and proposed methods. . . . . . . . . . . . . . . . . . . . . . . . .

35

38

39

40

41

42

43
46
51

53

List of Tables
12

Compositions of musical instruments . . . . . . . . . . . . . . . .
Spatial conditions of each dataset . . . . . . . . . . . . . . . . . .

viii

33
51

1. Introduction
1.1 BackgroundIn recent years, the main format for obtaining musical tunes has become electronic data such as music ﬁles, which can be made available over the Internet
owing to progress in information technology. Hence, users can easily obtain andedit music tunes, resulting in the active creation of new contents. These consumers’ activities have rapidly increased in the past few years with the expansionof social networking services and video-sharing websites. However, it is still diﬃcult to edit a speciﬁc instrumental signal in general music tunes containing many
instruments because almost all the commercially available music data are mixed
down, and consumers cannot obtain each solo-played instrumental signal in advance. Such audio editing of each sound source will enable us to engage in newactivities based on the appreciation of musical tunes and can be applied to many
valuable techniques including audio remixing by users [1, 2], musical instrument
education, 3D audio reproduction [3], and automatic music transcription [4, 5](see Fig. 1). With this background, music signal separation technologies haveattracted considerable interest and been intensively studied [6, 7, 8, 9, 10] in recent years. However, it remains diﬃcult to freely extract a speciﬁc music signal,
particularly in the case of instruments that belong to the same family.
Signal separation can be classiﬁed into overdetermined and underdetermined
problems. In the former situation, the number of channels is greater than thenumber of sound sources, and many techniques have been studied and proposedfor overdetermined signal separation [11, 12, 13]. However, such separation techniques cannot be applied to the above-mentioned music signal separation problem
because almost all musical tunes are provided in a stereo format and the number of sources is greater than two. Therefore, techniques for underdetermined
separation are required and should be used to achieve music signal separation.

1.2 Prior works
As a means of addressing underdetermined signal separation, in recent years, nonnegative matrix factorization (NMF) [14], which is a type of sparse representation

1

Audio remixing

Musical instrument
education

Automatic musictranscription

3D audio reproduction

Figure 1. Applications of music signal separation.

algorithm, has received much attention. NMF for acoustical signals decomposes
an input spectrogram into the product of a spectral basis matrix and its activation
matrix. The methods of signal separation based on NMF are roughly classiﬁed
into unsupervised and supervised algorithms. The former method attempts separation without using any training sequences, instead being subjected to various
constraints, as proposed in [15, 16, 17, 18, 19]. However, these techniques havediﬃculty in clustering the decomposed spectral bases into a speciﬁc target soundbecause the entire procedure should be carried out in a blind fashion. To solve
this problem, supervised NMF (SNMF) [20] and its improved method, penalized
SNMF (PSNMF) [21, 23, 22], have been proposed. These methods include a priori
training, which requires some sound samples of a target instrument, and separate
the target signal using supervised bases. PSNMF can extract the target signal tosome extent, particularly in the case of a small number of sources. However, fora mixture consisting of many sources, such as more realistic musical tunes, thesource extraction performance is markedly degraded because of the existence of2

instruments with similar timbre.
To apply NMF-based separation methods to multichannel signals, multichannel NMF has been proposed as an unsupervised separation method [24, 25]. Thismethod is a natural extension of NMF for a stereo or multichannel signal and isa uniﬁed method that addresses the spatial and spectral separation problems simultaneously. However, such unsupervised separation is a diﬃcult problem, even
if the signal has multichannel components, because the decomposition is underspeciﬁed. Hence, these algorithms involve strong dependence on initial values
and lack robustness. For multichannel signal separation, directional clustering
has also been proposed as an unsupervised method [26, 27, 28]. This method
quantizes directional information via time-frequency binary masking under theassumption that the sources are completely sparse in the time-frequency domain.
However, there is an inherent problem that sources located in the same direction
cannot be separated using only the directional information. To cope with thisproblem, a hybrid method for multichannel signal separation, which concatenates
PSNMF after directional clustering, has been proposed [29]. However, this hybrid method also has a problem that the extracted signal suﬀers from considerable
distortion because the signal obtained by directional clustering has many spectral chasms, which mean spectral holes in the spectrogram. This results in the
cascaded SNMF being forced to incorrectly mimic such artiﬁcial spectral chasms.
In summary, no eﬀective technique has yet been proposed for separating thetarget source from a multichannel signal with high accuracy and satisfactoryrobustness. Therefore, attempts should be made to develop an eﬀective algorithm
for underdetermined signal separation. Such a robust signal separation method
for multichannel signals will be applicable to not only music signals but also
speech signals recorded by a microphone array to enhance the speech and suppress
interfering noise.

1.3 Scope of thesis
To achieve high-quality music signal separation with robustness, in this thesis, I
propose a new hybrid method that concatenates a new SNMF algorithm and an
unsupervised multichannel signal separation method. In addition, I also provide
a mathematical analysis for optimizing the proposed hybrid method. Figure 2
3

Higher

Separation performance

Proposed hybrid
method withdivergence diversity

Proposed
hybrid method

Scope of this thesis
Conventional
hybrid method
PSNMF
Multichannel
NMF

Directional
clustering

Lower
Lower

Robustness(availability for various signals)

HigherFigure 2. Relation between conventional methods and proposed hybrid methods.

depicts the relation between the conventional methods and the proposed hybrid
methods.
The hybrid method divides the stereo music signal separation problem into
two stages, namely, spatial separation and spectral separation. The spatial separation utilizes binary masking, which is performed by the directional clustering
technique, in the time-frequency domain. A clustering approach using spatial information is a common strategy used for multichannel signal separation because
it works well even in underdetermined situations. Then, in the spectral separation stage, a new SNMF algorithm is applied to separate the signals in the same
direction. In addition, this SNMF algorithm improves the sound quality of thetarget signal, which is deteriorated by the preceding binary masking performed
in directional clustering. Therefore, the proposed hybrid method is a divide-andconquer method that utilizes suitable decompositions in each separation problem

4and achieves robust multichannel signal separation with less sensitivity to theinitial values.
In the conventional hybrid method [29], the spectral chasms generated by binary masking degrade the sound quality of the separated target signal because
PSNMF is concatenated directly after directional clustering. To solve this problem, I propose a new SNMF with spectrogram restoration. By utilizing index
information generated from binary masking, the proposed SNMF regards the
spectral chasms as unseen observations, and ﬁnally reconstructs the target signal
components via spectrum extrapolation using supervised bases. In other words,
this SNMF can be categorized as a inpainting because the deteriorated spectrogram resulting from the preceding binary masking can be recovered. Note that an
SNMF-based extrapolation technique for acoustic signals has been proposed asa means of expanding the acoustic signal bandwidth [30]. However, this method
cannot be applied to signal separation.
The proposed SNMF with spectrogram restoration attempts both signal separation and basis extrapolation using the supervised bases. In previous studies, theanalysis of the optimal divergence criterion in SNMF has only been discussed forsignal separation [21, 22, 31], and the issue of the optimal divergence criterion in
SNMF for basis extrapolation has not been addressed. Therefore, in this thesis, I
analyze the ability of basis extrapolation for each divergence criterion in SNMF.

1.4 Outline of thesis
The thesis is organized as follows. First, I describe related works on singlechannel and multichannel signal separation methods in Sect. 2. In this section,
an overview of NMF is also given. In Sect. 3, I propose a new SNMF with
spectrogram restoration and derive its update rules for optimization. Also, therelation between the extrapolation ability and the divergence criterion in SNMF
is clariﬁed by theoretical analysis based on a signal generation model to ﬁnd
the optimal criterion for SNMF with spectrogram restoration. In addition, theeﬃcacy of the proposed hybrid method with the proposed SNMF is conﬁrmed
experimentally for musical signal separation. On the basis of the above-mentioned
ﬁndings, in Sect. 4, I propose a new method for switching the divergence criterionin SNMF with spectrogram restoration to adapt to various types of input signals
5and to separate the target signal robustly. The robustness of the proposed method
is conﬁrmed by experimental evaluations. Finally, I summarize the contributionsof this thesis and provide suggestions for future work in Sect. 5.

6

2. Conventional Signal Separation Methods
2.1 Introduction
In this section, I describe conventional music signal separation methods and theirproblems. In recent years, many types of signal separation methods have been
proposed and studied. These methods are roughly classiﬁed into single-channel
and multichannel signal separation algorithms. The former method attempts theunderdetermined separation using some constraints derived from the propertyof the target signal. The latter method uses a spatial cue, which is obtained as
diﬀerence between channels, as a directional information and separates the target
signals. Then, in this section, I review commonly used signal separation methods,
PSNMF, Directional clustering, and Multichannel NMF.
First, I outline conventional single-channel signal separation methods in Sect. 2.2.
Next, I give a brief review of multichannel signal separation methods and its problems in Sect. 2.3. Finally, Sect. 2.4 concludes this section.

2.2 Conventional single-channel signal separation methods
2.2.1 Overview of NMF
NMF is a type of sparse representation algorithm that decomposes a nonnegative
matrix into two nonnegative matrices as
X ≃ V W,

(1)

×N
where X(∈ RM) is an observed nonnegative matrix, which is an amplitude (or≥0a power) spectrogram for applying NMF to the acoustic signal; V (∈ R≥M0×D ) isoften called the basis matrix, which includes bases (frequently-appearing spectral
patterns in X) as column vectors; and W (∈ RD×N
) is often called the activation≥0matrix, which involves activation information of each basis of V . In addition, M
and N are the numbers of rows and columns of X, and D is the number of basesof V . Figure 3 depicts the decomposition model of NMF, where the number ofbases D equals two. The basis matrix includes two types of spectral patterns
as the bases to represent the observed matrix using time varying gains in theactivation matrix. In the decomposition of NMF, a cost function is deﬁned to7

Amplitude

Frequency

Frequency

Time

Activation matrix(Time-varying gain)
AmplitudeTime

Observed matrix(Spectrogram)

Basis matrix(Spectral patterns)

Figure 3. Decomposition model of simple NMF.

optimize the variables V and W using an arbitrary divergence between X andV W . The following equation represents the cost function of NMF:
JNMF = D(X∥V W ) ,(2)

where D(·∥·) is an arbitrary distance function, e.g., Itakura-Saito divergence (ISdivergence), generalized Kullback-Leibler divergence (KL-divergence), and Euclidean distance (EUC-distance). In this study, I use the following generalized
divergence called β-divergence [32] in the cost function:

{
}
∑

bβi,j
aβi,j bi,j aβ−1

i,j


+
−
(β ∈ R\ {0, 1} )β (β − 1)
β
β−1

i,j


{
}

∑
bi,j
+ ai,j − bi,j
(β = 1)bi,j log
Dβ (B∥A) =
(3)
,

a
i,j

i,j

}
{



∑
b
b

i,j
i,j


− log
−1
(β = 0)


ai,j
ai,j
i,j

where A(∈ RI×J ) and B(∈ RI×J ) are matrices whose entries are ai,j and bi,j ,respectively. This divergence is a family of cost functions parameterized by asingle shape parameter β that takes IS-divergence, KL-divergence, and EUCdistance as special cases (β = 0, 1, and 2, respectively) as shown in Figs. 6–8.
8

10
8
6
4
2
0
-5

-4

-3

-2

-1

0

1

2

3

4

5

Figure 4. Variation of β-divergence function when β = 0.

10
8
6
4
2
0
-5

-4

-3

-2

-1

0

1

2

3

4

Figure 5. Variation of β-divergence function when β = 1.

9

5

10
8
6
4
2
0
-5

-4

-3

-2

-1

0

1

2

3

4

5

Figure 6. Variation of β-divergence function when β = 2.

100
80
60
40
20
0
-5

-4

-3

-2

-1

0

1

2

3

4

Figure 7. Variation of β-divergence function when β = 3.

10

5

1000
800
600
400
200
0
-5

-4

-3

-2

-1

0

1

2

3

4

5

Figure 8. Variation of β-divergence function when β = 4.

The multiplicative update rules for V and W that minimize the cost function
based on β-divergence are given by [33])∑β−2 φ(β)
x
w
(
v
w
)
m,n
d,n
m,d
d,n
n
,
vm,d ← vm,d
∑
∑ d
β−1
n wd,n (d vm,d wd,n ))(∑∑β−2 φ(β)
d vm,d wd,n )m vm,d xm,n (,
wd,n ← wd,n∑∑β−1
d vm,d wd,n )m vm,d (
(∑

(4)

(5)

where xm,n , vm,d , and wd,n are the nonnegative entries of matrices X, V , and W ,respectively. In addition, φ(β) is given by


1


(β < 1) (2 − β)φ(β) = 1(6)(1 ≤ β ≤ 2) .




1



(β > 2)(β − 1)
We can optimize V and W by some iterations of these update rules. The convergence of these update rules is theoretically proven for any real-valued β.
11Training process

Supervised basis(dictionary)

Separation process

Fixed

UpdateFigure 9. Decomposition model in each process of SNMF.

2.2.2 SNMF and PSNMF
The signal separation using NMF is achieved by extracting only the target spectral bases. However, such unsupervised approaches have diﬃcultly in clustering
the decomposed spectral patterns into a speciﬁc target instruments. Furthermore, each basis may be forced to include a multi-instrumental spectral pattern.
To solve this problem, SNMF [20] and its improved method, PSNMF, have been
proposed [21, 22]. These supervised scheme consists of two processes, namely, apriori training and observed signal separation as shown in Fig. 9.
In SNMF, as the supervision, a priori spectral patterns (bases) should betrained in advance to achieve signal separation. Hereafter, we assume that we
can obtain speciﬁc solo-played instrumental sounds, which is the target of theseparation task. The trained bases are constructed by NMF as
Ytarget ≃ F Q,

(7)

where Ytarget (∈ RΩ×T
≥ 0 ) is an amplitude spectrogram of the speciﬁc instrumental
signal for training, F (∈ RΩ×K
≥ 0 ) is a nonnegative matrix that involves bases of thes12target signal as column vectors, and Q(∈ RK×T
) is a nonnegative matrix that≥0
corresponds to the activation of each basis of F . In addition, Ω is the numberof frequency bins, Ts is the number of frames of the training signal, and K is
the number of bases. Therefore, the basis matrix F constructed by (7) is thesupervision of the target instrumental spectra.
The following equation represents the decomposition model in separation process with trained supervision F :s

Y ≃ F G + HU ,

(8)

K×T
where Y (∈ RΩ×T
≥ 0 ) is an observed spectrogram, G(∈ R≥ 0 ) is an activation matrix
that corresponds to F , H(∈ RΩ×L
≥ 0 ) is the residual spectral patterns that cannot be
L×T
expressed by F G, and U (∈ R≥ 0 ) is an activation matrix that corresponds to H.
Moreover, T is the number of frames of the observed signal and L is the numberof bases of H. In SNMF, the matrices G, H, and U are optimized under thecondition that F is known in advance. Hence, ideally, F G represents the target
instrumental components, and HU represents other diﬀerent components fromthe target sounds after the decomposition. The cost function for (8) is deﬁned as

JSNMF = DβNMF (Y ∥F G + HU ) .

(9)

Also, the update rules for (9) are given by(∑

βNMF −2t yω,t ul,t zω,t∑βNMF −1
t ul,t zω,thω,l ← hω,l)φ(βNMF ),)βNMF −2 φ(βNMF )fyz
ω ω,k ω,t ω,t,gk,t ← gk,t∑βNMF −1
f
z
ω,k
ω,t
ω
(∑
)
βNMF −2 φ(βNMF )
h
y
z
ω,l
ω,t
ω,t
ω
ul,t ← ul,t,∑βNMF −1ω hω,l zω,t

(10)

(∑

(11)

(12)

where yω,t , fω,k , gk,t , hω,l , and ul,t are the nonnegative entries of the matrices Y ,F , G, H, and U , respectively, and∑∑zω,t =
fω,k gk,t +hω,l ul,t .
(13)
k

l

13

However, this SNMF incurs a risk of degrading the separation performanceowing to the simultaneous generation of similar spectral patterns in the supervised
basis matrix F and other basis matrix H (referred to as basis sharing problem).
This is because the cost function in SNMF is deﬁned as the divergence between
the observed and reconstructed matrices, and unique decomposition is not guaranteed. To solve this problem, PSNMF has been proposed [21, 22]. PSNMF
employs a penalty term in the cost function to force the other bases to become
as diﬀerent as possible from the supervised bases.
The cost function of PSNMF with orthogonality penalty is deﬁned as follows:
JPSNMF1 = DβNMF (Y ∥F G + HU ) + µ1 ∥F T H∥2Fr ,

(14)

∑
∑
where the conditions ω fω,k = 1 and ω hω,l = 1 are applied, µ1 is a weighting
parameter for the penalty term, and ∥ · ∥Fr indicates the Frobenius norm. The
minimization of the second term in (14) corresponds to the maximization oforthogonality between F and H. The update rule for H, which minimizes thecost function (14), is given by(hω,l ← hω,l∑βNMF −2t yω,t ul,t zω,t∑∑∑βNMF −1+ 2µ1 k fω,k ω′ fω′ ,k hω′ ,lt ul,t zω,t)φ(βNMF ).

(15)

The update rules for G and U are the same as (11) and (12).
Also, maximum-divergence penalty, which maximizes all divergence combinations between the supervised bases in F and the other bases in H, has been
proposed as another means of preventing the basis sharing problem. The costfunction of PSNMF with maximum-divergence penalty is deﬁned as follows:()
1 ∑Dβ (fω,k ∥hω,l ) , (16)JPSNMF2 = DβNMF (Y ∥F G + HU ) + µ2 exp −λm k,l,ω mwhere µ2 and λm are the weighting and sensitivity parameters, respectively. Here,
exponentiation is applied to make the penalty term nonnegative.
The update rule for H, which minimizes the cost function (16), is given by(hω,l ← hω,l)φ(βNMF )∑βNMF −2λm t yω,t ul,t zω,t+ µ2 hβω,lm −1 Cβm,∑∑
βNMF −1λm t ul,t zω,t+ µ2 hβω,lm −2 Cβm k fω,k
14

(17)

where)1 ∑Dβ (fω,k ∥hω,l ) .
Cβm = exp −λm k,l,ω m((18)

The update rules for G and U are the same as (11) and (12).
By imposing these penalty terms on the cost function, we can prevent thebasis sharing problem and separate the target signal with high accuracy. The
separation performance of PSNMF with orthogonality penalty and PSNMF withmaximum-divergence penalty are almost the same [22]. In this thesis, hereafter,
I use the orthogonality penalty and its update rules.
PSNMF can extract the target signal to some extent, particularly in the case ofa small number of sources. However, for the case of a mixture consisting of manysources, such as more realistic musical tunes, the source extraction performanceis markedly degraded because of the existence of instruments with similar timbre.

2.3 Conventional multichannel signal separation methods
2.3.1 Directional clustering
Decomposition methods employing directional information for the multichannel
signal have also been proposed as unsupervised separation techniques [26, 27,28]. These methods quantize directional information via time-frequency binary
masking under the assumption that the sources are completely sparse (double
disjoint) in the time-frequency domain. Figure 10 shows the separation algorithmof directional clustering. The target source in the center direction is separated
by hard clustering method, which corresponds to the binary masking in the timefrequency domain.
Such directional clustering works well, even in an underdetermined situation
where the number of sources is greater than that of inputs. However, there is an
inherent problem that sources located in the same direction cannot be separatedusing the directional information. Furthermore, the extracted signal is likely to
be distorted because of the eﬀect of binary masking.

15

2.3.2 Hybrid method of directional clustering and PSNMF
To separate the sources in the same direction, a hybrid method that concatenates
PSNMF after directional clustering has been proposed [29]. Figure 11 indicatesthe signal ﬂow of the hybrid method. This hybrid method can eﬀectively extract
the target instrument because the directionally clustered signal contains only few
instruments. Moreover, the residual interfering signal in the same direction canbe removed by PSNMF.
However, this hybrid method has a problem that the extracted signal suﬀers
from the generation of considerable distortion. This is due to the binary masking
in directional clustering. The signal in the target direction, which is obtainedby directional clustering, has many spectral chasms because the assumption ofsparseness in the time-frequency domain does not always hold completely. In
other words, the resolution of the spectrogram clustered as the target-direction
component is degraded by time-frequency binary masking. Figure 12 shows anexample of the spectrum of a signal separated by directional clustering. The
obtained spectrum has many chasms owing to the binary masking. These spectral
losses may deteriorate the performance of separation because PSNMF is forced
to incorrectly ﬁt these spectral chasms using supervised bases.
2.3.3 Multichannel NMF
Multichannel NMF, which is a natural extension of NMF for a stereo or multichannel music signal, has been proposed as an unsupervised signal separation
method [24, 25]. These algorithms employ Hermitian positive deﬁnite matrix thatmodels the spatial property of each NMF basis and each frequency bin. Therefore, multichannel NMF utilizes a frequency-wise transfer function between signal
source and microphone as a cue for basis clustering. However, such unsupervised
separation is a diﬃcult problem, even if the signal has multichannel components,
because the decomposition is underspeciﬁed. Hence, these algorithms involve
strong dependence on initial values and lack robustness.

16Frequency ofsource component

Left

Frequency ofsource component

Target source(a) Observed
spectraCenterDirection

Right

CenterDirection

Right(b) After
directional
clustering

Left

Figure 10. Directional source distribution of (a) observed stereo signal, (b) separated target components in the center cluster.

17

Input stereo signal
L-ch
R-ch
STFT
Directional clustering
Target clustercomponents
L-ch

R-ch

PSNMF

PSNMF

ISTFT

ISTFT
MixingExtracted signal
Figure 11. Signal ﬂow of conventional hybrid method; PSNMFs are cascaded
after stereo output of directional clustering.

18

40

Amplitude [dB]

30
20
10
0
-10
-20
-30
-40
0

200

400

600

800

1000

Frequency [Hz]Figure 12. Example of spectrum of signal separated by directional clustering.

2.4 Conclusion
In this section, conventional single-channel signal separation methods were denoted. Next, conventional multichannel signal separation methods were reviewed.
Since each method has its own drawback, I propose a new hybrid method of directional clustering and a new SNMF with spectrogram restoration in the next
section to cope with the problems eﬀectively.

19

3. SNMF with Spectrogram Restoration and Its
Hybrid Method
3.1 Introduction
In the previous section, I described two types of conventional signal separation
methods and the problems of each method. To solve these problems, in thissection, I propose a new algorithm of SNMF with spectrogram restoration andits hybrid method as a multichannel signal separation method.
First, I describe a strategy and a derivation of update rules for SNMF with
spectrogram restoration in Sect. 3.2. Second, theoretical analysis of basis extrapolation ability based on generation model is shown in Sect. 3.3. Third, I compare
separation performance of the proposed method and the other conventional methods via some experiments in Sect. 3.4 for a validation of the proposed method.
Finally, Sect. 3.5 concludes this section.

3.2 SNMF with spectrogram restoration
3.2.1 Motivation and strategy
The separated signal by the conventional hybrid method described in Sect. 2.3.2
suﬀers from the generation of considerable distortion owing to the binary masking
in directional clustering. To solve this problem, in this section, I propose a new
SNMF with spectrogram restoration as an alternative to the conventional PSNMF
for the hybrid method.
Figure 13 shows a signal ﬂow of the proposed hybrid method that includes
SNMF with spectrogram restoration. The algorithm of SNMF with spectrogram
restoration utilizes index information determined in directional clustering. For
example, if the target instrument is localized in the center cluster along with
the interference, SNMF is only applied to the existing center components using
index information (active binary mask). Therefore, the spectrogram of the target
instrument is reconstructed using more matched bases because spectral chasms
are treated as unseen, and these chasms have no impact on the cost function in
SNMF with spectrogram restoration. In addition, the components of the target

20Amplitude

Input stereo signal
L-ch
R-ch
STFT
Directional clustering
Index oftargetcluster

Chasm

Frequency

Target clustercomponents

Spectrum separated bydirectional clustering

R-ch
SNMF with
spectrogram
restoration

ISTFT

ISTFT

Amplitude

L-ch
SNMF with
spectrogram
restoration

MixingFrequency

Extracted signal

Spectrum reconstructedby extrapolation

Figure 13. Signal ﬂow of proposed hybrid method; SNMF with spectrogram
restoration concatenates after directional clustering.
instrument lost after directional clustering can be extrapolated using the supervised bases. In other words, the deteriorated target spectrogram is recovered withthe spectrogram restoration by the supervised basis extrapolation.
To illustrate the separation mechanism step by step, Fig. 14 (a) shows the
conﬁguration of source components in the stereo signal, (b) shows the separated
components that are clustered around the center direction by directional clustering, and (c) shows the separated target component obtained by SNMF with
spectrogram restoration. In Fig. 14 (a), the source components are distributedin all directions with some overlapping. After directional clustering (Fig. 14 (b)),
the center sources lose some of their components (i.e., the tails on both sides),and the other source components leak in the center cluster. After SNMF with
spectrogram restoration, the proposed algorithm restores the lost components
using the supervised bases (Fig. 14 (c)).

21Frequency ofsource component
Frequency ofsource component

Left

CenterDirection

Right

CenterDirection

Right(b) After
directional
clustering

Left
Frequency ofsource component

Target source(a) Observed
spectra(c) After
SNMF with
spectrogram
restoration

Left

Extrapolated
target source

CenterDirection

Right

Figure 14. Directional source distribution of (a) observed stereo signal, (b) separated components in center cluster, and (c) component separated and extrapolated by spectrogram restoration.
22

However, this basis extrapolation includes an underlying problem. If the timefrequency spectra are almost unseen in the spectrogram, which means that theindexes are almost zero, a large extrapolation error may occur. Then, incorrectbases are chosen and ﬁtted to a small number of spectral grids by incorrectly
modifying the activation matrix G. In the worst case, the activation matrix G
contains very large values and the extracted signal is overloaded. To avoid this,we should add a new penalty term in the cost function, as described in the next
section.
3.2.2 Cost function and update rules
In this section, we derive the update rules of SNMF with spectrogram restoration
based on β-divergence-based. Here, the index matrix I(∈ RΩ×T
{0, 1} ) is obtained fromthe binary masking preceding the directional clustering. This index matrix has
speciﬁc entries of unity or zero, which indicates whether or not each grid of thespectrogram belongs to the target directional cluster. The cost function in SNMF
with spectrogram restoration is deﬁned using the index matrix I as∑∑∑
J (Θ) =
iω,t DβNMF (yω,t ∥ k fω,k gk,t + l hω,l ul,t )
ω,t

+λ

∑

iω,t Dβreg (0∥∑k fω,k gk,t ) + µ∥F

T

H∥2F ,(19)

ω,twhere Θ = {G, H, U } is the set of objective variables, iω,t is a entry of the index
matrix I, λ and µ are the weighting parameters for each term, and · represents thebinary complement of each entry in the index matrix. The ﬁrst term represents
the main cost of separation in SNMF. Since the divergence DβNMF (·∥·) is only
deﬁned in grids whose index is one, the chasms in the spectrogram are ignored inthis SNMF decomposition. The second term forces the minimization of the value∑of k fω,k gk,t . Hence, the supervised bases are chosen so as to minimize the scaleof F G in proportion to the number of zeros in the index matrix I in each frame
to avoid the extrapolation error. In other words, this penalty term regulates theextrapolation. In addition, the third penalty term has the same property as thatin the cost function of conventional PSNMF (14).
The update rules based on (19) are obtained by the auxiliary function approach, similarly to [33]. Here, we can rewrite the cost function (19) using β23

divergence asJ (Θ) = J1 + λJ2 + µJ3 ,
)
(
βNMF
βNMF
βNMF −1∑yω,tzω,tyω,t zω,t
J1 =
+
−
,
iω,t
β
(β
−
1)
β
β
−
1
NMF
NMF
NMF
NMF
ω,t
∑
∑
( k fω,k gk,t )βreg
J2 =
iω,t
,
β
reg
ω,t
(
)2
∑ ∑
J3 =
fω,k hω,l .

(20)
(21)
(22)

(23)

ω

k,l

First, I deﬁne the upper bound function for J1 . The second term of J1 isconvex for βNMF ≥ 1 and concave for βNMF < 1, and the third term is convex for
βNMF ≥ 2 and concave for βNMF < 2. Applying Jensen’s inequality to the convex
function and the tangent line inequality to the concave function, we can deﬁne theupper bound function J1+ using auxiliary variables αω,t,k ≥ 0, γω,t,l ≥ 0, η1 ≥ 0,∑∑
η2 ≥ 0, and σω,t that satisfy k αω,t,k = 1, l γω,t,l = 1, and η1 +η2 = 1 as∑(β)
J1 ≤ J1+ =iω,t Pω,tNMF ,
(24)
ω,t

where

(β
)
Pω,tNMF =(βNMF )(β−1)
− yω,t Mω,tNMF

Nω,t
(β

)

(β

(βNMF < 1)

−1)

(25)
Mω,tNMF − yω,t Mω,tNMF(1 ≤ βNMF ≤ 2) ,M(βNMF ) − yω,t N (βNMF −1)(βNMF > 2)
ω,t
ω,t
{
(
)βNMF ∑
)βNMF }
(
∑
h
u
1
f
g
ω,l l,tω,k k,t(β)γω,t,l η2
Mω,tNMF =αω,t,k η1
+
,
βNMF
α
γ
ω,t,k η1
ω,t,l η2lk
(26)
βNMF −1(zω,t − σω,t ) +Nω,tNMF = σω,t
(β

)

βNMF
σω,t

βNMF

24

.

(27)

The equality in (24) holds if and only if the auxiliary variables are set as follows:fω,k gk,t,k′ fω,k′ gk′ ,thω,l ul,tγω,t,l = ∑,l′ hω,l′ ul′ ,t∑′ gk ′ ,tk′ fω,k∑,η1 = ∑′ gk ′ ,t +l′ hω,l′ ul′ ,tk′ fω,k∑ul′ ,tl′ hω,l′∑,η2 = ∑
k′ fω,k′ gk′ ,t +l′ hω,l′ ul′ ,t∑∑fω,k′ gk′ ,t +hω,l′ ul′ ,t .
σω,t =

αω,t,k = ∑

k′

(28)
(29)
(30)
(31)
(32)

l′

Second, I deﬁne the upper bound function for J2 . This term is convex for
βreg ≥ 1 and concave for βreg < 1. Similarly to (24)-(27), we can deﬁne the upperbound function J2+ using auxiliary variables αω,t,k and ρω,t as
J2 ≤ J2+ =

∑

(β

)

iω,t Sω,treg ,

(33)

ω,t

where


βreg

ρω,t

βreg −1 ∑ρω,t ( k fω,k gk,t − ρω,t ) + β(βreg )(
)βreg regSω,t =
fω,k gk,t1 ∑k αω,t,kβαω,t,kreg(βreg < 1).

(34)

(1 ≤ βreg )The equality in (33) holds if and only if the auxiliary variables are set as (28)
and as follows:∑ρω,t =
fω,k′ gk′ ,t .
(35)
k′Third, I deﬁne the upper bound function for J3 using auxiliary variables∑
δk,l,ω ≥ 0 that satisfy ω δk,l,ω = 1 as
J3 ≤ J3+ =2∑ fω,k
h2ω,l
.
δ
k,l,ω
k,l,ω

25

(36)

The equality in (36) holds if and only if the auxiliary variables are set as follows:fω,k hω,lδk,l,ω = ∑.
(37)
ω ′ fω ′ ,k hω ′ ,lFinally, using (24), (33), and (36), we can deﬁne the upper bound function
J +(Θ, Θ̂) as
J +(Θ, Θ̂) = J1+ + λJ2+ + µJ3+ ,(38)

where Θ̂ is the set of auxiliary variables. The update rules with respect to eachvariable are determined by setting the gradient to zero.
From ∂J +(Θ, Θ̂)/∂gk,t = 0, we obtain()∑
iω,t VβNMF (Θ, Θ̂) − WβNMF (Θ, Θ̂) + λXβreg (Θ, Θ̂) = 0,
(39)
ω

where


σ βNMF −1 fω,kω,tVβNMF (Θ, Θ̂) =
β
−1
βNMF
NMF
g
(αk,ω,t η1 )1−βNMF fω,kk,t
2−βNMF βNMF −1
y g β−2 (αfω,kω,t k,tk,ω,t η1 )WβNMF (Θ, Θ̂) =yω,t σ βNMF −2 fω,kω,t∑βreg −1 ω iω,t ρω,t ( fω,k )βreg −1Xβreg (Θ, Θ̂) = ∑fω,k gk,t ω iω,t fω,k α(βNMF < 1)
(1 ≤ βNMF )(βNMF ≤ 2)

,

(40)

,

(41)

(2 < βNMF )(βreg < 1)
(1 ≤ βreg )

.

(42)

ω,t,k

By solving (39) for gk,t under the nonnegativity, we obtain() 1∑2−βNMF βNMF −1 2−βNMF


i
y
(α
η
)
f
k,ω,t 1ω,kω ω,t ω,t


∑

β
−1
NMF

fω,k + λXβregω iω,t σω,t





(βNMF < 1)


∑

2−β
β
−1
β
−1

NMF
NMF

fω,k
gk,tNMF ∑ ω iω,t yω,t (αk,ω,t η1 )βNMF −1
βNMF
gk,t =. (43)(αk,ω,t η1 )1−βNMF fω,k+ λXβregω iω,t gk,t
(1 ≤ βNMF ≤ 2)
) β 1 −1(
∑βNMF −2
βNMF −1

NMF

i
y
σ
f
g

ω,k k,tω ω,t ω,t ω,t∑βNMF −1βNMF(αk,ω,t η1 )1−βNMF fω,k+ λXβregω iω,t gk,t

(2 < βNMF )26

Then we can obtain the update rule of gk,t by substituting (28), (30), (32), and(35) into (43) as follows:(
gk,t ← gk,t∑βNMF −2ω iω,t yω,t fω,k zω,t∑∑∑βNMF −1+ λ ω iω,t fω,k ( k′ fω,k′ gk′ ,t )βreg −1ω iω,t fω,k zω,t)φ(βNMF ).
(44)

The update rules of the other variables are similarly obtained as follows:(∑

βNMF −2t iω,t yω,t ul,t zω,thω,l ← hω,l ∑∑∑βNMF −1+ 2µ k fω,k ω′ fω′ ,k hω′ ,lt iω,t ul,t zω,t)(∑βNMF −2 φ(βNMF )iyhzω ω,t ω,t ω,l ω,tul,t ← ul,t.
∑βNMF −1ω iω,t hω,l zω,t)φ(βNMF )
,

(45)

(46)

The convergence of these update rules is theoretically proven for any real-valued
βNMF and βreg .

3.3 Theoretical analysis of basis extrapolation based on
generation model
3.3.1 Optimal divergence for basis extrapolation and generation model
The proposed method attempts both signal separation and basis extrapolation
using the supervised bases F . In previous studies, the analysis of optimal divergence only for signal separation has been discussed [21, 22, 31]. However, there
has been no discussion on the optimal divergence for the extrapolation techniques
using NMF. In this section, I analyze the extrapolation ability based on a statistical generation model of the observed data Y , and determine the optimal
divergence for basis extrapolation w.r.t. various βNMF and βreg values.
In NMF decomposition, the minimization of β-divergence between Y andF G corresponds to a log-likelihood maximization under the assumption of thegeneration model of Y for each βNMF [34]. The minimization of DβNMF (yω,t ∥ϑ)is equivalent to the maximization of exp(−DβNMF (yω,t ∥ϑ)). Here, we can rewrite27

exp(−DβNMF (yω,t ∥ϑ)) as

(
)

y
y

ω,t
ω,t


exp −
+1
(βNMF = 0)


ϑ
ϑ



(
)yω,t



ϑe


exp (−ϑ)(βNMF = 1) yω,t(),
exp (−DβNMF (yω,t ∥ϑ)) =
(47)
2

(y
−
ϑ)
ω,t



exp −
(βNMF = 2)


2


(
)


βNMF −1
βNMF

ϑ
y
ϑ

ω,t


−
(βNMF ≥ 3)
exp
βNMF − 1
βNMF∑
where ϑ = k fω,k gk,t represents a parameter of the maximum likelihood estimation. A probability density function (p.d.f.) that corresponds to (47) is given
by

(
)

1
yω,t



exp −
(βNMF = 0)


ϑ1
ϑ1





ϑ2 yω,t(βNMF = 1)
 Γ (y + 1) exp (−ϑ2 )ω,t()
yω,t ∼ p (yω,t ) =,(48)(yω,t − ϑ4 )2
1



√
exp −
(βNMF = 2)



2πϑ3
2ϑ3 2


( β −1 )ϑ5 NMF yω,t


C
exp
(βNMF ≥ 3)

βNMF − 1
where Γ(·) is a gamma function. These generation models of βNMF = 0, 1, and2 are equivalent to exponential distribution, Poisson distribution, and Gaussian
distribution, respectively. The generation models for βNMF ≥ 3 correspond to a
distribution in which the probability increases exponentially with increasing yω,t .
Strictly, this distribution is not a p.d.f. because it diverges when yω,t increases.
Thus, we set the upper bound of yω,t to a constant M and deﬁne the corresponding
p.d.f. with normalization coeﬃcient Cm , which is given by))−1(( β −1
ϑ5 NMF−1
βNMF −1C = ϑ5Cm − 1.
(49)(βNMF − 1)
exp
βNMF − 1
Using (48), we can generate the most probable spectrogram for each βNMF .
28

3.3.2 Simulation conditions
To analyze the net extrapolation ability, I simulate the spectrogram restoration
task. In this simulation, I generated random i.i.d. values, which obey the corresponding generation model (48) for each βNMF , as the observed data matrix
Y . I compared βNMF = 0, 1, 2, 3, 4 and βreg = 0, 1, 2, 3, and I used the same
divergence βNMF in the training and separation processes. The size of this data
matrix was set to Ω = 5000 and T = 200. I set the parameters of each p.d.f. toϑ1 = 1, ϑ2 = 5, ϑ3 = 10, ϑ4 = 50, ϑ5 = 2, and Cm = 15. These parameters
are determined so as to generate the nonnegative random i.i.d. values that obey
each corresponding generation model. Note that the parameters θ1 –θ5 simply
determine the scales of the input random variables, and basically can be set toarbitrary value without loss of generality. In addition, I used two types of datamissing patterns I, in which 75% or 98% of the grids were missing in a uniform
manner, and the missing data I ◦ Y imitated the binary-masking procedure. The
supervised bases F were obtained by training using the same data matrix Y ,namely, Ytarget = Y in Fig. 9. The number of supervised bases, K, was 100,
which is the half size of T , and the number of other bases, L, was 30. Therefore,
the task was to reconstruct original Y from the observations with missing data,
I ◦ Y , using the trained bases.
3.3.3 Simulation results and discussion
I used sources-to-artifacts ratio (SAR) deﬁned in [35] as the accuracy of theextrapolation. Here, the estimated signal ŝ (t) is deﬁned asŝ (t) = starget (t) + sinterf (t) + sartif (t) ,(50)

where starget (t) is the allowable deformation of the target source, sinterf (t) is theallowable deformation of the sources that account for the interferences of theundesired sources, and sartif (t) is an artifact term that may correspond to theartifacts of the separation algorithm, such as musical noise, or simply undesirable
deformation induced by the nonlinear property of the separation algorithm. The
formulas for SAR is deﬁned as∑(t) + einterf (t)}2t {starget∑(51).
SAR = 10 log102t eartif (t)29Therefore, SAR indicates the absence of artiﬁcial distortion.
Figure 15 shows the SAR result for each divergence and regularization. From
this result, it is conﬁrmed that a higher βNMF provides better basis extrapolation regardless of the type of regularization (βreg ). In NMF decomposition,
if we set βNMF to a large value, the trained bases tend to become anti-sparse(smooth). In contrast, if βNMF is close to zero, the trained bases become moresparsity-aware, and this property is suitable for normal NMF-based music source
separation because of the inherent sparsity of music spectrograms (e.g., βNMF = 1
is recommended in [21, 22, 31]). However, for basis extrapolation, sparse bases
are not suitable because it is diﬃcult to extrapolate them only from the observable data. Therefore, we speculate that the optimal divergence in SNMF with
spectrogram restoration, which attempts to ﬁt the trained bases using spectral
components except for chasms, is shifted to βNMF > 1 rather than KL-divergence(βNMF = 1) because of the trade-oﬀ between separation and extrapolation abilities, as illustrated in Fig. 16. This issue will be conﬁrmed experimentally in thenext section.

30

βreg= 1βreg= 3βreg= 0βreg= 225(a)

SAR [dB]

20
15
10
5
0
0
25

1

2

3

4

2

3

4

βNMF

(b)

SAR [dB]

20
15
10
5
0
0

1

βNMF

Figure 15. Extrapolation abilities for (a) 75%-binary-masked data and (b) 98%binary-masked data.

31

Performance

Optimum in SNMF with
spectrogram restoration
Optimum in
SNMF
Total performance

Shift
Extrapolation

Separation

0

1

2

3

4

β NMF
Figure 16. Trade-oﬀ between separation and extrapolation abilities. Overall
performance is highest when βNMF > 1.

3.4 Comparison between proposed hybrid method and conventional methods
3.4.1 Experimental conditions
I conducted objective evaluation to conﬁrm the eﬀectiveness of the proposed hybrid method described in the previous section. In this experiment, I comparedﬁve methods, namely, simple directional clustering [26], Multichannel NMF basedon IS-divergence [25], PSNMF [21, 22], conventional hybrid method that concatenates PSNMF after directional clustering [29], and proposed hybrid method including SNMF with spectrogram restoration after directional clustering, in terms
321 1 1 ¶ ED? E E E ED E E EE E EE E DD¶
¶ EE E EEE E E E E D´E E E EE E D
Figure 17. Scores of each part.
Table 1. Compositions of musical instruments

Dataset Melody 1
C1
Oboe
C2
TrumpetC3
Horn

Melody 2
Midrange
Bass
Flute
Piano
Trombone
ViolinHarpsichord
FagottoClarinet
Piano
Cello

of their ability to separate music artiﬁcial and real-recorded signals. Also, I compared some evaluation scores with various βNMF and βreg for PSNMF and the
proposed hybrid method by setting ﬁve divergences and regularizations, namely,
β = 0, 1, 2, 3, and 4. I used the same divergence (βNMF ) in the training and separation processes for PSNMF and proposed SNMF with spectrogram restorationin the proposed hybrid method. In this experiment, I conducted two experiments
to consider artiﬁcial signal and real-recorded signal cases. I used stereo signals
containing four melody parts (depicted in Fig. 17) with three compositions (C1–
C3) of instruments shown in Table 1. These signals were artiﬁcially generated
by a MIDI synthesizer. In particular, these stereo signals were mixed down toa monaural format only when we evaluate the separation accuracy of PSNMF
because PSNMF is a separation method for a monaural input signal.
In the artiﬁcial signal case, the observed signals Y were produced by mixing
33

Left

Center

Right

4
2

1

3

θ θ
Figure 18. Panning of four sources with sine law used in artiﬁcial signal caseexperiment. Numbered black circles represent locations of instruments in stereo
format. For example, if target is Ob., No.1 is set to Ob. and Nos.2, 3, and 4 are
combinations of Fl., Tb., and Pf.
four sources with the same power. The observed signal contained one source inthe left and right directions and two sources in the center direction based on a sinelaw (see Fig. 18). The target instrument is always located in the center direction
along with another interfering instrument, and we prepared two patterns in which
the left and right sources are located at θ = 15◦ and 45◦ . In addition, I used thesame MIDI sounds of the target instruments as supervision for a priori training.
The training sounds contained two octave notes that cover all notes of the target
signal in the observed signal. The sampling frequency of all signals was 44.1 kHz.
The spectrograms were computed using a 92-ms-long rectangular window witha 46-ms overlap shift. The number of iterations for the training and separation
were 500. Moreover, the number of clusters used in directional clustering was 3,
the number of a priori bases, K, was 100, and the number of bases for matrix H,
H, was 30. The weighting parameters λ and µ were empirically determined.
34

Center

4

Left

2

2.5 m

Right

1
1.5 m

1.5 m

3
1.5 m

Dummy head
Figure 19. Geometry of the loudspeaker and binaural microphone (dummy head).
Numbered black circles represent locations of loudspeakers. Target source andsupervision sound is always located in No.1 position.
In the real-recorded signal case, I recorded each instrumental solo signal andthe supervision sound, which are the same as those in the artiﬁcial case, in anexperimental room whose reverberation time was 200 ms. A geometry of theloudspeaker and binaural microphone NEUMANN KU-100 is shown in Fig. 19.
The target source and the supervision sound is always located in No.1 position in
Fig. 19. The observed signal Y was produced by mixing these recorded signals as
the same power. Other conditions were the same as those of the artiﬁcial signal
case.
35

3.4.2 Experimental results
I used the signal-to-distortion ratio (SDR), source-to-interference ratio (SIR), andSAR deﬁned in [35] as the evaluation scores. The formulas for SDR and SIR aredeﬁned as∑2
t starget (t)SDR = 10 log10 ∑(52)
2,
t {einterf (t) + eartif (t)}∑2t starget (t)∑SIR = 10 log10.
(53)
2t einterf (t)SDR indicates the quality of the separated target sound, and SIR indicates thedegree of separation between the target and other sounds. Therefore, SDR indicates the total evaluation score that involves SIR and SAR.
First, I compare the variation of separation performance for various βNMF andβreg . Figures 20 and 21 show the average SDR, SIR, and SAR of the proposed
hybrid method for each divergence (βNMF ) and each regularization (βreg ) in theartiﬁcial signal case with θ = 15◦ and θ = 45◦ , where the four instruments are
shuﬄed with 12 combinations in each of compositions C1–C3. Therefore, theseresults are the averages of 36 input signal patterns. Also, Fig. 22 show the average
SDR, SIR, and SAR in the real-recorded signal case. From the SDRs in Figs.
20, 21, and 22, the regularization with KL-divergence (βreg = 1) is slightly better
than the other divergences but the diﬀerence is not signiﬁcant, except for thecase of βreg = 0. In addition, we can conﬁrm that the EUC-distance-based costfunction (βNMF = 2) is an optimal divergence for the proposed hybrid method
including SNMF with spectrogram restoration.
Next, I compare the separation performance of the proposed hybrid methodwith the other conventional methods, where I compare the evaluation scores of the
proposed hybrid method only when βreg = 1 because this KL-divergence-based
regularization achieves the highest separation performance. Figures 23 and 24
show the average SDR, SIR, and SAR for each method in the artiﬁcial signal
case with θ = 15◦ and θ = 45◦ . Also, Fig. 25 shows the average SDR, SIR,
and SAR in the real-recorded signal case. Similarly to the previous results, theseresults are the averages of 36 input signal patterns, which contain all compositions
and instrumental combinations.

36

From the SDRs in Figs. 23, 24, and 25, we can conﬁrm that directional clustering does not have suﬃcient performance because this method cannot discriminate
the sources in the same direction. Multichannel NMF also cannot achieve the sufﬁcient separation because this method strongly depends on the initial value andlack robustness. In contrast, the methods using SNMF can give better results
and the proposed hybrid method with SNMF with spectrogram restoration outperforms all other methods in both artiﬁcial and real-recorded signal cases. In
addition, the conventional hybrid method is inferior to PSNMF when βNMF ≤ 1
whereas this hybrid method utilizes both directional clustering and PSNMF. Thisis because the conventional hybrid method is aﬀected by the spectral chasms andcannot reconstruct such lost components. Furthermore, we can conﬁrm that the
EUC-distance-based cost function (βNMF = 2) is an optimal divergence for the
proposed hybrid method, whereas KL-divergence (βNMF = 1) is the best divergence even for conventional PSNMF [21, 22, 31]. This marked shift of the optimal
divergence in SNMF with spectrogram restoration is due to the trade-oﬀ between
the separation and extrapolation abilities, as predicted in Sect. 3.3.

37

14

(a)

SIR [dB]

SDR [dB]

12
10
8
6
4
2

βreg= 0βreg= 3

0
0

1

22
20
18
16
14
12
10
8
6
4
2
0
0

βreg= 1βreg= 4β reg= 2

2

3

4

βNMF

(b)

βreg= 0βreg= 3βreg= 1βreg= 4βreg= 2

2

3

1

4

βNMF

12

(c)

SAR [dB]

10
8
6
4
2

βreg= 0βreg= 3001βreg= 1βreg= 4β reg= 2

2

3

4

βNMF
Figure 20. Average scores with various divergences and regularizations in artiﬁcial
signal case when θ = 15◦ : (a) shows SDR, (b) shows SIR, and (c) shows SAR for
proposed methods.
38

14

(a)

SIR [dB]

SDR [dB]

12
10
8
6
4
2

βreg= 0βreg= 3

0
0

1

22
20
18
16
14
12
10
8
6
4
2
0
0

βreg= 1βreg= 4βreg= 2

2

3

4

βNMF

(b)

βreg= 0βreg= 3βreg= 1βreg= 4β reg= 2

2

3

1

4

βNMF

12

(c)

SAR [dB]

10
8
6
4
2

βreg= 0βreg= 3001βreg= 1βreg= 4β reg= 2

2

3

4

βNMF
Figure 21. Average scores with various divergences and regularizations in artiﬁcial
signal case when θ = 45◦ : (a) shows SDR, (b) shows SIR, and (c) shows SAR for
conventional and proposed methods.
39

14

(a)

SIR [dB]

SDR [dB]

12
10
8
6
4
2

βreg= 0βreg= 3

0
0

1

22
20
18
16
14
12
10
8
6
4
2
0
0

βreg= 1βreg= 4β reg= 2

2

3

4

βNMF

(b)

βreg= 0βreg= 3βreg= 1βreg= 4β reg= 2

2

3

1

4

βNMF

12

(c)

SAR [dB]

10
8
6
4
2

βreg= 0βreg= 3001βreg= 1βreg= 4βreg= 2

2

3

4

βNMF
Figure 22. Average scores with various divergences and regularizations in realrecorded signal case: (a) shows SDR, (b) shows SIR, and (c) shows SAR for
conventional and proposed methods.
40

Directional clustering
Multichannel NMF
PSNMF
Conventional hybrid method
Proposed hybrid method (βreg= 1)

14

SDR [dB]

12 (a)
10
8
6
4
2
0

SIR [dB]012220 (b)
18
16
14
12
10
8
6
4
2
0
0

1

SAR [dB]

3

4

2

3

4

3

4

βNMF

12
10

2

βNMF

(c)

8
6
4
2
0
0

1

2

βNMF
Figure 23. Average scores in artiﬁcial signal case when θ = 15◦ : (a) shows SDR,(b) shows SIR, and (c) shows SAR for conventional and proposed methods.
41

Directional clustering
Multichannel NMF
PSNMF
Conventional hybrid method
Proposed hybrid method (βreg= 1)

14

SDR [dB]

12 (a)
10
8
6
4
2
0

SIR [dB]012220 (b)
18
16
14
12
10
8
6
4
2
0
0

1

SAR [dB]

3

4

2

3

4

3

4

βNMF

12
10

2

βNMF

(c)

8
6
4
2
0
0

1

2

βNMF
Figure 24. Average scores in artiﬁcial signal case when θ = 45◦ : (a) shows SDR,(b) shows SIR, and (c) shows SAR for conventional and proposed methods.
42

Directional clustering
Multichannel NMF
PSNMF
Conventional hybrid method
Proposed hybrid method (βreg= 1)

14

SDR [dB]

12 (a)
10
8
6
4
2
0

SIR [dB]012220 (b)
18
16
14
12
10
8
6
4
2
0
0

1

SAR [dB]

3

4

2

3

4

3

4

βNMF

12
10

2

βNMF

(c)

8
6
4
2
0
0

1

2

βNMF
Figure 25. Average scores in real-recorded signal case: (a) shows SDR, (b) shows
SIR, and (c) shows SAR for conventional and proposed methods.
43

3.5 Conclusion
In this section, ﬁrst, I derived the update rules of SNMF with spectrogram restoration and proposed hybrid method that concatenates SNMF with spectrogram
restoration after directional clustering. This SNMF attempts both signal separation and basis extrapolation simultaneously.
Next, I analyzed the net extrapolation ability based on generation modelsof each divergence. This analysis revealed the mechanism of marked shift of
optimal divergence in SNMF with spectrogram restoration and trade-oﬀ between
separation and extrapolation abilities owing to the diﬀerence of sparseness in eachdivergence.
Finally, the eﬀectiveness of the proposed hybrid method was conﬁrmed by
the experiments for artiﬁcial and real-recorded signals. The results showed themarked shift of the optimal divergence for the proposed hybrid method becauseof the trade-oﬀ between separation and extrapolation abilities.

44

4. Optimal Divergence Diversity for SNMF with
spectrogram restoration
4.1 Introduction
In the previous section, I revealed the mechanism of optimal divergence shift in the
SNMF methods. This divergence shift is due to the trade-oﬀ between separationand extrapolation abilities. The optimal divergence for SNMF with spectrogram
restoration depends on the rate of spectral chasms in each time frame of thespectrogram obtained by preceding directional clustering. Therefore, the optimal
divergence temporally ﬂuctuates because the spatial condition is not consistentin the general music signal, and the divergence of SNMF should be changed ineach time frame automatically. To solve this problem, in this section, I propose
a new scheme for frame-wise divergence selection to separate the target signal
using optimal divergence.
First, I describe about a new scheme of optimal divergence diversity andderive update rules of the proposed method in Sect. 4.2. Second, I conduct anexperiment to compare separation performance of the proposed method withdivergence diversity and the divergence-ﬁxed hybrid method in Sect. 4.3. Finally,
Sect. 4.4 concludes this section.

4.2 SNMF with spectrogram restoration based on multidivergence4.2.1 Divergence dependency on local chasms condition
The optimal divergence for SNMF with spectrogram restoration depends on therate of spectral chasms in each time frame of the spectrogram obtained by preceding directional clustering because of the trade-oﬀ between separation and extrapolation abilities. If there are many chasms in a frame of the binary-masked
spectrogram, SNMF is preferred to have high extrapolation ability. In contrast,
if the rate of chasms is low value, the separation ability is required rather thanthe extrapolation. Therefore, it is expected that EUC-distance should be usedin the frames that have many chasms, and KL-divergence should be used in the45Frequency

Binary-masked
spectrogramTime
Calculation of rate
ofCalculationchasms of rate
ofCalculationchasms of rate(EUC) Yesof chasmsNo(KL)Calculation of rate(EUC) Yes(KL)of chasms
No(EUC) Yes(KL)
No
(EUC) YesNo
EUC-distanceKL-divergence-

EUC-distanceKL-divergencebased
cost function based
cost function
EUC-distanceKL-divergencebased
cost function based
cost function
based cost function basedEUC-distancecost function
based cost function(KL)

KL-divergencebased cost function

SNMF with spectrogram restoration
Figure 26. Divergence diversity algorithm of proposed method.

other frames. To improve total separation performance of SNMF with spectrogram restoration for any types of input signals, we propose a new frame-wisedivergence switching method as described in the next section.
4.2.2 Cost function and update rules
Considering the above-mentioned divergence dependency on the local chasm condition, we propose to switch the divergence in each frame of the spectrogram according to the rate of chasms in each frame, rt , and a threshold value τ (0 ≤ τ ≤ 1),
where the rate of chasms rt can be calculated from the index matrix I. Figure 26
depicts an algorithm of the frame-wise divergence switching. This divergenceswitching method is implemented by switching the cost function in each frame,46

asJdiversity =

∑

Jt ,

(54)

t

∑
(EUC)

)

ω iω,t Dβ=2 (yω,t ∥sω,t∑ (EUC)
∑


(EUC)

gk,t )
i
D
(0∥
+
λ
ω,t
β

reg
k fω,kω
 + µ(EUC) ∥F (EUC)T H∥2(rt ≥ τ )Fr
Jt = ∑,(KL)
iω,t Dβ=1 (yω,t ∥sω,t )

ω


∑
∑ (KL)+ λ(KL) ω iω,t Dβreg (0∥ k fω,k gk,t ) + µ(KL) ∥F (KL)T H∥2(rt < τ )
Fr
∑
∑
(∗)
(∗)
fω,k gk,t +hω,n un,t ,sω,t =
rt =

(56)

n

k

∑

(55)

ω iω,t

Ω

,

(57)

(EUC)
where F (KL) (∈ RΩ×K(∈ RΩ×K≥ 0 ) and F≥ 0 ) are the supervised basis matrices trainedin advance using KL-divergence-based NMF and EUC-distance-based NMF, re(KL)(EUC)
spectively. Also, fω,k and fω,kare the entries of F (KL) and F (EUC) , respectively, µ(∗) and λ(∗) are the weighting parameters for each term, and ∗ ={KL, EUC}. The divergence is determined depending on rt and τ in each frame.
Therefore, this method can be considered as a frame-wise diversity of the divergence to achieve both of optimal separation and extrapolation.
The update rules based on (54) is obtained by the auxiliary function approach.
Similarly to Sect. 3.2.2, we can design the upper bound function J + using aux(∗)(∗)(∗)
iliary variables ζk,l,ω ≥ 0, κω,t,k ≥ 0, γω,t,l ≥ 0, ε1 ≥ 0, ε2 ≥ 0, and ξω,t ≥ 0 that47

satisfy

∑

∑ (∗)
∑
(∗)
ω ζk,l,ω = 1,k κω,t,k = 1,l γω,t,l = 1, and ε1 + ε2 = 1, as+
Jdiversity ≤ Jdiversity
=

∑

Jt+ ,

(58)

t

∑
( 2
)


ω iω,t yω,t + pω,t + 2qω,t



∑
(EUC)

(EUC)

+
λ

ω iω,t Rβreg


(EUC)


fω,k 2 h2ω,l
∑


(EUC)

+µ
(rt ≥ τ )

k,l,ω
(EUC)

ζk,l,ω
+)(,Jt ≤ Jt = ∑
∑
(KL)

i
−y
κ
γ
Q
+
c

ω,t
ω ω,t
k,l ω,t,k ω,t,l



∑

(KL)


+ λ(KL) ω iω,t Rβreg



(KL)


fω,k 2 h2ω,l
∑

(KL)

(rt < τ )
+µ

k,l,ω

(KL)
ζk,l,ω

(59)

wherepω,t =(EUC) 2 2
∑ fω,kgk,t(EUC)

κω,t,kk(qω,t =∑(EUC)
fω,k gk,t+∑ hω,l ul,tγω,t,l

l

)(

∑

,

(60)
)

hω,l ul,tlk− yω,t

∑

(EUC)

fω,k

gk,t − yω,t∑k) ξ (∗) βreg
(

ω,t

(∗)
(∗)
(∗) βreg −1 ∑k fω,k gk,t − ξω,t +
ξω,t
βreg
(∗)
( (∗) )βreg
Rβreg =
1 ∑ (∗)
fω,k gk,t


κ

ω,t,k
k
β
(∗)
reg
κω,t,khω,l ul,t ,l(61)(βreg < 1),(62)

(1 ≤ βreg )Q = ε1 log Φ + ε2 log Ψ,()∑ (KL)(KL)c = − yω,t
κω,t,k γω,t,l log κω,t,k γω,t,l + ε1 log ε1 + ε2 log ε2 ,

(63)
(64)

k,l
(KL)
Φ = γω,t,l fω,k gk,t ,(KL)Ψ = κω,t,k hω,l ul,t .

(65)
(66)

The equality in (59) holds if and only if the auxiliary variables are set as (29)48and as follows:(∗)
fω,k hω,l(∗)ζk,l,ω = ∑ (∗),ω ′ fω ′ ,k hω ′ ,l(∗)
fω,k gk,t(∗),κω,t,k = ∑ (∗)
′ ,t
g
f
′
′
k
k ω,k

(67)

(68)

Φ
,
Φ+Ψ
Ψ
ε2 =
,
Φ+Ψ
∑
(∗)
(∗)
ξω,t =
fω,k gk,t .
ε1 =

(69)
(70)
(71)

k

The update rules are obtained from the derivative of the upper bound function(58) w.r.t. each objective variable and substitution of the equality condition (67)–(71), as

∑
(EUC)


ω iω,t yω,t fω,k

gk,t · ∑

(
)βreg

∑

(EUC)
(EUC) (EUC)(EUC) ∑

(EUC)
′,t

f
g
i
f
s
+
λ
i
f
′
′
k
ω,t
ω,t
ω,t
ω,k
ω,k

k ω,k
ω
ω




(rt ≥ τ )gk,t ←,∑(KL) (KL) −1iω,t yω,t fω,k sω,tωgk,t ·

(
)βreg

∑
∑

(KL)
(KL)
(KL) ∑

(KL)

′
f
g
i
f
+
λ
i
f

k′ ω,k′ k ,t
ω ω,t ω,k
ω ω,t ω,k(rt < τ )(72)∑
iω,t yω,t ul,t Nω,thω,l ← hω,l · ∑ t,t iω,t ul,t Dω,t + Pω,l∑iω,t yω,t hω,lul,t · ∑ ω(EUC)ω iω,t hω,l sω,tul,t ←∑(EUC) −1ω iω,t yω,t hω,l sω,t∑ul,t ·
ω iω,t hω,l(73)
(rt ≥ τ ),(rt < τ )

49

(74)

where Nω,t , Dω,t , and Pω,l are given by1Nω,t =
s(KL) −1ω,ts(EUC)ω,tDω,t =1µ(EUC) ∑ f (EUC) ∑ ′ f (EUC)
hω′,l
k ω,k
ω ω ′,kPω,l =
∑
∑
(KL)
(KL)
µ(KL)
f
hω′,l
′ f ′
k

ω,k

ω

ω ,k

(rt ≥ τ )

,

(75)

,

(76)

.

(77)

(rt < τ )(rt ≥ τ )(rt < τ )(rt ≥ τ )(rt < τ )In total, the update rules of proposed SNMF with frame-wise divergence diversity
are deﬁned as (72)–(74).

4.3 Evaluation experiment4.3.1 Experimental conditions
To conﬁrm the eﬀectiveness of the proposed hybrid method with divergence diversity, I compared ﬁve methods, namely, PSNMF [22] based on KL-divergence,
PSNMF based on EUC-distance, the hybrid method using SNMF with spectrogram restoration based on only EUC-distance, the hybrid method including
SNMF with spectrogram restoration based on only KL-divergence, and the proposed hybrid method that switches the divergence to the optimal one framewisely.
In this experiment, I used stereo signals containing four melody parts (depicted inFig. 27) with three compositions (C1–C3) of instruments shown in Table 1. Similarly to Sect. 3.4.1, these signals were artiﬁcially generated by a MIDI synthesizer,
and the observed signals Y were produced by mixing four sources with the same
power. Also, these stereo signals were mixed down to a monaural format only
when we evaluate the separation accuracy of PSNMF. The sources were mixed as
Fig. 18, where the target source was always located in the center direction withanother interfering source.
I prepared four spatially diﬀerent dataset patterns of the observed signals,
SP1–SP4, as shown in Table 2. In the hybrid method, many chasms were produced by directional clustering in the measures where θ = 45◦ compared with50aaaaaaanaaaaaaaaaaaaaaaaaaaaaaagaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaa
angianag
¶ ED?E E E DE E E EE E EE E DD ED?E E E ED E E EE E EE E DD¶
¶ EE E EEE E E E E D EE E EEE E E E E D´ E EEEE E D E EEEE E D

Figure 27. Scores of each part. The observed signal consists of four measures.
Table 2. Spatial conditions of each dataset
Spatialpattern
SP1
SP2
SP3
SP4

Measure1stθ = 45◦
θ = 45◦
θ = 45◦
θ = 45◦2ndθ = 0◦
θ = 45◦
θ = 45◦
θ = 45◦3rd
θ = 0◦
θ = 0◦
θ = 45◦
θ = 45◦4thθ = 0◦
θ = 0◦
θ = 0◦
θ = 45◦those of θ = 0◦ . Therefore, we can expect that EUC-distance-based hybrid
method is suitable for SP4 rather than the dataset of SP1.
In addition, I used the same MIDI sounds of the target instruments as supervision for a priori training. The threshold value for the divergence diversity, τ ,was set to 20%. The other experimental conditions were the same as those in
Sect. 3.4.1.
4.3.2 Experimental results
Figure 28 shows the average SDR, SIR, and SAR scores for each method andeach dataset pattern, where four instruments are shuﬄed with 12 combinations
in each of compositions C1–C3. Therefore, these results are the averages of 36 input signals. In addition, the SDR scores of PSNMF are the same for any datasets
51because the input signals for PSNMF are mixed down to a monaural format.
From this result, KL-divergence-based hybrid method achieves high separation
accuracy for the dataset of spatial patterns SP1, SP2, and SP3 because thesesignals do not have much spectral chasms. On the other hand, EUC-divergencebased hybrid method achieves high separation accuracy for SP4. This dataset
has many spectral chasms because the signals are always mixed with a widepanning (θ = 45◦ ), which yields many chasms, and the extrapolation ability ishighly required. In addition, the proposed hybrid method with frame-wise divergence diversity can always achieve better separation for any datasets regardlessof the condition whether many chasms exist or not. This is because the proposed
method provides the appropriate diversity of the divergence and can automatically apply the optimal divergence to each time frame.

52KL-divergence-based PSNMF
EUC-distance-based PSNMF
14
12
10
8
6
4
2 KL-divergence-based hybrid method
0EUC-distance-based
hybrid
SP1
SP2
SP3 method
SP4
Proposed hybrid method with diversity

14
12

(a)

SDR [dB]

10
8
6
4
2
0
SP1

SP2

SP3

SP4

Spatial patterns
20(b)

SIR [dB]

15

10

5

0
SP1

SP2

SP3

SP4

Spatial patterns
10(c)

SAR [dB]

8
6
4
2
0
SP1

SP2

SP3

SP4

Spatial patterns

Figure 28. Average scores of each method and each spatial condition.: (a) shows
SDR, (b) shows SIR, and (c) shows SAR for conventional and proposed methods.
53

4.4 Conclusion
In this section, ﬁrst, I proposed a new divergence selection method as a improvement scheme of SNMF with spectrogram restoration and its hybrid method
to separate the target signal using optimal divergence. The proposed method
switches the optimal divergence in each time frame using a threshold value forthe rate of the chasms to separate and extrapolate the target signal with highaccuracy.
Second, I derived the update rules of proposed SNMF with divergence diversity. These update rules can switch the divergences framewisely and optimize thevariable matrices, simultaneously.
Finally, I conducted the evaluation experiment to conﬁrm the eﬃcacy of thedivergence diversity. Experimental results show that the proposed hybrid method
can always achieve high separation performance under any spatial conditions.

54

5. Conclusion
5.1 Summary of thesis
In this thesis, I proposed a new multichannel signal separation method, i.e., ahybrid method that concatenates SNMF with spectrogram restoration after directional clustering, which reconstructs the target components lost by preceding
binary masking. From theoretical analysis based on the generation model of thesignal, it was revealed that the optimal divergence in SNMF with spectrogram
restoration, which attempts to ﬁt the trained bases using spectral components
except for the chasms, is shifted to an anti-sparse criterion rather than KLdivergence because of the trade-oﬀ between separation and extrapolation abilities. Based on this ﬁnding, I also proposed an improved hybrid method thatswitches the divergence to the optimal one framewisely. According to the resultsof the evaluation experiments with artiﬁcial and real-recorded signals, the proposed method is advantageous to the conventional methods in terms of robustness
and separation performance.
In Sect. 3, I proposed a new SNMF with spectrogram restoration and its
hybrid method for multichannel signal separation. By utilizing the index information generated from binary masking, the proposed SNMF regards the spectral
chasms as unseen observations and ﬁnally reconstructs the target signal components via spectrum extrapolation using supervised bases. In other words, thisSNMF can be categorized as a inpainting-based method because the deteriorated
spectrogram resulting from the preceding binary masking can be recovered by
the supervised basis extrapolation. In addition, a regularization term is addedin the cost function to avoid extrapolation error. The theoretical analysis of thebasis extrapolation ability revealed the mechanism of the marked shift of optimal
divergence in SNMF with spectrogram restoration and the trade-oﬀ between separation and extrapolation abilities owing to the diﬀerence of sparseness in eachdivergence. Furthermore, the eﬀectiveness of the proposed hybrid method was
conﬁrmed by the evaluation experiments with artiﬁcial and real-recorded signals.
In Section 4, I proposed an improved hybrid method. This method switches
the divergence in each frame of the spectrogram according to the rate of chasms
in each frame and a threshold value. Therefore, this method can be considered55

as a frame-wise diversity of the divergence to achieve both optimal separationand extrapolation. Experimental results show that the proposed hybrid methodwith divergence diversity can always achieve high separation performance under
all spatial conditions.

5.2 Future work
The following points still remain to be investigated or clariﬁed.
• I have not analyzed other types of divergence, such as the divergence between KL-divergence and EUC-distance. I speculate that the optimal divergence strictly depends on the balance between inherent sparseness ofthe signal and the rate of spectral chasms generated by the preceding binary masking. Therefore, mathematical analysis of the relation between
the divergence and sparseness of the signal is an important future task.
• The proposed SNMF with spectrogram restoration can be used as a postﬁlter for target source extraction because it reconstructs the target components lost by the preceding process using supervision. For example, we caniterate the proposed hybrid method to increase the extraction performanceof the target source. The separation accuracy of the iteration method using
SNMF with spectrogram restoration as a postﬁlter should be analyzed.

56

Acknowledgements
This thesis is a summary of two years of study carried out at Graduate School ofInformation Science, Nara Institute of Science and Technology, Japan.
I would like to express my sincere thanks to Emeritus Professor Kiyohiro
Shikano and Professor Satoshi Nakamura of Nara Institute of Science and Technology, my thesis advisers, for their valuable guidance and constant encouragement.
I would also like to express my appreciation to Professor Kazushi Ikeda of
Nara Institute of Science and Technology, a member of the thesis committee, for
his valuable comments on the thesis.
I would especially like to express my deep gratitude to Associate Professor
Hiroshi Saruwatari of Nara Institute of Science and Technology for his continuous
teaching and essential advice on both technical and non technical issues. This
work could not have been accomplished without his well-directed advice, helpful
suggestions, and fruitful discussions with him. I have learned many valuable
aspects of being a researcher from his attitude toward study and have always
enjoyed conducting research with him.
I would like to express my gratitude to Visiting Associate Professor Hirokazu
Kameoka of The University of Tokyo and Associate Professor Nobutaka Onoof National Institute of Informatics, for their valuable suggestions, as well as
Associate Professor Tomoki Toda and Assistant Professors Hiromichi Kawanami,
Sakriani Sakti, and Graham Neubig of Nara Institute of Science and Technology,
and Assistant Professor Sunao Hara of Okayama University for their instructive
comments.
This work could not have been achieved without the collaboration of many
researchers. I especially thank Dr. Yu Takahashi, and Dr. Kazunobu Kondo,
researchers at Yamaha Corporation, for their beneﬁcial and valuable comments.
Many staﬀ and member s of my research group have supported me in carrying out experiments and writing this thesis at Nara Institute of Science andTechnology; I would especially like to express my appreciation of the valuable
discussions with them on technical issues of speech signal processing and digital
signal processing, and their provision of a comfortable computing environment in
our laboratory. I also wish to express my deep gratitude to Ms. Toshie Nobori
57and Ms. Manami Matsuda, secretaries at our laboratory, for their kind help andsupport in all aspects of my research.
I appreciate the opportunity to have studied with all the students in our
research group at Nara Institute of Science and Technology. I thank Mr. Ryoichi
Miyazaki and Ms. Fine Dwinita April, who are Ph.D. candidates at Nara Instituteof Science and Technology, for useful discussions on this work.
Finally, I wish to thank all members of my family for their support over many
years.

58

References[1] N. Kamado, H. Nawata, H. Saruwatari, K. Shikano, “Interactive controller
for audio object localization based on spatial representative vector operation,” Proc. International Workshop on Acoustic Echo and Noise Control,
2010.
[2] H. Nawata, N. Kamado, H. Saruwatari, K. Shikano, “Automatic musicalthumbnailing based on audio object localization and its evaluation,” Proc.
IEEE International Conference on Acoustics, Speech and Signal Processing,pp.41-44, 2011.
[3] A. J. Berkhout, “A holographic approach to acoustic control,” Journal of
Audio Engineering Society, vol.36, no.12, pp.977–995, 1988.
[4] H. Kirchhoﬀ, S. Dixon, A. Klapuri, “Missing template estimation for userassisted music transcription,” Proc. IEEE International Conference on
Acoustics, Speech and Signal Processing, pp.26–30, 2013.
[5] E. Benetos, S. Dixon, D. Giannoulis, H. Kirchhoﬀ, A. Klapuri, “Automaticmusic transcription: Breaking the glass ceiling,” Proc. 13th International
Conference on Music Information Retrieval, 2012.
[6] N. Ono, K. Miyamoto, J. Le Roux, H. Kameoka, S. Sagayama, “Separationof a monaural audio signal into harmonic/percussive components by complementary diﬀusion on spectrogram,” Proc. 16th European Signal Processing
Conference, 2008.
[7] A. Mesaros, T. Virtanen, A. Klapuri, “Singer identiﬁcation in polyphonicmusic using vocal separation and pattern recognition methods,” Proc. 8thInternational Conference on Music Information Retrieval, pp.375–378, 2007.
[8] M. Every, J. Szymanski, “A spectral-ﬁltering approach to music signal separation,” Proc. 7th International Conference on Digital Audio Eﬀects, pp.197–
200, 2004.

59

[9] L. Atlas, C. Janssen, “Coherent modulation spectral ﬁltering for singlechannel music source separation,” Proc. IEEE International Conference on
Acoustics, Speech and Signal Processing, pp.461–464, 2005.
[10] Z. Duan, Y. Zhang, C. Zhang, Z. Shi, “Unsupervised single-channel musicsource separation by average harmonic structure modeling,” IEEE Trans.
on Audio, Speech, and Language Processing, vol.16, no.4, pp.766–778, 2008.
[11] P. Comon, “Independent component analysis, a new concept?,” Signal processing, vol.36, no.3, pp.287–314, 1994.
[12] H. Sawada, R. Mukai, S. Araki, S. Makino, “A robust and precise method
for solving the permutation problem of frequency-domain blind source separation,” IEEE Trans. on Speech and Audio Processing, vol.12, no.5, pp.530–
538, 2004.
[13] M. Joho, H. Mathis, R. H. Lambert, “Overdetermined blind source separation: Using more sensors than source signals in a noisy mixture,” Proc.
2nd International Conference on Independent Component Analysis and Blind
Signal Separation, pp.81–86, 2000.
[14] D. D. Lee, H. S. Seung, “Algorithms for non-negative matrix factorization,”
Proc. Advances in Neural Information Processing Systems, vol.13, pp.556–
562, 2001.
[15] T. Virtanen, “Monaural sound source separation by nonnegative matrix factorization with temporal continuity and sparseness criteria,” IEEE Trans.
Audio, Speech and Language Processing, vol.15, no.3, pp.1066–1074, 2007.
[16] S. A. Raczynski, N. Ono, S. Sagayama, “Multipitch analysis with harmonic
nonnegative matrix approximation,” Proc. 8th International Conference onMusic Information Retrieval, pp.381–386, 2007.
[17] A. Ozerov, C. Fevotte, M. Charbit, “Factorial scaled hidden Markov model
for polyphonic audio representation and source separation,” Proc. IEEE
Workshop on Applications of Signal Processing to Audio and Acoustics,
pp.121–124, 2009.
60

[18] W. Wang, A. Cichocki, J. A. Chambers, “A multiplicative algorithm for convolutive non-negative matrix factorization based on squared Euclidean distance,” Signal Processing, vol.57, no.7, pp.2858–2864, 2009.
[19] H. Kameoka, M. Nakano, K. Ochiai, Y. Imoto, K. Kashino, S. Sagayama,“Constrained and regularized variants of non-negative matrix factorization
incorporating music-speciﬁc constraints,” Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, pp.5365–5368, 2012.
[20] P. Smaragdis, B. Raj, M. Shashanka, “Supervised and semi-supervised separation of sounds from single-channel mixtures,” Proc. 7th International Conference on Independent Component Analysis and Signal Separation, pp.414–
421, 2007.
[21] K. Yagi, Y. Takahashi, H. Saruwatari, K. Shikano, K. Kondo, “Music signal
separation by orthogonality and maximum-distance constrained nonnegative
matrix factorization with target signal information,” Proc. AES 45th Conference on Applications of Time-Frequency Processing in Audio, 2012.
[22] D. Kitamura, H. Saruwatari, K. Shikano, K. Kondo, Y. Takahashi, “Robust
music signal separation based on supervised nonnegative matrix factorization
with prevention of basis sharing,” Proc. IEEE International Symposium on
Signal Processing and Information Technology, 2013.
[23] D. Kitamura, H. Saruwatari, K. Shikano, K. Kondo, Y. Takahashi, “Musicsignal separation by supervised nonnegative matrix factorization with basis
deformation,” Proc. IEEE 18th International Conference on Digital Signal
Processing, T3P(C)-1, 2013.
[24] A. Ozerov, C. Fevotte, “Multichannel nonnegative matrix factorization in
convolutive mixtures for audio source separation,” IEEE Trans. Audio,
Speech and Language Processing, vol.18, no.3, pp.550–563, 2010.
[25] H. Sawada, H. Kameoka, S. Araki, N. Ueda, “Eﬃcient algorithms for multichannel extensions of Itakura-Saito nonnegative matrix factorization,” Proc.
IEEE International Conference on Acoustics, Speech and Signal Processing,
pp.261–264, 2012.
61

[26] S. Araki, H. Sawada, R. Mukai, S. Makino, “Underdetermined blind sparsesource separation for arbitrarily arranged multiple sensors,” Signal Processing, vol.87, no.8, pp.1833–1847, 2007.
[27] S. Miyabe, K. Masatoki, H. Saruwatari, K. Shikano, T. Nomura, “Temporal quantization of spatial information using directional clustering for multichannel audio coding,” Proc. IEEE Workshop on Applications of Signal
Processing to Audio and Acoustics, pp.261–264, 2009.
[28] Y. Li, D. Wang, “On the optimality of ideal binary time-frequency masks,”
Speech Communication, vol.51, no.3, pp.230–239, 2009.
[29] Y. Iwao, H. Saruwatari, K. Shikano, K. Kondo, Y. Takahashi, “Stereo musicsignal separation combining directional clustering and nonnegative matrix
factorization,” Proc. IEEE International Symposium on Signal Processing
and Information Technology, 2012.
[30] P. Smaragdis, B. Raj, “Example-driven bandwidth expansion,” Proc. IEEE
Workshop on Applications of Signal Processing to Audio and Acoustics,
pp.135–138, 2007.
[31] D. FitzGerald, M. Cranitch, E. Coyle, “On the use of the beta divergencefor musical source separation,” Proc. Irish Signals and Systems Conference,
2009.
[32] S. Eguchi, Y. Kano, “Robustifying maximum likefood estimation,” Technical
Report of Institute of Statistical Mathematics, 2001.
[33] M. Nakano, H. Kameoka, J. Le Roux, Y. Kitano, N. Ono, S. Sagayama,“Convergence-guaranteed multiplicative algorithms for non-negative matrix
factorization with beta-divergence,” Proc. IEEE International Workshop on
Machine Learning for Signal Processing, 2010.
[34] A. T. Cemgil, “Bayesian inference for nonnegative matrix factorisation models,” Computational Intelligence and Neuroscience, vol.2009, p.1–17, 2009.

62

[35] E. Vincent, R. Gribonval, C. Fevotte, “Performance measurement in blindaudio source separation,” IEEE Trans. Audio, Speech and Language Processing, vol.14, no.4, pp.1462–1469, 2006.

63

List of Publications
Jornal Papars
1. D. Kitamura, H. Saruwatari, K. Yagi, K. Shikano, Y. Takahashi, and K. Kondo,“Music signal separation based on supervised nonnegative matrix factorization with orthogonality and maximum-divergence penalties,” IEICE Transactions on Fundamentals of Electronics, Communications and Computer
Sciences, (in printing)Peer Reviewed International Conference Proceedings
1. D. Kitamura, H. Saruwatari, K. Shikano, K. Kondo, and Y. Takahashi,“Superresolution-based stereo signal separation with regularization of supervised basis extrapolation,” Proc. 5th International Conference on 3D
Systems and Applications 2013, June S10-4, 2013.
2. D. Kitamura, H. Saruwatari, Y. Iwao, K. Shikano, K. Kondo, and Y. Takahashi, “Superresolution-based stereo signal separation via supervised nonnegative matrix factorization,” Proc. 18th International Conference on Digital Signal Processing 2013, July T3C-2, 2013.
3. D. Kitamura, H. Saruwatari, K. Shikano, K. Kondo, and Y. Takahashi,“Music signal separation by supervised nonnegative matrix factorization
with basis deformation,” Proc. 18th International Conference on Digital
Signal Processing 2013, July T3P(C)-1, 2013.
4. D. Kitamura, H. Saruwatari, K. Yagi, K. Shikano, Y. Takahashi, and K. Kondo,“Robust music signal separation based on supervised nonnegative matrix
factorization with prevention of basis sharing,” Proc. IEEE International
Symposium on Signal Processing and Information Technology 2013, December 2013.
5. D. Kitamura, H. Saruwatari, S. Nakamura, Y. Takahashi, K. Kondo, and
H. Kameoka, “Online divergence switching for superresolution-based nonnegative matrix factorization,” Proc. 2014 RISP International Workshop
64

on Nonlinear Circuits, Communications and Signal Processing, pp.485–488,
March 2014.
6. T. Miyauchi, D. Kitamura, H. Saruwatari, S. Nakamura, “Depth estimationof sound images using directional clustering and activation-shared nonnegative matrix factorization,” Proc. 2014 RISP International Workshop on
Nonlinear Circuits, Communications and Signal Processing, pp.437–440,
March 2014.
7. Y. murota, D. Kitamura, S. Nakai, H. Saruwatari, S. Nakamura, Y. Takahashi, and K. Kondo, “Music signal separation based on bayesian spectral
amplitude estimator with automatic target prior adaptation,” Proc. IEEE
International Conference on Acoustics, Speech and Signal Processing, May2014 (in printing).

Technical Reports
1. D. Kitamura, H. Saruwatari, K. Shikano, K. Kondo, and Y. Takahashi,“Signal separation for real instruments based on supervised NMF with basis deformation,” IEICE Technical Report, EA2012-121, vol.112, no.338,
pp.13–18, March 2013 (in Japanese).
2. D. Kitamura, H. Saruwatari, K. Shikano, K. Kondo, and Y. Takahashi,“Importance of regularization in superresolution-based multichannel signal
separation with nonnegative matrix factorization,” 99th IPSJ Special Interest Group on Music and Computer, vol.2013-MUS-99, no.14, pp.1–6, May2013.
3. D. Kitamura, H. Saruwatari, K. Shikano, K. Kondo, and Y. Takahashi,“Study on optimal divergence for superresolution-based supervised nonnegative matrix factorization,” IEICE Technical Report, EA2013-14, vol.113,
no.27, pp.79–84, May 2013.
4. D. Kitamura, H. Saruwatari, K. Yagi, K. Shikano, Y. Takahashi, and K. Kondo,“Music signal separation using supervised nonnegative matrix factorization

65

with orthogonality and maximum-divergence penalties,” 28th Signal Processing Symposium, C1-4, pp.539–544, November 2013.
5. Y. Murota, D. Kitamura, S. Nakai, H. Saruwatari, S. Nakamura, Y. Takahashi, and K. Kondo, “Postﬁlter-based nonnegative matrix factorization
with statistical model parameter estimation,” IEICE Technical Report, EA2013116, vol.113, no.413, pp.75–80, January 2014.
6. T. Miyauchi, D. Kitamura, H. Saruwatari, and S. Nakamura, “Depth estimation of sound images in mixed source using activation-shared nonnegative matrix factorization,” IEICE Technical Report, EA2013-117, vol.113,
no.413, pp.81–86, January 2014 (in Japanese).

Domestic Meetings
1. D. Kitamura, H. Saruwatari, K. Shikano, K. Kondo, and Y. Takahashi,“Evaluation of separation accuracy for various real instruments based onsupervised NMF with basis deformation,” 2013 Spring Meeting of AcousticalSociety of Japan, 3-1-11, pp.1057-1060, March 2013.
2. D. Kitamura, H. Saruwatari, S. Nakamura, K. Kondo, and Y. Takahashi,“Divergence optimization based on trade-oﬀ between separation and extrapolation abilities in superresolution-based nonnegative matrix factorization,”2013 Autumn Meeting of Acoustical Society of Japan, 1-1-6, pp.583-586,
September 2013.
3. D. Kitamura, H. Saruwatari, S. Nakamura, K. Kondo, Y. Takahashi, and
H. Kameoka, “Optimal divergence diversity for superresolution-based nonnegative matrix factorization,” 2014 Spring Meeting of Acoustical Societyof Japan, 3-2-9, March 2014.
4. T. Miyauchi, D. Kitamura, H. Saruwatari, and S. Nakamura, “Automatic
depth estimation of sound images using directional clutering and nonnegative matrix factorization,” 2013 Autumn Meeting of Acoustical Society ofJapan, 2-1-19, pp.673–676, September 2013 (in Japanese).

66

5. T. Miyauchi, D. Kitamura, H. Saruwatari, and S. Nakamura, “Depth estimation of sound images in mixed source using activation-shared nonnegative
matrix factorization,” 2014 Spring Meeting of Acoustical Society of Japan,
1-1-5, March 2014 (in Japanese).
6. Y. Murota, D. Kitamura, S. Nakai, H. Saruwatari, S. Nakamura, Y. Takahashi, and K. Kondo, “Experimental evaluation of postﬁlter-based nonnegative matrix factorization with statistical model parameter estimation,”
2014 Spring Meeting of Acoustical Society of Japan, 2-1-5, March 2014.

Awards
1. Student Award of Acoustical Society of Japan, D. Kitamura, September
2013.
2. SIP Young Researcher’s Award of IEICE Technical Group on Signal Processing, D. Kitamura, February 2014.
3. Student Paper Award of 2014 RISP International Workshop on Nonlinear
Circuits, Communications and Signal Processing, D. Kitamura, H. Saruwatari,
S. Nakamura, K. Kondo, Y. Takahashi, and H. Kameoka, March 2014.

67