VOL. E97-A NO. 5
MAY 2014
The usage of this PDF file must comply with the IEICE Provisions
on Copyright.
The author(s) can distribute this PDF file for research andeducational (nonprofit) purposes only.
Distribution by anyone other than the author(s) is prohibited.

IEICE TRANS. FUNDAMENTALS, VOL.E97–A, NO.5 MAY 2014

1113

LETTER

Music Signal Separation Based on Supervised Nonnegative Matrix
Factorization with Orthogonality and Maximum-Divergence
Penalties
Daichi KITAMURA†a) , Student Member, Hiroshi SARUWATARI† , Member, Kosuke YAGI† , Nonmember,
Kiyohiro SHIKANO† , Fellow, Yu TAKAHASHI†† , Nonmember, and Kazunobu KONDO†† , Member

SUMMARY
In this letter, we address monaural source separation basedon supervised nonnegative matrix factorization (SNMF) and propose a newpenalized SNMF. Conventional SNMF often degrades the separation performance owing to the basis-sharing problem. Our penalized SNMF forcesnontarget bases to become diﬀerent from the target bases, which increases
the separated sound quality.
key words: music signal separation, nonnegative matrix factorization, supervised method

1.

Introduction

In this study, we address monaural music signal separation using nonnegative matrix factorization (NMF) [1]–[3],which decomposes an observed spectrogram into a basis
matrix and an activation matrix. The basis matrix involves
frequently appearing spectral patterns in the observed spectrogram as the column vectors, and the activation matrix involves the time-varying gain corresponding to each basis. In
particular, supervised NMF (SNMF) [4] has been proposed,
which utilizes some sound samples of a target signal as supervision for a priori training. In SNMF, a spectrogram ofthe supervision sound is decomposed by conventional NMF
in a training process to generate the supervised basis matrix.
In the separation process, an observed spectrogram that consists of target and interference sources is decomposed using
the supervised bases and other bases. Finally, the separated
target signal can be reconstructed from the supervised bases
and their activation.
Conventional SNMF incurs a risk of degrading the separation performance owing to the simultaneous generationof similar spectral patterns in the supervised bases and otherbases. Here, we explain this phenomenon via a simplified example. Let X be an observed nonnegative matrix
whose rank is one. We can represent X using a (nonnegative
column) supervised basis vector a1 and the corresponding(nonnegative column) activation vector b as X = a1 bT +a2 0T ,where a2 is an arbitrary basis vector and 0 is a zero vector. If a2 is identical to a1 , we can also represent X using
Manuscript received September 3, 2013.
Manuscript revised January 11, 2014.
†
The authors are with Nara Institute of Science and Technology, Ikoma-shi, 630-0192 Japan.
††
The authors are with Research & Development, Yamaha Corporation, Iwata-shi, 438-0192 Japan.
a) E-mail: daichi-k@is.naist.jp
DOI: 10.1587/transfun.E97.A.1113

other activation vectors b1 and b2 as X = a1 bT1 +a2 bT2 , whereb = b1 + b2 ; the original activation vector b is split into b1
and b2 , and then the separated signal a1 bT1 is distorted. In
SNMF, such basis sharing between the supervised bases andother bases often occurs. This is because the cost function
in NMF is defined as the divergence D (··) between the observed and reconstructed matrices, and unique decomposition is not guaranteed (D(Xa1 bT ) = D(Xa1 bT1 + a2 bT2 ) in
the above case).
To solve this problem, we propose a new penalized
SNMF (PSNMF), which employs a penalty term in the costfunction to force the other bases to become as diﬀerent as
possible from the supervised bases. In this study, we introduce two types of penalty term based on orthogonality anddivergence maximization, and we confirm their eﬃcacy via
experimental evaluations.
2.

Proposed PSNMF

2.1 Decomposition Model
The following equation represents the decomposition modelof PSNMF:
Y  FG + HU,

(1)

Ω×K
where Y(∈ RΩ×T
≥ 0 ) is an observed spectrogram, F(∈ R≥ 0 ) isa matrix that involves supervised spectral bases (dictionary)of the target source as column vectors, G(∈ R≥K×T0 ) is the activation matrix that corresponds to F, H(∈ RΩ×L
≥ 0 ) is anotherbasis matrix that ideally involves residual spectral patterns,
and U(∈ R≥L×T
0 ) is the activation matrix that corresponds to
H. Moreover, Ω is the number of frequency bins, T is thenumber of frames of the observed signal, K is the number ofsupervised bases, and L is the number of the other bases. In
PSNMF, the supervised basis matrix F is trained in advance
via a target sound sample. After fixing F, the matrices G,
H, and U are optimized. Hence, FG ideally represents thetarget source components and HU represents the other different components after decomposition.

2.2 Cost Functions
In this section, we propose two types of PSNMF algorithm.

c 2014 The Institute of Electronics, Information and Communication Engineers
Copyright IEICE TRANS. FUNDAMENTALS, VOL.E97–A, NO.5 MAY 2014

1114

Hereafter, we denote the entries of the nonnegative matrices
Y, F, G, H, and U as yω,t , fω,k , gk,t , hω,l , and ul,t , respectively. The cost function in NMF is defined as the arbitrary
divergence between Y and FG + HU. In this study, we propose the use of the following generalized cost function:
JNMF = Dβ (YFG + HU) ,where Dβ (··) indicates β-divergence [5], defined as
⎫
⎧
β
⎪
⎪
⎪
aβm,n bm,n aβ−1m,n ⎪⎬⎨ bm,n−Dβ (BA) =
+
,
⎪
⎪
⎪
⎭
⎩ β (β − 1)
β
β−1 ⎪

(2)

(3)

m,n

M×N

arg min FT H2Fr ,(4)

H

where the conditions ω fω,k = 1 and ω hω,l = 1 are applied, and  · Fr indicates the Frobenius norm. This minimization corresponds to the maximization of orthogonality
between F and H. The cost function with the orthogonality
penalty is given by
J1 = JNMF + μ1 FT H2Fr
⎛
⎞2
 ⎜⎜
⎟⎟
⎜
⎜
= JNMF + μ1
fω,k hω,l ⎟⎟⎟⎠ ,
⎜⎝
k,l

(5)

ω

where μ1 is the weighting parameter for the penalty term.
As another means for making H diﬀerent from F, themaximization of all divergence combinations between the
supervised bases in F and the other bases in H can be used,which is given by
arg max
Dβm fω,k hω,l ,(6)
H

In this section, we derive the update rules based on the costfunctions Eqs. (5) and (7), similarly to [6]. Since it is difficult to analytically derive the optimal G, H, and U, wedefine auxiliary functions J1+ and J2+ that represent the upper bounds of J1 and J2 , respectively. Here, we can rewrite
Eq. (2) as
⎫
⎧
β
⎪
⎪
⎪
zβω,t yω,t zβ−1⎬
⎨ yω,tω,t ⎪JNMF =
+
−
,
(8)
⎪
⎪
⎪
⎭
⎩ β (β − 1)
β
β−1 ⎪
ω,t

where A(∈ R ) and B(∈ R ) are matrices whose entries are am,n and bm,n , respectively. This generalized divergence is a family of cost functions parameterized by a single
shape parameter β that takes Itakura-Saito divergence (ISdivergence), generalized Kullback-Leibler divergence (KLdivergence), and Euclidean distance (EUC-distance) as special cases (β = 0, 1, and 2, respectively).
In PSNMF, to avoid the sharing of bases, we make H
as diﬀerent as possible from F. We impose the following
minimization in addition to the cost function:
M×N

2.3 Auxiliary Functions

where zω,t is defined asfω,k gk,t +hω,l ul,t .
zω,t =k(9)lFirst, we define the upper bound for the second term on
the right-hand side of Eq. (8). This term is convex for β ≥ 1
and concave for β < 1. If β satisfies β ≥ 1, the upper boundfunction Q(β)ω,t is defined using auxiliary variables αω,t,k ≥ 0,γω,t,l ≥ 0, η1 ≥ 0, and η2 ≥ 0 that satisfy k αω,t,k = 1, l γω,t,l =1, and η1 + η2 = 1. Applying Jensen’s inequality to this, we
have
⎧



 ⎫
⎪
⎪
zβω,t 1 ⎪fω,k gk,t β hω,l ul,t β ⎪⎨⎬≤ ⎪αω,t,k η1+ γω,t,l η2
⎪
⎪
⎪
⎩
⎭
β β k
αω,t,k η1
γ
η
ω,t,l
2
l
(β)

≡ Qω,t .

(10)

The equality in Eq. (10) holds if and only if the auxiliary
variables are set as follows: αω,t,k = fω,k gk,t / k fω,k gk ,t ,
(11)

 

(12)
γω,t,l = hω,l ul,t / l hω,l ul ,t , 
η1 = k fω,k gk,t / k fω,k gk ,t + l hω,l ul ,t ,
(13)

 

η2 = l hω,l ul,t / k fω,k gk ,t + l hω,l ul ,t .
(14)
(β)

If β satisfies β < 1, the upper bound function Rω,t is definedusing the auxiliary variable σω,t ≥ 0. Applying the tangent
line inequality to this, we haveβzβω,t σω,t(β)β−1 ≤ σω,t zω,t − σω,t +≡ Rω,t .
β
β

(15)

k,l,ω

where βm is the shape parameter of the divergence for thispenalty, and the conditions ω fω,k = 1 and ω hω,l = 1 areapplied. This maximization forces the other bases in H to
become diﬀerent from the target spectral patterns. The costfunction with the maximum-divergence penalty is given by⎞⎛⎜⎜⎜ 1 ⎟⎟⎟⎟⎜Dβ fω,k hω,l ⎟⎟⎠,
J2 = JNMF + μ2 exp ⎜⎜⎝−
(7)
λ k,l,ω mwhere μ2 and λ are the weighting and sensitivity parameters,
respectively. Here, exponentiation is applied to make thepenalty term nonnegative.

The equality in Eq. (15) holds if and only if the auxiliary
variable is set toσω,t =
fω,k gk ,t +hω,l ul ,t .
(16)
k

l

Second, we define the upper bound function for thethird term on the right-hand side of Eq. (8). This term is convex for β ≥ 2 and concave for β < 2. Similarly to Eqs. (10)and (15), we can derive the auxiliary function for the third
term of Eq. (8) as⎧ (β−1)⎪⎪zβ−1(β ≥ 2)⎨−Qω,t
ω,t
≤⎪
.
(17)
−
(β−1)
⎪
⎩
β−1
(β < 2)−Rω,t

LETTER

1115

Third, for the orthogonality penalty term in Eq. (5),the upper bound function P+ is defined using an auxiliary
variable δk,l,ω ≥ 0 that satisfies ω δk,l,ω = 1. Similarly to
Eq. (10), we obtain
⎛
⎞2
2 2
 ⎜⎜
 fω,k⎟⎟hω,l⎜μ1
fω,k hω,l ⎟⎟⎟⎠ ≤ μ1
≡ P+ ,
(18)
⎜⎜⎝
δ
k,l,ω
ω
k,l
k,l,ω
where the equality in Eq. (18) holds if and only if the auxiliary variable is set to

 

(19)
δk,l,ω = fω,k hω,l / ω fω ,k hω ,l .

⎧⎪1/(2 − β)⎪⎪⎪⎨ϕ(β) = ⎪1⎪⎪⎪⎩1/(β − 1)⎞⎛⎜⎜⎜ 1 

⎟⎟⎟⎟
+
+
⎜
J2 = JNMF + μ2 exp ⎜⎜⎝−
Dβm fω,k hω,l ⎟⎟⎠,
λ

⎛
⎞ϕ(β)
⎜⎜⎜ λ t yω,t ul,t zβ−2+μ2 hβω,lm −1Cβm ⎟⎟⎟ω,t⎟⎟⎟ , (29)hω,l ← hω,l ⎜⎜⎝⎜⎠βm −2
λ t ul,t zβ−1
ω,t +μ2 hω,l C βm k fω,k
where

⎞
⎛
⎜⎜⎜ 1 ⎟⎟⎟⎟⎜Cβm = exp ⎜⎜⎝−
Dβm fω,k hω,l ⎟⎟⎠.
λ

k,l,ω

where
+
=
JNMF




yβω,t
(β)
Sω,t ,
+
(β
β
−
1)
ω,t
ω,t

⎧ (β)
(β−1)
⎪
⎪
Rω,t − yω,t Qω,t
⎪
⎪
⎪
⎪
⎨ (β)
(β)
(β−1)
Sω,t = ⎪Qω,t − yω,t Qω,t
⎪
⎪
⎪
⎪
(β−1)
⎪
⎩Q(β)
ω,t − yω,t Rω,t(22)

(β < 1)(1 ≤ β ≤ 2) .

(30)

k,l,ω

(20)
(21)

(28)

Similarly to Eq. (27), we can obtain the update
rule of hω,l with the maximum-divergence penalty from
∂J2+ /∂hω,l = 0 as

Finally, using Eqs. (10), (15), (17), and (18), we candefine the upper bound functions J1+ and J2+ as+
J1+ = JNMF
+ P+ ,(β < 1)(1 ≤ β ≤ 2) .
(β > 2)The update rules of the activation matrices are obtained asfollows:⎛⎞ϕ(β)⎜⎜⎜ ω fω,k yω,t zβ−2
⎟⎟⎟
ω,t ⎟
⎜
(31)
gk,t ← gk,t ⎜⎝
⎟ ,
β−1 ⎠
ω fω,k zω,t⎛⎞ϕ(β)⎜⎜⎜ ω hω,l yω,t zβ−2
⎟⎟⎟
ω,t ⎟
⎜
(32)
ul,t ← ul,t ⎜⎝
⎟ .
β−1 ⎠
ω hω,l zω,t

(23)
3.

(β > 2)

Experiment for Artificial Signals

3.1 Experimental Conditions
2.4 Update Rules for PSNMF
The update rules with respect to each variable are determined by setting the gradient of the cost function to zero.
From ∂J1+ /∂hω,l = 0, we obtain2 fω,k


hω,l
(β)
(β)
= 0,
Vω,t,l − Wω,t,l + 2μ1
δk,l,ω
t
k

where

(24)

⎧

β−1
⎪
⎪
(β < 1)⎨σω,t ul,t(β)=⎪,
Vω,t,l
β−1⎪⎩hω,l γω,t,l η2 1−β uβl,t (β ≥ 1)⎧
2−β β−1
β−2 
⎪
ul,t
(β ≤ 2)⎪⎨yω,t hω,l γω,t,l η2(β)
Wω,t,l = ⎪.
⎪β−2⎩yω,t σ ul,t(β > 2)
ω,t

(25)
(26)

To confirm the eﬃcacy of PSNMF, we compared the applicability of PSNMF and conventional SNMF (μ1 = μ2 = 0 in
Eqs. (27) and (29)) to a separation task involving monaural multiple instrumental sources. We produced the four
melodies depicted in Fig. 1. These instrumental signals wereartificially generated by a MIDI synthesizer. As the supervision sound for a priori training, we used the same MIDI
sounds of the target instruments containing two octave notesthat cover all the notes of the target signal in the observed
signal. The spectrograms were computed using a 92-mslong rectangular window with a half-size shift. The numberof iterations for the training and separation processes was
500. Moreover, the number of supervised bases in F was
100 and the number of the other bases in H was 50. In thisexperiment, the parameters μ1 , μ2 , and λ were changed to

By solving Eq. (24) for hω,l assuming nonnegativity, andsubstituting Eqs. (11)–(14), (16), and (19) into the solution,
we can obtain the update rule of hω,l with the orthogonality
penalty as⎛⎜⎜hω,l ← hω,l ⎜⎜⎜⎝

β−2t yω,t ul,t zω,tβ−1
t ul,t zω,t + 2μ1

k fω,kω fω ,k hω ,l

⎞ϕ(β)
⎟⎟⎟
⎟⎟⎠ ,
(27)

where ϕ (β) is given by

Fig. 1

Scores of each instrument.

IEICE TRANS. FUNDAMENTALS, VOL.E97–A, NO.5 MAY 2014

1116

evaluate their dependence on the separation performance.
We conducted two experiments to consider two-source
and four-source cases. In the two-source case, the observed
signal Y was produced by mixing two sources selected fromfour instruments with the same power. Therefore we prepared 12 combinations of the observed signals. In the foursource case, we produced an observed signal that consistedof four instruments with the same power. Then we calculated the average evaluation scores for each combination of
the instruments.
In NMF decomposition, the parameter β of the divergence directly aﬀects the separation accuracy. Hence, weused three values, namely, β = 0 (IS-divergence), β = 1(KL-divergence), and β = 2 (EUC-distance). Also, we used
βm = 0, 1, and 2 for the maximum-divergence penalty.

the type of penalties, the deviation in 12 combinations of theobserved signals is within 4 dB. The other results of β = 0
and 2 showed the similar tendency to that of β = 1.
Next, Tables 1, 2, and 3 show the average scores of
SDR, SIR, and SAR in the two-source case for each valueof β. Here, the SDR values are the average of 12 combinations of the observed signals with optimization on the parameters μ1 , μ2 , and λ, and the SIR and SAR values are thecorresponding ones. Also, Tables 4, 5, and 6 show the average scores in the four-source case. From the SDR results,
we can confirm that conventional SNMF cannot achieve ahigh separation accuracy owing to basis sharing between
the supervised bases and other bases, whereas the proposed
methods can avoid this basis-sharing problem by using thepenalty terms. In addition, the performances of the orthog-

3.2 Experimental Results
We used the signal-to-distortion ratio (SDR), source-tointerference ratio (SIR), and sources-to-artifacts ratio (SAR)
defined in [7] as the evaluation scores. SDR indicates thequality of the separated target sound, SIR indicates the degree of separation between the target and other sounds, andSAR indicates the absence of artificial distortion. Therefore, SDR indicates the total evaluation score that involves
SIR and SAR.
First, we depict the dependence of SDR values on theparameters μ1 , μ2 , and λ in the two-source case, where only
β = 1 is selected owing to the limit of the space because KLdivergence-based NMF is often used for many signal separation tasks. Figure 2 shows the variation of SDR values of
PSNMF with the orthogonality penalty. We plot the average SDR values of 12 combinations of the observed signals
and its deviation in the error bar. From this result, we canconfirm that the separation performance improves with increasing the value of μ1 because of the prevention of basis
sharing, and high SDR values can be retained when μ1 is set
to be large enough. Figures 3–5 show the variation of SDR
values of PSNMF with the maximum-divergence penalty.
From these results, we can also prevent the basis-sharing
problem by setting the parameter μ2 to be large. However,the fluctuation exists in the average SDR along with μ2 ; itindicates a slight diﬃculty in optimizing the parameters in
the maximum-divergence penalty. In all cases regardless ofFig. 2 Variation of SDR values of PSNMF with orthogonality penalty
when β = 1 in two-source case. Circle and error bar represent average SDRof 12 combinations of observed signals and its deviation, respectively.

Fig. 3 Variation of SDR values of PSNMF with maximum-divergencepenalty when β = 1 and βm = 0 in two-source case: (a) λ = 105 and (b)λ = 106 . Circle and error bar represent average SDR of 12 combinations ofobserved signals and its deviation, respectively.

Fig. 4 Variation of SDR values of PSNMF with maximum-divergencepenalty when β = 1 and βm = 1 in two-source case: (a) λ = 1 and (b) λ = 10.
Circle and error bar represent average SDR of 12 combinations of observed
signals and its deviation, respectively.

LETTER

1117
Table 5

Average scores in four-source case of artificial signals (β = 1).

MethodSDR SIR SAR
Conventional SNMF
7.1 14.2 4.3
PSNMF with orthogonality penalty
10.8 14.1 10.6
PSNMF with maximum-divergence penalty (βm = 0) 11.1 13.8 10.9
PSNMF with maximum-divergence penalty (βm = 1) 9.7 12.2 9.9
PSNMF with maximum-divergence penalty (βm = 2) 10.1 12.0 11.8

Table 6

Average scores in four-source case of artificial signals (β = 2).

MethodSDR SIR SAR
Conventional SNMF
7.3 13.0 6.4
PSNMF with orthogonality penalty
9.6 11.8 11.4
PSNMF with maximum-divergence penalty (βm = 0) 10.3 12.1 12.6
PSNMF with maximum-divergence penalty (βm = 1) 9.2 12.1 11.3
PSNMF with maximum-divergence penalty (βm = 2) 8.6 11.1 11.3
Fig. 5 Variation of SDR values of PSNMF with maximum-divergencepenalty when β = 1 and βm = 2 in two-source case: (a) λ = 10−4 and (b)λ = 10−3 . Circle and error bar represent average SDR of 12 combinationsof observed signals and its deviation, respectively.

Table 1

Average scores in two-source case of artificial signals (β = 0).

MethodSDR SIR SAR
Conventional SNMF
6.1 18.5 1.9
PSNMF with orthogonality penalty
9.6 17.8 6.7
PSNMF with maximum-divergence penalty (βm = 0) 9.3 16.9 6.3
PSNMF with maximum-divergence penalty (βm = 1) 8.5 14.5 6.2
PSNMF with maximum-divergence penalty (βm = 2) 8.9 18.1 5.1

Table 2Average scores in two-source case of artificial signals (β = 1).

MethodSDR SIR SAR
Conventional SNMF
3.9 18.1 −0.2
PSNMF with orthogonality penalty
13.6 19.0 11.3
PSNMF with maximum-divergence penalty (βm = 0) 12.0 18.6 9.0
PSNMF with maximum-divergence penalty (βm = 1) 11.8 15.9 10.5
PSNMF with maximum-divergence penalty (βm = 2) 13.0 17.5 11.6

Table 3

Average scores in two-source case of artificial signals (β = 2).

MethodSDR SIR SAR
Conventional SNMF
6.7 18.4 3.4
PSNMF with orthogonality penalty
11.9 18.1 9.8
PSNMF with maximum-divergence penalty (βm = 0) 13.0 17.8 11.4
PSNMF with maximum-divergence penalty (βm = 1) 10.8 17.0 9.5
PSNMF with maximum-divergence penalty (βm = 2) 12.1 15.8 12.3

Table 4

Average scores in four-source case of artificial signals (β = 0).

MethodSDR SIR SAR
Conventional SNMF
6.9 14.9 3.6
PSNMF with orthogonality penalty
8.6 12.6 6.8
PSNMF with maximum-divergence penalty (βm = 0) 8.6 13.8 6.0
PSNMF with maximum-divergence penalty (βm = 1) 8.8 14.3 6.3
PSNMF with maximum-divergence penalty (βm = 2) 8.6 13.8 6.0

Fig. 6 Spectrograms of (a) observed signal consisting of cello and oboe,(b) oracle signal of target cello signal, (c) cello signal extracted by conventional SNMF, and (d) cello signal extracted by proposed PSNMF withorthogonality penalty.

that consists of cello and oboe signals. The signal extracted
by conventional SNMF loses some of the target spectra (seeFig. 6(c)) because of basis sharing, but the proposed method
extracts the target source with high accuracy (see Fig. 6(d)).
Finally, as another means of preventing the basis sharing problem, some people may guess that the orthogonality penalty on activations, GUT 2Fr , can also be introduced
instead of the proposed penalty, FT H2Fr . However, thispenalty term has a risk to force G to become 0, which yields
that the input matrix Y is represented using only the othermatrix HU. Indeed, for instance, the average SDR valueof KL-divergence-based PSNMF with this penalty term was−8.8 dB in two source case, and the output signal did notcontain the suﬃcient target components.
4.

onality and maximum-divergence penalties are roughly thesame.
Figure 6 shows sample spectrograms after the separation, which extracts the cello signal from the observed signal

Experiment for Real-Recorded Signals

4.1 Experimental Conditions
We also conducted another experiment using real-recorded

IEICE TRANS. FUNDAMENTALS, VOL.E97–A, NO.5 MAY 2014

1118
Table 7

Average scores in two-source case of recorded signals (β = 0).

MethodSDR SIR SAR
Conventional SNMF
7.5 19.6 3.5
PSNMF with orthogonality penalty
10.6 19.1 11.8
PSNMF with maximum-divergence penalty (βm = 0) 10.5 18.2 7.1
PSNMF with maximum-divergence penalty (βm = 1) 10.5 17.2 7.3
PSNMF with maximum-divergence penalty (βm = 2) 10.5 19.7 6.7

Table 8Average scores in two-source case of recorded signals (β = 1).

MethodSDR SIR SAR
Conventional SNMF
4.4 20.2 −0.1
PSNMF with orthogonality penalty
14.4 20.0 11.7
PSNMF with maximum-divergence penalty (βm = 0) 13.8 20.3 10.4
PSNMF with maximum-divergence penalty (βm = 1) 13.6 18.2 11.0
PSNMF with maximum-divergence penalty (βm = 2) 14.4 18.5 12.4

Table 9Average scores in two-source case of recorded signals (β = 2).

MethodSDR SIR SAR
Conventional SNMF
7.5 19.6 3.5
PSNMF with orthogonality penalty
11.9 19.0 9.2
PSNMF with maximum-divergence penalty (βm = 0) 14.7 19.5 12.4
PSNMF with maximum-divergence penalty (βm = 1) 11.9 18.2 9.9
PSNMF with maximum-divergence penalty (βm = 2) 13.2 16.3 12.8

Table 10

Average scores in four-source case of recorded signals (β = 0).

MethodSDR SIR SAR
Conventional SNMF
8.3 17.4 4.6
PSNMF with orthogonality penalty
11.3 16.7 8.8
PSNMF with maximum-divergence penalty (βm = 0) 11.4 15.3 9.5
PSNMF with maximum-divergence penalty (βm = 1) 10.9 16.2 8.2
PSNMF with maximum-divergence penalty (βm = 2) 11.6 16.1 9.2

Table 11

Average scores in four-source case of recorded signals (β = 1).

MethodSDR SIR SAR
Conventional SNMF
8.2 16.2 4.6
PSNMF with orthogonality penalty
13.4 16.2 12.4
PSNMF with maximum-divergence penalty (βm = 0) 14.0 15.3 12.0
PSNMF with maximum-divergence penalty (βm = 1) 12.2 16.1 11.6
PSNMF with maximum-divergence penalty (βm = 2) 13.0 16.2 11.6

Table 12

Average scores in four-source case of recorded signals (β = 2).

MethodSDR SIR SAR
Conventional SNMF
9.1 13.9 6.8
PSNMF with orthogonality penalty
12.3 15.1 11.6
PSNMF with maximum-divergence penalty (βm = 0) 12.9 15.0 13.2
PSNMF with maximum-divergence penalty (βm = 1) 11.9 14.3 12.5
PSNMF with maximum-divergence penalty (βm = 2) 10.3 11.8 12.2

that our proposed method can achieve higher separation accuracy compared with the conventional method even in thecase of real-recorded signals.
5.

Conclusion

In this study, we propose a new penalized SNMF with
two types of penalty that force the other bases to become
diﬀerent from the target bases trained in advance. Fromthe experimental results, it can be confirmed that the proposed method prevents the simultaneous generation of similar spectral patterns in the supervised bases and other bases,
and increases the separation performance compared with theconventional method.
Referencesmusic signals. We recorded each instrumental solo signal
and the supervision sound, which are the same as those in
the previous section, in an experimental room whose reverberation time was 200 ms. The distance between a loudspeaker and binaural microphone NEUMANN KU-100 was
1.5 m. The binaurally recorded signals in both ears weremixed down to a monaural signal. The observed signal Y
was produced by mixing these recorded signals as the same
power. Other conditions were the same as those of the previous experiment, and we prepared the observed signals in
two-source and four-source cases.
4.2 Experimental Results
Tables 7, 8, and 9 show the average scores of SDR, SIR, andSAR in the two-source case for each value of β. Here, theSDR values are the average of 12 combinations of the observed signals with optimization on the parameters μ1 , μ2 ,and λ, and the SIR and SAR values are the corresponding
ones. Also, Tables 10, 11, and 12 show the average scoresin the four-source case. From these results, we can confirm[1] D.D. Lee and H.S. Seung, “Algorithms for non-negative matrix factorization,” Neural Information Processing Systems, vol.13, pp.556–
562, 2001.
[2] T. Virtanen, “Monaural sound source separation by nonnegative matrix factorization with temporal continuity and sparseness criteria,”
IEEE Trans. Audio Speech Language Process., vol.15, pp.1066–1074,
2007.
[3] H. Kameoka, M. Nakano, K. Ochiai, Y. Imoto, K. Kashino,
S. Sagayama, “Constrained and regularized variants of non-negative
matrix factorization incorporating music-specific constraints,” Proc.
ICASSP, pp.5365–5368, 2012.
[4] P. Smaragdis, B. Raj, and M. Shashanka, “Supervised and semisupervised separation of sounds from single-channel mixtures,” Proc.
LVA/ICA 2010, LNCS 6365, pp.140–148, 2010.
[5] S. Eguchi and K. Yano, “Robustifying maximum likelihood estimation,” Technical Report, Institute of Statistical Mathematics, 2001.
[6] M. Nakano, H. Kameoka, J. Le Roux, Y. Kitano, N. Ono, and
S. Sagayama, “Convergence-guaranteed multiplicative algorithms for
non-negative matrix factorization with beta-divergence,” Proc. 2010
IEEE International Workshop on Machine Learning for Signal Processing, pp.283–288, 2010.
[7] E. Vincent, R. Gribonval, and C. Fevotte, “Performance measurementin blind audio source separation,” IEEE Trans. Audio Speech Language Process., vol.14, no.4, pp.1462–1469, 2006.