ISMIR 2008 – Session 1c – Timbre

INSTRUMENT EQUALIZER FOR QUERY-BY-EXAMPLE RETRIEVAL:IMPROVING SOUND SOURCE SEPARATION BASED ON
INTEGRATED HARMONIC AND INHARMONIC MODELS
Katsutoshi Itoyama† * Masataka Goto‡ Kazunori Komatani†
Tetsuya Ogata† Hiroshi G. Okuno†† Graduate School of Infomatics, Kyoto University * JSPS Research Fellow (DC1)‡ National Institute of Advanced Industrial Science and Technology (AIST){itoyama,komatani,ogata,okuno}@kuis.kyoto-u.ac.jp m.goto@aist.go.jp

ABSTRACTface. The interface enables a user to boost or cut the volumeof each instrument part of an existing musical piece. With
this interface, a user can easily give an alternative query witha different mixing balance to obtain reﬁned results of the
QBE retrieval. The issue in the above example of ﬁnding
another piece with weaker vocal or drum sounds can thus
be resolved. Note that existing graphic equalizers or tone
controls on the market cannot control each individual instrument part in this way: they can adjust only frequency characteristics (e.g., boost or cut for bass and treble). Although
remixing stereo audio signals [8] had reported previously,
it had tackled to control only harmonic instrument sounds.
Our goal is to control all instrument sounds including both
harmonic and inharmonic ones.
This paper describes our music remixing interface, called
Instrument Equalizer, in which a user can listen to and remix
a musical piece in real time. It has sliders corresponding
to different musical instruments and enables a user to manipulate the volume of each instrument part in polyphonicaudio signals. Since this interface is independent of the succeeding QBE system, any QBE system can be used. In ourcurrent implementation, it leverages the standard MIDI ﬁle(SMF) corresponding to the audio signal of a musical piece
to separate sound sources. We can assume that it is relatively
easy to obtain such SMFs from the web, etc. (especially forclassical music). Of course, given a SMF, it is quite easy tocontrol the volume of instrument parts during the SMF playback, and readers might think that we can use it as a query.
Its sound quality, however, is not good in general and users
would lose their drive to use the QBE retrieval. Moreover,
we believe it is important to start from an existing favorite
musical piece of high quality and then reﬁne the retrieved
results.

This paper describes a music remixing interface, called Instrument Equalizer, that allows users to control the volumeof each instrument part within existing audio recordings
in real time. Although query-by-example retrieval systems
need a user to prepare favorite examples (songs) in general,
our interface gives a user to generate examples from existing ones by cutting or boosting some instrument/vocal parts,
resulting in a variety of retrieved results. To change the volume, all instrument parts are separated from the input soundmixture using the corresponding standard MIDI ﬁle. For
the separation, we used an integrated tone (timbre) model
consisting of harmonic and inharmonic models that are initialized with template sounds recorded from a MIDI soundgenerator. The remaining but critical problem here is to deal
with various performance styles and instrument bodies thatare not given in the template sounds. To solve this problem,
we train probabilistic distributions of timbre features by using various sounds. By adding a new constraint of maximizing the likelihood of timbre features extracted from eachtone model, we succeeded in estimating model parametersthat better express actual timbre.
1 INTRODUCTION
One of promising approaches of music information retrieval
is the query-by-example (QBE) retrieval [1, 2, 3, 4, 5, 6, 7]
where a user can receive the list of musical pieces ranked
by their similarity to a musical piece (example) that the user
gives as a query. Although this approach is powerful anduseful, a user has to prepare or ﬁnd favorite examples andsometimes feels difﬁculty to control/change the retrieved
pieces after seeing them because the user has to ﬁnd anotherappropriate example to get better results. For example, if auser feels that vocal or drum sounds are too strong in the
retrieved pieces, the user has to ﬁnd another piece that has
weaker vocal or drum sounds while keeping the basic moodand timbre of the piece. It is sometimes very difﬁcult to ﬁndsuch a piece within a music collection.
We therefore propose yet another way of preparing an example for the QBE retrieval by using a music remixing inter-

2 INSTRUMENT EQUALIZERThe Instrument Equalizer enables a user to remix existing
polyphonic musical signals. The screenshot of its interface
is shown in Figure 1 and the overall system is shown in Figure 2. It has two features for remixing audio mixtures asfollows:133

ISMIR 2008 – Session 1c – Timbre

Figure 1. Screenshot of main window.
Figure 3. System architecture.

ually improved to represent actual sounds in the sound mixture.
2.1 Internal architectures
This section describes the internal architectures of controlling the volume of each instrument part. The procedures
described in this section are performed in real time under
the assumption that the musical signals of each instrument
part already have been obtained in advance from the target
polyphonic musical signal, as described in Section 3. Let
xk (c, t) and yk (c, t) be a separated signal and the volume of
instrument k at channel c and time t, respectively. yk (c, t)satisﬁes the following condition:

Figure 2. Instrument Equalizing System.

∀k, c, t : 0 ≤ yk (c, t) ≤ 1,
1. Volume control function. It provides the remixing
function by boosting or cutting the volume of each
instrument part, not by controlling the gain of a frequency band. A user can listen to the remixed soundmixture as soon as the user manipulates the volume.

and yk (c, t) is obtained asyk (c, t) = (value of volume slider k) · (value of the pan c).
The overview of the architecture is shown in Figure 3.
1. Volume control function. The output signal, x(c, t), is
obtained as
x(c, t) =yk (c, t) · xk (c, t).

2. Interlocking with the hardware controller. In addition
to a typical mouse control on the screen, we allow
a user to use a hardware controller shown in Figure
2 with multiple faders. It enables the user to manipulate the volume intuitively and quickly. This hardware
controller makes it easy to manipulate the volume ofmultiple parts at the same time, while it is difﬁcult on
a mouse control.

kEach yk (t) is obtained in real-time from the volumesliders in the GUI in Figure 1.
2. Interlocking with the hardware controller. The GUI
and the hardware controller communicate by MIDI.
If users control the hardware fader, a MIDI message
which represents the new volume is sent to the GUI,and vice varsa. Since a motor is embeded in the fader,
MIDI messages from the GUI move the fader to theposition corresponding value of the volume.

To remix a polyphonic musical signal, the signal must be
separated into each instrument part. We use an integrated
weighted mixture model consisting harmonic-structure andinharmonic-structure tone models [9] for separating the signal, but improve the parameter estimation method of thismodel by introducing better prior distributions. This separation method needs a standard MIDI ﬁle (SMF) that issynchronized to the polyphonic signal. We assume that
the SMF has already been synchronized with the input
signal by using audio-to-score alignment methods such as[10, 11, 12]. For the separation using the integrated model,
the parameters of the model are initialized by templatesounds recorded from a MIDI sound generator. and grad-

3 SOUND SOURCE SEPARATION CONSIDERINGTIMBRE VARIETIESIn this section, we ﬁrst deﬁne our sound source separation
problem and the integrated model. We then describe timbre varieties and timbre feature distributions for estimating
parameters of the model.

134

ISMIR 2008 – Session 1c – Timbre

3.1 Integrated model of harmonic and inharmonic models

Table 1. Parameters of the integrated model.
Symbol(J)wkl
rkl (c)(H)(I)
wkl , wkl

The sound source separation problem is to decompose theinput power spectrogram, X(c, t, f ), into the power spectrogram corresponding to each musical note, where c, t, andf are the channel (e.g., left and right), the time, and the frequency, respectively. We assume that X(c, t, f ) includes K
musical instruments and the k-th instrument performs Lk
musical notes. We use the tone model, Jkl (c, t, f ), to represent the power spectrogram of the l-th musical note performed by the k-th musical instrument ((k, l)-th note), andthe power spectrogram of a template sound, Ykl (t, f ), to initialize the parameters of Jkl (c, t, f ). Each musical note of
the SMF is played back on a MIDI sound generator to record
the corresponding template sound. Ykl (t, f ) is monaural because SMFs may not include any sound localization (channel) information. Ykl (t, f ) is normalized to satisfy the following relation, where C is the total number of the channels:X(c, t, f ) dt df = Ccukl (m)vkl (n)τkl
φkl
ωkl (t)σkl(I)Ekl (t)(I)
Fkl (t, f )where M is the number of Gaussian kernels representing the
temporal power envelope and N is the number of Gaussian
kernels representing the harmonic components. ukl (m),
(I)
(I)
(H)
(I)
vkl (n), Ekl (t), Fkl (t, f ), wkl , and wkl satisfy the following conditions:∀k, l :
ukl (m) = 1, ∀k, l :vkl (n) = 1,

Ykl (t, f ) dt df.

k,l

For this source separation, we deﬁne an integrated model,
Jkl (c, t, f ), as the sum of harmonic-structure tone models,
Hkl (t, f ), and inharmonic-structure tone models, Ikl (t, f ),(J)multiplied by the whole amplitude of the model, wkl , andthe relative amplitude of each channel, rkl (c):m∀k, l :(J)

Jkl (c, t, f ) = wkl rkl (c) Hkl (t, f ) + Ikl (t, f ) ,and(J)
where wkl and rkl (c) satisfy the following constraints:

(J)

X(c, t, f ) dt df, ∀k, l :

wkl =



Description
overall amplituderelative amplitude of each channel
relative amplitude of harmonic and inharmonic
tone models
coefﬁcient of the temporal power envelope
relative amplitude of n-th harmonic component
onset time
diffusion of a Gaussian of power envelope
F0 trajectory
diffusion of a harmonic component along thefreq. axis
power envelope of inharmonic tone model
relative amplitude of frequency f at time t ofinharmonic tone modeln(I)
Ekl (t) dt = 1,
(H)∀k, l, t :(I)

Fkl (t, f ) df = 1,

(I)∀k, l : wkl + wkl = 1.

The goal of this separation is to decompose X(c, t, f )into Jkl (c, t, f ) by estimating a spectrogram distributionfunction, Δ(J) (k, l; c, t, f ), which satisﬁes

rkl (c) = C.

ck,l∀k, l, c, t, f : 0 ≤ Δ(J) (k, l; c, t, f ) ≤ 1
Δ(J) (k, l; c, t, f ) = 1.
∀c, t, f :All parameters of Jkl (c, t, f ) are listed in Table 1. The
harmonic model, Hkl (t, f ), is deﬁned as a constrained
two-dimensional Gaussian mixture model (GMM), which is (H)a product of two one-dimensional GMMs,
Ekl (m, t)
 (H)
and
Fkl (n, t, f ), and is designed by referring to
the harmonic-temporal-structured clustering (HTC) source
model [13]. The inharmonic model, Ikl (t, f ), is deﬁned asa product of two nonparametric functions. The deﬁnition ofthese models is as follows:(H)

Hkl (t, f ) = wkl

M 
N


(H)

k,l

With Δ(J) (k, l; c, t, f ), the separated power spectrogram,(J)
Xkl (c, t, f ), is obtained as(J)

Xkl (c, t, f ) = Δ(J) (k, l; c, t, f )X(c, t, f ).
Furthermore, let Δ(H) (m, n; k, l, t, f ) and Δ(I) (k, l, t, f )be spectrogram distribution functions which decompose(J)
Xkl (c, t, f ) into each Gaussian distribution of the harmonic model and the inharmonic model, respectively. These
functions satisfy(H)

Ekl (m, t)Fkl (n, t, f ),m=1 n=1

(t − τkl − mφkl )2
,
−
2φkl2


vkl (n)(f − nωkl (t))2
(H)
exp −
Fkl (n, t, f ) = √
, and
2σkl2
2πσkl
ukl (m)(H)
Ekl (m, t) = √
exp
2πφkl

(I)

(I)

and

∀k, l, m, n, t, f : 0 ≤ Δ(H) (m, n; k, l, t, f ) ≤ 1,
∀k, l, t, f : 0 ≤ Δ(I) (k, l, t, f ) ≤ 1, and
Δ(H) (m, n; k, l, t, f ) + Δ(I) (k, l, t, f ) = 1.
∀k, l, t, f :(I)

Ikl (t, f ) = wkl Ekl (t)Fkl (t, f ),

m,n

135

ISMIR 2008 – Session 1c – Timbre

To evaluate the ‘effectiveness’ of this separation, we canuse a cost function deﬁned as the Kullback-Leibler (KL) di(J)vergence from Xkl (c, t, f ) to Jkl (c, t, f ):
(J)

Qkl =



3.3 Cost function without considering timbre feature
distributions
In Itoyama’s previous study [9], they used template sounds
instead of timbre feature distributions to evaluate the ‘good(Y )ness’ of the feature vector. The cost function, Qkl , used in[9] can be obtained by replacing the negative log-likelihood,(p)(Y )Qkl , with the KL divergence, Qkl , from Ykl (t, f ) (the(t, f ):
power spectrogram of a template sound) to Jkl

(J)

(J)

Xkl (c, t, f ) logcXkl (c, t, f )dt df.
Jkl (c, t, f )(J)

By minimizing the sum of Qkl over (k, l) pertaining to
Δ(J) (k, l; c, t, f ), we obtain the spectrogram distributionfunction and model parameters (i.e., the most ‘effective’ decomposition).
(J)
By minimizing the Qkl pertaining to each parameter ofthe integrated model, we obtain model parameters estimatedfrom the distributed spectrogram. This parameter estimation
is equivalent to a maximum likelihood estimation. The parameter update equations are described in appendix A.

(Y )

Qkl =

c

+


c

2. using the template sounds (previous method [9], section 3.3).
4.1 Experimental conditions
We used 5 SMFs from the RWC Music Database (RWCMDB-P-2001 No. 1, 2, 3, 8, and 10) [16]. We recorded allmusical notes of these SMFs by using two different MIDI
sound generators made by different manufacturers. We used
one of them for the test (evaluation) data and the other for
obtaining the template sounds or training the timbre feature
distributions in advance.
The experimental procedure is as follows:

Xkl (c, t, f )dt df
Jkl (c, t, f )1. initialize the integrated model of each musical note byusing the corresponding template sound,

1
(u)
(u)
(ukl (m) − μk (m))2 /Σk (m)
2 m
1
(v)
(v)
(vkl (n) − μk (n))2 /Σk (n)
+
2 n
1
2

(I)

(F )

rkl (c)Ykl (t, f )dt df.
Jkl (c, t, f )1. using the log-likelihood of timbre feature distributions (proposed method, section 3.2),

2. estimate all the model parameters from the input
sound mixture, and++rkl (c)Ykl (t, f ) log

4 EXPERIMENTAL EVALUATION

(J)

(J)

Xkl (c, t, f ) log



Xkl (c, t, f )dt df
Jkl (c, t, f )We conducted experiments to conﬁrm whether the performance of the source separation using the prior distribution isequivalent to the one using the template sounds. In the ﬁrst
experiment, we separated the sound mixtures which weregenerated from a MIDI sound generator. In the other one,the sound mixtures were created from the signals with multiple tracks [15] which were before mixdown. In this experiment, we compared the following two conditions:

Even within the same instrument, different instrument bodies have different timbres, although its timbral difference issmaller than the difference among different musical instruments. Moreover, in live performances, each musical note
could have slightly different timbre according to the performance styles. Instead of preparing a set of many templatesounds to represent such timbre varieties within each instrument, we represent them by using a probabilistic distribution.
We use parameters of the integrated model, ukl (m),(I)
vkl (n), and Fkl (t, f ), to represent the timbre variety of instrument k by training a diagonal Gaussian distribution with
(u)
(v)
(F )
(u)
mean μk (m), μk (m), μk (f ) and variance Σk (m),(v)
(F )Σk (n), Σk (f ), respectively. Note that other probability
distributions, such as a Dirichlet distribution, are available
in this case. The model parameters for training the prior distribution are extracted from instrument sound database [14](i.e., the parameters are estimated without any prior distributions).
By minimizing the cost function,
(p)

(J)

(J)

Xkl (c, t, f ) log

c

3.2 Timbre varieties within each instrument

Qkl =



3. calculate the SNR in the frequency domain for theevaluation.
The SNR is deﬁned as follows:(F )(Fkl (t, f ) − μk (f ))2 /Σk (f ) dt df,

SNR =where the last term is an additional cost by using the priordistribution, we obtain the parameters by taking into accountthe timbre varieties. This parameter estimation is equivalent
to a maximum A Posteriori estimation.

1
C(T1 − T0 ) c

SNRkl (c, t) dt,(J)

SNRkl (c, t) = log10136

Xkl (c, t, f )2
(J)

(R)

Xkl (c, t, f ) − Xkl (c, t, f )2 df,

ISMIR 2008 – Session 1c – Timbre

the better prior distributions.

Table 2. Experimental conditions.
Frequency
analysisParameters

MIDI soundgenerator

Sampling rate
Analyzing method
STFT window
STFT shift
C
M
N
Test data
Template sounds

5 CONCLUSION

44.1 kHz
STFT
2048 points Gaussian
441 points
2
10
30
YAMAHA MU-2000
Roland SD-90

In this paper, we have proposed the novel use of a music
remixing interface for generating queries for the QBE retrieval, explained our Instrument Equalizer on the basis ofsound source separation using an integrated model consisting of harmonic and inharmonic models, and described a
new parameter estimation method for the integrated modelby using the timbre feature distributions. We conﬁrmed that
this method increased separation performance for most instrument parts simply by using the basic timbre features.
Although the use of the timbre feature distributions is
promising, it has not been fully exploited in our experiments. For example, we have not tried to use training data
including various performance styles and instrument bodies.
We plan to evaluate our method by using such various training data as well as more advanced timbre features. Some
performance benchmark for audio source separation [17]will helpful to compare our separation method with otherones. Future work will also include the usability evaluationof the Instrument Equalizer for the use of the QBE retrieval.

Table 3. SNRs of the signals separated from sound mixturesgenerated from a MIDI tone generator. P1, P2, and P3 are
the SNRs which are based on the prior distribution trainedusing 1, 2, and 3 instrument bodies, respectively. T is the
SNR which is based on the template sounds.
P1
P2
P3
T
P001 11.5 12.1 11.6 14.0
P002 12.3 12.3 12.5 12.3
P003 11.5 11.8 12.4 10.8
P008
8.1
7.8
8.3
4.9
9.1
8.8
8.9 12.2
P010
Ave. 10.5 10.6 10.8 10.8

6 ACKNOWLEDGEMENTS
This research was partially supported by the Ministry of
Education, Science, Sports and Culture, Grant-in-Aid for
Scientiﬁc Research of Priority Areas, Primordial Knowledge Model Core of Global COE program and CrestMuse
Project.

Table 4. SNRs of the signals separated from sound mixturesgenerated from CD recordings.
P1
P2
P3
T
P001
9.3
9.4
9.4
9.0
P002 11.4 11.5 11.7 11.6
P003
4.0
4.1
4.2
3.1
P008
7.1
7.2
7.3
6.1
8.4
8.5
8.5
8.0
P010
Ave.
8.1
8.2
8.3
7.6

7 REFERENCES[1] Rauber, A., Pampalk, E., Merkl, D., “Using Psycho-acoustic
Models and Self-organizing Maps to Create a Hierarchical
Structuring of Music by Sound Similarity”, Proc. ISMIR, pp.
71–80, 2002.
[2] Yang, C., “The MACSIS Acoustic Indexing Framework forMusic Retrieval: An Experimental Study”, Proc. ISMIR, pp.
53–62, 2002.

where T0 and T1 are the beginning and ending times of(R)
the input power spectrogram, X(c, t, f ), and Xkl (c, t, f )is the ground-truth power spectrogram corresponding to the(k, l)-th note (i.e., the spectrogram of an actual sound before
mixing). We used a 40-parameters for the prior distributions(1–10 dimensions: ukl (1), . . ., ukl (M ), 11–40 dimensions:vkl (1), . . ., vkl (N )), where M = 10 and N = 30. Other
experimental conditions are shown in Table 2.
4.2 Experimental results[3] Allamanche, E., Herre, J., Hellmuth, O., Kastner, T., Ertel, C.,“A Multiple Feature Model for Musical Similarity Retrieval”,
Proc. ISMIR, 2003.
[4] Feng, Y., Zhuang, Y., Pan, Y., “Music Information Retrieval by
Detecting Mood via Computational Media Aesthetics”, Proc.
of Web Intelligence, pp. 235–241, 2003.
[5] Thoshkahna, B. and Ramakrishnan, K. R., “Projekt Quebex: A
Query by Example System for Audio Retrieval”, Proc. ICME,
pp. 265–268, 2005.

The results are listed in Tables 3 and 4. In both experiments,
the SNRs were improved by increasing the number of instrument bodies for training the prior distributions. Furthermore, the average SNR of P3 is equal to that of T in Table 3
and the average SNRs in Table 4 is improved from the average SNR of T. This means that although the SNRs decreasein some cases where the timbre difference between templatesounds and input sounds is large, it can be resolved by using[6] Vignoli, F. and Pauws, S., “A Music Retrieval System Basedon User-driven Similarity and Its Evaluation”, Proc. ISMIR,
pp. 272–279, 2005.
[7] Kitahara, T., Goto, M., Komatani, K., Ogata, T., Okuno, H.
G., “Musical Instrument Recognizer “Instrogram” and Its Application to Music Retrieval based on Instrumentation Similarity”, Proc. ISM, pp. 265–274, 2006.

137

ISMIR 2008 – Session 1c – Timbre
(H)

(I)

A.3 wkl , wkl : amplitude of harmonic and inharmonic tone models (H)

Xklmn (c, t, f ) dt df
c,m,n
(H)

wkl =   
(H)
(I)
c
m,n Xklmn (c, t, f ) + Xkl (c, t, f ) dt df[8] Woodruff, J., Pardo, B., and Dannenberg, R., “Remixing
Stereo Music with Score-informed Source Separation”, Proc.
ISMIR, pp. 314–319, 2006.
[9] Itoyama, K., Goto, M., Komatani, K., Ogata, T., Okuno,
H.G., “Integration and Adaptation of Harmonic and Inharmonic Models for Separating Polyphonic Musical Signals”,
Proc. ICASSP, pp. 57–60, 2006.

and

 

(I)

(I)

Xkl (c, t, f ) dt df

[10] Cano, P., Loscos, A., and Bonada, J., “Score-performance
Matching using HMMs”, Proc. ICMC, pp. 441–444, 1999.

wkl =   [11] Adams, N., Marquez, D., and Wakeﬁeld, G., “Iterative Deepening for Melody Alignment and Retrieval”, Proc. ISMIR, pp.
199–206, 2005.

A.4 ukl (m): coefficient of the temporal power envelope  (H)(u)
Xklmn (c, t, f ) dt df + μk (m)c,nukl (m) = .
 (H)
Xklmn (c, t, f ) dt df + 1
c,m,n

(H)
(I)
m,n Xklmn (c, t, f ) + Xkl (c, t, f )c[12] Cont, A., “Realtime Audio to Score Alignment for PolyphonicMusic Instruments using Sparce Non-negative Constraints andHierarchical HMMs”, Proc. ICASSP, pp. 641–644, 2006.

c

A.5 vkl (n): relative amplitude of n-th harmonic component  (H)(v)
Xklmn (c, t, f ) dt df + μk (m).
vkl (n) = c (H)Xklmn (c, t, f ) dt df + 1c,n[13] Kameoka, H., Nishimoto, T., Sagayama, S., “Harmonictemporal Structured Clustering via Deterministic Annealing
EM Algorithm for Audio Feature Extraction”, Proc. ISMIR,
pp. 115–122, 2005.

A.6 τkl : onset time[14] Goto, M., Hashiguchi, H., Nishimura, T., and Oka, R., “RWC
Music Database: Music Genre Database and Musical Instrument Sound Database”, Proc. ISMIR, pp. 229-230, 2003.

(H)(t − mφkl )Xklmn (c, t, f ) dt df.
 (H)

Xklmn (c, t, f ) dt df
c,m,n

c,m,n

τkl =

[15] Goto, M., “AIST Annotation for the RWC Music Database”,
Proc. ISMIR, pp. 359–260, 2006.

A.7 ωkl (t): F0 trajectory[16] Goto, M., Hashiguchi, H., Nishimura, T., and Oka, R.,“RWC Music Database: Popular, Classical, and Jazz Music
Databases”, Proc. ISMIR, pp. 287–288, 2002.

ωkl (t) = 

c,m,n
c,m,n

A DERIVATION OF THE PARAMETER UPDATEEQUATION

(φ)

A1

(H)

n2 Xklmn (c, t, f ) df

.



(H)

m(t − τkl )Xklmn (c, t, f ) dt df, and

c,m,n



(H)

(t − τkl )2 Xklmn (c, t, f ) dt df.

c,m,n

A.9 σkl : diffusion of harmonic component along thefrequency axis



(H)
 c,m,n (f − nωkl (t))2 Xklmn(c, t, f ) dt df
σkl = .
 (H)

Xklmn (c, t, f ) dt df
c,m,n

(J)

(J)

Xkl (c, t, f ) = Δ(I) (k, l, t, f )Xkl (c, t, f ).

(J )A.1 wkl : overall amplitude


 (H)

(J)
(I)
wkl =
Xklmn (c, t, f ) + Xkl (c, t, f ) dt df.
c

=

(φ)
A0 =

Xklmn (c, t, f ) = Δ(H) (m, n; k, l, t, f )Xkl (c, t, f )(I)

nf Xklmn (c, t, f ) df

c,m,n

In this section, we describe the update equations of eachparameter derived from the M-step of the EM algorithm.
By differentiating the cost function about each parameter,(H)the update equations were obtained. Let Xklmn (c, t, f ) and(I)
Xkl (c, t, f ) be the decomposed power:

and

(H)



A.8 φkl : diffusion of a Gaussian of power envelope

(φ)
(φ)2
(φ) (φ)−A1 + A1 + 4A2 A0
φkl =, where
(φ)
2A1

(φ)
(H)
Xklmn (c, t, f ) dt df,
A2 =[17] Vincent, E., Gribonbal, R., Févotte, C., “Performance Measurement in Blind Audio Source Separation”, IEEE Trans. on
ASLP, vol. 14, No. 4, pp. 1462–1469, 2006.

(H)

.
dt df

(I)

(I)

A.10 Ekl (t), Fkl (t, f ): inharmonic tone model
 

m,n

A.2 rkl (c): relative amplitude of each channel

 
(H)
(I)
C
m,n Xklmn (c, t, f ) + Xkl (c, t, f ) dt df
.
rkl (c) =   
(H)
(I)
c
m,n Xklmn (c, t, f ) + Xkl (c, t, f ) dt df

138

(I)
Ekl (t) = 

c


c


(I)
Fkl (t, f ) =(I)

Xkl (c, t, f ) df
(I)

Xkl (c, t, f ) dt df
(I)

(F )

and

c Xkl (c, t, f ) + μk (f ).
  (I)c Xkl (c, t, f ) df + 1