Ab t t
Abstractalgorithm
lg ith ffor non-negative
g ti matrix
t i factorization
f t i ti (NMF)()ddeveloped
l p dby Lee et al.
al
Even though the terms “complex”
complex and “non-negative”
non negative may seemt conflicttofli t with
ith eachh otherth andd theth decomposition
dp iti iis no longerl g amatrix factorization
factorization, we named this framework “Complex
Complex NMF”
NMF ,because it shares with NMF the ability to generate non-negative
non negative
matrices
tiandd , while
hil ththe iinput
p t matrixti
iis assumed
d tto bbe acomplex matrix and the algorithm also generates a third
third-rank
rankcomplex-valued
complexvalued tensor.

This p
Thi
paper
p p
presents
t a new sparseprepresentationpt ti ffor acoustic
ti signals
ig lbased on a mixing model defined in the complex
complex-spectrum
spectrum domain(where additivity holds):

This allows
Thi
ll
us to
t extractt t recurring
i g patterns
p tt
off magnitude
g it d spectrap t thatth tunderlie observed complex spectra and the phase estimates of
constituent signalssignals. An efficient iterative algorithm is derived
derived, which
reducesdunderd a particularp ti l condition
diti tot theth multiplicative
ltipli ti update
pd t

1 Introduction
1.

2 Research objective
2.

Sparse representation

Develop
thatt concurrently
advantages
Dl p new fframeworkk thtly offers
ffdt g off
NMF based and Sparse coding/Semi-NMF
coding/Semi NMF * -based
based acoustic modeling
NMF-based
S i NMF* Semi-NMF
variety of acousticAllowstoprojectallsignals((Dingg et aal.,, 2006)006)
phenomena withthat have the same spectral(Le Roux et al., 2008)[11]
shape onto a single basis
compact set ofspectral
p t l basesb
 In
I NMF fframework,k, Assume mixing model defined in magnitude domain.
domain Model can be optimized by efficient algorithm.
additive. →inaccurate modeling Magnitude spectra are assumed additive Phase information are discarded. →limits
limits the range of application

Given a set of observed vectors, the goal is to find a set of basis
vectorstsuchh ththatt anyy observations
bti
can b
be succinctly
i tly represented
pt d
as a linear combination of a small number of “active”
active basesbases.

p)[ ]
Sparsecodingd g (SC)( ) ((Olshausen et al.,, 1996)[1]Non-negativegmatrix ffactorization ((NMF)) ((Lee et al.,, 1999)[2])[ ]sparse((or other measure))

becomes sparse as aresult of a side effectcauseddb
by theh nonnegativity constraints

NMF model
Observed magnitude spectrogram[multiplicative update algorithm] Æ guarantees convergence to a stationary point
frequency

ObObservedd spectrogramtMusic transcription [3,4]
[3 4]
 1ch
1 h source separation
p ti [5] Speech denoising Bandwidthd d h expansionp Audio classification
 Dereverberation [7]etcetc.
Recurring spectral


Acctiva
atio
ontimeti

NMF applied
ppli d tto audio
di pprocessing
i gp
problems
blÆ parts-based
parts based decomposition ofmagnitude spectrogram

ObObserved
d complex
pl spectrogram
pgMagnitude
spectrum bases#Bases: 10

Static magnitude spectrum templatesTimeTime-varying
ing activation
ti tion coefficientsoeffi ient
h complex
plthspectrum
pbasis

SC/Semi-NMF
SC/Semi
NMF model

sparseOptimized modelth magnitude
it dspectrum
tbbasisi
activation ofth basis

activation ofth basisfrequency
time

Static complex spectrum templatesTime a ing activation
Time-varying
acti ation coefficients

patterns

3 C
3.
Complexpl NMF ((model)
d l))
Objective function

Model
Assume that the short
short-term
term Fourier transform (STFT) of an acoustic
signal is composed ofcomplex-valued
complex valued elements:

Given a set of observations
the optimal estimate of(1) Model reconstruction accuracy
Generative model:(frequencytitime

Each constituent
constituent,, is assumed to have a magnitude spectrum whichis constant up
p to the ggain over time:
Gain at frame
Normalized
l d magnitude
gd
spectrum template:
and a time-varying
time varying phase spectrum:

P
Proposedd modeld lObObserved
d complexo pspectrogram

Complex NMF model
thh magnitude
g i dspectrum
pbbasisactivation ofth basis

whereandare independent if

Statica magnitudeag ud spectrum
pu templates
paTime-varying
Timevarying gains & phase spectra)(likelihood)((2)) Sparsity
pa y ofo basis
ba aactivationsa o
As we want to achieve as parsimoniouspa representationp
as p
possible,, we would
like the’ss to be sparsesparse.
This basically means that any observed complex spectrum should bewell
ell represented
ep esented using
sing onl
only a fe
few acti
activee magnit
magnitudede spectspectrum
m baseseach of which is paired with an arbitrary phase spectrum.

(prior)
Generalized Gaussian distribution (GGD)
Æ promotes sparsity when
andth norm offtheiis bboundedd d
MAP (Maximum A Posteriori) estimation problem leads to...

frequency
time
Cannot be expressedin matrix notation!
Æ new class of sparse
representationt ti model
d l, the goal is to find4 C
4.
Complexl NMF ((algorithm)
l ith )
Utilizing auxiliary function concept

Update formulae

Definition.
i an auxiliary
isilifunction
f ti off theth objective
bj ti function
fti
if it satisfies.

Step 1)Non-negativities
Non negativities are preserved if we start with
non-negative
nonnegative initial conditions.

Theorem 1 (Auxiliary function method).
is non-increasing
non increasing under the updates
and
Proof
Assume that we set

at an arbitrary value . Letandd
It is obvious that
. We deduce
ffromththatt. By definition
definition, fromwe verify
ify thatth t.

(P j ti onto((Projectiont ththe constraintt i t space)p).
Step 2)

Theoremeo e 2 ((Auxiliaryy functiono foro Complexo pNMF)).

Condition for equivalence to NMF with Frobenius norm criterion
We assume a particular situation:
Update formulafor:withauxiliary function foris an, if

Update formulafor:

Proof(1) By differentiating the Lagrangianis a parameter that can bechosen arbitrarily subject toWhat if we update it
at each iteration
ite ation bybypartially w.r.t.
and setting it at zero, wedetermine the minimum value.
(2)amounts to a convex
quadratic
qd ti function
fti thatth t isi tangentt g t tto
att
argument.

?

Reduces to Lee’s
Lee s multiplicativeupdate
pd t fformulal ffor FFrobenius
b inorm criterion!

5 Experiments
5.
Activations

Conditions

Convergence of Complex NMF algorithm

SpeechSph data:
d t excerpted
pt d ffrom ATR B
B-sett speechphdatabase monaural and sampled at 16kHz
database,STFT was computed
i gHHanningi g window
i dththatt wasp t d using
32ms long with 16ms overlap.
reconstruction
t tierror

objective
bj i functionfisparsenessp

Iteration #: 30

Initialization: random

Iteration #How did we obtain the separate signals?

M
it d spectrum
ttMagnitude
atoms

The magnitude
gspectrum
patom closest to the true
spectrum was selected for each frame
frame, and theframewise signals,
signals each of which we constructed
using
i th
the selectedl t d atom
tandd ththe corresponding
diactivation coefficient and phase spectrum, wereconcatenated to synthesize the whole signal stream
stream.

Optimized
pmodelThe aim of the experiments wasto determine whether Complex
NMF hhas, similarly
i il l tto NMF
NMF, ththeeffect of extracting
g recurring
gspectral patterns underlying theobserved audio datadata.

Speechp1

Reconstructed 1
Complex NMF

0 87dB
0.87dB

4 3dB
4.3dB

Mixed speech

Speech 2

Reconstructed 2-0.87dB
0 87dB

References cited in the paper
p p
[1] B
B.A.
A Olshausen and DD.J.
J Field
Field, “Emergence
Emergence of simple
simple-cell
cell receptive field properties by learning a sparse code for natural images
images,”Nature 381,
Nature,381 607–609
607 609, 1996.
1996[[2]] D.D. Lee and H.S. Seung,g, “Algorithmsg
for nonnegativegmatrix factorization,”, Proc. NIPS’00,, 556–562,, 2000.
[3] M.D. Plumbley, S.A. Abdallah, J.P. Bello, M.E. Davies, G. Monti and M.B. Sandler, “Automatic music transcription and audio source
separation,” Cybernetics and Systems, 33(6), 603–627, 2002.
[4] P.
P S
Smaragdis
di andd JJ.C.
C BBrown, “N“Non-negativeti matrix
t i factorization
f t i ti forf musici transcription,”ti ti ” Proc.
PWASPAA’03
WASPAA’03, 177–180,
177 180 2003
2003.
[5] TT. Virtanen
Virtanen, “SoundSound source separation using sparse coding with temporal continuity objective,objective ” Proc
Proc. ICMC’03
ICMC 03, 231
231–234,
234 2003
2003.

4 0dB
4.0dB

[6] S.A. Abdallah, M.D. Plumbley, “Unsupervised
Unsupervised analysis of polyphonic music using sparse coding,coding,” IEEE Trans. NN, 17(1), 179–196,
179 196, 2006.
[[7]] H. Kameoka T. Nakatani and T. Yoshioka,, “Robust speech
pdereverberation based on non-negativitygy and sparse
pnature of speech
pspectrograms,”
pg,
Proc. ICASSP’09, to appear.
[8] R.M. Parry and I. Essa, “Phase-Aware Non-negative Spectrogram Factorization,” Proc. ICA’07, 536–543, 2007.
[9] C.
C FFevotte,
tt N
N. B
Bertinti andd JJ.-L.
L Durrieu,
D i“Nonnegative“Nti matrix
t i ffactorizationt i ti with
ith ththe ItItakura-SaitokS it divergence
diwithith application
li ti tto musici analysis,”l i ”
Technical Report TELECOM ParisTech 2008D006
2008D006, 20082008.
[10] T. Blumensath and M.E. Davies, “SparseSparse and shift
shift-invariant
invariant representations of music,music,” IEEE Trans. ASLP, 14(1), 50–57,
50 57, 2006.
[[11]] J. Le Roux,, A. de Cheveigneg and L.C. Parra,, “Adaptiveptemplatepmatchingg with shift-invariant semi-NMF,”, Proc. NIPS’08,, 2008.
[12] A.T. Cemgil, “Bayesian inference for nonnegative matrix factorisation models,” Technical Report CUED/F-INFENG/TR.609, University ofCambridge, 2008.