Music Signal Separation Using Supervised NMF with All-Pole-Model-Based Discriminative Basis Deformation
H. Nakajima (The University of Tokyo: UTokyo), D. Kitamura (SOKENDAI), N. Takamune (UTokyo), S.Koyama (UTokyo), H. Saruwatari (UTokyo), N. Ono (NII) , Y. Takahashi (Yamaha Corp.), K. Kondo (Yamaha Corp.)

4. DISCRIMINATIVE BASIS TRAINING

1. INTRODUCTION• Nonnegative matrix factorization (NMF) can beused to decompose a spectrogram into BASISmatrix and ACTIVATION matrix.
• Supervised NMF (SNMF) can extract the target source byusing pre-trained basis matrix F in advance.
• DRAWBACK of SNMF
→Timbre mismatch between pre-trained basis and actual
target signal causes big error in source separation.

Problem in determining appropriate order of all-pole model
Small order leads to insufficient deformation (basis mismatch cannot be resolved).
Large order results in exceeding deformation that wrongly represents other (non-target) signal.

RESEARCH PURPOSE• Introduction of All-Pole-Model-Based Basis Deformation• Discriminative Basis Training for obtaining appropriate degree of freedom of deformation

We should solve Bilevel Optimization that can supply the appropriate solution holding
both conditions: (a) spectral mismatch becomes as small as possible, and (b) mixture canbe modeled as accurate as possible using HU.

2. CONVENTIONAL METHODSNMF and SNMF [1]
SNMF:

NMF:
Sparse representation using nonnegative
matrices F and G

Using pre-trained basis matrix 𝑭, we can decompose
𝒀 ≅ 𝑭𝑮 + 𝑯𝑼 to get 𝑭𝑮 (target source spectrogram).

Since the bilevel optimization is generally hard to solve, we introduce an approximatediterative parameter update rule as follows.

SNMF with Additive Basis Deformation (SNMF-ABD) [2]
The method deforms the original basis 𝑭 to be 𝑭 + 𝑫 to adapt the actual target spectrogram in 𝒀mix.
Deformation model

Difficult to optimize because D is NOT limited within
nonnegative.
Hard to simultaneously perform separation and deformation.

𝒀mix ≈ 𝑭 + 𝑫 𝑮 + 𝑯𝑼

3. PROPOSED METHOD FOR BASIS DEFORMATION

5. EXPERIMENTAL EVALUATION

Statistical Postfilter: Generalized MMSE Short-Time
Spectral Amplitude (MMSE-STSA) Estimator [3,4]Experimental Condition
Type of instruments

High accuracy NMF postfilter based on non-target spectral variance and a priori statistical target model.
Thanks to the a priori target statistical model, we canobtain higher separation result rather than SNMF only.

𝒀 = 𝑱 ○ 𝒀mix
: Hadamard product
Assuming target amplitude spectrum
𝒀target obeys chi distribution, we solve
min 𝐸[(𝒀target − 𝒀)2 ]J can be used as the measure of seldom overlapping
components among target and non-target signals.
⇒ we define a matrix I as the binary version of J for
sampling convincing components𝐽

Target sound (MIDI)
Sampling frequency
FFT length

Oboe (Ob.), Piano (Pf.), Trombone (Tb.)
2 octave notes generated byGarritan Professional Orchestra
Generated by Microsoft GS Wavetable SW Synth (*)
44100 Hz4096 points (about 92 ms)

Shift length

512 points (about 12 ms)

Number of bases
Iteration
Evaluation score
Comparison

Target: 100, Non-target: 30
Training : 1000, Separation: 1000
Signal-to-Distortion Ratio (SDR)
SNMF, Penalized SNMF (PSNMF) [5], SNMF-ABD [2]Training sample (MIDI)

g can be calculated by NMF output: confluent hyper-geometric function
G(): gamma function*Music score usedin this experimente, g: a priori and a posteriori SNR andr: shape parameter of chi distribution

Basic Idea and System Overview
Motivation:• Simultaneous processing for separation and basis
deformation results in bad side effect each other.
• We want to separate the whole process into 2 parts:
deformation and separation.

Results

Proposed method outperforms
the conventional NMF methods
in almost all the cases.

variables

Deformation model

𝑰 ○ 𝒀 ≈ 𝑰 ○ (𝑨𝑭org 𝑮): Binary masking to sample convincing target components: Output of Generalized MMSE-STSA estimator: Diagonal matrix with All-Pole-Model spectrum gain
: Original basis matrix: Activation matrix

Smoothed spectral envelope toreduce mismatch between thesampled target spectra andCost Function and Parameter Update Rule
Cost function to optimize all-pole-model A
using Y and I (KL-divergence-based).

Update Rulea is a vector with entry ak

where：element of I

：element of 𝒀

：element of 𝑭org

: element of 𝑮

Ω: Nyquist frequency

REFERENCES[1] D. D. Lee, et al., Proc. Advances in Neural information Processing Systems, 2001.
[2] D. Kitamura, et al., Proc. IEEE DSP 2013, 2013.
[3] C. Breithaupt, et al., IEEE Trans. ASLP 2008.
[4] Y. Murota, et al., Proc. ICASSP, pp. 7490-7494, 2014.
[5] D. Kitamura, et al., IEICE Trans. Fundamentals, vol. E97-A, no. 5, pp. 1113-1118, 2014.