Linear Multichannel Blind Source Separation Based on
Time-Frequency Mask Obtained by Harmonic/Percussive Sound Separation

2381

Soichiro Oyabu, Daichi Kitamura (National Institute of Technology, Kagawa College, Japan), and Kohei Yatabe (Waseda University, Japan)
1. Background

2. Proposed Method• Blind source separation (BSS)• Process flow (overview)

3. Experiments• Conducted experiments

Audio source separation problem without any prior information or training
We only aim at the separation of harmonic and percussive audio sources

1. Investigation of the optimal number of iterations in HPSS process
2. Investigation of the optimal smoothing parameter in smoothing process
3. Performance comparison with state-of-the-art existing BSS algorithms• Single-channel BSS

Impulse response E2A inRWCP database [Nakamura+, 2000](reverberation time: 300 ms)
Music dataset
SiSEC2016 MUS [Liutkus+, 2016]
Other source
Drum source(dry sources)“Drums” and “Other” sources of 20 songs(harmonic)(Percussive)
Windowing in STFT 128-ms-long Hann window with half-overlap shifting2m• Conditions

BSS problem for monaural-recorded audio signals (difficult)
Harmonic/percussive sound separation (HPSS) [Ono+, 2008], etc.

• Multichannel BSS
Blind estimation of demixing system
Mixing system(inverse of mixing system)

Number of iterations
500
in TFMBSS

Demixing system

Subjectiveevaluation score

50

Improvement of source-to-distortion ratio (SDR)[Vincent+, 2006]

50

5.66 cm• Experiment 1: optimal number of iterations in HPSS
12

Average SDR imp. [dB]

Independent vector analysis (IVA) [Hiroe+, 2006], [Kim+, 2006], [Ono, 2011]
Independent low-rank matrix (ILRMA) [Kitamura+, 2016]
Time-frequency-masking-based BSS (TFMBSS) [Yatabe+, 2019]• HPSS [Ono+, 2008], [FitzGerald, 2010], [Duong+, 2011], [Tachibana+, 2012], etc.
BSS focusing on “smoothness” along with time or frequency directions inspectrogram
TFMBSS iteratively optimizes the linear spatial demixing system
In each iteration of TFMBSS, HPSS-based new mask calculation andmask smoothing are performed
After TFMBSS is converged, the estimated signals are obtained via
inverse STFT

Percussive
components
Harmonic
components

Harmonic estimateHPSSTimeEstimateandPercussive estimate

by iteratively minimizing smoothness cost function• TFMBSS [Yatabe+, 2019]

6The performance issaturated for morethan 9 iterations

421357911
13
Number of iterations in HPSS blocks

1520• Experiment 2: optimal smoothing parameter
16• HPSS-based new mask calculation

14

Two HPSS are independently applied to each of temporarily estimatedsignalsand
Two Wiener-like masks
andare constructed using the results ofHPSS

12

Linear multichannel BSS with plug-and-play source models
Source model is input as a time-frequency mask

Fewer iterations arenot preferable

80

SDR improvements [dB]

Frequency

Mixture1012101086

βold=04

βold=0.52βold=0.75

Average SDR imp. [dB]High separation quality because of utilization of spatial features

Masking process: entrywise productOptimization of TFMBSS is based on primal-dual splitting algorithm
Drastic change of masks in each iteration will cause instability ofparameter optimization (see experiment 1)
Introduce mask smoothing process based on weighted geometric mean
Mask calculated in
the current iteration

EntrywiseproductMask calculated in
the previous iteration

Our research aim

HPSS-based source model is effective, but its separation mechanism is
non-linear, resulting in the generation of artificial distortions
For multichannel signals, linear distortion-less separation can beachieved by estimating the spatial demixing system

We propose high-quality multichannel blind HPSS

Intensity of smoothing can be controlled byand00100
200300400500
Numbers of iterations of proposed method

00.5

0.75 0.875 0.9375

Smoothing parameter

Smoothing process can drastically stabilize the SDR behavior• Experiment 3: comparison with existing methods
Linear multichannel BSS
12Average SDR imp. [dB]

Generate time-frequency
mask based on temporalestimated sources• Mask smoothing using previous mask4

βold=0.9375-2

These masks enhance the harmonic or percussive components / byeliminating the other components

62

βold=0.875

08

10Non-linear
single-channel
6BSS
8Greatlyoutperforms
other existing
HPSS methods

420Single-channel Multichannel
HPSSHPSS

AuxIVA

ILRMA

Proposedmethod