Part II.
Multichannel blind inverse filtering䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎕䎚Two approaches for signal dereverberation
Signaldereverberation

Reverberation
suppression

Partialdeconvolution“Robust” blindinverse filteringis the main topic
of part II[Lebart 2001], [Habets 2005],[Löllman 2009], [Erkelens 2010],[Kameoka 2009], [Jeub 2010]
and others䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎕䎛

Multichannel inverse filtering
Reverberant Inverse
speechfilterCleanspeech

st

h
h

(1)
t
( 2)
t
(M )
t

w

x

w

(1)
t

( 2)
t

h

Linear filtering:

x

(1)
t

RIRs

( 2)
t

w
K

(m) (m)w¦¦ W xt W
m 1Ǽ 0

Goal: estimate {w (m ) } s.t.
t

yt

(M )
t

x
yt

+

侇
侇
侇

(M )
t
M

Dereverberated
signalyt䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊stm : mic. index
t : time index{} : a set ofvariables
for all t and m䎕䎜Part II. Multichannel blind inverse filtering- Example applications- Professional audio post production- Meeting recognition with microphone arrays- Fundamentals: dereverberation with inverse filtering- What is inverse filter- Robust ‘approximate’ inverse filter- Blind inverse filtering- Overview of basic approaches- Closer look: multichannel linear prediction with
time-varying source model- Integration with blind source separation䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎖䎓

Application to audio post-production[Movies/TV creation]
Step1:
Sound&video recording (on location)
Actor/actress

Step2:
Audio post-production(de-noising, de-reverb, sound effects)

Microphone(s)䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎖䎔Dereverberation system for audio post production [Kinoshita 2008]
䯂Dereverberation plug-in for Pro Tools: NML RevCon-RR(sold by TAC System, Inc.)䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎖䎕

Online meeting recognition [Hori 2012]
Simultaneous
SpeechReverberation

Recognize speech andother audio events

Backgroundnoise

Who Spoke When, What,
To-whom and How?
Show&Tell:
ST-3.2: Thursday,
March 29, 10:30-12:30
䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊

Real-time Meeting Browser
䎖䎖Online/offline processing flow of meeting recognition
Mic signals

Preprocessing
for all following
signal processing
units

Dereverberation
Dereverberated
microphone signals
Voice
activity
detection
Separated
signals

Speechseparation

Cleanedsignals

Noise
suppression䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊Wordsequence
ASR䎖䎗Baseline:
Distant microphone(w/o enhancement)
w/o derev:
BSS+denoisew/ derev:
derev.+BSS+denoise
Headset:Close microphone(w/o enhancement)

27.4 % Headset

35.9 % w/ derev

38.0 % w/o derev

Baseline 78.9 %30.6 % Headset

56.6 % w/ derev

w/o derev 72.1 %90
Word 80
error 70rate 60(%) 50
40
30
20
10
0

Baseline 86.5 %ASR performance w/ and w/o dereverberation

Offline processing
Online processing(Latency=1s for preprocessing, (w/ unsupervised
speaker adaptation)
w/o speaker adaptation)

Test data:
Meeting by 4 speakers (15 min x 8 sessions)
Recording: 8 mics. (T60: about 350 ms, Speaker-mic distance: 100 cm)
Acoustic model:
Trained on CSJ (corpus of spontaneous Japanese): headset recording
Language model:
Vocabulary size: 156K (LVCSR)
䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎖䎘Questions to be answered

st

ht(1)
ht( 2 )
(M )
t

h

xt(1)

wt(1)

x

wt( 2 )( 2)t

xt( M )+侇侇侇( M )

yt
Dereverberated
signalwt• What is inverse filtering ?
• Is the inverse filter robust against interferences ?
• Can we estimate the inverse filter with blind
processing ?

䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎖䎙

Answers at a glance
What is inverse filtering ?

Inversion of room impulse responses (RIRs)
Is the inverse filter robust against interferences ?

Unfortunately no,
but there is a robust ‘approximate’ inverse filter
Can we estimate the inverse filter with blind processing ?

Yes, we can,by using cues for distinguishing speech from RIRs
䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎖䎚

Part II. Multichannel blind inverse filtering- Example applications

Assume non-blind- Professional audio post production processing for- Meeting recognition with microphonearrays purpose
analysis- Fundamentals: dereverberation with inverse filtering- What is inverse filter- Robust ‘approximate’ inverse filter- Blind inverse filtering- Overview of basic approaches- Closer look: multichannel linear prediction with
time-varying source model- Integration with blind source separation䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎖䎛Inversion of RIRs = Inversion of matrix transformation
Reverberant speech
Clean
speechst

Dereverberated
speechxt(1)RIRs

Viewed as
matrixtransformation

(M )
t

x

yt

Inversefiltering

Inversion
Viewed as
matrix inversion

䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎖䎜Matrix/vector representations of RIR convolution/filtering
Single channel RIR convolutionª xt( m ) º« (m) »
« xt 1 »«  »« (m) »«¬ xt  K »¼

x

Multichannel RIR convolutionª h0( m ) h1( m )  hL( mh ) 0  0 º ª s t º»« s »
«
(m)
(m)
(m)

h
h

0
h
Lh
1
0
» « t 1 »««    
 0 »«  »»«(m) » «
(m)
(m)
h
s
h
0

h
0

«¬
Lh »
1
0
¼ «¬ t  K 0 »¼

(m)t

x t( m )H
H ( m )s t(m)K

K 0  Lhª H (1)«« « H (M )
¬

xt

H

xt

ª xt( m ) º
K
»
(m)
(m) «
(m) (m)w0 ,  , wK «  »wW xt W¦
W 0
« xt(mK) »( m) T¬¼
w
Tw ( m ) x t( m )x t(m )@

M¦w

( m )Tm 1

䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊

x

(m)
t

yt

º
»
»s t
»
¼

Hs t

Multichannel filtering

Single channel filtering>stª x t(1) º
«
»
«  »
« x t( M ) »
¬
¼

>w

(1) T

T

w xt
䎗䎓

ª x t(1) º»( M )T « w«»(M ) »
«
x
¬ t ¼
wT
xt

@

Existence of inverse filter

st

xt

yt
Hs t
yt w x t• A column vector w is an inverse filter when it satisfies:
T
T
s
[
s
,
s
,

,
s
]
st yt w Hs t where t t t 1
tK

xt

T

0

T

w H

e

T where e[1,0,  ,0]T

• An inverse filter w exists, when H is invertible, i.e.,
it is full column rank, and w is obtained as

w

T

T

e H



where

H

䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊( H T H ) 1 H T

䎗䎔M (#mics) > 1 is required for single source case
• H is invertible, or full column
rank, if and only if

H

H (1)

(#rows of H) >= (#columns of H)
and all columns are linearly
independent• In the case of single source,(#rows of H) >= (#columns of H)
is satisfied if and only if

H ( 2)
H (M )#columns

M (#mics) > 1

䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊

䎗䎕

#rows

Generalization to N sources 䐙 M microphones caseMultiple-input/output inverse theorem (MINT) [Miyoshi 1988]sst( N )
st(1)(t 2 )

H

xt((12))xt
( M )xtWyt((12) ) st(1()2 )
yt
st

(N )

yt

s

(N )
t

– Inverse filter exists when H is full column rank
Equivalent• M (#mics) >N (#sources)• H(z) does not contain common zeros䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎗䎖

Part II. Multichannel blind inverse filtering- Example applications- Professional audio post production- Meeting recognition with microphone arrays- Fundamentals: dereverberation with inverse filtering- What is inverse filter- Robust ‘approximate’ inverse filter- Blind inverse filtering- Overview of basic approaches- Closer look: multichannel linear prediction with
time-varying source model- Integration with blind source separation䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎗䎗Problem of inverse filtering
Assumptions for inverse filtering• Invertible RIRs
• No additive noise• Time-invariant RIRs

Not realistic !

Inverse filter is too sensitive tomodeling errors (noise or RIR change)䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎗䎘Inverse filter greatly amplifies noise
Noise-free reverberant case
• Clean speech• Reverberant speech– Synthesized using a fixed RIR (RT60=0.5 s)• Dereverberated speech using an inverse filterfor known RIRs (2-channel)
Noisy reverberant case
• Noisy reverberant speech (SNR=30dB)• Speech processed using the same inverse filter(2-channel)䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎗䎙Why inverse filter is so sensitive to additive noisestH

xt  nt

w

T

T

e H



where

Minimum singularvalue of H

Maximum singularvalue of H 

Omin

inv
max
often extremely
large

O

often extremely
small

~
nt

T



e H ntExtremely
amplifies
noise(compared to maximum
singular value)䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊~st  nt䎗䎚Standard numerical approach for robustness [Engl 1996]•Regularization

Noisy rev.

Processed– A general technique for robust matrix inversion
T
H
H• Add a very small positive constant G to diagonal offor calculating the pseudo-inversion of H

H



T

1

(H H ) HT~
H( H T H ǬI ) 1 H T

I 䰆Identity matrix– It can reduce the maximum singular value Oinv
max
~

of H
Noise amplification is greatly mitigated䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎗䎛Room acoustics motivated approach for robustness• Channel shortening Noisy rev.

Processed– Set “direct signal + early reflections” as target signal, andreduce only late reverberation
TRR
Illustration of an RIR
Target e.g., -3 dB
DirectInverseEarly
pathfilteringreflections h e
e
wT

Late
reverberationhl

about 30 mst~eT H 

ChannelshorteningwTe.g., 8 dB~
(eT  h Te ) H 

Target to reverberation ratio (TRR) w/ channel shortening
is much higher than TRR w/ inverse filtering䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎗䎜Intermediate summary II-1
• Dereverberation: inversion of RIRs
– Assuming RIRs to be a time-invariant linear system• Inverse filter exists– When we have more microphones than sources– But it may be very sensitive to additive noise• ‘Approximate’ inverse filter is robust against noise– Based on regularization and channel shortening䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎘䎓

Part II. Multichannel blind inverse filtering- Example applications- Professional audio post production- Meeting recognition with microphone arrays- Fundamentals: dereverberation with inverse filtering- What is inverse filter- Robust ‘approximate’ inverse filter- Blind inverse filtering- Overview of basic approaches- Closer look: multichannel linear prediction with
time-varying source model- Integration with blind source separation䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎘䎔Blind inverse filtering based dereverberation
EstimateinversefilterUnknown
Speechproduction
system

Unknown

st

RIRs

Clean
speech

ŵ

Mics.



xt(m )

InversefilteringReverberant
speechyt
Dereverberated
speechTwo approaches• RIR estimation + RIR inversion• Direct estimation of inverse filter䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎘䎕

Conventional decorrelation approaches for stationary white signal^ ` 0

E st s t 'Increasecorrelation E xt( m ) xt('m )^for t z t 'Unknownst

RIRs

Stationary
white signal

`

z0



Approach:
Estimate w
thatdecorrelates

Estimateinversefilterŵ

xt(m )x(m)t

Inversefilteringyt E ^yt yt ' ` 0• SOS approach assumes st to be stationary white GaussianMultichannel linear prediction (MCLP) [Slock 1994], [Abed-Meraim 1997]• HOS approach assumes st to be an i.i.d. sequence
Higher order decorrelation [Sato 1975], [Bellini 1994]
䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎘䎖

Multichannel linear prediction (MCLP)

x

(m )
t W

Predict(1)rreverberation t
in observation

M Lx

c x
¦¦
W
m1

r

(m) (m)Wt W(1)t : reverberation

1

Past observation
Current
observation

Mic.1

x
䱏䱏䱏

䱏䱏䱏

(1)
t

Mic.M
Time
䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎘䎗st  rt(1)

MCLP based decorrelation [Slock 1994], [Abed-Meraim 1997]• x(1)
t is modeled by
M K(1)(m) (m)tt W
W
m 1W 1x¦¦c x  st

Predicted signal (= reverberation)Tt 1

x c  stTt 1

st x  x c(1)t

Prediction error (= direct signal)

(1)
(1)
(M )
(M ) T
c
[
c
,

,
c
,


,
c
,

,
c
where
1
K
1
K ] is prediction coeffs.

– c is equivalent to inverse filter w• c can be estimated by minimizing prediction error
when sources are stationary and uncorrelated in time
2
T
t 1

cˆ argmin ¦ x  x c
c

t

(1)
t

– Quadratic form: optimized using a closed form solution䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎘䎘Why dereverberation can be achieved by MCLP
2

§
·
(1)
T
¨¨ ¦ hW st W  xt 1c ¸¸x  c xt 1¦¦
t 1
t 1 ©W 0
¹
2
·
§
(1)
T
¨¨ st  ¦ hW st W  xt 1c ¸¸
¦
t 1©
W 1
¹
2
§
·
2
(1)
T
| st | ¦¨¨ ¦ hW st W  xt 1c ¸¸¦
t 1
t 1 ©W 1
¹
(1)
t

T

2

t ¦| st |2
t 1

h0(1) 1 is usuallyassumed for MCLP
without loss of generality

¦s s
t 1

t t'

0 for t z t '(and thus¦ st x Tt1

0)

t 1

Minimization is achieved only whenTruereverberation(1)h¦ W st W=

Predicted
reverberationxTt1cW 1䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎘䎙Robustness of MCLP against noiseLetzt( m )

xt( m )  nt( m ) be noisy reverberant observation.
Additive noise (or can be viewed as modeling error)

Assume xt and nt to be uncorrelated,
then, the cost function becomes
(m )

(m )

¦ z z c
t 1

(1)
t

T
t 1

2

¦ x x c ¦ n n c
t 1

(1)
t

Cost function for
dereverberation

T
t 1

2

t 1

T
t 1

(1)
t

2

Cost function fornoise amplification

Regularization is inherently included䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎘䎚Problem of decorrelation approach for speech dereverberation
Both are decorrelated

Unknown

Unknown

Speechproduction
system

RIRs

st

ŵ

Mics.



Problem:Not only dereverberate xt(m)but also decorrelate st

Approach:
Estimate w
thatdecorrelates

Estimateinversefilterxt(m )x(m)t

Inversefilteringyt

Key to the solution:
Use cues
to separate
speech and RIRs

䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎘䎛

Cues for separating speech and RIRs

Cues

SpeechRIRs

Inter-channel
difference

Common to all the
microphone signals

Different for eachmicrophone

Auto-correlation Correlated only within Correlated within longshort time interval of time interval overduration
100 msthe order of 30 ms
Nonstationarity

Stationary only withinshort time period of
the order of 30 ms䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊

Stationary over longtime period of theorder of 1000 ms orlarger䎘䎜

Approaches to blind inverse filtering
Cues• Subspace method (RIR estimation + inversion)

Inter-channel
difference–[Furuya 1997], [Gannot 2003], [Gaubitch 2006]• Pre-whitening + decorrelation–Second-order statistics (SOS): [Gaubitch 2003],[Furuya 2007], [Triki 2007]–Higher-order statistics (HOS): [Gillespie 2001]

Auto-correlation
duration• Channel shortening–[Gillespie 2003], [Kinoshita 2009]

Auto-correlation
durationandnonstationarity• Joint speech and reverberation modeling–[Hopgood 2003], [Buchner (TRINICON) 2010],[Yoshioka 2007], [Nakatani 2008]䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎙䎓Pre-whitening + decorrelation

Reverberant
speechxt

Hst

Estimate w
thatdecorrelates

Estimateinverse filter
Pre-whitening
Reducecorrelation~
within shortxt(m)time interval

ŵ
H~st

( m)
~
xt
~
yt(m)

Inversefilteringŵ
Inversefilteringyt(m)• A typical method for pre-whitening–Low-dimensional (e.g., 12-dim) single channel linear prediction often used

Hs t , andAssumption: pre-whitening can decorrelate only s t in x t~~~
we can obtain xt
H st where st is an unknown decorrelated speech䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎙䎔Channel shortening• Introduce constraints so that dereverberation
reduces only late reverberation
Directpath

Early
reflections

ChannelshorteningLatereverberation

Make derev. robust and do not decorrelate speech• Techniques:– Correlation shaping [Gillespie 2003]– Multistep MCLP [Kinoshita 2009]䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎙䎕

Multistep MCLP [Gesbert 1997], [Kinoshita 2009]

x

(m )
t W

Predict late(1)
reverberation rt
in observation

Past observation

M

K

m1

D

rt(1) : late

c x
¦¦
W

(m) (m)Wt Wreverberation

Delay D (=30-50 ms)
Current
observation

Mic.1
䱏䱏䱏

䱏䱏䱏

xt(1)

st  rt (1)st : direct signal+ early
reflections

Mic.M

Time
䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎙䎖

Approaches to blind inverse filtering
Cues• Subspace method (RIR estimation + inversion)

Spatial diversity–[Furuya 1997], [Gannot 2001], [Gaubitch 2006]• Pre-whitening + decorrelation–Second-order statistics (SOS): [Gaubitch 2003],[Furuya 2006], [Triki 2006]–Higher-order statistics (HOS): [Gillespie 2001]

Duration of
auto-correlation• Channel shortening–[Gillespie 2003], [Kinoshita 2006]

Duration of
auto-correlation
andnonstationarity• Joint speech and reverberation modeling–[Hopgood 2003], [Buchner (TRINICON) 2010],[Yoshioka 2007], [Nakatani 2008]䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎙䎗Joint speech and reverberation modeling for derev.
Model of generative system

Sourcemodel

Reverberation
model

xt ~ p ( xt ;T ) T^T s ,T h `Parameter estimation by

Parametricmodel T

Parametricmodel T

䯂Likelihood maximizationh[Hopgood 2003], [Yoshioka 2007],[Nakatani 2008]

Distinguishable ?

䯂Kullback-Leibler
divergence minimizations[Buchner (TRINICON) 2010]

Sourceprocess

Reverberation
process

xt

Reverberant
observation

Unknown true generative system䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎙䎘Models for source process and reverberation process

Source model(SOS or HOS)

Reverberation
model

Time-varying&
Correlated
only withinshort interval

Stationary&
Correlated
overlong intervalDistinguishable䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎙䎙

Multichannel blind partial deconvolution (MCBPD) by TRINICON
Cost function for SOS-TRINICON [Buchner 2010]

J SOS¦ ^log det Rˆ  log det Rˆ `t

Autocorrelation
matrix ofobserved signal

s ,t

Goal

y ,t

Autocorrelation matrix of yt

Goal R̂ s,t
Decorrelation(deconvolution)

MCBPD by
TRINICON䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎙䎚

Part II. Multichannel blind inverse filtering- Example applications- Professional audio post production- Meeting recognition with microphone arrays- Fundamentals: dereverberation with inverse filtering- What is inverse filter- Robust ‘approximate’ inverse filter- Blind inverse filtering- Overview of basic approaches- Closer look: multichannel linear prediction with
time-varying source model- Integration with blind source separation䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎙䎛

MCLP with time-varying source model for dereverberation

Reverberation
process

Source process(SOS)
Time-varying
short-time
Gaussian

Distinguishable bylikelihood
maximization

MCLP[Yoshioka 2007][Nakatani 2008, 2011]䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎙䎜

Reformulation of MCLP based on likelihood maximization
L (c )

log p x ({x t }; c)

Conditional
probability rule

(1)
log
p
(
x
¦ x t | {xt '}t ' 1:t 1;T )  const.
t 1

xt(1)  x Tt1c¦ log ps ( st )  const . where st
t 1

xt(1)

xTt1c  st

Source model

Assume p s ( st )L (c )N ( st ;0,1) (stationary white Gaussian), then(1 / 2)¦ | xt(1)  cT x t 1 |2  const .
t 1

max L (c)cMaximize likelihood

min ¦ | xt(1)  cT x t 1 |2ct 1

Minimize prediction error䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎚䎓

Time-varying Gaussian source model (TVGSM)
30 ms order

1. Each short time segment st [ st st 1  st  N 1 ]T is
stationary multivariate Gaussian, which can becharacterized byp s st ; R t
where R t
2.

N st ;0, R t^ ` is an autocorrelation matrix

E st stTR t varies over different time segments

Ts^R t ` : parameters to be estimated䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎚䎔

MCLP with multivariate source model

x

(1)
t

T
t 1

x c  stxt(1)ª xt(1) º« (1) »
xt 1 »«
Xt 1c  st or«  »« (1) »«¬ xt  N 1 »¼
x

(1)
t

ª xTt1 º ª st º»
« T » «
« xt 2 »c  « st 1 » 30ms
«  » «  » order»« T » ««¬xt  N »¼ «¬st  N 1 »¼

st

Xt 1

• Prediction error s is assumed to follow TVGSMt䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎚䎕Likelihood function of MCLP with TVGSM¦ log p (s ; c,{R })L (c, {R t })

s

t

t

t

 X t 1c and p s ( st ; R t )N ( st ;0, R t )where stx

L ( c , { R t })

 ¦ || x t(1)  X t 1c || R t  log | R t |

(1)
t

t

Normalization term

Prediction error
weighted by R 1twhere || s || RT1

s R s (quadratic form)䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎚䎖

Iterative optimization procedure
InitializeˆRt
Closedform

E{x Tt x t }Update prediction coeffs. ĉcˆ

arg min ¦ || x t(1)  X t 1c ||Rˆ
c

t

ĉ

t

ˆ }
{R
t

A few iterations
are sufficient for
convergenceˆ }Update source model {Rt
1. Dereverberate{x t }sˆt

x t(1)  X t 1cˆˆRt

E{sˆtT sˆt }2. Calculate autocorrelation matrix ofŝt{sˆt }
䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎚䎗Importance of time-varying source model

Frequency 䋨kHz䋩

Source signal

Observation

T60 : 0.5 s
Source-mic
distance: 1.5 m# mics : 2
Processed A(A) MCLP withstationary white
Gaussian source
model

Processed B

Recording: 2.5 s

A few seconds ofobservation are
sufficient for
dereverberation(B) MCLP with
TVGSM䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎚䎘Blind inverse filtering works in noisy environments
Noisy reverberant speech# mics: 8
source-mics distance: 2 m

SNR
15 dB
10 dB

5.8 dB(T60 =0.39s)

15 dB
10 dB

0.1 dB(T60 =0.65s)

Dereverberation(w/ multistep MCLP
w/ TVGSM)

TRR*

Reducereverberation

Noise: additive white noise(reproduced andrecorded by 8 mics)

Processed signal

TRR
15.2 dB
13.8 dB
11.4 dB
10.3 dB
Noise may slightly increase,but not significantly*TRR: Target-to-reverberation ratio (target = direct signal + early reflections)䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎚䎙Computationally efficient implementation• Subband decomposition approach [Nakatani 2010],[Yoshioka 2009b]{xn( m,1) }{ yn ,1}

{x

(m )
t

}

Subbandanalysis{xn( m, 2) }{xn( m, F) }MCLP with TVGSM
MCLP with TVGSMMCLP with TVGSM{ yn , 2 }{ yn ,F }Subbandsynthesis• Computational efficiency largely improves

Real-time factor (RTF)
using MATLAB(RT60: 0.5 s, # mics: 2)

Time-domain

Subband170

0.8䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎚䎚{ yt }

Processing flow with subband decomposition [Nakatani 2010]
1.Set analysis parameters
D : prediction delay (D should be # of subband

samples corresponding to 30 ms, or larger)
L : length of prediction filter, M : # of mics,
m0 : index of target channel to be dereverberated
D : a coeff. for flooring constant (e.g.,D 10 4 )2.Decompose a multichannel observed signal
into a set of subband signals

xn( m, f) : subband signal (e.g., [Weiss 2000], or STFT
can also be used)m : channel index, n : sample index
f : subband index
E.g., # of subbands is 512 (including negative
frequencies) for 16 kHz sampling

3.In each subband f , set initial estimates ofsource variance V n, f as
H k D max | xn( m, f0 ) |2n

V n, f^max | xn( m, f0 ) |2 , H f`

where H f is a flooring constant for V n, f(m)

4.Obtain vector representation of xn , f in all
channels as
TTT

x n, f[ x (n1,)f , x (n2, )f ,  x (nM, f ) ]T

where T is non-conjugate transposition, andx (nm, f)[ xn( m, f) , xn( m1), f ,  xn( m L) 1, f ]T

5.In each subband f, iterate the following untilconvergence is achieved
i. Obtain prediction filter c f ascf

§ x n D, f x¨¦¨ n
V n, f©

*Tn D, f·
x n  D , f xn( m, f0 )¸ ¦¸ n
V n, f¹*

where  and* are Moore-Penrose pseudoinverse and complex conjugate operations.
(see [Yoshioka, 2009b] for efficient calculation)
ii. Obtain dereverberated subband signal y n, f
as
(m )
*T

y n, fxn , f0  c f x n  D , fiii. Update source variance estimates V n, f as

V n, f^max | y n , f |2 , H f`

6.Compose a dereverberated signal from a set of
dereverberated subband signals y n, f䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎚䎛

Part II. Multichannel blind inverse filtering- Example applications- Professional audio post production- Meeting recognition with microphone arrays- Fundamentals: dereverberation with inverse filtering- What is inverse filter- Robust ‘approximate’ inverse filter- Blind inverse filtering- Overview of basic approaches- Closer look: multichannel linear prediction with
time-varying source model- Integration with blind source separation䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎚䎜BSS+dereverberation

Reverberation
Clean
speechDirectsignalClean
Integrated
speechMixedreverberant
speechReverberant
speechBSS+
estimate
dereverberation

Reverberation

Approaches:• MCLP based approach [Yoshioka 2009b, 2011]• TRINICON [Buchner 2010]䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎛䎓

Generative model for reverberant sound mixture
Source process 1
Time-varying s
Gaussian(1)t

Source process 2( 2)
Time-varying st
Gaussian

Mixture process

 (1)
xt

Reverb. processxt(1)Instantaneous ( 2 ) Multi-input
xt
mixingmulti-output
processMCLP

xt( 2 )

䯂䯂䯂䯂䯂䯂 (m )
xt : non-reverberant mixture
xt(m ) : reverberant mixture

Jointly optimized by maximum likelihood
estimation approach [Yoshioka 2009b, 2011]䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎛䎔

Optimization procedure (subband-based implementation)

Initialization
Compute source estimates䎦䏄䏏䏆䏘䏏䏄䏗䏈 䎃䏖䏒䏘䏕䏆䏈䎃䏓䏒䏚䏈䏕䎃 䏖䏓䏈䏆䏗䏕䏄䏅䏄䏖䏈䏇 䎃䏒䏑䎃䏈䏖䏗䏌䏐䏄䏗䏈䏖 䎃䏒䏉 st(m ) .

Update source model parameters
Update de-mixing matrices
Update prediction coefficients

Converged ?

䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎤䏓䏓䏏䏜 䎃䏄䎃䏆䏒䏑䏙䏈䏑䏗䏌䏒䏑 䏄䏏䎃䎥䎶䎶䎃䏗䏒䎃䏈䏖䏗䏌䏐䏄䏗䏈䏖 䎃䏒䏉 xt(m ) .

Closed-form optimization

䎛䎕

2
0

Dereverb + BSS

BSS

16
SIR
14
[dB]
12
10
8
6
4

Unprocessed

Improvement in signal-to-interference ratio (SIR)

T60=0.3 s# sources: 2# mics: 4
Source-mic
distance: 1.5 m
Recording: 1 to 8 s(average: 3.5 s)
BSS: [Sawada 2007]

T60=0.5 s

Results averaged over 672 pairs of utterances (TIMIT test set)䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎛䎖Live demo䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎛䎗TRINICON: general framework for blind MIMO signal processing
s



x (1)

(1)

Unknown
mixingsystem



s(P)

x

(M )

y (1)

Unmixing
system Wb



y (P)

b: index of signal blocks

Source models
Cost function [Buchner 2010]

J ( Wb )

f

N0

i 0

j 0

^

`

¦ E (i, b)¦ log( pˆ s , PD (y (i, j )))  log( pˆ y , PD (y (i, j )))with PD-variate pdfs (P: source number, D: filter length)• pˆ s , PD (y (i, j )) for source (assumed or estimated)• pˆ y , PD (y (i, j )) for output
䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎛䎘Comparison of SOS and HOS by TRINICON [Buchner 2010]
Signal-to-reverberation ratio(SRR) improvement (dB)

SIR improvement (dB)
SOS+HOSSOS+HOSSOSSOS

BSS (w/o derev.)

Number of iterations

Number of iterations# mics.: 4, # sources: 2, T60 : 700 ms,
Source-mic distance: 1.65 m, Recording: 30 sec
䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎛䎙

Summary II-2
• Robust blind inverse filtering is possible– Using joint speech and reverberation modeling• Based only on a few seconds of observation (e.g., 2.5 s)• With a relatively small computational cost (e.g., RTF<1)• In an online processing manner (e.g., latency=1s)– Under low SNR conditions (e.g., 10 dB SNR)

• Future challenges– Realtime adaptation of inverse filter [Yoshioka 2009a],[Evers 2011]– Single channel inverse filtering [Gillespie 2001]– Processing under more adverse noise conditions such as
nonstationary diffuse noise– Optimal integration of inverse filtering and spectral
enhancement based dereverberation䎱䏄䏎䏄䏗䏄䏑䏌䎏䎃䎶䏈䏋䏕䎏䎃䎮䏈䏏䏏䏈䏕䏐䏄䏑䏑䎝䎃䎵䏈䏙䏈䏕䏅䏈䏕䏄䏑䏗䎃䎶䏓䏈䏈䏆䏋䎃䎳䏕䏒䏆䏈䏖䏖䏌䏑䏊䎛䎚