3-Q-8
Joint Noise Suppression and Dereverberation Combining Frequency-Domain
Blind Signal Extraction and Multichannel Wiener Filter for Hands-Free Spoken
Dialogue System *○Fine Dwinita Aprilyanti, Hiroshi Saruwatari, Satoshi Nakamura (NAIST), Tomoya Takatani(TOYOTA)1

has also been proposed for this method that isbased on the assessment of generated musicalnoise as an effect of nonlinear postprocessing[7].
The method performs well in heavily reverberant
environment, but fails to achieve optimum speech
recognition performance when interferences are
not so severe.
In this paper, we propose improvement for thejoint noise suppression and dereverberation
method based on FD-BSE and multichannel WF.
This joint method takes advantage of the FD-BSE
capability to separate speech component fromnoise mixture. Assume that the speech extraction
is effective, the late reverberation is synthesized
and suppressed directly from the speech
estimation of FD-BSE. Therefore, the processing
path becomes shorter and target speech distortiondue to processing can be mitigated.
The remaining of this paper will be organized
as follows; Section 2 shows the sound mixing
model and review on FD-BSE. Section 3
describes the proposed method including
optimization scheme. The experimental result andASR performance evaluation are presented in
Section 4. Finally, we conclude our work in
Section 5.

Introduction

Hands-free spoken dialogue system provides
convenience and flexibility for user inhuman-machine interaction. This system uses anarray of microphones to capture user’s utterance
from the distance, and it is preferable to beimplemented in real environment rather thanclose-talking system, where the user must use
hand-held or body-mounted microphone.
However, the effect of interference sound, suchas background noise and reverberation, is more
severe in this system, thus degrades its
performance[1].
To solve this problem, many speechenhancement methods utilizing microphone arrayhas been studied. The blind signal separation(BSS) aims to separate the signal component, forexample by emphasizing on the statisticalcharacteristics of the component of each signal,
also known as independent component analysis(ICA)[2]. The blind spatial subtraction array(BSSA)[3]methodhasshownthat[4]frequency-domain ICA (FD-ICA)
performs
better in estimating the diffuse background noise
than the target speech. Hence, a combination
technique, which consists of FD-ICA as a noise
estimator and nonlinear postprocessing to2 Related Work
suppress the estimated noise, has been proved to
2.1 Sound Mixing Model at Microphone Arraybe effective in improving the target sound quality.
The observed signal
captured at theSeveral methods have been proposed tomicrophone array is given by
suppress the effect of both background noise and(1)(2)()
reverberation. Some of the authors have proposeda method that is based on frequency domain blind
whereandare the clean speech[3]signal extraction (FD-BSE) combined with twosource and noise, respectively, andandstages of multichannel Wiener filter (WF) as
indicate the early and late room
[6]
nonlinear postfilters . An optimization scheme
impulse responses. Most hidden Markov*周波数領域ブラインド信号抽出と多チャネルウィーナフィルタを用いたハンズフリー音声対
話向けブラインド雑音・残響抑圧。
○アプリリヤンティフィネ、猿渡洋、中村智（奈良先端大）、高谷智哉 (トヨタ)
日本音響学会講演論文集- 805 -

2013年9月

Figure 1 Block diagram of FD-BSE-based joint methodmodel (HMM) based speech recognizers arecapable to handle the effect of
up tocertain time delay
for example byapplying cepstral mean normalization.
Therefore, many speech dereverberation methods
focus on suppressing the effect of latereverberation instead of attempting to recover thenon-reverberant signal.
In time-frequency domain, we can simplify
the mixture model in each frequency bin without
explicitly separate the early and late reverberation
component, as given by
𝑿 𝑓 ≈ 𝑯𝜃 𝑆 𝑓
𝑵 𝑓(3)
where 𝑆 𝑓
is the clean speech component
and 𝑵 𝑓is a vector containing n componentsof diffuse background noise.
With no loss of generality, we can reformulate
the Eq. 2 as given by(4)
𝑿 𝑓
𝑨 𝑓 𝑺 𝑓 .
Here, we assume that 𝑆1 𝑆 𝑓
and[𝑆2 𝑓… 𝑆𝑛 𝑓 ] 𝑵 𝑓. It is also
realistic to assume that, in frequency bin, thespeech component is statistically independent tonoise component.
2.2 Frequency-Domain
BlindSignal
Extraction
Given the observation 𝑿 𝑓 , FD-BSE works
by applying extracting vector 𝑩 𝑓 in eachfrequency bin to obtain 𝑌 𝑓 , as given bŷ 𝑓𝑌 𝑓𝑋
𝑩 𝑓 𝑿 𝑓
(5)
On the contrary to the conventional FD-BSS,
FD-BSE algorithm only aims at extracting thespeech components from noise mixture, rather
than separating signal according to each source.
Therefore, the statistical independence withineach noise source is not required.
The extracting vector is updated using gradientdescent method to minimize the cost function
given by

日本音響学会講演論文集- 806 -𝐽(𝑩 𝑓 )12

𝐸{|𝑌 𝑓|}2(6)𝐸{|𝑌 𝑓 |2 } 1 .
(7)This cost function is minimized if the extracted
component has a modulus with a small mean anda large variance. This implies the sparseness ofthe modulus of extracted component, in the sensethat only a few of the values are significantly
large and the rest is close to zero. This is also
applied in speech and noise mixture, wherespeech component is considered sparser thandiffuse background noise that has more flat
distribution. Consequently, Eq. (6) is minimized
when the target speech component is extracted.

3

Proposed Joint Method

3.1 Main Algorithm
The capability of FD-BSE in suppressing
diffuse background noise has been confirmed inprevious work[5]. However, this method cannot
perform well in heavily reverberant environment.
Therefore, an additional postprocess is required
to improve its performance.
The block diagram of the proposed joint
method is shown in Fig. 1. In this proposed
method, FD-BSE is used to extract the target
speech component from the noise mixture.
Assume that the process is effective, the extracted
component will only consist of reverberant
speech. The next step is to synthesize andsuppress the late reverberation component fromthe target speech. The estimation of latereverberation can be separated into two task:estimating late impulse response
andclean speech signal.
In this method,
is approximated bygenerating synthetic tail from decayed Gaussian[8]
random variable, given by(8)𝑎𝑢 𝑒 − 𝜏−𝜏𝑑𝑑ln 106
2 𝑇60 −𝜏𝑑(9)2013年9月

where 𝑎 is the scaling factor. Consequently, thesynthesize ofrequires a priori
information of 𝑇60 .
The direct speech
is approximated bŷ 𝑓projecting back the ouput of FD-BSE 𝑋
tothe truncated FD-BSE filter. Then, according tô 𝑓equation (2), the estimate 𝑿
is obtainedby applying convolution in the time domain.
After that, a set of multichannel Wiener filter isapplied to suppressed the late reverberation
component, as given bŷ 𝑓𝑋

̂ 𝑓|𝑋|̂|̂(̂|𝑒(10))|2|2 +𝛽𝑅 | ̂𝐿(11)|2whereis a parameter for controlling thestrength of dereverberation.
3.2 Optimization Strategy for Joint Method
By combining FD-BSE and WF filters in theproposed joint method, we gain more flexibility
in suppressing the interferences, thus making themethod more robust to various acousticalconditions. On the other hand, it is required tofind the value of parameterto gain theoptimum performance of this method.
Since the proposed method is intended to be
implemented in hands-free spoken dialoguesystem, the method should achieve best
performance in terms of speech recognition
accuracy. One common measure to evaluate theperformance is word accuracy (WA), given by
WA

100 ×

𝑁− 𝐼+ +𝐷(12)𝑁

where 𝑁 is the number of words in the reference,
𝐼 is the number of insertions, 𝑆 is the number ofsubstitutions, and 𝐷 is the number of deletions.
In speech recognizer, a series of fixed size[𝑜1 … 𝑜 𝑇 ]acousticvectors
𝒐is
extracted from the output of joint method withparameter
through some feature extraction
process. During decoding, it attempts tohypothesize
thewordsequence
𝐖 [𝑤1 … 𝑤𝐾 ] which is the most probable togenerate the sequence 𝒐, as stated by
̂𝑾

日本音響学会講演論文集

arg max 𝑃 𝑾|𝒐𝑾.

(13)- 807 -However, the recognition system cannot
compute the posterior probability 𝑃 𝑾|𝒐directly. Instead, Eq. (13) is transformed into thefollowing form based on Bayes' theorem:
̂𝑾

arg max𝑾𝑃 𝒐 𝛽𝑅 |𝑾 𝑃 𝑾
𝑃 𝒐 𝛽𝑅(14)|𝑾 is the acoustic likelihood orwhere 𝑃 𝒐acoustic score, representing the probability thatfeature sequence 𝒐 is observed given that word
sequence 𝑾 was spoken, and 𝑃 𝑾 is thelanguage score, i.e., the a priori probability of aparticular word sequence 𝑾. The former term is
calculated from acoustic model, while latter term
is computed using a language model.
Since Eq. (14) is maximized with respect tothe word sequence 𝑾 for a given observed
sequence 𝒐 that is fixed, the denominator term
𝑃 𝒐can be ignored. Thus, the parameteris optimized by maximizing the likelihood of
acoustic model of speech recognizer, as writtenbŷR4arg max 𝑃 𝒐𝜷𝐑

R |𝑾 .

(15)

Experiment and Result

We used 25 utterances from female JNAS
database[8] as the source signals. An eight-channel
microphone array (inter microphone spacing of2.0 cm) was used to record the diffuse
background noise from the railway station, androom impulse response at various distance
between microphone and speaker at a large
lecture room. The estimated T60 value is 500 ms.
Speech is convoluted with room impulse
response, and was mixed with noise signal at
SNR 10 dB. The τd value is set to 75 ms. This
corresponds to the effect of room impulse
response that still can be handled by mostHMM-based speech recognizer.
The objective evaluation and recognition
results are shown in Fig. 2. For the recognitiontask, Julius 4.2[9] is used as the decoder andnoise-free early reverberated speech is used as
reference speech. We compare the output of
proposed scheme with the estimated speech from
FD-BSE and the former 2-stages joint method,which is optimized via higher-order statistics and2013年9月

recognition
condition.

accuracy

invarious

acousticalAcknowledgement
This work was partly supported by JST Core
Research of Evolutional Science and Technology(CREST), Japan.

Figure 2 Experimental result of proposed
method, input signal SNR = 10 dB. (a) NRR
result, (b) cepstral distortion, and (c) word
accuracy of recognition task.
acoustic likelihood[7]. We also include therecognition result from unprocessed signal.
From the result it is shown that while the NRR
result of proposed method is not optimum in mostuser distances, it succeeded to achieve the best
recognition result compared to other methods.
The shorter processing in the proposed method
results in lower distortion to the target signal,thus the proposed method can achieve higher
recognition performance compared to 2-stage
joint method.

5Conclusion

We have proposed a method to jointly suppress
diffuse background noise and late reverberationby combining FD-BSE and multichannel WF.
Experimental result confirmed the effectivenessof the proposed method in improving the日本音響学会講演論文集- 808 -References[1] R. Prasad, H. Saruwatari, K. Shikano,“Robots that can hear, understand and talk,”
Advanced Robotics, vol.18, no.5, 2004.
[2] P. Comon, “Independent component analysis,a new concept?,” Signal Process., vol. 36, pp.
287–314, 1994.
[3] Y. Takahashi, et al., “Blind spatial
subtraction array for speech enhancement innoisy environment,” IEEE Trans. Audio,Speech, Lang. Process., vol. 17, no. 4, pp.
650-664, May 2009.
[4] S. Ikeda and N. Murata, “A method of ICA
in time-frequency domain,” Proc. ICA 1999,pp.365-371.
[5] J. Even, et al., “Blind signal extraction basedspeech enhancement in presence of diffuse
background noise,” Proc. IEEE SSP, pp.
513-516, 2009.
[6] J. Even, et al., “Blind signal extraction based
joint suppression of diffuse backgroundnoise and late reverberation,” Proc.
EUSIPCO, pp. 1534-1538, 2010.
[7] F. D. Aprilyanti, et al.，“Optimization
scheme of joint noise suppression anddereverberation based on higher-orderstatistics,” Proc. APSIPA ASC, Dec. 2012.
[8] K. Lebart and J.M. Boucher, “A new method
based on spectral subtraction for speech
dereverberation,” Acta Acoustica, vol.87,pp.359--356, 2001.
[9] K. Ito et al., “JNAS: Japanese speech corpus
for large vocabulary continuous speech
recognition research,” Journal of Acoust.
Soc. of Japan, vol. 20, pp. 196-206, 1999.
[10] A. Lee, T. Kawahara, and K. Shikano,“Julius - An open source real-time large
vocabulary recognition engine,” Proc.
Eurospeech, pp.1691–1694, 2001.

2013年9月