1-P4-19
Joint Suppression of Background Noise and Late Reverberation Combining BlindSpeech Extraction and Generalized MMSE-STSA Estimator*☆Fine Dwinita Aprilyanti, Hiroshi Saruwatari, Satoshi Nakamura (NAIST)
1Introduction

2

Hands-free automatic speech recognition(ASR) system provides more convenience andflexibility to the user under many conditions.
However, the presence of background noise androom reverberation, particularly the latereverberation component, significantly degrade
the speech recognition performance. Thus, arraysignal processing is required as the front-end forsuch a system.
One of the microphone array processing
techniques is frequency-domain blind source
separation (FD-BSS), which separates signal
components according to the statisticalindependence of each source, e.g., by using
independent component analysis (ICA)[1].
FD-BSS does not require a priori information
about the input signal; however, it cannot
perform well in the presence of non-point source
noise[2].
Frequency-domain
blindsignal[3]
extraction (FD-BSE) to effectively extracts
speech from a mixture of speech and noise byutilizing the difference in sparseness between
their modulus. However, the reverberation effectis not considered in this method.
In this paper, we utilize a nonlinear
postprocessing method to extend the capability ofFD-BSE for dereverberation. We introduce theminimum mean-square error short-time spectral
amplitude (MMSE-STSA) estimator[4] and its
generalized method[5] to suppress the latereverberation component in addition to diffuse
background noise. The related works arereviewed in the next section. In Section 3, we
describe our proposed method including theoptimization
strategy.
Theexperimental
evaluation and a discussion of the results aregiven in Section 4, followed by conclusion inSection 5.

Related Works

2.1 Frequency-Domain
BlindSignalExtraction
The time-frequency domain model of thesignal captured by a microphone array 𝑿(𝑓, 𝑘)
at the fth frequency bin is given by
𝑿(𝑓, 𝑘) = 𝑨(𝑓)𝒁(𝑓, 𝑘),(1)where 𝑨(𝑓) is the mixing matrix. Without lossof generality, we may assume that the first-row
component of 𝒁(𝑓, 𝑘) is the contribution ofspeech and its reverberation 𝑿𝑆 (𝑓, 𝑘), and theremainder is the contribution of diffuse noise𝑿𝑁 (𝑓, 𝑘). We may also assume that the speechcomponent is statistically independent of thenoise component.
In FD-BSE, the extracted output 𝑌(𝑓, 𝑘) isobtained by applying an extracting vector to theobserved signal, as given by
𝑌BSE (𝑓, 𝑘) = 𝐵(𝑓)𝑿(𝑓, 𝑘)= 𝐵 (𝑓)𝑨(𝑓)𝒁(𝑓, 𝑘). (2)The vector 𝐵(𝑓) is updated using the gradientdescent method to minimize the cost function
𝐽(𝐵 (𝑓)) given by
1

𝐽(𝐵(𝑓)) = 2 𝐸{|𝑌BSE (𝑓, 𝑘)|}2 ,(3)(4)
𝐸{|𝑌BSE (𝑓, 𝑘)|2 } = 1.
The speech modulus has a sparser distribution
than the diffuse background noise as most of its
values are close to zero and only a few are
significantly large. Thus, the cost function
becomes minimum when the target speechcomponent is extracted.
2.2 Generalized MMSE-STSA Estimator
The generalized MMSE-STSA estimator isalso referred to as MMSE estimation with
optimizable speech model and inhomogeneous
error criterion (MOSIE) estimator [5]. In MOSIE,the probability density function of the cleanspeech power spectrum is modeled by a chi*

ブラインド信号抽出と一般化 MMSE-STSA 推定器を用いた背景雑音・後部残響抑圧☆アプリリヤンティフィネ、猿渡洋、中村哲（奈良先端大）日本音響学会講演論文集- 789 -

2014年3月

Figure 1 Block diagram of FD-BSE combinedwith single channel MOSIE estimator.

Figure 2 Block diagram of FD-BSE combinedwith multichannel MOSIE estimator.

distribution as23𝜌𝜌𝑝(𝑠) = Γ(𝜌) (𝑃 (𝑓))𝜌 𝑠 2𝜌−1 exp (− 𝑃 (𝑓) 𝑠 2 ),𝑆𝑆(5)where 𝜌 is the shape parameter of the speechmodel, 𝑃𝑆 (𝑓) is the mean of speech spectral
amplitude, and Γ(𝑎) is the complete Gammafunction. Also, 𝜌 = 1 indicates a Gaussianspeech signal[4]. In this paper, we employ
0 < 𝜌 < 1 to model the speech power spectrum
as a super-Gaussian signal.
The MOSIE estimator is mainly used forsingle-channel noise suppression by applying thegain function 𝐺 (𝑓, 𝑘), as given by
𝑌MOSIE (𝑓, 𝑘) = 𝐺 (𝑓, 𝑘)𝑋(𝑓, 𝑘),(6)√𝜐(𝑓, 𝑘)𝐺 (𝑓, 𝑘) =.
𝛾̂(𝑓, 𝑘)[𝜐(𝑓, 𝑘) =𝛽2𝛽2𝛾(𝜌+ ) Φ(1−𝜌− ,1,−𝜐(𝑓,𝑘)𝛾(𝜌)1/𝛽. Φ(1−𝜌,1,−𝜐(𝑓,𝑘)) ]𝜉̂ (𝑓,𝑘)𝛾̂(𝑓, 𝑘),1+𝜉̂ (𝑓,𝑘)

(7)
(8)

where Φ(𝑎, 𝑏, 𝑐) is the confluent hypergeometricfunction. 𝜉̂ (𝑓, 𝑘) and 𝛾̂(𝑓, 𝑘) are the estimateda priori and a posteriori signal-to-noise ratio(SNR), respectively, as given by𝜉̂ (𝑓, 𝑘) = α𝛾̂(𝑓, 𝑘 − 1)𝐺 2 (𝑓, 𝑘 − 1) +(1 − 𝛼 ) max[𝛾̂(𝑓, 𝑘) − 1,0],(9)|𝑋(𝑓,𝑘)|2𝛾̂(𝑓, 𝑘) = |𝑋̂ (𝑓,𝑘)|2 ,𝑁(10)

where α is the forgetting parameter in thedecision-directed approach, and 𝛽 is thecompression parameter of the error function
given by𝛽e (𝑆𝑜 (f, k), 𝑆𝑝 (f, k)) = |𝑆𝑜 (f, k)|𝛽 − |𝑆𝑝 (f, k)| ,(11)where 𝑆𝑜 (f, k) and 𝑆𝑝 (f, k) are the original andprocessed
speechspectral
amplitude,respectively.

日本音響学会講演論文集

Proposed Joint Method

The reverberant speech in the mixture in Eq.
(1) is composed of clean speech with the room
impulse response, as given by
𝑋𝑆 (𝑓, 𝑘) = 𝑋 (𝑓, 𝑘) + 𝑋 (𝑓, 𝑘)= 𝑆 (𝑓, 𝑘) (𝑓) + 𝑆(𝑓, 𝑘) (𝑓),(12)()()
where𝑓 and𝑓 indicate the early andlate room impulse responses, respectively. This isbecause the speech signal tends to loses its
correlation after some delays, owing to its
nonstationary characteristics. Therefore, the latereverberation component can be suppressed in thesame manner as additive noise.
3.1 Main Algorithm
Assuming that the diffuse background noise
has been suppressed effectively, the extracted̂𝑆 (𝑓, 𝑘) will only consist of clean
component 𝑋
speech and its reverberation. First, we synthesize
the late reverberation in the time domain by
applying convolution according to Eq. (12). The
late room impulse response is estimated by(13)𝒉 (𝜏) = 𝑢(𝜏)𝑒 −𝑑(𝜏−𝜏𝑑 ) ,ln 106

𝑑 = 2(𝑇 −𝜏 ),(14)
where 𝑢(𝜏) is a Gaussian random function, 𝜏𝑑
is the cutoff time between early and late impulse
responses, and 𝑇60 is the reverberation time. The
clean speech estimate 𝑠̂ ( ) is approximated by
̂𝑆 (𝑓, 𝑘) to the truncatedprojecting back 𝑋
FD-BSE filter.
Next, we will compare the performance of
single-channel and multichannel MOSIE
estimator postprocessing, as shown in Fig. 1 andFig.2, respectively. Parametric postprocessing
allows flexible control of the level ofdereverberation. However, it is important to set
the parameter to obtain the optimum output. In
this paper, we only focus on optimizing the shape- 790 -60𝑑

2014年3月

Table 1 Speech recognizer specification
Frame length
Frame period
Pre-emphasis
Feature vectors

Figure 3 Room configuration for the experiment.
parameter 𝜌. Other parameters, such as 𝑇60 , areassumed to be known.
3.2 Optimization Scheme Based on Acoustic
Likelihood
Signal processing front-end in ASR system canonly be expected to improve the recognition
performance if it generates outputs that maximize
the probability of the correct transcription relative
to other possible candidates. Therefore, we canoptimize the parameter of the front-end based on
the likelihood in the acoustic model of ASR.
In speech recognizer, a series of fixed size
acoustic vectors 𝒐(𝜌) = [𝑜1 , … , 𝑜𝑇 ] is extracted
from the output of joint method with parameter 𝜌.
During decoding, it attempts to hypothesize theword sequence 𝑾 = [𝑤1 , … , 𝑤𝐾 ] which is themost probable to generate the sequence 𝒐(𝜌), asstated bŷ = arg max 𝑃(𝑾|𝒐(𝜌)).
𝑾𝑾(15)

However,
theposteriorprobability
𝑃(𝑾|𝒐(𝜌)) cannot be computed directly. Thus,
Bayes’ theorem is applied, as given bŷ = arg max 𝑃(𝒐(𝜌)|𝑾)𝑃(𝑾).
𝑾
𝑃(𝒐(𝜌))𝑾(16)𝑃 (𝒐(𝜌)|𝑾) is the acoustic likelihood(acoustic score), representing the probability thatfeature sequence 𝒐 is observed given that word
sequence 𝑾 was spoken, and 𝑃(𝑾) is thelanguage score, i.e., the a priori probability of aparticular word sequence 𝑾.
Since Eq. (16) is maximized with respect tothe word sequence 𝑾 for a given observed
sequence 𝒐 that is fixed, the denominator term
𝑃(𝒐(𝜌)) can be ignored. Thus, the parameter

日本音響学会講演論文集

25 ms
10 ms
1 − 0.97𝑧 −1
12-order MFCC,
12-order ΔMFCC.
1-order ΔE
Acoustic model HMM phonetic tied mixture(PTM), 2000 states, GMM
64 mixtures
Language model Standard word trigram model
Training data
Adult JNAS database𝜌 is optimized by maximizing the likelihoodof acoustic model, as written by𝜌̂ = arg max 𝑃(𝒐(𝜌)|𝑾).
𝜌(17)This optimization problem requires the correcttranscription 𝑾. However, if it is already known,
the speech recognizer is not required anymore.
Therefore, in practice, we apply MOSIE
estimator postprocessing using a set of 𝜌parameters, from which the output with thehighest acoustic score is chosen. 𝑾 is selectedfrom the result of the first iteration.

4

Experimental Evaluation

Two experiments have been carried out forevaluation
purposes.
An
eight-channel
microphone array (inter-microphone spacing of2.1 cm) was used to record the room impulse
response with the configuration shown in Fig. 3.
The estimated 𝑇60 is 500 ms. The observed
signal was created by convolution of the clean
speech with the impulse response, and recorded
real noise was added at SNR of 10 dB.
Recognition performance was evaluated using
Julius with the specifications shown in Table 1.
Since the goal of the proposed method is tosuppress diffuse background noise and latereverberation, we used clean speech convoluted
with the early impulse response as the baseline.
The word accuracy measure used in theevaluation is calculated as
WA = 100 ×𝑁−(𝐼+𝑆+𝐷)
𝑁,(18)

where 𝑁, 𝐼, 𝑆, and 𝐷 are the number of words
in the correct transcription and the number ofinsertions,substitutions,anddeletions,- 791 -

2014年3月

Table 2 Word accuracy results for Experiment 1

Table 3 Word accuracy results for Experiment 2

improvement of 12.9%. This implies that
optimization based on acoustic likelihood iseffective for our method.
respectively. The frequency domain processing
was carried out with a 512-point Hamming
window and 50% overlap of the STFT. FD-BSE
was performed in 600 iterations with anadaptation step of 0.3. The parameter 𝜏𝑑 was set
to 75 ms, corresponding to the delay that can stillbe handled by the speech recognizer.
The first experiment was carried out using
5-male and 5-female utterances from JNAS
database. We employed several parameter sets,e.g., 𝛽 of 0.001 to represent the MOSIE-LSA
estimator and 1 to represent the MOSIE-STSA
estimator[5]. We manually selected the best result
among these combinations.
The results are shown in Table 2. It is shownthat the multichannel MOSIE estimator improves
the recognition accuracy compared with FD-BSE.
On the other hand, the single-channel
MOSIE-STSA estimator’s performance is inferior.
This is understandable as single-channel
processing tends to result in higher speech output
distortion.
It can be observed that the MOSIE-STSA
estimator performs better than the MOSIE-LSA
estimator for dereverberation. It is also shownthat the α = 0.96 results in better word accuracy
than α = 0.98 , which is known to give theoptimum result for speech enhancement such asthat for hearing aid system. This may be because
a high quality output signal waveform is lessimportant for speech recognition purposes.
The optimization scheme was evaluated in thesecond experiment. We used the utterances from
50-male and 50-female speakers. Parameters αand β were set to 0.96 and 1, respectively. The
results in Table 3 show that the optimized
methods outperform FD-BSE with an average
日本音響学会講演論文集

5Conclusion

We combined FD-BSE and MOSIE estimatorto suppress the diffuse background noise and latereverberation for a hands-free ASR system,
which was optimized based on the acoustic
likelihood. Experimental results show that theproposed method improves the word recognition
accuracy compared with the FD-BSE method.
Future work may include the utilization ofFD-BSE in estimating the speech and latereverberation statistical model.
References[1] P. Comon, “Independent component analysis, anew concept?,” Signal Process., vol. 36, pp.
287–314, 1994.
[2] Y. Takahashi, T. Takatani, K. Osako, H.
Saruwatari, and K. Shikano, “Blind spatialsubtraction array for speech enhancement innoisy environment,” IEEE Trans. Audio
Speech Lang. Process., vol. 17, no. 4, pp. 650–664, 2009.
[3] J. Even, H. Saruwatari, and K. Shikano, “Blindsignal extraction based speech enhancement inpresence of diffuse background noise,” Proc.
IEEE SSP 2009, pp. 513–516, 2009.
[4] Y. Ephraim and D. Malah, “Speechenhancement using a minimum mean-squareerror short-time spectral amplitude estimator,”
IEEE Trans. Acoust. Speech Signal Process.,
vol. 32, no. 6, pp. 1109.1121, 1984．
[5] C. Breithaupt and R. Martin, “Analysis ofthe decision-directed SNR estimator forspeech enhancement with respect to
low-SNR and transient condition,” IEEETrans. Audio Speech Lang. Process., vol. 19,
no. 2, pp. 277–289, 2011．

- 792 -2014年3月