2009 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics

October 18-21, 2009, New Paltz, NY

STATISTICAL MODELS FOR SPEECH DEREVERBERATION

Takuya Yoshioka'r', Hirokazu Kameoka', Tomohiro Nakatani', and Hiroshi G. Okuno'
1 NTT Communication Science Laboratories, NTT Corporation
2-4, Hikari-dai, Seika-cho, Soraku-gun, Kyoto 619-0237, Japan
2Graduate School of Informatics, Kyoto University
Yoshida-hommachi, Sakyo-ku, Kyoto 606-8501, Japanemail: takuya @ cslab.kecl.ntt.co.jp, web: http://www.kecl.ntt.co.jp/icl/signal/takuya/index.htrnl

ABSTRACTcessing techniques (e.g., [5]). This paper further investigates thestatistical-model-based approach to dereverberation.
Now, we briefly review the concept of the statistical-modelbased approach to dereverberation. We begin by formulating the
dereverberation problem. In this paper, we deal with a single microphone case. Let Sn,l and Yn,l denote clean and reverberant
speech signals, respectively, represented in the short-time Fouriertransform (STFT) domain, where nand 1 are time frame and frequency bin indices, respectively. Assume that we observe Yn,l over
N consecutive time frames. We represent the observed data set as
Y = {Yn,l}O~n<N,O~l<L, where L is the number of frequency
bins. In response to this, we denote the corresponding set of clean
speech samples by S = {Sn,l }O~n<N,O~l<L. Dereverberation is aprocess for estimating S when Y is given.
With the statistical-model-based approach, we first define statistical models of a room transmission channel and a clean speech
signal by using probability density functions (pdfs) p(y Is, 8) andp(sI8), respectively, where 8 is the set of all parameters. We refer to the respective pdfs as the room acoustics pdf and the clean
speech pdf. Then, we estimate 8 values using the observed data, y.
We denote by 8 the set of the estimated parameter values. Finally,
we compute the minimum mean square error (MMSE) estimate of
the clean speech signal as

This paper discusses a statistical-model-based approach to speech
dereverberation. With this approach, we first define parametric statistical models of probability density functions (pdfs) for a clean
speech signal and a room transmission channel, then estimate themodel parameters, and finally recover the clean speech signal byusing the pdfs with the estimated parameter values. The key to thesuccess of this approach lies in the definition of the models of theclean speech signal and room transmission channel pdfs. This paper presents several statistical models (including newly proposedones) and compares them in a large-scale experiment. As regards
the room transmission channel pdf, an autoregressive (AR) model,
an autoregressive power spectral density (ARPSD) model, and amoving-average power spectral density (MAPSD) model are considered. A clean speech signal pdf model is selected according
to the room transmission channel pdf model. The AR model exhibited the highest dereverberation accuracy when a reverberant
speech signal of 2 sec or longer was available while the other twomodels outperformed the AR model when only a l-sec reverberant
speech signal was available.

Index Terms- Dereverberation, statistical model.
1. MOTIVATION

s = E s {s;p(s ly , 8 )},Reverberation degrades the quality of speech picked up by distant microphones and thereby limits the applicability of existing speech processing products. Therefore, dereverberation techniques, which mitigate the unfavorable reverberation effect, are vital for the further expansion of existing speech processing products
as well as for the development of new ones. In fact, many dereverberation methods have been proposed including a maximum kurtosis method [1], a spectral subtraction method [2], and a weighted
prediction error (WPE) method [3].
In practical situations, speech quality may be degraded not
only by reverberation but also by ambient noise and interferingspeech. Therefore, dereverberation techniques should be designed
so that they can be effectively combined with noise reduction andspeech separation techniques.
A statistical-model-based approach has been employed extensively for noise reduction and speech separation [4]. Thus,
this approach affords a good guide to combining the dereverberation techniques with noise reduction and speech separation techniques. In fact, on the basis of the WPE method for dereverberation [3], which is based on this approach, we have developed several dereverberation methods coupled with such other signal pro-

978-1-4244-3679-8/09/$25.00 ©2009 IEEE

(1)

where p(sly,8) ex p(Yls,8)p(sI8) and Ex{f(x); q(x)} =f q(x)f(x)dx. In summary, we need to perform the following
three steps to derive a specific dereverberation method.
(sl) Define a room acoustics pdf, p(Yls, 8), and a clean speech
pdf, p( s 18), along with a parameter set, 8.
(s2) Define an estimation algorithm for 8.
(s3) Derive the MMSE signal estimator given by (1).
The key to the successful design of a dereverberation method
is the selection of statistical models of room acoustics and clean
speech pdfs. Different models lead to dereverberation methods
with different characteristics. Therefore, we must understand whatkind of dereverberation method is derived based on each model.
With the above motivation, this paper describes several statistical models of room acoustics and clean speech pdfs, and experimentally compares the dereverberation methods derived fromthese models. The models considered in this paper include both
newly proposed and already reported models [3]. The comparison
is carried out in terms of maximum dereverberation accuary and145

2009 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics

October 18-21, 2009, New Paltz, NY

where hk,l is a complex number and * is a complex conjugate.
Therefore, the conditional pdf, P(Yn,lly~-l, S, 8), is described as

the amount of observation data required for model parameter estimation. (Although computational cost and memory size are also
important, we do not cover these performance indices because they
are dependent on implementation.)P(Yn,lly~-l, S, 8)
Dl+Kl-1= NC{Yn'l;

2. MODELS CONSIDERED IN THIS PAPER

E

h'k,IYn-k,I+Sn,I,0"2} (0"2-. 0 ) .

(7)

k=Dl

In Section 2, we describe several statistical models of room acoustics and clean speech pdfs.

By substituting (7) into (4), we obtain the AR-modelbased room acoustics pdf. This model is parameterized by{{hk,l} DlS;k<Dl+Kl }OS;l<L; in other words, we have hk,l E 8.
The validity of the AR model is discussed in [3].
Reverberation generally affects both the power and phase
spectra of a clean speech signal. We see that (7) takes accountof the reverberation effects on both the power and phase spectra.
By contrast, the remaining two room acoustics models consider
only the reverberation effect on the power spectra.

2.1. Basic PDFs
First, we present two basic pdfs that are used in the definition ofroom acoustics and clean speech pdfs.
• Complex normal distribution
Nc {x; u, (12} denotes the pdf of a complex normal distribution with mean J-l and variance (12, Le.,1
{ Ix - 2J-l1
JVC{X; J-l, (1 } = - - 2 exp
ft.

r

2

1r(1

Autoregressive psd (ARPSD) model

2
}

(1

for x E C.

(2)

The second room acoustics model is an autoregressive power spectral density (ARPSD) model. This model is new and inspired by
the Lebart et al.'s work [2]. Lebart et al. assumed that the power
spectral density (psd) of the reverberation component of a reverberant speech signal is the same as the psd of a delayed versionof the reverberant speech signal up to a constant scale factor. This
assumption was derived from the observation that sound energy
exponentially decays in a room. (Later, Habets et al. reported thatthe exactness of this assumption can be improved by further taking into consideration direct-to-reverberation ratio [6].) The delay
amount is set at about 50 msec, and the constant scale factor isdetermined with respect to the reverberation time.
The ARPSD model regards the reverberation component, rn,l,as an additive noise. In other words, we assume that rn,l is uncorrelated with Sn,l. Furthermore, generalizing the assumption madeby Lebart et al., we assume that rn,l is normally distributed witha mean of 0 and a variance (Le., a psd component) given by anautoregressive form as• Generalized gamma distribution
QQ{x; K" (), p} denotes the pdf of a generalized gamma distribution with scale parameter () and two shape parameters K,
and p, Le.,

pXK,p-1{(X)P}(jfor x > O. (3)QQ{x; K" (), p} = r(K,)(}K,p exp 2.2. Room acoustics PDFs

Now, we describe the room acoustics pdfs, p(Yls, 8), that are considered in this paper. We assume that reverberant speech signals at
different frequency bins are statistically independent of each other.
Then, p(Yls, 8) can be decomposed as
L-1 N-1
p(Yls,8) =II II P(Yn,z1y~-l,s, 8),(4)

l=O n=O

Dl+Kl-1p(rn,de) = Nc{rn,l; 0, (where y~-l = {Ynl ,l}OS;n' <n,OS;l<L. The pdf on the right handside of (4), P(Yn,z1y~-l, s, 8), represents the probability of observing Yn,l at time frame n on condition that we have the clean
speech signal and the past observed signal. In this paper, we consider three statistical models for this conditional pdf. Two of thesethree models are presented for the first time.

E

gk,dYn-k,d

2)}.

(8)

k=Dl

Note that the gk,l value is non-negative. Based on (8), we obtainthe following conditional pdf:
P(Yn,lly~-l, S, 8)

Dl+Kl-1= N c{Yn,l; Sn,l, (Autoregressive (AR) modelE

gk,dYn-k,d

2)

}.

(9)

k=Dl

The first room acoustics model is an autoregressive (AR) model,
which we described in our previous paper [3]. (Note however that,in [3], we did not restrict the number of microphones to one.) The
reverberant speech signal, Yn,l, is divided into its clean component, Sn,l, and reverberation component, denoted by r-». as
Yn,l = Sn,l + rn,l·If we set Ki at 1 and Di at about 50 msec, (9) becomes equivalent to the model of [2]. The ARPSD model is parameterized by{{gk,l} DlS;k<Dl+Kl }OS;l<L; thus we have gk,l E 8.
Incidentally, the method described in [2] determines the gk,lvalue based on the reverberation time. Lebart et al. proposed
estimating the broadband reverberation time by exploiting shortpauses in speech activity. However, because a reverberation time
generally depends on a frequency bin, we may not achieve an acceptable dereverberation accuracy by using only the broadband reverberation time. In constrast, we can estimate gk,l without using the reverberation time by combining the ARPSD model with asparseness-constrained non-stationary Gaussian model of a clean
speech pdf, which we describe later.

(5)

The AR model assumes that a room transmission channel is modeled by an AR system, or equivalently, that rn,l is given by
Dl+Kl-1rn,l =

Eh'k,lYn-k,l,

(6)

k=Dl

146

2009 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics

Moving-average psd (MAPSD) model

October 18-21, 2009, New Paltz, NY

where K, (J, P are prescribed constants.
In summary, we consider the following three combinations ofroom acoustics and clean speech pdfs:

The third model is a moving-average psd (MAPSD) model. Thismodel also regards reverberation component rn,l as an additivenoise. The difference from the ARPSD model lies in the assumption that the variance (i.e., the psd component) of rn,l is expressedby using a moving-average (MA) form. Specifically, the conditional pdf on the right hand side of (4) is assumed to be given by(ml) an AR model combined with an UCNSG model;(m2) an ARPSD model combined with an SCNSG model;(m3) an MAPSD model combined with an SCNSG model.
Thus, step (s1) has been completed.
3. DEREVERBERATION METHODS

where ''In,l is the reciprocal of the psd component, which is sometimes called the accuracy, of clean speech signal Sn,l. This modelis parameterized by {{gk,l}Dl~k<Dl+Kl}O~l<L; thus, we have
gk,l E 8. The MAPSD model aims to remove the exponential
decay assumption made by the ARPSD model.
The above three room acoustics models are compared in Section 4.

By carrying out steps (s2) and (s3) for each statistical model combination described in Section 2, we can derive dereverberation
methods corresponding to the respective combinations. Combination (ml) leads to the WPE method [3]. In this section, we describe the parameter estimation algorithm (step (s2» and MMSE
signal estimator (step (s3» for combination (m2). The parameter
estimation algorithm and MMSE signal estimator for combination(m3) can be derived in a similar way to those for (m2).
The statistical models for (m2) are defined by (4), (9), (11),(12), and (13). Hence, the set of parameters is given by

2.3. Clean speech PDFs
Next, let us describe the clean speech models. We start withthe assumption that Sn,l and Sn' ,l' are statistically independent if(n, l) i= (n', l'). Then, we obtain
L-1 N-1
p(sI8) =II II p(sn,z18).

(11)

We estimate these parameter values by means of maximum a posteriori (MAP) estimation, i.e., we computethat maximizes thefollowing posterior pdf:el=O n=OAs regards p(sn,z18), we consider the following two models:• an unconstrained non-stationary Gaussain (UCNSG) model;p(8Iy) ex: p(8)Es{p(yls, 8);p(sI8)}.

• a sparseness-constrained non-stationary Gaussian (SCNSG)
model.

The first term on the right hand side of (15) is defined as

0::0p(8) =
Of=~l P("ln,z), where we ignore the prior for gk,l.
p(yls, 8 ) is given by (4) and (9), while p(sI8) is given by (11)1

A clean speech pdf model should be selected in accordancewith a room acoustics model so that we can derive an effective
parameter estimator in step (s2). The UCNSG model is used in
combination with the AR model. On the other hand, the SCNSG
model is used with the ARPSD or MAPSD models.

and (12).
that maximizes the posterior pdf, (15), cannot be analytically calculated. There are two factors for this difficulty. One
is that each variance of the ARPSD model, (9), is defined as a(weighted) sum of gk,l over several k values. The other is thatthe log of the prior, given by (13), involves the computation of
p power of ''In,l. To cope with the first problem, we use an
expectation-maximization (EM) algorithm by introducing latentvariables rDl,n,l,' .. ,rDl+Kl-l,n,l for each nand l. rk,n,l represents the component of reverberation component r n,l that comesfrom the reverberant speech signal at the (n - k)th time frame,
Yn-k,l. Therefore, rk,n,l satisfies the following two conditions.

e

Unconstrained non-stationary Gaussian (UCNSG) model
In the field of independent component analysis, it is well known
that effective speech separation algorithms can be obtained by assuming a clean speech signal to be a realization of a non-stationary
Gaussian process. With this as a basis, we consider the following
modelofp(sn,lI8):p(sn,z18) = NC{Sn,l; 0, l/"ln,l}.

(15)

(12)

The parameter set for this model is the set of accuracies,{''In,l}O~n<N,O~l<L; thus, we have ''In,l E 8.

+ ... + rDl+Kl-1,n,l = rn,l(16)p(rk,n,z18) = Nc{rk,n,l; 0, gk,z1Yn-k,lI2}.

(17)

rDl,n,lSparseness-constrained non-stationary Gaussian (SCNSG)
model

We cope with the latter problem by using an auxiliary function
method, which is commonly used in the field of non-negative matrix factorization (NMF) [7].
The parameter estimation algorithm may be described as follows (we omit the detailed derivation owing to the limited space).
In the following, we use vector rn,l, which is defined as rn,l =[rDl,n,l,' .. ,rDl+Kl-l,n,dT. First, we set initial parameter values, denoted by 8 [0]. Then, we repeat the following two steps untilconvergence.

A clean speech signal generally contains short pauses and has a
harmonic spectral structure. Therefore, the psd components of aclean speech signal are sparsely distributed. To express this sparseness, the second model uses the following prior pdf for accuracy''In,l in addition to (12):
(13)

147

2009 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics

October 18-21, 2009, New Paltz, NY

1. For all nand l values, compute p(r n ,t1Y, e[i J) according to
the following equations (E-step) :p(rn,lly, eli!) = Nc{rn,l; "~~l ' I;~~l},,[i Jn,l(18)=(w[iJ)HlYn ,In,l(19)

I;[iJ =(A[iJ 1'T/ [iJ )(II'T/[iJ +
n ,l

n ,l

n ,l

n,l

11 T A[iJ )-1
n ,l

(20)

W[i! =A[i! (II [iJ + 11T A[iJ )-1 .
n ,l

n ,l

- •• Observed-+oAR' 00- ARPSD.. ct ' MAPSD'TJn ,1

2
3
4
Amount of observed data (sec)

(21)

n ,l

I and 1 denote the Kl-dimensional identity matrix and allone vector, respectively, and A~~l is given as follows:

Figure I: Average cepstral distances . Note that the lines for
ARPSD and MSPSD models are almost completely overlapping.

2
J 1[iJ = d'lag ([i
An,l
gDI,1
Yn-Dz,l1 ,~

2... , gDI +KI- I,IIYn- DI- KI+I,t1 ).

speech signals processed by the dereverberation methods for (m2)and (m3) sounded somewhat distorted due to Wiener filtering, they
were much less reverberant than the input speech signals .
From the above results, we conclude that, for a room acoustics pdf, we should use an AR model when we can assume thata speaker does not move frequently. For situations where this assumption is invalid, ARPSD and MAPSD models are better. Effectively combining these two models would achieve dereverberation
with high accuracy using only a small amount of observation data.
In this paper, we focused on maximum dereverberation accuracy and observation data amount. Another important performanceindex is sensitivity to environmental changes . We will conductcomparative tests in relation to these other performance indices .

(22)

Note that the matrix inversions in (20) and (21) can be
avoided by using the Woodbury fomula .
2. For all n, l, and k values, update the parameter values according to the following equations (M-step) :1 /'T/~fJ = E" n.l {IYn,1 - gT r n,t1 2;p(rn ,t1Y, e[iJ)} /lI:p+ 1/ I1:BP('T/[in,lJ )I-P

(23)

~ E r k • n •l {l rk, n,t1 ; p(rn,l;Y, e[i J)}- L.J
NIYn-k 11 2
• (24)
2[HIJ _

gk,l

n=O

5

'

5. REFERENCES

Because p(r n,lly , e[i J) is based on a complex normal distribution as shown by (18), the expectated values in (23)and (24) can be readily computed.

[I) B. W. Gillespie, H. S. Malvar, and D. A. F. Florencio, "Speech
dereverberation via maximum-kurtosis subband adaptive filtering," in Proc. Int 'l Conf. Acou st., Speech. Signal Process.,
vol. VI, 2001, pp. 3701-3704.

As regards step (s3), it is obvious that the MMSE signal estimator is given by a Wiener filter. This is a consequence of the factthat we regarded the reverberation component, r n,l, as an additivenoise. We omit the details due to lack of space.

[2] K. Lebart, J. M. Boucher, and P. N. Denbigh , "A new method
based on spectral subtraction for speech dereverberation,"
Acta Acustica united with Acustica, vol. 87, pp. 359-366,
2001.

4. EXPERIMENT AND CONCLUSION[3) T. Nakatani, T. Yoshioka, K. Kinoshita, M. Miyoshi, and B.-H.
Juang, "Blind speech dereverberation with multi-channel linear prediction based on short time Fourier transform representation," in Proc. Int 'l Con! Acoust. Speech, Signal Process.,
2008, pp. 85-88.

We conducted an experiment to compare the dereverberation methods derived from the statistical models described in Section 2. We
used utterances of 306 speakers contained in the JNAS database .
For each speaker, we made 1-, 2-, 3-, 4-, and 5-sec clean speech
signals with a sampling rate of 16 kHz. The individual clean
speech signals were convolved with an impulse response measured
in a room with a reverberation time of 0.6 sec to simulate reverberant speech signals. The frame size and frame shift for STFT
were set at 512 points (32 msec) and 128 points (8 msec), respectively. Di, Ki , 11:, p, and B values were experimentally determined.
The experimental results were evaluated in terms of the 12th-order
cepstral distances (CD) between the target (i.e., processed or reverberant) speech signals and the corresponding clean speech signals.
The CD was shown to be highly correlated with subjective qualityof dereverberated speech [8].
Figure I shows the CDs averaged over the 306 speakers
against the amount of observed data. We see that combination(ml) provided the smallest average CDs when reverberant speech
signals of 2 sec or longer were available. However, with (m I) , theaverage CD increased when we used only I-sec reverberant speech
signals . By contrast, we find that combinations (m2) and (m3) constantly improved the average CDs to some degree. Although the[4] J. Benesty, S. Makino, and J. Chen, Speech Enhancement.
Springer, 2005.
[5] T. Yoshioka, T. Nakatani, and M. Miyoshi , "Integrated speechenhancement method using noise suppression and dereverberation," IEEE Trans. Audio, Speech. Lang. Process., vol. 17,
no.2,pp.231-246,2009.
[6) E. A. P. Habets, S. Gannot, and I. Cohen, "Dual-microphone
speech dereverberation in a noisy environment," in Proc. Int ',Symp. Signal Process., In! Tech., 2006, pp. 651-655 .
[7] Y. Lin and D. D. Lee, "Bayesian Ll-norm sparse learning,"
in Proc. Int 'l Conf. Acoust., Speech. Signal Process., vol. V,
2006, pp. 605-608.
[8] K. Furuya and A. Kataoka, "Factor analysis of speech quality improved by dereverberation processing ," IEICE Trans.
Fund., vol. E91-A, no. 8, pp. 763-771 , 2008, in Japanese .

148