JAIST Repository
https://dspace.jaist.ac.jp/

Title

A study on the IMTF‑based filtering for themodulation spectrum of reverberant speech

Author(s)Morita, Shota; Unoki, Masashi; Akagi, Masato

Citation2010 International Workshop on Nonlinear
Circuits, Communication and Signal Processing(NCSP'10): 265‑268

Issue Date

2010‑03‑04

Type

Conference Paper

Text version

publisher

URL

http://hdl.handle.net/10119/9969

RightsThis material is posted here with permission of
the Research Institute of Signal Processing
Japan. Shota Morita, Masashi Unoki, and Masato
Akagi, 2010 International Workshop on Nonlinear
Circuits, Communication and Signal Processing(NCSP'10), 2010, pp.265‑268.

Description

Japan Advanced Institute of Science and Technology

2010 International Workshop on Nonlinear Circuits,
Communication and Signal Processing
NCSP'10, Waikiki, Hawaii, March 3-5, 2010

A study on the IMTF-based filtering on the modulation spectrum of reverberant speech
Shota Morita, Masashi Unoki, and Masato Akagi
School of Information Science, Japan Advanced Institute of Science and Technology
1-1 Asahidai, Nomi, Ishikawa 923-1292, Japan
Phone/FAX: +81-761-51-1391/+81-761-51-1149
Email: {s-morita, unoki, akagi}@jaist.ac.jp

Abstract
Many methods of speech dereverberation have been proposed
to reduce the effects of reverberation. The IMTF (InverseMTF)-based filtering on the power envelope does not needto measure the room impulse response (RIR), while the RIR
has to be precisely measured before the dereverberation in
the typical methods. However, improvement of restoration
accuracy of the restored power envelope is saturated as thereverberation time increases. This is a remaining problem.
This paper proposes IMTF-based filtering on the modulation
spectrum to resolve the problem. The proposed method estimates the reverberation time on the modulation spectrum andthen dereverberates the modulation spectrum of reverberant
speech using the IMTF. Three simulations were carried out toevaluate the proposed method. Results showed that the proposed method can adequately restore the power envelope of areverberant signal in comparison with the previous method.

tering on the power envelope. This can restore the temporalenvelope of original speech from reverberant speech.
In this method, spectrum of the power envelope, that is,modulation spectrum, can be restored by using IMTF-based
filtering in which modulation frequencies of the temporal
power envelope are limited to 20 Hz using the low-pass filter (LPF). However, since the remains on the power envelopethat are higher modulation spectra over 20 Hz were overemphasized by the IMTF-based filtering, the reverberation
time (TR ) is underestimated due to these remains and improvement of restoration accuracy by this method is saturated
as TR increases. This is a remaining problem of the method.
In this paper, we propose IMTF-based filtering on the modulation spectrum, not on the power envelope, to solve theabove problem. The remains can be removed completely
in the modulation frequency domain so that the proposed
method can effectively restore the modulation spectrum of theoriginal speech signal from reverberant speech in comparison
with the IMTF-based filtering on the power envelope.

1. Introduction
In real environments, significant features of speech signals
are smeared due to reverberation so that the sound quality
and intelligibility of speech signals are significantly degraded.
Therefore, restoration of an original speech from a reverberant speech in room acoustics is an important issue such asconcerning for robust speech recognition systems.
Many methods have been proposed to dereverberate theoriginal speech from the reverberant speech in the room
acoustics. For example, minimum-phase inverse filtering
method was proposed by Neely and Allen [1]. This method
can only be used for room acoustics with minimum phasecharacteristics. Miyoshi and Kaneda proposed the multiple input/output inverse theorem (MINT) method [2]. Wangand Itakura proposed the method of acoustic inverse filtering through multi-microphone sub-band processing. However, all of these methods have to measure the room impulse
response (RIR) before the dereverberation.
On the other hand, the power envelope inverse filtering
method has been proposed to improve speech intelligibility,
degraded by reverberation, by Unoki et al. [5, 6]. Thismethod is based on the modulation transfer function (MTF)[4] so that this is referred as inverse MTF (IMTF)-based fil-2. Modulation Transfer Function (MTF)
The MTF concept was proposed by Houtgast andSteeneken to predict speech intelligibility in the room acoustics [4]. The MTF can be characterized as the modulationindex that accounts for a relation between a transfer function in an enclosure with regard to the envelopes of input andoutput signals. For example, the modulation index of the output signal is decreased by MTF (due to reverberation) whenthe modulation index of input signal is 1.0 (100% amplitude
modulation). The MTF can be represented as functions ofmodulation frequency and reverberation time.
We explain the MTF in reverberant environments. Room
impulse response (RIR) we used is defined as

6.9t
h(t) = eh (t)nh (t) = a exp −TRnh (t),(1)

where eh (t) is envelope of the RIR, nh (t) is white noise as
carrier, a is amplitude term and TR is reverberation time. This
RIR was proposed by Schroeder [7]. Here, the MTF of h(t)- 265 -

T̂R = max arg

1
T = 0.1 sRmin

TR,min ≤TR ≤TR,max

Z T
0min ê2x,TR (t), 0





dt ,
(6)

MTF

0.8
0.6

TR = 0.3 s

0.4

TR = 0.5 s

where T is signal duration and ê2x,TR (t) is the set of candidatesof the restored power envelope as a function of TR .
The power envelope e2y (t) from y(t) is extracted ase2y (t) = LPF |y(t) + j · Hilbert(y(t))|2 ,(7)

TR = 1 s

where LPF[·] is a low-pass filtering and Hilbert[·] is theHilbert transform. This method is used in the LPF as postprocessing to remove the component of higher modulation
0
0
5
10
15
20
spectrum in the power envelope. The cut-off frequency ofModulation Frequency, f (Hz)the LPF is 20 Hz because the dominant component of modm
Figure 1: Theoretical curves representing the MTF, m(fm ), ulation region for speech perception and speech recognition
for various conditions with TR = 0.1, 0.3, 0.5, 1.0 and 2.0 s. exists from 1 to 16 Hz.
Figure 2(a) shows a block diagram of the IMTF-based inverse filtering on the power envelope. In this method, specis represented as [5]trum of the power envelope, that is, modulation spectrum, can"be restored by using IMTF-based filtering in which modu
2 #− 21TR
lation frequencies are limited to 20 Hz using the LPF. Them(fm ) = 1 + 2πfm,
(2)estimation method of reverberation time in the time domain
13.8
can calculate the best reverberation time T̂R for the reasonwhere fm is the modulation frequency. Figure 1 shows the able power envelope restoration. However, the actual LPF
theoretical curves of MTF m(fm ), with various TR s. From cannot completely remove the remains that are higher modthis figure, MTF can be regarded as characteristics of a low- ulation spectra over 20 Hz. Since the remains on the powerpass filtering in the modulation frequency domain.
envelope are over-emphasized by the IMTF-based filtering,
the emphasized remains affect to degrade the dips of powerenvelope that dominate the estimation accuracy of reverber3. IMTF-based filtering on power envelope
ation time. Thus, T̂R is underestimated due to the remains
In the IMTF-based filtering on the power envelope [5, 6], and improvement of restoration accuracy by this method isthe following useful relation is used.
saturated as TR increases. This is a remaining problem of the*Z2 +IMTF-based filtering on the power envelope.
∞y 2 (t) =x(τ )h(t − τ )dτ−∞
4. IMTF-based filtering on modulation spectrum
Z
0.2

TR = 2 s∞

e2x (τ )e2h (t − τ )dτ = e2y (t),

=

(3)

−∞

where e2x (t), e2h (t), and e2y (t), are the power envelopes of the
input x(t), the RIR h(t), and the output y(t), respectively.
On the basis of this result, e2x (t) can be recovered by deconvoluting e2y (t) = e2x (t) ∗ e2h (t) with e2h (t). Here, the transmission functions of power envelopes Ex (z), Eh (z), and Ey (z)are assumed to be the z-transforms of e2x (t), e2h (t), and e2y (t).
Thus, Ex (z) can be determined fromEy (z)
13.8−1
Ex (z) =z
1 − exp −, (4)a2TR · f s

We propose another type of the IMTF-based filtering tosolve the above problem. Figure 2(b) shows the proposed
method. In order to remove the remains on the power envelope, the proposed method represents the power envelopee2y (t) by down-sampling from 20k Hz to 40 Hz (M=500) andthen represents modulation spectrum of e2y (t) within 20 Hz.
In this method, we incorporated the estimation method of reverberation time as blind-method by Hiramatsu and Unoki[8] into the proposed method, to estimate TR at the dominant
modulation frequency. Here, Eq (5) is used to determine theparameter of â. Then, IMTF-based filtering on the modulation spectrum in Eq. (4) is used to restore the modulation
spectrum of reverberant signal. Finally, the restored powerenvelope ê2x (t) is obtained from the modulation spectrum ofEx (z) by the inverse Fourier transform.

where fs is the sampling frequency. This means that modulation spectrum Ex (z) of e2x (n) can be obtained from Ey (z)times inverse MTF, 1/Eh (z). Therefore, e2x (t) can then be
obtained from the inverse z-transform of Ex (z). Here, twoparameters (TR and a) are obtained as [5, 6].
5. Evaluation
s Z


∞
13.8t
We evaluate the proposed method as to whether it can reâ = 1/
dt,
(5)
exp −
solve the above problem. Original signals x(t) consistedT̂R0- 266 -

(a) IMTF-based filtering on power envelope
Reverberant
signalPower envelope
extractiony(t)Recovered
power envelope

IMTF-based filteringon power envelope^e 2x (t)Parameter estimation(time domain)(b) IMTF-based filtering on modulation spectrum
Reverberant
signaly(t)Power envelope
extraction

M

IMTF-based filteringon modulation spectrum

FFT

Recovered
power envelope
IFFTM^e 2x (t)Parameter estimation(modulation frequency domain)

Figure 2: Block diagram of IMTF-based filtering (a) on the power envelope and (b) on the modulation spectrum.
white noise multiplied by three types of power envelope:5

1. Sinusoidal, e2x (t) = 1 − cos (2πF t);0e2x (t) = 1 +

K
1 X

K

Normalized Modulation Spectrum [dB]

2. Harmonics power envelope,sin(2πkF0 t + θk );k=1

3. Band-limited noise, e2x (t) = LPF[nω (t)].

−5

−10

−15

−20

Here，F = 10 Hz，F0 = 1 Hz，K = 20，θk is a random
phase, and the cut-off frequency of LPF[·] is 20 Hz. The−25
RIRs, h(t)s, consisted of five types of envelope: eh (t) with
Clean
−30
TR = 0.1, 0.3, 0.5, 1.0, and 2.0 s in which a was set in Eq.(5)Reverberation
IMTF−based filtering on power envelopewith each TR , multiplied by 100 white noise carriers. All
IMTF−based filtering on modulation spectrum−35
stimuli y(t) were composed through 1, 500 (= 3 × 5 × 100)
0
2
4
6
8
10
12
14
16
18
20
Modulation frequency [Hz]convolutions of x(t) with h(t).
In this paper, to evaluate both the error and similarity in Figure 3: Restoration modulation spectrum for sinusoidal
the terms of the power envelopes, we thus used (i) correlation power envelope with TR = 1.0 s.
and (ii) SNR (S was power envelope of original signal and N
was power envelope of recovered power envelope)
because TR was underestimated in the time domain due toCorr(e2x , ê2x )the remains and this caused saturation of the improvement of
RT 2
restoration accuracy.
(ex (t)−e2x (t))(ê2x (t)−ê2x (t))dt= qR 0,(8)
Figures 4–6 show the improvements of restoration accuR T
T(e2x (t)−e2x (t))2 dt(ê2x (t)−ê2x (t))2 dt
racy for the three types of the power envelope. In these00figures, panel (a) shows the improved correlation and panel
RT 2
2
(e
(t))
dt
(b)
shows the improved SNR. From these results, it was
SNR(e2x , ê2x ) = 10 log10 R T 0 x,(9) found that the proposed method can effectively improve the222(ex (t) − êx (t)) dt
0restoration accuracy as well in comparison with the previous
method. These improvements are not so much in the cases
22
where the notation of ex (t) means the averaged ex (t).
of last two power envelopes. This maybe caused by different
Figure 3 shows the modulation spectrum of the case in
shapesof dominant peaks in the modulation spectrum.
the sinusoidal power envelope where the peak is in 10 Hz.
This peak indicates the dominant component of the sinusoidal
power envelope. Around this dominant component, the shapeof the restored power envelope by the proposed method corresponded with that of original one. In contrast, the shapein the previous method is under that of original one. This is

6. Conclusion
In this paper, we studied a possibility of solving the remaining problem of the IMTF-based filtering on the power- 267 -

0.5

0.5

0.3
0.2
0.1

0

Improved SNR [dB]

10
5
0
0

0.5

1
1.5
Reverberation time T (s)IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum

0.1

20
IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum

(b)

0.2

0

(b)

IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum

0.3

20
15

(a)

0.4

Improved SNR [dB]

Improved correlation

0.4

Improved correlation

IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum

(a)

15
10
5
0
0

2

0.5

R

1
1.5
Reverberation time T (s)

2

R

(a)

IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum(b)

IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum

0.4
0.3
0.2
0.1
0
20

Improved SNR [dB]envelope and then proposed the IMTF-based filtering on themodulation spectrum. Three simulations were carried out toevaluate the proposed method as to whether it can resolve theproblem. As the results, it was found the proposed method
can adequately improve restoration accuracy of the power envelopes in comparison with our previous method. Improvements are power envelopes, however improvement degree
was not bigger as we expected. Therefore, we presented the
IMTF-based filtering on modulation spectrum had advantage.
We confirmed the influence with the harmonic component ofover 20 Hz in modulation frequency that one of the causes
saturated the accuracy of improvement of IMTF-based filtering on power envelope.

Improved correlation

Figure 4: Comparison with the envelope restoration accuracy Figure 5: Comparison with the envelope restoration accuracy
for a sinusoidal power envelope: (a) improved correlation and for a harmonic power envelope
0.5(b) improved SNR

15
10
5
0
0

0.5

1
1.5
Reverberation time T (s)2R

7. Acknowledgements
This work was supported by the Strategic Information andCOmmunications R&D Promotion ProgrammE (SCOPE)(071705001) of the Ministry of Internal Affairs and Communications (MIC), Japan.

Figure 6: Comparison with the envelope restoration accuracy
for a band-limited noise power envelope

speech intelligibility in auditoria,” J. Acoust. Soc. Am.,77, 1069–1077, 1985.
[5] M. Unoki, M. Furukawa, K. Sakata and M. Akagi, “An
improved method based on the MTF concept for restoring
References
the power envelope from a reverberant signal,” Acoust.
[1] S. T. Neely and J. B. Allen,“Invertibility of a room imSci. Tech., 25(4), 232–242, 2004.
pulse response,” J. Acoust. Soc. Am., 66(1), 165–169, [6] M. Unoki, K. Sakata, M. Furukawa and M. Akagi, “A
1979.
speech dereverberation method based on the MTF con[2] M. Miyoshi and Y. Kaneda, “Inverse filtering of room
cept in power envelope restoration,” Acoust. Sci. Tech.,acoustics,” IEEE Trans. Speech Signal Process., ASSP,
25(4), 243–254, 2004.
36, 145–152, 1988.
[7] M. R. Schroeder, “Modulation transfer function: definition and measurement,” Acoustica, 49, 179–182, 1981.
[3] H. Wang and F. Itakura,“Realization of acoustic inverse
filtering through multi-microphone sub band processing,” [8] S. Hiramatsu and M. Unoki, “A speech dereverberation
IEICE Trans. Fundam., E75-A, 1474–1483, 1992.
method based on the MTF concept in power envelope[4] T. Houtgast and H. J. M. Steeneken, “A review of therestoration,” J. Signal Processing, 12(6), 351–361, 2008.
MTF concept in room acoustics and its use for estimating- 268 -