JAIST Repository
https://dspace.jaist.ac.jp/Title

A Study on the IMTF‑Based Filtering on theModulation Spectrum of Reverberant Signal

Author(s)Morita, Shota; Unoki, Masashi; Akagi, Masato

Citation

Journal of Signal Processing, 14(4): 269‑272

Issue Date

2010‑07

Type

Journal Article

Text version

author

URL

http://hdl.handle.net/10119/9519

RightsCopyright (C) 2010 Research Institute of Signal
Processing Japan. Shota Morita, Masashi Unoki,
and Masato Akagi, Journal of Signal Processing,
14(4), 2010, 269‑272.

Description

Japan Advanced Institute of Science and Technology

A Study on the IMTF-Based Filtering on the Modulation Spectrum of Reverberant Signal
Shota Morita, Masashi Unoki and Masato Akagi
School of Information Science, Japan Advanced Institute of Science and Technology
1-1 Asahidai, Nomi, Ishikawa 923-1292, Japan
Phone/FAX: +81-761-51-1391/+81-761-51-1149
E-mail: {s-morita, unoki, akagi}@jaist.ac.jp

Abstract
Many methods of speech dereverberation have been proposed
to reduce the effects of reverberation. The IMTF (inverseMTF)-based filtering on the power envelope does not needto measure the room impulse response (RIR), but the RIR has
to be precisely measured before the dereverberation in typicalmethods. However, improvement of the restoration accuracyof the restored power envelope is saturated as the reverberation time increases. This is a remaining problem. This paper
proposes IMTF-based filtering on the modulation spectrum
to resolve the problem. The proposed method estimates thereverberation time on the modulation spectrum and then dereverberates the modulation spectrum of reverberant signal using the IMTF. Three simulations were carried out to evaluate
the proposed method. The results showed that the proposedmethod could adequately restore the power envelope of a reverberant signal in comparison with the previous method.
1. Introduction
In real environments, significant features of speech signals
are deteriorated due to reverberation so that the sound quality
and intelligibility of speech signals are significantly degraded.
Therefore, restoration of the original speech from the reverberant speech in room acoustics is an important issue in, forexample, robust speech recognition systems.
Many methods have been proposed to dereverberate theoriginal speech from the reverberant speech in room acoustics. For example, the minimum-phase inverse filtering
method was proposed by Neely and Allen [1]. This method
can only be used for room acoustics with minimum-phasecharacteristics. Miyoshi and Kaneda proposed the multiple input/output inverse theorem (MINT) method [2]. Wangand Itakura proposed the method of acoustic inverse filtering
through multi-microphone sub-band processing [3]. However, all of these methods have to measure the room impulse
response (RIR) before the dereverberation.
On the other hand, the power envelope inverse filtering
method has been proposed to improve speech intelligibility that has been degraded by reverberation, by Unoki et al.
[5, 6]. This method is based on the modulation transfer function (MTF) [4] so that this is referred to as inverse MTF(IMTF)-based filtering on the power envelope. This can restore the temporal envelope of original speech from reverberant speech.
In this method, the spectrum of the power envelope, that is,
the modulation spectrum, could be restored by using IMTFbased filtering in which modulation frequencies of the temporal power envelope were limited to 20 Hz using a low-pass filter (LPF). However, since the remains of the power envelopethat were higher modulation spectra over 20 Hz were overemphasized by the IMTF-based filtering, the reverberation
time (TR ) was underestimated and improvement of restoration accuracy by this method was saturated as T R increased.
This is a remaining problem of the method.
In this paper, we propose IMTF-based filtering on the modulation spectrum, not on the power envelope, to solve theabove problem. The remains could be removed completelyin the modulation frequency domain so that the proposed
method could effectively restore the modulation spectrum of
the original signal from reverberant in comparison with theIMTF-based filtering on the power envelope.

2. Modulation Transfer Function (MTF)
The MTF concept was proposed by Houtgast andSteeneken to predict speech intelligibility in room acoustics[4]. The MTF can be characterized as the modulation index that accounts for a relation between a transfer function
in an enclosure with regard to the envelopes of input and output signals. For example, the modulation index of the output
signal is decreased by MTF (due to reverberation) when themodulation index of the input signal is 1.0 (100% amplitudemodulation). The MTF can be represented as functions ofmodulation frequency and reverberation time.
We explain the MTF in reverberant environments. Room
impulse response (RIR) that we used is defined as
6.9t
h(t) = eh (t)nh (t) = a exp −nh (t)
TR(1)where eh (t) is the envelope of the RIR, n h (t) is white noise
as carrier, a is amplitude term and T R is reverberation time.
This RIR was proposed by Schroeder [7]. Here, the MTF ofTT̂R = max arg

1minTR,min ≤TR ≤TR,max

TR = 0.1 s0min ê2x,T (t), 0  dt

R(6)m(fm)0.80.6

TR = 0.3 s

0.4

TR = 0.5 s

0.2

TR = 1 s

where T is signal duration and ê 2x,TR (t) is the set of candidatesof the restored power envelope as a function of T R .
The power envelope e 2y (t) from y(t) is extracted ase2y (t) = LPF |y(t) + j · Hilbert(y(t))|2(7)where LPF[·] is a low-pass filtering and Hilbert[·] is theHilbert transform. This method is used in the LPF as post0
processing to remove the component of higher modulation
05101520
Modulation frequency, f (Hz)
spectrum in the power envelope. The cut-off frequency ofm
Figure 1: Theoretical curves representing the MTF and the LPF is 20 Hz because the dominant component of modm(fm ) for various conditions with T R = 0.1, 0.3, 0.5, 1.0, ulation region for speech perception and speech recognition
exists from 1 to 16 Hz.
and 2.0 s
Figure 2(a) shows a block diagram of the IMTF-based inverse filtering on the power envelope. In this method, theh(t) is represented as [5]spectrum of the power envelope, that is, the modulation spec1
trum, could be restored by using IMTF-based filtering in−
2 2TRwhich modulation frequencies were limited to 20 Hz usingm(fm ) = 1 + 2πfm
(2)the LPF. The estimation method of reverberation time in the13.8
time domain could calculate the best reverberation time T̂R
where fm is the modulation frequency. Figure 1 shows the for reasonable power envelope restoration. However, thetheoretical curves of MTF, m(f m ), with various T R s. From actual LPF could not completely remove the remains that
this figure, the MTF can be regarded as characteristics of a were higher modulation spectra over 20 Hz. Since the relow-pass filtering in the modulation frequency domain.
mains on the power envelope were over-emphasized by theIMTF-based filtering, the emphasized remains cause the dipsof power envelope that dominate the estimation accuracy of
3. IMTF-Based Filtering on Power Envelope
reverberation time. Thus, T̂R was underestimated due toIn the IMTF-based filtering on the power envelope [5, 6], the remains and improvement of restoration accuracy by thisthe following useful relation is used.
method was saturated as T R increases. This is a remaining
problemof the IMTF-based filtering on the power envelope.
2∞ 2 y (t) =x(τ )h(t − τ )dτ−∞
4. IMTF-Based Filtering on Modulation Spectrum
TR = 2 s=∞

e2x (τ )e2h (t − τ )dτ = e2y (t)(3)

We propose another type of IMTF-based filtering to solve
the above problem. Figure 2(b) shows the proposed method.
where e2x (t), e2h (t), and e2y (t), are the power envelopes of the To remove the remains on the power envelope, the proposed method represents the power envelope e 2y (t) by downinput x(t), the RIR h(t), and the output y(t), respectively.
2On the basis of this result, e x (t) can be recovered by decon- sampling from 20 kHz to 40 Hz (M = 500) and then repvoluting e 2y (t) = e2x (t) ∗ e2h (t) with e2h (t). Here, the transmis- resents the modulation spectrum of e 2y (t) within 20 Hz. Wesion functions of power envelopes E x (z), Eh (z), and Ey (z) incorporated the estimation method of reverberation time as a
are assumed to be the z-transforms of e 2x (t), e2h (t), and e2y (t), blind-method by Hiramatsu and Unoki [8] into the proposed
method to estimate T R at the dominant modulation frequency.
respectively. Thus, E x (z) can be determined from
Here, Eq. (5) was used to determine the parameter of â.
Ey (z)13.8−1Then,IMTF-based filtering on the modulation spectrum inEx (z) =1 − exp −z(4)a2
T R · fs
Eq. (4) was used to restore the modulation spectrum of rever2
where fs is the sampling frequency. This means that modu- berant signal. Finally, the restored power envelope ê x (t) was2
obtainedfromthemodulation
spectrumofE(z)byinversexlation spectrum Ex (z) of ex (n) can be obtained from E y (z)Fouriertransform.
2
times inverse MTF, 1/Eh (z). Therefore, e x (t) can then beobtained from the inverse z-transform of E x (z). Here, twoparameters (a and T R ) are obtained as [5, 6].
5. Evaluation

∞
13.8t
We evaluate the proposed method as to whether it can reâ = 1/exp −dt(5)solve the above problem. Original signals x(t) consisted ofT̂R0−∞

(a) IMTF-based filtering on power envelope
Reverberant
signalPower envelope
extractiony(t)Recoveredpower envelope

IMTF-based filteringon power envelope^e 2x (t)Parameter estimation(time domain)(b) IMTF-based filtering on modulation spectrumReverberant
signaly(t)Power envelope
extraction

MIMTF-based filteringon modulation spectrum

FFTRecoveredpower envelope
IFFTM^e 2x (t)Parameter estimation(modulation frequency domain)

Figure 2: Block diagram of IMTF-based filtering (a) on the power envelope and (b) on the modulation spectrum
white noise multiplied by three types of power envelope:

2. Harmonics power envelope
e2x (t) = 1 +1KK

sin(2πkF0 t + θk )k=1

3. Band-limited noise e 2x (t) = LPF[nω (t)]0Normalized modulation spectrum [dB]

1. Sinusoidal e2x (t) = 1 − cos (2πF t)5−5−10−15−20Here, F = 10 Hz, F0 = 1 Hz, K = 20, θk is a random−25
phase, and the cut-off frequency of LPF[·] was 20 Hz. TheRIRs, h(t)s, consisted of five types of envelope: e h (t) with
Clean
−30
Reverberation
TR = 0.1, 0.3, 0.5, 1.0, and 2.0 s in which a was set in Eq.
IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum(5) with each T R , multiplied by 100 white noise carriers. All−3502468101214161820stimuli y(t) were composed through 1, 500 (= 3 × 5 × 100)Modulation frequency [Hz]convolutions of x(t) with h(t).
Figure 3: Restoration modulation spectrum for sinusoidal
In this paper, to evaluate both the error and similarity in thepower envelope with T R = 1.0 s
terms of the power envelopes, we thus used (i) correlation and(ii) SNR (S was the power envelope of the original signal andN was the power envelope of the recovered power envelope). was underestimated in the time domain due to the remains
and this caused saturation of the improvement of restoration
Corr(e2x , ê2x )accuracy.
T 2(ex (t)−e2x (t))(ê2x (t)−ê2x (t))dt
Figures 4–6 show the improvements of restoration accu0= (8) TTracy for the three types of power envelope. In these figures,(e2x (t)−e2x (t))2 dt(ê2x (t)−ê2x (t))2 dt
00
panel (a) shows the improved correlation and panel (b) shows
T 2
the improved SNR. From these results, it was found that the(ex (t))2 dt
2 20(9) proposed method could effectively improve the restoration
SNR(ex , êx ) = 10 log10  T
2 (t) − ê2 (t))2 dt
accuracy in comparison with the previous method. These(exx0improvements were not so great for the last two power en2
2velopes. This may have been caused by different shapes ofwhere the notation of e x (t) means the averaged e x (t).
Figure 3 shows the modulation spectrum of the sinusoidal dominant peaks in the modulation spectrum.
power envelope where the peak is 10 Hz. This peak indicatesthe dominant component of the sinusoidal power envelope. 6. Conclusion
Around this dominant component, the shape of the restored
In this paper, we studied the possibility of solving the repower envelope by the proposed method corresponded withthat of the original one. In contrast, the shape in the previous maining problem of IMTF-based filtering on the power envemethod is under that of the original one. This is because T R lope and then proposed IMTF-based filtering on the modula-

0.4
0.3
0.2
0.1

Improved correlation

0.5IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum

(a)

50.5

11.5

Reverberation time TR (s)Figure 4: Comparison with the envelope restoration accuracy
for a sinusoidal power envelope: (a) improved correlation and(b) improved SNRtion spectrum. Three simulations were carried out to evaluate
the proposed method as to whether it could resolve the problem. It was found that the proposed method could adequately
improve restoration accuracy of the power envelopes in comparison with our previous method. There were improvementsin power envelopes, however, the degree of improvement was
not as big as we expected. Therefore, we propose that theIMTF-based filtering on modulation spectrum had an advantage. We confirmed the influence of the harmonic componentof over 20 Hz in modulation frequency as one of the causesof saturation of the accuracy of improvement of IMTF-based
filtering on the power envelope.

1510500

20.5

11.5

Reverberation time TR (s)2

Figure 5: Comparison with the envelope restoration accuracy
for a harmonic power envelope
0.5

Improved correlation

00

Improved SNR [dB]

10IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum

0.10

IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum

(b)

0.2

20(b)

IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum

0.3

2015(a)0.40(a)IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum(b)

IMTF−based filtering on power envelope
IMTF−based filtering on modulation spectrum

0.40.30.20.1020

Improved SNR [dB]

Improved SNR [dB]

Improved correlation

0.5

15105000.5

11.5

Reverberation time T (s)2RAcknowledgements
This work was supported by the Strategic Information andCOmmunications R&D Promotion ProgrammE (SCOPE)(071705001) of the Ministry of Internal Affairs and Communications (MIC), Japan.
References[1] S. T. Neely and J. B. Allen: Invertibility of a room impulse response, J. Acoust. Soc. Am., Vol. 66, No. 1, pp.
165–169, 1979.
[2] M. Miyoshi and Y. Kaneda: Inverse filtering of roomacoustics, IEEE Trans. Speech Signal Process., ASSP,Vol. 36, pp. 145–152, 1988.
[3] H. Wang and F. Itakura: Realization of acoustic inverse
filtering through multi-microphone sub band processing, IEICE Trans. Fundam., Vol. E75-A, pp. 1474–1483,
1992.
[4] T. Houtgast and H. J. M. Steeneken: A review of theMTF concept in room acoustics and its use for estimating

Figure 6: Comparison with the envelope restoration accuracy
for a band-limited noise power envelope
speech intelligibility in auditoria, J. Acoust. Soc. Am.,
Vol. 77, pp. 1069–1077, 1985.
[5] M. Unoki, M. Furukawa, K. Sakata and M. Akagi: An
improved method based on the MTF concept for restoring the power envelope from a reverberant signal, Acoust.
Sci. Tech., Vol. 25, No. 4, pp. 232–242, 2004.
[6] M. Unoki, K. Sakata, M. Furukawa and M. Akagi: Aspeech dereverberation method based on the MTF concept in power envelope restoration, Acoust. Sci. Tech.,Vol. 25, No. 4, pp. 243–254, 2004.
[7] M. R. Schroeder: Modulation transfer function: definition and measurement, Acoustica, Vol. 49, pp. 179–182,
1981.
[8] S. Hiramatsu and M. Unoki: A speech dereverberation
method based on the MTF concept in power envelope
restoration, J. Signal Processing, Vol. 12, No. 6, pp. 351–361, 2008.