JAIST Repository
https://dspace.jaist.ac.jp/

Title

An MTF‑based Blind Restoration Method for
Improving Intelligibility of Bone‑conducted
SpeechAuthor(s)Kinugasa, Kota; Unoki, Masashi; Akagi, Masato

Citation

2009 International Workshop on Nonlinear Circuits
and Signal Processing (NCSP'09): 105‑108

Issue Date

2009‑03‑01

Type

Conference Paper

Text version

publisher

URL

http://hdl.handle.net/10119/9966

RightsThis material is posted here with permission of
the Research Institute of Signal Processing
Japan. Kota Kinugasa, Masashi Unoki, and Masato
Akagi, 2009 International Workshop on Nonlinear
Circuits and Signal Processing (NCSP'09), 2009,
pp.105‑108.

Description

Japan Advanced Institute of Science and Technology

2009 International Workshop on Nonlinear Circuits and Signal Processing
NCSP'09, Waikiki, Hawaii, March 1-3, 2009

An MTF-based blind restoration method for improving intelligibility of bone-conducted speech
Kota Kinugasa, Masashi Unoki, and Masato Akagi
School of Information Science, Japan Advanced Institute of Science and Technology
1–1 Asahidai, Tatsunokuchi, Nomi, Ishikawa, 923–1292 Japan
Phone/Fax: +81–761–51–1699 ex, 1391/+81–761–51–1149
Email: {k-kota, unoki, akagi}@jaist.ac.jp

Abstract
Bone-conducted (BC) speech is useful for speech communications in extremely noisy environments. However, soundquality and intelligibility of BC speech are very poor andlow. In this paper, we propose a blind restoration method
for improving BC speech based on the concept of modulationtransfer function (MTF). We first investigate the relationshipbetween the power envelope of air-conducted (AC) and BC
speech signals by analyzing AC/BC database. We then model
these relations by fitting some models to the MTF derived
from the database. The best-fitted model has two parameters.
One is the gain factor, and another is the attenuation factor.
We then propose a method for determining the parameters ofmodel without the AC speech. We finally evaluate the proposed method using SNR, correlation, RMS of MTF, PESQ,
and LSD. As the results, we showed the proposed method iseffective for blind BC speech restoration.
1. Introduction
It is very difficult for us to accomplish speech communications in extremely noisy environments such as factory anddisaster sites. One of the solutions is to use the speech witha bone conduction microphone because BC speech can berecorded by this microphone without interference of external noises. However, sound quality and intelligibility of BC
speech are very poor and low [1]. Therefore, compensatingthese losses of BC speech is needed for speech communications using BC speech and it is a challenging topic.
Generally, the power attenuation of BC speech is stronger
than that of AC speech at higher frequencies. A straightforward method for restoring BC speech is to compensate theseattenuated frequency components by using high-pass filtering. Since these attenuations vary in a complex manner depending on BC pickup points, speakers, and pronounced syllables, it is very difficult to design one unique type of highpass filtering with these variations. There are various methods of deriving inverse filtering such as cross-spectrum [2]and long-term Fourier transform methods [3], however, theseyield the restored speech signals with artifacts, i.e., echoes.
Therefore, there is less improvement in voice quality. Furthermore, the AC speech is needed to design the inverse filtering in these models [2, 3].
On the other hand, by considering the relationship between
AC and BC speech signals as the transfer function, we havebeen studied a common strategy based on the source-filter^ey1(t)#1^ex1(t)Envelope restoration
X

BoneCarrier extraction
conducted N-channel y1(t) ^^cx1(t)cy1(t)
speechConst. BWy(t)Filterbank(Analysis)^eyN(t)#N^exN(t)N-channel
Const. BW
Filterbank(Synthesis)

Resotored
speech^x(t)Envelope restoration
X

yN(t)

Carrier extraction^cyN(t)^cxN(t)Figure 1: Restoration method based on MTF.
model for improving the intelligibility and sound quality ofBC speech. As the results, we found that the filter characteristics are more important than the source characteristics
for improving sound quality and intelligibility of BC speechbased on the source-filter model [4, 5, 6]. Vu et al. thus proposed the LP-based blind BC speech restoration method thatoriginates from the idea of the source-filter model in the frequency domain [4]. Although this can restore the BC speech
blindly, this method required learning AC-LP (AC-LSF) coefficients. Kimura et al. proposed a BC speech restoration
method based on MTF concept in the time domain [5]. Thismethod compensates the reduced modulation index of eachtemporal power envelope in the filterbank model. Becauseof MTF relating to speech intelligibility, this method can improve the intelligibility of BC speech directly. However, AC
speech is needed for restoring the BC speech in their method.
Moreover, their method was exactly effective to restore the
BC speech, but it is debatable to determine what type ofmodel of MTF is the most useful to restore the BC speech.
Since it is very important to improve the loss of speech
intelligibility as well as voice quality for speech communications, in this paper, we aim to propose an MTF-based method
for blindly restoring BC speech.
In this paper, first, we investigate the relationship between
power envelopes of AC and BC speech. Secondly, the MTF
derived from the database is modeled by fitting. Thirdly, we
propose the method determining the parameters of the model
without the AC speech. Finally, we propose an MTF-based
method for blindly restoring BC speech.
2. Concept of MTF-based restoration method
The MTF concept was proposed by Houtgast et al. [7] topredict the speech intelligibility. Drullman revealed [8] that a
temporal envelope information appears to be more importantfor speech intelligibility than carrier information. We think- 105 -

n=1

n=1

N
X

N
X

y(t) :=n=1

yn (t) =eyn (t) · cyn (t),Correration

SNR (dB)
100

150

0
−5

200

0

50

(2)

n=1

150

Gain (dB)

(c)

200

0
−0.05

(d)
100
50
0

50

100

150

200

0

2000

Channel number

(1)

100

Channel number

0.05

0

e xn (t) · c xn (t),

50

4000

6000

8000

Frequency (Hz)
−20

10
5
0

(e)

Power (dB)xn (t) =

0

(b)

5

Channel number

2x(t) :=

N
X

(a)

0.5

0

Parameter of 1/a (dB)NX

1Slope of the MTF

that differences between AC and BC temporal envelopes affect speech intelligibility and sound quality significantly.
The BC speech restoration method in the filterbank isshown in Fig. 1. AC and BC speech signals are decomposed into the temporal envelopes, e x (t) and ey (t), and thecarriers, c x (t) and cy (t), by N-channel bandpass filterbankswith a constant bandwidth (40-Hz). Here, we assume that thesignals,x(t) and y(t), can be represented as

−5

(f)
−40
−60
−80

0

50

100

150

200

0

50

Channel number

100

150

200

Channel number

3. Characteristics of bone conduction

SNR (dB)

Gain (dB)

(d)

0
−0.05

100
50
0

0

50

100

150

200

0

2000

Channel number

4000

6000

8000

Frequency (Hz)
−20

10
5
0

(e)

−5

Power (dB)

where Eh (z), E x (z), and Ey (z) are respectively the z-transformof e2h (t), e2x (t), and e2y (t). The impulse response between AC
and BC power envelopes, eh (t), was defined in previous paper[5] as eh (t) = a exp(−bt), where a is the gain factor and b is
the factor to control attenuation. However, it was not surewhether an MTF model as a exp(−bt) is the best model or not
to restore the BC speech.

(c)
0.05

2

(5)

Parameter of 1/a (dB)Eh−1 (z) = E x (z)/Ey (z),Slope of the MTF

Correration

where, xn (t) and yn (t), are the bandpass signals. Power enve- Figure 2: Analysis results of all dataset (solid line: mean,
lope and carrier of the BC speech are calculated asdashed line: mean ± standard deviation). (a) correlation, (b)e2yn (t) = LPF[|yn (t) + jHilbert(yn (t))|2 ],(3) SNR, (c) slope of MTF, (d) transfer function, (e) mean ofpower ratio of power envelope (parameter 1/a of model) andcyn (t) = yn (t)/eyn (t),(4)
regression curve, and (f) mean of e2y (t) of each channel.
1
where Hilbert( · ) is the Hilbert transform and LPF[ · ] de(a)(b)
5notes the low-pass filtering with a 20-Hz cut-off frequency.
00.5
e x (t) and c x (t) can also be calculated from x(t) usnig the same
−5
−10
method. Then, inverse filtering, Eh−1 (z) is used to restore the
0
0
50
100
150
200
0
50
100
150
200
BC speech as follows
Channel number
Channel number

(f)
−40
−60
−80

0

50

100

150

Channel number

200

0

50

100

150

200

Channel number

Figure 3: Analysis results at measurement point 2. Format is
A question we have is how to design the inverse filter tothe same that in Fig. 2.
restore the BC speech. We analyze the characteristics of bone
conduction to design the inverse filtering, Eh−1 (z).
AC and BC power envelopes. Here F [·] is the long-term
Fourier transform. Kimura et al. had been analyzed the char3.1. AC/BC speech databaseacteristics for the measurement at point 5 (calvaria). We anWe used the AC/BC speech database [6] to analyze the alyze the characteristics for the measurements at all points,characteristics between AC and BC power envelopes. BC and we carefully investigate the characteristics of bone conspeech was collected at five measurement points (1: mandibu- duction.
lar angle, 2: temple, 3: philtrum, 4: forehead, and 5: cal3.3. Results and discussion
varia). Different microphones were used at points from 1 to
4 and at the point 5. 25 Japanese words of each degree of faFigure 2 shows the results of all dataset and Figure 3 shows
miliarity were chosen from NTT-database [9]. Speakers were the results at measurement point 2. Solid lines show the mean
5 males and 5 females.
and dashed lines show the mean ± standard deviation. Figures
2 (a) and (b) show the distortion of BC power envelope. Fig3.2. Analysis of Characteristics of bone conduction
ure 2 (c) shows the slope of regression line of MTF (1 to 10
We analyzed the characteristics between AC and BC Hz). The reason to limit the range of MTF from 1 to 10-Hz
power envelopes with the AC/BC speech database using is that MTF of at the higher modulation frequencies is influthe following measures: (1) correlation of AC and BC enced by the internal noise such as blood flow, and noise of
power envelopes, (2) SNR (S: e2x (t), N: e2x (t) − e2y (t), (3) transmission-line, and noise flooring. If the value of slope


.R ∞
R∞
is negative, the MTF has low-pass characteristics and if the
MTF M(ω) = 0 e2h (t) exp(− jωt)dt 0 e2h (t)dt , (4) trans- value of slope is positive, the MTF has high-pass characterisfer function (| F [y(t)]/ F [x(t)]|), and (5) power ratio between tics. This result shows the MTF has low-pass characteristics- 106 -

1

in most channels. Figure 2 (d) shows that the characteristicof bone conduction is low-pass. Figure 3 (e) shows the meanof power ratio of power envelope of BC signal to AC signal
in each channel. We fitted some curves to the mean of power
ratio and investigated that the power ratio can be approximated by regression curve as −cω−1 + d, where c and d areparameters depend on measurement point. These characteristics were also shown in the results at the other measurement
points in our analysis. We can approximate the power ratio of
BC power envelope to AC power envelope of each channel bychanging the parameter of regression curve. Also, the analysis results show that the regression curve does not depend on
the difference of speaker and difference of syllable so much.

0.8

Modulation index

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

0

2

4

6

8

10

12

14

16

18

20

Modulation frequency (Hz)Figure 4: Comparison between MTFs derived from thedatabase and model. MTF without internal noise: MTF thatremoved internal noise derived from database. at exp(−bt):
one of exponential curve. a exp(−bt): the model of MTF usedin previous method. MTF: MTF derived from database. LPF:
low-pass filter.
0.01

Slope of the MTF

0
−0.01
−0.02
−0.03
−0.04
−0.05

mean
mean ± std

−0.06
−0.07
3

mean
mean+std

2.5
2

RMS

3.4. Modeling the MTF between AC and BC power envelopes.
In the previous method [5], MTF was represented as an exponential model. However, it was not evident whether MTF
can be represented by an exponential curve. We confirmedthat the MTF has low-pass characteristics from the analysis
result. Thus, we model the MTF by fitting three low-passmodels (one of exponential curve eh (t) = at exp(−bt), themodel of MTF used in previous method eh (t) = a exp(−bt),
and eh (t) = low-pass filter) to the MTF derived from thedatabase. Figure 4 shows a case of the MTF derived fromthe database and MTF that removed internal noise derived
from database and three models. The shape of the MTF derived from the database seems to be rippled. Because MTF
without internal noise did not fluctuate, influence of internal
noise might ripple the shape of MTF. The model a exp(−bt) is
especially suitable in this case. Figure 5 shows the mean andthe standard deviation of the analysis results using a exp(−bt).
Upper panel shows the slope of regression line of model that
fitted MTF derived from the database. If the value of slope
is zero, the MTF does not influence the BC speech. Lower
panel shows the root mean squared (RMS) differences between model and that of MTF derived from database. Because the RMS difference for a exp(−bt) is the smallest in allmodels, we assumed that the MTF relation, eh (t), can be represented by the model as eh (t) = a exp(−bt). Then, we canmodel the MTF by fitting models to the MTF derived fromthe database and finally determined inverse filtering as follow
(
!)
1
2b
,
Eh−1 (z) = 2 1 − exp −
(6)
fs
a

MTF without internal noise
a*t*exp(−b*t)
a*exp(−b*t)
LPF
MTF

0.9

1.5
1
0.5
0

0

20

40

60

80

100

120

140

160

180

200

Channel number

Figure 5: Analysis results of MTF by fitting model. Upper:
Slope of regression line of model that fitted MTF derived
from database. Lower: RMS of difference between modeland that fitted MTF derived from database.
determine the parameter a without AC speech to study eachmeasurement of the regression curve.
To estimate the parameter b, we used the method proposed
where f s is the sampling frequency of 16 kHz.
by Hiramatsu and Unoki [10] which was originally introduced for estimating reverberation time based on MTF con4. MTF-based blind restoration method
cept. The utilization of this method is that our research and
The previous method [5] needs AC speech to determine the their work are based on exactly the same model. The paramparameter of MTF model and to set restoring condition. From eter b can be estimated as
the analysis results, we improve the previous method from the(7)b̂ = arg min Êy (0) · |M( fdm , b)|/Êy ( fdm ) ,following two standpoints.
b4.1. Parameter determination

where fdm is dominant frequency of BC power envelope andOne of the stand point is how we determine the two param- Êy (·) is power envelope of the restored speech.
eters of model, a and b, for restoring the BC speech. For the 4.2. Modification in recovering condition
method for determining parameter a, Figure 3 (e) shows theAnother stand point is that the previous method restores theparameter a can be approximated by the regression curve andthis curve depends only on the measurement point. We can BC speech when the correlation between AC and BC power- 107 -

speech restoration.

Correration

1
0.8
0.6

6. Conclusion

0.4
0.2

BC power envelope
Restored power envelope

We analyzed the characteristics between the AC and BC
power envelopes with the AC/BC speech database. As the5results, we found that the characteristic of MTF is the low0
pass filtering and each measurement point of power ratio
can be approximated by regression curve. We then mod−5
BC power envelope
Restored power envelope
eled the MTF as a exp(−bt) and proposed the methods to−10
determine the parameters of model without the AC speech.
15
As the results, we proposed the MTF-based blind restoration method. Finally, we evaluated the proposed method and10
showed that the proposed method was effective for blind BC
BC power envelope
5
Restored power envelope
speech restoration.
0
20
40
60
80
100
120
140
160
180
200
Channel number
As the next step in our research, we will carry out the comFigure 6: Improving of correlation, SNR and RMS of MTF. prehensive evaluation and investigate the performance of the
Solid line: BC speech. Dashed line: restored speech by pro- proposed method.
posed method.
Acknowledgmentenvelopes is not over 0.8 and the relative power of AC powerenvelope is over −20 dB. To restore the BC speech without This work was supported by a Grant Program by the YAZAKI
Memorial Foundation for Science and Technology It was also parAC speech, we change these conditions to an appropriately tially supported by the Strategic Information and COmmunications
condition that the method restores the BC speech when the R&D Promotion ProgrammE (SCOPE) (071705001) of the Ministry
relative power of BC power envelope is over −40 dB. The of Internal Affairs and Communications (MIC), Japan.
reason we set such a condition is that the internal noise in
the BC speech appears when the relative power of BC power
References
envelope is not over −40 dB. Because internal noise increased
the power of DC of MTF, the method cannot estimate the true [1] Kitamori, S., and Takizawa, M., “An analysis of bone conducted speech signal by articulation tests,” IEICE Trans., J72value of parameter b.
A(11), 1764-1771, Nov. 1989.
Based on the above results, an MTF-based method for[2] Ishimitsu, S., Kitakaze, H., Tsuchibushi, Y., Yanagawa, H.,
blindly restoring BC speech was proposed.
and Fukushima, M., “A noise-robust speech recognition sys0

RMS (dB)

SNR

10

5. Evaluations
We carried out simulations to evaluate the proposed
method using the AC/BC database. The correlation and SNRof the power envelopes of AC and the restored speech signals
or the power envelopes of AC and BC speech signals wereused to evaluate the improvement of restoration of the powerenvelopes. Perceptual evaluation of sound quality (PESQ)[11] that was recommended by ITU-T P. 862, and log spectral distortion (LSD) were used to evaluate the improvementsof speech quality. RMS of MTF was used to evaluate the improvement of intelligibility of the restored speech. RMS of
MTF means RMS difference between MTF that is attenuation and MTF that is not attenuation. If the all modulation
frequency of MTF is 0 dB, the MTF does not influence the
BC speech. Therefore, the closer to 0 the value of RMS of
MTF, the better the improvement of MTF. Figure 6 shows anexample of the SNR, correlation, and RMS of MTF. The solidlines show the result of the BC speech and the dashed lines
show the result of the restored speech by the proposed method
using parameter a derived from regression curve. These results showed that the power envelopes can be adequately restored by the proposed method and RMS of MTF showed
the improvement of intelligibility. LSD value has been decreased by 1.5 dB and PESQ value has been increased by 0.7
points. These results showed that the sound quality can be
improved by the proposed method. The results of evaluations
showed that the proposed method is effective for blind BC

tem making use of body-conducted signals,” Acoust. Sci. &,Tech., 25(2), 166-169, Mar. 2004.
[3] Tamiya, T., and Shimamura, T., “Reconstruction filter designfor bone-conducted speech,” Proc. ICSLP2004, ll, 1085-1088,
Oct. 2004.
[4] Vu, T. T., Seide, G., Unoki, M., and Akagi, M., “Method of LPbased blind restoration for improving intelligibility of boneconducted speech,” Proc. Interspeech2007, 966-969, Belgium,
Aug. 2007.
[5] Kimura, K., Unoki, M., and Akagi, M., “A study on a boneconducted speech restoration method with the modulation filterbank,” NCSP05, 411-414, Honolulu, USA, Mar. 2005.
[6] Vu, T. T., Kimura, K., Unoki, M., and Akagi, M., “A study
on restoration of bone-conducted speech with MTF-based andLP-based models,” J. Signal Processing, 10(6), 407-417, Nov.
2006.
[7] Houtgast, T., and Steenken, H. J. M., “The Modulation Transfer Function in Room Acoustics as a Predictor of Speech Intelligibility,” ACUSTICA. 54, 557, Aug. 1973.
[8] Drullman, M. “Temporal envelope and fine structure cues forspeech intelligibility,” J. Acoust. Soc. Am., 97, 585-592, Jan.
1995.
[9] Database for speech intelligibility testing using Japanese wordlists. NTT–AT, Mar. 2003.
[10] Hiramatsu, S., and Unoki, M. “A Study on the Blind Estimation of Reverberation Time in Room Acoustics,” J. Signal Processing, 12(4), 323-326. July 2008.
[11] Yi, H., and Philipos, C. L., “Evaluation of objective measures
for speech enhancement,” Interspeech2006, 1447-1450, Pittsburgh, Pennsylvania, Sept. 2006.

- 108 -