JAIST Repository
https://dspace.jaist.ac.jp/

Title

An MTF‑based method of blind restoration forimproving intelligibility of bone‑conducted
speechAuthor(s)Kinugasa, Kota; Unoki, Masashi; Akagi, Masato

Citation

Journal of Signal Processing, 13(4): 339‑342

Issue Date

2009‑07

Type

Journal Article

Text version

publisher

URL

http://hdl.handle.net/10119/9182

RightsCopyright (C) 2009 信号処理学会. Kota Kinugasa,
Masashi Unoki and Masato Akagi, Journal of Signal
Processing, 13(4), 2009, 339‑342.

Description

Japan Advanced Institute of Science and Technology

Journal of Signal Processing. Vol. 13. No.4. pp. 339-342. July 2009

SELECTED PA PER

An MTF-Based Method of Blind Restoration for Improving Intelligibility of Bone-Conducted
SpeechKota Kinugasa, Masashi Unoki and Masato Akagi
School of Infonnation Science, Japan Advanced Institute of Science and Technology
1-1 Asahidai, Tatsunokuchi, Nomi, Ishikawa, 923-1292, Japan
E-mail: {k-kota.unoki.akagi}@jaist.ac.jp

Abstract
Bone-conducted (BC) speech is useful for speech communications in extremely noisy environments. However, boththe sound quality and intelligibility of BC speech are very
poor. We propose a method of blind restoration for improving
BC speech based on the concept of the modulation transfer
function (MTF). We investigated the relationship between the
power envelope of air-conducted (AC) and BC speech signals
by analyzing an ACIBC database. We then modeled these relations by fitting some models to the MTF derived from thedatabase. The best-fitted model had two parameters. The firstwas the gain factor and the second was the attenuation factor. We also propose a method of detennining the parameters of the model without AC speech. We evaluated the new
method using SNR, correlation, MTF, PESQ, and LSD. The
results revealed that the proposed approach could effectively
and blindly restore BC speech.

1. Introduction
It is very difficult for us to communicate through speech inextremely noisy environments such as those in factories anddisaster sites. One of the best solutions is to use speech witha bone conduction microphone because BC speech can berecorded with this without interference from external noises.
However, both the sound quality and intelligibility of BC
speech are very poor [1]. Therefore, we need to compensate
for these losses in BC speech to accomplish communicationand this is a concern that presents numerous challenges.
The attenuation of BC speech is stronger than that of AC
speech at higher frequencies. A straightforward method ofrestoring BC speech is to compensate for these attenuated frequency components by using high-pass filtering. Since attenuation in BC speech signals varies and can be complex depending on the BC pickup points, speakers, and pronounced
syllables, it is very difficult to design one unique type of highpass filtering with these variations. There are various methods of deriving inverse filtering such as cross-spectrum [2]and long-tenn Fourier transfonn methods [3]; however, theseyield the restored speech signals with artifacts, i.e., echoes,
so there are only slight improvements in voice quality.
However, by considering the relationship between AC andBC speech signals as a transfer function, we studied a common strategy based on the source-filter model to improve
the intelligibility and sound quality of BC speech. As a result, we found that filter characteristics are more important

Journal of Signal Processing, Vol. 13, No.4, July 2009

than source characteristics to restore BC speech based on thesource-filter model [4, 5]. Vu et al. proposed an LP-based
method of blindly restoring BC speech that originated fromthe idea of the source-filter model in the frequency domain[5]. Although this could restore BC speech blindly, machinelearning methods had to be used to predict the AC-LP coefficients from BC-LP coefficients.
In contrast, Kimura et al. proposed a method of restoring BC speech based on the MTF concept in the time domain[4]. This method compensated for the reduced modulation index of all temporal power envelopes in the filterbank model.
Because MTF is related to speech intelligibility, this method
could directly improve the intelligibility ofBC speech. However, AC speech was needed to restore BC speech with their
method. Although it was extremely effective in restoring BC
speech, it is debatable what type of model ofMTF is the mostuseful for restoring BC speech.
Since it is very important to improve the loss ofintelligibility as well as the voice quality of BC speech for speech communications, we propose an MTF-based method of blindly
restoring BC speech. Thus, we attempted to analyze the characteristics of AC and BC power envelopes to design an MTFbased inverse filter and to find a method of blindly determining the parameters of the inverse filter without AC speech.
2. MTF-Based Model
The MTF concept was proposed by Houtgast andSteeneken [6] to predict speech intelligibility. Drullman revealed [7] that temporal envelope appears to be more important in speech intelligibility than carrier infonnation. We
think that the differences between AC and BC envelopes significantly affect speech intelligibility and sound quality.
Our method of restoring BC speech involves a filterbank,which is outlined in Fig. 1. Let x ( t) be AC speech and y (t)be associated BC speech. We assume that the signals with the
N -channel bandpass filterbank can be represented as
NNx(t) = I:Xn(t) = I:eXn(t). CXn(t)n=ln=ly(t) =

N

N

n=l

n=l

I: Yn(t) = I: e (t) . cyJt)
Yn

(1)

(2)

where xn(t) and Yn(t) are the bandpass signals, ex(t) andey(t) are the temporal envelopes, and cx(t) and cy(t) are339

'r:"~~-'- "'-~"~"-(~)':

Envelope restoration
Boneconducted N-channel
speechConst. By(t)Filterbank(Analysis)

0.5 :f""~,Carrier extraction
N-channel

Cons!. BW

Resotored
speechFilterbank(Synthesis)~....... _ _ ~--

"

~

0.02

~

' , __ ;' _..

o·0-0.02.

!]C),". --..-., '.'_____ .;..----.-----........... -

~ 100

.f:

i,

i-o.o.t ". . . _-. "., - ."',' -',;," -.~Carrier extraction-.~~l·'~)- - - - - - _ "..... _. _ "_~ ,"'- . "0·5 ; : ' • . ,,' .' .• , . . . . . . .'-~-~-~--'°0'~ ~: b~.Channe~ ~u~:~r ~)~(t)Envelope restoration·I:(-"...• ..
~"~.

- • " " ..' " .~ -. .. "; "' _ ",

;

l3

50

<IJ

Channel number (n)N~l' " ,.

~

0

..

-,

1 f\

Frequency (Hz)jJ:(r-~~-/J L~~~~~<~.

Figure 1: Method of restoration based on MTF
the carriers in the n-th channel of the fiIterbank at a constantbandwidth (40-Hz).

LPF [IYn(t) + jHilbert(Yn(t))1 2 ](3)Yn(t)/e yn (t)(4)

where Hilbert(·) is the Hilbert transfonn and LPF[·] denotes
low-pass filtering at a 20-Hz cut-off frequency. ex(t) andC x (t) can also be calculated from x( t) using the same method.
Then, E;:l(z) is used to restore the BC speech as(5)where Eh(Z), Ex(z), and Ey(z) correspond to the ztransfonn of e~(t), e~(t), and e~(t). Relation between e~(t)and e~(t), e~(t), was defined in our previous paper [4] ase~(t) = a~ exp( -2bnt), where an is the gain factor and bn
is the factor to control attenuation. However, there is no evidence whether the MTF model as a~ exp( -2bn t) is the best
representation for restoring BC speech.

Channel number (n)·200Channel number (n)Figure 2: Results of analysis for all datasets (solid line: mean
and dashed line: mean ± standard deviation): (a) correlation,(b) SNR, (c) slope ofMTF, (d) transfer function, (e) mean ofpower ratio of power envelope (1/ a~) and regression curve,and (f) mean of e~(t) for each channel(4) the transfer function, IF[y( t) l/ F[x( t) II, and (5) the power
ratio, a~ = 10 10gloCr: e;n (t)dt/ 1;{ e~n (t)dt). Here .1'[.]is the long-tenn Fourier transfonn. Kimura et al. had analyzed the characteristics for measurement at point 5 (calvaria). We carefully analyzed and investigated the characteristics for measurements at all points.

3.3 Results and discussion

Figure 2 presents the results for all datasets and Fig. 3
has the results at measurement point 2. The solid lines indicate the mean and the dashed lines are the mean ± standard
3. Characteristics of Bone Conduction
deviation. Figures 2(a) and (b) show the distortion in the BC
The question is how can we design an inverse filter to re- power envelope. Figure 2( c) shows the slope of the regression
store BC speech? We analyzed the characteristics of bone line of MTF (1 to 10 Hz). The reason we limited the range
of MTF from 1 to 10Hz was that MTF at higher modulation
conduction to design an MTF-based inverse filter, E;:l(z).
frequencies is influenced by internal noise such as blood flow,transmission-line noise, and noise flooring. If the value of
3.1 ACIBC speech databasethe slope is negative, MTF has low-pass characteristics and if
the value of the slope is positive, MTF has high-pass characWe used an ACIBC speech database [4] to analyze the char- teristics. This indicates MTF has low-pass characteristics inacteristics of AC and BC power envelopes. BC speech was most channels. Figure 2( d) indicates that bone conduction has
collected at five measurement points, i.e., (l: mandibular an- low-pass characteristics. Figure 3(e) shows the mean for thegle, 2: temple, 3: philtrum, 4: forehead, and 5: calvaria). power ratio of the power envelope of BC signals to AC signals
Different microphones were used at points 1 to 4 and at point in all channels. We fitted various curves to the mean of the
5. Twenty-five Japanese words of each degree of familiarity power ratio and investigated whether the power ratio could be
were chosen from an NTT-database [8]. The speakers were approximated by a regression curve as 1/ it ~ = . ._. en -1 + d,five males and five females.
where c and d are parameters that depend on the measurement point. These characteristics were also in the results atthe other measurement points in our analysis. We could ap3.2 Analysis of characteristics of bone conduction
proximate the power ratio of the BC power envelope to the
We analyzed the characteristics of AC and BC power en- AC power envelope for all channels by changing the paramvelopes using five measures: (1) the correlation between eters of the regression curve. Also, the results obtained frome;(t) and e~(t) (2) the SNR (S: e~(t), N: e;(t) - e~(t), (3) analysis reveal that the regression curve did not greatly de00
00
the MTF, M(w) =e~(t)exp(-jwt)dt/
e~(t)dtl, pend on different speakers or different syllables.

1J0

340

10

Journal of Signal Processing, Vol. 13, No.4, July 2009

y-~-.~~:.'l\""_"'_"~~'_"_.. ' .... ~~ :: f;' -- ~:m~' ~~ , ) ....

l··ilf'···'· ....
0:,-". -"'--'~~-'"

0;

.5

. 10:

-'-·~· · · · ·-·. -.. '········J"../--:.,t,- .- ,-_ '-'~--. ~

,..,'

'\

.".,;

I

,.f

:::;;".0'~o~\.,..-~--~j-O.04~
0402 .'i"\CIl_ ... ,"

, __ • _ . . . . . . . .

'.

,"~ •• ,1Q)a.

~ ".06 H-meanII]..... IL ..

'0

c:
'iii

(!)

II,!

j.

Channel number (n)

t..

.

l .,.

~

0

a.

100

::;

Frequency (Hz)(~"~Et~~~lNImean ± SId I

50

0

~

;~

'0..0.04

J!i.. _'00...--.--------

....

I-

0

Channel number (n)Channel number (n)·~o--~--~--~~~--~~'m--~--~--~--~
Channel number (n)Figure 3: Results of analysis for dataset at point 2 in the sameformat as Fig. 2

Figure 5: Results of analysis for MTF obtained by fitting
model: Slope of regression line offitted model (top) and RMS
difference between model and MTF (bottom)

II

MTF without internal noise
a't'exp(-b't)
a'exp(-b't)." ..... lPF- - -MTF

I0.8

results of analysis using an exp( -bnt). The top panel indicates the slope of the regression line of the model that was
fitted by MTF derived from the database. If the value of theslope is ze.ro, .MTF does not influence BC speech. The bottom panel mdlcates the root mean squared (RMS) differences
between the model and the MTF. As the RMS difference between MTF and an exp( -bnt) was the smallest for all models, we assumed that the MTF relation, eh(t), could be represented by the model as eh(t) = an exp( -bnt). Then we candetermine inverse filtering as
'

0.7
)(

~ 0.6

..

.5:
c:

.g 0.51"0~I~ 0.4 •

0.3

Et: (z) = a~ {I - exp ( - 2;sn ) Z

0.2

1

0.1

' - - _ L____-'___._-'-----"-_

o

o

2

4

6

8

10

'

.•.....•;..•.

12
Modulation frequency (Hz)

14

,

L_

16

18

20

Figure 4: Comparison between MTF derived from ACIBC
speech database and three fitted models

3.4 Functional modeling of MTF
In our previous method [4], MTF was represented as an
exponential model. However, it was not evident whether
MTF could be represented with an exponential curve. We
confirmed that MTF has low-pass characteristics from the results of analysis. We modeled MTF by fitting three low-passfunctions, i.e., exponential curve eh(t) = ant exp( -bnt),
the model of MTF used in the previous method eh(t) =an exp( -bnt), and the eh(t) = IIR low-pass filter to the MTF
derived from the database.
Figure 4 plots the results of comparing the MTF derived
from the database and the three fitted functions. The shapeof the MTF seems to be rippled. Because MTF without internal noise did not fluctuate, the influence of internal noise
might rip~le the ~hape .of tv:tTF. The model, an exp( -bnt),
was especially sUItable m thIS comparison.
Figure 5 shows the mean and the standard deviation for the

Journal of Signal Processing, Vol. 13, No.4, July 2009

-1 }

(6)

where Is is the sampling frequency of 16 kHz.

4. Method of MTF-Based Blind Restoration
Our previous method [4] needed information on AC speech
to determine the parameters of the MTF model and to set the~onditions for rest?ration. We improved our previous method
m two respects usmg the results from analysis.
The first improvement was how we determined the two parameters of the model, an and bn , to restore BC speech. From
the results of analysis, parameter an can be approximated bya regression curve and this curve only depends on the measurement point. We can determine parameter an without AC
speech to study each measurement of the regression curve.
.To estimate parameter bn , we used the method proposed by
Huamatsu and Unoki [9], which was originally introduced toestim~t~ the r~verberation time based on the MTF concept.
We utlhzed thIS method because our research and theirs werebased on exactly the same model. bn can be estimated as

bn = argb~in (IEy(O) ·IM(Jdml bn)11 EYn (Jdm) I)(7)

where I d"!, is the dominant frequency of the BC power envelope and Ey (.) is the power envelope of restored speech.

341

The. results of evaluations demonstrated that the new method
could effectively and blindly restore BC speech.
6. Conclusion

Channel number (n)We analyzed the characteristics of AC and BC power envelopes with an ACIBC speech database. As a result, wefound that a characteristic ofMTF was the low-pass filteringand each measurement point of power ratio could be approximated by a regression curve. We then modeled the MTF as
an exp( -bnt) and proposed methods of determining the parameters for the model without AC speech. We consequently
proposed the MTF-based method of blind restoration. Finally, we evaluated the new approach and found that it could
effectively and blindly restore BC speech.
We next intend to carry out comprehensive evaluations andassess how well the proposed method performs in these.

Figure 6: Improved Correlation, SNR and RMS of MTF
Acknowledgments
Our previous method restored BC speech when the corre- This work was supported by a Grant-in-Aid by the YAZAKI
lation between AC and BC power envelopes was not over 0.8 Memorial Foundation for Science and Technology It wasand the relative power of AC power envelope was over - 20 also partially supported by the Strategic Information and
dB. The second improvement was that we changed these con- COmmunications R&D Promotion ProgrammE (SCOPE)
ditions so that the method restored BC speech when the rel- (071705001) of the Ministry of Internal Affairs and Commuative power of the BC power envelope was over -40 dB to nications (MIC), Japan.
restore BC speech without using AC speech. The reason weset such conditions is that internal noise in BC speech appears
References
when the relative power of the BC power envelope is not over
-40 dB. Because internal noise increased the power of the[1] S. Kitamori and M. Takizawa: An analysis of bone conducted
DC of MTF, the method could not estimate the true value ofspeech signal by articulation tests, IEICE Trans., Vol. J72-A,
No. 11, pp. 1764-1771, 1989.
parameter bn • We therefore propose an MTF-based method
for blindly restoring BC speech based on these results.
[2] S. Ishimitsu, H. Kitakaze, Y. Tsuchibushi H. Yanagawa and M.
5. Evaluations
We carried out simulations to evaluate the new method using a subset of the ACIBC speech database. The correlationand SNR of the power envelopes for AC and the restored
speech signals or the power envelopes for AC and BC speech
signals were used to evaluate the improved restoration in the
power envelopes. The perceptual evaluation of sound quality(PESQ) [10] that was recommended by ITU -T P. 862, and log
spectral distortion (LSD) were used to evaluate the improved
speech quality. The RMS of MTF was used to evaluate theimproved intelligibility of restored speech. The RMS ofMTF
means the RMS difference between MTF that includes attenuation and MTF that does not include attenuation. If all modulation frequencies ofMTF are 0 dB, MTF does not influence
BC speech. Therefore, the closer the value of the RMS of
MTF is to 0, the better the improvement in MTF.
Figure 6 shows an example of the SNR, correlation, and
RMS of MTF. The solid lines indicate the results for BC
speech and the dashed lines indicate the results for speech
restored with the proposed method using parameters derived
from the regression curve. These results demonstrate thatthe power envelopes can be adequately restored by the proposed approach. The LSD value decreased by 1.5 dB and thePESQ value increased by 0.7 points. These results revealed
that sound quality can be improved with the new approach.

342

Fukushima,: A noise-robust speech recognition system making use of body-conducted signals, Acoust. Sci. &,Tech., Vol.
25, No.2, pp. 166-169,2004.
[3] T. Tamiya and T. Shimamura: Reconstruction filter designfor bone-conducted speech, Proc. ICSLP2004, pp. 1085-1088,
2004.
[4] T. T. Vu, K. Kimura, M. Unoki and M. Akagi: A study on
restoration of bone-conducted speech with MTF-based andLP-based models, J. Signal Processing, Vol. 10, No.6, pp.
407-417, 2006.
[5] T. T. Vu, G. Seide, M. Unoki and M. Akagi: Method of LPbased blind restoration for improving intelligibility of boneconducted speech, Proc. Interspeech2007, pp. 966-969, 2007.
[6] T. Houtgast and H. 1. M. Steeneken: The Modulation Transfer
Function in Room Acoustics as a Predictor of Speech Intelligibility, Acustica, Vol. 28, pp. 66-73, 1973.
[7] M. Drullman: Temporal envelope and fine structure cues forspeech intelligibility, 1. Acoust. Soc. Am., Vol. 97, pp. 585592, 1995.
[8] Database for speech intelligibility testing using Japanese wordlists. NTT-AT, 2003.
[9] S. Hiramatsu and M. Unoki: A Study on the Blind Estimationof Reverberation Time in Room Acoustics, J. Signal Processing, Vol. 12, No.4, pp. 323-326. 2008.
'[10] H. Yi and C. L. Philipos, :Evaluation of objective measures forspeech enhancement, Interspeech2006, pp. 1447-1450, 2006.

Journal of Signal Processing, Vol. 13, No.4, July 2009