APSIPA ASC 2011 Xi’an

Computationally Efﬁcient Body-Conducted Voice
Conversion with Original Excitation Signals
Daisuke Deguchi, Tomoki Toda, Hironori Doi, Hiroshi Saruwatari, and Kiyohiro Shikano
Graduate School of Information Science, Nara Institute of Science and Technology, Nara, Japan
E-mail: tomoki@is.naist.jp Tel/Fax: +81-743-72-5261
Abstract—In this paper, we propose a computationally efﬁcientmethod of body-conducted voice conversion. A body-conductedvoice is robust against to external noise but its voice quality
is severely degraded by mechanisms of body-conduction. The
conventional body-conducted voice conversion method effectively
enhances the body-conducted voice by converting both spectral
and excitation features. On the other hand, its computational
cost is relatively high. To signiﬁcantly reduce the computational
cost while keeping the enhanced voice quality as high as possible,
we propose a conversion method of using an original excitation
signal of the body-conducted voice and computationally efﬁcientfeature extraction. The effectiveness of the proposed method is
conﬁrmed in the objective and subjective evaluations.

I. I NTRODUCTION
Towards noise-robust human-to-human speech communication, there have been several attempts to explore sensing
devices as alternatives to the air-conductive microphone. It
has been reported that speech signals detected by the boneconductive microphone can be effectively used to enhancespeech sounds under heavy noise conditions [1]. As one of
the sensing devices to detect body-conducted voices, we focus
on the nonaudible murmur (NAM) microphone [2]. It was
originally developed to detect extremely soft murmur called
NAM, which is so quiet that people around the speaker barely
hear its emitted sound. Placed on the neck below the ear, theNAM microphone can effectively detect air vibrations in the
vocal tract from the skin through only the soft tissues of thehead. High-quality body-conductive recording of various typesof speech, such as a very soft murmur as NAM and a normal
voice, is possible from this position because the conduction
through obstructions, such as bones whose acoustic impedanceis different from that of soft tissues, is avoided. It is also robust
against external noise owing to its noise-proof structure likein other body-conductive microphones. One serious drawbackof the NAM microphone is that severe degradation of speech
quality is caused by essential mechanisms of body conduction.
Therefore, its speech quality improvements are essential if it
is used in human-to-human speech communication.
To improve speech quality of the body-conducted voicedetected with the NAM microphone, body-conducted voice
conversion based on a statistical voice conversion technique[3], [4] has been proposed [5]. Joint probability density functions of the acoustic features of the body-conducted voice andthose of the normal voice are modeled by Gaussian mixture
models (GMMs). By using these GMMs, the acoustic featuresof the body-conducted voice are converted to those of thenormal voice in a probabilistic manner. In the conventional
conversion system, STRAIGHT [6] is used as a high-quality
analysis-synthesis method to extract the acoustic features as
accurately as possible and its mixed excitation model [7] isalso used to convert an excitation signal as well. They arevery effective for improving the converted voice quality but
the computational cost of STRAIGHT analysis is relatively
high. To develop a real-time conversion system, it is required
to reduce the computational cost as much as possible. This requirement becomes more essential if only the limited resources
are available to implement the system.
In this paper, we present a computationally efﬁcient bodyconducted voice conversion system. By assuming that the
acoustic differences in the excitation signal between the bodyconducted voice and normal voice less affect the converted
voice quality, the original excitation signal of the bodyconducted voice is used in synthesis without any modiﬁcations. Several methods are proposed to considerably reduce thecomputational cost while keeping the converted voice quality
as high as possible. To demonstrate the effectiveness of the
proposed system, both objective and subjective experimental
evaluations are conducted.
II. B ODY-C ONDUCTED VOICE C ONVERSION
There are several acoustic differences between the bodyconducted voice detected with the NAM microphone and thenormal voice detected with the air-conductive microphone.
High-frequency components of the body-conducted voice areseverely attenuated by lack of radiation characteristics fromlips and effect of low-pass characteristics of the soft tissues.
Consequently, some phonemes with large power at highfrequency bands, such as unvoiced fricatives, often lose theirown speciﬁc spectral structures. Waveform power differences
are also noticeable. It is hard to compensate for these acoustic
differences using simple modiﬁcations, such as global linear transformation. Moreover, aperiodic components, which
capture noisy strength on each frequency band of the excitation signal [7], are also quite different between the bodyconducted voice and the normal voice. These complicatedacoustic differences are well dealt with in statistical voice
conversion from the body-conducted voice into the normal
voice uttered by the same speaker. The conversion processof the conventional body-conducted voice conversion system[5] is shown in Figure 1.

Body-conducted voice

STRAIGHT analysis

STRAIGHT-based melcepstrum sequence
Combining
multiple frames
STRAIGHT-based
spectral segment
feature sequence

Aperiodicity
sequence

Conversion

Aperiodicity sequenceof normal voice

GMM for aperiodicity estimation
F0 sequence

Mixed excitation
generation

GMM for spectral
estimation

Conversion

Excitation signalof normal voice

Filtering
STRAIGHT-based
mel-cepstrum sequenceof normal voice

Normal voiceFig. 1. Conversion process of conventional body-conducted voice conversion
system (referred as system A). The excitation signal is modeled withSTRAIGHT mixed excitation and its aperiodic components are converted.

STRAIGHT analysis is employed to extract time-varying
speech parameters of the body-conducted voice. In this paper,
mel-cepstrum is used as a spectral feature and F0 and aperiodic
components are used as excitation features. To compensate forthe spectral structures often lost at several phonemes throughthe body-conductive recording, a spectral segment feature
vector is extracted at each frame and is used as the source
feature vector in the spectral conversion [5]. The spectral
segment feature vector X t at frame t is calculated as





X t = A c
+ b, (1)t−L , ct−L+1 , · · · , ct , · · · , ct+L


(x)
(x)
(x)
is a D-dimensional
where ct = ct (1), · · · , ct (D)mel-cepstrum vector of the body-conducted voice at framet. The transformation matrix A and the bias vector b are
determined with principal component analysis (PCA) to extract the lower dimensional spectral segment feature vector
by removing redundant components of 2L + 1 mel-cepstrum
vectors.
Two GMMs are used in the body-conducted voice conversion: a GMM for spectral conversion and a GMM foraperiodicity conversion. Using parallel data of the bodyconducted voice and normal voice as training data, which arerecorded simultaneously with the NAM microphone and an
air-conductive microphone, joint feature vectors of the source
and target features are developed. For the spectral conversion,
the spectral segment feature vector of the body-conductedfeaturevoice X t and a joint static and dynamic mel-cepstralvector of the target normal voice Y t =

(y) 

ct

(y) 



, Δct

are concatenatedframe by frame to develop the joint feature. Using the joint feature vectors, a joint
vector X t ,Y tprobability
densityfunctionof the source and target features
P X t , Y t |λ(X,Y ) is modeled by a GMM, where λ(X,Y ) isa parameter set of the GMM. For the aperiodicity conversion,
the joint feature vector is developed by concatenating staticand dynamic features of aperiodic components of the bodyconducted voice and those of the normal voice, and then its
joint probability density function is modeled by the other
GMM.
In conversion,
 given a time sequence of the source feature
vectors X = X 
1 , · · · , XT

, a time sequence of the target



(y)
(y)
static feature vectors y = c1 , · · · , cT
is determinedbymaximizing the conditional probability density function
P Y |X, λ(X,Y ) of a time sequence of the target static
and dynamic feature vectors Y = Y . This1 ,··· ,Y T
estimation is performed under a constraint between static anddynamic feature vectors given by Y = W y, where W is atransformation matrix from the static feature vector sequence
to the joint static and dynamic feature vector sequence. The
spectral conversion and the aperiodicity conversion are performed independently. The global variance is also consideredin the spectral conversion [4]. On the other hand, F0 values
with unvoiced/voiced (U/V) information extracted from the
body-conducted voice are not converted.
In synthesis, the converted excitation signal e(ŷ) (n) is
generated with STRAIGHT mixed excitation using the converted aperiodic components and the extracted F0 values with
U/V information. Then, the converted voice signal s(ŷ) (n) is
generated by ﬁltering the excitation signal with the converted
spectral ﬁlter h(ŷ) (n). This ﬁltering process is written as
S (ŷ) (z) = H (ŷ) (z)E (ŷ) (z),
(ŷ)

(ŷ)

(2)

(ŷ)

where S (z), E (z), and H (z) are Z-transforms ofs(ŷ) (t), e(ŷ) (t), and h(ŷ) (t), respectively. Using Mel Log
Spectral Approximation (MLSA) ﬁlter [8], H (ŷ) (z) is givenby
H (ŷ) (z) =

D

exp

(ŷ)

ct (d)z̃ −d ,

(3)

d=0
(ŷ)

where ct (d) is the dth coefﬁcient of the converted melcepstrum at frame t and z̃ −1 is the all-pass ﬁlter for frequency
warping.
III. B ODY-C ONDUCTED VOICE C ONVERSION WITHO RIGINAL E XCITATION S IGNALSSTRAIGHT analysis is capable of extracting highly accurate spectral parameters by removing periodic components ofthe excitation signal from spectral envelope using F0 information. Moreover, STRAIGHT mixed excitation enables toconvert the excitation signal. On the other hand, STRAIGHT
analysis is computationally expensive and STRAIGHT mixed
excitation sometimes suffers from errors of the F0 extraction
and U/V estimation. To address these issues, a computationally efﬁcient conversion method using the original excitation
signals is proposed.
A. Use of Original Excitation Signal
The original excitation signal of the body-conducted voice
is well approximated with a residual signal r(x) (n) if it
is extracted by inversely ﬁltering the body-conducted voice
signal s(x) (n) with the sufﬁciently accurate spectral ﬁlter
h(x) (n), such as given by mel-cepstrum extracted from the
body-conducted voice with STRAIGHT. The inverse ﬁltering
process is given by1(4)
S (x) (z),R(x) (z) =
(x)
H (z)

Body-conducted voice

Body-conducted voice

STRAIGHT analysis

STRAIGHT-based melcepstrum sequence
Combining
multiple frames

FFT analysis

Equivalent to filtering
excitation signal ofbody-conducted voice

GMM for spectral
estimation

STRAIGHT-based
spectral segment
feature sequence

Conversion+

STRAIGHT-based
difference melcepstrum sequence

FFT-based melcepstrum sequence

Normal voiceFig. 2. Conversion process of body-conducted voice conversion system(referred as system B) using original excitation signals.

where R(x) (z), S (x) (z), and H (x) (z) are Z-transforms ofr(x) (n), s(x) (n), and h(x) (n), respectively. In the proposed
conversion method, the residual signal is used as the excitation
signal in synthesizing the converted voice as follows:
S (ŷ) (z) = H (ŷ) (z)R(x) (z) =H (ŷ) (z) (x)S (z).
H (x) (z)(5)

This is equivalent to ﬁltering the body-conducted voice signal
with the difference spectral ﬁlter given byD

H (ŷ) (z)=H (x) (z)

exp
d=0
D

exp

(ŷ)

ct (d)z̃ −d

D

= exp
(x)

ct (d)z̃ −dd=0(ŷ−x)(ŷ)(x)ct(d) = ct (d) − ct (d).

(ŷ−x)

ct-

Combining
multiple frames

Filtering

STRAIGHT-based
mel-cepstrum sequenceof normal voice

Equivalent to filtering
excitation signal ofbody-conducted voice(d)z̃ −d , (6)GMM for spectral
estimation

FFT-based
spectral segment
feature sequence

Conversion+

FFT/STRAIGHTbased differencemel-cepstrum sequence

STRAIGHT-based
mel-cepstrum sequenceof normal voice

Filtering

Normal voiceFig. 3. Conversion process of computationally efﬁcient body-conducted voice
conversion system (referred as system C) using original excitation signals.
Body-conducted voice

FFT analysis

Equivalent to filtering
excitation signal ofbody-conducted voice

FFT-based melcepstrum sequence
Combining
multiple frames
FFT-based
spectral segment
feature sequence

GMM for spectral
estimation

Conversion

Filtering

STRAIGHT-based
difference melcepstrum sequence

Normal voiceFig. 4. Conversion process of computationally efﬁcient body-conducted voice
conversion system (referred as system D) using original excitation signals anddirect estimation of difference mel-cepstrum sequence.
TABLE IC OMPARISON OF BODY- CONDUCTED VOICE CONVERSION SYSTEMSSHOWN IN Figures 1, 2, 3, AND 4

d=0

(7)

This conversion process is shown in Figure 2. Although theexcitation signal is not converted, this process is free fromerrors of the F0 extraction and U/V estimation.

System indexA(conventional)
B
C
D

Excitation
w/ conversion
of aperiodicity
w/o conversion
w/o conversion
w/o conversion

Analysis
STRAIGHT
STRAIGHT
FFT
FFTConversion target
Target mel-cepstrum
and aperiodicity
Target mel-cepstrum
Target mel-cepstrum
Difference mel-cepstrum

B. Reducing Computational Cost
In the statistical voice conversion, it is important to use ahighly accurate spectral parameter for the target feature since it
directly affects the converted voice quality. On the other hand,the spectral parameter for the source feature does not directly
affect the converted voice quality as far as it is still useful as an
explanatory variable to estimate the target features. Therefore,
simple FFT-based spectral analysis is used to quickly extract
the source mel-cepstrum. This conversion process is shown in
Figure 3. Note that a GMM for the spectral conversion needs
to be trained with the joint feature vectors consisting of thespectral segment feature vectors developed from the FFT-based
mel-cepstra of the body-conducted voice and the STRAIGHTbased mel-cepstrum feature vectors of the normal voice. This
conversion process is very computationally efﬁcient since
STRAIGHT analysis is avoided.
One drawback of this conversion process is that the extraction accuracy of the source mel-cepstrum still affects theﬁltering process even if it does not affect the conversion accuracy of the target mel-cepstrum. Since the FFT-based spectral
analysis is sensitive to the periodicity of excitation signal,
the source mel-cepstrum often captures harmonic componentsof the periodic excitation signal. This causes the degradationof converted voice quality because the difference spectral
ﬁlter given by Eq. (6) is contaminated by the periodicity of
excitation signal.
To address this issue, the conversion to a differencemel-cepstrum is proposed. In training, the difference mel(y−x)(y)(x)= ct − ct is calculated by subtracting
cepstrum ct
the STRAIGHT-based mel-cepstrum of the body-conductedvoice from the STRAIGHT-based mel-cepstrum of the normal
voice. Thus, the difference mel-cepstrum does not capture
the periodic components of the excitation signal thanks to
STRAIGHT analysis. A GMM is trained with the joint feature
vectors consisting of the spectral segment feature vectors basedon the FFT-based mel-cepstra of the body-conducted voice andthe difference mel-cepstral feature vectors. This conversion
process is shown in Figure 4.
Table I shows a comparison of the body-conducted voice
conversion systems depicted in Figures 1, 2, 3, and 4. The
systems B, C, and D use the original excitation signals. The
computational costs of the systems C and D are much lower
than those of the systems A and B. The system D estimates
the difference mel-cepstrum to keep the spectral ﬁlter from
capturing the periodicity of excitation signal.

B. Experimental Results
Figure 5 shows the result of objective evaluation. The
system B yields lower mel-cepstral distortion than the system
A. This is because the conversion accuracy of the system A
suffers from the errors of F0 extraction and U/V estimation.
The system C causes very large mel-cepstral distortion since
the converted voice signal is directly affected by poor accuracy
in mel-cepstrum extraction of the body-conducted voice. Thisissue is effectively addressed in the system D, the conversion
accuracy of which is equivalent to that of the system B.
The result of the opinion test is also shown in Figure 5.
The systems B, C, and D using the original excitation signals
yield the better converted voice quality than the system A. This
result implies that the converted voice quality is sensitive to
the F0 extraction error or the U/V estimation error rather thana mismatch of aperiodic components. The system C causes
the signiﬁcant degradation in the converted voice quality but
this degradation is effectively alleviated by the system D.
Consequently, the system D yields the best converted voice
quality equivalent to that of the system B while keeping its
computational cost much lower than that of the system B.
These results show the effectiveness of the system D. It
could depend on individual speakers whether the system D
outperforms the system A. We plan to evaluate the performance of these systems for various speakers.

㪤㪼㪸㫅㩷㫆㫇㫀㫅㫀㫆㫅㩷㫊㪺㫆㫉㪼㩷㩿㪤㪦㪪㪀㩷
Mean opinion score (MOS)

Mel-cepstral distortion [dB]

IV. E XPERIMENTAL E VALUATION
A. Experimental Conditions
The body-conducted voice and normal voice were simultaneously recorded by detecting a natural voice uttered by aJapanese male speaker with the NAM microphone and an airconductive microphone. The sampling frequency was set to
8 kHz. A sentence set consisting of 100 phonetically balanced
sentences was recorded with each of three different types of
the NAM microphone. In each set, 50 sentences were used
for training and the remaining 50 sentences were used forevaluation.
The 0th through 16th mel-cepstral coefﬁcients were used asa spectral feature. Aperiodic components on ﬁve frequency
bands (i.e., 0-1, 1-2, 2-4, 4-6, and 6-8 kHz) were used as an
excitation feature. The shift length was set to 5 ms. In theextraction of the spectral segment feature, a 34-dimensional
vector was extracted at each frame from concatenated melcepstrum vectors at current and ± 4 preceding/succeeding
frames. The number of mixture components of each GMM
was set to 32.
Objective and subjective evaluations were conducted tocompare the conversion performance of the systems A, B,
C, and D. In the objective evaluation, mel-cepstral distortion
between the converted voice signal and the target voice signal
was used as an evaluation metric. In the subjective evaluation,
the opinion test on voice quality was conducted using a 5-point
opinion scale, such as 1: very bad, 2: bad, 3: fair, 4: good,and 5: excellent. Eight listeners participated in the test. Eachlistener evaluated 120 samples consisting of 30 samples foreach system, which were randomly selected for each listener.

4.4
4.2
4.0
3.8
3.6
3.4

A

B

C

D

5

4

3

2

1

Conversion systems

A

B

C

D

Conversion systems

Fig. 5. Mel-cepstral distortion (left) and mean opinion score (right) for eachbody-conducted voice conversion system shown in Figures 1, 2, 3, and 4.

V. C ONCLUSIONSIn this paper, we proposed computationally efﬁcient bodyconducted voice conversion based on FFT-based spectral analysis, conversion from the source spectral feature to a difference
spectral feature between the body-conducted voice and thenormal voice, and the use of a residual signal of the bodyconducted voice as an excitation signal in synthesis. The
experimental results showed that the proposed conversion
yields better converted voice quality than the conventional
conversion since the proposed conversion is free from theerrors of F0 extraction and unvoiced/voiced estimation. Further
investigation for various speakers’ voices will be performed.
ACKNOWLEDGMENT
This work was supported in part by MEXT Grant-in-Aid forYoung Scientists (A). The authors are grateful to Prof. Hideki
Kawahara of Wakayama University, in Japan for permission
to use the STRAIGHT analysis-synthesis method.
R EFERENCES[1] A. Subramanya, Z. Zhang, Z. Liu, and A. Acero. Multisensory processing
for speech enhancement and magnitude-normalized spectra for speechmodeling. Speech Communication, Vol. 50, No. 3, pp. 228–243, 2008.
[2] Y. Nakajima, H. Kashioka, N. Cambell, and K. Shikano. Non-Audible
Murmur (NAM) recognition. IEICE Trans. Information and Systems, Vol.
E89-D, No. 1, pp. 1–8, 2006.
[3] Y. Stylianou, O. Cappé, and E. Moulines. Continuous probabilistic transform for voice conversion. IEEE Trans. Speech and Audio Processing,Vol. 6, No. 2, pp. 131–142, 1998.
[4] T. Toda, A.W. Black, and K. Tokuda. Voice conversion based on
maximum likelihood estimation of spectral parameter trajectory. IEEE
Trans. Audio, Speech and Language Processing, Vol. 15, No. 8, pp. 2222–
2235, 2007.
[5] T. Toda, K. Nakamura, H. Sekimoto, and K. Shikano. Voice conversion
for various types of body transmitted speech. Proc. ICASSP, pp. 3601–
3604, Taipei, Taiwan, Apr. 2009.
[6] H. Kawahara, I. Masuda-Katsuse, and A. de Cheveigné. Restructuring
speech representations using a pitch-adaptive time-frequency smoothing
and an instantaneous-frequency-based F0 extraction: possible role of arepetitive structure in sounds. Speech Communication, Vol. 27, No. 3–4,
pp. 187–207, 1999.
[7] H. Kawahara, J. Estill, and O. Fujimura. Aperiodicity extraction andcontrol using mixed mode excitation and group delay manipulation
for a high quality speech analysis, modiﬁcation and synthesis system
STRAIGHT. Proc. MAVEBA, Firenze, Italy, Sep. 2001.
[8] T. Fukada, K. Tokuda, T. Kobayashi, and S. Imai, An adaptive algorithm
for mel-cepstral analysis of speech. Proc. ICASSP, Vol. 1, pp. 137–140,
San Francisco, USA, Mar. 1992.