Available online at www.sciencedirect.com

Speech Communication 54 (2012) 134–146
www.elsevier.com/locate/specom

Speaking-aid systems using GMM-based voice conversion
for electrolaryngeal speech
Keigo Nakamura ⇑, Tomoki Toda, Hiroshi Saruwatari, Kiyohiro Shikano
Graduate School of Information Science, Nara Institute of Science and Technology, 8916-5 Takayama-cho, Ikoma-shi, Nara 630-0192, Japan
Received 23 February 2011; received in revised form 2 July 2011; accepted 14 July 2011
Available online 26 July 2011

Abstract
An electrolarynx (EL) is a medical device that generates sound source signals to provide laryngectomees with a voice. In this article we
focus on two problems of speech produced with an EL (EL speech). One problem is that EL speech is extremely unnatural and the otheris that sound source signals with high energy are generated by an EL, and therefore, the signals often annoy surrounding people. To
address these two problems, in this article we propose three speaking-aid systems that enhance three diﬀerent types of EL speech signals:
EL speech, EL speech using an air-pressure sensor (EL-air speech), and silent EL speech. The air-pressure sensor enables a laryngectomee
to manipulate the F0 contours of EL speech using exhaled air that ﬂows from the tracheostoma. Silent EL speech is produced with a newsound source unit that generates signals with extremely low energy. Our speaking-aid systems address the poor quality of EL speechusing voice conversion (VC), which transforms acoustic features so that it appears as if the speech is uttered by another person. Our
systems estimate spectral parameters, F0 and aperiodic components independently. The result of experimental evaluations demonstrates
that the use of an air-pressure sensor dramatically improves F0 estimation accuracy. Moreover, it is revealed that the converted speech
signals are preferred to source EL speech.
Ó 2011 Elsevier B.V. All rights reserved.
Keywords: Electrolaryngeal speech; Voice conversion; Speaking-aid system; Speech enhancement; Airpressure sensor; Silence excitation;
Non-audible murmur; Laryngectomee

1. IntroductionMore than 12000 people in the United States are estimated to receive the diagnosis of laryngeal cancer in 2008
(Jemal et al., 2008). There are some treatments to cure laryngeal cancer such as chemotherapy, radiation therapy,
partial laryngectomy (Forastiere et al., 2003; Laccourreye
et al., 1996) or total laryngectomy. Total laryngectomy,
which removed the whole larynx including the vocal folds,
was the standard treatment for locally advanced disease
until the early 1990s (Forastiere et al., 2003). It means that
there are many total laryngectomees that have been under⇑ Corresponding author. Tel.: +81 743 72 5287; fax: +81 743 72 5289.

E-mail address: kei_go@nifty.com (K. Nakamura).
0167-6393/$ - see front matter Ó 2011 Elsevier B.V. All rights reserved.
doi:10.1016/j.specom.2011.07.007

gone the total laryngectomy.1 For example, it is said that
there are 12000 laryngectomees in Japan (Ifukube, 2003).
After laryngectomy, laryngectomees cannot speak using
the vibration of their vocal folds as shown in Fig. 1. By
undergoing a laryngectomy, the quality of life of laryngectomees suﬀers because of the loss of their original voices(Carr et al., 2000). Therefore, voice rehabilitation for laryngectomees is an important research topic.
There are three major types of alaryngeal speech called
esophageal speech, tracheo-esophageal (T-E) shunt speech,
and artiﬁcial laryngeal speech such as electrolaryngeal
speech (EL speech) (Singer and Blom, 1980; Hashiba
1
The words of laryngectomee and laryngectomy respectively denotetotal laryngectomee and total laryngectomy in this article.

K. Nakamura et al. / Speech Communication 54 (2012) 134–146

Flow of air
Flow of sound source signals

Nasal cavity
Oral cavity

Vocal folds
Trachea

Esophagus Tracheostoma

Expired air
Laryngeal speaker

Expired air
Total Laryngectomees

Fig. 1. Anatomical image of laryngal speakers and total laryngectomees.

et al., 2001; Hayes, 1951; Williams and Watson, 1985;
Hočevar-Boltežar and Žargi, 2001). The main diﬀerence
among these alaryngeal speech signals is how the speakerobtains an alternative sound source vibration to vocal foldvibration. There are some types of artiﬁcial larynxes suchas an electrolarynx(EL) or a pneumatic artiﬁcial larynx(also known as a whistle-type artiﬁcial larynx). An EL is
one of the most commonly used alaryngeal speech (Williams and Watson, 1985; Carr et al., 2000), and therefore,
we focus on EL speech.
When a speaker produces EL speech, an EL is pressedagainst the speaker’s lower jaw, and the signals are transmitted to the speaker’s oral cavity through the speaker’s
skin. Finally, the speaker produces EL speech by articulating with the sound source signals generated by the EL. Oneof the advantages of using an EL is that speakers easily
learn how to produce EL speech. Another advantage isthat EL speech is possible for people with less physical ﬁtness such as elderly laryngectomees. Two problems with
EL speech that are focused on in this article are (1) the
unnaturalness of EL speech and (2) the radiation of noisy
sound source signals. One of the main defects of EL speech
is its unchangeable F0 pattern. Because of this unchangeable F0 pattern, EL speech sounds obviously artiﬁcial andunnatural even when speakers are proﬁcient in using an
EL. Another defect of EL speech is the high energy ofvibrations generated by the EL itself. The sound source signals are often radiated from the location of attachment of
the EL, and the radiated noise often annoys surrounding
people; especially in quiet environments such as inside alibrary. Moreover, the speaker might also be concernedthat he or she will annoy people nearby because of the radiated sound source signals. Therefore, the enhancement of
EL speech quality is an important issue in ensuring thesmoothinterpersonal
speechcommunicationoflaryngectomees.
Many researchers and developers have attempted toaddress problems of EL speech (Uemi et al., 1994;
secom.co., 2011; Murakami et al., 2004; Saikachi et al.,2009; Liu et al., 2006; Takahashi et al., 2001; Goldstein
et al., 2004). Uemi et al. developed an EL with an airpressure sensor (Uemi et al., 1994), which enables

135

laryngectomees to control the F0 pattern using the exhaled
air that ﬂows from the tracheostoma that is a hole locatedon the speaker’s neck for breathing. Murakami et al.
enhanced EL speech by employing a feature transformation technique (Murakami et al., 2004). In their approach,numerous conversion rules are developed from training
data and then applied to test data. Saikachi et al. proposeda method to control F0 of EL speech based on root-meansquare amplitude (Saikachi et al., 2009). Their concept ofobtaining reasonable F0 contours to make EL speech naturally sound is same as that of this article. Liu et al. proposed a method to reduce the radiation noises based on
spectral subtraction considering auditory masking (Liuet al., 2006). Novel artiﬁcial larynxes were also proposed.
For example, Takahashi et al. developed an intra-oral
electrolarynx for people who could not acquire common
alaryngeal speech or for early post-surgery speech rehabilitation (Takahashi et al., 2001). Goldstein et al. also
developed hands-free EL triggered by neck muscle electromyographic activity (Goldstein et al., 2004). Developmentof hands-free EL is another very important mission forimproving the quality of life of laryngectomees. However,developing new devices is out of the scope of this article.
Despite these studies for enhancing EL speech, some problems still remain. In the study of Murakami et al. (2004),
input utterances cannot be converted when no conversion
rules are found. A method proposed by Saikachi et al.
(2009) requires pre-laryngectomy speech. Moreover, notonly amplitude but also spectral information might beneeded to estimate reasonable F0 contours. The approachof Liu et al. (2006) might remove radiation noises; however, the problem of radiation noises is not the only problem in EL speech. There are huge gaps between EL speechand normal speech, and these gaps have not been ﬁlled bythese previous studies.
We proposed a speaking-aid system that enhanced EL
speech using a statistical voice conversion (VC) techniquewith Gaussian mixture models (GMMs) to address theunnaturalness of EL speech and the problem of radiationnoises (Nakamura et al., 2007). Our speaking-aid system
consists of four parts that (1) generate sound source signals, (2) record the produced EL speech, (3) convert therecorded EL speech, and (4) present the converted speech.
We also introduced a new sound source device that generates signals with extremely low energy. Since the soundsource signals of EL speech is given from outside, the voice
quality of EL speech is mainly deﬁned by the given source
signals. Existing ELs have to generate sound source signals
with high energy to make the volume of EL speech close tothat of normal speech, although the energy of the radiatednoises is also getting higher when the volume of soundsource signals is raised. Our aid system proposed in ourprevious work, on the other hand, includes a VC procedure, and therefore, using an existing EL is not essential.
We have had an idea to suppress the energy of radiatednoises by using a new sound source unit that generates signals with extremely low energy. The produced EL speech

136

K. Nakamura et al. / Speech Communication 54 (2012) 134–146

with the silent signals (silent EL speech) is also very low
energy. It is diﬃcult to make an absolute deﬁnition whatsilent EL speech is. In this article, we regard EL speech
produced with the new sound source unit generating extremely low energy as silent EL speech. Although a usual airconductive microphone is diﬃcult to detect the silent EL
speech owing to its sensitivity to external noises, a special
body-conductive microphone is capable of detecting it withhigh quality. As a result of experimental evaluations, theeﬀectiveness of this system was demonstrated by using
silent EL speech produced by a laryngeal speaker (Nakamura et al., 2007). However, two important problems havestill remained; (1) the system has been evaluated using
silent EL speech produced by not a laryngectomee but alaryngeal speaker and (2) input speech data have been converted into not normal speech but whispered voice that
does not have F0 information.
In this article we address two problems of EL speech;
one is the unnaturalness of EL speech, and the otherone is noisy sound source signals radiated from the location of the EL attachment. For addressing the problem
of unnaturalness, we propose a speaking-aid system thatconverts EL speech into normal speech using a VC technique. This system presents converted EL speech that
includes F0 information estimated from the EL speech.
Moreover, this system removes noisy sound source signals
using the VC technique. For more improving the naturalness, we propose another speaking-aid system that accepts
EL-air speech. In this system, VC eﬀectively uses F0 information of EL-air speech. We also propose the otherspeaking-aid system that accepts silent EL speech. Thissystem addresses the problem of noisy sound source signals; however, the speech quality of silent EL speech isstill same as that of EL speech. Therefore, VC should beapplied to silent EL speech as well. The shared part
between these three aid systems and the aid systemdescribed in our previous article (Nakamura et al., 2007)is the four components constructing each system. On theother hand, the following points are updated from ourprevious work; (1) two more aid systems are proposedto accept EL speech and EL-air speech, (2) source speech
is converted into not whispered voice but normal speech,and (3) three aid systems are all experimentally evaluatedin the same experimental conditions, in which source
speech signals are produced by not a laryngeal speaker
but a laryngectomee. Moreover, there are three featuresof our aid systems compared to other aid systems proposed in previous studies described above; (1) each system
can convert an arbitrary sentence, (2) each system uses
statistics of normal speech to estimate F0 contours, and(3) each system can address two problems focused on inthis article simultaneously.
This article is organized as follows. In Section 2, the key
techniques used in this article are explained. Our proposedsystems are described in Section 3 and experimentally evaluated in Section 4 and discussed in Section 5. Finally, we
conclude this article in Section 6.

2. Key techniques
2.1. VC based on maximum likelihood estimation (Kain andMacon, 1998; Toda et al., 2007)
Not only the primary role of speech to convey linguistic
information but also secondary information conveyed byspeech such as speaker individuality play an important rolein interpersonal speech communication. VC generally modiﬁes speech signals of a given source speaker while maintaining the linguistic information so that it appears as if
another speaker (the target speaker) utters the speech.
VC is useful for many applications such as voice responses,
text reader systems, and so forth. It is often convenient tospecify the desired modiﬁcation of source acoustic characteristics with reference to an existing target acoustic
characteristic.
A statistical VC method using a GMM-based maximum
likelihood estimation is used in applicative studies such as
Toda and Shikano (2005). This VC technique consists oftraining and conversion parts. In the training part, a parallel data set, which consists of sentence pairs of the source
and the target speakers, is used to train a GMM. Namely,
these two speakers need to utter the same sentence set todevelop training data. In each sentence pair, time alignment is automatically performed by a dynamic time warping (DTW) procedure to build joint vectors of the source
and the target features. Next, the joint probability density
consisting of the source and the target data is modeledby a GMM (Kain and Macon, 1998). In the conversion
part, the conditional probability density function of thetarget data given the source data is used to generate theconverted target data (Toda et al., 2007). After the
GMM training, any sentences can be converted with thetrained GMM, even if it is not included in the training
data. A main problem in maximum likelihood estimationis that the estimated parameters tend to be oversmoothed.
Oversmoothing of acoustic parameters tends to signiﬁcantly reduce their variations, and it causes noticeable degradation in naturalness of synthetic speech quality (Toda
et al., 2007). To address the oversmoothing problem, Toda
et al. proposed the consideration of the global variance(GV) of acoustic features over a time sequence (Toda
et al., 2007).
2.1.1. Training procedure to describe joint probability density
function (Kain and Macon, 1998)
Let xt = [xt(1), . . . , xt(dx)]> and yt = [yt(1), . . . , yt(dy)]> bestatic source and target feature vectors at frame t where dx
and dy denote the dimensions of xt and yt, respectively, andT denotes transposition.
The
joint probability density of> >these vectors zt ¼ x>;
yis
described by a GMM asttfollows:
P ðzt jkðzÞ Þ ¼

M
X
m¼1



ðzÞ
xm N zt ; lðzÞm ; Rm ;ð1Þ

K. Nakamura et al. / Speech Communication 54 (2012) 134–146

137

where m is the index of mixture components, M is the number of mixture components, xm is the weight of the mth
mixture component, and N ð; l; RÞ represents a Gaussian
distribution including a mean vector l and a covariancematrix R. k(z) is a model parameter set including weights,
mean vectors, and covariance matrices. The mth mean vector and the covariance matrix are written as
"
#
"
#
lðxÞ
RðxxÞ
RðxyÞ
m
ðzÞ
ðzÞ
m
m
lm ¼
; Rm ¼;
ð2ÞlðyÞRmðyxÞ RðyyÞ
mm

To eﬃciently perform the conversion process, weemploy the approximation method proposed in (Toda etal., 2007). Originally, all mixture component sequences
should be considered to calculate P(Yj X, k(Z)) such asX
P ðmjX; Y; kðZÞ ÞP ðYjX; m; kðZÞ Þ;
ð6Þ
P ðYjX; kðZÞ Þ ¼

ðyÞ
where lðxÞm and lm represent the mean vectors of the mth
mixture component for the source and target features.
RðxxÞand RðyyÞ
represent the covariance matrices of themm
mth mixture component for the source and target features,and RmðyxÞ represent the cross-covariance matrices
and RðxyÞ
mof the mth mixture component for the source and target
features, respectively. The model parameters are estimatedby the expectation-maximization (EM) algorithm(Dempster et al., 1977).
The probability density of the GV of the target staticfeature vectors over a time sequence is modeled by aGaussian distribution as^^ kðZÞ Þ;
Y; kðZÞ ÞP ðYjX; m;
P ðYjX; kðZÞ Þ  P ðmjX;

ð7Þ

ðZÞ

ð8Þ

P ðvðyÞjkðvÞ Þ ¼ N ðvðyÞ; lðvÞ ; RðvvÞ Þ;

ð3Þ
>

where the GV v(y) = [v(1), . . . , v(d), . . . , v(D)] is calculated
as follows:
!2
T
T
1 X
1 X
vðdÞ ¼y t ðdÞ y ðdÞ :
ð4Þ
T t¼1
T t¼1 t
The parameter set k(v) consists of the mean vector l(v) andthe diagonal covariance matrix R(vv). The GV is calculated
utterance by utterance in this article.
2.1.2. Conversion procedure considering dynamic features(Toda et al., 2007) 


> >
> >
Let X t ¼ x>
and Y t ¼ y>
be the sourcet ; Dxtt ; Dyt
and target joint feature vectors of the static and dynamic
feature vectors for frame t, respectively. Then, time
sequences of the source and target feature vectors are writ


>
> >
ten as X ¼ X >and Y ¼ Y >
1 ; . . . ; Xt ; . . . ; XT
1 ;...
> >
;
.
.
.
;
Y

,
respectively.
The
converted
staticfeature
;Y >
t
T
 >
>
^¼ y
^1 ; . . . ; y^>^>sequence y
is determined by maxit ;...;yT
mizing the product of the conditional probability density
of Y given X and the probability density of the GV as
follows:

all m

where m = {m1, . . . , mt, . . . , mT} is a mixture component sequence. Eq. (6) is eﬃciently approximated by suboptimum^ as follows:
mixture component sequence m^ ¼ arg max P ðmjX; k Þ:mUsing this suboptimum mixture component sequence, thetarget static feature sequence ^y is estimated as^y ¼ arg max P ðWyjX; m;^ kðZÞ Þx P ðvðyÞjkðvÞ Þ:
2.2. Air-pressure sensor for an EL (Uemi et al., 1994)
The air-pressure sensor shown in Fig. 2 enables laryngectomees to manipulate the number of vibrations of an
EL using exhaled air from the tracheostoma. This air-pressure sensor is connected to an existing EL with the name‘yourtone’ (Hashiba et al., 2001). EL-air speech is produced by (1) drawing air into the lungs, (2) covering thetracheostoma with the air-pressure sensor, (3) expelling this
air to drive the vibrator, and (4) articulating the soundsource signals while holding the EL and the air-pressure
sensor with both hands. Note that the location of attachment of the EL and the method of articulation are the sameas those in the production of EL speech. Because this airpressure sensor is a closed-type that does not allow the passage of air when the air-pressure sensor covers the tracheostoma, the speaker needs to repeat the above process
after every pause in speech. The circuit to convert the airpressure to the number of vibrations is built into the main
body of the EL. A threshold that deﬁnes the lower limit of
the air pressure required to turn on the vibrations is set inthis EL. If this threshold is too low, EL generates
Headset microphone
Electrolarynxx^
y ¼ arg max P ðYjX; kðZÞ Þ P ðvðyÞjkðvÞ Þ subject to Y ¼ Wy;

ð5Þ
where W extends the static feature vector to a joint feature
vector consisting of the static and dynamic feature vectors.
The constant x denotes the weight factor used to control
the balance between these two likelihoods. In this article
we set x as the ratio between the numbers of dimensionsof v(y) and Y, namely, 2T1 .

ð9ÞAir-pressure
sensorFig. 2. Photograph of man using EL with air-pressure sensor.

K. Nakamura et al. / Speech Communication 54 (2012) 134–146

Frequency [Hz]EL
speechAmplitude

30k

0

-30k

2.5

5.0
7.5
Time [sec]Amplitude

0

-30k

2.5

5.0
7.5
Time [sec]

10.0

0

-30k

5.0
7.5
Time [sec]

10.0

2.5

5.0
7.5
Time [sec]

10.0

2.5

5.0
7.5
Time [sec]

10.0

200
100
50

Frequency [Hz]Amplitude

30k
Normalspeech

2.5

400

30k
EL-airspeech

400
200
100
50

10.0

Frequency [Hz]

138

2.5

5.0
7.5
Time [sec]

10.0

400
200
100
50

Fig. 3. Examples of waveforms and F0 contours of EL speech, EL-air speech, and normal speech.

vibrations all the time. On the other hand, if it is too high,the speaker needs very strong air pressure to generate thevibrations. Therefore, a suitable threshold should be
adjusted for each speaker in advance.
Fig. 3 shows examples of waveforms and F0 contours of
EL speech, EL-air speech, and normal speech. The EL andEL-air speech signals are produced by the same laryngectomee, and the normal speech is produced by a laryngeal
speaker. As shown in the ﬁgure, rich F0 contours areobserved in EL-air speech, resulting in it sounding morenatural than EL speech. However, EL-air speech is still lessnatural than normal speech. Moreover, it is diﬃcult for a
speaker to intentionally switch between unvoiced andvoiced (U/V) sounds when producing EL-air speech, sincean EL always vibrates when the speaker produces EL-air
speech.
2.3. Non-audible murmur (NAM) microphone (Nakajima
et al., 2006)
Nakajima et al. deﬁned a special speech that consists ofarticulated respiratory sound without vocal-fold vibrationtransmitted through the soft tissues of the head as nonaudible murmur (NAM) (Nakajima et al., 2006). Since
speech signals of NAM have extremely low energy, it is
too diﬃcult to capture those signals with normal air-conductive microphones such as a head-set microphone.
Therefore, it is required to capture speech signals of
NAM using a special microphone called a NAM microphone (Nakajima et al., 2005). NAM is expected to be used
as a novel technique in silent speech interfaces, andNAM
microphone
Sternocleidomastoid

Muscle(Sternocleidomastoid)

Skin
Soft silicone
Electretcondenser microphone

Air vibrationin the vocal tract

Electrodes
External noiseproofing

Oral cavity

Fig. 4. Location of attachment and structure of NAM microphone.

applicative studies on NAM and NAM microphones havebeen conducted (Toda and Shikano, 2005; Nakajima et al.,2006; Nakagiri et al., 2006; Miyamoto et al., 2009; Toda
et al., 2009).
A NAM microphone is attached to the body on the sternocleidomastoid behind the neck as shown in Fig. 4. The

K. Nakamura et al. / Speech Communication 54 (2012) 134–146

139

bone-conductive receiver, and therefore, it can generate
arbitrary signals.

Amplitude

32553

3. Proposed speaking-aid systems using GMM-based VC

Frequency[Hz]-327687

3.1. Speaking-aid system for EL speech

5
3
1
0

1

2
Time [sec]3

Fig. 5. Example of NAM signals.

waveform in Fig. 5 shows that the dynamic ranges of vowels and consonants are extremely diﬀerent and that plosive
consonants such as /k/ can be impulsively observed. The
dynamic range of NAM microphones is larger than thatof normal air-conductive microphones; this is one advantage of NAM microphones, since they can detect not only
speech signals of NAM but also those of normal speech. In
the spectrogram in Fig. 5, spectral components above
4 kHz almost disappear. It is said that this phenomenon
is caused by mainly two reasons; one reason is that theradiation features of the oral cavity are lost and the otherone is that NAM signals are low-pass ﬁltered when they
pass through the muscle (Nakajima et al., 2006). On theother hand, the ﬁrst and second formants, which areimportant in speech processing, are clearly observed.
2.4. Sound source unit generating signals with extremely low
energy (Hosoi and Sakaguchi, 2003)
Hosoi and Sakaguchi proposed a novel unit as a newsound source unit for silent speech communication (Hosoi
and Sakaguchi, 2003). This unit, which is available from
Toshin Co. Ltd. (http, xxxx) with the model number
BR-41, generates signals with extremely low energy, whichcannot be captured by people around the speaker. A photograph of this unit and the vibrator without the cover are
shown in Fig. 6. This unit was originally developed as a

3.1.1. Framework of the system
To address the problem of the unnaturalness of EL
speech, in this article we propose a speaking-aid systemfor enhancing EL speech. An overview of this system isshown in Fig. 7. This system consists of four parts that(1) generate sound source signals, (2) record the EL speech,(3) convert the EL speech, and (4) present the converted
normal speech. The location where the EL is attached isthe same as that where a laryngectomee usually sets the EL.
When a laryngectomee speaks with this system, listeners
would hear not only converted speech but also the source
EL speech, making the use of our proposed system limited.
In other words, it is not suitable for laryngectomees to use
this system in face-to-face conversations. This proposedsystem, however, is expected to be eﬀective in some situations where listeners do not have to hear the source EL
speech such as when used in telecommunication systems.
In telecommunication systems, listeners must understandwhat a laryngectomee says from only the speech signals,
and therefore, converted speech should dominate thesource EL speech to allow the conversation to smoothly
ﬂow. When our aid system is used in telecommunication
systems, it is enough to transfer only converted speech sothat listeners do not have to hear the source EL speech.
Regarding telecommunications, for example, conventional
telephones encode normal speech signals. In other words,
current EL speech is unsuitable for use with telecommunications, making it diﬃcult for laryngectomees to use telephones. However, our proposed system enables
laryngectomees to conduct smooth communication on thetelephone.
3.1.2. VC for EL speech
In the VC from EL speech to normal speech (EL-toSpeech), three acoustic features of the spectral parameters,

Fig. 6. Photographs of sound source unit generating signals with extremely low energy.

140

K. Nakamura et al. / Speech Communication 54 (2012) 134–146

System for
EL speech(2) Recording

Static feature
vectors(1) Electrolarynx(3) Voice
Conversion(4) Converted
normal speech

Concatenated feature
vectors

System for
EL-air speech(2) Recording
PCA(1) Electrolarynx
Air-pressure sensor
Air flow(4) Converted
normal speech
System forsilent EL speech(3) Voice
Conversion

Fig. 8. Flow chart of the construction of segmental feature vectors fromstatic feature vectors.

(2) NAM microphone
Amplifier(1) Sound source unit

(4) Converted
normal speech

Segmental feature
vectors(3)Voice
Conversion

Fig. 7. Overview of proposed speaking-aid system for three types of EL
speech. Top accepts EL speech, middle does EL-air speech, and bottom
does silent EL speech.

F0 , and aperiodic components (Kawahara et al., 2001) are
independently estimated from only source spectral parameters. Aperiodic components, which are used to construct
mixed excitation signals (Ohtani et al., 2006), indicate thestrength of noises in each frequency band.
For the source data, we set a segmental feature vector inthis article that includes information over multiple frames.
This idea originates from another VC framework used toconvert NAM to normal speech (Toda and Shikano,2005). Fig. 8 shows a ﬂow chart of the construction of segmental feature vectors from static spectral parameter vectors. Let Hx = {x1, . . . , xt, . . . , xT} be a set of source staticfeature vectors, where xt = [xt(1), . . . , xt(d), . . . , xt(Dx)] is a>
Dx-dimensional feature vector. Let ct ¼ ½x>tL ; . . . ; xt ; . . . ;> >xtþL  be a Dc = D(2L + 1)-dimensional concatenated feature vector over the current ±L(L P 1) frames. Then, thebD-dimensional
segmental feature vector Xt at frame t is
extracted from ct by principal component analysis (PCA).
For the target data, joint feature vectors consisting of static

Table 1
Source and target acoustic features for EL-to-Speech.
SourceTargetSpectrum

Spectrum
F0
Aperiodic components

and delta features are constructed. Delta features arecalculated from the previous and succeeding frames.
Table 1 shows the source and target acoustic features ofEL-to-Speech. The training data of the GMM to estimate
target spectral parameters are joint vectors consisting of
the segmental feature vectors of source spectra and joint
feature vectors of target spectra. In the estimation of F0
and aperiodic components, individual GMMs are trainedusing joint feature vectors of the segmental feature vectors
and those of target Log-scaled F0 or aperiodic components.
In the conversion procedure, source segmental feature
vectors are constructed by the same approach as that in
the training part. Acoustic parameters are estimated by
the method described in Section 2.
3.2. Speaking-aid system for EL-air speech
3.2.1. Framework of the system
In this article we propose another speaking-aid systemthat accepts EL-air speech. An overview of this system isshown in Fig. 7. This system is expected to estimate morenatural F0 contours by using the air-pressure sensor. The
four components consisting of this system, the usage, andthe situations in which this system can be used are the same

K. Nakamura et al. / Speech Communication 54 (2012) 134–146

as those for the proposed system for EL speech described in
Section 3.1.1.
3.2.2. VC for EL-air speech
As shown in Fig. 3, the F0 contours of EL-air speechvary more than those of EL speech. However, it is diﬃcult
for users to intentionally switch between U/V sounds when
producing EL-air speech, and therefore, many U/V errors
are observed in the F0 contours of EL-air speech. In otherwords, the F0 contours of EL-air speech are not suﬃciently
accurate to present natural speech, and therefore, it is necessary to convert not only the spectral parameters but also
the F0 features of EL-air speech.
Table 2 shows the relationship between the source andtarget acoustic features in the VC from EL-air speech tonormal speech (EL-air-to-Speech). For the estimation oftarget spectral parameters and aperiodic components, thesegmental feature vectors created from spectral parametersof EL-air speech are used for the source data, which areconstructed by the same manner shown in Fig. 8.
For the estimation of F0 , two methods are considered
for constructing source segmental features: (1) concatenate
the static feature vectors of spectra and F0 , and then construct segmental feature vectors, or (2) construct segmental
feature vectors of spectra and F0 separately, and then concatenate these vectors (see Fig. 9). The ﬁrst method
includes a risk that the F0 information might not be presented when the segmental feature vectors are constructed.
Therefore, in this article we employ the second method.
Table 2
Source and target acoustic features for EL-air-to-Speech.
SourceTargetSpectrum
Spectrum and F0
Spectrum

Spectrum
F0
Aperiodic components

141

Log-scaled F0 values are extracted from EL-air speechand normal speech to be used for the static F0 features.
For the target F0 features, joint feature vectors of staticand delta F0 values are constructed.
3.2.3. Data recording of training data for EL-air-to-Speech
It is essential in VC to use source and target featuresthat correlate with each other. To obtain these data, a laryngectomee trained how to control F0 using the air-pressure sensor for one month. The laryngectomee further
trained to control F0 for another three weeks so that thepitch of EL-air speech sounds similar to that of the target
normal speech. EL-air speech was recorded after thistraining.
However, we noticed that it was too diﬃcult for the laryngectomee to mimic the target pitch pattern by controlling F0 with exhaled air. Moreover, the F0 patterns of therecorded EL-air speech were signiﬁcantly diﬀerent fromthose of the target speech. Therefore, we additionallyrecorded target normal speech for the recorded EL-air
speech. In this recording, a target speaker uttered target
speech while mimicking the pitch patterns of the recorded
EL-air speech as naturally as possible. Note that the F0
contours of the recorded EL-air speech are still diﬀerent
from those of the re-recorded target speech. For example,
the F0 contours of the recorded EL-air speech vary discontinuously, although those of the target normal speechsmoothly vary. These diﬀerences are expected to be
removed by VC in our speaking-aid system.
3.3. Speaking-aid system for silent EL speech
3.3.1. Framework of the system
The speaking-aid system for silent EL speech is shown in
Fig. 7. In our previous work, input silent EL speech was
only converted to a whispered voice (Nakamura et al.,2007). However, in this article we convert silent EL speech

Spectral staticfeature vectors

F0 static
feature vectors

Spectral segmental
feature vectors

F0 segmental
feature vectors

Fig. 9. Flow chart of the construction of segmental feature vectors from spectral and F0 feature vectors.

142

K. Nakamura et al. / Speech Communication 54 (2012) 134–146

to normal speech. This system addresses not only the
unnaturalness of EL speech but also the problem of noisy
sound source signals. The four components of this system
are the same as those of the other speaking-aid systems
described above. To address the problem of noisy soundsource signals, this system employs the new sound source
unit described in Section 2.4. Produced silent EL speechis captured by the NAM microphone described in Section
2.3 as well as NAM speech signals. In this article, silent
EL speech is recorded in a sound proof room that is extremely quiet space. Note that silent EL speech has energy toprovide auditory feedback to the speaker; however, theenergy is extremely low. Therefore, silent EL speech ishardly heard by listeners, and it is easily masked by external noises. As a result of the masking phenomenon, it isexpected that this system can be used in daily conversations
as well as with telecommunication systems.

which were extracted by mel-cepstral analysis (Fukada
et al., 1992), were used as the source spectral parameters,
where the 0 coeﬃcient captured power information. The
concatenating frame length for the source segmental feature vectors was set to 8. After the concatenation of frames,
50- and 2-dimensional components were extracted frameby frame to construct spectral and F0 segmental feature
vectors, respectively. F0 contours of the source speech signals were automatically extracted using a robust algorithm
for pitch tracking (Talkin, 1995). Acoustic features of thetarget speech were extracted by STRAIGHT analysis(Kawahara et al., 1999).
Following mel-cepstral distortion was used to measure
the spectral conversion accuracy.
qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ
ﬃ
PD
2
T 10
2 d¼1 fðtart ½d  convt ½dÞ g
1 X
Mel  cd½dB ¼
;
T t¼1
ln 10

3.3.2. VC for silent EL speech
VC from silent EL to normal speech (silent-EL-toSpeech) is performed by the method used for EL-to-Speech
described in Section 3.1.2. Three acoustic features of spectral parameters, F0 , and aperiodic components are estimated from only the spectral parameters of silent EL
speech. Segmental feature vectors are constructed in themanner shown in Fig. 8 and are set as the source features.
Joint features consisting of static and delta features are set
as the target features.

ð10Þ

4. Experimental evaluations
4.1. Experimental conditions
The source speaker was a laryngectomee (Japanesemale), who was proﬁcient in speaking with an EL. The target speaker was a laryngeal speaker (also a Japanese male).
Both speakers recorded 50 phoneme-balanced sentences,which served as training data, and 30 utterances fromnewspaper articles, which served as test data. The source
speaker recorded three types of alaryngeal speech: EL
speech, silent EL speech using a pulse train with a frequency of 100 Hz, and EL-air speech. The source speaker
used an EL named ‘yourtone’ (Hashiba et al., 2001) to produce EL speech and EL-air speech. The target speakerrecorded normal speech. EL-air speech and normal speech
were recorded in the manner described in Section 3.2.3. The
recorded target speech data were shared among the settingsof VC for each source speech. All speech signals wererecorded in a sound proof room. Silent EL speech wasrecorded with a NAM microphone. Other speech signals,
which were EL speech, EL-air speech, and the target normal speech, were recorded with a head-set microphone.
The number of mixture components of the GMMs used
to estimate spectral parameters and aperiodic components
was set to 32 and the number of mixture components of the
GMM used to estimate F0 was set to 16, 32, 64, and 128,
respectively. The 0 through 24 mel-cepstral coeﬃcients,

where tard[d] and convt[d] were dth coeﬃcients of the target
and converted mel-cepstrum at the frame t, respectively. T
was the total number of frames. The accuracy of F0 was
evaluated by U/V decision error rates and the correlationcoeﬃcient between frames for which the estimated and target frames both correspond to be voiced as shown in
Fig. 10. Target F0 contours of the test data including U/
V information were given as references for calculating U/
V decision error rates. The correlation coeﬃcients were calculated utterance by utterance in this article.
Six laryngeal speakers subjectively and independently
evaluated stimuli in terms of (1) naturalness, (2) intelligibility, and (3) preference in this order, which were all ratedusing a ﬁve-point-scaled opinion score (1: Bad, 2: Poor,
3: Fair, 4: Good, 5: Excellent). Our aid systems were
expected to be used in our daily life. Therefore, we evaluated the systems using not isolated words but continuous
sentences. For conducting writing test to evaluate the intelligibility, we needed to prepare lots of sentences to prevent
subjects from learning the contents. On the other hand, it
was diﬃcult for the source speaker to produce many

Estimated F0 contours
Target F0 contours
Extraction ofcorrectly matchedvoiced frames

Calcuatingcorrelation coefficientFig. 10. Calculation of correlation coeﬃcients between voiced frames of
estimated and target F0 contours.

sentences for each source speech. Therefore, we conducted
an opinion test on intelligibility. Five diﬀerent sentencesout of 30 test sentences were randomly selected for eachspeaker. Note that the subjects conducted perceptual evaluations for three times using diﬀerent test data set. Seven
types of stimuli were evaluated: analysis-synthesized target
normal speech, three types of recorded source speech signals (EL speech, silent EL speech, and EL-air speech),and three types of converted speech signals from eachsource speech. All stimuli were randomly given using aheadphone in a sound proof room. Converted and the target speech waveforms were synthesized using a mel log
spectrum approximation ﬁlter (Imai et al., 1983). When
synthesizing the converted speech waveforms, the GVparameters of only the converted spectra were taken into
account. F0 contours of the test data were estimated using
64 mixture components.

4.2. Experimental results
4.2.1. Objective results
Table 3 shows the results of mel-cepstral distortion. As
shown in the table, VC strongly enhances the spectra ofsource speech signals. The result of spectral conversion
from EL speech or EL-air speech is better than that fromsilent EL speech. The result of EL-air speech conversion
is slightly better than that of EL speech conversion in terms

Table 3Mel-cepstral distortions without power information.
Source speech

Source – Target± S.D. [dB]

Converted – Target± S.D. [dB]

Silent EL speech
EL speech
EL-air speech

11.42 ± 0.36
8.96 ± 0.31
9.51 ± 0.30

4.55 ± 0.24
4.25 ± 0.26
4.12 ± 0.20

1.0

Silent EL speech

EL speech

Correlation coefficients

EL-air speech
0.8

0.6

0.4

0.2

0

16

32

64

128

Number of mixture components
Fig. 11. Correlation of F0 contours between target and converted F0
contours.

Error of voiced or
unvoiced decision [%]K. Nakamura et al. / Speech Communication 54 (2012) 134–146

7.5

143

Silent EL speech
EL-air speech

EL speech

6.5
5.5
4.5
16

32

64

128

Number of mixture components
Fig. 12. U/V errors for converted F0 contours.

of the average and the standard deviation; however, theperformances of VC for EL speech and EL-air speech arevery close each other.
Fig. 11 shows the correlation coeﬃcients between voiced
values of target and converted F0 contours. From this ﬁgure, the results of using silent EL speech and EL speech arealmost the same. Overtraining, however, occurred in the
result of EL speech with 128 mixture components. On theother hand, the result using EL-air speech is better thanthat using the other two speech signals, and moreover, thisresult remains almost the same for diﬀerent numbers of
mixture components.
Fig. 12 shows U/V decision error rates for converted F0
contours. According this ﬁgure, EL-air speech produces
fewer U/V decision errors than the other two speech signals, even though overtraining is observed when GMM
has 128 mixture components.
Fig. 13 shows examples of F0 contours of source EL-air
speech, converted speech, and the corresponding target
speech. As shown in this ﬁgure, VC is highly eﬀective formaking the F0 contours of the EL-air speech smoothly
and continuously vary while suitably switching between
U/V decisions.
4.2.2. Subjective results
Fig. 14 shows the mean opinion score (MOS) for eachtest.
The result shows that our systems dramatically improve
the naturalness of each source speech. Namely, the resultsof the converted speech have the signiﬁcance to the corresponding source speech considering 5% signiﬁcance level.
However, the results of the three converted speech does
not have the signiﬁcance each other considering 5% significance level, since the naturalness of the converted silent
EL speech, the converted EL speech, and the converted
EL-air speech varies between 2.31 and 3.09, 2.94 and
3.53, and 3.08 and 3.92, respectively. This result shows that
our speaking-aid systems are highly eﬀective for improving
the naturalness of three types of EL speech signals.
The intelligibility of the converted silent EL speech isscored to vary between 1.77 and 2.49, and that of thesource silent EL speech is scored to vary between 1.13
and 1.47. Therefore, the converted silent EL speech has
the signiﬁcance to source EL speech on intelligibility.

144

K. Nakamura et al. / Speech Communication 54 (2012) 134–146
30k

Frequency [Hz]EL-air speech

Amplitude

400
0

200
100
50

-30k
0

1

2

3
4
5
Time[sec]

6

7

0

1

2

3
4
5
Time[sec]

6

7

0

1

2

3
4
5
Time[sec]

6

7

0

1

2

3
4
5
Time[sec]

6

7

30k

Converted
speechFrequency [Hz]Amplitude

400
0

200
100
50

-30k
0

1

2

3
4
5
Time[sec]

6

7

NormalspeechFrequency [Hz]Amplitude

30k

0

400
200
100
50

-30k
0

1

2

3
4
5
Time[sec]

6

7

Fig. 13. Examples of F0 contours in EL-air-to-Speech.

95% confidence intervals
MOS

5
3

MOS

1
5

Intelligibility

3
1
5

MOS

Naturalness

Preference

3
1

org

cv

VC fromsilent EL
speechorg

cv

VC from
EL speechorg

cv Target
speechVC fromEL-airspeechStimuliFig. 14. Result of subjective evaluation by six laryngeal speakers. ‘org’and ‘cv’ denote original and converted speech, respectively. Arrows inﬁgure shows that improvement was observed considering 5% signiﬁcancelevel.

However, other two VC experiments have no signiﬁcancebetween before and after VC procedure. Although themean value of the intelligiblity of the converted EL speech
is slightly degraded than that of the source EL speech, nosigniﬁcant diﬀerences are observed when we consider the
95% conﬁdence intervals. The intelligibility of the converted EL-air speech is similar to that of the source ELair speech, although each result is better than the resultof the converted silent EL speech.
The preference for each converted speech scored higher
than that for the corresponding source speech. The preference of the EL speech conversion also has the signiﬁcanceconsidering 5% signiﬁcance level. Namely, the preferenceof the source and the converted EL speech vary between
1.68 and 2.66 and 2.67 and 3.3, respectively. It is interestingthat the preference for the source EL-air speech is similar
to that for source EL speech, although EL-air speech
includes more F0 information than EL speech. Although
the mean value of the preference of converted EL-air
speech is better than that of the converted EL speech, there
is no statistical signiﬁcance considering 5% signiﬁcancelevel since the preference of the converted EL speech varies
between 2.67 and 3.33 and that of the EL-air speech varies
between 3.28 and 4.12.

K. Nakamura et al. / Speech Communication 54 (2012) 134–146

5. Discussion
Diﬀerences of mel-cepstral distortions after VC are notlarge, on the other hand, certain tendency is appeared.
Since the target speech is shared among each VC experiment, the result of the mel-cepstral distortion is causedby the diﬀerence of the type of the source speech, namely,
the amount of information source speech has. Since thesound source signals of EL speech and EL-air speech arelarger than those of silent EL speech, the result of VC from
EL speech or EL-air speech is better than that from silent
EL speech. Moreover, EL speech and EL-air speech signals
were captured by an air-conductive microphone, whereassilent EL speech signals are captured by a body-conductive
microphone. Therefore, EL and EL-air speech containmuch more information than silent EL speech. On theother hand, the diﬀerence between EL speech and EL-air
speech is only the existence of the air-pressure sensor,
which mainly aﬀects not spectral information but F0 information. Therefore, it is reasonable to obtain similar resultsof VC between EL speech and EL-air speech.
A reason why the mean value of the intelligibility of thesource EL speech is higher than that of the source EL-air
speech is a diﬀerence of training periods to produce EL
speech or EL-air speech. The source speaker has used an
EL in his daily life for more than 10 years, and therefore,
the speaker well knows how to produce intelligible EL
speech. The source speaker trained how to produce ELair speech and control its F0 using his exhaled air for around
one month and three weeks. After the speech recording, thesource speaker gave us a comment that he might be able toproduce more natural and intelligible EL-air speech if hecould get more training periods. As a result, it is reasonable
that the intelligibility of the source EL speech is higher thanthat of the source EL-air speech, since the duration of training of the source speaker to produce EL-air speech isshorter than that to produce EL speech.
VC surely reduced radiation noises. However, the current VC procedure might cause another distortion on intelligibility, which is our future work. As a result, there are no
signiﬁcant diﬀerences in the intelligibility before and afterconverting EL speech and EL-air speech.
The subjective result of preference has a similar tendency to that of naturalness than that of intelligibility. Thisis because the improvement of the naturalness aﬀected thesubjects much more than the diﬀerence of intelligibility.
Moreover, the result of preference of the converted ELair speech is better than that of the converted EL speech.
Considering the objective result of spectral and F0 estimation accuracy, the result of preference score comes from theimprovement of F0 estimation accuracy. The gap between
the converted EL-air speech and target speech in the preference result is the combination of a defect of intelligibility
due to the spectral estimation and that of naturalness due
to F0 estimation. For more improvement of F0 estimation,
it might be eﬀective to use more training data.

145

From these results of spectral and F0 estimation, we conclude that VC dramatically improves the spectral performance of source speech signals, and the use of the airpressure sensor eﬀectively improves the converted speech
quality.
6. Conclusion
In this article we proposed three speaking-aid systems
for EL speech, EL-air speech, and silent EL speech withthe aim of addressing two problems EL speech: its unnaturalness and noisy sound source signals. We introduced astatistical VC technique for addressing the unnaturalness
of EL speech and employed a new sound source unit, which
generated signals with extremely low energy, to address the
problem of noisy sound source signals. We also introduced
an air-pressure sensor that enables laryngectomees tomanipulate the F0 contours of an EL using their exhaledair. This device was expected to enable the estimation of
more natural F0 contours.
As the result of experimental evaluations, it was shownthat VC greatly enhanced the spectral quality. The use of
the air-pressure sensor dramatically improved the correlation coeﬃcients of F0 contours using only voiced frames
from 0.58 to 0.78 and also suppressed the proportion ofU/V decision errors from 5.44% to 4.73%. Moreover, subjective evaluations indicated that our speaking-aid systems
dramatically improved the naturalness of EL speech andconverted speech is preferred to the corresponding source
EL speech.
Acknowledgments
The authors are grateful to Professor Hideki Kawahara
of Wakayama University, Japan, for permission to use the
STRAIGHT analysis-synthesis method. This research was
also supported in part by MIC SCOPE and Grant-in-Aid
for JSPS Fellows.
References
Carr, M.M., Schmidbauer, J.A., Majaess, L., Smith, R.L., 2000.
Communication after laryngectomy: An assesment of quality of life.
Arch. Otolaryngol. Head and Neck Surg. 122 (1), 39–43.
Carr, M.M., Schmidbauer, J.A., Majaess, L., Smith, R.L., 2000.
Communication after laryngectomy: an assessment of quality of life.
Otolaryngol. Head and Neck Surg. 122 (1), 39–43.
Dempster, A.P., Laird, N.M., Rubin, D.B., 1977. Maximum likelihoodfrom incomplete data via the EM algorithm. J. Roy. Stat. Soc. Ser. B(Methodological) 39 (1), 1–38.
Forastiere, A.A., Goepfert, H., Maor, M., Pajak, T.F., Weber, R.,
Morrison, W., Glisson, B., Trotti, A., Ridge, J.A., Chao, C., Peters,
G., Lee, D., Leaf, A., Ensley, J., Cooper, J., 2003. Concurrent
chemotherapy and radiotherapy for organ preservation in advanced
laryngeal cancer. The New England J. Med. 349 (22), 2091–2098.
Fukada, T., Tokuda, K., Kobayashi, T., Imai, S. 1992. An adaptivealgorithm for mel-cepstral analysis of speech. In: Proc. Internat. Conf.
on Acoustics, Speech and Signal Processing (ICASSP), Vol. 1, pp. 137–
140.

146

K. Nakamura et al. / Speech Communication 54 (2012) 134–146

Goldstein, E.A., Heaton, J.T., Kobler, J.B., Stanley, G.B., Hillman, R.E.,
2004. Design and implementation of a hands-free electrolarynx device
controlled by neck strap muscle electromyographic activity. IEEE
Trans. Biomed. Eng. 51 (2).
Hashiba, M., Uemi, N., Oikawa, M., Yamaguchi, Y., Sugai, Y., Ifukube,T., 2001. Industrialization of the electrolarynx with a pitch control
function and its evaluation. IEICE Trans. Inform. Syst. J94-D-II (6),1240–1247, in Japanese.
Hayes, M., 1951. Rehabilitation of the laryngectomee. A Cancer J. Clinic.
1, 147–152.
Hočevar-Boltežar, I., Žargi, M., 2001. Communication after laryngectomy. Radiat. Oncol. 35 (4), 249–254.
Hosoi, Y., Sakaguchi, T. 2003. Silent Voice Input System Without
Exhalation –Theory and Applications, Technical Report of IEICE,
SP2003-105, pp. 13–16.
http://www.toshin-ha.co.jp/ (conﬁrmed on 23.02.2011, in Japanese).
Ifukube, T., 2003. Sound-based Assistive Technology for the Disabled.
Corona publishing Co. Ltd., in Japanese.
Imai, S., Sumita, K., Furuichi, C., 1983. Mel log spectrum approximation(MLSA) ﬁlter for speech synthesis. Electron. Commun. Jpn. 66 (2), 10–18.
Jemal, A., Siegel, R., Ward, E., Hao, Y., Xu, J., Murray, T., Thun, M.J.,
2008. Cancer statistics, 2008. A Cancer J. Clinic. 58, 71–96.
Kain, A., Macon, M.W. 1998. Spectral voice conversion for text-to-speechsynthesis. In: Proc. Internat. Conf. on Acoustics, Speech and Signal
Processing (ICASSP), pp. 285–288.
Kawahara, H., Masuda-Katsuse, I., de Cheveigné, A., 1999. Restructuring
speech representations using a pitch-adaptive time–frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible roleof a repetitive structure in sounds. Speech Commun. 27 (3–4), 187–207.
Kawahara, H., Estill, J., Fujimura, O., 2001. Aperiodicity extraction andcontrol using mixed mode excitation and group delay manipulation fora high quality speech analysis, modiﬁcation and synthesis system
STRAIGHT. 2nd Models Anal. Vocal Emissions for Biomed. Appl.
(MAVEBA).
Laccourreye, O., Weinstein, G., Naudo, P., Cauchois, R., Laccourreye,H., Brasnu, D., 1996. Supracricoid partial laryngectomy after failed
laryngeal radiation therapy. The Laryngoscope 106 (4), 495–498.
Liu, H., Zhao, Q., Wan, M., Wang, S., 2006. Enhancement of electrolarynx speech based on auditoy masking. IEEE Trans. Biomed. Eng.
53 (5), 865–874.
Miyamoto, D., Nakamura, K., Toda, T., Saruwatari, H., Shikano, K.
2009. Acoustic compensation methods for body transmitted speech
conversion. In: IEEE Internat. Conf. on Acoustics, Speech, and Signal
Processing (ICASSP2009), pp. 3901–3904.
Murakami, K., Araki, K., Hiroshige, M., Tochinai, K., 2004. A
method for speech transform from electrolaryngeal speech tonormal speech. IEICE Trans. Inform. Syst. J87-D-I (11), 1030–
1040, in Japanese.
Nakagiri, M., Toda, T., Kashioka, H., Shikano, K., 2006. Improving bodytransmitted unvoiced speech with statistical voice conversion. Proc.
Interspeech 2006, 2270–2273.
Nakajima, Y., Kashioka, H., Shikano, K., Campbell, N., 2005. Remodeling of the sensor for non-audible murmur (NAM). Proc. Interspeech
2005, 293–296.
Nakajima, Y., Kashioka, H., Campbell, N., Shikano, K., 2006. Nonaudible murmur (NAM) Recognition. IEICE Trans. Inform. Syst.
E89-D (1), 1–8.
Nakamura, K., Toda, T., Saruwatari, H., Shikano, K., 2007. A speechcommunication aid system for total laryngectomees using voice
conversion of body transmitted artiﬁcial speech. IEICE Trans. Inform.
Syst. J90-D (3), 780–787, in Japanese.
Ohtani, Y., Toda, T., Saruwatari, H., Shikano, K., 2006. Maximum
likelihood voice conversion based on GMM with STRAIGHT mixed
excitation. Proc. Interspeech 2006, 2266–2269.
Saikachi, Y., Stevens, K.N., Hillman, R.E., 2009. Development andperceptual evaluation of amplitude-based F0 control in electrolarynx
speech. J. Speech, Lang. Hearing Res. 52, 1360–1369.
http://www.secom.co.jp/personal/medical/myvoice.html (conﬁrmed on
23.02.2011, in Japanese).
Singer, M.I., Blom, E.D., 1980. An endoscopic technique for restorationof voice after laryngectomy. The Anal. Otol. Rhinol. Laryngol. 89,
529–533.
Takahashi, H., Nakano, M., Okusa, T., Hatamura, Y., Kikuchi, Y.,
Kaga, K., 2001. A voice-generation system using an intramouth
vibrator. J. Artif. Organs 4 (4), 288–294.
Talkin, D., 1995. A robust algorithm for pitch tracking. In: Speech Coding
and Synthesis. Elsevier Science, pp. 495–518.
Toda, T., Shikano, K., 2005. NAM-to-Speech conversion with Gaussian
mixture models. Proc. Interspeech 2005, 1957–1960.
Toda, T., Black, A.W., Tokuda, K., 2007. Voice conversion based on
maximum likelihood estimation of spectral parameter trajectory. IEEE
Trans. Audio, Speech Lang. Process. 15 (8), 2222–2235.
Toda, T., Nakamura, K., Nagai, T., Kaino, T., Nakajima, Y., Shikano,K., 2009. Technologies for processing body conductive speech detectedwith non-audible murmur microphone. Proc. Interspeech 2009, 632–
635.
Uemi, N., Ifukube, T., Takahashi, M., Matsushima, J. 1994. Design of a
new electrolarynx having a pitch control function. In: Proc. 3rd IEEE
Internat. Workshop of Robot and Human Communication, pp. 198–
203.
Williams, S.E., Watson, J.B., 1985. Diﬀerences in speaking proﬁciencies inthree laryngectomee groups. Arch. Otolaryngol. Head and Neck Surg.
111, 216–219.