An Enhanced Electrolarynx with Automatic Fundamental
Frequency Control based on Statistical Prediction
Kou Tanaka, Tomoki Toda, Graham Neubig, Sakriani Sakti and Satoshi Nakamura
Graduate School of Information Science, Nara Institute of Science and Technology
8916-5 Takayama-cho, Ikoma, Nara, Japan{ko-t, tomoki, neubig, ssakti, s-nakamura}@is.naist.jp
ABSTRACT
Production of morenaturally sounding speech

An electrolarynx is a type of speaking aid device which
is able to mechanically generate excitation sounds to help
laryngectomees produce electrolaryngeal (EL) speech. Al­though EL speech is quite intelligible, its naturalness suﬀers
from monotonous fundamental frequency patterns of the me­
chanical excitation sounds. To make it possible to generate
more natural excitation sounds, we have proposed a method
to automatically control the fundamental frequency of thesounds generated by the electrolarynx based on a statisticalprediction model, which predicts the fundamental frequency
patterns from the produced EL speech in real-time. In thispaper, we develop a prototype system by implementing theproposed control method in an actual, physical electrolarynx
and evaluate its performance.

MicSystem
Electrolarynx automatically Expired air(Real time prediction) controlled by predicted

Figure 1: Proposed system to directly control elec­
trolarynx using real-time statistical F0 prediction.
To generate more natural F0 patterns, we have proposeda method to control F0 based on the statistical F0 predic­
tion [1]. In our proposed system based on this method, F0patterns are predicted from the produced EL speech signals
as shown in Fig. 1. Relatively natural F0 patterns can bepredicted using statistics extracted in advance from parallel
data consisting of utterance pairs of EL speech and naturalspeech. Therefore, this system allows laryngectomees to di­rectly produce more natural EL speech in the same manneras in the traditional speaking method using the conventional
electrolarynx. Our preliminary experimental results throughsimulation [1] have demonstrated that the proposed method
yields signiﬁcant improvements in naturalness while causing
no degradation in listenability and intelligibility comparedto the original EL speech.
In this paper, we develop a prototype system by imple­
menting our proposed F0 control method in an actual, phys­
ical electrolarynx and evaluate its performance. The exper­
imental results demonstrate that the prototype system gen­erates more natural excitation sounds, as in the simulation.

Categories and Subject Descriptors
K.4.2 [Computers and Society]: Social Issues - Assistive
technologies for persons with disabilities

Keywords
electrolaryngeal speech, automatic fundamental frequency
control

1. INTRODUCTION
Electrolaryngeal (EL) speech is one of the major speaking
methods used by laryngectomees who are people who have
had their larynx usually due to laryngeal cancer. EL speech
is produced using an electrolarynx, which is typically heldagainst the neck to mechanically generate artiﬁcial excita­tion signals. The generated excitation signals are conducted
into the speaker’s oral cavity, and are articulated to pro­
duce EL speech. EL speech is relatively intelligible but its
naturalness is very low owing to unnatural fundamental fre­quency (F0 ) patterns of the mechanically generated excita­tion signals. Consequently, quality of life of laryngectomees
is signiﬁcantly degraded.

2.

METHODOLOGY

Direct Control of Excitation Signals of Electrolar­
ynx based on Statistical F0 Prediction: Our proposedsystem allows a laryngectomee to produce EL speech withpredicted F0 patterns using two processes: prediction andarticulation [1]. In the prediction process, the F0 value ispredicted frame by frame using the real-time voice conver­sion algorithm [2] from EL speech produced by the laryn­
gectomee. This process causes a constant processing de­lay of 50 ms to make it possible to predict relatively natu­
ral F0 patterns varying corresponding to linguistic contents.
In the articulation process, the laryngectomee produces theEL speech by articulating the excitation sounds generated

Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor proﬁt or commercial advantage and that copies bear this notice and the full cita­tion on the ﬁrst page. Copyrights for third-party components of this work must behonored. For all other uses, contact the Owner/Author(s). Copyright is held by theowner/author(s).
ASSETS’15, October 26–28, 2015, Lisbon, Portugal.
ACM 978-1-4503-3400-6/15/10.
DOI: http://dx.doi.org/10.1145/2700648.2811340.

435

EL speech
waveform

50ms

50ms

50ms

50ms

AB

CD200 a) Conventional EL speech100

Low-delay conversion
pattern…

A

B

50C
Frequency [Hz]Predicted

Writing signal into D/A converter
Excitation signalof electrolarynx

…

…

…

AFigure 2: Latency caused by each process200b) Proposed EL speech10050200 c) Target normal speech
100

from the electrolarynx based on the predicted F0 values.
These two processes are simultaneously and continuouslyperformed. Consequently, EL speech with the predicted F0patterns can be directly produced by the laryngectomee but
it always suﬀers from misalignment between the articulated
sounds and F0 patterns caused by the processing delay of 50ms. The results of our previous evaluation through simula­tion have demonstrated that the impact of this misalignment
on perception is small [1].
Development of Prototype System: A prototype sys­
tem based on our proposed technique has been developed
using a laptop and a digital/analog (D/A) converter shownin Table 1. As shown in Fig. 1, EL speech produced frommouth of a laryngectomee is detected with a standard closetalk microphone. The EL speech signal is recorded on alaptop and F0 patterns of normal speech are predicted on
the ﬂy by using the real-time statistical F0 prediction. The
predicted F0 values are linearly converted to voltage values
to control the F0 values of the excitation signal generated
by an electrolarynx. Then, an electric signal corresponding
to the determined voltage values is generated with the D/Aconverter connected from the laptop to the electrolarynx.
The electrolarynx changes the F0 values of the excitation
signal according to the input electric signal generated fromthe D/A converter. As shown in Fig. 2, additional latency
is caused by the D/A converter in the prototype system. It
takes around 50 ms to write the digital signal on the D/Aconverter. Moreover, the digital signal to be written needs
to be determined before starting writing. Consequently, theD/A part always causes 100 ms latency. In total, 150 mslatency is caused in the prototype system. Note that thislatency in the D/A part may be addressed by the develop­
ment of a special device for the electrolarynx.

5004Time [s]68Figure 3: Example of waveforms and F0 patternsof EL speech signals generated by conventional andproposed systems and those of target natural speech.
balanced sentence set [3]. We conducted a 5-fold cross vali­
dation test in which 40 utterance pairs were used for trainingof a statistical F0 prediction model, and the remaining 10 ut­
terance pairs were used for evaluation. Sampling frequency
was set to 16 kHz.
Experimental results: The F0 correlation coeﬃcient
between the prototype system and the simulation process
is 0.91. This result shows that F0 patterns predicted by
the prototype system strongly correlate to those by the sim­
ulation process, which have already been conﬁrmed to beeﬀective for improving naturalness of EL speech [1]. An ex­
ample of EL speech signals and their F0 patterns are shownin Fig. 3. We can see that the prototype system makes it
possible to produce EL speech with more naturally varying
F0 patterns compared to the conventional EL speech.

4.

CONCLUSIONSIn this paper, we have developed a prototype system byimplementing our proposed F0 control method of an elec­
trolarynx based on the statistical F0 prediction techniqueand evaluate its performance. The experimental results havedemonstrated that the prototype system enables a speakerto produce more naturally sounding electrolaryngeal speech.

5.
3. EVALUATION AND RESULTS

ACKNOWLEDGMENTS

This work was supported in part of JSPS KAKENHI Grant
Numbers: 26280060 and the authors would like to thank Mr.
Y. Sugai of Densei Communication Inc., Japan, for advise
to control an electrolarynx.

We conducted an objective evaluation for evaluating pre­diction accuracy of F0 patterns generated by the developed
prototype compared to that conﬁrmed in the simulation pro­
cess [1]. The source speech was EL speech uttered by one
non-disabled male speaker, and the target speech was nor­mal speech uttered by a professional female speaker. Eachspeaker uttered about 50 sentences in the ATR phonetically

6.

REFERENCES[1] K. Tanaka, T. Toda, G. Neubig, S. Sakti, andS. Nakamura, “Direct F0 control of an electrolarynx
based on statistical excitation feature prediction andits evaluation through simulation,” in Proc.
INTERSPEECH, Sep 2014.
[2] T. Toda, T. Muramatsu, and H. Banno,“Implementation of computationally eﬃcient real-time
voice conversion.” in Proc. INTERSPEECH, Sep 2012.
[3] M. Abe, Y. Sagisaka, T. Umeda, and H. Kuwabara,“Speech database,” ATR Technical Report, TR-I-0166,
Sep 1990.

Table 1: Electronic devices on the prototype system
Electrolarynx
Microphone
CPU of the laptop
D/A converter

2Yourtone
Crown CM-311AIntel(R) Core(TM) i5-4200U
AIO-160802AY-USB436