INVESTIGATIONS OF REAL-TIME GAUSSIAN FFTNET AND PARALLEL WAVENETNEURAL VOCODERS WITH SIMPLE ACOUSTIC FEATURES
Takuma Okamoto1 , Tomoki Toda2,1 , Yoshinori Shiga1 , and Hisashi Kawai1
1

National Institute of Information and Communications Technology, Japan
2
Information Technology Center, Nagoya University, Japan
ABSTRACTThis paper examines four approaches to improving real-time neuralvocoders with simple acoustic features (SAF) constructed from fundamental frequency and mel-cepstra rather than mel-spectrograms.
The investigations are as follows: 1) the effectiveness of single Gaussian (SG) autoregressive (AR) WaveNet and FFTNet vocoders with
SAF, 2) the possibility of SG parallel WaveNet vocoder training andsynthesis with SAF, 3) the impact of noise shaping on SG AR neuralvocoders, and 4) the efficacy of bandwidth extension to synthesize
speech waveforms at a sampling frequency of 24 kHz by SG AR
neural vocoders from SAF for that of 16 kHz. The results of experiments indicate that SG AR WaveNet and real-time SG AR FFTNet
vocoders with noise shaping using SAF can realize sufficient synthesis quality with bandwidth extension effect. Moreover, a real-time
SG parallel WaveNet vocoder can also be trained using SAF.
Index Terms— Speech synthesis, neural vocoder, FFTNet, parallel WaveNet, Gaussian inverse autoregressive flow, noise shaping
1. INTRODUCTION
Neural network-based raw audio autoregressive (AR) generativemodels such as WaveNet [1], SampleRNN [2], FFTNet [3], and
WaveRNN [4] have been recently investigated and outperform theconventional source-filter vocoders typically employed in conventional statistical parametric speech synthesis (SPSS) [5] and voice
conversion (VC) [6]. Such raw audio generative models can realize
end-to-end text to speech (TTS), converting text to raw speech waveforms. The speech quality of English synthesized by Tacotron 2
can match that of natural speech at a sampling frequency f s of
24 kHz [7].
In contrast to TTS, a WaveNet vocoder that directly synthesizes raw speech waveforms from acoustic features [8] has been
proposed to drive conventional source-filter vocoders within a neural network-based raw audio generative model framework. Neuralvocoders based on WaveNet [8] and SampleRNN [9] have been applied to SPSS [10, 11] and VC [12–14], and have been shown tooutperform conventional source-filter vocoders.
Compared with conventional source-filter vocoders, however,the synthesis speed of AR neural vocoders remains problematic, because the sequential synthesis of each sample requires a huge amountof calculation time [1,2]. To overcome this problem, FFTNet [3] hasa simpler structure based on a 1 × 1 convolutional network and rectified liner unit layers. As a result, FFTNet can synthesize raw audio
in real time. In addition, noise shaping [15, 16] and subband approaches [17, 18] have been applied to 256-way categorical FFTNet
to improve the synthesized speech quality [19]. However, its performance cannot reach that of 256-way categorical WaveNet with noise
shaping [19].

Parallel WaveNet [20] introduces teacher–student knowledge
distillation based on the inverse autoregressive flow (IAF) [21] andWaveRNN [4] introduces a single-layer recurrent neural networkwith sparse and subscale modifications. These methods enablereal-time synthesis with 16-bit raw audio prediction. However, thetraining in parallel WaveNet is unstable because of the intractable
Kullback–Leibler (KL) divergence between the student logistic andteacher mixture of logistics (MoL) distributions [22].
To solve this problem, single Gaussian (SG) parallel WaveNet
was proposed. This method is based on the Gaussian IAF [21];the non-AR student WaveNet is trained using a SG AR teacher
WaveNet. An SG-based AR WaveNet outperforms a MoL-based
AR WaveNet and the student network is better trained than in a
MoL-based parallel WaveNet. The entire end-to-end TTS based on
SG parallel WaveNet is called ClariNet [22].
The SG modeling is only applied to AR and parallel WaveNet
vocoders with 80-band mel-spectrograms for end-to-end TTS [22].
In conventional SPSS and VC, however, source-filter vocoders arestill widely employed to synthesize speech waveforms from estimated and converted simple acoustic features (SAF) mainly constructed from the fundamental frequency and mel-cepstra [23, 24]rather than mel-spectrograms [7, 14, 22]. Therefore, it is important
to investigate the effectiveness of SG modeling in AR WaveNet and
FFTNet with SAF, as employed in previous neural vocoders [8–10, 12, 13, 15, 16, 18, 19]. In addition, it is also important to determine how parallel WaveNet can be successfully trained from SAF
because this remains uninvestigated. Furthermore, the impact ofnoise shaping for these SG-based neural vocoders should also be
investigated at f s = 24 kHz because all previous investigations
used 256-way categorical WaveNet and FFTNet neural vocoders at
f s = 16 kHz [15, 16, 19].
In Tacotron 2 [7], high quality speech waveforms at f s =24 kHz are synthesized from 80-band mel-spectrograms with a frequency band of 125–7,600 Hz. The result indicates that SAF forlower sampling frequencies might also be able to synthesize highquality speech waveforms. Therefore, it is also important to investigate whether speech waveforms at f s = 24 kHz can be synthesized
by neural vocoders from SAF for f s = 16 kHz to explore the possibility of bandwidth extension. If this is possible, existing SPSS and
VC systems operating at f s = 16 kHz could be able to synthesize
speech waveforms at f s = 24 kHz.
Consequently, to improve real-time neural vocoders with SAF,
this paper investigates the following four topics: 1) the effectivenessof the SG AR WaveNet and FFTNet vocoders with SAF, 2) the possibility of SG parallel WaveNet training and synthesis with SAF, 3)
the impact of noise shaping on SG AR neural vocoders, and 4) the
efficacy of bandwidth extension to synthesize speech waveforms at
f s = 24 kHz by SG AR neural vocoders from SAF at f s = 16 kHz.

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including
reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, orreuse of any copyrighted component of this work in other works.

p
⇥ 0.5

l-th layer<latexit sha1_base64="LXZEjyTBNaLx5XJQhk2fvAWplYg=">AAACdnichVFNLwNBGH66vqq+iotEIo2muGjeFYnGSeLi6KtIkGZ3DTad7m52p6Uaf8AfcBAHEkT8DBd/wMFPEEcSDg7ebjcRBO9kZp555n3eeWbG9KQdKKKHmNbU3NLaFm9PdHR2dfcke/tWArfsWyJvudL110wjENJ2RF7ZSoo1zxdGyZRi1SzO1vdXK8IPbNdZVlVPbJaMHcfeti1DMVVI9soNJfZVbVztpqRRFf5hIZmmLIWR+gn0CKQRxbybvMIGtuDCQhklCDhQjCUMBNzWoYPgMbeJGnM+IzvcFzhEgrVlzhKcYTBb5HGHV+sR6/C6XjMI1RafIrn7rEwhQ/d0Tc90Rzf0SO+/1qqFNepeqjybDa3wCj1HA0uv/6pKPCvsfqr+9KywjVzo1WbvXsjUb2E19JWD4+el6cVMbYTO6Yn9n9ED3fINnMqLdbEgFk+Q4A/Qvz/3T7AykdUpqy9Mpmdy0VfEMYhhjPF7T2EGc5hHns/dwykucRV704a0jDbaSNVikaYfX0KjD648kRI=</latexit><latexit sha1_base64="Li6rjy9OEocqkY8bl/39fXwD7SM=">AAACdnichVG7ThtBFD3eBALmYZM0kSJFKywDlXXXvOxUSDQpMWBAwsjaXcZm5H1ld+yIWPxAfiBFREGkYKF8Rhp+gMKfgCiNBEWKXK8dRRTAXe3MuWfuuXNmxgocGSmibkJ78XJk9NXYeHJicmo6lZ55vRP5zdAWZdt3/HDPMiPhSE+UlVSO2AtCYbqWI3atxnp/fbclwkj63rY6DsSBa9Y9WZO2qZiqpmcqSroi0ivRp1C1Kbd8Uk1nKGfkC4XVJZ3zYtGgRQYrlCejqBs5iiODYWz46Q4qOIQPG024EPCgGDswEfG3DwOEgLkDtJkLGcl4XeAESdY2uUpwhclsg8c6Z/tD1uO83zOK1Tbv4vAfslJHlq7ognp0Sb/omv482qsd9+h7OebZGmhFUE19fbt196zK5Vnh6L/qSc8KNRRir5K9BzHTP4U90Le+fOttfdjMtufoB92w/zPq0m8+gde6tX+WxOZ3JPkB/t2y/jjYyecMxqWlzFph+BRjeIdZLPB9r2INH7GBMu/7Gac4Rydxr73Xstr8oFRLDDVv8CA0+gtGFpDh</latexit><latexit sha1_base64="YZNQ6PhZPSD898Nbq1aHfrioKkY=">AAAC0HichVE9b9RAEB2br2A+ckCDRGNxDqI67aZJRBWJhoIiuXBJpPPpZO9NLqvz7lr23pFgWYg2f4CCikgRQvwMGloKivwERBmkNBSMfZYQRMCsdnfmzbzZt7txmsjcMnbiuBcuXrp8ZeGqd+36jZuLrVu3t3IzzQT2hElMthNHOSZSY89Km+BOmmGk4gS348njKr89wyyXRj+zBykOVDTWcleKyBI0bE1Cjc+FUSrSoyKMZdnng6IIVWz2KTTJSEV2zw/aPCjL0gtjHEtdCNQWs9Lr4tOeH4TWBH7A/dBKhbnPgwaqsl6I1LepH7barMNq8887vHHa0Ni6ab2DEEZgQMAUFCBosOQnEEFOow8cGKSEDaAgLCNP1nmEEjziTqkKqSIidELrmKJ+g2qKq555zRZ0SkIzI6YPS+wLe89O2Sf2gX1lP/7aq6h7VFoOaI/nXEyHi4d3N8/+y1K0W9j7xfqnZgu7sFprlaQ9rZHqFmLOn714fbr5qLtUPGBH7Bvpf8tO2Ee6gZ59F8cb2H0DHn0A//O5zztbyx3OOnxjub222nzFAtyD+/CQ3nsF1uAJrEOPzv0MZw44jtt1992X7qt5qes0nDvwm7mHPwELorBg</latexit>

Residual connection<latexit sha1_base64="4tkf4QNJxZ4INFcYSSoExkg65CQ=">AAACeHichVHNTttAEP5iyk/DXygXpF4sIihcorEDTdITEpceSSA/UkCRvdnACse2bCdSiPoCfQEOHCoqoRb1MXrpC/SQR6g4glSp4tCJkwpxoMxqd2e/nW/22xnbd1QYEQ0S2sSLyanpmZfJ2bn5hcXU0qtK6HUCIcvCc7ygZluhdJQry5GKHFnzA2m1bUdW7dPd4X21K4NQee5B1PPlUds6dlVLCStiqJFaLslQNTuWowvPdaUYoWnK5AvGdn5Lp4xhFszcW3ay2UIuS7qRodjSGNuel/qCQzThQaCDNiRcROw7sBDyqMMAwWfsCH3GAvZUfC/xAUnmdjhKcoTF6Cmvx3yqj1GXz8OcYcwW/IrDM2CmjjX6Sdd0Sz/oG/2i+ydz9eMcQy093u0RV/qNxY8r+7+fZbV5j3DywPqv5ggt5GOtirX7MTL8hRjxu2fnt/vvSmv9dfpMN6z/kgb0nX/gdu/EVVGWLpDkBvyrsv60UzEzBrenaKZ38uNWzOA1VrHB9c5hB++xhzK/28MnfMV14o+ma2+0zVGolhhzlvHINPMvKauSOQ==</latexit>

ReLU ! 1 ⇥ 1 ! ReLU+
<latexit sha1_base64="MNQ8Guaynm3W8HzRQJbP8gFpqH4=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIxySlpmfmVSen5pWkFtVyqWircMWk5qXABOIFlA30DMBAAZNhCGUoM0BBQL7AcoYYhhSGfIZkhlKGXIZUhjyGEiA7hyGRoRgIoxkMGQwYCoBisQzVQLEiICsTLJ/KUMvABdRbClSVClSRCBTNBpLpQF40VDQPyAeZWQzWnQy0JQeIi4A6FRhUDa4arDT4bHDCYLXBS4M/OM2qBpsBckslkE6C6E0tiOfvkgj+TlBXLpAuYchA6MLr5hKGNAYLsFszgW4vAIuAfJEM0V9WNf1zsFWQarWawSKD10D3LzS4aXAY6IO8si/JSwNTg2YzcAEjwBA9uDEZYUZ6hgZ6hoFGyg4W0KjgYJBmUGLQAIa3OYMDgwdDAEMo0N5WhtUMWxi2MrEx6TAZM5lClDIxQvUIM6AAJjsATzeVPw==</latexit><latexit

Fig. 1. Proposed FFTNet with residual connections and additional
multiplications.
2. INVESTIGATIONS OF SINGLE GAUSSIAN NEURALVOCODERS WITH SIMPLE ACOUSTIC FEATURES
2.1. SG AR WaveNet and FFTNet neural vocoders with SAF
Given acoustic features h, AR WaveNet [8] and FFTNet [3] neuralvocoders model conditional probability distribution p(x|h) of raw
audio waveform x = [x1 , · · · , xT ] asp(x|h) =TY

t=1

p(xt |x<t , θ; h),(1)where θ are the parameters of the model. In AR WaveNet, Eq. (1) ismodeled by a stack of dilated causal convolution layers, allowing theefficient input of very long audio samples with relatively few layers.
However, the network model size of WaveNet vocoders is still toolarge to synthesize speech waveforms in real time. In contrast, tosignificantly reduce the network model size, FFTNet uses simple
1 × 1 convolution layers instead of the dilated causal convolution
layers, and can therefore synthesize speech waveforms in real timewith a fast generation algorithm [25].
Although the vanilla WaveNet and FFTNet models introduce
a categorical distribution of the next sample xt based on µ-law
companding algorithm defined in G.711 [26], the MoL distribution
has been employed in parallel WaveNet [20] and several WaveNet
vocoders [11, 13] including Tacotron 2 [7] for predicting 16 bit
law audio to improve synthesized speech quality. In these neuralvocoders, 10-component MoL is used and the output consists of 30
channels. In parallel WaveNet, the MoL distribution is also used
for teacher–student knowledge distillation-based training using the
IAF [21]. However, a parallel WaveNet employs a Monte Carlo
method to approximate the intractable KL divergence between thestudent logistic and teacher MoL distributions. Therefore, a doubleloop sampling is required for the student input and estimation of theintractable KL divergence. [22].
To solve this problem in MoL-based parallel WaveNet, an SG
output distribution is provided. The conditional distribution ofp(xt |x<t , θ; h) in Eq. (1) is defined as:p(xt |x<t , θ; h) = N (µ(x<t ; θ), σ(x<t ; θ)),(2)where µ(x<t ; θ) and σ(x<t ; θ) are the mean and standard deviation
predicted by the AR WaveNet, respectively. Network parameters θ
are trained using maximum likelihood estimation [22]. Comparedwith MoL distribution, the SG AR WaveNet can synthesize speech
waveforms with higher quality when using mel-spectrogram input
although only two output channels are used. In addition, the SGmodeling can efficiently train parallel WaveNet because the KL divergence between the two Gaussian distributions of the teacher andstudent models can be analytically calculated without sampling [22].
The SG modeling has only been applied to AR and parallel WaveNet vocoders with mel-spectrogram input for end-to-end

TTS [22]. Therefore, this paper investigates the effectiveness of the
SG modeling for these neural vocoders with SAF constructed fromthe fundamental frequency and mel-cepstra for directly applying theconventional SPSS and VC frameworks.
SG modeling can also be directly applied to an FFTNet vocoderwith SAF. Compared with the categorical FFTNet, which has 256
input and output channels, there are one input and two output channels in the SG WaveNet. Real-time synthesis is also realized by SG
FFTNet vocoder. In a previous work, the residual connections in alllayers were found to improve the FFTNet model while keeping the
network√ model size [19]. In this paper, additional multiplications
with 0.5 are introduced after the residual connections in all layers
to improve the stability of the network, as shown in Fig. 1 and theeffectiveness of MoL and SG FFTNet vocoders with SAF are also
investigated.
2.2. SG parallel WaveNet vocoder with SAF
The SG parallel WaveNet is based on the Gaussian IAF [21]. In
SG parallel WaveNet, white noise z (0) is first generated from theisotropic Gaussian distribution N (0, I) as the input of the non-AR
student WaveNet where I is a unit matrix. Using n stacked Gaussian
IAFs, where each flow is parameterized by a WaveNet, student input
z (0) is repeatedly transformed as z (0) → z (1) → · · · → z (n) .
For acoustic feature input, the upsampling layer trained in theteacher WaveNet is commonly used. Then, the student WaveNet
outputs a synthesized waveform xq and student Gaussian distribution q(xt |x<t , θ; h) with mean µq and standard deviation σq
where xq = z (n) = z (0) σ q + µq . To match student Gaussian distribution q(xt |x<t , θ; h) to teacher Gaussian distribution
p(xt |x<t , θ; h) provided by the SG AR teacher WaveNet, the KL
divergence between the two distributions is minimized. In contrast to
the intractable KL divergence in MoL-based parallel WaveNet [20],the KL divergence between two Gaussian distributions can be analytically calculated. In SG parallel WaveNet, the regularized KL
divergence loss is introduced as
KLreg (q||p) =λ| log σp − log σq |2+ log

σq2 − σp2 + (µp − µq )2
σp
+
,
σq
2σp2

(3)

where λ = 4 is used in [22].
However, only minimizing the regularized KL divergence loss
leads to whisper voices [20, 22]. To avoid the problem, the spectrogram frame loss between synthesized and ground truth waveforms
xq and x is additionally considered and total loss L is calculated as
L = KLreg (q||p) +1
B|STFT(xq )| − |STFT(x)|

2
2

(4)

in SG parallel WaveNet, where |STFT(x)| is the magnitude of theshort-term Fourier transform (STFT), B = 1025 is the number of
frequency bins, and the STFT size is 2048 with a 12.5 ms frame-shift
50 ms Hann window [22]. Using SG parallel WaveNet instead of SG
AR WaveNet enables speech waveforms to be directly generated by
the student SG WaveNet with white noise z (0) and acoustic features
h in real time.
As SG AR WaveNet vocoder, SG parallel WaveNet vocoder has
also only been investigated with mel-spectrogram input [22]. Therefore, the possibility of SG parallel WaveNet vocoder training andsynthesis with SAF instead of mel-spectrograms is also investigatedin this paper.

2.3. Impact of noise shaping on SG neural vocoders with SAF
The speech signals synthesized by neural vocoders often suffer fromnoise caused by prediction errors, and these noise signals tend tocause large spectral distortions in high-frequency bands. Thus, thenoise signals degrade the synthesized speech quality [16]. To reduce
the adverse effects of the noise signals generated by neural vocoders,predictive pulse code modulation (PPCM) [27]-based time-invariant
noise shaping, which is a perceptual weighting technique, has been
applied to categorical WaveNet and FFTNet vocoders [15, 16, 19].
This noise shaping should improve the synthesized speech qualityof SG AR WaveNet and FFTNet as well as SG parallel WaveNet
vocoders. Therefore, the impact of noise shaping on SG neuralvocoders with SAF is also investigated in this paper.
2.4. Efficacy of bandwith extention on SG neural vocoders withSAF
As described in Sec. 1, high quality speech waveforms at f s =24 kHz are synthesized by a MoL-based WaveNet vocoder using
80-band mel-spectrograms with a frequency band of 125–7600 Hz in
Tacotron 2 [7]. Therefore, SG WaveNet and FFTNet neural vocoders
are trained to synthesize speech waveforms at f s = 24 kHz by using
SAF for f s = 16 kHz to investigate the bandwidth extension effect.
3. EXPERIMENTS
3.1. Experimental conditions
To evaluate the impact of the SG modeling and noise shaping methods for AR WaveNet and FFTNet as well as parallel WaveNet neural vocoders with SAF at f s = 24 and 16 kHz, a series of objective and subjective experiments were conducted using a Japanesemale speech corpus recorded at f s = 48 kHz and downsampled to
24 kHz, as used in [17–19]. In the experiments, 5,697 utterances(about 3.7 h) were used as the training set and 20 utterances wereused as the test set.
In the experiments, acoustic features h were analyzed every
5 ms over a Hann window of length 25 ms. Fundamental frequency
fo , analyzed by an NDF algorithm [28], was used in all the vocoders
with SAF, as in [18, 19]. For the neural vocoders with SAF, the0–th to 34–th mel-cepstral coefficients (35 dimensions) were analyzed from a simple STFT of windowed speech waveforms at f s =24 kHz with warping coefficient α = 0.46. In addition, the 0–th to24–th mel-cepstral coefficients (25 dimensions) were also analyzed
from speech waveforms at f s = 16 kHz with warping coefficient
α = 0.42 to investigate the possibility of bandwidth extension. In
the neural vocoders with SAF for f s = 24 kHz, (1 + 1 + 35 =)37-dimensional vectors constructed from continuous logarithmic fo ,voice/unvoice one-hot vector, and mel-cepstrum coefficients (normalized to have a zero-mean and unit-variance) were used as acoustic features h. For the acoustic features for f s = 16 kHz, (1 + 1 +25 =) 27-dimensional vectors were employed.
The SG WaveNet with mel-spectrogram input is used as a reference. In this method, 80-dimensional log-mel-spectrograms wereanalyzed every 12.5 ms over a Hann window of length 85.3 ms witha frequency band 125 to 7,600 Hz and normalized to the range of [0,1], as in [7]. Similar in [19], transposed convolution [1] was applied
for upsampling the acoustic features in all the neural vocoders andthe upsampling layer was also trained with neural vocoder models.
To directly compare the results with those of the original SG
WaveNet vocoders [22], the same network parameters as those of

the original SG WaveNet vocoders were employed. Both the residual and skip channels of AR and parallel WaveNet vocoders were set
to 128. Twenty layers (10 dilations × 2 cycles) with a kernel size of
two were used for the dilated causal convolution layers, giving a receptive field of 2,047 samples for the AR teacher WaveNet vocoder.
In the parallel WaveNet vocoder, 60 layers, where each Gaussian
IAF is parameterized by a 10-layer WaveNet with a kernel size of
three, were used, as in [22].
In the FFTNet vocoders, L = 11 layers were introduced andthe receptive field was 211 = 2, 048 samples, as used in the original
FFTNet [3]. The channel number of each FFTNet layer was 256 [3].
In FFTNet, 10-component MoL models were also investigated.
The AR WaveNet, AR FFTNet, and parallel WaveNet vocoders
required 3,000,000, 3,000,000, and 1,500,000 parameter updates, respectively. In addition, an Adam optimization algorithm [29] updated the neural network parameters with learning rates of 0.0002,
0.001, and 0.0002, respectively. As in MoL-based parallel WaveNet
and Tacotron 2, exponential moving averaging [30] with a decay rateof 0.999 was used for the parameters. The minibatch sizes of allneural vocoders were 2 × 5,000 samples. They were trained using asingle GPU of an NVIDIA Tesla P100.
Similar to previous approaches [15, 16, 19], a mel-generalized
cepstrum [23]-based noise shaping filtering implemented by the mellog spectrum approximation (MLSA) filter [24] was introduced. A
parameter to control noise energy in the formant regions was set to
0.5 for noise shaping according to the results of the WaveNet and
FFTNet vocoder investigations [15, 16, 19]. Noise shaping was notapplied to the SG AR WaveNet with mel-spectrograms. The results of preliminary experiments showed that the speech qualities
synthesized by MoL and SG FFTNet without residual connections
and noise shaping were obviously lower than those of the others.
Hence, they were omitted after the objective and subjective evaluations. As a result, nine models were evaluated, as described inTable. 1 and Fig. 2. Although SG parallel WaveNet with noise shaping was also investigated, it was difficult to train to the same quality
as obtained without noise shaping because the loss score cannot be
reduced. Therefore, determining a method to effectively train SG
parallel WaveNet with noise shaping is a future task.
3.2. Objective evaluations
To objectively evaluate the synthesized test set speech waveforms,
the signal-to-noise ratio (SNR) and spectral distortion (SD) between
original waveform x and synthesized x̂ were computed. As in previous studies [8, 16, 18], a linear phase compensation for each frame
was introduced to calculate the SNR. For acoustic feature analysis,
the STFT analysis window function was also a Hann window witha frame length of 25 ms and frameshift of 5 ms. To consider thehuman auditory perception criterion in the objective evaluation, themel-cepstral distortion (MCD) was also computed with weighting
factor α = 0.46. The results of the objective evaluations are presented in Table 1.
3.3. Subjective evaluations
To subjectively evaluate the speech waveforms synthesized by ninemodels, mean opinion score (MOS) tests [31] were conducted. All
20 utterances of the test set were used as the evaluation set. These
were presented through headphones to 13 Japanese adult native
speakers without hearing loss (20 utterances × 9 conditions including the original test set waveforms = 180 utterances). The MOS
results are plotted in Fig. 2.

Table 1. Results of objective evaluations of 20 test set utterances. “NS” and “Parallel WN” denote noise shaping and parallel WaveNet,
respectively.
MethodNetwork
Type
Input features
Real-time NS
SNR [dB]SD [dB]MCD [dB](a):WN-SG-MSPC
WaveNetSG
Mel-spectrogram
3.30 ± 0.39 9.34 ± 0.20 3.71 ± 0.12(b):WN-SG-SAF
WaveNetSG
SAF 24 kHz
5.40 ± 0.44 8.02 ± 0.08 2.55 ± 0.07(c):WN-SG-SAF-NS
WaveNetSG
SAF 24 kHz
X
3.90 ± 0.73 7.57 ± 0.08 2.20 ± 0.04(d):WN-SG-SAF16k-NS
WaveNetSG
SAF 16 kHz
X
3.70 ± 0.69 8.26 ± 0.07 2.89 ± 0.05(e):PWN-SG-SAF
Parallel WN
SG
SAF 24 kHz
X
5.20 ± 0.41 8.09 ± 0.06 2.73 ± 0.07(f):FN-MoL-SAF-NS
FFTNetMoL
SAF 24 kHz
X
X
3.20 ± 0.66 7.96 ± 0.08 2.69 ± 0.05(g):FN-SG-SAF-NS
FFTNetSG
SAF 24 kHz
X
X
2.90 ± 0.66 8.01 ± 0.08 2.80 ± 0.05(h):FN-SG-SAF16k-NS
FFTNetSG
SAF 16 kHz
X
X
3.10 ± 0.66 8.53 ± 0.07 3.36 ± 0.05(a):SG-MSPC
WaveNeRea meneu a vocode

o ms snc ude spec a d s o ons and he syn hes s qua y wasno so h gh The eason m gh a so be due o he ack o a n ng da as nce a eas 9 hou s o a n ng da a has been used n he o g na
pa a e WaveNe 20 To mp ove he syn hes s qua y o he SG
pa a e WaveNe vocode w h SAF a huge amoun o a n ng da amo e han 10 hou s m gh a so be equ ed

2.92<latexit sha1_base64="2c6biWERg4gKpI+JU0iCqPfISkA=">AAADrnichVHLbtNAFL2ueZTwaAobJDYWaVCiQjSOkChIlaoioUog1DaPVoqTyHYmiVWP7dhOSDuZH+AHWLACCSHEgo9gww900Q2sEcsisWHB9SOqQnmM5Zl7z73nzJkZw7OtICTkSJqTz5w9d37+QubipctXFrKLV+uBO/RNWjNd2/V3DT2gtuXQWmiFNt31fKozw6Y7xt7DqL4zon5guU413Pdok+k9x+paph4i1F6UNjSHPjNdxnSnwzXDEg21ybnGDHeMqWt3mB72laWcuiSEyOS1rq+bXBV8XWjrVo9pE2WihXQc8kr1UVUUIgllLNp8IIqTO8ofi8WJknJbvIytZdSNSweixQukKBQtsJiiRTubus2figK5rRbTpkS8sHrCcIpidYavuR03jBEU6jE9ZiwnABsKVIq1fZb4avFp6tOBEIXBZOJhz3Im8V4u3Y8cTnseP5kh9KaEdjZHSiQeyulATYMcpGPTzb4FDTrggglDYEDBgRBjG3QI8GuACgQ8xJrAEfMxsuI6BQEZ5A6xi2KHjugezj3MGinqYB5pBjHbxF1s/H1kKpAnh+QdOSafyHvylfz8qxaPNSIv+7gaCZd67YXn1ys//stiuIbQP2H903MIXViJvVro3YuR6BRmwh8dvDiuPNjO81vkNfmG/l+RI/IRT+CMvptvtuj2S8jgA6i/X/fpoF4uqRhv3c2traRPMQ834CYU8L7vwRpswCbUwJQ+SIfSZ+mLTOS63JTbSeuclHKuwcyQ+78AJYkDIA==</latexit><latexit sha1_base64="eg+kTlAm5bstgA9mntZPagpa5zw=">AAADtXichZHLbtNAFIaPay4lXJrCBomNRRrkqLQaR0itKlWqWgmQuKVN0laqk8h2JqlVj+3YTkg7mRfgBViwAgkhxJ4XYMMLsKjEFiTEskhsWHBiu4rSchnLnnP+Od8/xzOm79hhRMihNCGfOXvu/OSFzMVLl69MZaevboZeN7Bo1fIcL9g2jZA6tkurkR05dNsPqMFMh26Ze2vD9a0eDULbcyvRvk9rzGi7dsu2jAilxrT0RHfpU8tjzHCbXDdtsaPVONeZ6fUx9ZwmM6JdZSanzQghMnm9FRgW1wRfFfqq3Wb6QBnoEe1HvFy5WxHq0ELpiwbviMJgTvnjYmGgpGydF7G0iL7x0oGoc5UUhKKHNlP04c6W4fDHQiW3tUJalJiryyPCLYjlMV73ml4UK2jUZkZMzCYC6wp0ir0DlvRV58dpQDtCqJ3BwMea2UzSu2oUlsr35h6VS2tiRD54OMa1j7lGNkfmSTyU04GWBjlIR8nLvgEdmuCBBV1gQMGFCGMHDAjx2QE(b):SG

3.80<latexit sha1_base64="iyjx+IB6qQ4YjGElilJHqVTeSLk=">AAADrnichZHLbtNAFIaPGy4lXJrCBomNRRrkqBCNAxIRUqWqSKgSCLVNmlaKk8h2JolVj+3YTkg7mRfgBViwAgkhxIKHYMMLdNENrBHLIrFhwfGlikK5jGXPOf+c75/jGcOzrSAk5Eiay5w5e+78/IXsxUuXryzkFq/WA3fom3TbdG3X3zX0gNqWQ7dDK7TprudTnRk23TH2HkbrOyPqB5br1MJ9jzaZ3nOsrmXqIUrtRWldc+gz02VMdzpcMyzRUJuca8xwx5i6dofpYV9eyqtLQohsQev6uslVwdeEtmb1mDaRJ1pIxyGv1h7VhBJZyGPR5gNRnNyR/7hYnMgp2+JlLC2jb7x0IFpcIUUha4HFZC3a2dRt/lQo5LZaTIsSc2VlSjhFsTLDa27HDWMFjXpMj4nlRGBDgU6xt8+Svlr8JPXpQAhlMJl4WLOcTXq/W6oQMUUeP5kBeidAO5cnJRIP+XSgpkEe0rHh5t6CBh1wwYQhMKDgQIixDToE+DRABQIeak3gqPkYWfE6BQFZZIdYRbFCR3UPvz3MGqnqYB55BjFt4i42vj6SMhTIIXlHjskn8p58JT//6sVjj6iXfZyNhKVee+H59eqP/1IM5xD6U+qfPYfQhUrcq4W9e7ES/YWZ8KODF8fVB1sFfou8Jt+w/1fkiHzEP3BG3803m3TrJWTxAtTfj/t0UC+XVIw37+VXK+lVzMMNuAkKnvd9WIV12IBtMKUP0qH0WfqSIZl6pplpJ6VzUspcg5mR6f8CIUUDHg==</latexit><latexit sha1_base64="j2d1dpWqzWWB38D1Wt++5tEtdRw=">AAADsHichZHLbtNAFIaPGy4lXJrCBomNRRpkq1CNIySqSpWqIgEChNomaSPFSWQ7k9TUYzu2E9JO5gV4ARasQEIIseIZ2PACCHUHS8SySGxYcHypqlAuY9lzzj/n++d4xvQdO4wI2ZemcidOnjo9fSZ/9tz5CzOF2YuboTcILFqzPMcL6qYRUsd2aS2yI4fW/YAazHTolrlzO17fGtIgtD23Gu36tMmMnmt3bcuIUGrPSvd1lz6xPMYMt8N10xYNrcm5zkxvhKnndJgRbctzRW1OCJEv6d3AsLgm+KrQV+0e08fyWI/oKOKV6p2qUGILeSTavC/U8Q35j4vqWM7YFi9jaRl9k6U90eIKUYWshzaT9Xhny3D4I6GQ65qaFaXmyvIR4apieYLXvY4XJQoa9ZiREPOpwAYCnRLvgKV9tfhhGtC+EEp/PPaxZj6f9q6Y6lLlrjiCHjycQHqHSLtQJAskGfLxQMuCImRjzSu8Bh064IEFA2BAwYUIYwcMCPFpgAYEfNSawFELMLKTdQoC8sgOsIpihYHqDn57mDUy1cU89gwT2sJdHHwDJGUokY/kDTkgH8hb8pX8/KsXTzziXnZxNlOW+u2Zp5crP/5LMZwj2D6i/tlzBF1YTHq1sXc/UeK/sFJ+uPfsoLK0UeLXyEvyDft/QfbJe/wDd/jderVON55DHi9A+/24jweb5QUN4/WbxZXF7Cqm4QpcBQXP+xaswD1YgxpY0jvpk/RZ+pIr5+q5ds5IS6ekjLkEEyP3+BfZ9APw</latexit>c SG NS

4.79<latexit sha1_base64="8SCXTFuGBgEWqWo12bNHXvTqxe4=">AAADrnichZHLbtNAFIaPGy4lXJrCBomNRRrkqBCNo0otSJWqIqFKINQ2l1aKk8h2JolVj+3YTkk7nhfgBViwAgkhxIKHYMMLdNENrBHLIrFhwYntKgrlMpY95/xzvn+OZwzPtoKQkGNpJnPu/IWLs5eyl69cvTaXm79eD9yhb9Ka6dquv2voAbUth9ZCK7TprudTnRk23TH2Ho7Xd/apH1iuUw0PPNpkes+xupaphyi156UNzaHPTJcx3elwzbBEQ21yrjHDHWHq2h2mh315Ia8uCCGyBa3r6yZXBV8X2rrVY1okR1pIRyGvVB9VhTK2kEeizQeiGN2T/7hYjOSUbfEylpbRN146FC2ukKKQtcBisjbe2dRt/lQo5K5aTIsSc2V1QjhFsTrFa27HDWMFjXpMj4nFRGBDgU6xt8+Svlr8NPXpQAhlEEUe1ixmk96XSsv3xQR5/GQK6J0C7VyelEg85LOBmgZ5SMemm3sLGnTABROGwICCAyHGNugQ4NMAFQh4qDWBo+ZjZMXrFARkkR1iFcUKHdU9/PYwa6Sqg/nYM4hpE3ex8fWRlKFAjsg7ckI+kffkK/n5Vy8ee4x7OcDZSFjqteee36z8+C/FcA6hP6H+2XMIXViJe7Wwdy9Wxn9hJvz+4YuTyoPtAr9DXpNv2P8rckw+4h84+9/NN1t0+yVk8QLU34/7bFAvl1SMt5byayvpVczCLbgNCp73MqzBBmxCDUzpg3QkfZa+ZEimnmlm2knpjJQyN2BqZPq/ADSCAyc=</latexit>d SG NS 16k

4.60

e Pa a e SG

3.27

<latexit sha1_base64="30i7j51VU2kryB4DsXfjpVkHJec=">AAADrnichZHLbtNAFIaPay4lXJrCBomNRRrkqBCNowoqpEpVkVAlEGqbSyvVSWQ7k8Sqx3ZsJ6SdzAvwAixYgYQQYsFDsOEFuugG1ohlkdiw4MR2FYVCGcuec/453z/HM6bv2GFEyJE0I587f+Hi7KXM5StXr81l56/XQq8fWLRqeY4X7JhGSB3bpdXIjhy64wfUYKZDt829R+P17QENQttzK9G+T+vM6Lh227aMCKXmvLSuu/S55TFmuC2um7bY1eqc68z0hph6TosZUVdZyGkLQohMXm8HhsU1wdeEvmZ3mD5SRnpEhxEvVx5XhDq2UIaiyXuiMLqn/HWxMFJStsFLWFpC33jpQDS4SgpC0UObKfp4Z8tw+DOhkrtaIS1KzNWVCeEWxMoUr3stL4oVNOowIyYWE4H1BTrF3gFL+mrwkzSgPSHU3mjkY81iJul9qXifiAny5OkU0DkBmtkcKZJ4KKcDLQ1ykI4NL/sOdGiBBxb0gQEFFyKMHTAgxGcXNCDgo1YHjlqAkR2vUxCQQbaPVRQrDFT38NvBbDdVXczHnmFMW7iLg2+ApAJ5ckjek2PymXwg38ivf3rx2GPcyz7OZsJSvzn34mb5538phnME3Ql1Zs8RtGE57tXG3v1YGf+FlfCDg5fH5YdbeX6HvCHfsf/X5Ih8wj9wBz+st5t06xVk8AK0P4/7dFArFTWMN5dyq8vpVczCLbgNKp73A1iFddiAKljSR+lQ+iJ9lYlck+tyMymdkVLmBkwNufsbHyMDHQ==</latexit><latexit sha1_base64="B/Cqv+pueh3ag/bRXHD7UXYKOfo=">AAADrnichZHLbtNAFIaPGy4lXJrCBomNRRrkqBCNA1IrpEpVkVAlEGqbSyvFSWQ7k8Sqx3Zsp6QdzwvwAixYgYQQYsFDsOEFuugG1ohlkdiw4MR2FYVCGcuec/453z/HM4ZnW0FIyJE0kzl3/sLF2UvZy1euXpvLzV+vB+7QN2nNdG3X3zH0gNqWQ2uhFdp0x/Opzgybbhu7j8br23vUDyzXqYb7Hm0yvedYXcvUQ5Ta89K65tDnpsuY7nS4ZliioTY515jhjjB17Q7Tw768kFcXhBDZgtb1dZOrgq8Jbc3qMS2SIy2ko5BXqo+rQhlbyCPR5gNRjO7Jf10sRnLKtngZS8voGy8diBZXSFHIWmAxWRvvbOo2fyYUclctpkWJubIyIZyiWJniNbfjhrGCRj2mx8RiIrChQKfY22dJXy1+kvp0IIQyiCIPaxazSe/3S+UlMUGePJ0CeidAO5cnJRIP+XSgpkEe0rHh5t6BBh1wwYQhMKDgQIixDToE+DRABQIeak3gqPkYWfE6BQFZZIdYRbFCR3UXvz3MGqnqYD72DGLaxF1sfH0kZSiQQ/KeHJPP5AP5Rn7904vHHuNe9nE2EpZ67bkXNys//0sxnEPoT6gzew6hC8txrxb27sXK+C/MhN87eHlcebhV4HfIG/Id+39Njsgn/ANn74f5dpNuvYIsXoD653GfDurlkorx5oP86nJ6FbNwC26Dgue9BKuwDhtQA1P6KB1KX6SvGZKpZ5qZdlI6I6XMDZgamf5vI2IDHw==</latexit>

4.32

4 3 Impac o no se shap ng on SG AR neura vocoders

4.22

h SG NS 16k

4.18

O g na4.86

F g 2 Resu s o MOS es w h 13 s en ng sub ec s “WN” “FN”
and “SAF” a e om ed as defined n Tab e 1

F om he esu s o b c
and g no se shap ng can s g
n fican y mp ove he SG WaveNe MoL and SG FFTNe neu a
vocode s o su fic en syn hes s qua y ove MOS va ues o 4 0 com
pa ed w h he SG WaveNe w hou no se shap ng when us ng no soa ge amoun o a n ng da a The esu s espec a y o FFTNe a e
mpo an s nce FFTNe can syn hes ze speech wave o ms n ea
me and ca ego ca FFTNe vocode s w h no se shap ng cannoeach su fic en syn hes s qua y n a p ev ous nves ga on 19

4 DISCUSSIONS

4 4 E ficacy o bandw d h ex ens on on SG AR WaveNe and
FFTNe vocoders w h SAF

FFTNe

MoL NSg SG NS<latexit sha1_base64="CMoM3WhVGzhhfCBGNmjJg0qWSUs=">AAADrnichVHLbtNAFL2ueZTwaAobJDYWaVCiQjQOlaiQKlVFQpVAqG0erRQnke1MEqse27GdkHYyP8APsGAFEkKIBR/Bhh/oohtYI5ZFYsOC60dVhUIZyzP3nnvPmTMzhmdbQUjIoTQjnzt/4eLspczlK1evzWXnr9cDd+ibtGa6tuvvGHpAbcuhtdAKbbrj+VRnhk23jd1HUX17RP3Acp1quOfRJtN7jtW1TD1EqD0vrWsOfW66jOlOh2uGJRpqk3ONGe4YU9fuMD3sKws5dUEIkclrXV83uSr4mtDWrB7TJspEC+k45JXq46ooRBLKWLT5QBQn95S/FosTJeW2eBlby6gbl/ZFixdIUShaYDFFi3Y2dZs/EwVyVy2mTYl4YeWE4RTFyhRfcztuGCMo1GN6zFhMADYUqBRr+yzx1eLHqU8HQhQGk4mHPYuZxPtS6X7k8LjnydMpQu+Y0M7mSInEQzkdqGmQg3RsuNl3oEEHXDBhCAwoOBBibIMOAX4NUIGAh1gTOGI+RlZcpyAgg9whdlHs0BHdxbmHWSNFHcwjzSBmm7iLjb+PTAXy5IC8J0fkM/lAvpFf/9TisUbkZQ9XI+FSrz334mbl539ZDNcQ+iesMz2H0IXl2KuF3r0YiU5hJvzR/sujysOtPL9D3pDv6P81OSSf8ATO6If5dpNuvYIMPoD653WfDurlkorx5lJudTl9ilm4BbehgPf9AFZhHTagBqb0UTqQvkhfZSLX5abcTlpnpJRzA6aG3P8NHP0DHA==</latexit><latexit sha1_base64="ZTxINqlkyBkvAeQP0qLld06J+pw=">AAADrnichVHLbtNAFL2ueZTwaAobJDYWaVCiQjSOKlEhVaqKhCqBUNs8WilOItuZJFY9tmM7Ie1kfoAfYMEKJIQQCz6CDT/QRTewRiyLxIYF14+oCuUxlmfuPfeeM2dmDM+2gpCQY2lOPnf+wsX5S5nLV65eW8guXq8H7tA3ac10bdffM/SA2pZDa6EV2nTP86nODJvuGvsPo/ruiPqB5TrV8MCjTab3HKtrmXqIUHtR2tQc+sx0GdOdDtcMSzTUJucaM9wxpq7dYXrYV5Zy6pIQIpPXur5uclXwDaFtWD2mTZSJFtJxyCvVR1VRiCSUsWjzgShO7il/LBYnSspt8TK2llE3Lh2KFi+QolC0wGKKFu1s6jZ/KgrkrlpMmxLxwtopwymKtRm+5nbcMEZQqMf0mLGcAGwoUCnW9lniq8WnqU8HQhQGk4mHPcuZxPtKqRw5nPY8fjJD6E0J7WyOlEg8lLOBmgY5SMeWm30LGnTABROGwICCAyHGNugQ4NcAFQh4iDWBI+ZjZMV1CgIyyB1iF8UOHdF9nHuYNVLUwTzSDGK2ibvY+PvIVCBPjsg7ckI+kffkK/n5Vy0ea0ReDnA1Ei712gvPb1Z+/JfFcA2hf8r6p+cQurAae7XQuxcj0SnMhD86fHFSebCT53fIa/IN/b8ix+QjnsAZfTffbNOdl5DBB1B/v+6zQb1cUjHeXsmtr6ZPMQ+34DYU8L7vwzpswhbUwJQ+SEfSZ+mLTOS63JTbSeuclHJuwMyQ+78AGtkDGw==</latexit><latexit sha1_base64="ZwKFRT6WalvY8oouFshT/y8qBkQ=">AAADrnichZHLbtNAFIaPGy4lXJrCBomNRRrkqBCNo0pESJWqIqFKINQ2l1aKk8h2JolVj+3aTkg7mRfgBViwAgkhxIKHYMMLdNENrBHLIrFhwfGlikK5jGXPOf+c75/jGcOzrSAk5Fiay5w7f+Hi/KXs5StXry3kFq83Anfom7Ruurbr7xp6QG3LofXQCm266/lUZ4ZNd4y9h9H6zoj6geU6tfDAoy2m9x2rZ5l6iFJnUdrQHPrMdBnTnS7XDEs01RbnGjPcMaau3WV6OJCX8uqSECJb0Hq+bnJV8HWhrVt9pk3kiRbSccirtUc1oUQW8lh0+L4oTu7Jf1wsTuSUbfMylpbRN146FG2ukKKQtcBishbtbOo2fyoUclctpkWJubI6JZyiWJ3hNbfrhrGCRn2mx8RyIrChQKfY22dJX21+mvp0XwhlfzLxsGY5m/S+UlIrYoo8fjID9E+BTi5PSiQe8tlATYM8pGPTzb0FDbrggglDYEDBgRBjG3QI8GmCCgQ81FrAUfMxsuJ1CgKyyA6ximKFjuoefvuYNVPVwTzyDGLaxF1sfH0kZSiQI/KOnJBP5D35Sn7+1YvHHlEvBzgbCUu9zsLzm9Uf/6UYziEMptQ/ew6hB5W4Vwt792Il+gsz4UeHL06qD7YL/A55Tb5h/6/IMfmIf+CMvptvtuj2S8jiBai/H/fZoFEuqRhvreTXKulVzMMtuA0Knvd9WIMN2IQ6mNIH6Uj6LH3JkEwj08p0ktI5KWVuwMzIDH4BJYcDIA==</latexit><latexit sha1_base64="spA8EJ0HMN4JgDRCSokX/D14xIk=">AAADrnichZHLbtNAFIaPay4lXJrCBomNRRqUqBCNowoipEpVkVAlEGqbSyvFSWQ7k8Sqx3ZsJ6SdzAvwAixYgYQQYsFDsOEFuugG1ohlkdiw4MR2FYVCGcuec/453z/HM4ZnW0FIyJE0J587f+Hi/KXU5StXry2kF6/XAnfgm7Rqurbr7xp6QG3LodXQCm266/lUZ4ZNd4y9R5P1nSH1A8t1KuG+RxtM7zpWxzL1EKXWorShOfS56TKmO22uGZaoqw3ONWa4I0xdu830sKcsZdQlIUQqq3V83eSq4OtCW7e6TBsrYy2ko5CXK48rIjexUEaixfsiP76n/HUxP1YStsmLWFpE32jpQDR5juSFogUWU7TJzqZu82ciR+6q+aQoNs+tTgknL1ZneM1tu2GkoFGX6RGxHAtsINAp8vZZ3FeTn6Q+7QuR64/HHtYsp+LeVwql+2KKPHk6A3RPgFY6QwokGsrpQE2CDCRj002/Aw3a4IIJA2BAwYEQYxt0CPCpgwoEPNQawFHzMbKidQoCUsgOsIpihY7qHn67mNUT1cF84hlEtIm72Pj6SCqQJYfkPTkmn8kH8o38+qcXjzwmvezjbMQs9VoLL26Wf/6XYjiH0JtSZ/YcQgdKUa8W9u5FyuQvzJgfHrw8Lj/czvI75A35jv2/JkfkE/6BM/xhvt2i268ghReg/nncp4NasaBivLWSWSslVzEPt+A25PC8H8AabMAmVMGUPkqH0hfpq0zkmtyQW3HpnJQwN2BmyL3fMD0DJQ==</latexit>

1

2
3
4
Mean opinion score<latexit sha1_base64="A8wgxxMU4sYuvJgM+Tfl3LldnCo=">AAADq3ichVFda9NQGH6z+DHrxzq9EbwJdpWWuXFSBIcwGBNFUGRb223YtCVJT7uwnCRN0trt5PwB/4AXXimIiP4Lb7wWvBj6B8TLCd544ZuPMuqmnpCc933O8zznyTmGZ1tBSMiBNCWfOn3m7PS53PkLFy/N5GcvbwbuwDdp3XRt19829IDalkProRXadNvzqc4Mm24Zu3fj9a0h9QPLdWrhnkebTO85Vtcy9RCh9qx0T3PoU9NlTHc6XDMs0VCbnGvMcEfYunaH6eGOMldQ54QQuaLW9XWTq4KvCm3V6jEtUiItpKOQV2v3a6IUWygj0eZ9UY4WlBMXy5GSaVu8gtQK+iZL+6LFS6QsFC2wmKLFO5u6zR+LErmpljNSal5aPlI4ZbE8odfcjhsmCBr1mJ4o5lOADQQ6Jd4+S3O1+Lj1aV+IUj+KPOTM59Lscbwx4eGjCXZvzG7nC2SRJEM5XqhZUYBsrLn5N6BBB1wwYQAMKDgQYm2DDgE+DVCBgIdYEzhiPlZWsk5BQA61A2RRZOiI7uK3h10jQx3sY88gUZu4i42vj0oFiuQzeUsOyUfyjnwjv/7qxROPOMsezkaqpV575tnV6s//qhjOIewcqf6ZOYQuLCVZLczuJUj8F2aqH+4/P6ze2SjyG+QV+Y75X5ID8gH/wBn+MF+v040XkMMLUP887uPFZmVRxXr9VmFlKbuKabgG16GE530bVuABrEEdTOm99En6In2VF+Sq/ETWUuqUlGmuwMSQ6W8tCgJp</latexit><latexit<latexit sha1_base64="aSSfvD3n13ROZIJucHpsP9TMZM8=">AAADq3ichVHLbtNAFL1ueJTwaAobJDYWaZCj0mockKiQKlVFICQQapukrYiTyHYmqVWP7dhOSDueH+AHWLACCSEEf8GGNRKLCn4AsSwSGxZcP6oolMdYnrn33HPO3JkxPNsKQkIOpKnciZOnTk+fyZ89d/7CTGH24mbgDnyT1k3Xdv1tQw+obTm0HlqhTbc9n+rMsOmWsXsnrm8NqR9YrlML9zzaZHrPsbqWqYcItWelu5pDn5guY7rT4ZphiYba5FxjhjvC1LU7TA935LmiOieEyJe0rq+bXBV8VWirVo9pkRxpIR2FvFq7VxNKbCGPRJv3RTlakP9YLEdypm3xClIr6JuU9kWLK6QsZC2wmKzFO5u6zR8JhVxXyxkpNVeWxwqnLJYn9JrbccMEQaMe0xPFfAqwgUCnxNtnaV8tfpT6tC+E0o8iDznz+bT3G2LMf/Bwgt07YrcLRbJIkiEfD9QsKEI21tzCa9CgAy6YMAAGFBwIMbZBhwC/BqhAwEOsCRwxHyMrqVMQkEftAFkUGTqiuzj3MGtkqIN57BkkahN3sfH3USlDiXwib8gh+UDekq/k51+9eOIR97KHq5FqqdeeeXq5+uO/KoZrCDtj1T97DqELS0mvFvbuJUh8CjPVD/efHVZvb5T4NfKSfMP+X5AD8h5P4Ay/m6/W6cZzyOMDqL9f9/Fgs7KoYrx+s7iylD3FNFyBq6Dgfd+CFbgPa1AHU3onfZQ+S19yC7lq7nFOS6lTUqa5BBMjR38BLy0Cag==</latexit><latexit<latexit sha1_base64="RjMHC4gSjFJ1UD64jld2R0Wh2Qo=">AAADq3ichVHLbtNAFL2ueZTwaAobJDYWaZCj0mocVaJCqlQVgZBAqG2StiJOItuZpFY9tmM7Ie14foAfYMEKJIQQ/AUb1kgsKvgBxLJIbFhwY7uKQnmM5Zl7zz3nzJ0Z03fsMCLkUJqST50+c3b6XO78hYuXZvKzl7dCrx9YtGZ5jhfsmEZIHdultciOHLrjB9RgpkO3zb07o/r2gAah7bnVaN+nDWZ0XbtjW0aEUGtWuqu79InlMWa4ba6btqhrDc51ZnpDTD2nzYxoV5kraHNCiFxR7wSGxTXB14S+ZneZHiuxHtFhxCvVe1WhjiyUoWjxnijFC8ofi6VYybRNXkZqGX2T0oFocpWUhKKHNlP00c6W4fBHQiU3tVJGSs3VlbHCLYmVCb3utb0oQdCoy4xEMZ8CrC/QKfEOWNpXkx+nAe0Jofbi2EfOfC7tfUmM+Q8eTrC7x+xWvkAWSTKUk4GWBQXIxrqXfw06tMEDC/rAgIILEcYOGBDiVwcNCPiINYAjFmBkJ3UKAnKo7SOLIsNAdA/nLmb1DHUxH3mGidrCXRz8A1QqUCSfyBtyRD6Qt+Qr+flXL554jHrZx9VMtdRvzTy9WvnxXxXDNYLdseqfPUfQgeWkVxt79xNkdAor1Q8Onh1Vbm8W+Q3yknzD/l+QQ/IeT+AOvluvNujmc8jhA2i/X/fJYKu8qGG8sVRYXc6eYhquwXVQ8b5vwSrch3WogSW9kz5Kn6Uv8oJckR/LekqdkjLNFZgYMv0FMVACaw==</latexit><latexit

5<latexit sha1_base64="1IUVcuZW32jBoBTVFm1lo5hQHp4=">AAADq3ichVHLbtNAFL1ueJTwaAobJDYWaZCj0mocgaiQKlVFICQQapukrYiTyHYmqVWP7dhOSDueH+AHWLACCSEEf8GGNRKLCn4AsSwSGxZcP6oolMdYnrn33HPO3JkxPNsKQkIOpKnciZOnTk+fyZ89d/7CTGH24mbgDnyT1k3Xdv1tQw+obTm0HlqhTbc9n+rMsOmWsXsnrm8NqR9YrlML9zzaZHrPsbqWqYcItWelu5pDn5guY7rT4ZphiYba5FxjhjvC1LU7TA935LmiOieEyJe0rq+bXBV8VWirVo9pkRxpIR2FvFq7VxNKbCGPRJv3RTlakP9YLEdypm3xClIr6JuU9kWLK6QsZC2wmKzFO5u6zR8JhVxXyxkpNVeWxwqnLJYn9JrbccMEQaMe0xPFfAqwgUCnxNtnaV8tfpT6tC+E0o8iDznz+bT3m2LMf/Bwgt07YrcLRbJIkiEfD9QsKEI21tzCa9CgAy6YMAAGFBwIMbZBhwC/BqhAwEOsCRwxHyMrqVMQkEftAFkUGTqiuzj3MGtkqIN57BkkahN3sfH3USlDiXwib8gh+UDekq/k51+9eOIR97KHq5FqqdeeeXq5+uO/KoZrCDtj1T97DqELS0mvFvbuJUh8CjPVD/efHVZvb5T4NfKSfMP+X5AD8h5P4Ay/m6/W6cZzyOMDqL9f9/Fgs7KoYrx+o7iylD3FNFyBq6Dgfd+CFbgPa1AHU3onfZQ+S19yC7lq7nFOS6lTUqa5BBMjR38BM3MCbA==</latexit><latexit<latexit sha1_base64="VLteJgqmf+VhX8IyWxbcYux9C7k=">AAACd3ichVHLSsNAFD2N7/qqdSO4sFiUrsqNG4urghs3QrW2FbSUJE51ME1Ckha1+AP+gAtBUBAVP8ONP+CinyAuFURw4W0aEC3qDZk598w9d87M6I4pPZ+oFVF6evv6BwaHosMjo2PjsYl40bPrriEKhm3a7qauecKUlij40jfFpuMKraaboqTvL7fXSw3hetK2NvxDR5Rr2q4lq9LQfKYqsfiq0KyE7UiL04Rn2K6oxJKUpiAS3UANQRJh5OzYNbaxAxsG6qhBwILP2IQGj78tqCA4zJXRZM5lJIN1gWNEWVvnKsEVGrP7PO5ythWyFuftnl6gNngXk3+XlQnM0SPd0gs90B090cevvZpBj7aXQ571jlY4lfGTqfzbv6oazz72vlR/evZRRSbwKtm7EzDtUxgdfePo9CW/tD7XnKdLemb/F9Siez6B1Xg1rtbE+hmi/ADqz+vuBsWFtEppdW0hmc2ETzGIacwixfe9iCxWkEOB9z3AOa5xE3lXZpR5JdUpVSKhZhLfQlE/AfGmkSg=</latexit>

4 1 E ec veness o SG AR WaveNe vocoder w h SAF
F om he esu s o a and b he syn hes s qua y o he SG AR
WaveNe vocode w h me spec og ams was nsu fic en and owe
han ha w h SAF Th s m gh be due o he ack o a n ng da aon y 3 7 hou s s nce o g na SG AR WaveNe vocode s w h me
spec og ams have been a ned us ng abou 20 hou s o a n ng
da a 22 The o he eason m gh be due o he ack o ne wo k
pa ame e s s nce Taco on 2 7 and a WaveNe based VC w h me
spec og ams 14 have used a ge numbe s o ne wo k pa ame e s
han hose o hese nves ga ons and o g na SG AR WaveNe 22
These esu s nd ca e ha SG AR WaveNe w h SAF m gh be eas
y a ned compa ed w h ha w h me spec og ams when us ng
no so a ge amoun o a n ng da a s nce SAF can m gh mo e s m
p y ep esen speech n o ma on han me spec og ams a hough
he undamen a equency es ma on s equ ed Howeve he synhes s qua y o he SG AR WaveNe vocode w hou no se shap ng
us ng SAF was no so h gh a hough h ghes SNR can be ach eved
To confi m h s hypo hes s and o mp ove he syn hes s qua y ohe SG AR WaveNe vocode w h SAF u he nves ga ons w h
a a ge amoun o a n ng da a a e equ ed as u u e wo k
4 2 Poss b

y o SG para e WaveNe vocoder w h SAF

F om he esu s o e SG pa a e WaveNe can be success u ya ned om SAF w h easonab e SNR a hough syn hes zed wave

F om he esu s o d and h he SG WaveNe and FFTNe
vocode s w h SAF o f s = 16 kHz can sea ze su fic en syn
hes s qua y ove MOS va ues o 4 0 and he bandw d h ex ens on
e ec was success u y va da ed n SG neu a vocode s a houghh s e ec was no confi med n a p ev ous subband app oach 18
5 FUTURE WORK
To mp ove he syn hes s qua y o SG AR FFTNe and SG pa ae WaveNe neu a vocode s w h SAF u he nves ga ons w h
a a ge amoun o a n ng da a shou d be equ ed and hey shou dbe compa ed w h o he ea me neu a vocode s 4 32 33 as u
u e wo k Fu he mo e an e fic en ea n ng me hod o a n ng SG
pa a e WaveNe w h no se shap ng shou d a so be nves ga ed
6 CONCLUSIONSTh s pape nves ga ed me hods o mp ov ng ea me neu a
vocode s w h SAF The esu s o hese nves ga ons nd ca ed
ha ea me MoL and SG AR FFTNe vocode s w h no se shap ng
us ng SAF can ea ze su fic en syn hes s qua y w h bandw d h exens on e ec Mo eove a ea me SG pa a e WaveNe vocode
can a so be a ned om SAF These nves ga ons a e mpo an o
mp ov ng he ex s ng SPSS and VC sys ems based on sou ce fi e
vocode s w h SAF

7. REFERENCES[1] A. van den Oord, S. Dieleman, H. Zen, K. Simonyan,
O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and
K. Kavukcuoglu, “WaveNet: A generative model for raw audio,” arXiv:1609.03499, Sept. 2016, (unreviewed manuscript).
[2] S. Mehri, K. Kumar, I. Gulrajani, R. Kumar, S. Jain, J. Sotelo,A. Courville, and Y. Bengio, “SampleRNN: An unconditional
end-to-end neural audio generation model,” in Proc. ICLR,
Apr. 2017.
[3] Z. Jin, A. Finkelstein, G. J. Mysore, and J. Lu, “FFTNet: A
real-time speaker-dependent neural vocoder,” in Proc. ICASSP,
Apr. 2018, pp. 2251–2255.
[4] N. Kalchbrenner, E. Elsen, K. Simonyan, S. Noury,
N. Casagrande, E. Lockhart, F. Stimberg, A. van den Oord,
S. Dieleman, and K. Kavukcuoglu, “Efficient neural audio synthesis,” in Proc. ICML, July 2018, pp. 2415–2424.
[5] K. Tokuda, Y. Nankaku, T. Toda, H. Zen, J. Yamagishi, andK. Oura, “Speech synthesis based on hidden Markov models,”
Proc. IEEE, vol. 101, no. 5, pp. 1234–1252, May 2013.
[6] S. H. Mohammadi and A. Kain, “An overview of voice conversion systems,” Speech Commun., vol. 88, pp. 65–82, Apr.
2017.
[7] J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang,
Z. Chen, Y. Zhang, Y. Wang, RJ Skerry-Ryan, R. A. Saurous,
Y. Agiomyrgiannakis, and Y. Wu, “Natural TTS synthesis byconditioning WaveNet on mel spectrogram predictions,” inProc. ICASSP, Apr. 2018, pp. 4779–4783.
[8] A. Tamamori, T. Hayashi, K. Kobayashi, K. Takeda, andT. Toda, “Speaker-dependent WaveNet vocoder,” in Proc. Interspeech, Aug. 2017, pp. 1118–1122.
[9] Y. Ai, H.-C. Wu, and Z.-H. Ling, “SampleRNN-based neuralvocoder for statistical parametric speech synthesis,” in Proc.
ICASSP, Apr. 2018, pp. 5659–5663.
[10] X. Wang, J. Lorenzo-Trueba, S. Takaki, L. Juvela, and J. Yamagishi, “A comparison of recent waveform generation andacoustic modeling methods for neural-network-based speechsynthesis,” in Proc. ICASSP, Apr. 2018, pp. 4804–4808.
[11] L. Juvela, V. Tsiaras, B. Bollepalli, M. Airaksinen, J. Yamagishi, and P. Alku, “Speaker-independent raw waveform modelfor glottal excitation,” in Proc. Interspeech, Sept. 2018, pp.
2012–2016.
[12] K. Kobayashi, A. Tamamori, T. Hayashi, and T. Toda, “Statistical voice conversion with WaveNet-based waveform generation,” in Proc. Interspeech, Aug. 2017, pp. 1138–1142.
[13] J. Niwa, T. Yoshimura, K. Hashimoto, K. Oura, Y. Nankaku,
and K. Tokuda,“Statistical voice conversion based on
WaveNet,” in Proc. ICASSP, Apr. 2018, pp. 5289–5293.
[14] K. Chen, B. Chen, J. Lai, and K. Yu, “High-quality voice conversion using spectrogram-based WaveNet vocoder,” in Proc.
Interspeech, Sept. 2018, pp. 1993–1997.
[15] T. Hayashi, A. Tamamori, K. Kobayashi, K. Takeda, andT. Toda, “An investigation of multi-speaker training for
WaveNet vocoder,” in Proc. ASRU, Dec. 2017, pp. 712–718.
[16] K. Tachibana, T. Toda, Y. Shiga, and H. Kawai, “An investigation of noise shaping with perceptual weighting for WaveNetbased speech generation,” in Proc. ICASSP, Apr. 2018, pp.
5664–5668.

[17] T. Okamoto, K. Tachibana, T. Toda, Y. Shiga, and H. Kawai,“Subband WaveNet with overlapped single-sideband filterbanks,” in Proc. ASRU, Dec. 2017, pp. 698–704.
[18] T. Okamoto, K. Tachibana, T. Toda, Y. Shiga, and H. Kawai,“An investigation of subband WaveNet vocoder covering entire audible frequency range with limited acoustic features,” inProc. ICASSP, Apr. 2018, pp. 5654–5658.
[19] T. Okamoto, , T. Toda, Y. Shiga, and H. Kawai, “Improving
FFTNet vocoder with noise shaping and subband approaches,”in Proc. SLT, Dec. 2018, pp. 304–311.
[20] A. van den Oord, Y. Li, I. Babuschkin, K. Simonyan,
O. Vinyals, K. Kavukcuoglu, G. van den Driessche, E. Lockhart, L. C. Cobo, F. Stimberg, N. Casagrande, D. Grewe,
S. Noury, S. Dieleman, E. Elsen, N. Kalchbrenner, H. Zen,
A. Graves, H. King, T. Walters, D. Belov, and D. Hassabis,“Parallel WaveNet: Fast high-fidelity speech synthesis,” inProc. ICML, July 2018, pp. 3915–3923.
[21] D. P. Kingma, T. Salimans, R. Jozefowicz, X. Chen,
I. Sutskever, and M. Welling, “Improved variational inferencewith inverse autoregressive flow,” in Proc. NIPS, Dec. 2016,
pp. 4743–4751.
[22] W. Ping, K. Peng, and J. Chen, “ClariNet: Parallel wave generation in end-to-end text-to-speech,” in Proc. ICLR, May 2019.
[23] K. Tokuda, T. Kobayashi, T. Masuko, and S. Imai, “Melgeneralized cepstral analysis — A unified approach to speech
spectral estimation,” in Proc. ICSLP, Sept. 1994, pp. 1043–
1046.
[24] T. Fukada, K. Tokuda, T. Kobayashi, and S. Imai, “An adaptive algorithm for mel-cepstral analysis of speech,” in Proc.
ICASSP, Mar. 1992, vol. 1, pp. 137–140.
[25] P. Ramachandran, T. L. Paine, P. Khorrami, M. Babaeizadeh,
S. Chang, Y. Zhang, M. Hasegawa-Johnson, R. Campbell, andT. Huang, “Fast generation for convolutional autoregressivemodels,” in Proc. ICLR, Apr. 2017.
[26] ITU-T. Recommendation G. 711,
(PCM) of voice frequencies, 1988.

Pulse Code Modulation[27] B. S. Atal and M. R. Schroeder, “Predictive coding of speech
signals and subjective error criteria,” IEEE Trans. Acoust.
Speech, Signal Process., vol. 27, no. 3, pp. 247–254, June
1979.
[28] H. Kawahara, A. de Cheveigné, H. Banno, T. Takahashi, andT. Irino, “Nearly defect-free F0 trajectory extraction for expressive speech modifications based on STRAIGHT,” in Proc.
Interspeech, Sept. 2005, pp. 537–540.
[29] D. P. Kingma and J. L. Ba, “Adam: A method for stochastic
optimization,” in Proc. ICLR, May 2015.
[30] B. T. Polyak and A. B. Juditsky, “Acceleration of stochastic
approximation by averaging,” SIAM J. Control Optim., vol.
30, no. 4, pp. 838–855, July 1992.
[31] ITU-T Recommendation P. 800, Methods for subjective determination of transmission quality, 1996.
[32] R. Prenger, R. Valle, and B. Catanzaro, “WaveGlow: A
flow-based generative network for speech synthesis,” in Proc.
ICASSP, May 2019.
[33] X. Wang, S. Takaki, and J. Yamagishi, “Neural source-filterbased waveform model for statistical parametric speech synthesis,” in Proc. ICASSP, May 2019.