INTERSPEECH 2020
October 25–29, 2020, Shanghai, China

Quasi-Periodic Parallel WaveGAN Vocoder: A Non-autoregressive Pitchdependent Dilated Convolution Model for Parametric Speech Generation
Yi-Chiao Wu1 , Tomoki Hayashi1 , Takuma Okamoto2 , Hisashi Kawai2 , and Tomoki Toda121

2

Nagoya University, Japan
National Institute of Infomation and Communications Technology, Japanyichiao.wu@g.sp.m.is.nagoya-u.ac.jp, tomoki@icts.nagoya-u.ac.jp

Abstract

However, because of the data-driven nature and the lack ofprior speech knowledge, it is hard for these NN-based SS models to deal with unseen data. For instance, if the pitches of testing acoustic features are scaled or outside the observed pitch
range of training data, the pitch accuracy and speech qualityof the WN-generated speech samples markedly degrade. Since
the pitch controllability is an essential feature for a vocoder,
NN-based SS models with carefully designed periodic and aperiodic inputs [24–26] greatly improve the pitch modeling capability. Furthermore, in our previous work, we proposed a quasiperiodic WN vocoder (QPNet) [27, 28], which adopts pitchdependent dilated convolution networks (PDCNNs) to dynamically change the network architecture according to the input
pitches, to improve the pitch controllability and speech modeling ability of WN.
To tackle the slow generation of AR WN/QPNet, we apply aquasi-periodic (QP) structure to a compact non-AR model parallel WaveGAN (PWG) [21]. Since PWG transforms a noise
sequence sampled from a standard Gaussian distribution into
speech samples by taking conventional acoustic features as theauxiliary feature, PWG is more flexible than the models required specific periodic and aperiodic inputs. Moreover, thenon-AR fashion also makes the parallelized generation avaliable for real-time generation. In this paper, we propose a fast
and flexible QPPWG vocoder to improve the pitch controllability and speech modeling efficiency of PWG. Both objective
and subjective evaluations are conducted, and the experimental
results show the higher pitch accuracy, comparable speech quality, and smaller model size of the QPPWG vocoder than that of
the PWG vocoder.

In this paper, we propose a parallel WaveGAN (PWG)-likeneural vocoder with a quasi-periodic (QP) architecture to improve the pitch controllability of PWG. PWG is a compact nonautoregressive (non-AR) speech generation model, whose generative speed is much faster than real time. While utilizing
PWG as a vocoder to generate speech on the basis of acoustic
features such as spectral and prosodic features, PWG generates
high-fidelity speech. However, when the input acoustic features
include unseen pitches, the pitch accuracy of PWG-generated
speech degrades because of the fixed and generic network of
PWG without prior knowledge of speech periodicity. The proposed QPPWG adopts a pitch-dependent dilated convolution
network (PDCNN) module, which introduces the pitch information into PWG via the dynamically changed network architecture, to improve the pitch controllability and speech modelingcapability of vanilla PWG. Both objective and subjective evaluation results show the higher pitch accuracy and comparable
speech quality of QPPWG-generated speech when the QPPWG
model size is only 70 % of that of vanilla PWG.
Index Terms: neural vocoder, parallel WaveGAN, quasi- periodic WaveNet, pitch-dependent dilated convolution

1. Introduction
Because of the high temporal resolution of speech signals,
speech waveform modeling is challenging. The general technique to tackle speech synthesis (SS) is called a vocoder [1, 2],which encodes speech into low-dimensional acoustic featuresand decodes speech on the basis of these acoustic features. The
conventional vocoders such as STRAIGHT [3] and WORLD(WD) [4] are usually designed on the basis of a source-filter
model [5], which decomposes speech into spectral and prosodic
acoustic features. However, the ad hoc signal-processing mechanisms imposed on the conventional vocoders cause the loss ofphase information and temporal details, which results in marked
speech quality degradation.
To achieve high fidelity SS, many neural network (NN)based autoregressive (AR) SS models such as SampleRNN [6]and WaveNet (WN) [7] have been proposed to directly model
the probability distributions of speech waveforms without many
ad hoc assumptions of SS. The NN-based vocoders [8–11] arealso proposed on the basis of these AR SS models to replace thesynthesizers of the conventional vocoders for recovering the lostphase information and temporal details and generating highquality speech. Furthermore, because of the extremely slow
generation of WN and SampleRNN, many AR models withcompact networks and specific knowledge [12–14] and non-AR
models such as flow-based [15–19] and generative adversarial
network (GAN) [20]-based models [21–23] have been proposed
for real-time speech generation.

Copyright © 2020 ISCA

2. Parallel WaveGAN
As shown in Fig. 1, PWG is composed of a discriminator (D), a
generator (G), and a multi-resolution short-time Fourier transform (STFT) loss module. The discriminator is trained to detect
synthesized samples as fake speech and natural samples as realspeech. The training criterion of the discriminator is to minimize the loss LD , which is formulated as
LD (G, D)= Ex∈pdata (1 − D(x))2 + Ez∈N (0,I) D(G(z))2 , (1)where x denotes the natural samples, pdata denotes the data
distribution of the natural samples, N (0, I) denotes a Gaussian
distribution with zero mean and standard deviation, and z denotes the input noise of the generator drawn from the Gaussian distribution. Note that all auxiliary features of G are omitted in this chapter for simplicity. The discriminator is a fullyconvolutional network, which consists of several dilated convolution network (DCNN) [29] layers with LeakyReLU [30]3535

http://dx.doi.org/10.21437/Interspeech.2020-1070

Training

LG

λadv

Lsp

<=66

LD

A$80$8&`@)'8.9/-*0*:-*:8/-'1%8*-/?%.8,+&/
;=

Ladv

B'--*:&`A
C:0$8

Multi-resolution
STFT loss

Discriminator
(D)

)<=66

222 ;4`@ 222>*.*08'3*/?'*1A$80$8&:`@\;=

Synthesis Generated
speechB'--*:&:`A\;=
C:0$8

Generator (G)@??*.8'3*/+*.*08'3*/?'*1-

Acoustic
features

Gaussiannoise

NaturalspeechFigure 2: Pitch-dependent dilated convolution.
PWG modeling all components is inefficient. Specifically, a receptive field is a region in the input space that the outputs areaffected by, and the receptive field length is highly related tothe modeling capacity. However, modeling speech using fixedlength receptive fields may lead to the receptive fields including many redundant samples from the oversampled periodic
components. To tackle this problem, we proposed a QPPWG
vocoder to respectively model periodic and aperiodic components using cascaded fixed and pitch-adaptive modules.

Figure 1: Architecture of Parallel WaveGAN.
activation functions. The dilation size of each DCNN layer isextending in an exponential growth manner with a base of two
and an exponent of its layer index. The generator is trained togenerate speech samples, which makes the discriminator difficult to distinguish between the synthesized and natural samples.
The training criterion of the generator is to minimize the generator loss (LG ) formulated as
LG (G, D) = Lsp (G) + λadv Ladv (G, D),(2)

3. Quasi-Periodic parallel WaveGAN

which is the weighted sum of an adversarial loss (Ladv ) fromthe GAN structure and a spectral loss (Lsp ) from the multiresolution STFT loss module with a weight λadv . The Ladv
is formulated asLadv (G, D) = Ez∈N (0,I) (1 − D(G(z)))2 .
(3)

Since the GAN architecture and the multi-resolution STFT lossof PWG have shown the effectiveness for training a speech generator, the proposed QPPWG inherits the discriminator and the
Lsp regularizer from PWG and focuses on improving the generator using a QP structure. The main modules of the QP structure
are PDCNN components and a cascaded architecture, which are
inspired by the pitch filtering and the cascaded recursive structure of the code-excited linear prediction (CELP) codec [31].

Unlike flow-based models [15–19] adopting an invertible network to transform the distribution of the input noise to the real
data distribution, PWG adopts a GAN structure to make thegenerator learn the transformation via the feedback from thediscriminator. The generator adopts a WN-like network but
without the AR mechanism and causality, so the generation of
PWG markedly faster than that of WN. To ensure the stability
and efficiency of the GAN training, PWG adopts an extra Lsp
loss as a regularizer of the generator. Specifically, the Lsp is asummation of a spectral convergence loss (Lsc ) and a log STFT
magnitude loss (Lm ). The Lsp is formulated as
k|STFT(x)| − |STFT(x̂)|kF
Lsc (x, x̂) =,kSTFT(x)|kF

3.1. Pitch-dependent dilated convolution
As shown in Fig. 2, there are gaps between the inputs of a
DCNN kernel, and the length of each gap is a predefined hyperparameter called a dilation size (rate). PDCNN is an extension of DCNN, and its dilation size is pitch-dependent anddynamically changed according to the input pitch. Specifically,a dilated convolution with a kernel size three is formulated as
(o)

yt

(4)

(i)

(i)

(6)

where y (i) and y (o) are the input and output of the DCNN layer.
W (c) , W (p) and W (f) are the trainable 1×1 convolution filtersof current, previous, and following samples, respectively. ∗ isthe convolution operator, and the dilation size d of DCNN is atime-invariant constant. By contrast, PDCNN adopts a pitchdependent dilated factor Et to dynamically change the dilation
size d′ in each time step t as

and the Lm is formulated as
Lm (x, x̂) =(i)= W (c) ∗ yt + W (p) ∗ yt−d + W (f) ∗ yt+d ,1
k log |STFT(x)| − log |STFT(x̂)|kL1 , (5)N

where x̂ denotes the generated samples from the generator,
k·kF is Frobenius norm, k·kL1 is L1 norm, |STFT (·)| denotes
STFT magnitudes, and N is the number of the magnitude elements. Moreover, to avoid the generator overfitting to a specific
STFT resolution causing a suboptimal problem, the final Lsp
is a summation of several Lsp values calculated on the basis of
STFT features with different analysis parameters such as FFT
size and frame length and shift.
Although PWG achieves high fidelity speech generation,
the fixed and generic network architecture of PWG is not appropriate. Speech is a quasi-periodic signal consisting of periodic
components with long-term correlations and aperiodic components with short-term correlations, so the fixed architecture ofd′ = Et × d.

(7)

The dilated factor Et is derived from
Et = Fs /(F0,t × a),(8)

where Fs is the sampling rate, F0 is the fundamental frequency,
and a is a hyperparameter called a dense factor, which indicatesthe number of samples in one cycle taken as the PDCNN inputs
for each time step. The higher the dense factor, the lower thesparsity of PDCNN. For simplicity, the d′ of each time step isrounded. For the unvoiced segments, we find that the models

3536

E42%9+>:*;C

Table 1: Numbers of generator parameters.

A'"'.%&'#)(1''$*999

I'WX

E42%9+
>:*;C

LgL

I'WX

LgL

8'D&).'(!#+%2)-2/$48(;'*>:*;C+7

999

DA24:4('G+,%()A'%6

D9(3)4@%>:*;C
D9(3)4@%
>:*;C

LgL

H$/+(&!$)
0'%&+.'(

A%+((!%")"/!('PWG 20QPPWGafQPPWGf a

BF 30C3-

BF 20C2-

BA 10C2
BF 10C1

BF 10C1
BA 10C2Size (M)

1.16

0.78

0.79

0.79

LgL
BC43+
;*&&%;)4*&
DA24:4('G+,%()A'%6

LgL

Table 2: Objective evaluation results.

A%&'#

LgL
D9(3)4@%+F+E42%9+
GgL)#!2%&'#8(;'*>:*;C+H
X1(%,12'PWG 30Macro 0Macro 1

BC43+;*&&%;)4*&

5.'B!/+().'(!#+%2)-2/$4

D9(3)4@%+F+E42%9+'%649A(:+>:*;CFigure 3: Architecture of QPPWG generator.
with interpolated F0 outperform the models setting d′ to one,
so our implementation [32] adopts interpolated F0 . In conclusion, PDCNN introduces the pitch information to the network,makes each sample have a pitch-dependent receptive field size,and efficiently enlarges receptive fields.

Vocoder
Blocks

WD
-

30

PWG

1 × F0
1/2 × F0
2 × F0
Average

0.10
0.14
0.10
0.11

RMSE of log F0
0.12
0.15
0.27
0.32
0.15
0.15
0.18
0.21

1 × F0
1/2 × F0
2 × F0
Average

2.58
3.89
3.79
3.42

3.69
4.47
5.24
4.46

20

MCD (dB)
3.74
4.39
5.06
4.40

QPPWG
af
fa
0.11
0.19
0.11
0.14

0.11
0.20
0.11
0.14

3.80
4.52
4.92
4.41

4.54
5.18
5.61
5.11

4.2. Experimental settings
All NN-based vocoders were trained in a multi-speaker manner.
The training corpus consisted of 2200 utterances of the “slt” and“bdl” speakers of CMU-ARCTIC [33] and 852 utterances of all
speakers of Voice Conversion Challenge 2018 (VCC2018) [34].
The total data length was around 2.5 hours. The testing corpus
was the SPOKE set, which consisted of two male and two female speakers, of VCC2018 corpus, and the number of testingutterances of each speaker was 35. All speech data were set toa sampling rate of 22,050 Hz and a 16-bit resolution.
The auxiliary features of these speech generation models
consisted of one-dimensional continuous F0 , one-dimensional
unvoiced/voiced binary code (U/V ), 35-dimensional melcepstrum (mcep), and two-dimensional coded aperiodicity(codeap). The WD vocoder was first adopted to extract onedimensional F0 and 513-dimensional spectral feature (sp) andaperiodicity (ap) with a frameshift of 5 ms. F0 was interpolated
to the continuous F0 and converted to the U/V , ap was coded
into the codeap, and sp was parameterized into the mcep. To
simulate unseen data, the continuous F0 was scaled by the ratiosof 1/2 and 2 while keeping other features the same. Moreover,the dilated factor Et of QPPWG was calculated with the continuous F0 because of the higher speech quality, and the dense
factor of QPPWG was empirically set to four.
The models were trained with a RAdam optimizer [35](ǫ = 1e−6 ) with 400 k iterations. For the stability, the generators of the models were trained with only multi-resolution
STFT losses, which were calculated on the basis of three different FFT sizes (1024 / 2048 / 512), frameshifts (120 / 240 /50), and frame lengths (600 / 1200 / 240), for the first 100 k iterations and then jointly trained with the discriminators for thefollowing 300 k iterations. The balance weight λadv of Ladv
was set to 4.0. The learning rates, which decayed 50 % every
200 k iterations, of the generators were 1e−4 and the discriminators were 5e−5 . The minibatch size was six and the batch
length was 25,520 samples.

3.2. Generator of QPPWG
The architecture of the QPPWG generator is shown in Fig. 3,
and it is similar to the WN-like PWG generator. The maindifference is the hierarchical architecture of the stacked residual blocks. Specifically, QPPWG includes two cascaded macroblocks with different types of residual blocks while PWG
only contains one type of residual blocks. The first macroblockof QPPWG consists of stacked adaptive blocks with PDCNN
layers to model the periodic components with long-term correlations, and the second macroblock of QPPWG consists ofstacked fixed blocks with DCNN layers to model the aperiodic
components with short-term correlations.

4. Experiments
4.1. Model descriptions
In this paper, the QPPWG models with two different orders ofmacroblocks and the PWG models with two different numbersof residual blocks were evaluated. Specifically, the QPPWG
model, whose first macroblock included 10 adaptive blocks
with 2 cycles of the dilation size expansions (BA 10C2) and thefollowing macroblock included 10 fixed blocks with one cycle (BF 10C1), was denoted as QPPWGaf and the one with areverse macroblock order was QPPWGf a. The generator ofvanilla PWG (PWG 30) contained 30 fixed blocks with 3 cycles (BF 30C3) and the compact PWG (PWG 20) contained 20
fixed blocks with 2 cycles (BF 20C2). The channel and kernel sizes of the non-causal convolution of these generators were64 and three. As shown in Table 1, the generator size of the
proposed QPPWG is around 70 % of that of the vanilla PWG
because of the less stacked residual blocks. However, because
the time-variant mechanism degrades the parallelism of CNNs,
the real-time factor (RTF) of QPPWG generation with a Titan
V GPU is around 0.020, which is higher than 0.016 of PWG 30
and 0.011 of PWG 20. Moreover, these four models adopted
the the same discriminator architecture including 10 non-causal
DCNN layers with 64 convolution channels, three kernels, andLeakyReLU (α is 0.2) activation functions, and the number ofthe discriminator parameters was around 0.1 M.

4.3. Objective evaluations
Root mean square error (RMSE) of log F0 and mel-cepstral
distortion (MCD) were adopted to the objective evaluations.
Both measurements were calculated using the auxiliary features and the features extracted from the generated speech.
As shown in Table 2, the proposed QPPWG vocoders achieve3537

MEAN OPINION SCORE

5QPPWGf a

QPPWGaf

Intermediate cumulative outputs of 1–10 blocks

432
3.5 4.3 3.8 4.1

2.2 2.9 2.7 3.2

2.9 2.5 2.2 2.7

1PREFERENCE SCORE (%)

1×!₀
½×!₀
2×!₀
WD
PWG_30
PWG_20QPPWG"#
Figure 4: MOS results of speech quality with 95 % CI.

Cumulative outputs of 1–20 blocks

100
80
60
40
Figure 6: Spectra of cumulative outputs.

20
27.1

72.9

20.3

79.7

curacies between the QPPWGaf and PWG 30 vocoders withscaled F0 inputs. In conclusion, introducing the pitch information to the PWG model by the QP structure markedly improves
the pitch and speech modeling capabilities of the PWG vocoder,which results in compact model size and better pitch controllability of the QPPWG voder.

0
½×!₀
2×!₀
PWG_30QPPWG"#
Figure 5: ABX results of pitch accuracy with 95 % CI.

markedly higher F0 accuracy than the PWG vocoders whenconditioned on the scaled F0 , and it confirms the effectiveness of the proposed QP structure. The results also show that
QPPWGaf achieves a comparable spectral prediction accuracy as the vanilla and compact PWGs. Moreover, QPPWGaf
achieving lower MCDs than QPPWGf a implies that modelinglong-term correlations first get a better overall spectral structure.
In conclusion, the proposed QP structure improves the accuracyof pitch modeling of the PWG vocoder and efficiently extends
the receptive field length. The QPPWG with an adaptive to fixedorder outperforms the QPPWG with a reverse order.

4.5. Discussion
Since the model capacity is highly related to the receptive field
length [27, 28], and the length of PWG 30 is 6139 samples(20 + · · · + 29 = 1023 with three cycles and two sides plusone), QPPWG attains a longer effective receptive field lengtharound 3,000–16,000 samples. Specifically, the size is 2047 of
the BF 10C1 and 124×ET (20 + · · · + 24 = 31 with two cycles
and two sides) of the BA 10C2, and the ET is around 11–110 of
the 500–50 Hz pitches when the dense factor is four.
Moreover, as the intermediate cumulative outputs shownin Fig. 6, the first ten adaptive blocks of QPPWGaf focus onmodeling the pitch and harmonic components, which have longterm correlations, while the first ten fixed blocks of QPPWGf afocus on modeling the non-harmonic components, which haveshort-term correlations. The results confirm our assumptions of
the QP structure, and the behavior, which is similar to the harmonic plus noise model [36, 37], of QPPWG is more tractable
and interpretable than that of vanilla PWG. More details anddemo samples can be found on our website [38].

4.4. Subjective evaluations
The subjective evaluation set consisted of 960 selected utterances of four testing speakers, four vocoders (WD, PWG 30,
PWG 20, and QPPWGaf ), and three F0 scaled ratios (1, 1/2,
and 2). For each speaker, vocoder, and F0 ratio, we randomly
selected 20 utterances from the 35 testing utterances for bothmean opinion score (MOS) and ABX tests. Specifically, thespeech quality of each utterance was evaluated by listeners assigning MOSs (1–5). The higher the MOS, the better the speech
quality. For the ABX test, every time listeners compared two
testing utterances with one reference to pick up the utterance
whose pitch contour was more consistent with that of the reference. The WD-generated utterances were taken as the references, and the pitch accuracies of the QPPWGaf -generated
utterances with 1/2 and 2 F0 inputs were compared with thatof the PWG 30-generated utterances. Eight listeners involvedin both tests and each utterance was evaluated by at least twolisteners. Most listeners were audio-related researchers.
As shown in Fig. 4, the QPPWGaf vocoder markedly outperforms the same sized PWG 20 vocoder for all F0 inputs.
Even compared with PWG 30, QPPWGaf still achieves higher
speech qualities for the scaled F0 inputs and a comparable
speech quality for the unchanged F0 input. In addition, the results of Fig. 5 show the perceptible differences of the pitch ac-

5. Conclusions
In this paper, we integrate a fast and compact PWG vocoderwith a QP structure to improve its pitch controllability. The
proposed QPPWG vocoder reduces the model size and achieves
higher speech quality and pitch accuracy than the PWG vocoderwhen the input F0 sequence is scaled. In conclusion, the QPPWG vocoder is more in line with the definition of a vocoder,which attains acoustic controllability.

6. Acknowledgments
This work was supported in part by JST CREST Grant Number JPMJCR19A3. The initial investigation in this study was
performed while Y.-C. Wu was interning at NICT.

3538

7. References[20] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. WardeFarley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Proc. NIPS, Dec. 2014, pp. 2672–2680.

[1] H. Dudley, “The vocoder,” Bell Labs Record, vol. 18, no. 4, pp.
122–126, 1939.

[21] R. Yamamoto, E. Song, and J.-M. Kim, “Parallel WaveGAN: A
fast waveform generation model based on generative adversarial
networks with multi-resolution spectrogram,” in Proc. ICASSP,May 2020, pp. 6199–6203.

[2] M. R. Schroeder, “Vocoders: Analysis and synthesis of speech,”Proc. IEEE, vol. 54, no. 5, pp. 720–734, 1966.
[3] H. Kawahara, I. Masuda-Katsuse, and A. De Cheveigne, “Restructuring speech representations using a pitch-adaptive time–
frequency smoothing and an instantaneous-frequency-based f0
extraction: Possible role of a repetitive structure in sounds,”
Speech communication, vol. 27, no. 3-4, pp. 187–207, 1999.

[22] K. Kumar, R. Kumar, T. de Boissiere, L. Gestin, W. Z. Teoh,
J. Sotelo, A. de Brébisson, Y. Bengio, and A. C. Courville, “MelGAN: Generative adversarial networks for conditional waveform
synthesis,” in Proc. NeurIPS, Dec. 2019, pp. 14 910–14 921.

[4] M. Morise, F. Yokomori, and K. Ozawa, “World: a vocoder-based
high-quality speech synthesis system for real-time applications,”
IEICE TRANSACTIONS on Information and Systems, vol. 99,
no. 7, pp. 1877–1884, 2016.

[23] M. Bińkowski, J. Donahue, S. Dieleman, A. Clark, E. Elsen,
N. Casagrande, L. C. Cobo, and K. Simonyan, “High fidelity
speech synthesis with adversarial networks,” in Proc. ICLR, Apr.
2020.

[5] R. McAulay and T. Quatieri, “Speech analysis/synthesis basedon a sinusoidal representation,” IEEE Transactions on Acoustics,
Speech, and Signal Processing, vol. 34, no. 4, pp. 744–754, 1986.

[24] X. Wang, S. Takaki, and J. Yamagishi, “Neural source-filter-based
waveform model for statistical parametric speech synthesis,” inProc. ICASSP, May 2019, pp. 5916–5920.

[6] S. Mehri, K. Kumar, I. Gulrajani, R. Kumar, S. Jain, J. Sotelo,A. Courville, and Y. Bengio, “SampleRNN: An unconditional
end-to-end neural audio generation model,” in Proc. ICLR, Apr.
2017.

[25] X. Wang, S. Takaki, and J. Yamagishi, “Neural source-filter
waveform models for statistical parametric speech synthesis,”
IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 28, pp. 402–415, 2020.

[7] A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals,
A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu,“WaveNet: A generative model for raw audio,” in Proc. SSW9,
Sept. 2016, p. 125.

[26] K. Oura, K. Nakamura, K. Hashimoto, Y. Nankaku, andK. Tokuda, “Deep neural network based real-time speech vocoderwith periodic and aperiodic inputs,” in Proc. SSW10, Sept. 2019,
pp. 13–18.

[8] A. Tamamori, T. Hayashi, K. Kobayashi, K. Takeda, and T. Toda,“Speaker-dependent WaveNet vocoder,” in Proc. Interspeech,
Aug. 2017, pp. 1118–1122.

[27] Y.-C. Wu, T. Hayashi, P. L. Tobing, K. Kobayashi, and T. Toda,“Quasi-periodic WaveNet vocoder: A pitch dependent dilated
convolution model for parametric speech generation,” in Proc. Interspeech, Sept. 2019, pp. 196–200.

[9] T. Hayashi, A. Tamamori, K. Kobayashi, K. Takeda, and T. Toda,“An investigation of multi-speaker training for WaveNet vocoder,”in Proc. ASRU, Dec. 2017, pp. 712–718.

[28] Y.-C. Wu, T. Hayashi, P. L. Tobing, K. Kobayashi, and T. Toda,“Quasi-periodic WaveNet: An autoregressive raw waveform generative model with pitch-dependent dilated convolution neuralnetwork,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, (submitted).

[10] K. Tachibana, T. Toda, Y. Shiga, and H. Kawai, “An investigation of noise shaping with perceptual weighting for wavenet-based
speech generation,” in 2018 IEEE International Conference on
Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018,
pp. 5664–5668.

[29] F. Yu and K. Vladlen, “Multi-scale context aggregation by dilated
convolutions,” in Proc. ICLR, May 2016.

[11] Y. Ai, H.-C. Wu, and Z.-H. Ling, “SampleRNN-based neural vocoder for statistical parametric speech synthesis,” in Proc.
ICASSP, Apr. 2018, pp. 5659–5663.

[30] A. L. Maas, A. Y. Hannun, and A. Y. Ng, “Rectifier nonlinearities
improve neural network acoustic models,” in Proc. ICML, June
2013, pp. 3–11.

[12] Z. Jin, A. Finkelstein, G. J. Mysore, and J. Lu, “FFTNet: A realtime speaker-dependent neural vocoder,” in Proc. ICASSP, Apr.
2018, pp. 2251–2255.

[31] M. Schroeder and B. Atal, “Code-excited linear prediction(CELP): High-quality speech at very low bit rates,” in Proc.
ICASSP, vol. 10, Apr. 1985, pp. 937–940.

[13] N. Kalchbrenner, E. Elsen, K. Simonyan, S. Noury,
N. Casagrande, E. Lockhart, F. Stimberg, A. van den Oord,
S. Dieleman, and K. Kavukcuoglu, “Efficient neural audio
synthesis,” in Proc. ICML, July 2018, pp. 2415–2424.

[32] Y.-C. Wu, QPPWG repository, Accessed:
Available: https://github.com/bigpon/QPPWG

2020. [Online].

[14] J.-M. Valin and J. Skoglund, “LPCNet: Improving neural speechsynthesis through linear prediction,” in Proc. ICASSP, May 2019,
pp. 5826–7830.

[33] J. Kominek and A. W. Black, “The CMU ARCTIC speech
databases for speech synthesis research,” in Tech. Rep. CMU-LTI03-177, 2003.

[15] A. van den Oord, Y. Li, I. Babuschkin, K. Simonyan, O. Vinyals,
K. Kavukcuoglu, G. van den Driessche, E. Lockhart, L. C. Cobo,
F. Stimberg, N. Casagrande, D. Grewe, S. Noury, S. Dieleman,
E. Elsen, N. Kalchbrenner, H. Zen, A. Graves, H. King, T. Walters, D. Belov, and D. Hassabis, “Parallel WaveNet: Fast highfidelity speech synthesis,” in Proc. ICML, July 2018, pp. 3915–
3923.

[34] J. Lorenzo-Trueba, J. Yamagishi, T. Toda, D. Saito, F. Villavicencio, T. Kinnunen, and Z. Ling, “The voice conversion challenge
2018: Promoting development of parallel and nonparallel methods,” in Proc. Odyssey, June 2018, pp. 195–202.
[35] L. Liu, H. Jiang, P. He, W. Chen, X. Liu, J. Gao, and J. Han, “Onthe variance of the adaptive learning rate and beyond,” in Proc.
ICLR, Apr. 2020.

[16] W. Ping, K. Peng, and J. Chen, “ClariNet: Parallel wave generation in end-to-end text-to-speech,” in Proc. ICLR, May 2019.

[36] Y. Stylianou, O. Cappé, and E. Moulines, “Continuous probabilistic transform for voice conversion,” IEEE Trans. Speech Audio
Process., vol. 6, no. 2, pp. 131–142, 1998.

[17] R. Prenger, R. Valle, and B. Catanzaro, “WaveGlow: A flowbased generative network for speech synthesis,” in Proc. ICASSP,May 2019, pp. 3617–3621.

[37] Y. Stylianou, “Applying the harmonic plus noise model in concatenative speech synthesis,” IEEE Trans. Speech Audio Process.,
vol. 9, no. 1, pp. 21–29, 2001.

[18] S. Kim, S.-G. Lee, J. Song, J. Kim, and S. Yoon, “FloWaveNet :A generative flow for raw audio,” in Proc. ICML, June 2019, pp.
3370–3378.

[38] Y.-C. Wu, QPPWG demo, Accessed: 2020. [Online]. Available:
https://bigpon.github.io/QuasiPeriodicParallelWaveGAN demo/[19] N.-Q. Wu and Z.-H. Ling, “WaveFFJORD: FFJORD-based
vocoder for statistical parametric speech synthesis,” in Proc.
ICASSP, May 2020, pp. 7214–7218.

3539