FIRNET: FUNDAMENTAL FREQUENCY CONTROLLABLE FAST NEURAL VOCODER
WITH TRAINABLE FINITE IMPULSE RESPONSE FILTER
Yamato Ohtani1 , Takuma Okamoto1 , Tomoki Toda2,1 , Hisashi Kawai1
1

National Institute of Information and Communications Technology, Japan
2
Information Technology Center, Nagoya University, Japan
ABSTRACT

Some neural vocoders with fundamental frequency (f0 ) control havesucceeded in performing real-time inference on a single CPU while
preserving the quality of the synthetic speech. However, comparedwith legacy vocoders based on signal processing, their inference
speeds are still low. This paper proposes a neural vocoder basedon the source-ﬁlter model with trainable time-variant ﬁnite impulse
response (FIR) ﬁlters, to achieve a similar inference speed to legacy
vocoders. In the proposed model, FIRNet, multiple FIR coefﬁcientsare predicted using the neural networks, and the speech waveformis then generated by convolving a mixed excitation signal with these
FIR coefﬁcients. Experimental results show that FIRNet can achieve
an inference speed similar to legacy vocoders while maintaining f0
controllability and natural speech quality.
Index Terms— Speech synthesis, neural vocoder, source-ﬁlter
model, ﬁnite impulse response, fundamental frequency control
1. INTRODUCTION
A neural vocoder is a well-known neural waveform generation
technique that allows us to convert acoustic features to high-quality
speech waveforms. Since the invention of WaveNet [1], many neuralvocoders have been proposed [2–6] and applied in speech generation systems, such as text-to-speech (TTS), voice conversion, andsinging voice synthesis. For practical use, they require fundamental
frequency (f0 ) controllability and real-time generation speed on asingle CPU. Therefore, it is important to develop neural vocoders
that satisfy these requirements.
HiFi-GAN [5] is a fast and high-ﬁdelity neural vocoder. Although HiFi-GAN achieves real-time inference, it does not have
ﬂexible f0 controllability. To solve this problem, Harmonic-Net+ [7]introduced the source-ﬁlter model [8] and the quasi-periodic (QP)
architecture [9, 10] into HiFi-GAN. In Harmonic-Net+, the downsampling network converts a source excitation signal generated
from the f0 to downsampled latent representations. The speech
waveform is then generated using the QP-HiFi-GAN module. Another approach based on HiFi-GAN, Source-ﬁlter HiFi-GAN (SiFiGAN) [11], has been proposed. This framework has two networks:
the source network, which applies the QP architecture, and theﬁlter network, which comprises the HiFi-GAN module. In SiFiGAN, the source network converts the f0 -dependent sine signal to
the source excitation representation, from which the ﬁlter network
generates the speech waveform. Although these neural vocoders
achieve real-time generation speed on a single CPU and ﬂexible f0
controllability, their real-time factors (RTFs) for 24-kHz sampling
waveforms range from approximately 0.4 to 0.8 [7, 11] because of
Corresponding author: yamato.ohtani@nict.go.jp

the high complexity of the ﬁltering process. In contrast, the TTS
acoustic models reported in [12] are much faster than the neuralvocoders described above on a single CPU. Therefore, for real-time
operation in speech generation systems, source-ﬁlter-based neuralvocoders require higher inference speed than those described above.
According to the theory of the source-ﬁlter model [8], a speech
waveform is generated by convolving a source excitation signal with
an impulse response of the vocal tract. That is, there is a possibility of realizing a high-speed neural vocoder based on the sourceﬁlter model by estimating appropriate impulse responses using neural networks. To test this hypothesis, we propose a neural vocoderwith trainable time-variant ﬁnite impulse response (FIR) ﬁlters. In
the proposed model, which we call FIRNet, neural networks convert
f0 -independent acoustic features to multiple FIR coefﬁcients. The
speech waveform is then generated through the intermediate residual signal by ﬁltering the source excitation signal with these FIR
coefﬁcients. Because the FIR ﬁlters do not depend on f0 parameters, ﬂexible f0 control is possible by changing source excitation
signals. In addition, this model achieves very high inference speed
because of the low computational cost incurred by simple linear ﬁltering. Experimental results show that the inference speed of the
proposed model on a single CPU is almost the same as that of legacy
vocoders, such as the WORLD synthesizer [13], while the speech
quality and f0 controllability are preserved 1 .
2. RELATED WORK
Several neural vocoders based on the source-ﬁlter model have been
proposed previously. In [7, 11, 14, 15], speech waveforms are generated by ﬁltering excitation signals nonlinearly with neural networks.
LPCNet [4, 16] predicts residual signals using neural networks andthen generates speech waveforms using linear prediction with linear
predictive coding parameters, that is, inﬁnite impulse response (IIR)
ﬁltering. The NITECH end-to-end TTS system [17] introduced thesource-ﬁlter model using FIR ﬁltering into the waveform generation
module. In this module, speech waveforms are generated from predicted residual signals by the differentiable mel-cepstral synthetic
ﬁlter [18]. This synthetic ﬁlter employs the cascade connection ofmultiple FIR ﬁlters instead of the IIR ﬁlter because of the need tosatisfy bounded-input bounded-output (BIBO) stability.
In contrast to the above neural vocoders, FIRNet generates
speech only by conversion from source excitation signals to residual
signals and speech waveforms, in a step-by-step manner, by linear
ﬁltering with multiple FIR ﬁlters whose coefﬁcients are predictedby neural networks. In addition, FIRNet guarantees stable speech
generation because of its BIBO stability.
1 Speech samples are available on https://ast-astrec.nict.
go.jp/demo_samples/firnet_icassp2024/© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including
reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, orreuse of any copyrighted component of this work in other works.

Input features

Cont. log fO

V/UV

BAP

Excitation
generator

Mixed excitation

MGC

Causal conv
block

Auxiliary feature
FC + GELU( , )Dilated Causal
Conv1dFIR filter( )ConvNeXtcore block

Causal conv
block

FC
Concatenate

Concatenate

FC + GELU

Causal depthwise
1D-conv( , )Dilated Causal
Conv1dFIR filter

Causal conv
block

( )

Layer
normalization

Residual
FIR network

Residual signal( )Input signal

Notations and specifications of input features
V/UV: Voiced / unvoiced decision flagCont. log f0 : Continuous logarithmic
fundamental frequency
BAP: Band aperiodicity (3 dims.)
MGC: mel-generalized cepstrum (40 dims. α=0.466)

ConvNeXtcore block

Pointwise
1D-conv + GELU
Causel ConvNeXt
core blockResonance
FIR network

Causal ConvNeXt
core block

Global response
normalization(

FC + GELU)( , )Dilated Causal
Conv1dFIR filter

Pointwise
1D-conv

Output signal

Speech waveform(a) Network architecture

FC
Concatenate(b) Causal convolution block( )(c) Residual/resonance FIR network

3.1. Details of generator modules
3.1.1. Excitation generator
The excitation generator generates a mixed excitation signal [13,21],
which consists of a weighted sum of the f0 -dependent pulse train and
the Gaussian noise based on BAP. The tth mixed excitation signals(t) is deﬁned as follows:{
gp v (k) ∗ p (t) + gn u(k) ∗ n (t) if voiceds (t) =,
(1)
gn n (t)if unvoiced
where p(t), n(t), gp , and gn denote the f0 -dependent pulse train,
Gaussian noise at time t, gain values for p(t), and gain values forn(t), respectively. v (k) and u(k) denote impulse responses of voiced
and unvoiced segments, respectively. These are calculated by applying the inverse Fourier transform to BAP at the kth frame corresponding to t. ∗ denotes the convolution operation. Following [14],gp and gn are set to 0.1 and 0.003, respectively.












7LPH>V@

















(a) Mixed excitation signal
$PSOLWXGH

Fig. 1 shows an overview of FIRNet and its sub-modules. FIRNet(Fig. 1(a)) is composed of the excitation generator, three causal
convolution blocks (Fig. 1(b)), and the residual FIR network andresonance FIR network (Fig. 1(c)). As input features, we employ
continuous f0 , voiced/unvoiced ﬂag (V/UV), band aperiodicity(BAP) [19], and mel-generalized cepstrum coefﬁcients (MGC) [20].
In inference, the excitation generator ﬁrst generates a mixed excitation signal [21] shown in Fig. 2(a), using continuous f0 , V/UV,
and BAP. BAP and MGC are converted to their latent representations by causal convolution blocks. Moreover, latent representations
for the residual FIR network are generated from the BAP and MGC
latent representations. Because the frequency characteristics of themixed excitation signal differ from those of the actual residual signal, the residual FIR network conditioned by its corresponding latent
representations converts the mixed excitation signal to the residual
signal, as illustrated in Fig. 2(b). Finally, the residual signal is converted to a speech waveform by the resonance FIR network conditioned by the MGC latent representation, as shown in Fig. 2(c).













7LPH>V@(b) Residual signal
$PSOLWXGH

3. PROPOSED METHOD: FIRNET

$PSOLWXGH

Fig. 1. Overview of FIRNet and sub-modules.













7LPH>V@(c) Speech waveform
Fig. 2. Generated signals from FIRNet.
3.1.2. Causal convolution block
Fig. 1(b) illustrates the structure of our causal convolution block.
To obtain highly accurate and effective latent representations, weemploy the core module of the ConvNeXt V2 architecture [22].
This consists of an input depthwise convolution layer, layer normalization [23], pointwise convolution with Gaussian error linear unit
(GELU) activation [24], global response normalization [22], andoutput pointwise convolution. In this paper, we apply causality to
the input depthwise convolution layer and set its kernel size to 5.
The causal convolution blocks include two ConvNeXt core blocks.
The numbers of output channels for BAP, MGC, and the residual
FIR network are 128, 256, and 128, respectively.
3.1.3. FIR network
Fig. 1(c) illustrates the structure of the FIR network architecture. In
this network, an input signal is converted to an output signal using
multiple FIR ﬁlters. In the ﬁgure, h(1,k) , h(2,k) , · · · , h(M,k) represent the impulse response coefﬁcients of the corresponding FIR

ﬁlters, x(0) (t) and x(M ) (t) are the input and output signals, respectively, and x(1) (t), x(2) (t), · · · , x(M −1) (t) represent the intermediate output signals, which are deﬁned as follows:x(m) (t) = h(m,k) ∗ x(m−1) (t) + x(m−1) (t) .

(2)

The impulse response coefﬁcients of the mth FIR ﬁlter are estimatedfrom the latent representations of an auxiliary feature and previous
impulse response coefﬁcients using the dilated causal convolution
layer. In this paper, the number of latent channels is set to 128, thekernel sizes of the dilated causal convolution layers are set to 3, andthe others are set to 1. The residual and resonance FIR networks
include 8 FIR ﬁlters with tap sizes of 256. The dilation sizes of the
8 dilated causal convolution layers are 1, 2, 4, 8, 1, 2, 4, and 8.

(a) First FIR ﬁlter in residual FIR network(b) Sixth FIR ﬁlter in residual FIR network
3.2. Training criteria
FIRNet is a generative adversarial network (GAN) [25]. In this paper, we employ four losses as the objective for the generator: an adversarial loss with least squares criteria Ladv [26], a feature matching loss Lf m [27], a mel-spectral L1 loss Lmel for speech waveforms, and a source excitation regularization loss Lreg [11] for residual signals. The objective function LG is deﬁned as follows:
LG = Ladv + λf m Lf m + λmel Lmel + λreg Lreg ,(c) Eighth FIR ﬁlter in residual FIR network(3)

where λf m , λmel , and λreg denote the balancing hyperparametersfor Lf m , Lmel , and Lreg , respectively. We empirically set λf m ,λmel , and λreg to 2.0, 50.0, and 20.0, respectively. In addition, we
employed HiFi-GAN’s discriminator [5] according to the results ofpreliminary experiments.

(d) First FIR ﬁlter in resonance FIR network

4. EXPERIMENTAL EVALUATION
4.1. Experimental setup
In experiments, we objectively and subjectively evaluated the performance of FIRNet in the analysis–synthesis scenario. We used aJapanese male speaker from the Hi-Fi-CAPTAIN corpus [28].We
randomly selected 1,000 utterances from the parallel and nonparallel training subsets to use as training data. We used 100 validation and 100 evaluation utterances from their corresponding subsets.
We downgraded all recording data from 48 kHz and 24 bits to 24
kHz and 16 bits and normalized them to −17 dB. In feature extraction, we set the frame shift to 5.0 ms and the FFT size to 1024. In
f0 extraction, we calculated f0 contours by using DIO [13], HARVEST [29], SWIPE [30], RAPT [31], and REAPER2 . From thesecontours, we then extracted the median of f0 and the majority voteof V/UV in each frame. Spectra were calculated by CheapTrick [32]and converted to 40-dimensional MGCs whose warping coefﬁcientswere 0.466. BAP was extracted by D4C [19] and the number of
BAP dimensions was 3.
The conventional systems that we used were the WORLD synthesizer [13] and SiFi-GAN [11]. All neural vocoders used in theexperiments were constructed using the Adam optimizer [33]. The
learning rate, β1 , β2 , and ϵ were set to 0.0002, 0.5, 0.8, and 1.0 ×
10−8 , respectively. These optimizer settings were made for both
the generator and discriminator. The number of update iterations
was 500,000, and the learning rate was halved every 100,000 iterations. While the minibatch size of SiFi-GAN was the original settingin [11], that of FIRNet set to one utterance.
2 https://github.com/google/REAPER(e) Fourth FIR ﬁlter in resonance FIR network(f) Eighth FIR ﬁlter in resonance FIR network
Fig. 3. Impulse responses of FIR ﬁlters at the vowel /a/ and the corresponding input and output signals. The left and right parts of theﬁgure show the input and output signals, respectively. The central
parts of the ﬁgure show (from top to bottom) the FIR coefﬁcients,
amplitude, and group delay characteristics.
4.2. Effectiveness of FIR ﬁlters
Fig. 3 shows the impulse responses of various FIR ﬁlters and theircorresponding input and output signals. In the impulse responses of
the residual FIR network, the ﬁrst FIR ﬁlter in Fig. 3(a) dramaticallychanges the waveform shape of the source excitation signal. The
remaining FIR ﬁlters, after the ﬁrst, only slightly modify the waveform shapes of the input signals. This suggests that we may be able
to reduce the number of FIR ﬁlters in the residual FIR network.

In contrast, in the resonance FIR network, the ﬁltered waveform shapes gradually change by convolving with successive FIR
ﬁlters. In addition, the impulse responses in the resonance FIR network have shapes that differ from those in the residual FIR network.
This implies that we need to make the impulse responses as long as
possible by using many FIR ﬁlters or a FIR ﬁlter with a long tap size
to obtain ﬁne spectral envelopes.
4.3. Objective evaluation
We compared FIRNet with conventional methods in aspects ofmel-cepstral distortion (MCD), root mean squared error of log f0(RMSE), V/UV decision error (VUVE), and RTF on a single CPU(AMD EPYC 7542). We used seven f0 scaling conditions: 1.0, 0.0,
0.25, 0.5, 2.0, 4.0, and 8.0.
Table 1 shows the objective results. f0 and V/UV could not
be calculated in the f0 scaling conditions of 0.0 and 8.0. The proposed method achieved the lowest MCD of all three methods in all
f0 conditions. With respect to generation speed, FIRNet was ﬁvetimes faster than SiFi-GAN because FIRNet does not require complex processing of time-domain signals due to the classical FIR ﬁlterings. Moreover, FIRNet has a higher generation speed than theWORLD synthesizer in the cases of large f0 scaling conditions because the waveform generation algorithm in WORLD depends on
pitch intervals.

5

OriginalWORLD
SiFi-GAN
FIRNet4
MOS

Table 1. Results of objective evaluation and numbers of model parameters. The best score in each f0 condition is highlighted in bold.
Model
MCD [dB] RMSE VUVE [%]RTF
1.0 × f0
WORLD
6.87
0.029
4.47
0.106
SiFiGAN
5.85
0.037
4.16
0.576
FIRNet (proposed)
4.41
0.030
3.65
0.103
0.00 × f0
WORLD
7.46
0.115
SiFiGAN
7.90
0.591
FIRNet (proposed)
6.23
0.103
0.25 × f0
WORLD
11.21
0.073
20.61
0.100
SiFiGAN
8.06
0.096
21.14
0.566
FIRNet (proposed)
6.64
0.095
22.31
0.104
0.5 × f0
WORLD
8.39
0.038
5.03
0.102
SiFiGAN
6.74
0.056
6.59
0.569
FIRNet (proposed)
6.04
0.192
16.42
0.103
2.0 × f0
WORLD
7.19
0.026
5.15
0.12
SiFiGAN
6.70
0.097
8.18
0.574
FIRNet (proposed)
5.09
0.068
6.13
0.103
4.0 × f0
WORLD
8.82
0.065
10.53
0.138
SiFiGAN
8.22
0.258
42.22
0.572
FIRNet (proposed)
6.99
0.179
25.62
0.104
8.0 × f0
WORLD
10.30
0.180
SiFiGAN9.20
0.574
FIRNet (proposed)
8.57
0.101
Number of model parameters [M]
SiFi-GAN
11.29
FIRNet (proposed)
9.21

3
2
1
0.00

0.25

0.50 1.00 2.00 4.00 8.00
f0 scaling condition
Fig. 4. Subjective results. Error bars show 95% conﬁdence intervals.

4.4. Subjective evaluation
We conducted ﬁve-level mean opinion score (MOS) tests to evaluate
the naturalness of the synthetic speech. Using the f0 scaling conditions in Section 4.3, we compared WORLD, SiFi-GAN, FIRNet, andthe original waveforms. We employed 20 listeners, who each evaluated 12 samples for each method and each f0 scaling condition.
Fig. 4 shows the subjective results. For SiFi-GAN, although thesynthetic speech with f0 scaling conditions of 1.0, 0.5, and 2.0 had
high quality, synthetic speech with other conditions was degradeddramatically. In contrast, FIRNet keeps a similar speech quality toWORLD in the f0 scaling conditions of 0.0, 0.25, 4.0, and 8.0. In
addition, FIRNet had higher speech quality than WORLD in the f0
scale condition of 1.0. Therefore, FIRNet could achieve both goodrobustness for f0 control and speech quality of the neural vocoder.
SiFi-GAN achieved higher speech quality than FIRNet in thecase of the f0 scaling conditions of 1.0. Moreover, the speech quality of FIRNet with a f0 scaling condition of 0.5 was worse than thatgenerated by any other method. By checking some samples of generated speech, we found that some FIR coefﬁcients include slightharmonic structures. We conclude that these structures caused thisdegradation of speech quality in FIRNet. Therefore, we need to improve the speech quality further in future work.
5. CONCLUSION
This paper proposed a neural waveform generation model, FIRNet,
which achieves fast waveform generation and f0 controllability. Because the inference speed of FIRNet is comparable to that of theWORLD synthesizer on a single CPU, we believe that FIRNet canbe applied in speech generation systems such as TTS, singing voicesynthesis, and voice conversion without a bottleneck caused by processing speed. In the future, to improve speech quality, we will introduce multiple collaborative discriminators [34] and more effective
structures into FIRNet. In addition, we will apply FIRNet to fullband neural waveform generation and in various end-to-end speech
generation systems.
6. REFERENCES[1] A. van den Oord, S. Dieleman, H. Zen, K. Simonyan,
O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and
K. Kavukcuoglu, “WaveNet: A generative model for raw audio,” in Proc. SSW9, Sept. 2016, p. 125.
[2] A. Tamamori, T. Hayashi, K. Kobayashi, K. Takeda, andT. Toda, “Speaker-dependent WaveNet vocoder,” in Proc. INTERSPEECH, Aug. 2017, pp. 1118–1122.
[3] N. Kalchbrenner, E. Elsen, K. Simonyan, S. Noury,
N. Casagrande, E. Lockhart, F. Stimberg, A. van den Oord,
S. Dieleman, and K. Kavukcuoglu, “Efﬁcient neural audio synthesis,” in Proc. ICML, July 2018, pp. 2415–2424.
[4] J.-M. Valin and J. Skoglund, “LPCNet: Improving neuralspeech synthesis through linear prediction,” in Proc. ICASSP,May 2019, pp. 5826–7830.
[5] J. Kong, J. Kim, and J. Bae, “HiFi-GAN: Generative adversarial networks for efﬁcient and high ﬁdelity speech synthesis,” inProc. NeurIPS, Dec. 2020, pp. 17022–17033.
[6] N. Chen, Y. Zhang, H. Zen, R. J. Weiss, M. Norouzi, andW. Chan, “WaveGrad: Estimating gradients for waveform generation,” in Proc. ICLR, May 2021.
[7] K. Matsubara, T. Okamoto, R. Takashima, T. Takiguchi,
T. Toda, and H. Kawai, “Harmonic-net: Fundamental frequency and speech rate controllable fast neural vocoder,”IEEE/ACM Trans. on Audio, Speech, and Lang. Proc., vol. 31,
pp. 1902–1915, 2023.
[8] T. Chiba and M. Kajiyama, The Vowel, Tokyo-Kaiseikan Pub.
Co., Ltd., 1942.
[9] Y.-C. Wu, T. Hayashi, P. L. Tobing, K. Kobayashi, and T. Toda,“Quasi-Periodic WaveNet: An autoregressive raw waveformgenerative model with pitch-dependent dilated convolution
neural network,” IEEE/ACM Trans. Audio, Speech and Lang.
Proc., vol. 29, pp. 1134–1148, feb 2021.
[10] Y.-C. Wu, T. Hayashi, T. Okamoto, H. Kawai, and T. Toda,“Quasi-Periodic Parallel WaveGAN: A non-autoregressive raw
waveform generative model with pitch-dependent dilated convolution neural network,” IEEE/ACM Trans. Audio, Speechand Lang. Proc., vol. 29, pp. 792–806, Jan. 2021.
[11] R. Yoneyama, Y.-C. Wu, and T. Toda, “Source-Filter HiFiGAN: Fast and pitch controllable high-ﬁdelity neural vocoder,”in Proc. ICASSP, June 2023.
[12] R. Luo, X. Tan, R. Wang, T. Qin, J. Li, S. Zhao, E. Chen, andT.Y. Liu, “Lightspeech: Lightweight and fast text to speechwith neural architecture search,” in Proc. ICASSP, 2021, pp.
5699–5703.
[13] M. Morise, F. Yokomori, and K. Ozawa, “WORLD: A
vocoder-based highquality speech synthesis system for realtime applications,” IEICE Trans. Info. & Syst., vol. E99-D,
no. 7, pp. 1877–1884, July 2016.
[14] X. Wang, S. Takaki, and J. Yamagishi, “Neural Source-Filter
Waveform Models for Statistical Parametric Speech Synthesis,” IEEE/ACM Trans. Audio, Speech, and Language Processing, vol. 28, pp. 402–415, 2020.
[15] R. Yoneyama, Y.-C. Wu, and T. Toda, “Uniﬁed Source-Filter
GAN with Harmonic-plus-Noise Source Excitation Generation,” in Proc. Interspeech, 2022, pp. 848–852.
[16] K. Subramani, J.-M. Valin, U. Isik, P. Smaragdis, and A. Krishnaswamy, “End-to-end LPCNet: A neural vocoder with fullydifferentiable lpc estimation,” in Proc. Iinterspeech, 2022, pp.
818–822.
[17] T. Yoshimura, S. Takaki, K. Nakamura, K. Oura, Y. Hono,
K. Hashimoto, Y. Nankaku, and K. Tokuda, “Embedding a
differentiable mel-cepstral synthesis ﬁlter to a neural speechsynthesis system,” in Proc. ICASSP, 2023, pp. 1–5.

[18] T. Yoshimura, T. Fujimoto, K. Oura, and K. Tokuda, “SPTK4:
An open-source software toolkit for speech signal processing,”
in 12th ISCASpeech Synthesis Workshop (SSW 2023), 2023,
pp. 211–217.
[19] M. Morise, “D4C, A band-aperiodicity estimator for highquality speech synthesis,” Speech Comm., vol. 84, pp. 57–65,
Nov. 2016.
[20] K. Tokuda, T. Kobayashi, T. Masuko, and S. Imai, “Melgeneralized cepstral analysis,” in Proc. ICASSP, 1994, pp.
1043–1046.
[21] H. Kawahara, Jo Estill, and O. Fujimura, “Aperiodicity extraction and control using mixed mode excitation and group delay
manipulation for a high quality speech analysis, modiﬁcationand synthesis system STRAIGHT,” in MAVEBA 2001, Sept.
2001.
[22] S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I.-S. Kweon, andS. Xie, “ConvNeXt V2: Co-designing and scaling convnets
with masked autoencoders,” Proc. CVPR, pp. 16133–16142,
2023.
[23] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,”
arXiv, 2016.
[24] D. Hendrycks and K. Gimpel, “Gaussian error linear units(gelus),” arXiv, 2016.
[25] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. WardeFarley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Proc. NIPS, Dec. 2014, pp. 2672–2680.
[26] X. Mao, Q. Li, H. Xie, R. K. Lau, Z. Wang, and S. Smolley, “Least squares generative adversarial networks,” in Proc.
ICCV, Oct. 2017, pp. 2813–2821.
[27] K. Kumar, R. Kumar, T. de Boissiere, L. Gestin, W. Z. Teoh,
J. Sotelo, A. de Brébisson, Y. Bengio, and A. C Courville,“MelGAN: Generative adversarial networks for conditional
waveform synthesis,” in Proc. NeurIPS, Dec. 2019, pp. 14910–
14921.
[28] T. Okamoto, Y. Shiga, and H. Kawai, “Hi-Fi-CAPTAIN:
High-ﬁdelity and high-capacity conversational speechsynthesis corpus developed by NICT,” https://astastrec.nict.go.jp/en/release/hi-ﬁ-captain/, 2023.
[29] M. Morise, “Harvest: A high-performance fundamental frequency estimator from speech signals,” in Proc. INTERSPEECH, 2017, pp. 2321–2325.
[30] A. Camacho, “SWIPE: A sawtooth waveform inspired pitch
estimator for speech and music,” Ph.D. Thesis, University of
Florida, 2007.
[31] D. Talkin, A Robust Algorithm for Pitch Tracking (RAPT),
chapter Speech Coding & Synthesis, pp. 495–518, Elsevier,
1995.
[32] M. Morise, “CheapTrick, A spectral envelope estimator for
high-quality speech synthesis,” Speech Comm., vol. 67, pp.
1–7, Mar. 2015.
[33] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. ICLR, 2015.
[34] D. S. Dang, T. L. Nguyen, B. T. Ta, T. T. Nguyen, T. N. A.
Nguyen, D. L. L., N. M. Le, and V. H. Do, “LightVoc: An
Upsampling-Free GAN Vocoder Based On Conformer And Inverse Short-time Fourier Transform,” in Proc. Interspeech,
Aug. 2023, pp. 3043–3047.