IMPROVING FFTNET VOCODER WITH NOISE SHAPING AND SUBBAND APPROACHES
Takuma Okamoto1 , Tomoki Toda2,1 , Yoshinori Shiga1 , and Hisashi Kawai1
1

National Institute of Information and Communications Technology, Japan
2
Information Technology Center, Nagoya University, Japan
ABSTRACT

Although FFTNet neural vocoders can synthesize speech waveforms in real time, the synthesized speech quality is worse than thatof WaveNet vocoders. To improve the synthesized speech quality of
FFTNet while ensuring real-time synthesis, residual connections are
introduced to enhance the prediction accuracy. Additionally, timeinvariant noise shaping and subband approaches, which significantly
improve the synthesized speech quality of WaveNet vocoders, areapplied. A subband FFTNet vocoder with multiband input is also
proposed to directly compensate the phase shift between subbands.
The proposed approaches are evaluated through experiments using
a Japanese male corpus with a sampling frequency of 16 kHz. The
results are compared with those synthesized by the STRAIGHT
vocoder without mel-cepstral compression and those from conventional FFTNet and WaveNet vocoders. The proposed approaches
are shown to successfully improve the synthesized speech quality of
the FFTNet vocoder. In particular, the use of noise shaping enables
FFTNet to significantly outperform the STRAIGHT vocoder.
Index Terms— speech synthesis, vocoder, WaveNet, FFTNet,noise shaping, subband processing
1. INTRODUCTION
In conventional statistical parametric speech synthesis (SPSS) [1]and voice conversion (VC) [2], source-filter vocoders are typically
employed to synthesize speech waveforms from estimated and converted acoustic features. These features are mainly constructed fromthe fundamental frequency and vocal tract spectrums. To improve
the synthesized speech quality of conventional SPSS and VC overthat offered by a simple mel-log spectrum approximation (MLSA)
filter [3], several sophisticated corpus-independent vocoders havebeen developed [4–8]. Compared with these corpus-independent
vocoders, deep learning-based corpus-dependent approaches, suchas acoustic feature extraction [9], glottal vocoder [10, 11], power
spectrum reconstruction for vocoded speech [12], and speech waveform generation [13, 14] from power spectrums estimated using
the Griffin–Lim algorithm [15], have been investigated as in deeplearning-based acoustic models for SPSS and VC [16]. However,the quality of their synthesized speech is limited by the analysis
errors, approximations, and assumptions inherent in conventional
vocoders.
WaveNet [17, 18] is a neural network-based raw audio autoregressive generative approach. In text-to-speech synthesis (TTS),
WaveNet directly synthesizes raw speech waveforms from linguistic features, allowing it to outperform state-of-the-art unit selectionand SPSS-based TTS systems. Other raw audio generative models
such as SampleRNN [19] and WaveRNN [20] have also been proposed. Such raw audio generative models can realize end-to-end
TTS, converting text to raw speech waveforms. Examples include

Char2Wav [21], Deep Voice [22–24], and Tacotron 2 [25–27]. The
speech quality of English synthesized by Tacotron 2 can match thatof natural speech with a sampling frequency of 24 kHz [25].
Compared with TTS, a WaveNet-based neural vocoder that
directly synthesizes raw speech waveforms from acoustic features [28] has been used to drive conventional source-filter vocoders
within a raw audio generative model framework. Neural vocoders
based on WaveNet [28] and SampleRNN [29] have been applied to
SPSS [30–34] and VC [35–39], and also outperform conventionalsource-filter vocoders. In addition, several speaker-independent
WaveNet vocoders have been investigated [23, 24, 34, 40, 41].
Although source-filter vocoders can synthesize speech waveforms in real time, the synthesis speed of WaveNet and SampleRNN
neural vocoders remains problematic, because the sequential synthesis of each sample requires a huge number of network parameters [17, 19]. To overcome this problem, Deep Voice uses smaller
networks that quickly synthesize speech waveforms in real time.
However, there is a tradeoff between the synthesis speed and thesynthesized speech quality [22]. Parallel WaveNet [18] and WaveRNN [20] use a probability density distillation and a single-layer
recurrent neural network with sparse and subscale modifications,
respectively. These methods enable real-time synthesis with 16-bit
linear pulse code modulation (PCM) raw audio prediction without
any degradation in synthesized speech quality. However, the detailed network structures of these methods, especially the linguisticfeature input network, are not disclosed, and complicated training
and synthesis strategies might be required.
An alternative raw audio autoregressive generative model, FFTNet [42], has a simpler structure based on a 1 × 1 convolutional network and rectified liner unit (ReLU) layers. As a result, FFTNet canrealize real-time raw audio synthesis. To improve the synthesized
speech quality, four modifications have been introduced, namely,
zero padding and noise injection in the training stage, argmax sampling of voice segments, and spectral subtraction [43]. However,despite these modifications, the synthesized speech quality of FFTNet vocoders remains inferior to that of WaveNet vocoders. In particular, argmax sampling can only be used when the fundamental
frequency is known, and spectral subtraction introduces additionalmusical noise. Therefore, alternative approaches are required.
To improve the synthesized speech quality of FFTNet vocoders
while ensuring that the network model size remains small enoughfor real-time synthesis, this paper presents the following four approaches. 1) Residual connections are introduced into FFTNet to improve the prediction accuracy. 2) Time-invariant noise shaping [40,44, 45] and 3) subband approaches [46, 47], which significantly improve the synthesized speech quality in WaveNet vocoders, are directly applied to FFTNet. 4) A subband FFTNet vocoder with multiband input is proposed for the direct compensation of the phase shift
between subbands.

© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including
reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, orreuse of any copyrighted component of this work in other works.

So max

So tmax

Fu y connec ed

Fu y connected
Res dua connect on+ 1 h aye
ReLU ! 1 ⇥ 1 ! ReLU

th ayer
ReLU ! 1 ⇥ 1 ! ReLU++

<latexit sha1_base64="MNQ8Guaynm3W8HzRQJbP8gFpqH4=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIxySlpmfmVSen5pWkFtVyqWircMWk5qXABOIFlA30DMBAAZNhCGUoM0BBQL7AcoYYhhSGfIZkhlKGXIZUhjyGEiA7hyGRoRgIoxkMGQwYCoBisQzVQLEiICsTLJ/KUMvABdRbClSVClSRCBTNBpLpQF40VDQPyAeZWQzWnQy0JQeIi4A6FRhUDa4arDT4bHDCYLXBS4M/OM2qBpsBckslkE6C6E0tiOfvkgj+TlBXLpAuYchA6MLr5hKGNAYLsFszgW4vAIuAfJEM0V9WNf1zsFWQarWawSKD10D3LzS4aXAY6IO8si/JSwNTg2YzcAEjwBA9uDEZYUZ6hgZ6hoFGyg4W0KjgYJBmUGLQAIa3OYMDgwdDAEMo0N5WhtUMWxi2MrEx6TAZM5lClDIxQvUIM6AAJjsATz

1⇥1

1⇥1

1 · · · 2L l 1

2L l 1 + 1 · · · 2L l<latexit sha1_base64="Y1MAI6Ab21gSGqqqJrpIIP/nmSQ=">AAACjXichVFNSxxBEH2O8WsS3Y25BHJZXBVPS40gioQg5JAc/ciq4MoyM5Zr40zPMNO7YBb/gAevOeSUQJCQa6568eIfyMGfEDwqeMkhtbMTgopaw3S/flWv+nW3FwcqNUTnPVbvk77+gcEh++mz4ZFC8fnoaho1E5+rfhREybrnphwozVWjTMDrccJu6AW85u2+7eTXWpykKtIfzF7Mm6Hb0Gpb+a4Rql6cqnncULrtszac7NvjTqlmVMhpyRm3a6y3/mXqxTJVKIvSXeDkoIw8FqPiEWrYQgQfTYRgaBjBAVyk8m3AASEWbhNt4RJBKssz9mGLtilVLBWusLsyNmS1kbNa1p2eaab2ZZdA/kSUJUzQL/pOl3RGP+g3/bm3Vzvr0fGyJ7PX1XJcLxy8XLl+VBXKbLDzX/WgZ4NtzGVelXiPM6ZzCr+rb338dLkyvzzRnqSvdCH+v9A5ncoJdOvK/7bEy59hywM4t6/7LlidrjhUcZamywtz+VMM4hXGMCX3PYsFvMciqrLvIX7iGCdWwZqxXltvuqVWT655gRthvfsL+xSYjg==</latexit><latexit sha1_base64="Y1MAI6Ab21gSGqqqJrpIIP/nmSQ=">AAACjXichVFNSxxBEH2O8WsS3Y25BHJZXBVPS40gioQg5JAc/ciq4MoyM5Zr40zPMNO7YBb/gAevOeSUQJCQa6568eIfyMGfEDwqeMkhtbMTgopaw3S/flWv+nW3FwcqNUTnPVbvk77+gcEh++mz4ZFC8fnoaho1E5+rfhREybrnphwozVWjTMDrccJu6AW85u2+7eTXWpykKtIfzF7Mm6Hb0Gpb+a4Rql6cqnncULrtszac7NvjTqlmVMhpyRm3a6y3/mXqxTJVKIvSXeDkoIw8FqPiEWrYQgQfTYRgaBjBAVyk8m3AASEWbhNt4RJBKssz9mGLtilVLBWusLsyNmS1kbNa1p2eaab2ZZdA/kSUJUzQL/pOl3RGP+g3/bm3Vzvr0fGyJ7PX1XJcLxy8XLl+VBXKbLDzX/WgZ4NtzGVelXiPM6ZzCr+rb338dLkyvzzRnqSvdCH+v9A5ncoJdOvK/7bEy59hywM4t6/7LlidrjhUcZamywtz+VMM4hXGMCX3PYsFvMciqrLvIX7iGCdWwZqxXltvuqVWT655gRthvfsL+xSYjg==</latexit><latexit sha1_base64="clztVNVcTTKFWfx0jLYYAT5jsJo=">AAACn3ichVHLahRBFD1pX7F9ZBI3gpvWScRNhtuzMWQVcKELkUniJJF0HLprbiZFeqqb7pqB2MwP+AMu3KggIn6GIO504yKfIC4juHHhnZ4W0aDeoqpOnbrn1qmqKI11bokOp5wTJ0+dPjN91j13/sLFmdrs3EaeDDLFbZXESbYVhTnH2nDbahvzVppx2I9i3oz2b433N4ec5Tox9+1Byjv9sGf0rlahFapTWw4i7mlTKDaWs5E773tF4I28QHUTm09w82FxdzFe9EfzbsCm+zO3U6tTg8rwjgO/AnVU0UpqrxCgiwQKA/TBMLCCY4TIpW3DByEVbgeFcJkgXe4zRnBFO5AsloxQ2H0Ze7Larlgj63HNvFQrOSWWnonSwwJ9otd0RO/pDX2m73+tVZQ1xl4OZI4mWk47M48vr3/7r6ovs8XeL9U/PVvsYqn0qsV7WjLjW6iJfvjoydH68tpCcZ1e0Bfx/5wO6a3cwAy/qpervPYUrnyA/+dzHwcbzYZPDX+1WV9Zqr5iGldwDTfkvW9iBXfQQlvOfYZ3+ICPzlXntnPPaU1SnalKcwm/hfPgB0WInv8=</latexit>

1st layer<latexit sha1_base64="ErPKpUNEdKFfnQh1dUZLUnMZAd4=">AAACt3ichVE9bxNBEJ1c+AjHR0xokGgsnCAqay4NgSoSTcrEwUkkn2Xtrcf2Knu7x93aYE73B/IHKKhAQgjxB+hp+AMUaWgBUQaJhoLx+SQEETCn233zdt7s290o0SpziMcL3uKZs+fOL13wL166fGW5dnVlL7PjVFJbWm3Tg0hkpJWhtlNO00GSkogjTfvR4f3Z+v6E0kxZ88BNE+rGYmjUQEnhmOrVWqGhR9LGsTD9PIxU0Qm6eR7GkX3MqdX9WLhRfbURrBZF4YcRDZXJJRlHaeEHmatrMaXUD4nlFd2rNbCJZdRPg6ACDahi29ZeQQh9sCBhDDEQGHCMNQjI+OtAAAgJc13ImUsZqXKdoACftWOuIq4QzB7yOOSsU7GG81nPrFRL3kXzn7KyDmv4AV/jCb7HN/gFf/y1V172mHmZ8hzNtZT0lo+u737/ryrm2cHol+qfnh0MYKP0qth7UjKzU8i5fvLk6cnuvdZafgtf4Ff2/xyP8R2fwEy+yZc71HoGPj9A8Od1nwZ7680Am8HOemNzo3qKJbgBN+E23/cd2IQt2IY27/sWPsIn+Ozd9XrewBvNS72FSnMNfgvv4U/gkasT</latexit><latexit sha1_base64="YZNQ6PhZPSD898Nbq1aHfrioKkY=">AAAC0HichVE9b9RAEB2br2A+ckCDRGNxDqI67aZJRBWJhoIiuXBJpPPpZO9NLqvz7lr23pFgWYg2f4CCikgRQvwMGloKivwERBmkNBSMfZYQRMCsdnfmzbzZt7txmsjcMnbiuBcuXrp8ZeGqd+36jZuLrVu3t3IzzQT2hElMthNHOSZSY89Km+BOmmGk4gS348njKr89wyyXRj+zBykOVDTWcleKyBI0bE1Cjc+FUSrSoyKMZdnng6IIVWz2KTTJSEV2zw/aPCjL0gtjHEtdCNQWs9Lr4tOeH4TWBH7A/dBKhbnPgwaqsl6I1LepH7barMNq8887vHHa0Ni6ab2DEEZgQMAUFCBosOQnEEFOow8cGKSEDaAgLCNP1nmEEjziTqkKqSIidELrmKJ+g2qKq555zRZ0SkIzI6YPS+wLe89O2Sf2gX1lP/7aq6h7VFoOaI/nXEyHi4d3N8/+y1K0W9j7xfqnZgu7sFprlaQ9rZHqFmLOn714fbr5qLtUPGBH7Bvpf8tO2Ee6gZ59F8cb2H0DHn0A//O5zztbyx3OOnxjub222nzFAtyD+/CQ3nsF1uAJrEOPzv0MZw44jtt1992X7qt5qes0nDvwm7mHPwELorBg</latexit>

F g 2 P oposed FFTNe w h es dua connec ons<latexit sha1_base64="y/JEGzEcRJVjWGGpkIreLhSNWRI=">AAACp3ichVG7btRAFD0xr2AeWUKDlMZiEwRCWY23IUoVQUOBlBe7WSkOK3v2ZhnFO7bs2ZWC5R/gByioQEIkymekoUoXQT4BUQaJhoK7thGCCLiWZ86ce8+dMzNBHKrUCHEyYZ07f+HipcnL9pWr165P1W5Mt9NomEhqySiMkk7gpxQqTS2jTEidOCF/EIS0Eew8Guc3RpSkKtJPzW5MWwO/r9W2kr5hqlt76AXUVzqTpA0luT3bfJY9mQ/n3fy+62Sekzue7EUmLXGZzGdtj3Tvp6Zbq4uGKMI5C9wK1FHFSlR7Dw89RJAYYgCChmEcwkfK3yZcCMTMbSFjLmGkijwhh83aIVcRV/jM7vDY59VmxWpej3umhVryLiH/CSsdzIljsS9OxQdxID6L73/tlRU9xl52eQ5KLcXdqZe31r/9VzXg2eD5L9U/PRtsY6Hwqth7XDDjU8hSP3rx6nR9cW0uuyPeii/s/404EYd8Aj36Kt+t0tpr2PwA7p/XfRa0mw1XNNzVZn1poXqKSczgNu7yfT/AEh5jBS3edw9H+IhP1j1r2WpbnbLUmqg0N/FbWP4PRHyh5w==</latexit>n SPSS and VC acous c ea u es o vocode s a e yp ca y
ana yzed eve y 5 ms Some ad us men o he me eso u on beween speech wave o m x and acous c ea u es h s hen equ ed
n WaveNe vocode s a s mp e app oach o ma ch ng he sequenceeng hs o x and h nvo ves copy ng h n each ame acco d ng
o he sh amoun o he ana ys s w ndow 28 40 47
n FFT
Ne vocode s a nea n e po a on me hod s app ed 42 Basedon emp ca esu s ansposed convo u on 17 s app ed o up
samp ng he acous c ea u es n bo h he WaveNe and FFTNe
vocode s a he han he s mp e copy and nea n e po a on ap
p oaches The base ne L aye FFTNe vocode s uc u e nves
ga ed n h s pape s us a ed n F g 1 whe e he ecep ve fie d s
2L samp es

ReLU ! 1 ⇥ 1 ! ReLU+
<latexit sha1_base64="MNQ8Guaynm3W8HzRQJbP8gFpqH4=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIxySlpmfmVSen5pWkFtVyqWircMWk5qXABOIFlA30DMBAAZNhCGUoM0BBQL7AcoYYhhSGfIZkhlKGXIZUhjyGEiA7hyGRoRgIoxkMGQwYCoBisQzVQLEiICsTLJ/KUMvABdRbClSVClSRCBTNBpLpQF40VDQPyAeZWQzWnQy0JQeIi4A6FRhUDa4arDT4bHDCYLXBS4M/OM2qBpsBckslkE6C6E0tiOfvkgj+TlBXLpAuYchA6MLr5hKGNAYLsFszgW4vAIuAfJEM0V9WNf1zsFWQarWawSKD10D3LzS4aXAY6IO8si/JSwNTg2YzcAEjwBA9uDEZYUZ6hgZ6hoFGyg4W0KjgYJBmUGLQAIa3OYMDgwdDAEMo0N5WhtUMWxi2MrEx6TAZM5lClDIxQvUIM6AAJjsATzeVPw==</latexit><latexit

1⇥1

1⇥1

1 · · · 2L 1

2L 1 + 1 · · · 2L

<latexit sha1_base64="Y1MAI6Ab21gSGqqqJrpIIP/nmSQ=">AAACjXichVFNSxxBEH2O8WsS3Y25BHJZXBVPS40gioQg5JAc/ciq4MoyM5Zr40zPMNO7YBb/gAevOeSUQJCQa6568eIfyMGfEDwqeMkhtbMTgopaw3S/flWv+nW3FwcqNUTnPVbvk77+gcEh++mz4ZFC8fnoaho1E5+rfhREybrnphwozVWjTMDrccJu6AW85u2+7eTXWpykKtIfzF7Mm6Hb0Gpb+a4Rql6cqnncULrtszac7NvjTqlmVMhpyRm3a6y3/mXqxTJVKIvSXeDkoIw8FqPiEWrYQgQfTYRgaBjBAVyk8m3AASEWbhNt4RJBKssz9mGLtilVLBWusLsyNmS1kbNa1p2eaab2ZZdA/kSUJUzQL/pOl3RGP+g3/bm3Vzvr0fGyJ7PX1XJcLxy8XLl+VBXKbLDzX/WgZ4NtzGVelXiPM6ZzCr+rb338dLkyvzzRnqSvdCH+v9A5ncoJdOvK/7bEy59hywM4t6/7LlidrjhUcZamywtz+VMM4hXGMCX3PYsFvMciqrLvIX7iGCdWwZqxXltvuqVWT655gRthvfsL+xSYjg==</latexit><latexit sha1_base64="Y1MAI6Ab21gSGqqqJrpIIP/nmSQ=">AAACjXichVFNSxxBEH2O8WsS3Y25BHJZXBVPS40gioQg5JAc/ciq4MoyM5Zr40zPMNO7YBb/gAevOeSUQJCQa6568eIfyMGfEDwqeMkhtbMTgopaw3S/flWv+nW3FwcqNUTnPVbvk77+gcEh++mz4ZFC8fnoaho1E5+rfhREybrnphwozVWjTMDrccJu6AW85u2+7eTXWpykKtIfzF7Mm6Hb0Gpb+a4Rql6cqnncULrtszac7NvjTqlmVMhpyRm3a6y3/mXqxTJVKIvSXeDkoIw8FqPiEWrYQgQfTYRgaBjBAVyk8m3AASEWbhNt4RJBKssz9mGLtilVLBWusLsyNmS1kbNa1p2eaab2ZZdA/kSUJUzQL/pOl3RGP+g3/bm3Vzvr0fGyJ7PX1XJcLxy8XLl+VBXKbLDzX/WgZ4NtzGVelXiPM6ZzCr+rb338dLkyvzzRnqSvdCH+v9A5ncoJdOvK/7bEy59hywM4t6/7LlidrjhUcZamywtz+VMM4hXGMCX3PYsFvMciqrLvIX7iGCdWwZqxXltvuqVWT655gRthvfsL+xSYjg==</latexit><latexit sha1_base64="sWI3cQ8QhDEZBQ/rziC5m6lyeXM=">AAACm3ichVHLahRBFD1po8b2kYluBAkMTiJuHG7PQkNWgWxEssjDSQLpOHTX3IxFeqqb7pqBpJkfyA+40I2CiPgXuhGSrYt8griM4MaFd3o6BA3qLarq1Kl7bp2qCpNIZ5boeMy5MH7x0uWJK+7Va9dvTFambq5ncS9V3FRxFKebYZBxpA03rbYRbyYpB90w4o1wd3G4v9HnNNOxeWr3Et7uBh2jd7QKrFCtykM/5I42uWJjOR24M14196sDX7VjmxWw8SxfeuANZlyfTfs0r1WpUZ2KqJ4HXglqKGM5rryDjzZiKPTQBcPACo4QIJO2BQ+ERLht5MKlgnSxzxjAFW1PslgyAmF3ZezIaqtkjayHNbNCreSUSHoqyipm6Qu9pxP6TB/oK/38a628qDH0sidzONJy0po8uL3247+qrswWz89U//RssYO5wqsW70nBDG+hRvr+/ouTtfnV2fwevaFv4v81HdMnuYHpf1dvV3j1JVz5AO/P5z4P1ht1j+reSqO2MFd+xQTu4C7uy3s/wgIeYxlNOfcVPuIQR860s+g8cZZGqc5YqbmF38Jp/gK8DJ3+</latexit><latexit sha1_base64="3QyFTltxfHkCzrTeXlsVgbzxcTw=">AAACoXichVFBa9RAFP4aq9ZY7dZehF4WtxVBXCZ7aSk9FHpR8LDdum1hU5dk9nUdmp2EZHahhvwB/4AHTxZESn9GL70q9NCfIB4rePHgSzZStKgvZOab773vzTczfhSoxAhxPmFdm7x+4+bULfv29J27M5XZe1tJOIwltWUYhPGO7yUUKE1to0xAO1FM3sAPaNvfX8/z2yOKExXqF+Ygot2B19dqT0nPMNWtrLo+9ZVOJWlDcWYvNF6mz5842WOnmrrVzJW90CQFzBPZgu2S7v2q7lZqoi6KqF4FTglqKKMZVj7CRQ8hJIYYgKBhGAfwkPDXgQOBiLldpMzFjFSRJ2SwWTvkKuIKj9l9Hvu86pSs5nXeMynUkncJ+I9ZWcWiOBNH4kKcimPxRfz4a6+06JF7OeDZH2sp6s68ub/5/b+qAc8Gry5V//RssIflwqti71HB5KeQY/3o9duLzZXWYvpQHIqv7P+9OBcnfAI9+iY/bFDrHWx+AOfP674Kthp1R9SdjUZtbbl8iinM4wEe8X0vYQ1P0USb9z3EKT7hs1WznllNqzUutSZKzRx+C6vzEwvMoDk=</latexit>

One-hot embedding<latexit sha1_base64="JkH+ghQ8YQivlqNGb+/YrCuhm0Y=">AAACknichVE9T9xAEH2YfIBJwgUoItGcOBGl4TSmAVGBaCgi8ZUDJA6dbN9wt8JeW/beSXDiD9BSUFAlUoQQvyAtNPwBCn5ClJJIaSiY8zmKEpRkLO++fTNv9u2uFwcqNUS3fVb/k6fPng8M2kMvXr4aLrwe2UijVuJzxY+CKNny3JQDpblilAl4K07YDb2AN729xW5+s81JqiL9wezHvBO6Da12le8aoWqFqarHDaU7PmvDyaG9rHmqGZkihx7X60o37Crr+s90rVCiMmVRfAycHJSQx0pUOEMVdUTw0UIIhoYRHMBFKt82HBBi4XbQES4RpLI84xC2aFtSxVLhCrsnY0NW2zmrZd3tmWZqX3YJ5E9EWcQk3dA53dE1XdBXuv9rr07Wo+tlX2avp+W4Nnz0Zv3Hf1WhzAbNX6p/ejbYxWzmVYn3OGO6p/B7+vbByd363Npk5y19om/i/yPd0pWcQLe/+59Xee0UtjyA8+d1PwYb02WHys7qdGl+Nn+KAYxjAu/kvmcwjyWsoCL7HuMLLnFljVlz1oK12Cu1+nLNKH4L6/0D/I6bsQ==</latexit>µ-law encoded 2L audio samples x<latexit sha1_base64="H+cB9AoDAbSCv/VFvxxznylSS+A=">AAAC33ichVE7b9RAEB4bCME8zoQGicbiHETDaX0NEVWkNBQUeXBJpPNxWtuTyyr2rmWvLw/LNRIFokOICiSEED+Dhj9AkY4WUQaJhoKxzxKCCJjV7s58O9/Mt7tBGotcM3ZsmGfOnps7P3/Bunjp8pWOfXVhM1dFFuIgVLHKtgOeYywkDrTQMW6nGfIkiHEr2Fupz7emmOVCyYf6MMVRwidS7IiQa4LG9pEvcT9UScJlVPqBqIbeqCz9JFAHFKo4Srjeddyu51ZVZfkBToQsQ5Qas8py/aRw78R830EZqggjx+0/Kh9UrsOLSCgn50kaY+64dWXnoHItH6lNSx/bXdZjjTmnHa91utDaqrLfgg8RKAihgAQQJGjyY+CQ0xiCBwxSwkZQEpaRJ5pzhAos4haUhZTBCd2jdULRsEUlxXXNvGGH1CWmmRHTgUX2ib1jJ+wje8++sB9/rVU2NWoth7QHMy6m486T6xvf/8tKaNew+4v1T80admCp0SpIe9og9S3CGX969Pxk4976YnmLvWZfSf8rdsw+0A3k9Fv4Zg3XX4JFH+D9+dynnc1+z2M9b63fXV5qv2IebsBNuE3vfReW4T6swoD6fjbmjI5hm9x8bD41n81STaPlXIPfzHzxE4Idttw=</latexit>

1⇥1

1⇥1

1 · · · 2L 1

2L 1 + 1 · · · 2L

<latexit sha1_base64="Y1MAI6Ab21gSGqqqJrpIIP/nmSQ=">AAACjXichVFNSxxBEH2O8WsS3Y25BHJZXBVPS40gioQg5JAc/ciq4MoyM5Zr40zPMNO7YBb/gAevOeSUQJCQa6568eIfyMGfEDwqeMkhtbMTgopaw3S/flWv+nW3FwcqNUTnPVbvk77+gcEh++mz4ZFC8fnoaho1E5+rfhREybrnphwozVWjTMDrccJu6AW85u2+7eTXWpykKtIfzF7Mm6Hb0Gpb+a4Rql6cqnncULrtszac7NvjTqlmVMhpyRm3a6y3/mXqxTJVKIvSXeDkoIw8FqPiEWrYQgQfTYRgaBjBAVyk8m3AASEWbhNt4RJBKssz9mGLtilVLBWusLsyNmS1kbNa1p2eaab2ZZdA/kSUJUzQL/pOl3RGP+g3/bm3Vzvr0fGyJ7PX1XJcLxy8XLl+VBXKbLDzX/WgZ4NtzGVelXiPM6ZzCr+rb338dLkyvzzRnqSvdCH+v9A5ncoJdOvK/7bEy59hywM4t6/7LlidrjhUcZamywtz+VMM4hXGMCX3PYsFvMciqrLvIX7iGCdWwZqxXltvuqVWT655gRthvfsL+xSYjg==</latexit><latexit sha1_base64="sWI3cQ8QhDEZBQ/rziC5m6lyeXM=">AAACm3ichVHLahRBFD1po8b2kYluBAkMTiJuHG7PQkNWgWxEssjDSQLpOHTX3IxFeqqb7pqBpJkfyA+40I2CiPgXuhGSrYt8griM4MaFd3o6BA3qLarq1Kl7bp2qCpNIZ5boeMy5MH7x0uWJK+7Va9dvTFambq5ncS9V3FRxFKebYZBxpA03rbYRbyYpB90w4o1wd3G4v9HnNNOxeWr3Et7uBh2jd7QKrFCtykM/5I42uWJjOR24M14196sDX7VjmxWw8SxfeuANZlyfTfs0r1WpUZ2KqJ4HXglqKGM5rryDjzZiKPTQBcPACo4QIJO2BQ+ERLht5MKlgnSxzxjAFW1PslgyAmF3ZezIaqtkjayHNbNCreSUSHoqyipm6Qu9pxP6TB/oK/38a628qDH0sidzONJy0po8uL3247+qrswWz89U//RssYO5wqsW70nBDG+hRvr+/ouTtfnV2fwevaFv4v81HdMnuYHpf1dvV3j1JVz5AO/P5z4P1ht1j+reSqO2MFd+xQTu4C7uy3s/wgIeYxlNOfcVPuIQR860s+g8cZZGqc5YqbmF38Jp/gK8DJ3+</latexit><latexit sha1_base64="Y1MAI6Ab21gSGqqqJrpIIP/nmSQ=">AAACjXichVFNSxxBEH2O8WsS3Y25BHJZXBVPS40gioQg5JAc/ciq4MoyM5Zr40zPMNO7YBb/gAevOeSUQJCQa6568eIfyMGfEDwqeMkhtbMTgopaw3S/flWv+nW3FwcqNUTnPVbvk77+gcEh++mz4ZFC8fnoaho1E5+rfhREybrnphwozVWjTMDrccJu6AW85u2+7eTXWpykKtIfzF7Mm6Hb0Gpb+a4Rql6cqnncULrtszac7NvjTqlmVMhpyRm3a6y3/mXqxTJVKIvSXeDkoIw8FqPiEWrYQgQfTYRgaBjBAVyk8m3AASEWbhNt4RJBKssz9mGLtilVLBWusLsyNmS1kbNa1p2eaab2ZZdA/kSUJUzQL/pOl3RGP+g3/bm3Vzvr0fGyJ7PX1XJcLxy8XLl+VBXKbLDzX/WgZ4NtzGVelXiPM6ZzCr+rb338dLkyvzzRnqSvdCH+v9A5ncoJdOvK/7bEy59hywM4t6/7LlidrjhUcZamywtz+VMM4hXGMCX3PYsFvMciqrLvIX7iGCdWwZqxXltvuqVWT655gRthvfsL+xSYjg==</latexit>

3 IMPROVING FFTNET VOCODER BY INTRODUCINGRESIDUAL CONNECTIONS AND SIGNAL PROCESSING
APPROACHES<latexit sha1_base64="3QyFTltxfHkCzrTeXlsVgbzxcTw=">AAACoXichVFBa9RAFP4aq9ZY7dZehF4WtxVBXCZ7aSk9FHpR8LDdum1hU5dk9nUdmp2EZHahhvwB/4AHTxZESn9GL70q9NCfIB4rePHgSzZStKgvZOab773vzTczfhSoxAhxPmFdm7x+4+bULfv29J27M5XZe1tJOIwltWUYhPGO7yUUKE1to0xAO1FM3sAPaNvfX8/z2yOKExXqF+Ygot2B19dqT0nPMNWtrLo+9ZVOJWlDcWYvNF6mz5842WOnmrrVzJW90CQFzBPZgu2S7v2q7lZqoi6KqF4FTglqKKMZVj7CRQ8hJIYYgKBhGAfwkPDXgQOBiLldpMzFjFSRJ2SwWTvkKuIKj9l9Hvu86pSs5nXeMynUkncJ+I9ZWcWiOBNH4kKcimPxRfz4a6+06JF7OeDZH2sp6s68ub/5/b+qAc8Gry5V//RssIflwqti71HB5KeQY/3o9duLzZXWYvpQHIqv7P+9OBcnfAI9+iY/bFDrHWx+AOfP674Kthp1R9SdjUZtbbl8iinM4wEe8X0vYQ1P0USb9z3EKT7hs1WznllNqzUutSZKzRx+C6vzEwvMoDk=</latexit>

Transposed convolution for upsampling<latexit sha1_base64="OGNtnLEBfh9kGYzTTmV+Idq4qs4=">AAAC03ichVFNaxRBEK2dRI3jRzZ6EbwMbiKelp5cDJ4CXjzma5PAzrL09NZumkx3Dz09a+IwF/EkePbgSUFE/Ble/AF6yE8QjxEE8WDN7IBoUKvp7qrX9apfdcdpIjPH2EnLm5s/d/7CwkX/0uUrVxfbS9d2M5NbgT1hEmP3Y55hIjX2nHQJ7qcWuYoT3IsP71fne1O0mTR6xx2nOFB8ouVYCu4IGrZ1pPGhMEpxPSqiWJb9cFAUkYrNEYUmGSnuDoLlTrhclqUfxTiRuhCoHdrS37FcZ6nJcBQIo6cmyauiwdjYIE8zrqgBPfEjpNINZdjusC6rLTjrhI3TgcY2TPsNRDACAwJyUICgwZGfAIeMRh9CYJASNoCCMEuerM8RSvCJm1MWUgYn9JDWCUX9BtUUVzWzmi3oloSmJWYAK+wTe8tO2Qf2jn1mP/5aq6hrVFqOaY9nXEyHi09vbH/7L0vR7uDgF+ufmh2MYa3WKkl7WiNVF2LGnz56frp9b2uluM1esS+k/yU7Ye+pAz39Kl5v4tYL8OkDwj+f+6yzu9oNWTfcXO2srzVfsQA34Rbcofe+C+vwADagR/d+hO+tuda81/MK77H3ZJbqtRrOdfjNvGc/Abl8taI=</latexit>

Acoustic features h<latexit sha1_base64="RQVTbKgtq/zpmMrvWbdf8pD+IKM=">AAACsXichVG7btRAFD0xr7A8skCDRGOxCaJaXQMSEVUQDQVFHmwStF4t49nZ7Ghtj2WPlwTLP8APUFCBhBCi4xdo+AEkUkKHKINEQ8FdryUEEXAtz5x75p47Z2aCJNSZJdqfc44cPXb8xPzJxqnTZ84uNM+d38xMnkrVkSY06XYgMhXqWHWstqHaTlIloiBUW8H4znR9a6LSTJv4vt1LVC8SO7EeaiksU/3mPT9Wj6SJIhEPCj/QZdfrFYUfBWaXUxMOImFH7mLLWyzLsuFbtWuL29LkmdXSHSph81RlbsnKYlT2my1qUxXuYeDVoIU6Vk3zFXwMYCCRI4JCDMs4hEDGXxceCAlzPRTMpYx0ta5QosHanKsUVwhmxzzucNat2Zjzac+sUkveJeQ/ZaWLJfpAr+mA3tMb+kI//tqrqHpMvezxHMy0KukvPLm48f2/qohni9Ev1T89WwyxXHnV7D2pmOkp5Ew/efz0YOPW+lJxhV7QV/b/nPbpHZ8gnnyTL9fU+jM0+AG8P6/7MNi81vao7a3daK0s108xj0u4jKt83zexgrtYRYf3fYuP+ITPznXngfPQCWalzlytuYDfwhn/BKjiqQQ=</latexit>

F g 1 Base ne FFTNe vocode s uc u e w h L aye s
2 WAVENET AND FFTNET NEURAL VOCODERSG ven he acous c ea u es h he WaveNe and FFTNe neu a
vocode s 28 42 mode he cond ona p obab y d s bu onp(x h) o he aw aud o wave o m x = x(1)x(T ) asp(x h) =TY=1

p(x(t) x(1)x(t − 1) h)

1n WaveNe Eq 1 s mode ed by a s ack o d a ed causa convo
u on aye s a ow ng he e fic en npu o ve y ong aud o sam
p es w h e a ve y ew aye s Howeve he ne wo k mode s ze o
WaveNe vocode s s soo a ge o syn hes ze speech wave o ms
n ea meTo s gn fican y educe he ne wo k mode s ze FFTNe uses
s mp e 1 × 1 convo u on aye s ns ead o he d a ed causa convo
u on aye s and can he e o e syn hes ze speech wave o ms n ea
me w h a as gene a on a go hm 48
Ra he han a con nuous d s bu on he WaveNe and FFT
Ne mode s ou pu a ca ego ca d s bu on o he nex samp e x(t)h ough a fina so max aye Th s app oach s e a ve y flex b e
and can eas y mode a b a y d s bu ons a hough aw wave o m
npu s a e yp ca y ea ed as con nuous va ues n van a WaveNeand FFTNe a µ aw compand ng a go hm defined n G 711 49 s
n oduced and aw aud o wave o ms a e quan zed n o one o 256
poss b e va ues

The s mp e ne wo k s uc u e o FFTNe enab es speech wave o ms
o be syn hes zed n ea me a hough he syn hes zed speech qua
y s no as good as ha o WaveNe To mp ove he syn hes zed
speech qua y o FFTNe wh e e a n ng a ne wo k mode s ze haa ows ea me syn hes s a ne wo k s uc u e mod fica on and wo
s gna p ocess ng me hods me nva an no se shap ng and subband p ocess ng a e now desc bed
3 1 In roduc ng res dua connec ons
To mp ove he FFTNe mode wh e ensu ng ea me syn hes s
wo ne wo k mod fica ons nsp ed by he WaveNe mode s uc u e
a e nves ga ed
The fi s s he n oduc on o sk p connec ons om a aye s
as used n WaveNe Howeve emp ca esu s nd ca e ha h scanno mp ove he FFTNe p ed c on accu acy and so s no usedn he expe men s
The o he mod fica on n oduces es dua connec ons n aaye s an app oach ha s a so emp oyed n WaveNe as shownn F g 2 Expe men a esu s see Sec 4 demons a e ha h s
mod fica on can s gn fican y mp ove he p ed c on accu acy andsyn hes zed speech qua y o FFTNe
3 2 FFTNe w h me nvar an no se shap ng me hod
The speech s gna s gene a ed by WaveNe o en su e om no secaused by p ed c on e o s and hese no se s gna s end o cause
a ge spec a d s o ons n h gh equency bands Thus he no ses gna s deg ade he syn hes zed speech qua y 44
To educe he adve se e ec s o he no se s gna s gene a ed by
neu a vocode s p ed c ve pu se code modu a on PPCM 50
based me nva an no se shap ng wh ch s a pe cep ua we gh ng

(a) Trainingstage(a) (a)Training
procedure
Training
procedure<latexit sha1_base64="mK1mqAOOttCffizbzSK39lI/LiU=">AAACfXichVFNSxtBGH6ytprGqlEvgpelIUVBwrteEnoK9OIxfsQIKmF2ncQhm91ldxJIQ/wB/QMePCmKSGn/hJf+gR78CaVHC70U6pvNQmmD+g4z88wz7/POMzN24KpIE92ljIkXLyen0q8y069nZuey8wu7kd8JHVl1fNcP92wRSVd5sqqVduVeEErRtl1Zs1vvh/u1rgwj5Xs7uhfIw7ZoeqqhHKGZqmeX+gd2w1wRq+ZOKJSnvKYZadGUg3o2RwWKwxwHVgJySKLiZ69xgCP4cNBBGxIeNGMXAhG3fVggBMwdos9cyEjF+xIDZFjb4SzJGYLZFo9NXu0nrMfrYc0oVjt8iss9ZKWJPH2jG7qnr/SJvtPvR2v14xpDLz2e7ZFWBvW5j0vbv55VtXnWOP6retKzRgOl2Kti70HMDG/hjPTdD6f32++28v23dEE/2P853dEt38Dr/nSuNuXWGTL8Adb/zz0OdtcLFhWszfVcuZR8RRrLeIMVfu8iythABVU+9wSX+IwvqT9G3lgzCqNUI5VoFvFPGMUHQSuTCQ==</latexit>

Amplitude
Amplitude
Amplitude
Amplitude
Amplitude

features
AuxiliaryfeaturesExa on o Auxiliary
Extracting
Extracting
featuresa Extracting
ouExtracting
a u Auxiliary
acoustic
featuresacoustic
features
Auxiliaryfeaturesacoustic
featuresacoustic
features
Filtering
Filtering
Filtering
Filtering
Filtering<latexit sha1_base64="JekowpVGRq8ye6Mlmpqdb77uHBw=">AAACcHichVHLSsNAFD2N7/qquhFcWC0VcVFupKC4Ety49FUr1FKSOK3BNInJtFiLP+APuHCjgoj4GW78ARd+grizghsX3qYB0aLeYWbOnLnnzpkZ3bVMXxI9RZSOzq7unt6+aP/A4NBwbGR023cqniEyhmM53o6u+cIybZGRprTEjusJraxbIqsfrDT3s1Xh+aZjb8maK/JlrWSbRdPQJFP54q4UR7Iez60e508KsQSlKIh4O1BDkEAYa07sBrvYgwMDFZQhYEMytqDB55aDCoLLXB515jxGZrAvcIIoayucJThDY/aAxxKvciFr87pZ0w/UBp9icfdYGUeSHumWGvRAd/RMH7/Wqgc1ml5qPOstrXALw6fjm+//qso8S+x/qf70LFHEYuDVZO9uwDRvYbT01eOzxubSRrI+Q1f0wv4v6Ynu+QZ29c24Xhcb54jyB6g/n7sdbM+nVEqp6+nEcjr8il5MYBqz/N4LWMYq1pDhcw9xhgtcRl6VcWVSmWqlKpFQM4Zvocx9AhSijwM=</latexit><latexit sha1_base64="Cn+BUEKd8xdKIt3Q5PRco3fP2Ow=">AAACbHichVG7SgNBFD1ZXzE+Eh+FIIIYFKtwVwTFKmJj6SuJoiK7m4ku7ovdSSAGf8DWwkItFETEz7DxByz8BBFsIthYeLNZEBX1Ljtz5sw9956Z0T3LDCTRY0xpaW1r74h3Jrq6e3qTqb7+fOCWfUPkDNdy/XVdC4RlOiInTWmJdc8Xmq1boqDvLzT2CxXhB6brrMmqJ7ZtbdcxS6ahSaY25m1uIstFsZNKU4bCGP0J1AikEcWSm7rGFopwYaAMGwIOJGMLGgL+NqGC4DG3jRpzPiMz3Bc4RIK1Zc4SnKExu8/jLq82I9bhdaNmEKoN7mLx77NyFOP0QDdUp3u6pSd6/7VWLazR8FLlWW9qhbeTPBpafftXZfMssfep+tOzRAmzoVeTvXsh0ziF0dRXDk7qq3Mr47UJuqRn9n9Bj3THJ3Aqr8bVslg5RYIfQP1+3T9BfiqjUkZdnkpnp6OniGMYY5jk+55BFotYQo772jjGGc5jL8qgMqyMNFOVWKQZwJdQJj4AhTONZQ==</latexit><latexit sha1_base64="DaKj1+ZEHV9X2j1UBMbfE4Y+o1k=">AAACcHichVG7SgNBFD1Z3/GRqI1gYTRExCLctVGsAjaWaowKGsLuOsYh+3J3E4jBH/AHLGyMICJ+ho0/YOEniJ0KNhbebBZEg3qHmTlz5p47Z2Z015R+QPQYU7q6e3r7+gfig0PDI4nk6NiW71Q9QxQMx3S8HV3zhSltUQhkYIod1xOapZtiW6+stPa3a8LzpWNvBnVXFC2tbMsDaWgBU8W8K4RxmPJl2dbMUjJNWQoj1QnUCKQRxZqTvMYe9uHAQBUWBGwEjE1o8LntQgXBZa6IBnMeIxnuC5wgztoqZwnO0Jit8Fjm1W7E2rxu1fRDtcGnmNw9VqaQoQe6oVe6p1t6oo9fazXCGi0vdZ71tla4pcTpRP79X5XFc4DDL9WfngMcYCn0Ktm7GzKtWxhtfe347DW/vJFpzNIlPbP/Jj3SHd/Arr0ZV+ti4xxx/gD153N3gq2FrEpZdX0hnVuKvqIfk5jBHL/3InJYxRoKfO4RznCBZuxFmVCmlOl2qhKLNOP4Fsr8J82LjuI=</latexit>

Amplitude

Sourcesignalf [Hz]signal
Sourcef [Hz]
Speechsignal
signalSourceSource signal

Amplitude
Amplitude

Calculating
Calculating<latexit sha1_base64="Cn+BUEKd8xdKIt3Q5PRco3fP2Ow=">AAACbHichVG7SgNBFD1ZXzE+Eh+FIIIYFKtwVwTFKmJj6SuJoiK7m4ku7ovdSSAGf8DWwkItFETEz7DxByz8BBFsIthYeLNZEBX1Ljtz5sw9956Z0T3LDCTRY0xpaW1r74h3Jrq6e3qTqb7+fOCWfUPkDNdy/XVdC4RlOiInTWmJdc8Xmq1boqDvLzT2CxXhB6brrMmqJ7ZtbdcxS6ahSaY25m1uIstFsZNKU4bCGP0J1AikEcWSm7rGFopwYaAMGwIOJGMLGgL+NqGC4DG3jRpzPiMz3Bc4RIK1Zc4SnKExu8/jLq82I9bhdaNmEKoN7mLx77NyFOP0QDdUp3u6pSd6/7VWLazR8FLlWW9qhbeTPBpafftXZfMssfep+tOzRAmzoVeTvXsh0ziF0dRXDk7qq3Mr47UJuqRn9n9Bj3THJ3Aqr8bVslg5RYIfQP1+3T9BfiqjUkZdnkpnp6OniGMYY5jk+55BFotYQo772jjGGc5jL8qgMqyMNFOVWKQZwJdQJj4AhTONZQ==</latexit><latexit sha1_base64="3VQFYMVMvz+mj0rqWVDBhPRNH08=">AAACinichVG7TtxAFD2YR4h5LdAgpUGsNkq1ut4GAg0SCFHyWkBi0co2l2WEd2zZsyvBih+AD0iRKpEQilKlDSUNP0DBJ0QpQaKh4K7XURRQkmt57pkz99w5M+NFgUoM0W2X1d3T2/eq/7U9MDg0PJIbHdtMwkbsc9kPgzDe9tyEA6W5bJQJeDuK2a17AW95hwvt9a0mx4kK9YY5ini37ta02le+a4Sq5goVj2tKt3zWhuMTe0kFkpWu2RXWe7/oai5PRUpj8iVwMpBHFith7gIV7CGEjwbqYGgYwQFcJPLtwAEhEm4XLeFiQSpdZ5zAFm1DqlgqXGEPZazJbCdjtczbPZNU7csugfyxKCdRoBv6Qnd0TV/pBz3+tVcr7dH2ciTZ62g5qo6cTqw//FdVl2xw8Fv1T88G+5hJvSrxHqVM+xR+R988/nC3PrtWaL2lz/RT/H+iW7qSE+jmvX++ymsfYcsDOM+v+yXYLBUdKjqrpfz8TPYU/XiDKbyT+57GPJaxgrLse4Zv+I5La9AqWe+tuU6p1ZVpxvFHWItP7oeYrA==</latexit>f [Hz]
ff [Hz][Hz]

Amplitude
Amplitude

Acoustic features<latexit sha1_base64="ID1VvDnLcgTfuzPowmtqCf7PM38=">AAACknichVE9T9xAEH2YEIiBcIQUkdKcOBGl4TSmAVGBaFKk4OsAiUMne5m7rPCtLXt9EjnxB2hTpEhFJISi/IK0ockfoOAnRClBokmROZ8RSlCSsbz79s282be7QRzq1BJdDjiDD4YeDo88ckfHxh9PlCafbKVRliiuqSiMkp3ATznUhmtW25B34oT9dhDydnCw0stvdzhJdWQ27WHMe22/ZXRTK98K1SjN1gNuadNVbCwnR+6yirLUalVusm+zhFO3zmb/Nt0oVahKeZTvA68AFRSxGpXOUMc+IihkaINhYAWH8JHKtwsPhFi4PXSFSwTpPM84givaTKpYKnxhD2RsyWq3YI2sez3TXK1kl1D+RJRlzNAFfaIr+kaf6Tv9/Guvbt6j5+VQ5qCv5bgxcfxs4+a/qrbMFm/uVP/0bNHEQu5Vi/c4Z3qnUH195+37q43F9ZnuC/pIP8T/CV3SuZzAdK7V6Rqvf4ArD+D9ed33wdZc1aOqtzZXWVoonmIEzzGNl3Lf81jCK6yiJvu+wxd8xbnz1Fl0lp2VfqkzUGim8Fs4r38BxwKcEg==</latexit>

Quantization
Quantization
Quantization
Quantization
Quantization<latexit sha1_base64="AxCp/0ADlFNJuGzcCQj4MYvpTXM=">AAACjXichVFNSxtRFD2ZftnpR6JuBDehweIq3BFKpYgILnRptImCkTAzXtOHkzfDzEtAg3+gC7cuXCmIlG671Y0b/4CL/ARxacGNi95MppRW2t5h3jvv3HvuO+89LwpUYoh6OevR4ydPnw09t1+8fPU6XxgeqSVhO/a56odBGK95bsKB0lw1ygS8FsXstryAV73t+X5+tcNxokL90exEvNFym1ptKd81QjUKk3WPm0p3fdaG4z270na1Ubtp1q6z3vyZaRRKVKY0ig+Bk4ESslgKC6eoYxMhfLTRAkPDCA7gIpFvHQ4IkXAb6AoXC1JpnrEHW7RtqWKpcIXdlrEpq/WM1bLu90xStS+7BPLHoixigq7oC93SJX2la7r/a69u2qPvZUdmb6DlqJH/PLZy919VS2aDT79U//RssIXp1KsS71HK9E/hD/Sd3YPblQ/LE923dEw34v+IenQhJ9Cd7/5JhZcPYcsDOH9e90NQmyo7VHYqU6W56ewphjCON5iU+36POSxiCVXZdx/fcIZzK2+9s2as2UGplcs0o/gtrIUfVteaLQ==</latexit>f [Hz]
ff [Hz][Hz]<latexit sha1_base64="JekowpVGRq8ye6Mlmpqdb77uHBw=">AAACcHichVHLSsNAFD2N7/qquhFcWC0VcVFupKC4Ety49FUr1FKSOK3BNInJtFiLP+APuHCjgoj4GW78ARd+grizghsX3qYB0aLeYWbOnLnnzpkZ3bVMXxI9RZSOzq7unt6+aP/A4NBwbGR023cqniEyhmM53o6u+cIybZGRprTEjusJraxbIqsfrDT3s1Xh+aZjb8maK/JlrWSbRdPQJFP54q4UR7Iez60e508KsQSlKIh4O1BDkEAYa07sBrvYgwMDFZQhYEMytqDB55aDCoLLXB515jxGZrAvcIIoayucJThDY/aAxxKvciFr87pZ0w/UBp9icfdYGUeSHumWGvRAd/RMH7/Wqgc1ml5qPOstrXALw6fjm+//qso8S+x/qf70LFHEYuDVZO9uwDRvYbT01eOzxubSRrI+Q1f0wv4v6Ynu+QZ29c24Xhcb54jyB6g/n7sdbM+nVEqp6+nEcjr8il5MYBqz/N4LWMYq1pDhcw9xhgtcRl6VcWVSmWqlKpFQM4Zvocx9AhSijwM=</latexit>f [Hz]
Residual
signalResidual
signalResidualf [Hz]signalTraining
Training ofTraining
WaveNetTraining
WaveNetWaveNet / FFTNet
Training
WaveNetWaveNet<latexit sha1_base64="Azd+72x9Z3Zft0FZGjZDm7tnYds=">AAACn3ichVFBTxNBFP5YRXFFqXIh8bLaYDjVt1wknEhMwBMp0FIIS5rd5bVO2M5udqdNsOEP+Ac8eFESQ4g/w8R404sHfoLxiIkXDrxulxglwJvMzDffvO/NNzNBEqnMEB2PWDdujt66PXbHvjt+7/5E6cHD9SzupiHXwziK043AzzhSmutGmYg3kpT9ThBxI9h9Mdhv9DjNVKxrZi/h7Y7f1qqlQt8I1SzNewG3le6HrA2n+3Yt9ZVWuu3ELc+zG36Pl9k4z5zFxZoA22O9c57bLJWpQnk4F4FbgDKKqMalQ3jYQYwQXXTA0DCCI/jIpG3BBSERbht94VJBKt9n7MMWbVeyWDJ8YXdlbMtqq2C1rAc1s1wdyimR9FSUDqbpBx3RCX2lT/STTi+t1c9rDLzsyRwMtZw0J95Mrf25VtWR2eDVX9W
sha1_base64="Azd+72x9Z3Zft0FZGjZDm7tnYds=">AAACn3ichVFBTxNBFP5YRXFFqXIh8bLaYDjVt1wknEhMwBMp0FIIS5rd5bVO2M5udqdNsOEP+Ac8eFESQ4g/w8R404sHfoLxiIkXDrxulxglwJvMzDffvO/NNzNBEqnMEB2PWDdujt66PXbHvjt+7/5E6cHD9SzupiHXwziK043AzzhSmutGmYg3kpT9ThBxI9h9Mdhv9DjNVKxrZi/h7Y7f1qqlQt8I1SzNewG3le6HrA2n+3Yt9ZVWuu3ELc+zG36Pl9k4z5zFxZoA22O9c57bLJWpQnk4F4FbgDKKqMalQ3jYQYwQXXTA0DCCI/jIpG3BBSERbht94VJBKt9n7MMWbVeyWDJ8YXdlbMtqq2C1rAc1s1wdyimR9FSUDqbpBx3RCX2lT/STTi+t1c9rDLzsyRwMtZw0J95Mrf25VtWR2eDVX9WVng1amMu9KvGe5MzgFuFQ33v99mRtfnW6/5QO6Jf4/0DH9FluoHu/w48rvPoOtnyA+/9zXwTrsxWXKu7KbHlhrviKMTzCE8zIez/HAl6iirqc+x5f8A3frcfWkrVsVYep1kihmcQ/YW2eAWTwn/0=</latexit>

Residual
signalResidual
signal<latexit sha1_base64="W0AMd2SFuzIBLnf0k4RLimj/aPE=">AAACkHichVFNSxxBEH2OSTSTD1e9BLwsWQyBwFLjxcWLihfxpGtWBVeWmdly09jbM8z0LujiH/DmKYecFCSE/IBcDbnkD+TgTxCPBrx4SO3shJBIkhqm+/WretWvu4NYq9QSXQw5w/fuPxgZfeg+evzk6VhhfGIjjTpJyLUw0lGyFfgpa2W4ZpXVvBUn7LcDzZvB3lI/v9nlJFWReW33Y95p+y2jdlXoW6EahVf1gFvK9EI2lpNDt8qpanZ8XUxVy/jarbNp/kw2CiUqUxbFu8DLQQl5rEaF96ijiQghOmiDYWAFa/hI5duGB0Is3A56wiWCVJZnHMIVbUeqWCp8YfdkbMlqO2eNrPs900wdyi5a/kSURUzTN/pA1/SVPtIl3f61Vy/r0feyL3Mw0HLcGDt6tn7zX1VbZos3v1T/9Gyxi0rmVYn3OGP6pwgH+u7B2+v1uep07wWd0pX4P6EL+iInMN3v4dkaV9/BlQfw/rzuu2BjpuxR2VubKS1U8qcYxRSe46Xc9ywWsIxV1GTfY3zCOT47E07FmXcWB6XOUK6ZxG/hrPwAki2bGw==</latexit>

WaveNetWaveNetWav N / FFTN
WaveNetWaveNetf [Hz]f [Hz]f [Hz]
ff [Hz][Hz]<latexit sha1_base64="JekowpVGRq8ye6Mlmpqdb77uHBw=">AAACcHichVHLSsNAFD2N7/qquhFcWC0VcVFupKC4Ety49FUr1FKSOK3BNInJtFiLP+APuHCjgoj4GW78ARd+grizghsX3qYB0aLeYWbOnLnnzpkZ3bVMXxI9RZSOzq7unt6+aP/A4NBwbGR023cqniEyhmM53o6u+cIybZGRprTEjusJraxbIqsfrDT3s1Xh+aZjb8maK/JlrWSbRdPQJFP54q4UR7Iez60e508KsQSlKIh4O1BDkEAYa07sBrvYgwMDFZQhYEMytqDB55aDCoLLXB515jxGZrAvcIIoayucJThDY/aAxxKvciFr87pZ0w/UBp9icfdYGUeSHumWGvRAd/RMH7/Wqgc1ml5qPOstrXALw6fjm+//qso8S+x/qf70LFHEYuDVZO9uwDRvYbT01eOzxubSRrI+Q1f0wv4v6Ynu+QZ29c24Xhcb54jyB6g/n7sdbM+nVEqp6+nEcjr8il5MYBqz/N4LWMYq1pDhcw9xhgtcRl6VcWVSmWqlKpFQM4Zvocx9AhSijwM=</latexit>

Ca time-invariant
u time-invariant
aCalculating
on o mnva annoise
Calculating noise
noweighting
hap filterngfiweightingfiltertime-invariant
noiseTime-Invariant
noiseTime-Invariantnoise
T m nva anweightingfilterweightingfilterTime-Invariant
noiseTime-Invariant
noise now gh ng fi
weightingfilterweighting filter

time-invariant noise
weightingfilterweightingfilter(b) (b)Synthesis
procedure
Synthesis
procedure
WaveNet(b)(b)
Synthesis
procedure
WaveNetSynthesis
procedure(b) Synthesis
stage
WaveNetWaveNetAuxiliaryfeatures
Predicting
Auxiliaryfeatures
Predicting
Dequantization
Dequantization
Auxiliaryfeatures
Acoustic
featuresresidual
residual
Predicting
G signal
n signal
a on oAuxiliaryfeatures
Predicting
Dequantization
D quan a on
Dequantization
residual
signaldua
gna of noise
residual
signalTime-invariant
noise
shapingfilter(inverse
weightingfilterfilter
Time-invariant noise shaping filter (inverse of noise
weighting
WaveNet/
FFTNetTime-invariant
noise
shapingfilter(inverseof noiseweightingfilterTime-invariant
noise
shapingfilter(inverseof noiseweightingfilter

Amplitude
Amplitude
Amplitude
Amplitude
Amplitude<latexit sha1_base64="WFpprREJHKk2M8jMxWebF7USDuM=">AAACkXichVFNTxRBEH2MCjh8rXIx8bJhA+FANjVc+Dht5GLiBVgXSBiymWmKpcNsz2SmdxPc8Ac8cvHgSRNjjH+AK3rxD3jgJxiOmHjxYO3sEKNEqcl0v35Vr/p1d5hEOrNEF0POnbv3hkdG77tj4xOTU6UHD7eyuJMqbqg4itOdMMg40oYbVtuId5KUg3YY8XZ4tNbPb3c5zXRsntvjhPfaQcvoA60CK1SztOCH3NKmp9hYTk/cesKsDsu+76o4TTqZ67PZv842SxWqUh7lm8ArQAVFrMel9/CxjxgKHbTBMLCCIwTI5NuFB0Ii3B56wqWCdJ5nnMAVbUeqWCoCYY9kbMlqt2CNrPs9s1ytZJdI/lSUZczSV/pAV/SFPtI3+vnPXr28R9/LsczhQMtJc+rlo/qPW1VtmS0Of6v+69niAMu5Vy3ek5zpn0IN9N0Xr67qq5uzvTl6S5fi/w1d0Gc5gel+V+82ePM1XHkA7+/rvgm2FqseVb2NxUptuXiKUTzGDOblvpdQw1OsoyH7nuIM5/jkTDsrTs15Mih1hgrNNP4I59kvvX6bJA==</latexit><latexit sha1_base64="Cn+BUEKd8xdKIt3Q5PRco3fP2Ow=">AAACbHichVG7SgNBFD1ZXzE+Eh+FIIIYFKtwVwTFKmJj6SuJoiK7m4ku7ovdSSAGf8DWwkItFETEz7DxByz8BBFsIthYeLNZEBX1Ljtz5sw9956Z0T3LDCTRY0xpaW1r74h3Jrq6e3qTqb7+fOCWfUPkDNdy/XVdC4RlOiInTWmJdc8Xmq1boqDvLzT2CxXhB6brrMmqJ7ZtbdcxS6ahSaY25m1uIstFsZNKU4bCGP0J1AikEcWSm7rGFopwYaAMGwIOJGMLGgL+NqGC4DG3jRpzPiMz3Bc4RIK1Zc4SnKExu8/jLq82I9bhdaNmEKoN7mLx77NyFOP0QDdUp3u6pSd6/7VWLazR8FLlWW9qhbeTPBpafftXZfMssfep+tOzRAmzoVeTvXsh0ziF0dRXDk7qq3Mr47UJuqRn9n9Bj3THJ3Aqr8bVslg5RYIfQP1+3T9BfiqjUkZdnkpnp6OniGMYY5jk+55BFotYQo772jjGGc5jL8qgMqyMNFOVWKQZwJdQJj4AhTONZQ==</latexit>

Training
Training
SpeechcorpusTraining
corpusTraining
corpuscorpuscorpusAmplitude
Amplitude
Amplitude
Amplitude
Amplitude(a)(a)
Training
procedure
Training
procedure<latexit sha1_base64="5pPIW82VZY/glz+6aZRHYYeFDCM=">AAACmnichVHLahRBFD1pX7F9ZDQbIVk0DpG4GW4HgsFVMBslmySTSQKZMHR37kyK9FQ3XTUDYzM/4A+4yEpBJPgVko2YtYt8griM4MaFd3paRIN6i6o6deqeW6eqwjRWxhKdTTiXLl+5em3yunvj5q3bU5U7d7dM0ssibkRJnGQ7YWA4VpobVtmYd9KMg24Y83Z4uDLa3+5zZlSiN+0g5b1u0NGqraLACtWqLDZD7iidR6wtZ0M3b4Ztbz586NUH2h6wUcYzNujw0G2y3v+Z1qpUqUZFeBeBX4IqylhLKm/RxD4SROihC4aGFRwjgJG2Cx+EVLg95MJlglSxzxjCFW1PslgyAmEPZezIardktaxHNU2hjuSUWHomSg9z9ImO6Zw+0Dv6TN//Wisvaoy8DGQOx1pOW1Mv7tW//VfVldni4Jfqn54t2lgqvCrxnhbM6BbRWN9//vK8/nhjLn9Ar+mL+H9FZ3QiN9D9r9Gbdd44gisf4P/53BfB1kLNp5q/vlBdXiq/YhIzuI95ee9HWMZTrKEh5x7hPT7i1Jl1njjPnNVxqjNRaqbxWzibPwAbi561</latexit><latexit sha1_base64="HoMLaaKrInn74ntzFJlCJE2q5gM=">AAACkXichVFNSxxBEH2OGnXy4UYvQi6Li8FD2NR4UXNaFEQIBL/WFVxZZsZybZztGWZ6F3TxD3j04iGnCEEkfyDXxEv+QA7+BPGokEsOqZ0dCVGS1DDdr1/Vq37d7UWBSgzRZY/V29f/aGBwyH785Omz4dzzkfUkbMY+l/0wCOMNz004UJrLRpmAN6KY3YYXcMXbm+/kKy2OExXqNbMf8VbDrWu1o3zXCFXLvap6XFe67bM2HB/aFbfF79jkX+cXFtYE2FXW23fZWq5ARUoj/xA4GSggi6Uwd4YqthHCRxMNMDSM4AAuEvk24YAQCbeFtnCxIJXmGYewRduUKpYKV9g9Geuy2sxYLetOzyRV+7JLIH8syjwm6Dud0w19o090RT//2qud9uh42ZfZ62o5qg0fja3++K+qIbPB7m/VPz0b7GAm9arEe5QynVP4XX3r4ORm9c3KRPslndK1+P9Al3QhJ9CtW//jMq+8hy0P4Ny/7odgfaroUNFZniqUZrKnGMQLjGNS7nsaJSxiCWXZ9xif8QVfrVFr1ipZc91SqyfTjOKPsN7+Ama9mn4=</latexit>

Filtering
Filtering
FilteringnvfiFilteringng

f [Hz]f [Hz]Reconstructed
Reconstructed
ff [Hz][Hz]f [Hz]
sourcesignalsourcesignalReconstructed
Reconstructed
Reconstructed
speech signal
sourcesignalsource signal<latexit sha1_base64="Cn+BUEKd8xdKIt3Q5PRco3fP2Ow=">AAACbHichVG7SgNBFD1ZXzE+Eh+FIIIYFKtwVwTFKmJj6SuJoiK7m4ku7ovdSSAGf8DWwkItFETEz7DxByz8BBFsIthYeLNZEBX1Ljtz5sw9956Z0T3LDCTRY0xpaW1r74h3Jrq6e3qTqb7+fOCWfUPkDNdy/XVdC4RlOiInTWmJdc8Xmq1boqDvLzT2CxXhB6brrMmqJ7ZtbdcxS6ahSaY25m1uIstFsZNKU4bCGP0J1AikEcWSm7rGFopwYaAMGwIOJGMLGgL+NqGC4DG3jRpzPiMz3Bc4RIK1Zc4SnKExu8/jLq82I9bhdaNmEKoN7mLx77NyFOP0QDdUp3u6pSd6/7VWLazR8FLlWW9qhbeTPBpafftXZfMssfep+tOzRAmzoVeTvXsh0ziF0dRXDk7qq3Mr47UJuqRn9n9Bj3THJ3Aqr8bVslg5RYIfQP1+3T9BfiqjUkZdnkpnp6OniGMYY5jk+55BFotYQo772jjGGc5jL8qgMqyMNFOVWKQZwJdQJj4AhTONZQ==</latexit><latexit sha1_base64="ID1VvDnLcgTfuzPowmtqCf7PM38=">AAACknichVE9T9xAEH2YEIiBcIQUkdKcOBGl4TSmAVGBaFKk4OsAiUMne5m7rPCtLXt9EjnxB2hTpEhFJISi/IK0ockfoOAnRClBokmROZ8RSlCSsbz79s282be7QRzq1BJdDjiDD4YeDo88ckfHxh9PlCafbKVRliiuqSiMkp3ATznUhmtW25B34oT9dhDydnCw0stvdzhJdWQ27WHMe22/ZXRTK98K1SjN1gNuadNVbCwnR+6yirLUalVusm+zhFO3zmb/Nt0oVahKeZTvA68AFRSxGpXOUMc+IihkaINhYAWH8JHKtwsPhFi4PXSFSwTpPM84givaTKpYKnxhD2RsyWq3YI2sez3TXK1kl1D+RJRlzNAFfaIr+kaf6Tv9/Guvbt6j5+VQ5qCv5bgxcfxs4+a/qrbMFm/uVP/0bNHEQu5Vi/c4Z3qnUH195+37q43F9ZnuC/pIP8T/CV3SuZzAdK7V6Rqvf4ArD+D9ed33wdZc1aOqtzZXWVoonmIEzzGNl3Lf81jCK6yiJvu+wxd8xbnz1Fl0lp2VfqkzUGim8Fs4r38BxwKcEg==</latexit><latexit sha1_base64="JekowpVGRq8ye6Mlmpqdb77uHBw=">AAACcHichVHLSsNAFD2N7/qquhFcWC0VcVFupKC4Ety49FUr1FKSOK3BNInJtFiLP+APuHCjgoj4GW78ARd+grizghsX3qYB0aLeYWbOnLnnzpkZ3bVMXxI9RZSOzq7unt6+aP/A4NBwbGR023cqniEyhmM53o6u+cIybZGRprTEjusJraxbIqsfrDT3s1Xh+aZjb8maK/JlrWSbRdPQJFP54q4UR7Iez60e508KsQSlKIh4O1BDkEAYa07sBrvYgwMDFZQhYEMytqDB55aDCoLLXB515jxGZrAvcIIoayucJThDY/aAxxKvciFr87pZ0w/UBp9icfdYGUeSHumWGvRAd/RMH7/Wqgc1ml5qPOstrXALw6fjm+//qso8S+x/qf70LFHEYuDVZO9uwDRvYbT01eOzxubSRrI+Q1f0wv4v6Ynu+QZ29c24Xhcb54jyB6g/n7sdbM+nVEqp6+nEcjr8il5MYBqz/N4LWMYq1pDhcw9xhgtcRl6VcWVSmWqlKpFQM4Zvocx9AhSijwM=</latexit><latexit sha1_base64="KB2z9I76/yW89h6Mf9iuEgOuNWs=">AAACnHichVHLShxBFD22eWjn4cRsBCEMGQxZDbfdKFkJughIQMfMKDgydNdcx8Ke6qarZsAM/kB+IIvgIoEQQj7DjQtdZuEniEsD2WSROz0dQiJJblFVp07dc+tUVZTG2jqi8zFv/MbNW7cnJv07d+/dnyo9mG7YpJcprqskTrKtKLQca8N1p13MW2nGYTeKeTPaXx7ub/Y5szoxL91ByjvdsGP0rlahE6pVWmhG3NFmoNg4zg79GqvEWJf1lON22abMaq9sdceEsd9k0/6Z2CpVqEp5lK+DoAAVFLGWlD6iiTYSKPTQBcPACY4RwkrbRgBCKtwOBsJlgnS+zziEL9qeZLFkhMLuy9iR1XbBGlkPa9pcreSUWHomyjLm6At9ois6oc90Qd//WmuQ1xh6OZA5Gmk5bU29ntn49l9VV2aHvV+qf3p22MVi7lWL9zRnhrdQI33/1ZurjWe1ucETek+X4v8dndOx3MD0v6oP61x7C18+IPjzua+Dxnw1oGqwPl9ZWiy+YgKzeIyn8t4LWMJzrKEu5x7hGKc48x55K96q92KU6o0Vmof4LbzGD8j2oFc=</latexit>

F g 3 T me nva an no se shap ng me hod o WaveNe and FFTNe neu a vocode sechn que has been app ed o WaveNe 40 44 Th s no se shapng me hod s expec ed o mp ove he syn hes zed speech qua yo FFTNe vocode s and s he e o e d ec y app ed o FFTNe A
b ock d ag am o me nva an no se shap ng s dep c ed n F g 3
S m a o WaveNe me hods 40 44 a me gene a zed ceps um 51 s emp oyed o ca cu a e he no se shap ng fi e The
ans e unc on o he fi e s g ven byMcX
H(z) = s−1cγ (0) +βcγ (m)z̃ −m2γs−1γ (ω) =

m=1

1

(1 + γω) γeω

0< γ ≤1
γ=0

3

whe e cγ (m) γ β and Mc a e he m h me gene a zed ceps acoe fic en s a powe pa ame e o he me gene a zed ceps um apa ame e o con o no se ene gy n he o man eg ons and heo de o he me gene a zed ceps um espec ve y z̃ −1 s he fi so de a pass unc on g ven byz̃ −1 =z −1 − α
1 − αz −1

4

whe e α s he equency wa p ng pa ame e
The ave aged me gene a zed ceps a coe fic en s a e ca cu
a ed n advance ove aames ex ac ed om he a n ng da a
and a e hen used o he me nva an no se shap ng fi e ng mp e
men ed by he MLSA fi e 3 n he a n ng s age F g 3 an he
syn hes s s age he nve se fi e s eas y de ved by mu p y ng heme gene a zed ceps a coe fic en s by −1 and he econs uc edspeech s gna s fina y ob a ned as shown n F g 3 b
3 3 Subband FFTNe
Fo ap d syn hes s and mp oved syn hes zed speech qua y a subband WaveNe based on mu a e s gna p ocess ng 52 53 has been
p oposed 46 47 By n oduc ng an ove apped s ng e s deband
SSB fi e bank based on a squa e oo Hann w ndow 46 47 subband WaveNe ach eves be e p ed c on accu acy compa ed w hu band WaveNe esu ng n acce e a ed syn hes s speed and mp oved syn hes zed speech qua y Thus he subband app oach s
a so d ec y app ed o FFTNe

A b ock d ag am o he p oposed subband FFTNe vocode s
us a ed n F g 4 n he a n ng s age u band speech waveo ms x = x(1)x(T ) w h a samp ng equency o f s
a e dec ma ed by a ac o o M and decomposed n o N subbands eams xn = xn (1)xn (T /M ) o sho eng h T /M andow samp ng equency f s/M by an ove apped SSB ana ys s fi ebank Each subband FFTNe ne wo k pn (xn h) s hen sepa a e yand e fic en y a ned by each subband wave o m xn w h common
acous c ea u es h n he syn hes s s age each subband s eam
x̂n = x̂n (1)x̂n (T /M ) s s mu aneous y gene a ed by hea ned ne wo k and upsamp ed by a ac o o M and he syn hes zed speech wave o m w h a samp ng equency o f s s ob a ned
by an ove apped SSB syn hes s fi e bank The p oposed subband
me hod can be comb ned w h he no se shap ng app oach and s
eva ua ed n he expe men s epo ed n Sec 4n he subband WaveNe vocode 47 he e s a phase sh
be ween subbands because each es ma ed samp e x̂n (t) s ndepen
den y gene a ed om a eady es ma ed pas samp es x̂n (1)x̂n (t − 1) and acous c ea u es h w h andom samp ng based onpn (xn h) Thus a max mum co e a on based phase sh com
pensa on be ween subbands s n oduced o he subband WaveNe
vocode Howeve he esu s o u he nves ga ons nd ca e hahe max mum co e a on based app oach canno mp ove he synhes zed speech qua y when ansposed convo u on s used o up
samp e he acous c ea u es The e o e he max mum co e a on
based app oach s no nc uded n he subband FFTNe vocode andano he phase sh compensa on app oach s p oposed n he nex
subsec on
3 4 Subband FFTNe w h mu band npu
To d ec y compensa e he phase sh be ween subbands w h n a
neu a ne wo k amewo k a subband FFTNe vocode w h muband npu s p oposed n h s app oach he n h band a n ng
and syn hes s uses o he band wave o ms as we as he n h band
un ke he subband me hod w h s ng e band npu desc bed n
Sec 3 3 The same s a egy s used n seve a o he me hods 1
p xe CNN 54 55 o cond ona co o mage gene a on whe ea b ue p xe s p ed c ed om he p ev ous ed g een and b ue
p xe s 2 he neu a pa ame c s ng ng syn hes ze 56 57 whe eFullband waveform x

Acoustic features h<latexit sha1_base64="6UplFInBxIwkw0BVbFG3LuUOb5A=">AAACvHichVG7btRAFD0xr7ABsoEGicZiE0S1GiMkohQoEQ1lHmw20nq1Gs/OZkfr8Vj2eCFY/oH8AAUVSAghvoGKhh+gSE2FgCpINCm49lpCEAHX8sy5Z+65c2YmiEOVWsaO5pwzZ8+dvzB/sbFw6fKVxebS1d3UZImQHWFCk+wFPJWhimTHKhvKvTiRXAeh7AaTB+V6dyqTVJnokT2IZV/z/UiNlOCWqEGz60fysTBa82iY+4Eqel4/z30dmCeUmnCouR27yy1vuSiKhl9mic43hMlSq4Sb+27hjiS3WSLToszKJu64GDRbrM2qcE8DrwYt1LFpmq/hYwgDgQwaEhEs4RAcKX09eGCIiesjJy4hpKp1iQIN0mZUJamCEzuhcZ+yXs1GlJc900otaJeQ/oSULlbYR/aGHbMP7C37zE7+2iuvepReDmgOZloZDxYPr+/8+K9K02wx/qX6p2eLEVYrr4q8xxVTnkLM9NOnz4531rZX8lvsJftC/l+wI/aeThBNv4tXW3L7ORr0AN6f130a7N5pe6ztbd1tra/WTzGPG7iJ23Tf97COh9hEh/Z9h0/4im/OfWfoTBw9K3Xmas01/BbO9Cc1K60+</latexit>

Acoustic features h<latexit sha1_base64="6UplFInBxIwkw0BVbFG3LuUOb5A=">AAACvHichVG7btRAFD0xr7ABsoEGicZiE0S1GiMkohQoEQ1lHmw20nq1Gs/OZkfr8Vj2eCFY/oH8AAUVSAghvoGKhh+gSE2FgCpINCm49lpCEAHX8sy5Z+65c2YmiEOVWsaO5pwzZ8+dvzB/sbFw6fKVxebS1d3UZImQHWFCk+wFPJWhimTHKhvKvTiRXAeh7AaTB+V6dyqTVJnokT2IZV/z/UiNlOCWqEGz60fysTBa82iY+4Eqel4/z30dmCeUmnCouR27yy1vuSiKhl9mic43hMlSq4Sb+27hjiS3
sha1_base64="6UplFInBxIwkw0BVbFG3LuUOb5A=">AAACvHichVG7btRAFD0xr7ABsoEGicZiE0S1GiMkohQoEQ1lHmw20nq1Gs/OZkfr8Vj2eCFY/oH8AAUVSAghvoGKhh+gSE2FgCpINCm49lpCEAHX8sy5Z+65c2YmiEOVWsaO5pwzZ8+dvzB/sbFw6fKVxebS1d3UZImQHWFCk+wFPJWhimTHKhvKvTiRXAeh7AaTB+V6dyqTVJnokT2IZV/z/UiNlOCWqEGz60fysTBa82iY+4Eqel4/z30dmCeUmnCouR27yy1vuSiKhl9mic43hMlSq4Sb+27hjiS3WSLToszKJu64GDRbrM2qcE8DrwYt1LFpmq/hYwgDgQwaEhEs4RAcKX09eGCIiesjJy4hpKp1iQIN0mZUJamCEzuhcZ+yXs1GlJc900otaJeQ/oSULlbYR/aGHbMP7C37zE7+2iuvepReDmgOZloZDxYPr+/8+K9K02wx/qX6p2eLEVYrr4q8xxVTnkLM9NOnz4531rZX8lvsJftC/l+wI/aeThBNv4tXW3L7ORr0AN6f130a7N5pe6ztbd1tra/WTzGPG7iJ23Tf97COh9hEh/Z9h0/4im/OfWfoTBw9K3Xmas01/BbO9Cc1K60+</latexit>

Fullband waveform x

Overlapped SSB analysis filterbank

Overlapped SSB analysis filterbank<latexit sha1_base64="hz3YcsLZyBQE4R2v8gIf+sob3fE=">AAACh3ichVE9TxtBEH1cAgFDsAkNEs0JC0TlzFEQRMVHQ8eHMSARZO0da1h5fXe6W1syVvqIP0BBBRKKIiRa6GnyByj4CVFKkGgoMj5OQoCAOd3u2zfzZt/uuqFWsSG67rA+fOzs+tTdk+nt+9yfzQ18WYuDeuTJkhfoINpwRSy18mXJKKPlRhhJUXO1XHer8+38ekNGsQr8VdMM5VZN7PiqojxhmCrnRhY5q0UYym27WJyzhS90M1axXVHayMgVfrWcy1OBkrBfAicFeaSxFOR+4Tu2EcBDHTVI+DCMNQRi/jbhgBAyt4UWcxEjleQlfiDD2jpXSa4QzFZ53OHVZsr6vG73jBO1x7to/iNW2hilK/pNN/SHTukv3b/aq5X0aHtp8uw+aGVYzu4PFe/eVdV4Nth9VL3p2aCCqcSrYu9hwrRP4T3oG3sHN8XpldHWGB3TP/Z/RNd0ySfwG7feybJcOUSGH8B5ft0vwdpEwaGCszyRn5lKn6IbwxjBON/3N8xgAUso8b4/cYZzXFg91ldr0kprrY5UM4gnYc3+B80EltA=</latexit><latexit sha1_base64="hz3YcsLZyBQE4R2v8gIf+sob3fE=">AAACh3ichVE9TxtBEH1cAgFDsAkNEs0JC0TlzFEQRMVHQ8eHMSARZO0da1h5fXe6W1syVvqIP0BBBRKKIiRa6GnyByj4CVFKkGgoMj5OQoCAOd3u2zfzZt/uuqFWsSG67rA+fOzs+tTdk+nt+9yfzQ18WYuDeuTJkhfoINpwRSy18mXJKKPlRhhJUXO1XHer8+38ekNGsQr8VdMM5VZN7PiqojxhmCrnRhY5q0UYym27WJyzhS90M1axXVHayMgVfrWcy1OBkrBfAicFeaSxFOR+4Tu2EcBDHTVI+DCMNQRi/jbhgBAyt4UWcxEjleQlfiDD2jpXSa4QzFZ53OHVZsr6vG73jBO1x7to/iNW2hilK/pNN/SHTukv3b/aq5X0aHtp8uw+aGVYzu4PFe/eVdV4Nth9VL3p2aCCqcSrYu9hwrRP4T3oG3sHN8XpldHWGB3TP/Z/RNd0ySfwG7feybJcOUSGH8B5ft0vwdpEwaGCszyRn5lKn6IbwxjBON/3N8xgAUso8b4/cYZzXFg91ldr0kprrY5UM4gnYc3+B80EltA=</latexit>

#M

#M

···

#M

#M

#M

···

#M

x1

x2

···

xN

x1

x2

···

xN

FFTNetfor 2nd band···

FFTNetfor N -th band

FFTNeo 1 bandFFTNeo 2nd band

···

FFTNe
o N h band

FFTNetfor 1st band<latexit sha1_base64="UzFSnDwdXjGUDDDdSyiYCnF0Ufo=">AAACn3ichVHPS9xAFP5MbdVtrdt6Ebxsuyo9LS+loHgSCrYnWX+sWjbLkmRHHUwyIZndakP+Af+BHnpRQaT0zxBKb+2lB/8E8ajgxYNvs4Gi0vaFzHzvm/e9+WbGCT0Za6LTPuNB/8NHA4NDhcdPhp+OFJ89X41VO3JFzVWeitYdOxaeDERNS+2J9TAStu94Ys3ZfttdX+uIKJYqWNG7oWj49mYgN6Rra6aaxVkrEB9d5ft20EosR6Z1s5Eklu+oHU6V1/JtvVWaKJsTaZoWLC12dDI/v7IgdNoslqlCWZTuAzMHZeRRVcVjWGhBwUUbPgQCaMYebMT81WGCEDLXQMJcxEhm6wIpCqxtc5XgCpvZbR43OavnbMB5t2ecqV3exeM/YmUJk/SbvtIF/aBvdEbXf+2VZD26XnZ5dnpaETZH9saWr/6r8nnW2Pqj+qdnjQ3MZF4lew8zpnsKt6fvfPp8sTy7NJlM0SGds/8DOqUTPkHQuXSPFsXSFxT4Acy7130frL6umFQxF9+U52bypxjEOF7iFd/3NObwHlXUeN99fMdP/DJeGO+MBaPaKzX6cs0oboXx4QZ6FqD2</latexit><latexit sha1_base64="/CNpbZaNxrRZBopTINvRZmAQksM=">AAACpXichVHPS9xAFP5Ma7VbW9f2UugldNX2tLyUgtLTgpdeCv7aVdgsS5Kd1cEkE5LZVRvyD/gP9NBTC1XEP6OXHnq14J8gHi148dC32YCoaF/IzPe+ed+bb2bcyJeJJjoZMR48HH00Nv649GTi6bPJ8tTzRqJ6sSfqnvJVvO46ifBlKOpaal+sR7FwAtcXa+7WwmB9rS/iRKpwVe9GohU4G6HsSs/RTLXLNTsU254KAifspLYrs6bVSlM7cNUOp8rvBI7eNKcr1nSWZSVbix2ddlVsWok2XdZk7XKFqpSHeRtYBaigiEVVPoCNDhQ89BBAIIRm7MNBwl8TFggRcy2kzMWMZL4ukKHE2h5XCa5wmN3icYOzZsGGnA96Jrna4118/mNWmpihYzqkc/pFR3RKl3f2SvMeAy+7PLtDrYjak3svVy7+qwp41ti8Ut3rWaOL+dyrZO9RzgxO4Q31/c9fzlc+LM+ks/Sdztj/Nzqhn3yCsP/X+7Eklr+ixA9g3bzu26DxrmpR1Vp6X6nNF08xjld4jbd833Oo4SMWUed99/Ebx/hjvDE+GatGY1hqjBSaF7gWRvsfQiujXw==</latexit><latexit sha1_base64="UzFSnDwdXjGUDDDdSyiYCnF0Ufo=">AAACn3ichVHPS9xAFP5MbdVtrdt6Ebxsuyo9LS+loHgSCrYnWX+sWjbLkmRHHUwyIZndakP+Af+BHnpRQaT0zxBKb+2lB/8E8ajgxYNvs4Gi0vaFzHzvm/e9+WbGCT0Za6LTPuNB/8NHA4NDhcdPhp+OFJ89X41VO3JFzVWeitYdOxaeDERNS+2J9TAStu94Ys3ZfttdX+uIKJYqWNG7oWj49mYgN6Rra6aaxVkrEB9d5ft20EosR6Z1s5Eklu+oHU6V1/JtvVWaKJsTaZoWLC12dDI/v7IgdNoslqlCWZTuAzMHZeRRVcVjWGhBwUUbPgQCaMYebMT81WGCEDLXQMJcxEhm6wIpCqxtc5XgCpvZbR43OavnbMB5t2ecqV3exeM/YmUJk/SbvtIF/aBvdEbXf+2VZD26XnZ5dnpaETZH9saWr/6r8nnW2Pqj+qdnjQ3MZF4lew8zpnsKt6fvfPp8sTy7NJlM0SGds/8DOqUTPkHQuXSPFsXSFxT4Acy7130frL6umFQxF9+U52bypxjEOF7iFd/3NObwHlXUeN99fMdP/DJeGO+MBaPaKzX6cs0oboXx4QZ6FqD2</latexit><latexit sha1_base64="76GOptAJbRM4+9AtMhEY8egvVMs=">AAACpXichVHLbtNAFD01j7bh0UA3lbqxmhZYRddVpVasIrFhg9RX0kpxFNnOpB3VnrHsSWix/AP8AIuuQKIV4jO6YcG2SP0ExLKV2LDgxrGEoAKu5Zlzz9xz58yMH4cyNUQXE9aNm7duT05NV+7cvXd/pvrgYSvVgyQQzUCHOtn1vVSEUommkSYUu3EivMgPxY5/8Gy0vjMUSSq12jZHsehE3p6SfRl4hqluteEq8TLQUeSpXub6Mm87nSxzI18fcqrDXuSZfXux5izmeV5xjTg0WV8n9rLq2T5r8m61RnUqwr4OnBLUUMa6rp7CRQ8aAQaIIKBgGIfwkPLXhgNCzFwHGXMJI1msC+SosHbAVYIrPGYPeNzjrF2yivNRz7RQB7xLyH/CShtLdE4f6JI+0Uf6Sj/+2isreoy8HPHsj7Ui7s68ntv6/l9VxLPB/i/VPz0b9LFWeJXsPS6Y0SmCsX746s3l1tPNpewRvaNv7P8tXdAZn0ANr4L3G2LzGBV+AOfP674OWst1h+rOxkqtsVY+xRTmsYAnfN+raOA51tHkfU/wGef4Yj22XljbVmtcak2Umln8Flb3Jxmwo0s=</latexit><latexit sha1_base64="UzFSnDwdXjGUDDDdSyiYCnF0Ufo=">AAACn3ichVHPS9xAFP5MbdVtrdt6Ebxsuyo9LS+loHgSCrYnWX+sWjbLkmRHHUwyIZndakP+Af+BHnpRQaT0zxBKb+2lB/8E8ajgxYNvs4Gi0vaFzHzvm/e9+WbGCT0Za6LTPuNB/8NHA4NDhcdPhp+OFJ89X41VO3JFzVWeitYdOxaeDERNS+2J9TAStu94Ys3ZfttdX+uIKJYqWNG7oWj49mYgN6Rra6aaxVkrEB9d5ft20EosR6Z1s5Eklu+oHU6V1/JtvVWaKJsTaZoWLC12dDI/v7IgdNoslqlCWZTuAzMHZeRRVcVjWGhBwUUbPgQCaMYebMT81WGCEDLXQMJcxEhm6wIpCqxtc5XgCpvZbR43OavnbMB5t2ecqV3exeM/YmUJk/SbvtIF/aBvdEbXf+2VZD26XnZ5dnpaETZH9saWr/6r8nnW2Pqj+qdnjQ3MZF4lew8zpnsKt6fvfPp8sTy7NJlM0SGds/8DOqUTPkHQuXSPFsXSFxT4Acy7130frL6umFQxF9+U52bypxjEOF7iFd/3NObwHlXUeN99fMdP/DJeGO+MBaPaKzX6cs0oboXx4QZ6FqD2</latexit><latexit sha1_base64="/YI1qpcvsoQiTDmJZPgWv3R+7G8=">AAACrnichVFNTxRBEH0MIrCCLHgh8TJxgXhhU2NMIJxIuOjF8OEuJDubzcxsL9thZnoy07t8TOYP8Ac8eNKEEOPBH+HFP8CBi56NR0y8eLB2dhKCRKxOd1e9qlf9utuNfJloossRY/Te2P3xicnSg6nphzPl2bl6onqxJ2qe8lW85zqJ8GUoalpqX+xFsXAC1xe77sHGIL/bF3EiVfhaH0eiGTj7oexIz9EMtcov7VAceioInLCd2q7MGlYzTe3AVUccKr8dOLprLlSshSzLSrYWRzrtqNjMzFfDYJnTLpOzVrlCVcrNvO1YhVNBYZuqfA4bbSh46CGAQAjNvg8HCY8GLBAixppIGYvZk3leIEOJuT2uElzhMHrA6z5HjQINOR70THK2x6f4PGNmmlikC/pAV/SFPtJ3+v3PXmneY6DlmHd3yBVRa+Z0fufXf1kB7xrda9admjU6WM21StYe5cjgFt6Q3z95c7Wztr2YLtF7+sH639ElfeYbhP2f3tmW2H6LEn+A9fdz33bqz6oWVa2t55X11eIrJvAYT/CU33sF63iBTdT43E+4wFd8M8ioG02jNSw1RgrOI9wwo/sHcbenMQ==</latexit>

Subband FFTNet<latexit sha1_base64="l093gwkyUi/ZdRKs3xm/0TJGjfg=">AAACp3ichVHLbtNAFD01rxIeTcsGiU1EWgSb6BohUXVVUaliBW3TpJHiKJpxpu2otseyJ+nD8g/wAyxYgYQA8RlsWLFD0E9ALIvEhgU3jiUEFXAtz5x75p47Z2ZkHOjUEh1POWfOnjt/Yfpi5dLlK1dnqrNz7dQME1+1fBOYpCNFqgIdqZbVNlCdOFEilIHaknsr4/WtkUpSbaJNexirXih2Ir2tfWGZ6lcfeJHa900YimiQeVLnXbeXZV4ozQGnJhiEwu7W5uvufJ7nFc+qA5s1h1JyeW11dfORsnm/WqcGFVE7DdwS1FHGmqm+gocBDHwMEUIhgmUcQCDlrwsXhJi5HjLmEka6WFfIUWHtkKsUVwhm93jc4axbshHn455pofZ5l4D/hJU1LNBHekMn9J7e0hf68ddeWdFj7OWQZznRqrg/8+R68/t/VSHPFru/VP/0bLGNxcKrZu9xwYxP4U/0o6OnJ82ljYXsFr2gr+z/OR3TOz5BNPrmv1xXG89Q4Qdw/7zu06B9t+FSw12/V19eLJ9iGjdwE7f5vu9jGQ+xhhbv+xof8AmfnTvOY6ftdCalzlSpuYbfwhE/ASO/pCU=</latexit>

Subband FFTNe w h mu band npu(a) Training stage(a) Training stage

Acoustic features h<latexit sha1_base64="6UplFInBxIwkw0BVbFG3LuUOb5A=">AAACvHichVG7btRAFD0xr7ABsoEGicZiE0S1GiMkohQoEQ1lHmw20nq1Gs/OZkfr8Vj2eCFY/oH8AAUVSAghvoGKhh+gSE2FgCpINCm49lpCEAHX8sy5Z+65c2YmiEOVWsaO5pwzZ8+dvzB/sbFw6fKVxebS1d3UZImQHWFCk+wFPJWhimTHKhvKvTiRXAeh7AaTB+V6dyqTVJnokT2IZV/z/UiNlOCWqEGz60fysTBa82iY+4Eqel4/z30dmCeUmnCouR27yy1vuSiKhl9mic43hMlSq4Sb+27hjiS3WSLToszKJu64GDRbrM2qcE8DrwYt1LFpmq/hYwgDgQwaEhEs4RAcKX09eGCIiesjJy4hpKp1iQIN0mZUJamCEzuhcZ+yXs1GlJc900otaJeQ/oSULlbYR/aGHbMP7C37zE7+2iuvepReDmgOZloZDxYPr+/8+K9K02wx/qX6p2eLEVYrr4q8xxVTnkLM9NOnz4531rZX8lvsJftC/l+wI/aeThBNv4tXW3L7ORr0AN6f130a7N5pe6ztbd1tra/WTzGPG7iJ23Tf97COh9hEh/Z9h0/4im/OfWfoTBw9K3Xmas01/BbO9Cc1K60+</latexit>

x̂1

x̂2

x̂N

Acou

c ea u e h
x̂

x̂

x̂N

FFTNetfor 2nd band···

FFTNetfor N -th band

x̂1

x̂2

···

x̂N

FFTNe
o 1 bandFFTNeo 2nd band

FFTNe
o N h band

"M

"M

···

"M

x̂

x̂

x̂N

Overlapped SSB synthesis filterbank

"M

"M

"M

FFTNetfor 1st band<latexit sha1_base64="UzFSnDwdXjGUDDDdSyiYCnF0Ufo=">AAACn3ichVHPS9xAFP5MbdVtrdt6Ebxsuyo9LS+loHgSCrYnWX+sWjbLkmRHHUwyIZndakP+Af+BHnpRQaT0zxBKb+2lB/8E8ajgxYNvs4Gi0vaFzHzvm/e9+WbGCT0Za6LTPuNB/8NHA4NDhcdPhp+OFJ89X41VO3JFzVWeitYdOxaeDERNS+2J9TAStu94Ys3ZfttdX+uIKJYqWNG7oWj49mYgN6Rra6aaxVkrEB9d5ft20EosR6Z1s5Eklu+oHU6V1/JtvVWaKJsTaZoWLC12dDI/v7IgdNoslqlCWZTuAzMHZeRRVcVjWGhBwUUbPgQCaMYebMT81WGCEDLXQMJcxEhm6wIpCqxtc5XgCpvZbR43OavnbMB5t2ecqV3exeM/YmUJk/SbvtIF/aBvdEbXf+2VZD26XnZ5dnpaETZH9saWr/6r8nnW2Pqj+qdnjQ3MZF4lew8zpnsKt6fvfPp8sTy7NJlM0SGds/8DOqUTPkHQuXSPFsXSFxT4Acy7130frL6umFQxF9+U52bypxjEOF7iFd/3NObwHlXUeN99fMdP/DJeGO+MBaPaKzX6cs0oboXx4QZ6FqD2</latexit><latexit sha1_base64="/CNpbZaNxrRZBopTINvRZmAQksM=">AAACpXichVHPS9xAFP5Ma7VbW9f2UugldNX2tLyUgtLTgpdeCv7aVdgsS5Kd1cEkE5LZVRvyD/gP9NBTC1XEP6OXHnq14J8gHi148dC32YCoaF/IzPe+ed+bb2bcyJeJJjoZMR48HH00Nv649GTi6bPJ8tTzRqJ6sSfqnvJVvO46ifBlKOpaal+sR7FwAtcXa+7WwmB9rS/iRKpwVe9GohU4G6HsSs/RTLXLNTsU254KAifspLYrs6bVSlM7cNUOp8rvBI7eNKcr1nSWZSVbix2ddlVsWok2XdZk7XKFqpSHeRtYBaigiEVVPoCNDhQ89BBAIIRm7MNBwl8TFggRcy2kzMWMZL4ukKHE2h5XCa5wmN3icYOzZsGGnA96Jrna4118/mNWmpihYzqkc/pFR3RKl3f2SvMeAy+7PLtDrYjak3svVy7+qwp41ti8Ut3rWaOL+dyrZO9RzgxO4Q31/c9fzlc+LM+ks/Sdztj/Nzqhn3yCsP/X+7Eklr+ixA9g3bzu26DxrmpR1Vp6X6nNF08xjld4jbd833Oo4SMWUed99/Ebx/hjvDE+GatGY1hqjBSaF7gWRvsfQiujXw==</latexit><latexit sha1_base64="UzFSnDwdXjGUDDDdSyiYCnF0Ufo=">AAACn3ichVHPS9xAFP5MbdVtrdt6Ebxsuyo9LS+loHgSCrYnWX+sWjbLkmRHHUwyIZndakP+Af+BHnpRQaT0zxBKb+2lB/8E8ajgxYNvs4Gi0vaFzHzvm/e9+WbGCT0Za6LTPuNB/8NHA4NDhcdPhp+OFJ89X41VO3JFzVWeitYdOxaeDERNS+2J9TAStu94Ys3ZfttdX+uIKJYqWNG7oWj49mYgN6Rra6aaxVkrEB9d5ft20EosR6Z1s5Eklu+oHU6V1/JtvVWaKJsTaZoWLC12dDI/v7IgdNoslqlCWZTuAzMHZeRRVcVjWGhBwUUbPgQCaMYebMT81WGCEDLXQMJcxEhm6wIpCqxtc5XgCpvZbR43OavnbMB5t2ecqV3exeM/YmUJk/SbvtIF/aBvdEbXf+2VZD26XnZ5dnpaETZH9saWr/6r8nnW2Pqj+qdnjQ3MZF4lew8zpnsKt6fvfPp8sTy7NJlM0SGds/8DOqUTPkHQuXSPFsXSFxT4Acy7130frL6umFQxF9+U52bypxjEOF7iFd/3NObwHlXUeN99fMdP/DJeGO+MBaPaKzX6cs0oboXx4QZ6FqD2</latexit><latexit sha1_base64="76GOptAJbRM4+9AtMhEY8egvVMs=">AAACpXichVHLbtNAFD01j7bh0UA3lbqxmhZYRddVpVasIrFhg9RX0kpxFNnOpB3VnrHsSWix/AP8AIuuQKIV4jO6YcG2SP0ExLKV2LDgxrGEoAKu5Zlzz9xz58yMH4cyNUQXE9aNm7duT05NV+7cvXd/pvrgYSvVgyQQzUCHOtn1vVSEUommkSYUu3EivMgPxY5/8Gy0vjMUSSq12jZHsehE3p6SfRl4hqluteEq8TLQUeSpXub6Mm87nSxzI18fcqrDXuSZfXux5izmeV5xjTg0WV8n9rLq2T5r8m61RnUqwr4OnBLUUMa6rp7CRQ8aAQaIIKBgGIfwkPLXhgNCzFwHGXMJI1msC+SosHbAVYIrPGYPeNzjrF2yivNRz7RQB7xLyH/CShtLdE4f6JI+0Uf6Sj/+2isreoy8HPHsj7Ui7s68ntv6/l9VxLPB/i/VPz0b9LFWeJXsPS6Y0SmCsX746s3l1tPNpewRvaNv7P8tXdAZn0ANr4L3G2LzGBV+AOfP674OWst1h+rOxkqtsVY+xRTmsYAnfN+raOA51tHkfU/wGef4Yj22XljbVmtcak2Umln8Flb3Jxmwo0s=</latexit><latexit sha1_base64="UzFSnDwdXjGUDDDdSyiYCnF0Ufo=">AAACn3ichVHPS9xAFP5MbdVtrdt6Ebxsuyo9LS+loHgSCrYnWX+sWjbLkmRHHUwyIZndakP+Af+BHnpRQaT0zxBKb+2lB/8E8ajgxYNvs4Gi0vaFzHzvm/e9+WbGCT0Za6LTPuNB/8NHA4NDhcdPhp+OFJ89X41VO3JFzVWeitYdOxaeDERNS+2J9TAStu94Ys3ZfttdX+uIKJYqWNG7oWj49mYgN6Rra6aaxVkrEB9d5ft20EosR6Z1s5Eklu+oHU6V1/JtvVWaKJsTaZoWLC12dDI/v7IgdNoslqlCWZTuAzMHZeRRVcVjWGhBwUUbPgQCaMYebMT81WGCEDLXQMJcxEhm6wIpCqxtc5XgCpvZbR43OavnbMB5t2ecqV3exeM/YmUJk/SbvtIF/aBvdEbXf+2VZD26XnZ5dnpaETZH9saWr/6r8nnW2Pqj+qdnjQ3MZF4lew8zpnsKt6fvfPp8sTy7NJlM0SGds/8DOqUTPkHQuXSPFsXSFxT4Acy7130frL6umFQxF9+U52bypxjEOF7iFd/3NObwHlXUeN99fMdP/DJeGO+MBaPaKzX6cs0oboXx4QZ6FqD2</latexit><latexit sha1_base64="/YI1qpcvsoQiTDmJZPgWv3R+7G8=">AAACrnichVFNTxRBEH0MIrCCLHgh8TJxgXhhU2NMIJxIuOjF8OEuJDubzcxsL9thZnoy07t8TOYP8Ac8eNKEEOPBH+HFP8CBi56NR0y8eLB2dhKCRKxOd1e9qlf9utuNfJloossRY/Te2P3xicnSg6nphzPl2bl6onqxJ2qe8lW85zqJ8GUoalpqX+xFsXAC1xe77sHGIL/bF3EiVfhaH0eiGTj7oexIz9EMtcov7VAceioInLCd2q7MGlYzTe3AVUccKr8dOLprLlSshSzLSrYWRzrtqNjMzFfDYJnTLpOzVrlCVcrNvO1YhVNBYZuqfA4bbSh46CGAQAjNvg8HCY8GLBAixppIGYvZk3leIEOJuT2uElzhMHrA6z5HjQINOR70THK2x6f4PGNmmlikC/pAV/SFPtJ3+v3PXmneY6DlmHd3yBVRa+Z0fufXf1kB7xrda9admjU6WM21StYe5cjgFt6Q3z95c7Wztr2YLtF7+sH639ElfeYbhP2f3tmW2H6LEn+A9fdz33bqz6oWVa2t55X11eIrJvAYT/CU33sF63iBTdT43E+4wFd8M8ioG02jNSw1RgrOI9wwo/sHcbenMQ==</latexit><latexit sha1_base64="qnLCyaWqFkm8ANESrQjxt+GYQM8=">AAACiHichVE9TxtBEH0c304CTtIg0ZywiFJZczQYKos06QIYY0sGWXfntVl5fXe6XVtyrPyA8AdSpEqkKIpSpE1qGv4ABT8hSmkkGoqMzyehxALmdLtv38ybfbvrRUpqQ3Q5ZU3PzM7NLyxmHj1+srScffrsUIfd2BdlP1RhXPVcLZQMRNlIo0Q1ioXb8ZSoeO1Xo3ylJ2Itw+DA9CNx3HFbgWxK3zVM1bO5N5xVbhSJhl0q7di6H5gToaW2m1IZEXtu0OYqylMS9iRwUpBDGrth9iuO0EAIH110IBDAMFZwofmrwQEhYu4YA+ZiRjLJC7xDhrVdrhJc4TLb5rHFq1rKBrwe9dSJ2uddFP8xK22s0wV9oyGd03f6TTd39hokPUZe+jx7Y62I6sunK6XrB1Udng1OblX3ejZoopB4lew9SpjRKfyxvvf2w7C0vb8+eEGf6Q/7/0SXdMYnCHpX/pc9sf8RGX4A5//rngSHG3mH8s7eRq5YSJ9iAatYw0u+700U8Rq7KPO+7/EDP/HLylhkbVpb41JrKtU8xz9h7fwFDjKXYA==</latexit>

Fullband waveform x̂

Ove apped SSB yn hefi e bank(b) Synthesis stage
Fu band wave o m x̂Fig. 4. Block diagram of proposed subband FFTNet vocoder.

b Syn heage

F g 5 B ock d ag am o p oposed subband FFTNe vocode w hmu band npu
a vocal tract spectrum is predicted using the previous spectrum andfundamental frequency trajectories; and 3) subscale WaveRNN [20].
However, when using many band waveforms, the network model
size increases because the number of 1 × 1 convolution layers in thefirst layer corresponds to the number of multiband waveforms. To reduce the network model size, the use of only the first and n-th band
waveforms for n-th band training and synthesis is investigated. A
block diagram of the proposed subband FFTNet vocoder with multiband input and its first-layer network structure for n-th band training
and synthesis are depicted in Figs. 5 and 6, respectively. Comparedwith the single-band input method, two additional 1 × 1 convolution
layer components are required for the first-band waveform input, asshown in Fig. 6. For first-band training and synthesis, the standard
subband method using only the first-band input is shown in Fig. 5.
The proposed method is expected to directly compensate the phaseshift between subbands in the synthesis with random sampling basedon pn (xn |h). Compared with WaveNet, the subband method withmultiband input can be easily implemented in FFTNet because of its
simple network structure, as shown in Fig. 6.
The proposed subband method with multiband input can also
be combined with the noise shaping approach. However, empirical results suggest that the synthesized speech waveforms sometimes include collapsed segments, which are also found in WaveNet
vocoders [58]. Therefore, subband FFTNet with multiband input
combined with noise shaping is not included in the experiments reported here, and further investigations will be conducted in futurework.

4 EXPERIMENTS
4 1 Exper men a cond ons
To eva ua e he e ec veness o he p oposed me hods o FFTNe
vocode s a se es o ob ec ve and sub ec ve expe men s we e
conduc ed us ng a Japanese ma e speech co pus eco ded w h asamp ng equency o 48 kHz and downsamp ed o 16 kHz as
used n 46 47 n he expe men s 5697 u e ances abou 3 7 h
we e used as he a n ng se and 20 u e ances we e used as hees se The expe men s compa ed a he base ne van a FFTNe
vocode 42 w h b FFTNe w h es dua connec ons c FFT
Ne w h no se shap ng d subband FFTNe e subband FFTNe
w h no se shap ng andsubband FFTNe w h mu band n
pu n add on esu s we e compu ed o g he van a WaveNe
vocode 28 h he WaveNe vocode w h me nva an no seshap ng me hod 44 andhe conven ona STRA GHT sou ce
fi e vocode w hou me ceps a comp ess on 4 Me hods c –a so nc uded es dua connec ons
n he expe men s acous c ea u es h we e ana yzed eve y
5 ms ove a Hann w ndow o eng h 25 ms The undamena equency fo ana yzed by an NDF a go hm mp emen ed n
STRA GHT 59 was used n a he vocode s
Fo he WaveNe and FFTNe vocode s he 0– h o 24– h me
ceps a coe fic en s 25 d mens ons we e ana yzed om a s mp e
sho me Fou e ans o m o w ndowed speech wave o ms w h asamp ng equency o 16 kHz and wa p ng coe fic en α = 0 42
n he STRA GHT vocode he o g na STRA GHT smoo h vo

1st layer<latexit sha1_base64="ErPKpUNEdKFfnQh1dUZLUnMZAd4=">AAACt3ichVE9bxNBEJ1c+AjHR0xokGgsnCAqay4NgSoSTcrEwUkkn2Xtrcf2Knu7x93aYE73B/IHKKhAQgjxB+hp+AMUaWgBUQaJhoLx+SQEETCn233zdt7s290o0SpziMcL3uKZs+fOL13wL166fGW5dnVlL7PjVFJbWm3Tg0hkpJWhtlNO00GSkogjTfvR4f3Z+v6E0kxZ88BNE+rGYmjUQEnhmOrVWqGhR9LGsTD9PIxU0Qm6eR7GkX3MqdX9WLhRfbURrBZF4YcRDZXJJRlHaeEHmatrMaXUD4nlFd2rNbCJZdRPg6ACDahi29ZeQQh9sCBhDDEQGHCMNQjI+OtAAAgJc13ImUsZqXKdoACftWOuIq4QzB7yOOSsU7GG81nPrFRL3kXzn7KyDmv4AV/jCb7HN/gFf/y1V172mHmZ8hzNtZT0lo+u737/ryrm2cHol+qfnh0MYKP0qth7UjKzU8i5fvLk6cnuvdZafgtf4Ff2/xyP8R2fwEy+yZc71HoGPj9A8Od1nwZ7680Am8HOemNzo3qKJbgBN+E23/cd2IQt2IY27/sWPsIn+Ozd9XrewBvNS72FSnMNfgvv4U/gkasT</latexit><latexit sha1_base64="YZNQ6PhZPSD898Nbq1aHfrioKkY=">AAAC0HichVE9b9RAEB2br2A+ckCDRGNxDqI67aZJRBWJhoIiuXBJpPPpZO9NLqvz7lr23pFgWYg2f4CCikgRQvwMGloKivwERBmkNBSMfZYQRMCsdnfmzbzZt7txmsjcMnbiuBcuXrp8ZeGqd+36jZuLrVu3t3IzzQT2hElMthNHOSZSY89Km+BOmmGk4gS348njKr89wyyXRj+zBykOVDTWcleKyBI0bE1Cjc+FUSrSoyKMZdnng6IIVWz2KTTJSEV2zw/aPCjL0gtjHEtdCNQWs9Lr4tOeH4TWBH7A/dBKhbnPgwaqsl6I1LepH7barMNq8887vHHa0Ni6ab2DEEZgQMAUFCBosOQnEEFOow8cGKSEDaAgLCNP1nmEEjziTqkKqSIidELrmKJ+g2qKq555zRZ0SkIzI6YPS+wLe89O2Sf2gX1lP/7aq6h7VFoOaI/nXEyHi4d3N8/+y1K0W9j7xfqnZgu7sFprlaQ9rZHqFmLOn714fbr5qLtUPGBH7Bvpf8tO2Ee6gZ59F8cb2H0DHn0A//O5zztbyx3OOnxjub222nzFAtyD+/CQ3nsF1uAJrEOPzv0MZw44jtt1992X7qt5qes0nDvwm7mHPwELorBg</latexit>

ReLU ! 1 ⇥ 1 ! ReLU+
<latexit sha1_base64="MNQ8Guaynm3W8HzRQJbP8gFpqH4=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIxySlpmfmVSen5pWkFtVyqWircMWk5qXABOIFlA30DMBAAZNhCGUoM0BBQL7AcoYYhhSGfIZkhlKGXIZUhjyGEiA7hyGRoRgIoxkMGQwYCoBisQzVQLEiICsTLJ/KUMvABdRbClSVClSRCBTNBpLpQF40VDQPyAeZWQzWnQy0JQeIi4A6FRhUDa4arDT4bHDCYLXBS4M/OM2qBpsBckslkE6C6E0tiOfvkgj+TlBXLpAuYchA6MLr5hKGNAYLsFszgW4vAIuAfJEM0V9WNf1zsFWQarWawSKD10D3LzS4aXAY6IO8si/JSwNTg2YzcAEjwBA9uDEZYUZ6hgZ6hoFGyg4W0KjgYJBmUGLQAIa3OYMDgwdDAEMo0N5WhtUMWxi2MrEx6TAZM5lClDIxQvUIM6AAJjsATzeVPw==</latexit><latexit

1⇥1<latexit sha1_base64="Y1MAI6Ab21gSGqqqJrpIIP/nmSQ=">AAACjXichVFNSxxBEH2O8WsS3Y25BHJZXBVPS40gioQg5JAc/ciq4MoyM5Zr40zPMNO7YBb/gAevOeSUQJCQa6568eIfyMGfEDwqeMkhtbMTgopaw3S/flWv+nW3FwcqNUTnPVbvk77+gcEh++mz4ZFC8fnoaho1E5+rfhREybrnphwozVWjTMDrccJu6AW85u2+7eTXWpykKtIfzF7Mm6Hb0Gpb+a4Rql6cqnncULrtszac7NvjTqlmVMhpyRm3a6y3/mXqxTJVKIvSXeDkoIw8FqPiEWrYQgQfTYRgaBjBAVyk8m3AASEWbhNt4RJBKssz9mGLtilVLBWusLsyNmS1kbNa1p2eaab2ZZdA/kSUJUzQL/pOl3RGP+g3/bm3Vzvr0fGyJ7PX1XJcLxy8XLl+VBXKbLDzX/WgZ4NtzGVelXiPM6ZzCr+rb338dLkyvzzRnqSvdCH+v9A5ncoJdOvK/7bEy59hywM4t6/7LlidrjhUcZamywtz+VMM4hXGMCX3PYsFvMciqrLvIX7iGCdWwZqxXltvuqVWT655gRthvfsL+xSYjg==</latexit>

1⇥1

1⇥1

<latexit sha1_base64="Y1MAI6Ab21gSGqqqJrpIIP/nmSQ=">AAACjXichVFNSxxBEH2O8WsS3Y25BHJZXBVPS40gioQg5JAc/ciq4MoyM5Zr40zPMNO7YBb/gAevOeSUQJCQa6568eIfyMGfEDwqeMkhtbMTgopaw3S/flWv+nW3FwcqNUTnPVbvk77+gcEh++mz4ZFC8fnoaho1E5+rfhREybrnphwozVWjTMDrccJu6AW85u2+7eTXWpykKtIfzF7Mm6Hb0Gpb+a4Rql6cqnncULrtszac7NvjTqlmVMhpyRm3a6y3/mXqxTJVKIvSXeDkoIw8FqPiEWrYQgQfTYRgaBjBAVyk8m3AASEWbhNt4RJBKssz9mGLtilVLBWusLsyNmS1kbNa1p2eaab2ZZdA/kSUJUzQL/pOl3RGP+g3/bm3Vzvr0fGyJ7PX1XJcLxy8XLl+VBXKbLDzX/WgZ4NtzGVelXiPM6ZzCr+rb338dLkyvzzRnqSvdCH+v9A5ncoJdOvK/7bEy59hywM4t6/7LlidrjhUcZamywtz+VMM4hXGMCX3PYsFvMciqrLvIX7iGCdWwZqxXltvuqVWT655gRthvfsL+xSYjg==</latexit><latexit sha1_base64="Y1MAI6Ab21gSGqqqJrpIIP/nmSQ=">AAACjXichVFNSxxBEH2O8WsS3Y25BHJZXBVPS40gioQg5JAc/ciq4MoyM5Zr40zPMNO7YBb/gAevOeSUQJCQa6568eIfyMGfEDwqeMkhtbMTgopaw3S/flWv+nW3FwcqNUTnPVbvk77+gcEh++mz4ZFC8fnoaho1E5+rfhREybrnphwozVWjTMDrccJu6AW85u2+7eTXWpykKtIfzF7Mm6Hb0Gpb+a4Rql6cqnncULrtszac7NvjTqlmVMhpyRm3a6y3/mXqxTJVKIvSXeDkoIw8FqPiEWrYQgQfTYRgaBjBAVyk8m3AASEWbhNt4RJBKssz9mGLtilVLBWusLsyNmS1kbNa1p2eaab2ZZdA/kSUJUzQL/pOl3RGP+g3/bm3Vzvr0fGyJ7PX1XJcLxy8XLl+VBXKbLDzX/WgZ4NtzGVelXiPM6ZzCr+rb338dLkyvzzRnqSvdCH+v9A5ncoJdOvK/7bEy59hywM4t6/7LlidrjhUcZamywtz+VMM4hXGMCX3PYsFvMciqrLvIX7iGCdWwZqxXltvuqVWT655gRthvfsL+xSYjg==</latexit>

1⇥1

1⇥1

<latexit sha1_base64="Y1MAI6Ab21gSGqqqJrpIIP/nmSQ=">AAACjXichVFNSxxBEH2O8WsS3Y25BHJZXBVPS40gioQg5JAc/ciq4MoyM5Zr40zPMNO7YBb/gAevOeSUQJCQa6568eIfyMGfEDwqeMkhtbMTgopaw3S/flWv+nW3FwcqNUTnPVbvk77+gcEh++mz4ZFC8fnoaho1E5+rfhREybrnphwozVWjTMDrccJu6AW85u2+7eTXWpykKtIfzF7Mm6Hb0Gpb+a4Rql6cqnncULrtszac7NvjTqlmVMhpyRm3a6y3/mXqxTJVKIvSXeDkoIw8FqPiEWrYQgQfTYRgaBjBAVyk8m3AASEWbhNt4RJBKssz9mGLtilVLBWusLsyNmS1kbNa1p2eaab2ZZdA/kSUJUzQL/pOl3RGP+g3/bm3Vzvr0fGyJ7PX1XJcLxy8XLl+VBXKbLDzX/WgZ4NtzGVelXiPM6ZzCr+rb338dLkyvzzRnqSvdCH+v9A5ncoJdOvK/7bEy59hywM4t6/7LlidrjhUcZamywtz+VMM4hXGMCX3PYsFvMciqrLvIX7iGCdWwZqxXltvuqVWT655gRthvfsL+xSYjg==</latexit><latexit sha1_base64="Y1MAI6Ab21gSGqqqJrpIIP/nmSQ=">AAACjXichVFNSxxBEH2O8WsS3Y25BHJZXBVPS40gioQg5JAc/ciq4MoyM5Zr40zPMNO7YBb/gAevOeSUQJCQa6568eIfyMGfEDwqeMkhtbMTgopaw3S/flWv+nW3FwcqNUTnPVbvk77+gcEh++mz4ZFC8fnoaho1E5+rfhREybrnphwozVWjTMDrccJu6AW85u2+7eTXWpykKtIfzF7Mm6Hb0Gpb+a4Rql6cqnncULrtszac7NvjTqlmVMhpyRm3a6y3/mXqxTJVKIvSXeDkoIw8FqPiEWrYQgQfTYRgaBjBAVyk8m3AASEWbhNt4RJBKssz9mGLtilVLBWusLsyNmS1kbNa1p2eaab2ZZdA/kSUJUzQL/pOl3RGP+g3/bm3Vzvr0fGyJ7PX1XJcLxy8XLl+VBXKbLDzX/WgZ4NtzGVelXiPM6ZzCr+rb338dLkyvzzRnqSvdCH+v9A5ncoJdOvK/7bEy59hywM4t6/7LlidrjhUcZamywtz+VMM4hXGMCX3PYsFvMciqrLvIX7iGCdWwZqxXltvuqVWT655gRthvfsL+xSYjg==</latexit>

1⇥1<latexit sha1_base64="Y1MAI6Ab21gSGqqqJrpIIP/nmSQ=">AAACjXichVFNSxxBEH2O8WsS3Y25BHJZXBVPS40gioQg5JAc/ciq4MoyM5Zr40zPMNO7YBb/gAevOeSUQJCQa6568eIfyMGfEDwqeMkhtbMTgopaw3S/flWv+nW3FwcqNUTnPVbvk77+gcEh++mz4ZFC8fnoaho1E5+rfhREybrnphwozVWjTMDrccJu6AW85u2+7eTXWpykKtIfzF7Mm6Hb0Gpb+a4Rql6cqnncULrtszac7NvjTqlmVMhpyRm3a6y3/mXqxTJVKIvSXeDkoIw8FqPiEWrYQgQfTYRgaBjBAVyk8m3AASEWbhNt4RJBKssz9mGLtilVLBWusLsyNmS1kbNa1p2eaab2ZZdA/kSUJUzQL/pOl3RGP+g3/bm3Vzvr0fGyJ7PX1XJcLxy8XLl+VBXKbLDzX/WgZ4NtzGVelXiPM6ZzCr+rb338dLkyvzzRnqSvdCH+v9A5ncoJdOvK/7bEy59hywM4t6/7LlidrjhUcZamywtz+VMM4hXGMCX3PYsFvMciqrLvIX7iGCdWwZqxXltvuqVWT655gRthvfsL+xSYjg==</latexit>

1st band samples x1 n-th band samples xn Acoustic features h<latexit sha1_base64="RQVTbKgtq/zpmMrvWbdf8pD+IKM=">AAACsXichVG7btRAFD0xr7A8skCDRGOxCaJaXQMSEVUQDQVFHmwStF4t49nZ7Ghtj2WPlwTLP8APUFCBhBCi4xdo+AEkUkKHKINEQ8FdryUEEXAtz5x75p47Z2aCJNSZJdqfc44cPXb8xPzJxqnTZ84uNM+d38xMnkrVkSY06XYgMhXqWHWstqHaTlIloiBUW8H4znR9a6LSTJv4vt1LVC8SO7EeaiksU/3mPT9Wj6SJIhEPCj/QZdfrFYUfBWaXUxMOImFH7mLLWyzLsuFbtWuL29LkmdXSHSph81RlbsnKYlT2my1qUxXuYeDVoIU6Vk3zFXwMYCCRI4JCDMs4hEDGXxceCAlzPRTMpYx0ta5QosHanKsUVwhmxzzucNat2Zjzac+sUkveJeQ/ZaWLJfpAr+mA3tMb+kI//tqrqHpMvezxHMy0KukvPLm48f2/qohni9Ev1T89WwyxXHnV7D2pmOkp5Ew/efz0YOPW+lJxhV7QV/b/nPbpHZ8gnnyTL9fU+jM0+AG8P6/7MNi81vao7a3daK0s108xj0u4jKt83zexgrtYRYf3fYuP+ITPznXngfPQCWalzlytuYDfwhn/BKjiqQQ=</latexit><latexit sha1_base64="kstbW15saLDJ3NwtbkCIi4d8f/M=">AAACtXichVFNaxRBEK1MoonrR9Z4EbwMbiKelmoREjwFcslxk7hJYGdZemZ7kybd08N072aTYf5A/kAOOSmIiHf/gBf/gIeA1wjiMYIXD9bMDogGtYbpfvW6XvXr7jBR0jrE8ylveuba9dm5G7Wbt27fma/fXdi2ZphGoh0ZZdLdkFuhZCzaTjoldpNUcB0qsRMerBXrOyORWmni5+4oEV3N92I5kBF3RPXqrSAWh5HRmsf9LAhl3mHdLAt0aMaUGtXX3O37iw22mOd5LXBi7DJmnR9SvW+5TpSwfl4o/XHey1jeqzewiWX4VwGrQAOqaJn6awigDwYiGIIGATE4wgo4WPo6wAAhIa4LGXEpIVmuC8ihRtohVQmq4MQe0LhHWadiY8qLnrZUR7SLoj8lpQ9L+BHf4CV+wLf4BX/8tVdW9ii8HNEcTrQi6c2f3N/6/l+VptnB/i/VPz07GMBK6VWS96RkilNEE/3o+PRy69nmUvYIX+JX8v8Cz/E9nSAefYtebYjNM6jRA7A/r/sq2H7SZNhkG08bqyvVU8zBA3gIj+m+l2EV1qEFbdr3HXyCC/jsLXtdr+8NJqXeVKW5B7+FZ34Cba+qDA==</latexit><latexit sha1_base64="u45fr71MQQ7Px7voS483A57/Lrk=">AAACtnichVHNbtNAEJ6avxJ+GuCCxMUiLeJCNK6QiDhV4sKxaUlbKQ7R2pmmq+6PZW9CiuUX4AU4cAIJIcQD8ABceAEOPXAFIY5F4sKBsWMJQQWM5d1vvp1v9tvdKFEyc4iHC96Jk6dOn1k82zh3/sLFpealy1uZnaQx9WKrbLoTiYyUNNRz0inaSVISOlK0He3fK9e3p5Rm0poH7iChgRZjI3dlLBxTw2Y3NPQotloLM8rDSBb9YJDnoY7sjFOrRlq4PX+5FSwXRdEwoaOZy28xFbHAz4ROFGV+UUr9WTHMTTFstrCNVfjHQVCDFtSxbpuvIIQRWIhhAhoIDDjGCgRk/PUhAISEuQHkzKWMZLVOUECDtROuIq4QzO7zOOasX7OG87JnVqlj3kXxn7LShxX8gK/xCN/jG/yCP/7aK696lF4OeI7mWkqGS0+ubn7/r0rz7GDvl+qfnh3sQqfyKtl7UjHlKeK5fvr46dHm3Y2V/Aa+wK/s/zke4js+gZl+i192aeMZNPgBgj+v+zjYWm0H2A66t1trnfopFuEaXIebfN93YA3uwzr0eN+38BE+wWev4z30yBvPS72FWnMFfgsv+Qnqiaqy</latexit>

Fig. 6. First layer of subband FFTNet vocoder with multiband input
for n-th band training and synthesis.
Table 1. Number of network model parameters.
Model
Num of params
WaveNet (g) and (h)44,592,721
FFTNet (a) to (c)2,251,857
Subband FFTNet (d) and (e)1,857,105(each subband)
Subband FFTNet with multiband input (f)1,988,117(each subband)

cal tract spectrum (1025 dimensions) and aperiodicity (1025 dimensions) coefficients without mel-cepstral compression were directly
used. Although they are typically compressed as mel-cepstrum andband aperiodicity coefficients [60] for dimensional reduction, theoriginal high-dimensional coefficients were used in the experiments
to evaluate the potential of the proposed approaches.
In FFTNet vocoders (a)–(c), L = 11 layers were introduced andthe receptive field was 211 = 2048 samples, as used in the original
FFTNet [42]. A square-root Hann window-based overlapped SSB
filterbank was also introduced for subband FFTNet vocoders (d)–(f). A decimation factor of M = 4 and division number N = 2M +1 = 9 were also used. The length of the analysis and synthesis
prototype FIR filters was 1024 samples. The sampling frequencyof each subband waveform was (16/4 =) 4 kHz. The frequency
response of the filterbank is plotted in Fig. 7. In subband FFTNet
vocoders (d)–(f), L = 9 layers were used and the receptive field was
29 = 512 samples. The channel number of each FFTNet layer was
256 [42].
The dilation channel, residual channel, and skip channel of
WaveNet vocoders (g) and (h) were set to 512, 512, and 256, respectively. Thirty layers (10 dilations × 3 cycles) with a kernel sizeof 2 were used for the dilated causal convolution layers, giving a
receptive field of 3070 samples [17, 28].
In the WaveNet and FFTNet vocoders, (1 + 1 + 25 =) 27dimensional vectors constructed from the continuous logarithmic
fo , voice/unvoice one-hot vector, and mel-cepstrum coefficients(normalized to have a zero-mean and unit-variance) were used as
the acoustic features h. The WaveNet and FFTNet vocoders required 400,000 and 1,000,000 parameter updates, respectively, andan Adam optimization algorithm [61] updated the neural network
parameters with a learning rate of 0.001. The minibatch sizes of
WaveNet, FFTNet, and subband FFTNet were 1 × 20,000, 5 ×
5,000, and 5 × 1,250 samples, respectively. They were trained using
a single GPU of an NVIDIA Tesla P100. A value of β = 0.5 in
Eq. (2) was used for the noise shaping methods (c), (e), and (h) according to the results of the WaveNet vocoder investigations [40,44].
The number of network model parameters in the FFTNet andWaveNet vocoders is listed in Table 1. The FFTNet vocoders haveabout 1/20 the number of the WaveNet vocoders, and the FFTNet

Fig. 7. Frequency response of a square-root Hann window-based
overlapped SSB filterbank with decimation factor M = 4 and division number N = 9 for proposed subband FFTNet vocoder.
vocoders can synthesize speech waveforms in real time. In particular, the subband FFTNet vocoders can realize more than M = 4
times faster generation as a result of the decimation and smaller network, as in subband WaveNet [46, 47].
4.2. Objective evaluations
To objectively evaluate the synthesized test set speech waveforms,
the signal-to-noise ratio (SNR) and the spectral distortion (SD) between the original waveform x(t) and the synthesized x̂(t) werecomputed:!
PT2
t=1 x̂(t)SN R = 10 log10 PT,(5)2
t=1 (x(t) − x̂(t))
v
!2
A u
F
|X̂(f, a)|
1 Xu
1 X
t
SD =
20 log10,(6)
A a=1 F
|X(f, a)|f =1

where X(f, a) and X̂(f, a) are the short-time Fourier spectrums ofx(t) and x̂(t) in frame a for frequency bin f , and A is the total
number of frames. As in previous studies [28, 44, 47], a linear phasecompensation for each frame was introduced to calculate the SNR.
For acoustic feature analysis, the short-time Fourier transform analysis window function was also a Hann window with a frame lengthof 25 ms, a frameshift of 5 ms, and F = 257. To consider thehuman auditory perception criterion in the objective evaluation, themel-cepstral distortion (MCD) was also computed. This is defined
as:
v
u B
X
10 u
t2
M CD =(c(b) − ĉ(b))2 ,(7)
log 10b=1

where c(b) and ĉ(b) are the b-th mel-cepstral coefficients obtained
from X(f, a) and X̂(f, a) with α = 0.42 and B = 24. The resultsof the objective evaluations are presented in Table 2. In addition,
the training softmax loss scores averaged over 10,000 iterations are
given in Table 2 to evaluate the model accuracy. In subband methods(d)–(f), the loss scores were averaged over all nine bands.
4.3. Subjective evaluations
To subjectively evaluate the proposed approaches, mean opinion
score (MOS) tests [62] were conducted. All 20 utterances of the test
set were used as the evaluation set. These were presented throughheadphones to 10 Japanese adult native speakers without hearing
loss (20 utterances ×10 conditions including the original test set
waveforms = 200 utterances). The MOS results are plotted inFig. 8.

Table 2. Results of objective evaluations of 20 test set utterances. Bold and italic entries indicate best scores of all six FFTNet vocoders andof all nine methods, respectively.
Training softmax loss score
SNR [dB]SD [dB]MCD [dB](a):vanilla FFTNet (baseline)
1.89
5.20 ± 0.26 10.29 ± 0.15 3.66 ± 0.11(b):FFTNet with residual connections
1.81
5.50 ± 0.25
9.68 ± 0.12
3.33 ± 0.08(c):FFTNet with noise shaping
2.19
4.00 ± 0.47
8.19 ± 0.05
2.84 ± 0.06(d):subband FFTNet
1.39
4.00 ± 0.27 10.76 ± 0.30 2.96 ± 0.04(e):subband FFTNet with noise shaping
1.55
2.90 ± 0.39
9.62 ± 0.22
2.84 ± 0.06(f):subband FFTNet with multiband input
1.35
5.80 ± 0.36 10.84 ± 0.36 3.13 ± 0.39(g):vanilla WaveNet
1.50
6.60 ± 0.36
9.16 ± 0.12
2.50 ± 0.08(h):WaveNet with noise shaping
1.80
5.50 ± 0.60
7.58 ± 0.06
2.00 ± 0.07(i):STRAIGHT–
0.10 ± 0.47
7.09 ± 0.07
2.78 ± 0.08(a):vanilla<latexit sha1_base64="9kDhgs6xFmY/rQuobNJV7rK0/zo=">AAACaXichVG7SgNBFD1ZXzG+ojZBGzEoVuFuGoOVIAQrMZqHoCK766irm91ldxKIwR+wshO1UhARP8PGH7DIJ0jKCDYW3mwWREW9w8ycOXPPnTMzumuZviRqRJSu7p7evmh/bGBwaHgkPjpW9J2KZ4iC4ViOt6FrvrBMWxSkKS2x4XpCK+uWKOlHS+39UlV4vunYeVlzxXZZ27fNPdPQJFPFbDa/IuROPEkpCmLqJ1BDkEQYq078DlvYhQMDFZQhYEMytqDB57YJFQSXuW3UmfMYmcG+wAlirK1wluAMjdkjHvd5tRmyNq/bNf1AbfApFnePlVOYoWe6pxY90QO90PuvtepBjbaXGs96RyvcnZHTxPrbv6oyzxIHn6o/PUvsIRN4Ndm7GzDtWxgdffX4vLW+sDZTn6UbarL/a2rQI9/Arr4atzmxdoUYf4D6/bl/gmI6pVJKzaWTi5nwK6KYxDTm+L3nsYhlrKLA5x7iDBe4jDSVUSWhTHRSlUioGceXUJIf0E2LrQ==</latexit>

FFTNet<latexit sha1_base64="WxxwMN0bt7yLkYZq78T0lbsfw5c=">AAACbnichVHLSsNAFD2N7/qqCiKIKBalbsqNG0tXghuXvqpCFZnEUQenSUjSghZ/wL24EBQFEfEz3PgDLvwEcSMouHHhbRoQFfWGyZw5c8+dM3csT6sgJHpIGA2NTc0trW3J9o7Oru5UT+9y4JZ9WxZsV7v+qiUCqZUjC6EKtVz1fClKlpYr1u5MbX+lIv1Auc5SuOfJ9ZLYdtSWskXIVDEjJvIV4SitxUYqTVmKYuQnMGOQRhxzbuoKa9iECxtllCDhIGSsIRDwV4QJgsfcOqrM+YxUtC9xgCRry5wlOUMwu8v/bV4VY9bhda1mEKltPkXz8Fk5gjG6p2t6oTu6oUd6/7VWNapR87LHs1XXSm+j+3Bg8e1fVYnnEDufqj89h9hCLvKq2LsXMbVb2HV9Zf/4ZTG/MFYdpwt6Yv/n9EC3fAOn8mpfzsuFEyT5Aczv7f4JliezJmXN+cn0dC5+ilYMYhQZ7vcUpjGLORSijh3hFGeJZ6PfGDKG66lGItb04UsYmQ8wAo2r</latexit>(b):res<latexit sha1_base64="NWGDzAfJ5Q9yNcoVui5gnwwaKdA=">AAACanichVG7SgNBFD1Z3/GRGBslTTBGtAl30yipBBtLXzGBKLK7TnTJZnfZnQQ0+AN2VoJWCiLiZ9j4Axb5BNEugo2FN5sFUVHvMDNnztxz58yM7lqmL4laEaWnt69/YHAoOjwyOhaLjye2fKfuGaJgOJbjlXTNF5Zpi4I0pSVKrie0mm6Jol5d7uwXG8LzTcfelIeu2Klp+7ZZMQ1NMlWc0+fznvB342nKUhCpn0ANQRphrDrxG2xjDw4M1FGDgA3J2IIGn1sZKggucztoMucxMoN9gWNEWVvnLMEZGrNVHvd5VQ5Zm9edmn6gNvgUi7vHyhQy9Ei31KYHuqMnev+1VjOo0fFyyLPe1Qp3N3YyufH2r6rGs8TBp+pPzxIVLAZeTfbuBkznFkZX3zg6a2/k1zPNWbqiZ/Z/SS265xvYjVfjek2sXyDKH6B+f+6fYCuXVSmrruXSS4vhVwwiiWnM8XsvYAkrWEUhcHeKc1xEXpSEMqUku6lKJNRM4EsoMx9WyIvn</latexit>(c):ns<latexit sha1_base64="QCwblVFe9Lj9fYNrlYR0T1geWRc=">AAACaXichVG7SgNBFD1ZXzE+ErUJ2ohBiU24m0axCthY5mESIQbZXUdds9lddjcBDf6AlZ2olYKI+Bk2/oBFPkFSKthYeLNZEA3qHWbmzJl77pyZUW1Ddz2idkgaGBwaHgmPRsbGJyajsanpkms1HE0UNcuwnC1VcYWhm6Lo6Z4htmxHKHXVEGW1tt7dLzeF4+qWuekd2aJaV/ZNfU/XFI+pUlJbXjPdnViCUuTHfD+QA5BAEFkrdodt7MKChgbqEDDhMTagwOVWgQyCzVwVLeYcRrq/L3CCCGsbnCU4Q2G2xuM+ryoBa/K6W9P11RqfYnB3WDmPRXqme3qlJ3qgF/r4tVbLr9H1csSz2tMKeyd6Gi+8/6uq8+zh4Ev1p2cPe1j1vers3faZ7i20nr55fP5aWMsvtpbohjrs/5ra9Mg3MJtv2m1O5K8Q4Q+Qfz53PyilUzKl5Fw6kVkNviKMOSwgye+9ggw2kEWRzz3EGS5wGepIU1Jcmu2lSqFAM4NvISU+AV+Li3U=</latexit>(d):sb<latexit sha1_base64="NfOLGhU33Phv0IR0a0SQxc0nNMc=">AAACZ3ichVHLSsNAFD2Nr1ofrQoiuFFLxVW5KYLVleDGpa/aQi0liVMNTZOQpIVa/AEXbhVcKYiIn+HGH3DhJxSXFdy48CYNiBb1DjNz5sw9d87MqLahux7RS0Tq6x8YHIoOx0ZGx8bjiYnJfdeqO5rIaZZhOQVVcYWhmyLn6Z4hCrYjlJpqiLxa3fD38w3huLpl7nlNW5RqypGpV3RN8Xwqk6bVciJJaQpirhfIIUgijC0rcYcDHMKChjpqEDDhMTagwOVWhAyCzVwJLeYcRnqwL3CKGGvrnCU4Q2G2yuMRr4oha/Lar+kGao1PMbg7rJxDip7pnjr0RA/Upo9fa7WCGr6XJs9qVyvscvxsZvf9X1WNZw/HX6o/PXuoIBt41dm7HTD+LbSuvnFy0dld20m1FumGXtn/Nb3QI9/AbLxpt9ti5wox/gD553P3gv1MWqa0vL2cXM+GXxHFLBawxO+9gnVsYgs5PvcY57jAZaQtxaVpaaabKkVCzRS+hTT/CQawil0=</latexit>

2.56<latexit sha1_base64="DY+PuNQlkj1/6DMLqkXOSRVCBNQ=">AAACZ3ichVHLSsNAFD2N7/poVRDBTbVUXJUb8YWrghuXtrUPqEWSONVgmoQkLWjxB1y4reBKQUT8DDf+gIt+grhUcOPCmzQgKuodZubMmXvunJlRbUN3PaJOROrp7esfGByKDo+MjsXi4xNF12o4miholmE5ZVVxhaGbouDpniHKtiOUumqIknq44e+XmsJxdcvc9o5sUa0r+6Ze0zXF86nF9PLKbjxJaQoi8RPIIUgijC0rfoMd7MGChgbqEDDhMTagwOVWgQyCzVwVLeYcRnqwL3CCKGsbnCU4Q2H2kMd9XlVC1uS1X9MN1BqfYnB3WJlAih7pll7oge7oid5/rdUKavhejnhWu1ph78ZOp/Nv/6rqPHs4+FT96dlDDWuBV5292wHj30Lr6pvH7Zf8ei7Vmqcremb/l9She76B2XzVrrMid4Eof4D8/bl/guJiWqa0nF1KZtbCrxjEDOawwO+9igw2sYUCn3uAM7RxHnmSYtKUNN1NlSKhZhJfQpr9AAq1il8=</latexit>

3.02<latexit sha1_base64="6AjU9c1tGslCnYuCV4fq2fkktoA=">AAACZ3ichVHLSsNAFD2Nr1ofrQpScKOWiqtyo4LFVcGNS63WFmopSZxqaJqEJC1o8QdcuK3gSkFE/Aw3/oALP0FcKrhx4U0aEC3qHWbmzJl77pyZUW1Ddz2ip4jU1z8wOBQdjo2Mjo3HExOTu67VdDRR0CzDckqq4gpDN0XB0z1DlGxHKA3VEEW1vu7vF1vCcXXL3PGObFFpKAemXtM1xfOp5QwtVRMpylAQs71ADkEKYWxaiRvsYR8WNDTRgIAJj7EBBS63MmQQbOYqaDPnMNKDfYETxFjb5CzBGQqzdR4PeFUOWZPXfk03UGt8isHdYeUs0vRIt/RKD3RHz/Txa612UMP3csSz2tUKuxo/TW6//6tq8Ozh8Ev1p2cPNWQDrzp7twPGv4XW1beOO6/ba/l0e4Gu6IX9X9IT3fMNzNabdr0l8heI8QfIP5+7F+wuZWTKyFsrqVw2/IooZjCPRX7vVeSwgU0U+NxDnKGD88izFJempWQ3VYqEmil8C2nuE/qkilc=</latexit>

2.65<latexit sha1_base64="PTAvj3VjKvJBqZ2CDcEHsLB/NYI=">AAACZ3ichVHLSsNAFD2N7/poVRDBTbVUXJUb8YWrghuXtrUPqEWSONVgmoQkLWjxB1y4reBKQUT8DDf+gIt+grhUcOPCmzQgKuodZubMmXvunJlRbUN3PaJOROrp7esfGByKDo+MjsXi4xNF12o4miholmE5ZVVxhaGbouDpniHKtiOUumqIknq44e+XmsJxdcvc9o5sUa0r+6Ze0zXF86nF9MrybjxJaQoi8RPIIUgijC0rfoMd7MGChgbqEDDhMTagwOVWgQyCzVwVLeYcRnqwL3CCKGsbnCU4Q2H2kMd9XlVC1uS1X9MN1BqfYnB3WJlAih7pll7oge7oid5/rdUKavhejnhWu1ph78ZOp/Nv/6rqPHs4+FT96dlDDWuBV5292wHj30Lr6pvH7Zf8ei7Vmqcremb/l9She76B2XzVrrMid4Eof4D8/bl/guJiWqa0nF1KZtbCrxjEDOawwO+9igw2sYUCn3uAM7RxHnmSYtKUNN1NlSKhZhJfQpr9AAq2il8=</latexit>(e) sb w th ns

2.72( ) sb w th mb

2.61(g) van a2.95(h) ns

4.01( ) STRA GHT

2.89

Original

4.24<latexit sha1_base64="gd+Tu0DGGQXrp+RIN/2+37arWF0=">AAACZ3ichVHLSsNAFD2Nr1ofrQoiuFFLxVW4KYLFleDGpa/agkpJ4rQNzYskLWjxB1y4VXClICJ+hht/wIWfIC4V3LjwJg2IFvUOM3PmzD13zsxormn4AdFTQurp7esfSA6mhoZHRtOZsfEd32l6uijqjul4ZU31hWnYohgYgSnKridUSzNFSWushvullvB8w7G3g0NX7FtqzTaqhq4GIZWXl/KVTJZkimKmGygxyCKOdSdzgz0cwIGOJiwI2AgYm1Dhc9uFAoLL3D7azHmMjGhf4Bgp1jY5S3CGymyDxxqvdmPW5nVY04/UOp9icvdYOYMcPdItvdID3dEzffxaqx3VCL0c8qx1tMKtpE+mtt7/VVk8B6h/qf70HKCKQuTVYO9uxIS30Dv61tHZ69byZq49T1f0wv4v6Ynu+QZ2602/3hCbF0jxByg/n7sb7ORlhWRlYzG7Uoi/IolpzGGB33sJK1jDOop8bh2nOMN54llKS5PSVCdVSsSaCXwLafYTBreKXQ==</latexit><latexit sha1_base64="c2dfRfYc1bnWnQ58UviPdu0v87s=">AAACZ3ichVHLSsNAFD2Nr1ofrQpScFMtiqtyI6LFVcGNS632AVUkiWMbmiYhSQta/AEXbiu4UhARP8ONP+DCTxCXCm5ceJMGRIt6h5k5c+aeO2dmVNvQXY/oKSL19Q8MDkWHYyOjY+PxxMRk0bWajiYKmmVYTllVXGHopih4umeIsu0IpaEaoqTW1/39Uks4rm6ZO96RLfYaStXUD3VN8XxqKbMi7yfSlKEgUr1ADkEaYWxaiRvs4gAWNDTRgIAJj7EBBS63CmQQbOb20GbOYaQH+wIniLG2yVmCMxRm6zxWeVUJWZPXfk03UGt8isHdYWUK8/RIt/RKD3RHz/Txa612UMP3csSz2tUKez9+mtx+/1fV4NlD7Uv1p2cPh8gGXnX2bgeMfwutq28dd1631/Lz7QW6ohf2f0lPdM83MFtv2vWWyF8gxh8g/3zuXlBcysiUkbeW07ls+BVRzGAOi/zeq8hhA5so8Lk1nKGD88izFJempWQ3VYqEmil8C2n2EwK2ils=</latexit><latexit sha1_base64="j53NQuelycP7vJoTPq+b58fTRAE=">AAACZ3ichVHLSsNAFD2N7/poVZCCG7UorsqNKD5WBTcu+7C2UIskcdRgmoQkLWjxB1y4reBKQUT8DDf+gIt+QnFZwY0Lb9KAqKh3mJkzZ+65c2ZGtQ3d9YhaEamnt69/YHAoOjwyOhaLj0/suFbN0URBswzLKamKKwzdFAVP9wxRsh2hVFVDFNXjTX+/WBeOq1vmtndii0pVOTT1A11TPJ9aSq2v7MWTlKIgZn4COQRJhJGx4nfYxT4saKihCgETHmMDClxuZcgg2MxV0GDOYaQH+wJniLK2xlmCMxRmj3k85FU5ZE1e+zXdQK3xKQZ3h5UzmKdnuqcOPdEDten911qNoIbv5YRntasV9l7sPJF/+1dV5dnD0afqT88eDrAWeNXZux0w/i20rr5+2uzkN3LzjQW6oRf2f00teuQbmPVX7TYrcleI8gfI35/7J9hZSsmUkrPLyfRa+BWDmMYcFvm9V5HGFjIo8LlHuEATl5G2FJOmpEQ3VYqEmkl8CWn2AxC5imI=</latexit>

5 FUTURE WORK

<latexit sha1_base64="3WpXrD1Nmpw4kjTISmIXKxANqdM=">AAACZ3ichVHLSsNAFD2Nr1ofrQpScKMWxVW5kYLFVcGNy9raKqhIEqc1NE1CkhZq8QdcuK3gSkFE/Aw3/oCLfkJxqeDGhTdpQFTUO8zMmTP33Dkzo9qG7npE3Yg0MDg0PBIdjY2NT0zGE1PTZddqOJooaZZhObuq4gpDN0XJ0z1D7NqOUOqqIXbU2oa/v9MUjqtb5rbXssVBXamaekXXFM+nMmmSDxMpSlMQ8z+BHIIUwshbiVvs4wgWNDRQh4AJj7EBBS63Pcgg2MwdoM2cw0gP9gVOEWNtg7MEZyjM1nis8movZE1e+zXdQK3xKQZ3h5XzWKInuqMXeqR76tH7r7XaQQ3fS4tnta8V9mH8LFl8+1dV59nD8afqT88eKsgGXnX2bgeMfwutr2+edF6K64Wl9jJd0zP7v6IuPfANzOardrMlCpeI8QfI35/7JyivpmVKy1uZVC4bfkUUc1jECr/3GnLYRB4lPvcY5+jgItKT4tKslOynSpFQM4MvIS18APqnilc=</latexit><latexit sha1_base64="8PJK6Nm83Ibv5e25Lu3eDvaOolc=">AAACanichVG7SgNBFD1Z3/EVtVFsxBixCndtDFaCjZXkYRIhhrC7jrpks7vsTgIx+AN2VoJWCiLiZ9j4Axb5BNFOwcbCm82CqKh3mJkzZ+65c2ZGdy3Tl0TtiNLT29c/MDgUHR4ZHRuPTUwWfKfuGSJvOJbjbeuaLyzTFnlpSktsu57Qarolinp1vbNfbAjPNx17SzZdUa5p+7a5ZxqaZKpY1BpiU8hKLE5JCmLuJ1BDEEcYaSd2jR3swoGBOmoQsCEZW9DgcytBBcFlrowWcx4jM9gXOEKUtXXOEpyhMVvlcZ9XpZC1ed2p6Qdqg0+xuHusnEOCHuiGXuiebumR3n+t1QpqdLw0eda7WuFWxo+nc2//qmo8Sxx8qv70LLGHVODVZO9uwHRuYXT1jcPTl9xqNtFapEt6Yv8X1KY7voHdeDWuMiJ7jih/gPr9uX+CwnJSpaSaWY6vpcKvGMQs5rHE772CNWwgjXzg7gRnOI88K5PKjDLbTVUioWYKX0JZ+ABfWYxq</latexit>

WaveNet<latexit sha1_base64="cMc516U9a+f/IuOlnuho9eK5b80=">AAACaXichVG7SgNBFD1ZXzG+ojaiTTAosQl3bQxWARtLk5gHqMjuZhLXbHaX3U0gBn/Ayk7USkFE/Awbf8AinyCWCjYW3mwWRIN6h5k5c+aeO2dmVNvQXY+oE5IGBoeGR8KjkbHxicmp6PRMwbUajibymmVYTklVXGHopsh7umeIku0Ipa4aoq
sha1_base64="cMc516U9a+f/IuOlnuho9eK5b80=">AAACaXichVG7SgNBFD1ZXzG+ojaiTTAosQl3bQxWARtLk5gHqMjuZhLXbHaX3U0gBn/Ayk7USkFE/Awbf8AinyCWCjYW3mwWRIN6h5k5c+aeO2dmVNvQXY+oE5IGBoeGR8KjkbHxicmp6PRMwbUajibymmVYTklVXGHopsh7umeIku0Ipa4aoqjWNrr7xaZwXN0yt72WLfbqStXUK7qmeEwVEuWVdVfdj8YpSX7E+oEcgDiC2LKit9hFGRY0NFCHgAmPsQEFLrcdyCDYzO2hzZzDSPf3BY4RYW2DswRnKMzWeKzyaidgTV53a7q+WuNTDO4OK2NYoie6o1d6pHt6po9fa7X9Gl0vLZ7VnlbY+1Mnc7n3f1V1nj0cfKn+9OyhgpTvVWfvts90b6H19M2js9fcenapvUzX9ML+r6hDD3wDs/mm3WRE9hIR/gD553P3g8JqUqaknFmNp1PBV4SxgEUk+L3XkMYmtpDncw9xinNchF6kaWlOmu+lSqFAM4tvIcU/AUmUi2o=</latexit>

p oduces h ghe SNR sco es han he o he FFTNe vocode s How
eve me hodcanno ea ze h ghe qua y syn hes s because he
syn hes zed speech wave o ms nc ude some add ona no se s g
na s n a h ghe equency band compa ed w h subband me hods
w h a s ng e band npu A hough he subband me hods w h as ng e band npu nco po a e ewe no se componen s he syn hes zed speech qua y s no as h gh because he e s a ack o phasesh compensa on be ween subbands The e o e subband FFTNe
w h mu band npu shou d be u he nves ga ed o educe heno se componen s

2.09<latexit sha1_base64="/mlc+F+7AeJYnDOfegcsbFH6xI0=">AAACa3ichVG7SgNBFD1ZXzG+ojaiFsEQsQp3bRSrgI2dz2ggiuyukzhkX+xuAhr8AUsbC20URMTPsPEHLPwEESsFGwvvbhZEg3qX2Tlz5p47Z+7orin9gOgxoXR0dnX3JHtTff0Dg0Pp4ZFN36l7higajul4JV3zhSltUQxkYIqS6wnN0k2xpdcWw/2thvB86dgbwYErdiytasuKNLSAqdKyJ6vS1szddJbyFEWmHagxyCKOFSd9jW3swYGBOiwI2AgYm9Dg81eGCoLL3A6azHmMZLQvcIQUa+ucJThDY7bG/yqvyjFr8zqs6Udqg08xeXiszCBHD3RDr3RPt/REH7/WakY1Qi8HPOstrXB3h47H1t//VVk8B9j/Uv3pOUAF85FXyd7diAlvYbT0jcPT1/WFtVxzmi7pmf1f0CPd8Q3sxptxtSrWzpDiB1B/trsdbM7mVcqrq7PZwnz8FElMYAoz3O85FLCEFRSjPp/gDOeJF2VUGVcmW6lKItaM4lsouU9904zv</latexit><latexit sha1_base64="ya1fw+zABm7zMksY4oePmZonxeA=">AAACZ3ichVHLSsNAFD2N7/poVRDBTbVUXJUbEayuCm5cttZWoYokcarBNAlJWtDiD7hwW8GVgoj4GW78ARf9hOKyghsX3qQB0aLeYWbOnLnnzpkZ1TZ01yNqRaS+/oHBoeGR6OjY+EQsPjlVcq2ao4miZhmWs6sqrjB0UxQ93TPEru0IpaoaYkc92fD3d+rCcXXL3PZObbFfVY5MvaJriudTy+nM2kE8SWkKItEL5BAkEUbOit9jD4ewoKGGKgRMeIwNKHC5lSGDYDO3jwZzDiM92Bc4R5S1Nc4SnKEwe8LjEa/KIWvy2q/pBmqNTzG4O6xMIEUv9EAdeqZHatPHr7UaQQ3fyynPalcr7IPYxWzh/V9VlWcPx1+qPz17qCATeNXZux0w/i20rr5+1uwU1rdSjUW6pVf2f0MteuIbmPU37S4vtq4R5Q+Qfz53Lygtp2VKy/mVZDYTfsUw5rCAJX7vVWSxiRyKfO4xLtHEVaQtxaQZababKkVCzTS+hTT/CRa4imU=</latexit><latexit sha1_base64="QfJMIyRQbg1EiTpWxCMR4+0AHuw=">AAACZ3ichVHLSsNAFD2N7/poVZCCm2pRXJWbUlBcCW5c9mGrUIskcdTQNAlJWqjFH3DhVsGVgoj4GW78ARf9hOKyghsX3qQBUVHvMDNnztxz58yMahu66xF1ItLA4NDwyOhYdHxicioWn54pu1bD0URJswzL2VUVVxi6KUqe7hli13aEUlcNsaPWNv39naZwXN0yt72WLap15cjUD3VN8Xwqm85k9+MpSlMQyZ9ADkEKYeSs+B32cAALGhqoQ8CEx9iAApdbBTIINnNVtJlzGOnBvsApoqxtcJbgDIXZGo9HvKqErMlrv6YbqDU+xeDusDKJJXqme+rREz1Ql95/rdUOavheWjyrfa2w92NnieLbv6o6zx6OP1V/evZwiLXAq87e7YDxb6H19c2Ti15xvbDUXqYbemH/19ShR76B2XzVbvOicIUof4D8/bl/gnImLVNazmdTG2vhV4xiHotY4fdexQa2kEOJzz3GOS5wGelKMWlOSvRTpUiomcWXkBY+AAS4ilw=</latexit>

12
3
4
Mean opinion score<latexit sha1_base64="B9icVl7J9+4mxkv3fBFtLYkXSUQ=">AAACZHichVG7SgNBFD1Z3zGaqAiCIGKIWIW7IihWgo2lUWMElbC7jrpkX+xOAhr8AW0VCysFEfEzbPwBC39AEMsINhbe3SyIBvUOM3PmzD13zszonmUGkugpobS1d3R2dfcke1N9/enMwOB64FZ9QxQN13L9DV0LhGU6oihNaYkNzxearVuipFcWw/1STfiB6Tpr8sAT27a255i7pqFJpgpqOZOlPEUx3grUGGQRx7KbucEWduDCQBU2BBxIxhY0BNw2oYLgMbeNOnM+IzPaFzhCkrVVzhKcoTFb4XGPV5sx6/A6rBlEaoNPsbj7rBxHjh7plhr0QHf0Qh+/1qpHNUIvBzzrTa3wyunjkdX3f1U2zxL7X6o/PUvsYi7yarJ3L2LCWxhNfe3wvLE6v5KrT9IVvbL/S3qie76BU3szrgti5QJJ/gD153O3gvXpvEp5tTCTXZiLv6Ibo5jAFL/3LBawhGUU+VyBE5ziLPGspJQhZbiZqiRizRC+hTL2CWVKiac=</latexit><latexit<latexit sha1_base64="KZjta3BSg4MPfJx2vCaBZwt1c3c=">AAACZHichVHLSsNAFD2Nr1ofrRZBEKRYFFflpggWVwU3Lq21tlBLSeKooWkSkrRQiz+gW8WFKwUR8TPc+AMu+gOCuFRw48LbNCAq6h1m5syZe+6cmVFtQ3c9ok5I6usfGBwKD0dGRsfGo7GJyS3XajiaKGiWYTklVXGFoZui4OmeIUq2I5S6aoiiWlvt7hebwnF1y9z0Wrao1JU9U9/VNcVjKpeuxpKUIj8SP4EcgCSCWLdi19jGDixoaKAOARMeYwMKXG5lyCDYzFXQZs5hpPv7AoeIsLbBWYIzFGZrPO7xqhywJq+7NV1frfEpBneHlQnM0wPd0Avd0y090fuvtdp+ja6XFs9qTyvsavRoOv/2r6rOs4f9T9Wfnj3sIuN71dm77TPdW2g9ffPg7CW/sjHfXqBLemb/F9ShO76B2XzVrnJi4xwR/gD5+3P/BFvplEwpObeUzGaCrwhjBnNY5PdeRhZrWEeBzxU4xglOQ4/SqBSXpnqpUijQxPElpNkPZ0qJqA==</latexit><latexit<latexit sha1_base64="Ys7eS+klwBCZ5J7UyDpzHftULAc=">AAACZHichVHLSsNAFD2Nr1ofrRZBEEQsiqtyo4LFVcGNyz5sLVSRJI4amiYhSQu1+AO6VVy4UhARP8ONP+CiPyCIywpuXHibBkRFvcPMnDlzz50zM6pt6K5H1ApJPb19/QPhwcjQ8MhoNDY2XnStmqOJgmYZllNSFVcYuikKnu4ZomQ7QqmqhthUK2ud/c26cFzdMje8hi22q8q+qe/pmuIxlV3aiSUoSX7M/ARyABIIImPFbrCFXVjQUEMVAiY8xgYUuNzKkEGwmdtGkzmHke7vCxwhwtoaZwnOUJit8LjPq3LAmrzu1HR9tcanGNwdVs5gjh7pltr0QHf0TO+/1mr6NTpeGjyrXa2wd6LHk/m3f1VVnj0cfKr+9OxhDynfq87ebZ/p3ELr6uuH5+38am6uOU9X9ML+L6lF93wDs/6qXWdF7gIR/gD5+3P/BMXFpExJObucSKeCrwhjCrNY4PdeQRrryKDA5wqc4BRnoSdpWIpLE91UKRRo4vgS0vQHaUqJqQ==</latexit><latexit<latexit sha1_base64="nSQEQpvtQ9UCC3/FtNQtFipFFvA=">AAACYHichVHLSsNAFD2N7/iquhF0USyKq3IjguKq4MalrbYKWkoSRw2mSUimhVr8Ad0qLlwpiIif4cYfcNEfEMRlBTcuvEkDoqLeYWbOnLnnzpkZw7OtQBI1E0pHZ1d3T2+f2j8wODScVEeKgVv1TVEwXdv1Nw09ELbliIK0pC02PV/oFcMWG8bBcri/URN+YLnOuqx7olTR9xxr1zJ1yVRuvpxMU4aiSP0EWgzSiGPVTd5gGztwYaKKCgQcSMY2dATctqCB4DFXQoM5n5EV7QscQWVtlbMEZ+jMHvC4x6utmHV4HdYMIrXJp9jcfVamME2PdEsteqA7eqb3X2s1ohqhlzrPRlsrvPLw8fja27+qCs8S+5+qPz1L7GIx8mqxdy9iwluYbX3t8Ly1tpSfbszQFb2w/0tq0j3fwKm9mtc5kb+Ayh+gfX/un6A4l9Eoo6Wzi/FH9GICU5jl115AFitYRYFPFTjBKc4ST8qAMtZOVBKxYhRfQpn8ABTuiQM=</latexit><latexit
sha1_base64="z+3VOl2tLdnBpWLya1lPoJ502TI=">AAACZHichVHLSsNAFD2Nr1ofrRZBEKRYFFflRgoWVwU3Lq21tlBLSeKooWkSkrRQiz+gW8WFKwUR8TPc+AMu+gOCuFRw48LbNCAq6h1m5syZe+6cmVFtQ3c9ok5I6usfGBwKD0dGRsfGo7GJyS3XajiaKGiWYTklVXGFoZui4OmeIUq2I5S6aoiiWlvt7hebwnF1y9z0Wrao1JU9U9/VNcVjKpeuxpKUIj8SP4EcgCSCWLdi19jGDixoaKAOARMeYwMKXG5lyCDYzFXQZs5hpPv7AoeIsLbBWYIzFGZrPO7xqhywJq+7NV1frfEpBneHlQnM0wPd0Avd0y090fuvtdp+ja6XFs9qTyvsavRoOv/2r6rOs4f9T9Wfnj3sIuN71dm77TPdW2g9ffPg7CW/sjHfXqBLemb/F9ShO76B2XzVrnJi4xwR/gD5+3P/BFtLKZlSci6dzGaCrwhjBnNY5PdeRhZrWEeBzxU4xglOQ4/SqBSXpnqpUijQxPElpNkPa0qJqg==</latexit><latexit sha1_base64="VLteJgqmf+VhX8IyWxbcYux9C7k=">AAACd3ichVHLSsNAFD2N7/qqdSO4sFiUrsqNG4urghs3QrW2FbSUJE51ME1Ckha1+AP+gAtBUBAVP8ONP+CinyAuFURw4W0aEC3qDZk598w9d87M6I4pPZ+oFVF6evv6BwaHosMjo2PjsYl40bPrriEKhm3a7qauecKUlij40jfFpuMKraaboqTvL7fXSw3hetK2NvxDR5Rr2q4lq9LQfKYqsfiq0KyE7UiL04Rn2K6oxJKUpiAS3UANQRJh5OzYNbaxAxsG6qhBwILP2IQGj78tqCA4zJXRZM5lJIN1gWNEWVvnKsEVGrP7PO5ythWyFuftnl6gNngXk3+XlQnM0SPd0gs90B090cevvZpBj7aXQ571jlY4lfGTqfzbv6oazz72vlR/evZRRSbwKtm7EzDtUxgdfePo9CW/tD7XnKdLemb/F9Siez6B1Xg1rtbE+hmi/ADqz+vuBsWFtEppdW0hmc2ETzGIacwixfe9iCxWkEOB9z3AOa5xE3lXZpR5JdUpVSKhZhLfQlE/AfGmkSg=</latexit>

F g 8 Resu s o MOS es w h 10 s en ng sub ec s “ es ” “ns ”“sb ” and “mb ” deno e es dua connec on no se shap ng subband
and mu band npu espec ve y
4 4 D scuss on
Compa ed w h he van a FFTNe a he MOS es esu s F g 8
show ha he syn hes zed speech qua y can be s gn fican y mp oved by app oaches b –a hough hey canno ma ch he peo mance o he WaveNe w h no se shap ng h Th s ve fies hee ec veness o he no se shap ng and subband app oaches o FFT
Ne vocode s n pa cu a FFTNe w h no se shap ng c s gn
can y ou pe o ms he STRA GHT vocode w hou me ceps acomp ess on t es esu w h p = 0 049 < 0 05 The SNR esu so he no se shap ng me hods c and h we e owe han hose ohe me hods w hou no se shap ng b and g as he no se shaped
speech s gna s a e wh e han he o g na s gna s and he h ghe
oss sco es nd ca e ha hey a e mo e d ficu o p ed c Howeve
he syn hes zed speech qua y o he me hods w h no se shap ng
was success u y mp oved n con as he subband speech s gna s
a e cove ed by he squa e oo Hann w ndow based ove apped SSB
fi e bank and he owe oss sco es sugges ha hey a e mo e eas
y p ed c ed 46 The es dua connec ons can success u y mp ove he FFTNe mode accu acy and he no se shap ng and subband app oaches can mp ove he syn hes zed equency esponsen add on he subband me hod w h mu band npu
can e
ec ve y compensa e he phase sh be ween subbands because

To enhance he syn hes zed speech qua y o FFTNe o ha o
WaveNe and he o g na speech s gna s a e na ve ne wo ks uc u e mod fica ons such as he ga ed ac va on un s used n
WaveNe 17 and s gna p ocess ng echn ques such as subband
me hods w h mu band npu comb ned w h no se shap ng w benves ga ed a h ghe samp ng equenc es such as 24 18 25 34
and 48 kHz 24 47
n add on expe men s us ng a ema espeech co pus w be conduc ed Fu he mo e bandw d h exens on 41 47 63 16 b nea PCM aw aud o p ed c on w h
d sc e zed og s c m x u e ke hood 18 25 34 55 o a dua so
max aye 20 syn hes s w h me spec og am npu 25 39 andsyn hes s speed eva ua ons o FFTNe vocode s o e s gn ficanscope o u u e s ud es
6 CONCLUSIONS
To mp ove he syn hes zed speech qua y o FFTNe vocode s
wh e e a n ng he sma ne wo k mode s ze equ ed o ea me
syn hes s h s pape has desc bed he o ow ng ou app oaches 1
Res dua connec ons we e n oduced n o FFTNe o mp ove hep ed c on accu acy 2 No se shap ng and 3 subband app oaches
we e d ec y app ed o FFTNe vocode s 4 A subband FFTNe
vocode w h mu band npu was used o d ec y compensa e hephase sh be ween subbands The p oposed app oaches we e eva
ua ed h ough a se es o ob ec ve and sub ec ve expe men s us ng
a Japanese ma e co pus w h a samp ng equency o 16 kHz andhe esu s we e compa ed w h hose om STRA GHT w houme ceps a comp ess on as we as van a FFTNe and WaveNe
vocode s The esu s sugges ha he p oposed app oaches can
s gn fican y mp ove he syn hes zed speech qua y o FFTNe
vocode s n pa cu a he p oposed FFTNe vocode w h no seshap ng s gn fican y ou pe o ms he STRA GHT vocode

7. REFERENCES[1] K. Tokuda, Y. Nankaku, T. Toda, H. Zen, J. Yamagishi, andK. Oura, “Speech synthesis based on hidden Markov models,”
Proc. IEEE, vol. 101, no. 5, pp. 1234–1252, May 2013.
[2] S. H. Mohammadi and A. Kain, “An overview of voice conversion systems,” Speech Commun., vol. 88, pp. 65–82, Apr.
2017.
[3] T. Fukada, K. Tokuda, T. Kobayashi, and S. Imai, “An adaptive algorithm for mel-cepstral analysis of speech,” in Proc.
ICASSP, Mar. 1992, vol. 1, pp. 137–140.
[4] H. Kawahara, I. Masuda-Katsuse, and A. de Cheveigné, “Restructuring speech representations using a pitch-adaptive timefrequency smoothing and an instantaneous-frequency-based
F0 extraction: Possible role of a repetitive structure in sounds,”
Speech Commun., vol. 27, no. 3–4, pp. 187–207, Apr. 1999.
[5] M. Morise, F. Yokomori, and K. Ozawa, “WORLD: a vocoderbased high-quality speech synthesis system for real-time applications,” IEICE trans. Inf. Syst., vol. E99-D, no. 7, pp. 1877–
1884, July 2016.
[6] C. Demiroglu, O. Buyuk, A. Khodabakhsh, and R. Maia,“Postprocessing synthetic speech with a complex cepstrum
vocoder for spoofing phase-based synthetic speech detectors,”
IEEE J. Sel. Topics Signal Process., vol. 11, no. 4, pp. 671–
683, June 2017.
[7] D. Erro, I. Sainz, E. Navas, and I. Hernaez, “Harmonics plus
noise model based vocoder for statistical parametric speechsynthesis,” IEEE J. Sel. Topics Signal. Process., vol. 8, no.
2, pp. 184–194, Apr. 2014.
[8] Y. Agiomyrgiannakis, “Vocaine the vocoder and applicationsin speech synthesis,” in Proc. ICASSP, Apr. 2015, pp. 4230–
4234.
[9] S. Takaki and J. Yamagishi, “A deep auto-encoder based lowdimensional feature extraction from FFT spectral envelopes forstatistical parametric speech synthesis,” in Proc. ICASSP, Mar.
2016, pp. 5535–5539.
[10] M. Airaksinen, B. Bollepalli, L. Juvela, Z. Wu, S. King, and
P. Alku, “GlottDNN — A full-band glottal vocoder for statistical parametric speech synthesis,” in Proc. Interspeech, Sept.
2016, pp. 2473–2477.
[11] M. Airaksinen, L. Juvela, B. Bollepalli, J. Yamagishi, and
P. Alku, “Comparison between STRAIGHT, glottal, and sinusoidal vocoding in statistical parametric speech synthesis,”
IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 26, no.
9, pp. 1658–1670, Sept. 2018.
[12] T. Okamoto, K. Tachibana, T. Toda, Y. Shiga, and H. Kawai,“Deep neural network-based power spectrum reconstruction to
improve quality of vocoded speech with limited acoustic parameters,” Acoust. Sci. Tech., vol. 39, no. 2, pp. 163–166, Mar.
2018.
[13] S. Takaki, H. Kameoka, and J. Yamagishi, “Direct modeling of
frequency spectra and waveform generation based on phase recovery for DNN-based speech synthesis,” in Proc. Interspeech,
Aug. 2017, pp. 1128–1132.
[14] Y. Wang, RJ Skerry-Ryan, D. Stanton, Y. Wu, R. Weiss,
N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, Q. Le,
Y. Agiomyrgiannakis, R. Clark, and R. A. Saurous, “Tacotron:
Towards end-to-end speech synthesis,” in Proc. Interspeech,
Aug. 2017, pp. 4006–4010.

[15] D. Griffin and J. S. Lim, “Signal estimation from modified
short-time fourier transform,” IEEE Trans. Acoust. Speech Signal Process., vol. 32, no. 2, pp. 236–243, Apr. 1984.
[16] Z.-H. Ling, S.-Y. Kang, H. Zen, A. Senior, M. Schuster, X.-J.
Qian, H. Meng, and L. Deng, “Deep learning for acoustic modeling in parametric speech generation: A systematic review ofexisting techniques and future trends,” IEEE Signal Process.
Mag., vol. 32, no. 3, pp. 35–52, May 2015.
[17] A. van den Oord, S. Dieleman, H. Zen, K. Simonyan,
O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and
K. Kavukcuoglu, “WaveNet: A generative model for raw
audio,” arXiv preprint arXiv:1609.03499, Sept. 2016, (unreviewed manuscript).
[18] A. van den Oord, Y. Li, I. Babuschkin, K. Simonyan,
O. Vinyals, K. Kavukcuoglu, G. van den Driessche, E. Lockhart, L. C. Cobo, F. Stimberg, N. Casagrande, D. Grewe,
S. Noury, S. Dieleman, E. Elsen, N. Kalchbrenner, H. Zen,
A. Graves, H. King, T. Walters, D. Belov, and D. Hassabis,“Parallel WaveNet: Fast high-fidelity speech synthesis,” inProc. ICML, July 2018, pp. 3915–3923.
[19] S. Mehri, K. Kumar, I. Gulrajani, R. Kumar, S. Jain, J. Sotelo,A. Courville, and Y. Bengio, “SampleRNN: An unconditional
end-to-end neural audio generation model,” in Proc. ICLR,
Apr. 2017.
[20] N. Kalchbrenner, E. Elsen, K. Simonyan, S. Noury,
N. Casagrande, E. Lockhart, F. Stimberg, A. van den Oord,
S. Dieleman, and K. Kavukcuoglu, “Efficient neural audio synthesis,” in Proc. ICML, July 2018, pp. 2415–2424.
[21] J. Sotelo, S. Mehri, K. Kumar, J. F. Santos, K. Kastner,
A. Courville, and Y. Bengio, “Char2Wav: End-to-end speechsynthesis,” in Proc. ICLR, Apr. 2017.
[22] S. O. Arik, M. Chrzanowski, A. Coates, G. Diamos, A. Gibiansky, Y. Kang, X. Li, J. Miller, A. Ng, J. Raiman, S. Sengupta, and M. Shoeybi, “Deep voice: Real-time neural text-tospeech,” in Proc. ICML, Aug. 2017, pp. 195–204.
[23] S. O. Arik, G. Diamos, A. Gibiansky, J. Miller, K. Peng,
W. Ping, J. Raiman, and Y. Zhou, “Deep voice 2: Multispeaker neural text-to-speech,” in Proc. NIPS, Dec. 2017, pp.
2966–2974.
[24] W. Ping, K. Peng, A. Gibiansky, S. O. Arik, A. Kannan,
S. Narang, J. Raiman, and J. Miller, “Deep voice 3: 2000speaker neural text-to-speech,” in Proc. ICLR, Apr. 2018.
[25] J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang,
Z. Chen, Y. Zhang, Y. Wang, RJ Skerry-Ryan, R. A. Saurous,
Y. Agiomyrgiannakis, and Y. Wu, “Natural TTS synthesis byconditioning WaveNet on mel spectrogram predictions,” inProc. ICASSP, Apr. 2018, pp. 4779–4783.
[26] RJ Skerry-Ryan, E. Battenberg, Y. Xiao, Y. Wang, D. Stanton,
J. Shor, R. J. Weiss, R. Clark, and R. A. Saurous, “Towards
end-to-end prosody transfer for expressive speech synthesis
with tacotron,” in Proc. ICML, July 2018, pp. 4700–4709.
[27] Y. Wang, D. Stanton, Y. Zhang, RJ Skerry-Ryan, E. Battenberg, J. Shor, Y. Xiao, F. Ren, Y. Jia, and R. A. Saurous, “Styletokens: Unsupervised style modeling, control and transfer in
end-to-end speech synthesis,” in Proc. ICML, July 2018, pp.
5167–5176.
[28] A. Tamamori, T. Hayashi, K. Kobayashi, K. Takeda, andT. Toda, “Speaker-dependent WaveNet vocoder,” in Proc. Interspeech, Aug. 2017, pp. 1118–1122.

[29] Y. Ai, H.-C. Wu, and Z.-H. Ling, “SampleRNN-based neuralvocoder for statistical parametric speech synthesis,” in Proc.
ICASSP, Apr. 2018, pp. 5659–5663.
[30] X. Wang, J. Lorenzo-Trueba, S. Takaki, L. Juvela, and J. Yamagishi, “A comparison of recent waveform generation andacoustic modeling methods for neural-network-based speechsynthesis,” in Proc. ICASSP, Apr. 2018, pp. 4804–4808.
[31] N. Adiga, V. Tsiaras, and Y. Stylianou, “On the use of
WaveNet as a statistical vocoder,” in Proc. ICASSP, Apr. 2018,
pp. 5674–5678.
[32] J. Vı́t, Z. Hanzlı́ček, and J. Matousěk, “On the analysis oftraining data for WaveNet-based speech synthesis,” in Proc.
ICASSP, Apr. 2018, pp. 5684–5688.
[33] Y. Gu and Y. Kang, “Multi-task WaveNet: A multi-task generative model for statistical parametric speech synthesis without
fundamental frequency conditions,” in Proc. Interspeech, Sept.
2018, pp. 2007–2011.
[34] L. Juvela, V. Tsiaras, B. Bollepalli, M. Airaksinen, J. Yamagishi, and P. Alku, “Speaker-independent raw waveform modelfor glottal excitation,” in Proc. Interspeech, Sept. 2018, pp.
2012–2016.
[35] K. Kobayashi, A. Tamamori, T. Hayashi, and T. Toda, “Statistical voice conversion with WaveNet-based waveform generation,” in Proc. Interspeech, Aug. 2017, pp. 1138–1142.
[36] A. van den Oord, O. Vinyals, and K. Kavukcuoglu, “Neuraldiscrete representation learning,” in Proc. NIPS, Dec. 2017,
pp. 6306–6315.
[37] J. Niwa, T. Yoshimura, K. Hashimoto, K. Oura, Y. Nankaku,
and K. Tokuda,“Statistical voice conversion based on
WaveNet,” in Proc. ICASSP, Apr. 2018, pp. 5289–5293.
[38] B. Sisman, M. Zhang, and H. Li, “A voice conversion framework with tandem feature sparse representation and speakeradapted WaveNet vocoder,” in Proc. Interspeech, Sept. 2018,
pp. 1978–1982.
[39] K. Chen, B. Chen, J. Lai, and K. Yu, “High-quality voice conversion using spectrogram-based WaveNet vocoder,” in Proc.
Interspeech, Sept. 2018, pp. 1993–1997.
[40] T. Hayashi, A. Tamamori, K. Kobayashi, K. Takeda, andT. Toda, “An investigation of multi-speaker training for
WaveNet vocoder,” in Proc. ASRU, Dec. 2017, pp. 712–718.
[41] W. B. Kleijn, F. S. C. Lim, A. Luebs, J. Skoglund, F. Stimberg,
Q. Wang, and T. C. Walters, “WaveNet based low rate speechcoding,” in Proc. ICASSP, Apr. 2018, pp. 676–680.
[42] Z. Jin, A. Finkelstein, G. J. Mysore, and J. Lu, “FFTNet: A
real-time speaker-dependent neural vocoder,” in Proc. ICASSP,
Apr. 2018, pp. 2251–2255.
[43] S. F. Boll, “Suppression of acoustic noise in speech using spectral subtraction,” IEEE Trans. Acoust. Speech, Signal Process.,
vol. 27, no. 2, pp. 113–120, Apr. 1979.
[44] K. Tachibana, T. Toda, Y. Shiga, and H. Kawai, “An investigation of noise shaping with perceptual weighting for WaveNetbased speech generation,” in Proc. ICASSP, Apr. 2018, pp.
5664–5668.
[45] T. Yoshimura, K. Hashimoto, K. Oura, Y. Nankaku, andK. Tokuda, “Mel-cepstrum-based quantization noise shaping
applied to neural-network-based speech waveform synthesis,”
IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 26, no.
7, pp. 1173–1180, July 2018.

[46] T. Okamoto, K. Tachibana, T. Toda, Y. Shiga, and H. Kawai,“Subband WaveNet with overlapped single-sideband filterbanks,” in Proc. ASRU, Dec. 2017, pp. 698–704.
[47] T. Okamoto, K. Tachibana, T. Toda, Y. Shiga, and H. Kawai,“An investigation of subband WaveNet vocoder covering entire audible frequency range with limited acoustic features,” inProc. ICASSP, Apr. 2018, pp. 5654–5658.
[48] P. Ramachandran, T. L. Paine, P. Khorrami, M. Babaeizadeh,
S. Chang, Y. Zhang, M. Hasegawa-Johnson, R. Campbell, andT. Huang, “Fast generation for convolutional autoregressivemodels,” in Proc. ICLR, Apr. 2017.
[49] ITU-T. Recommendation G. 711, Pulse Code Modulation(PCM) of voice frequencies, 1988.
[50] B. S. Atal and M. R. Schroeder, “Predictive coding of speech
signals and subjective error criteria,” IEEE Trans. Acoust.
Speech, Signal Process., vol. 27, no. 3, pp. 247–254, June
1979.
[51] K. Tokuda, T. Kobayashi, T. Masuko, and S. Imai, “Melgeneralized cepstral analysis — A unified approach to speech
spectral estimation,” in Proc. ICSLP, Sept. 1994, pp. 1043–
1046.
[52] R. E. Crociere and L. R. Rabiner, Multirate Digital Signal
Processing, Prentice Hall, Englewood Cliffs, 1983.
[53] P. P. Vaidyanathan, Multirate Systems and Filter Banks, Prentice Hall, Upper Saddle River, 1993.
[54] A. van den Oord, N. Kalchbrenner, O. Vinyals, L. Espeholt,
A. Graves, and K. Kavukcuoglu, “Conditional image generation with pixelCNN decoders,” in Proc. NIPS, Dec. 2016, pp.
4790–4798.
[55] T. Salimans, A. Karpathy, X. Chen, and D. P. Kingma, “PixelCNN++: Improving the pixelCNN with discretized logistic
mixture likelihood and other modifications,” in Proc. ICLR,
Apr. 2017.
[56] M. Blaauw and J. Bonada, “A neural parametric singing synthesizer,” in Proc. Interspeech, Aug. 2017, pp. 4001–4005.
[57] M. Blaauw and J. Bonada, “A neural parametric singing synthesizer modeling timbre and expression from natural songs,”
Appl. Sci., vol. 7, no. 12, 1313, Dec. 2017.
[58] Y.-C. Wu, K. Kobayashi, T. Hayashi, P. L. Tobing, andT. Toda, “Collapsed speech segment detection and suppression for WaveNet vocoder,” in Proc. Interspeech, Sept. 2018,
pp. 1988–1992.
[59] H. Kawahara, A. de Cheveigné, H. Banno, T. Takahashi, andT. Irino, “Nearly defect-free F0 trajectory extraction for expressive speech modifications based on STRAIGHT,” in Proc.
Interspeech, Sept. 2005, pp. 537–540.
[60] H. Zen, T. Toda, M. Nakamura, and T. Tokuda, “Details of the
Nitech HMM-based speech synthesis system for the Blizzard
Challenge 2005,” IEICE Trans. Inf. Syst., vol. E90-D, no. 1,
pp. 325–333, Jan. 2007.
[61] D. P. Kingma and J. L. Ba, “Adam: A method for stochastic
optimization,” in Proc. ICLR, May 2015.
[62] ITU-T Recommendation P. 800, Methods for subjective determination of transmission quality, 1996.
[63] Y. Gu and Z.-H. Ling, “Waveform modeling using stacked dilated convolutional neural networks for speech bandwidth extension,” in Proc. Interspeech, Aug. 2017, pp. 1123–1127.