## 2 Background  
### Hierarchical Variational Auto-Encoder  
The variational autoencoder is a neural network generative model [10]. Given a dataset \(X=[x_{1},x_{2},..,x_{N}]\), VAE defines a prior distribution \(p(z)\) and models the joint distribution \(p(x,z)\) with \(p(z)p(x|z)\), where \(p(x|z)\) is the decoder to generate data point x from latent variable z. Due to true posterior \(p(z|x)\) being usually intractable, VAE utilizes an encoder \(q(z|x)\) to approximate the posterior distribution. Generally, the prior distribution \(p(z)\) usually adopts the standard Gaussian distribution with independent components. VAE is optimized by minimizing reconstruction loss and KL divergence between prior distribution \(p(z)\) and the estimated posterior \(q(z|x)\) using resampling tricks.  
\[\log p_{\theta}(x)\geq\mathbb{E}_{q_{\theta}(z|x)}\log p_{\theta}(x\mid z)- KL\big{[}q_{\theta}(z\mid x)||p(z)\big{]}\mid \tag{1}\]  
However, the assumption of a Gaussian distribution makes it difficult to model a more complex distribution. If the latent variable needs to model a more complex distribution, an ordinary VAE will lead to oversmoothing. Hierarchical variational autoencoders (HVAEs) [11,12] improve the complexity of both prior and posterior distributions by modeling hierarchically-dependent latent variables. A hierarchical VAE can be viewed as a series of VAEs stacked together. Apart from observed variables \(x\), it contains hierarchical latent variables \(\{z_{1},z_{2},\ldots,z_{L}\}\), where \(L\) represents the number of hierarchies. According to the chain rule, the joint distribution can be decomposed into:  
\[p(x,z_{1},\ldots,z_{L})=p(x\mid z_{\geq 1})\prod_{\ell=1}^{L}p(z_{\ell}\mid z_{< \ell}) \tag{2}\]  
The prior and estimated posterior distributions are \(p_{\theta}(z)=\prod_{\ell}p_{\theta}(z_{\ell}\mid z_{<\ell})\) and \(q_{\phi}(z\mid x)=\prod_{\ell}q_{\phi}(z_{\ell}\mid z_{<\ell},x)\), respectively. The conditional distribution of each level of prior and estimated posterior adopts the standard Gaussian distribution with independent components. Hierarchical VAEs extract the posterior distribution from the data based on a bottom-up path and follow a top-down path to generate the prior distribution and reconstruct samples. This architecture helps the model learn the hierarchy between latent variables efficiently. The evidence lower bound (ELBO) of HVAE are  
\[\log p_{\theta}(x)\geq\mathbb{E}_{q_{\phi}(z|x)}\log p_{\theta}(x\mid z)-\sum _{\ell=1}^{L}\mathbb{E}_{q_{\phi}(z_{<\ell}|x)}\big{[}KL\big{[}q_{\phi}(z_{ \ell}\mid x,z_{<\ell})\mid p_{\theta}(z_{\ell}\mid z_{<\ell})\big{]} \tag{3}\]  
where \(q_{\phi}(z_{\ell}\mid x,z_{<\ell})\) and \(p_{\theta}(z_{\ell}\mid z_{<\ell})\) are the approximate posterior and prediction priors at the \(\ell\)th layer, respectively.  
HVAE was first applied on the high-resolution image task, and then BVAE-TTS [13], and VARA-TTS [14] applies it to the acoustic model of speech synthesis. However, they only modeled the generation process from the phoneme to the Mel spectrogram. They did not consider the hierarchical characteristic of speech in the network structure's design. In addition, due to the posterior collapse, some hidden variables in the hierarchical structure are not activated. The Mel spectrogram generated by BVAE-TTS is very blurred. Unlike these two approaches, we built hierarchical networks that directly generate waveforms. Furthermore, we closely combine the hierarchical properties of speech with hierarchical VAE to further improve prosodic expressiveness.