# An Emotion Speech Synthesis Method Based on VITS
## 2 Vits  
VITS is a high-performance speech synthesis model that combines variational inference augmented with normalizing flows and an adversarial training process. VITS uses Variational AutoEncoder (VAE) and applies normalized flows [21] to conditional prior distribution and adversarial training on waveform domains to improve the quality of the speech waveform. By connecting the two modules of the TTS system with latent variables, complete end-to-end learning is achieved, which can produce a more natural sound audio than the current two-stage model. Through random modeling on hidden variables and using a random duration predictor, the diversity of synthesized speech is improved. The same text input can synthesize speech with different tones and prosody. VITS can be regarded as a conditioned VAE for a maximized variational Lower Bound, namely ELBO (Evidence Lower Bound):  
\[\log p_{\theta}(x|c)\geq\mathbb{E}_{q_{\theta}(z|x)}\bigg{[}\log p_{\theta}(x| z)-\log\frac{q_{\phi}(z|x)}{p_{\theta}(z|c)}\bigg{]} \tag{1}\]  
In the formula, \(p_{\theta}(z|c)\) and \(q_{\phi}(z|x)\), respectively, represent the prior distribution of the potential variable z and an approximate posterior distribution, and \(\log p_{\theta}(x|z)\) is the likelihood function of a data point x. The sum of reconstructed loss, \(-\log p_{\theta}(x|z)\), and \(kl\) divergence, \(\log q_{\phi}(z|x)-\log p_{\theta}(z|c)\), can be used as the loss of training, namely -ELBO. VITS use mel-spectrogram instead of the original waveforms, represented by \(x_{mel}\). \(L1\) losses are calculated between the predicted and target mel-spectrogram as reconstruction losses in training:  
\[L_{recon}=\parallel x_{mel}-\hat{x}_{mel}\parallel_{1} \tag{2}\]  
The \(kl\) divergence is as follows, where \(x_{lin}\) is the linear-scale spectrogram of target speech:  
\[L_{kl}=\log q_{\phi}(z|x_{lin})-\log p_{\theta}(z|c_{text},A), \tag{3}\]  
\[z\sim q_{\phi}(z|x_{lin})=N\big{(}z;\mu_{\phi}(x_{lin}),\sigma_{\phi}(x_{lin} )\big{)}\]  
To estimate the alignment between the input text and the target speech \(A\), VITS adopts the Monotonic Alignment Search (MAS), which maximizes the likelihood of data parameterized by a normalizing flow, \(f\):  
\[ \begin{split} A&=\underset{\hat{A}}{\text{argmax}} \log p\big{(}x|c_{text},\hat{A}\big{)}\\ &=\underset{\hat{A}}{\text{argmax}}\log N\big{(}f(x);\mu\big{(}c_ {text},\hat{A}\big{)},\sigma\big{(}c_{text},\hat{A}\big{)}\big{)}\end{split}  \tag{4}\]  
VITS can calculate the duration of each input tag, \(d\), by adding all the columns in each of the estimated aligned rows. In order to achieve a more realistic rhythm of human speech, VITS adds a stochastic duration predictor and introduces two random variables, \(u\) and \(v\), which have the same time resolution and dimension as the duration series, d, and are used for variational dequantization and variational data augmentation, respectively. The two variables are sampled by an approximate posterior distribution. The resulting goal is a variational lower bound on the logarithmic likelihood value of the phoneme duration. Training loss, \(L_{dur}\), is the lower bound of negative variation:  
\[\log p_{\theta}(d|c_{text})\geq\mathbb{E}_{q_{\phi}(u,v|d,c_{text})}\bigg{[} \log\frac{p_{\theta}(d-u,v|c_{text})}{q_{\phi}(u,v|d,c_{text})}\bigg{]} \tag{5}\]VITS uses adversarial training, with a discriminator, \(D\), distinguishing between the output generated by the decoder, \(G\), and the ground-truth waveform, \(y\).