# Comparative Study for Multi-Speaker Mongolian TTS with a New Corpus
## 4 Speech Synthesis Experiments
### Experimental Setup
#### 4.1.1 Experimental Steps of TTS Based on the FastSpeech2 Model and HiFi-GAN Vocoder  
We used the TensorFlowTTS toolkit ([[https://github.com/TensorSpeech/TensorFlowTTS](https://github.com/TensorSpeech/TensorFlowTTS)]([https://github.com/TensorSpeech/TensorFlowTTS](https://github.com/TensorSpeech/TensorFlowTTS)), accessed on 11 November 2022) to build an end-to-end TTS model based on the FastSpeech2 model. As shown in Figure 3, the FastSpeech2 model converts the input Mongolian text into Mel-spectrogram features, and as shown in Figure 4, the HiFi-GAN vocoder reconstructs the waveform from the Mel-spectrogram features. We added a speaker encoder layer to FastSpeech2 to implement a multi-speaker TTS system as a way to match our multi-speaker dataset. The speaker embedding layer inputs the speaker id as an integer code and initializes them with random weights for learning speaker embeddings. The embedding layer does not need to be trained separately and can be directly embedded in the FastSpeech2 model to learn together. The speaker encoder consists of a speaker embedding layer, a dense layer, and a softplus layer. For the network model structure, we set the number of speakers to 3, the dimension of speaker embedding to 384, and the hidden layer to 4. In the decoder, the hidden layer size was 384, and the number of hidden layers was 4.  
Figure 3: The structure of the FastSpeech2+HiFi-GAN model. We implemented the multi-speaker FastSpeech2 by adding the speaker encoder module.  
The HiFi-GAN vocoder built the network through a generative adversarial network to convert Mel-spectrogram into high-quality audio. The generator of HiFi-GAN consists of an upsampling structure, which consists of a one-dimensional transposed convolution and a multi-receptive filed fusion module, which is responsible for optimizing the upsampling points. HiFi-GAN, as a generative adversarial network, has two kinds of discriminators, including multiscale and multi-period discriminators. The generator kernel size of HiFi-GAN was 7, and the upsampling ratio was (8, 8, 2, 2). The list of discriminators for the cycle scale was (2, 3, 5, 7, 11). The Conv filters of each periodic discriminator were eight. The pooling type of output downsampling in the discriminator was AveragePooling1D, the kernel size was (five, three), and the activation function was LeakyReLU. HiFi-GAN was trained independently of FastSpeech2. For each speaker, the generator with only soft loss was first trained for 100 k steps, and then the generator and discriminator were trained for 100 k steps. This gave us the corresponding vocoder for each of the three speakers.  
Note that the teacher Tacotron2 model with 80 k steps for each speaker was trained for extracting durations from attentional contrasts for subsequent FastSpeech2 model training. After that, a multi-speaker FastSpeech2 model was trained in 200 k steps to complete the final speech generation. The generator of HiFi-GAN was trained for 100 k steps, and the generator and discriminator were jointly trained for 100 k steps. The above models were trained on 2 Tesla V100 GPUs.