# Comparative Study for Multi-Speaker Mongolian TTS with a New Corpus
## 4 Speech Synthesis Experiments
### Speaker Similarity Evaluation  
We further performed listening experiments to evaluate the speaker similarity performance of FastSpeech+HiFi-GAN and VITS baseline systems. The results of the Speaker Similarity Mean Opinion Score (SS-MOS) are reported in Table 3.  
We synthesized 20 audios for each speaker with the FastSpeech2+HiFi-GAN and VITS baseline systems, respectively. Ten native Mongolian-speaking volunteers were also invited to participate in the scoring. Each volunteer was asked to assess whether the speaker was the same person in the synthesized audio and the ground truth audio.  
In the FastSpeech2+HiFi-GAN baseline system, the SS-MOS scores for F1, F2, and F3 were 4.58, 4.04, and 4.12, respectively, with large differences across speakers. In contrast, the scores of the VITS model were more consistently concentrated, with F1, F2, and F3 achieving scores of 4.56, 4.67, and 4.54, respectively, which is encouraging. The results show that the audio synthesized by the FastSpeech2+HiFi-GAN system and VITS system performs well in terms of speaker similarity. Note that FastSpeech2 model achieved the highest SS-MOS score for speaker F1. One possible reason is that the F1 dataset contains a large number of sentences with fewer words instead of long sentences. Another reason is that the HiFi-GAN vocoder of F2 and F3 did not work well during the FastSpeech2+ HiFi-GAN synthesis experiment. Due to time and funding constraints, we first recruited volunteers to perform SS-MOS scoring on F1. Volunteers were re-recruited to update the data for F2 and F3 after subsequent training of the F2 and F3 vocoders was completed. The scoring of F1 and the division of F2 and F3 into two parts led to a greater subjectivity of the volunteers so that F1 scored higher, while F2 and F3 scored lower, and both scored similarly. We found that VITS can effectively synthesize high-quality speech in the face of longer utterance due to its generative and fully end-to-end structure. In conclusion, this experiment shows that the MnITS2 dataset can be used for speech synthesis work in multi-speaker scenarios.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**System****Speaker ID** & **F1** & **F2** & **F3** \\ \hline FastSpeech2+Griffin-Lim & 3.56 \(\pm\) 0.18 & 3.59 \(\pm\) 0.04 & 3.86 \(\pm\) 0.12 \\
**FastSpeech2+HiFi-GAN** & **4.02 \(\pm\) 0.18** & **4.15 \(\pm\) 0.06** & **4.29 \(\pm\) 0.11** \\
**VITS** & **4.60 \(\pm\) 0.07** & **4.55 \(\pm\) 0.08** & **4.62 \(\pm\) 0.09** \\ \hline Ground Truth & **4.73 \(\pm\) 0.08** & **4.70 \(\pm\) 0.14** & **4.68 \(\pm\) 0.09** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Naturalness mean opinion score (N-MOS) results for all systems with 95% confidence intervals.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**System****Speaker ID** & **F1** & **F2** & **F3** \\ \hline FastSpeech2+HiFi-GAN & 4.58 \(\pm\) 0.21 & 4.04 \(\pm\) 0.16 & 4.12 \(\pm\) 0.10 \\ VITS & 4.56 \(\pm\) 0.09 & 4.67 \(\pm\) 0.08 & 4.54 \(\pm\) 0.07 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Speaker Similarity Mean Opinion Score (SS-MOS) results for the FastSpeech2+HiFi-GAN system with 95% confidence intervals.