# Research on a Mongolian Text to Speech Model Based on Ghost and ILPCnet
## 3 Mongolian Speech Synthesis Model Based on Ghost-ILPCnet
### Mongolian Phoneme Pre-Training Model Based on Bang  
For the cases where the generated audio had reduced naturalness and insufficient authenticity, we further analyzed the reasons for this and found that there was a phenomenon in Mongolian where the glyphs had the same shape, different pronunciations, and different meanings, that is, the same letters in different contexts had different meanings and different pronunciations. For example, the Mongolian text \"\(\Leftarrow\)\" is placed in the context of describing tenses and means \"now\". The transliterated Latin form is \"vdv\" and the corresponding phoneme sequence is \"[? d :]\". If you put it in the context of describing the sky scene, it means \"star\", which is translated into the Latin form as \"vd\", and the corresponding phoneme sequence is [? d]. Therefore, in the front-end processing part, this paper proposes to build a Mongolian phoneme pre-training model based on Bang to further strengthen the training of Mongolian phonemes, which can determine the status of Mongolian in different contexts. Bang consists of a multi-layer stacked transformer encoder using a self-attention mechanism and a multi-layer stacked transformer decoder using a cross-stream visible multi-stream self-attention mechanism [16]. The Mongolian phoneme pre-training model is based on the Bang structure and uses the input sequence \(X=\left\{x_{1},x_{2},\ldots,x_{|X|}\right\}\) to generate a prediction target sequence: \(Y=\left\{y_{1},y_{2},\ldots,y_{|Y|}\right\}\). Its purpose is to achieve an accurate alignment of Mongolian phonemes and acoustic features [17] to reduce the rates of word errors and word omissions in synthesized Mongolian speech [18]. In the Bang pre-training model, for each phoneme, \(y_{\text{t}}\), in the original Mongolian target sequence, \(Y\), the predicted Mongolian target, \(\hat{Y}\), considers masking, [MASK], its \(i\) previous phonemes to optimize the entire output sequence. Masked language modeling (MLM) and next sentence prediction (NSP) are the two important pre-training tasks in Bang [19]. The workflow is shown in Figure 4. M-S (mainstream) indicates that it contains real Mongolian phonemes, while P-S (predicting stream) predicts Mongolian phonemes, which contains [M]. In order to obtain the previous real-word and [MASK] phoneme information, [M] in the P-S stream uses the information in M-S and its previous P-S for the attention calculation. Taking the prediction of \(y_{\text{4}}\) as an example, the specific calculation processes are as follows. Figure 3: Structure of Mongolian speech synthesis model based on Ghost-ILPCnet. First-line predicted phoneme \(y_{4}\): M-S and first P-S are shown, using [MASK] to analyze \(y_{1},y_{2},y_{3}\) in M-S, and \(y_{4}\) is predicted with conditional probability \(P(y_{4}|y_{1},y_{2},y_{3})\). All the phoneme information in the first P-S is autoregressively predicted with the complete previous information. Second-line predicted phoneme \(y_{4}\): the [M] of \(y_{4}\) in the second P-S focuses on the true \(y_{1},y_{2}\) values from the M-S and [M] of \(y_{3}\) from the first P-S. \(y_{3}\) in the first prediction stream and \(y_{4}\) in the second prediction stream are generated with the conditional probability \(P(y_{3},y_{4}|y_{1},y_{2})\). As the attention flow increases, the previous information is masked, and the generation method also moves from autoregressive to non-autoregressive. The last row shows that \(y_{4}\) in the fourth P-S ultimately performs predictions in a non-autoregressive manner. At this time, the predicted [M] of \(y_{4}\) in the fourth P-S is calculated from the [M] of \(y_{1}\) in the first P-S, the [M] of \(y_{2}\) in the second P-S, and the [M] of \(y_{3}\) in the third P-S. Therefore, this can be used to quickly predict the next Mongolian word phoneme without considering the surrounding phoneme information. Assuming that the length of the target sequence \(|Y|=n\), [PERSON] sets n prediction streams.