# Using Transfer Learning to Realize Low Resource Dungan Language Speech Synthesis
## 3 Results
### Evaluation on Transfer Learning-Based Dungan Acoustic Models
#### 3.2.2 Experimental Setup  
Three kinds of TTS frameworks, including Tacotron+Griffin-Lim, Tacotron2+WaveNet, and Tacotron2+WaveRNN, were compared in the experiments. Some hyperparameters of the frameworks are provided in Table 4.  
All three frameworks comprise a front-end text analyzer module, an acoustic model training module, and a vocoder. The text analyzer module transforms Dungan or Chinese sentences into a Pinyin-represented unit sequence, including initials, finals, and their tones and prosodic boundary labels. In the acoustic model training module, we derive the log magnitude spectrogram from the speech signal using Hann windowing with an 80 ms frame length, 12.5 ms frameshift, and a 2048-point Fourier transform. For the Tacotron+Griffin-Lim framework, acoustic models are trained using an output layer reduction factor of \(r=3\) and the Adam optimizer with a decaying learning rate. The learning rate commences at 0.001 and is subsequently reduced to 0.0005, 0.0003, and 0.0001 after 5, 20, and 50 epochs, respectively. A straightforward loss function is employed for the seq2 seq decoder (Mel spectrogram) and the postprocessing network (linear spectrogram). The training batch size is set to 32, with all sequences padding to a maximum length by reconstructing the zero-padded frames. The Griffin-Lim algorithm is utilized as the vocoder for Mel spectrum-to-speech conversion. For the Tacotron2+WaveNet-based framework, we train the acoustic models using the standard maximum-likelihood training procedure, which involves feeding the correct output instead of the predicted output on the decoder side. This was completed with a batch size of 32. The Adam optimizer was utilized with parameters set as follows: \(\beta=0.9\), \(\beta=0.999\), \(\epsilon=10^{-6}\). The learning rate was initialized at \(10^{-3}\) and then exponentially decayed to \(10^{-5}\) after \(50,000\). Additionally, we applied \(L2\) regularization with a weight of \(10^{-6}\). For the Mel spectrum-to-speech conversion, the WaveNet was employed as the vocoder. In our Tacotron2+WaveRNN-based transfer learning framework, we initially employ a large-scale Mandarin corpus to pre-train a Mandarin acoustic model for subsequent model transfer. This pre-trained model is then used to train the Dungan acoustic model via transfer learning from the Mandarin-Dungan corpus. For vocoding, we utilize the WaveRNN for Mel spectrum-to-speech conversion. Given that parameter settings significantly impact model accuracy and robustness, we optimized these parameters through iterative training and updates. Each TTS framework implements a monolingual speech synthesis for Mandarin or Dungan and a bilingual one based on transfer learning. We trained several models across three \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Tacotron** & **Tacotron2** & **Forward-Attention Tacotron2** \\ \hline \hline \multicolumn{2}{c}{Vocoder} & Griffin-Lim & WaveNet & WaveRNN \\ \hline \multirow{4}{*}{Encoder} & Embedding & Phoneme (256) & Phoneme (512) & Phoneme (512) \\ \cline{2-5}  & Pre-net & FFN (256, 128) & - & FFN Phoneme (512, 256) \\ \cline{2-5}  & Encoder core & CBHG (256) & \begin{tabular}{c} CNN (512) \\ Bi-LSTM (512) \\ \end{tabular} & \
\begin{tabular}{c} CNN (256) \\ Bi-LSTM (256, 512) \\ \end{tabular} \\ \hline \multirow{4}{*}{Decoder} & Post-net & CBHG (256) & CNN (512) & CNN (512) \\ \cline{2-5}  & Decoder RNN & GRU (256, 256) & - & LSTM (512, 256) \\ \cline{2-5}  & Attention & Additive (256) &
\begin{tabular}{c} Location-sensitive \\ (128) \\ \end{tabular} & Forward (256) \\ \cline{2-5}  & Attention RNN & GRU (256) & LSTM (1024, 1024) & LSTM (256) \\ \cline{2-5}  & Pre-net & FFN (256, 128) & FFN (256, 256) & FFN (256, 128) \\ \hline \multicolumn{2}{c}{Parameter} & \(7.6\times 10^{6}\) & \(28.9\times 10^{6}\) & \(23.7\times 10^{6}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Model hyperparameters of Tacotron and Tacotron2. TTS frameworks to assess the synthesized speech's quality and clarity.