# Using Transfer Learning to Realize Low Resource Dungan Language Speech Synthesis
## 2 Models and Methods
### Text Analyzer of Dungan Language
#### 2.1.4 Prosodic Boundary Prediction  
Our approach utilizes initials and finals, along with their prosodic labels, as the input unit sequence for the acoustic model. Thus, extracting the prosodic structure from Dungan sentences is crucial for synthesizing high-quality speech. Like Mandarin, Dungan's prosodic hierarchy can be segmented into prosodic words, prosodic phrases, intonation phrases, and sentence pauses. The boundary of intonation phrases can be easily identified using Dungan punctuation marks. In this study, we employed a BiLSTM with a conditional random field (BiLSTM_CRF)-based method, as illustrated in Figure 4, to predict the boundaries of prosodic words and phrases [42].  
We employed four distinct prosodic word-position labeling sets (#1, #2, #3, #4) to categorize Dungan words into prosodic phrases. Specifically, #1 was utilized to denote the prosodic words, #2 designated the prosodic phrases, #3 marked the  
\begin{table}
\begin{tabular}{c c} \hline \hline \multirow{2}{*}{intials} & /b/. /p/. /m/. /t/. /v/. /z/. /c/. /s/. /d/. /t/. /n/. /l/ /zh/. /ch/. /sh/. /r/. /j/. /q/. /x/. /g/. /k/. /ng/. /h/. /\phi/ /ii/. /ii/. /u/. /y/. /z/. /ia/. /ua/. /e/. /ue/. /ye/. /iE/ /ap/. /ai/, /uai/. /ei/. /ui/. /ao/. /iao/. /ou/. /iou/. /an/. /ian/ /uan/. /yan/. /aN/. /iaN/. /uN/. /uN/. /iN/. /yN/ \\ \hline \hline \end{tabular}
\end{table}
Table 1: The initials and finals of Dungan Language.  
Figure 4: The framework of BLSTM_CRF-based Dungan Prosodic Boundary Prediction. The input is a Dungan sentence with prosodic information.  
word, and #4 indicated a pause within a sentence. The labeling process incorporated phrase and prosodic information derived from Dungan text, which was tagged manually. During this phase, linguists sporadically reviewed and amended selected sentences. We attained a high level of consistency with linguistic experts through iterative corrections.  
Despite the BiLSTM's capability to learn context-dependent information, its independent classification decisions are constrained by strong dependencies across the output label. To address this, we employ a CRF layer that considers neighboring tags, as illustrated in Figure 4. For a normalized input sentence \(\mathbf{X}=\{x_{1},x_{2},\cdots,x_{n}\}\) containing \(n\) words and a tag sequence of sentence \(y=(y_{1},y_{2},\ldots,y_{n})\), each word is represented as a \(d\)-dimensional vector by word2 vec. We define its prediction score \(s(\mathbf{X},y)\) to be as follows:  
\[s(\mathbf{X},y)=\sum_{i=1}^{n}P_{i,y_{i}}+\sum_{i=0}^{n}A_{y_{i},y_{i}+1} \tag{1}\]  
where \(\mathbf{P}\) is the matrix of scores output by the BLSTM network. \(P_{i,y_{i}}\) corresponds to the score of the \(y_{i}\) tag of the \(i\)th word in a sentence. \(\mathbf{A}\) is the transition scores matrix of the CRF layer, and \(A_{y_{i},y_{i+1}}\) corresponds to the score from \(\text{tag}\,y_{i}\) to \(\text{tag}\,y_{i+1}\).  
In the training, we maximize the following log-likelihood functions:  
\[\log(p(y\mid\mathbf{X}))=s(\mathbf{X},y)-\log\left(\sum_{\tilde{y}\in\mathbf{ Y_{X}}}e^{s(\mathbf{X},\tilde{y})}\right) \tag{2}\]  
where \(\mathbf{Y_{X}}\) represents all possible tag sequences for an input text \(\mathbf{X}\).  
In the decoding, the optimal sequence \(y^{*}\) is given as follows:  
\[y^{*}=\underset{\tilde{y}\in\mathbf{Y_{X}}}{\text{argmax}}(\mathbf{X},\tilde{y}) \tag{3}\]