# A Kazakh-Chinese Cross-Lingual Joint Modeling Method for Question Understanding
## 4 Experiments
### Experimental Results and Analysis  
#### 4.5.1 Comparative Experiments  
To comprehensively compare the cross-lingual transfer ability of the Bi-XTM model, this experiment uses three different training methods, including the following: (1) Zero-shot training: Only the source language is used during the training process and the model is not exposed to the target language. (2) Single-language training: The model is trained and evaluated separately on different language corpora without distinguishing between the source and target languages. (3) Mixed-language training: The source language is used during the training process, and the model is exposed to the target language. In the zero-shot training method, the main focus is on the model's cross-lingual transfer ability to Kazakh. Since the source languages are Chinese and English, the evaluation results for the Chinese and English corpora are the same as those for single-language training. The comparison results of the MTOD zero-shot training experiment are shown in Table 2.  
Table 2-4 show the comparative experimental results of the Bi-XTM model on the MTOD dataset. From the experimental results, it can be seen that since mBERT did not include Spanish and Thai training data during the pre-training phase, the model performed poorly on the target language data. The XLM-R model, compared to the mBERT model, used more parallel corpora and monolingual data during the pre-training phase, but this does not mean that it can be directly applied to all languages. The coverage of Thai data in the training dataset is relatively low, so the XLM-R model may not fully learn and capture the linguistic features of Thai, leading to poor performance on Thai data. The CINO model introduced more low-resource corpora for secondary pre-training, addressing the issue of insufficient data in low-resource languages and improving the model's adaptability to these languages, thus achieving a significant performance improvement on target language data. The Co-Transformer model, using CINO as the shared encoder, incorporated an explicit bidirectional interaction module, which not only shares lower-level representations but also more finely controls the interaction and information transfer between subtasks. The Co-Transformer model effectively improved intent accuracy on target language data; however, due to the error propagation problem in sentence-level intent vector representations during the transfer to slot vector representations, the model \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{3}{c}{**Spanish**} & \multicolumn{3}{c}{**Thai**} \\ \cline{2-7}  & **Acc** & \(F1\) & **Sent.Acc** & **Acc** & \(F1\) & **Sent.Acc** \\ \hline \multicolumn{7}{c}{Mixed-language training} \\ \hline mBERT & 69.7 & 51.2 & 47.4 & 61.3 & 18.8 & 11.4 \\ XLM-R & 77.2 & 53.8 & 59.1 & 67.7 & 21.9 & 20.0 \\ CINO & 90.7 & 71.5 & 68.5 & 72.8 & 27.4 & 22.5 \\ Co-Trans & 92.1 & 76.9 & 68.6 & 77.4 & 43.8 & 37.4 \\ Bi-XTM & 95.4 & 80.1 & 75.7 & 77.6 & 43.5 & 37.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: MTOD mixed-language training comparative experiment results. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{3}{c}{**Spanish**} & \multicolumn{3}{c}{**Thai**} \\ \cline{2-7}  & **Acc** & \(F1\) & **Sent.Acc** & **Acc** & \(F1\) & **Sent.Acc** \\ \hline \multicolumn{7}{c}{Single-language training} \\ \hline mBERT & 71.3 & 45.2 & 41.7 & 58.5 & 16.3 & 10.6 \\ XLM-R & 78.5 & 49.5 & 46.3 & 66.2 & 23.6 & 19.8 \\ CINO & 91.3 & 71.3 & 66.5 & 65.8 & 44.1 & 34.7 \\ Co-Trans & 95.6 & 71.8 & 67.1 & 66.1 & 42.7 & 34.2 \\ Bi-XTM & 94.1 & 77.9 & 69.5 & 66.4 & 44.6 & 36.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: MTOD single-language training comparative experiment results. negatively impacted the evaluation of slot \(F1\) scores, with a slight decline in performance on Thai data.