## 4 Experimental Evaluation
### Experiment Analysis
However, it is important to note that despite this, the performance of the student model obtained through LightChatGLM after mixed compression demonstrates remarkable proximity to that of the teacher model within a 9.4% gap. LightChatGLM effectively facilitates the migration of bilingual knowledge from the teacher model to the student model, which proves to be particularly effective in task-independent knowledge distillation scenarios during the pretraining stage. This successful knowledge transfer highlights the efficacy of LightChatGLM in preserving the essential knowledge and capabilities of the teacher model while achieving significant compression, thereby demonstrating its potential for practical deployment in real-world applications. Table 3 presents a comprehensive comparison of the accuracy achieved in computerized English to Chinese translations using a combination of fine-tuning and knowledge distillation, with both target annotated language data and source language data. Upon examining the table, it becomes evident that the accuracy of Chinese and English translations for all models significantly improves after employing hybrid data fine-tuning. This enhancement can be attributed to the utilization of labeled target data in conjunction with further knowledge distillation, which collectively contributes to the model's enhanced performance in the domain of Chinese and English bilingual translation. Furthermore, it is noteworthy that LightChatGLM exhibits slightly lower performance, with 57.4% in English comprehension compared with the other models. This disparity arises from the relatively smaller English corpus available in the dataset utilized, as compared with the teacher model. However, by augmenting the training data with a more extensive Chinese corpus, LightChatGLM has the potential to achieve superior results with 33.3% in English-to-Chinese comprehension. This result underscores the importance of dataset composition and the need for mixed data to effectively train models for bilingual translation tasks. The dataset used in Tables 2 and 3 is Ai Challenger. According to Tables 2 and 3, in the mixed-data scenario, the LightChatGLM model achieves its highest Chinese translation accuracy, suggesting that the mixed dataset may enhance the model's generalization ability to some extent. The hybrid dataset incorporates characteristics of both the source and target languages, allowing the model to capture complex linguistic relationships, thereby improving translation quality. Specifically, the model benefits from mixed-data training by learning contextual and semantic relationships, particularly in computer-related terminology. However, in non-hybrid data scenarios, the model's translation accuracy is significantly lower. This may be due to the disproportionate presence of single-language data, which leads the model to focus excessively on the source language, thus neglecting the target language during optimization. \begin{table}
\begin{tabular}{c c c c} \hline
**Student/Teacher Model** & **English** & **Chinese** & **AVG** \\ \hline mBERT\_drop/mBERT & 58.4/60.4 & 30.4/37.6 & 44.4/49.0 \\ DistilmBERT/mBERT & **59.2**/60.4 & 33.3/37.6 & **46.3**/49.0 \\ ChatGLM-6B/ChatGLM & 57.6/60.2 & **32.8**/39.5 & 45.2/49.9 \\ LightChatGLM(Ours)/ChatGLM-6B & 57.2/57.6 & 30.1/32.8 & 43.7/45.2 \\ \hline \end{tabular}
\end{table} 
Table 2: Comparison of the experimental results of different compression methods under fine-tuning without target data (translation accuracy). \begin{table}
\begin{tabular}{c c c c} \hline
**Student/Teacher Model** & **English** & **Chinese** & **AVG** \\ \hline mBERT\_drop/mBERT & 58.7/61.2 & 32.4/41.1 & 45.6/51.2 \\ DistilmBERT/mBERT & **59.8**/61.2 & 32.8/41.1 & **46.3**/51.2 \\ ChatGLM-6B/ChatGLM & 58.2/60.5 & 33.1/40.5 & 45.7/50.5 \\ LightChatGLM(Ours)/ChatGLM-6B & 57.4/58.2 & **33.3**/33.1 & 45.4/45.7 \\ \hline \end{tabular}
\end{table} 
Table 3: Comparison of the experimental results of different compression methods under fine-tuning with mixed data (translation accuracy).