## 4 Experiments
### Ablation Study  
In this subsection, we explore the impact of the hyperparameters \(k_{1},k_{2},k_{3}\) in the loss function on the model performance as well as the contribution of the pinyin prediction task, the self-distillation module, and the adaptive weighting method to the SPMSpell model. For this ablation experiment, we evaluate the model using the SIGHAN 2015 test set. According to what was mentioned earlier, in the pinyin prediction task, the gap between the roles of tones and rhymes is small and theoretically larger than the roles provided by tones, so for exploring the effects of hyperparameters \(k_{1},k_{2},k_{3}\) can be transformed into exploring the effects of hyperparameter \(k\). The transformation is as follows:  
\[k_{1},k_{2},k_{3}\rightarrow\frac{(1-k)}{2},\frac{(1-k)}{2},k \tag{13}\]  
The effect of hyperparameter k taking value on F1 value of model performance is shown in Figure 5. Based on the experimental results, we can find that the F1 scores are basically increasing with increasing values of \(k\). However, after the value exceeds 0.3, the F1 scores begin to decrease. Therefore, setting \(k\) to 0.2 achieves the overall best corrected F1 score. We further explored the reasons for this and found that in Mandarin, pinyin is the official system used for phonetic transcription. It uses the three components of consonants, rhymes, and tones to express the pronunciation and spelling of Chinese characters. Since the similarity of pronunciation of Chinese characters is mainly determined by their consonants or rhymes rather than their tones, its proportion of tones should be relatively low, and its proportion of consonants and rhymes high, i.e., the value of \(k\) should be taken to be relatively low. \begin{table}
\begin{tabular}{l l c c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multirow{2}{*}{**Model**} & \multicolumn{3}{c}{**Detection-Level**} & \multicolumn{3}{c}{**Correction-Level**} & \multicolumn{3}{c}{**Char-Level**} \\  & & **D-P** & **D-R** & **D-F** & **C-P** & **C-R** & **C-F** & **D-F** & **C-F** \\ \hline \multirow{8}{*}{SIGHAN15} & GAD & 75.6 & 80.4 & 77.9 & 73.2 & 77.8 & 75.4 & 88.2 & 90.1 \\  & DCN & 77.1 & 80.9 & 79.0 & 74.5 & 78.2 & 76.3 & 85.0 & 84.9 \\  & REALISE & 77.3 & 81.3 & 79.3 & 75.9 & 79.9 & 77.8 & 87.4 & 86.2 \\  & uChecker & 75.4 & 72.0 & 73.7 & 70.6 & 67.3 & 68.9 & - & - \\  & MDCSpell & 80.8 & 80.6 & 80.7 & 78.4 & 78.2 & 78.3 & - & - \\  & ECOPE & 78.2 & 82.3 & 80.2 & 76.6 & 80.4 & 78.4 & - & - \\  & UMRSpell & 77.2 & 72.2 & 75.0 & 69.3 & 64.8 & 67.0 & 83.0 & 91.5 \\ \cline{2-10}  & **SPMSpell (our)** & **81.7** & **85.6** & **83.6** & **79.4** & **83.4** & **81.3** & **88.3** & **92.8** \\ \hline \hline \end{tabular} Note: The test sets are SIGHAN13, SIGHAN14, and SIGHAN15. In the table results, Detection-level and Correction-level represent sentence-level metrics, Char-level represents character-level metrics, D-P means detection module precision, D-R means detection module recall, D-F means detection module F1 value, C-P, C-R, and C-F are correction module corresponding metrics, and black bold in the table is the optimal result.  
\end{table} 
Table 1: _Cont._  
Figure 5: Effect of hyperparameter k on F1 value. To further validate the effect of analyzing several constituent modules, this paper also performs ablation experiments on the SIGHAN 2015 test set in the following settings to explore the contribution of each module: (a) w/o PP refers to the removal of the pinyin prediction task from the SPMSpell model; (b) w/o SD refers to the removal of the self-distillation module from the SPMSpell model; and (c) w/o AW refers to the removal of the adaptive weighting method from the SPMSpell model. The experimental results, as presented in Table 2, showcase precision (P), recall (R), and F1 scores for sentence-level detection/correction, along with the F1 scores for character-level detection/correction.