# MLSL-Spell: Chinese Spelling Check Based on Multi-Label Annotation
## 4 Experiments  
### Dataset  
Train dataThe train data used by MLSL-Spell model for spelling detection are from the misspelling dataset constructed by [PERSON] et al. [4] and the training sets of SIGHAN 2013, SIGHAN 2014 and SIGHAN 2015 provided in the SIGHAN competition. MLSL-Spell model uses the SIGHAN training sets as the training data for the XGBoost classifier. SIGHAN datasets are written in Traditional Chinese, so we use OpenCC ([[https://github.com/BYVoid/OpenCC](https://github.com/BYVoid/OpenCC)]([https://github.com/BYVoid/OpenCC](https://github.com/BYVoid/OpenCC)) (accessed on 1 March 2023)) to convert them into Simplified Chinese. The statistics of train data are shown in Table 2.  
Test dataMLSL-Spell model uses the test sets of SIGHAN 2013, SIGHAN 2014 and SIGHAN 2015 as the test data for spelling detection and correction. The statistics of test data are shown in Table 2.  
CorpusMLSL-Spell model uses People's Daily Segmented Corpus (2014 edition) to build a 4-gram language model based on word granularity. We adopt the Modified Kneser-Ney [38] smoothing algorithm in the language model.  
Confusion SetMLSL-Spell model uses the confusion set provided by [PERSON] et al. [18].  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Training Data** & **\#Sent** & **Avg.Length** & **\#Errors** \\ \hline ([PERSON] et al., 2018 [4]) & 271,329 & 44.4 & 382,704 \\ SIGHAN 2013 & 350 & 49.2 & 350 \\ SIGHAN 2014 & 6526 & 49.7 & 10,087 \\ SIGHAN 2015 & 3174 & 30.0 & 4237 \\ Total & 281,379 & 44.4 & 397,378 \\ \hline
**Test Data** & **\#Sent** & **Avg.Length** & **\#Errors** \\ \hline SIGHAN 2013 & 1000 & 74.1 & 1227 \\ SIGHAN 2014 & 1062 & 50.1 & 782 \\ SIGHAN 2015 & 1100 & 30.5 & 715 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Statistics of CSC datasets. #Sent represents the total number of sentences in the corresponding dataset. #Errors represents the total number of misspelled characters in the corresponding dataset.  
Figure 4: Architecture of MLSL-Spell spelling correction module.  
Evaluation MetricsWe adopt the evaluation criteria proposed by [PERSON] et al. [2], and use the evaluation metrics of precision, recall rate, and F1 score to evaluate the performance of the spelling detection and correction of the model. The calculation formulas are as follows:  
\[Precision=\frac{TP}{TP+FP} \tag{11}\]  
\[Recall=\frac{TP}{TP+FN} \tag{12}\]  
\[F_{1}=\frac{2*Precision*Recall}{Precision+Recall} \tag{13}\]  
where TP means that the model correctly corrects errors in the sentence, FP means that the model incorrectly changes the correct part of the sentence, and FN represents that the model does not correct errors in the sentence.  
Benchmark ModelsWe compare the following 11 models in terms of spelling correction or spelling detection. The first 3 are benchmark models, followed by 7 ablation experiments of MLSL-Spell model, and the last model is used to explore the impact of the data set on the model.  
* **LMC Model**[PERSON] et al. [8] used the confusion set provided by the SIGHAN competition to replace the suspicious characters, adopted the traditional N-gram language model to score the perplexity of the replaced sentences, and selected the sentence with the highest score as the CSC result. We call this method LMC.
* **SL Model**[PERSON] et al. [4] used sequence labeling to detect positions of misspelled characters. The correct position is labeled as 1, and the wrong position is labeled as 0. Sequence labeling is abbreviated as SL.
* **PN Model**[PERSON] et al. [7] proposed an end-to-end pointer network model, through which the correct character is copied from the input character list or generated from the confusion set instead of from the entire vocabulary. Pointer network is abbreviated as PN.
* **[PERSON]**[PERSON] et al. [5] proposes a framework from pre-training to fine-tuning.The Model is pre-trained on a large corpus and then fine-tuned when it is used for specific tasks.
* **FASpell**[PERSON] et al. [16] utilizes a denoising autoencoder (DAE) to generate candidate characters.