# MLSL-Spell: Chinese Spelling Check Based on Multi-Label Annotation
## 4 Experiments
### Ablation Studies  
Pruning of MLSL-SpellWe prune the MLSL-Spell, the pruned model is named and explained in this section. * **MLSL-Spell(Pinyin)** ModelWe remove the Pinyin vectors and only use character-based pre-trained context vectors in the spelling detection module of MLSL-Spell. * **MLSL-Spell(Top 1)** ModelIn the spelling correction system of MLSL-Spell, we adopt the candidate character with the maximum probability (generated by the MLM model) as the spelling correction result, instead of using XGBoost classifier to screen out candidate characters. * **MLSL-Spell(lg)** ModelWe use Logistic Regression [40] classifier instead of XGBoost classifier to screen out candidates. * **MLSL-Spell(knn)** ModelWe use K-Nearest Neighbor [41] classifier instead of XGBoost classifier to screen out candidates. * **MLSL-Spell(svm)** ModelWe use the Support Vector Machines [3] classifier instead of XGBoost classifier to screen out candidates. * **MLSL-Spell(lgb)** ModelWe use LightGBM [42] classifier instead of XGBoost classifier to screen out candidates. * **MLSL-Spell(-multi label)** ModelWe screen out candidate characters by directly using XGBoost classifier instead of considering the label types. * **MLSL-Spell(sighan)** ModelWe only use SIGHAN2013, SIGHAN2014 and SIGHAN 2015 training sets to train MLSL-Spell model. Effectiveness of Pinyin VectorsPinyin information is an essential feature in CSC task and Pinyin vectors contain similar Pinyin information. We compare the spelling detection performance of MLSL-Spell model and MLSL-Spell(-Pinyin) model. The latter solely adopts character-based pre-trained context vectors. The experimental results in Table 4 indicate that on SIGHAN 2013, 2014 and 2015 test sets, MLSL-Spell's spelling detection model outperforms MLSL-Spell(-Pinyin) model on all three indicators (Precision, Recall, and F1), proving the effectiveness of the Pinyin vectors. It is worth noticing that the spelling detection performance of MLSL-Spell model in Table 4 seems poorer than that of MLSL-Spell model in Table 3. This is because MLSL-Spell detection module might detect the correct character as the misspelled one, but MLSL-Spell correction module can identify the mis-detected character. In the experiment to verify the effectiveness of Pinyin vectors, we don't need to consider the case where the spelling correction module recognizes the mis-detected characters. Effectiveness of Spelling Correction ModuleWe compare the correction performance of MLSL-Spell model and MLSL-Spell (Top 1) model, the latter of which adopts candidate characters with the maximum probability (generated by MLM model) as the result of spelling correction. The experimental results shown in Table 5 present that three indicators (Precision, Recall and F1) of MLSL-Spell's spelling correction module are better than those of MLSL-Spell (Top 1) model on the test sets of SIGHAN 2013, 2014 and 2015. With these results, the effectiveness of MLSL-Spelling's spelling correction module is validated. Effectiveness of XGBoost ClassifierIn terms of using the classifier to screen out correct characters, we compare MLSL-Spell model, MLSL-Spell(lg) model, MLSL-Spell(knn) model, MLSL-Spell(svm) model and MLSL-Spell(lgb) model. The experimental results shown in Table 6 indicate that MLSL-Spell(knn) model and MLSL-Spell(svm) model achieve comparable performance in spelling correction, while MLSL-Spell(lg) model and MLSL-Spell model gain comparable performance. Moreover, the spelling correction module of MLSL-Spell has achieved the best performance on the test sets of SIGHAN 2013, SIGHAN 2014 and SIGHAN 2015, which validates the effectiveness of using XGBoost classifier to screen out correct characters.