# MLSL-Spell: Chinese Spelling Check Based on Multi-Label Annotation
## 4 Experiments
### Ablation Studies
\begin{table}
\begin{tabular}{c c c c} \hline
**Methods** & \multicolumn{3}{c}{**Chinese Spelling Detection**} \\ \hline SIGHAN 2013 & Precision & Recall & F1 \\ \hline MLSL-Spell (-Pinyin) & 82.1 & 87.2 & 84.6 \\ MLSL-Spell & **82.4** & **87.7** & **85.0** \\ \hline SIGHAN 2014 & Precision & Recall & F1 \\ \hline MLSL-Spell (-Pinyin) & 60.4 & 76.1 & 67.4 \\ MLSL-Spell & **63.8** & **75.1** & **69.0** \\ \hline SIGHAN 2015 & Precision & Recall & F1 \\ \hline MLSL-Spell (-Pinyin) & 69.1 & 81.8 & 74.9 \\ MLSL-Spell & **70.9** & **82.6** & **76.3** \\ \hline \end{tabular}
\end{table} 
Table 4: Comparison of spelling detection performance between MLSL-Spell model and MLSL-Spell (-Pinyin) model. The best results for each test set are in bold. \begin{table}
\begin{tabular}{c c c c} \hline
**Methods** & \multicolumn{3}{c}{**Chinses Spelling Correction**} \\ \hline SIGHAN 2013 & Precision & Recall & F1 \\ \hline MLSL-Spell (Top 1) & 63.6 & 63.9 & 63.8 \\ MLSL-Spell & **79.7** & **78.3** & **79.0** \\ \hline SIGHAN 2014 & Precision & Recall & F1 \\ \hline MLSL-Spell (Top 1) & 37.3 & 42.0 & 39.5 \\ MLSL-Spell & **66.4** & **61.8** & **64.0** \\ \hline SIGHAN 2015 & Precision & Recall & F1 \\ \hline MLSL-Spell (Top 1) & 40.7 & 45.8 & 43.1 \\ MLSL-Spell & **74.2** & **69.3** & **71.7** \\ \hline \end{tabular}
\end{table} 
Table 5: Comparison of spelling correction performance between MLSL-Spell model and MLSL-Spell (Top 1) model. The best results for each test set are in bold. Effectiveness of Multi-label Sequence AnnotationThe misspelled characters in MLSL-Spell's spelling detection module are labeled according to error kinds using a multi-label sequence labeling method. We compare the spelling correction performance of MLSL-Spell model and MLSL-Spell(-multi label) model. The experimental results in Table 7 indicate that MLSL-Spell attains the relatively best performance in terms of three indicators (Precision, Recall, F1). To be specific, on the SIGHAN 2013 test set, the spelling correction F1 score of MLSL-Spell model is 79.0%, which is 11.1% higher than that of MLSL-Spell(-multi label) model; on the SIGHAN 2014 test set, the spelling correction F1 score of MLSL-Spell model is 64.0%, which is 2.9% higher than that of MLSL-Spell(-multi label) model; on the SIGHAN 2015 test set, the spelling correction F1 score of MLSL-Spell model is 71.7%, which is 3.4% higher than that of MLSL-Spell(-multi label) model. These results validate the effectiveness of multi-label sequence annotation. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Methods** & \multicolumn{3}{c}{**Chinese Spelling Correction**} \\ \hline SIGHAN 2013 & Precision & Recall & F1 \\ \hline MLSL-Spell(lg) & **79.7** & 77.3 & 78.5 \\ MLSL-Spell(knn) & 69.3 & 65.9 & 67.6 \\ MLSL-Spell(svm) & 65.1 & 62.7 & 63.9 \\ MLSL-Spell(gb) & 77.8 & 75.4 & 76.6 \\ MLSL-Spell & **79.7** & **78.3** & **79.0** \\ \hline SIGHAN 2014 & Precision & Recall & F1 \\ \hline MLSL-Spell(lg) & 60.5 & 61.0 & 60.8 \\ MLSL-Spell(knn) & 52.6 & 48.8 & 50.6 \\ MLSL-Spell(svm) & 59.0 & 54.8 & 56.8 \\ MLSL-Spell(gb) & 64.2 & 59.7 & 62.0 \\ MLSL-Spell & **66.4** & **61.8** & **64.0** \\ \hline SIGHAN 2015 & Precision & Recall & F1 \\ \hline MLSL-Spell(lg) & 68.3 & 68.0 & 68.1 \\ MLSL-Spell(knn) & 57.0 & 53.5 & 55.2 \\ MLSL-Spell(svm) & 68.9 & 64.7 & 66.8 \\ MLSL-Spell(gb) & 72.3 & 68.0 & 70.1 \\ MLSL-Spell & **74.2** & **69.3** & **71.7** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison of spelling correction performance between MLSL-Spell model and MLSL-Spell(lg) model, MLSL-Spell(knn) model, MLSL-Spell(svm) model, and MLSL-Spell(lgb) model. The best results for each test set are in bold.