# Think Twice: A Post-Processing Approach for the Chinese Spelling Error Correction
## 7 Ablation Studies  
In this part, we analyze the effect of several components on the classifier's performance, including selection of the different feature subsets and the setting of the **SVM**'s threshold. The ablation experiments were performed, using a test set, which contains 290 positive samples and the same number of negative samples. Then, we test the effects of post-processing operations on model performance under different error rates. We use the model's F1 value as the comparison index from the two subtasks: error detection and error correction.  
**Different Feature Subsets:** Figure 6 shows the specific performance of the trained classifier on the test set when selecting different feature subsets. The related concept of text generation has a positive effect. Because the classification results used the probability-based feature, the subsets (pro and rank) are the same as the results when using the text generation features (topP and topK). Simultaneously, among all the classifiers trained on different subsets, these two classifiers have a better specific ability for negative samples. The classifier trained by using the uncertainty and the sentences similarity can recognize positive samples better than other sub-features classifiers. However, its ability to recognize negative samples is the lowest. When we put all the features together, the resulting classifier achieves an ideal effect between positive samples and negative samples.  
**Different error rate:** At the end, we analyzed the effect of the post-processing operation under different error rates. In order to simulate real-life Chinese error texts, we used the official training set provided by the 2018 NLPCC competition to construct them. The NLPPCC [38] is a Chinese misdiagnosis competition in which data have a similar distribution with the SIGHAN dataset. The no-error sentences from the competition's data are entirely correct. We randomly selected 3000 right sentences from the training set as our extra data. Table 4 gives the details of this data. We used BERT and post-processed BERT to correct the extra data and illustrate the results. Then we selected a part of these sentences and combined it with the test data to construct some more texts with different error rates (50%, 40%, 30%, 20%, and 10%). At last, we applied our BERT-PoP models as well as the baseline models to correct these texts and record the F1 score of the sentence level correction. The results are shown in Figure 7.  
With the decrease in text error rate, higher model precision will result in an excellent F1 score. Figure 7 shows that when the error rate of the text is less than 20%, the BERT-PoP's F1 score is higher than that of BERT. The model's F1 score can increase by 10% or more when the error rate of the text is less than 10%. Therefore, the post-processing operation significantly promotes the model's performance when the text has a low error rate.  
**Different SVM's thresholds:** The classifier's results in the test set, according to different thresholds, are shown in Table 5. It can be seen that the higher the threshold, the higher the precision. At the same time, it reduces the recall and accuracy of the classifier. So we should set the threshold according to the actual situation. When the situation requires a high precision rate, the classifier's threshold needs to be a higher value. In contrast, the classifier threshold needs to be lowered.  
Figure 6: The different sets of features classifier on test set which contains 290 positive samples and 290 negative samples. The feature subsets used from left to right are **pro + rank**, **topP + topK**, **so + sp**, **uncertainty + similarity (unc+sim)**, and **all features (AII)**. The classification results of the classifier on the test set are marked with different colors.