# Think Twice: A Post-Processing Approach for the Chinese Spelling Error Correction
## 4 Approach
### Implementation of Post-Processing
#### 4.2.1 Features Set
Therefore, the degree of pronunciation and form proximity is an essential factor to judge whether the candidate character is reasonable. We introduced two features, **sos** and **sop**, to measure the similarity between the candidate character and original character in terms of shape and sound. To get a clear numerical expression, we built a Chinese character disassembly and pronunciation statistics file, and then calculated these two feature values by the heuristic method described in FASPell [2], as shown in Figure 3.  
Figure 2: Across the candidate set, our approach obtains some features of \(\
ot{\times}\) and \(\
ot{\mp}\): **pro**, **rank**, **topK**, **topP**. Text Generation's Inspiration  
Text generation is another hot research topic in natural language processing. Both text generation and spell checking try to output correct and high-quality text. Researchers directly used characters with the largest probability value in the model's last layer as the generation results in the early stage of text generation. This generation method leads to low diversity of the generated text. Therefore, researchers introduced sampling methods to improve diversity. However, sampling methods may lead to another problem: characters with very low probability will inevitably be sampled, which is extremely harmful to the generated text's overall fluency. Top-k sampling [30] and top-p sampling [31] are two improved sampling algorithms. They truncate low-probability words with accumulated rank or probability to improve the generated text's quality. Inspired by text generation tasks, we introduced the two attributes of **topP** and **topK** in order to help the final output results to get rid of the character's frequency. Following the specific implementation in the text generation task, we redefined these two attributes' calculation method. Firstly, we constructed a probability set (the uppercase English letter **P** represents this set in the formula) by replacing all characters with their corresponding probabilities in the candidate set (1). The original character's value of **rank** plus one is the value of **topK** since their definitions are very close. Then, we added up all the probability values higher than the character's probability value (the lowercase English letter p represents it in the formula), including itself as the value of topP, as shown in Figure 3.  
\[topK=rank+1 \tag{2}\]  
\[topP=\sum p_{i},\quad p_{i}\subset P,\quad p_{i}\geq p \tag{3}\]  
Due to the characteristics of the Chinese text itself, we found that changing some correct characters to many others can still ensure that the entire sentence is grammatically right but the original sentence's meaning is changed. When this happens, the original character should not be replaced, even if there are other reasonable candidates. Therefore, it is significantly inaccurate to determine that the error character relies on the character with the highest probability. We put forward an extreme hypothesis that if the probabilities of all characters appearing in a particular position of the sentence are the same, it means that every character is reasonable in that position. In this case, this position has the greatest uncertainty [32], and the error's possibility is the smallest. So, we introduced the feature of **uncertainty** to indicate the probability of error. We calculated this value as follows, as indicated in Figure 4.  
\[uncertainty=-\sum p_{i}*\log p_{i},\quad p_{i}\subset P \tag{4}\]  
Figure 3: Our method calculates the phonetic similarity (**so**) and glyph similarity (**sos**) between \(\bigstar\) and \(\bigstar\) based on the edit distance. A qualified correction will not change a sentence's meaning, or it should make the semantic change of the sentence as small as possible. Therefore, we introduced the attribute **similarity** to represent the semantic similarity of sentences before and after spelling correction. [PERSON] et al.