# Think Twice: A Post-Processing Approach for the Chinese Spelling Error Correction
## 2 Related Work  
In the early days, researchers adopted the idea of combining rules and language models to solve error correction in Chinese texts. They relied on confusion sets to modify common errors in sentences. Each element in the confusion set is a dictionary. The key is an error-prone Chinese character or word, and the corresponding values contain a series of characters or phrases similar to the key in sound or form. In error processing, the sentence is segmented into many characters and words. Researchers chose one of the values to replace the error if the confusion set included the element corresponding to these characters or words. After that, they used the language model to score the replaced sentences and then selected the sentence with the highest score as the error correction result [6; 7]. However, typos affected the segmentation results in the sentence and reduced the effect of error correction. [PERSON] and [PERSON] proposed the noisy channel model [8] to address this limitation. [PERSON] and [PERSON] utilized this phenomenon that a misspelled word is often split into two or more characters, and used the Single Source Shortest Path (SSSP) algorithm to correct Chinese spelling errors [9]. In [PERSON] and [PERSON]'s work, they used Minimized-Path Segmentation [10] to solve this problem. With the development of deep learning research and the continuous improvement of computing power, researchers began to use neural networks to solve Chinese error correction problems. [PERSON] and [PERSON] used two stages to solve the CSC problem in their research. First, they used the confusion set and language model to determine the suspected error position in the sentence and then used the Sequence-to-Sequence [11] model to correct the mistakes [12]. At the same time, [PERSON] and [PERSON] combined the Long Short Term Memory network (LSTM [13]) network with the Conditional Random Fields(CRF [14]) to accomplish Chinese text error detection through sequence labeling [15]. After that, they incorporated the Sequence-to-Sequence model and attention to realize the error correction of Chinese text based on Bi-LSTM [16] model [17]. [PERSON] and [PERSON] introduced the pronunciation and structure of words as model inputs and then combined Bi-LSTM and CRF to find errors in sentences. After that, they used the Masked Language model to correct suspected errors [18]. On the basis of the original LSTM model, [PERSON] and [PERSON] added the pronunciation and confusion set information to the model by introducing the fusion cell form to help realize text error correction [19]. [PERSON] and [PERSON] rethought the attention mechanism [20], combined the confusion set and regard the Chinese error correction task as a prediction problem. They introduced more global information as a new attention mechanism distribution and used LSTM to predict the target character [21]. More recently, pretrained language models have played a significant role in achieving impressive gains in a wide varity of Natural language processing tasks. These modelsare trained on a large amount of unsupervised corpora to learn sufficient corresponding semantic knowledge. Fine-tuning these models on the professional corpus can obtain good performance while alleviating the problem of insufficient labeled corpora. Bidirectional Encoder Representations from Transformers (BERT) [1], proposed by the Google team, achieved leading results in various natural language processing tasks once it came out. Therefore, researchers began to apply BERT and its variant models to Chinese Spelling Error Correction tasks. The ByteDance team eliminated the soft-mask mechanism. They used Bi-GRU to softly cover the error and corrected whole sentences by the Masked Language model [3]. The Alibaba team introduced the Chinese character information's glyph and phonetic information into the BERT model's uppermost layer and expanded the training data. This algorithm obtained new, state-of-the-art results for error correction tasks [4].