Chunk-based C hinese Spelling Check with Global Optimization (2020)

Abstract: Chinese spelling check is a challenging task due to the characteristics of the Chinese language, such as the large character set, no word boundary, and short word length. On the one hand, most of the previous works only consider corrections with similar character pronunciation or shape, failing to correct visually and phonologically irrelevant typos. On the other hand, pipeline-style architectures are widely adopted to deal with different types of spelling errors in individual modules, which is difficult to optimize. In order to handle these issues, in this work, 1) we extend the traditional confusion sets with semantical candidates to cover different types of errors; 2) we propose a chunk-based framework to correct single-character and multi-character word errors uniformly; and 3) we adopt a global optimization strategy to enable a sentence-level correction selection. The experimental results show that the proposed approach achieves a new state-of-the-art performance on three benchmark datasets, as well as an optical character recognition dataset.

## introduction spelling check is a task to automatically detect and correct spelling errors in human writings. spelling check is well-studied for languages such as english, and many resources and tools have been developed. however, the characteristics of the chinese language make the chinese spelling check (csc) foot_0 quite different from the english one in three aspects: for the first challenge, previous research demonstrates that most of the chinese spelling errors come from similar pronunciations, shapes, or meanings @xcite . previous csc models usually employ the characters with similar pronunciation or shape as the confusion set to reduce the search space, but the visually and phonologically irrelevant typos cannot be handled. recent work aims at replacing the pronunciation and shape confusion sets with a dynamically generated confusion set by masked language models, which retrieve the semantically related candidates according to the contextual information @xcite . however, due to the lack of knowledge about human errors, masked language models correct the spelling errors ignoring the pronunciation or shape similarity. therefore, combining the two comes as a natural solution. for the second challenge, early works rely on the segmentation results from a chinese word segmentation system @xcite . however, as the segmentation system is trained on the clean corpus, the spelling errors often lead to incorrect segmentation results. the accumulated errors make the spell checking even more difficult. thus, characterbased models are proposed to perform the correction at the character-level directly, which are more robust to segmentation errors @xcite @xcite . however, the character-based model cannot effectively utilize the word-level semantic information, and the correction is also more difficult to interpret. in order to explore and utilize the word-level information, the word-based methods are designed to do word segmentation and spelling error corrections jointly. previous works show that the wordbased correction models often perform better than their character-based counterparts @xcite @xcite . since word-based correction models usually apply a pipeline of submodules and handle special cases (e.g., single-character words) individually, the complex architecture makes it difficult to perform global optimization. for the third challenge, previous works mainly rely on the local context features such as point-wise mutual information (pmi), part-of-speech (pos) n-gram, and perplexity from an n-gram language model @xcite @xcite . as these statistical features are limited within a fixed-size window, it is difficult to capture the deep contextual information. in the paper, we propose a unified framework combining features and benefits from previous works. we employ confusion sets from similar pronunciations, shapes, and semantics to deal with different types of spelling errors. a chunk-based decoding approach is proposed to model both singlecharacter and multi-character words in a uniform way. we also finetune an error model based on the large-scale pretrained language model to include deep semantic information. a global optimization algorithm is adopted to combine different features and select the best correction. the experiment results show that the proposed approach achieves a new state-of-the-art performance on the three benchmark datasets. a further experiment shows that our method is also effective for optical character recognition (ocr) errors. our contributions are summarized as follows: ## approach the workflow of the proposed approach is shown in figure 1 . the proposed spelling check method adopts the chunk-based decoding, which processes single-character and multi-character words in a uniform way. during decoding, the candidates with variable length are dynamically generated according to the input sentence and the partially decoded sentence. for selecting the best correction, a global ranking optimization is used to combine different features. foot_1 ## related work previous work of csc is closely related to a series of shared tasks @xcite @xcite . the workflow of csc systems can be roughly divided into two phases, candidate generation and candidate selection. for the candidate generation phase, most of the previous work retrieves the candidates according to pronunciation or shape @xcite @xcite . recently, @xcite propose to replace the traditional confusion sets with a dynamically generated one. they treat the csc as a sequence labeling problem and finetune a pretrained masked language model to generate candidates. for reducing the false alarm rate, they filter the result with pronunciation and shape similarity. their method inspired us to finetune the masked language model for generating semantically related candidates. for the candidate selection phase, the perplexity from language models is frequently used for selecting the most reasonable candidate @xcite @xcite @xcite . rules are effective and often included in the csc model for handling singlecharacter errors @xcite @xcite . recent researchers rely on supervised methods to achieve further improvement. the supervised error model is frequently involved in previous work @xcite @xcite . @xcite uses the support vector machines (svms) to rerank the candidate list. @xcite employ a maximum entropy (me) model for correction selection. @xcite use conditional random fields (crfs) to handle two types of misspelled single-character word. @xcite propose to incorporate phonological and visual similarity knowledge into the csc models via a graph convolutional network. due to the limited size of csc training data, the supervised models suffer from the lack of annotated data. @xcite generate pseudo data by replacing the character in the training sentence with characters in the confusion set. similarly, @xcite generate homophonous pseudo data to pretrain the detection and correction network jointly. web texts are in large quantities and contain more errors than published articles. @xcite propose to extract spelling error samples from the google web 1t corpus. @xcite propose the ocr-based and asr-based methods to mimic human errors. they further proposed a pointer network to model the csc task under the framework of a seq2seq model @xcite . ## conlusion in this work, we present a new framework for chinese spelling check. we include the masked language model for generating semantically related candidates. the chunk-based decoding is employed to handle single-character and multicharacter errors in a uniform way. a global optimization strategy is adopted for combining different features. the effectiveness of the proposed method is verified on three csc benchmark datasets and an ocr subtitle dataset. as for the future work, we plan to extend the proposed framework to chinese grammatical error correction and explore the possibilities of training in an end-to-end style.