MDCS pell: A Multi-task Detector-Corrector Framework for C hinese Spelling Correction (2022)

Abstract: Chinese Spelling Correction (CSC) is a task to detect and correct misspelled characters in Chinese texts. CSC is challenging since many Chinese characters are visually or phonologically similar but with quite different semantic meanings. Many recent works use BERT-based language models to directly correct each character of the input sentence. However, these methods can be sub-optimal since they correct every character of the sentence only by the context which is easily negatively affected by the misspelled characters. Some other works propose to use an error detector to guide the correction by masking the detected errors. Nevertheless, these methods dampen the visual or phonological features from the misspelled characters which could be critical for correction. In this work, we propose a novel general detector-corrector multi-task framework where the corrector uses BERT to capture the visual and phonological features from each character in the raw sentence and uses a late fusion strategy to fuse the hidden states of the corrector with that of the detector to minimize the negative impact from the misspelled characters. Comprehensive experiments on benchmarks demonstrate that our proposed method can significantly outperform the state-of-the-art methods in the CSC task.

## introduction chinese spelling correction (csc) is a fundamental task that aims to automatically detect and correct spelling errors in chinese texts. these spelling errors are typically caused by human writing, automatic speech recognition (asr) or optical character recognition (ocr) systems @xcite . csc is essential since it is crucial for many downstream tasks like search engine @xcite and essay scoring @xcite . input 我这一次写信给你是想跟你安排一 下关(guān)以(yǐ)我们要见面的。 baseline 我这一次写信给你是想跟你安排一 下所(suǒ)以(yǐ)我们要见面的事。 ground truth 我这一次写信给你是想跟你安排一 下关(guān)于(yú)我们要见面的事。 translation i am writing to you this time to make arrangements with you about our meeting. input 为了减少急遍(biàn)的生孩子率，需 要呼吁适当的生育政策。 baseline 为了减少急速(sù)的生孩子率，需要 呼吁适当的生育政策。 ground truth 为了减少急变(biàn)的生孩子率，需 要呼吁适当的生育政策。 translation in order to reduce the rapidly changing rate of childbirth, it is necessary to call for an appropriate childbirth policy. to address the above issues, we propose a novel general multi-task detector-corrector csc framework (mdcspell) which can both employ the visual and phonological features of the misspelled characters while eliminating their misleading impact on the context. specifically, the correction and detection tasks are executed simultaneously where the corrector uses bert to capture the visual and phonological features of all characters directly from the raw sentence and the detector uses a light-weight transformer to detect the positions of misspelled characters. a late-fusion strategy is employed to fuse the hidden states of the corrector with that of the detector and enable the elimination of the misleading impact from the misspelled characters with an end-to-end joint training. this framework is simple to implement and any bertbased csc model can be easily adapted in this framework. experimental results on three open benchmarks demonstrate that mdcspell can significantly outperform the competitors. in summary, our contributions are concluded as follows: ## related work chinese spelling correction (csc) is an important and challenging task. it mainly needs to detect the wrong characters based on the judgment of the semantics and correct these wrong characters with a full understanding of the context. most of the early work used unsupervised language models and rules for detection and correction, and used the perplexity of language model for determination @xcite @xcite . recently a lot of works tend to transform csc into a sequence tagging task, modeling each character in the sentence to determine the position of error and correct it into the right character @xcite @xcite . with the development of large-scale pretraining in nlp, an increasing number of works follow the way of solving sequence labeling problems, i.e., using the bert-like model to directly map every character in the sentence to the correct ones. @xcite proposed a model named spell-gcn which incorporates phonological and visual similarity knowledge into bert via a specialized graph convolutional network. @xcite utilized phonological and morphological knowledge to model the similarities of the characters for correction. @xcite proposed a global attention decoder that learns the global relationship of the potential correct input characters and the candidates of potential error characters. @xcite proposed a dynamic connected network to model the dependencies between two adjacent chinese characters to improve the corrector. some other works use an error detector as the preliminary for correction which turns the csc into a two-stage pipeline. @xcite proposed the faspell model to predict candidate characters based on the bert model and exploit the phonological and visual similarity information to select candidate characters. @xcite use a two-stage detection and correction method named soft-masked bert, which masked the detected error characters with error probability and then turn the masked input into the bert model for error correction. @xcite proposed a two-stage cloze-style detector-corrector framework for correction. although the above methods have achieved good results on csc tasks, they either suffer from the misleading effect on the context of the misspelled characters or miss the critical visual and phonological features of the spelling errors. in order to solve the above problems, we propose a multi-task learning architecture via late fusion to effectively use detection information for correction decisionmaking and improve the precision of the model. ## conclusions spelling errors have two sides to the csc task. specifically, their visual and phonological features are critical for substitution for the correct characters, but their misleading impact on the context can mislead the correction model in turn. in this paper, we proposed a general detector-corrector multi-task framework mdcspell which exploits the visual and phonological features of the misspelled characters and meanwhile minimizes their misleading impact on the context. the experiments demonstrate the effectiveness of our method. for future work, we will explore how to make better use of external knowledge to strengthen our model and other ways of using the detection information and extend the proposed framework to other problems like grammatical error correction.