A Frustratingly Easy Plug-and-Play Detection-and-Reasoning Module for C hinese Spelling Check (2023)

Abstract: In recent years, Chinese Spelling Check (CSC) has been greatly improved by designing task-specific pre-training methods or introducing auxiliary tasks, which mostly solve this task in an end-to-end fashion. In this paper, we propose to decompose the CSC workflow into detection, reasoning, and searching subtasks so that the rich external knowledge about the Chinese language can be leveraged more directly and efficiently. Specifically, we design a plug-and-play detection-and-reasoning module that is compatible with existing SOTA non-autoregressive CSC models to further boost their performance. We find that the detection-and-reasoning module trained for one model can also benefit other models. We also study the primary interpretability provided by the task decomposition. Extensive experiments and detailed analyses demonstrate the effectiveness and competitiveness of the proposed module.

## introduction spelling check aims to detect and correct spelling errors contained in the text @xcite . it benefits various applications, such as search engine @xcite , and educational scenarios @xcite @xcite @xcite . particularly, in this paper, we consider spelling check for the chinese language, i.e., chinese spelling check (csc), because of the challenge brought by the peculiarities of chinese to the task @xcite . suffering from many confusing characters, chinese spelling errors are mainly caused by phonologically and visually similar characters @xcite . as shown in figure 1 , "侍(shì)" is the visual error character of "待(dài)" because they have almost the same strokes. recently, with the rapid development of pretrained language models (plms) @xcite @xcite , methods based on plms have gradually become the mainstream of csc research @xcite @xcite @xcite , most of which are non-autoregressive models making predictions for each input chinese character. previous works can be roughly divided into two categories: (1) design tailored pre-training objectives to guide the model to learn the deep semantic information @xcite @xcite ; (2) perform different fine-tuning processes like adding auxiliary tasks to fine-tune the models @xcite . despite the remarkable success achieved by previous works, existing models seem to have some shortcomings in incorporating external expert knowledge, such as the confusion set, which contains a set of similar character pairs @xcite . @xcite makes an attempt that replaces the classifier's weight according to the confusion set using graph neural networks. @xcite further proposes a neural model that learns knowledge from dictionary definitions. however, these approaches implicitly learn external knowledge, thus lacking the desired level of interpretability and efficiency. we argue that the csc task can be decomposed into three subtasks, including detection, reasoning, and searching, aiming to incorporate the external knowledge more naturally and provide more interpretability. these subtasks answer the following questions respectively, "which character is misspelled?", "why did this error occur?" and "how to correct it?". by addressing these questions, the model gains access to error position, misspelling type, and external knowledge from three subtasks, respectively. as a result, a deeper understanding of the underlying processes involved in correcting misspelled characters can be achieved. to this end, we propose a frustratingly easy plug-and-play detection-and-reasoning module for chinese spelling check (dr-csc) to incorporate external expert knowledge naturally and effectively into existing non-autoregressive @xcite csc models. the proposed method is designed to be compatible with a wide range of nonautoregressive csc models, such as soft-masked bert @xcite , macbert @xcite , and scope @xcite . we decompose the csc task into three subtasks from easy to hard: (1) detection: we model the process of detecting misspellings from a sentence through the sequence labeling task, that is, predicting for each character in the sentence whether it is correct or wrong. (2) reasoning: for the wrong characters predicted in the detection step, we use the representations of csc models to further enable the model to consider the attribution of spelling errors by classifying them to know whether they are errors caused by phonetics or vision. (3) searching: after determining the spelling error type, we search within its corresponding phonological or visual confusion set to determine the correct character as the correction result. the proposed dr-csc module performs multi-task learning on these three subtasks. by doing so, it naturally incorporates the confusion set information, and effectively narrows the search space of candidate characters. as a result, this module greatly helps to improve the performance of csc models. in summary, the contributions of this work are in three folds: csc-targeted pre-training tasks researchers design different pre-training strategies to obtain plms that are more suitable for csc. plome @xcite proposes to pre-train models with their designed particular character masking strategy guided by the confusion set and apply task-specific pre-training to enhance the csc models. spell-bert @xcite , dcn @xcite , and mlm-phonetics @xcite leverage the phonetic information to improve the adaptation of the pre-training process to the csc task. for more recent research, wspeller @xcite trains a word-masked language model to utilize the segment-level information which provides appropriate word boundaries. ## methodology as shown in figure 2 , dr-csc consists of an encoder (i.e., e) and three progressive subtasks. dr-csc is a module that can be combined with various existing non-autoregressive csc models or proper plms. ## conclusion in this paper, we present an enhanced approach for the csc model by decomposing it into three subtasks. the main objective is to integrate external knowledge and offer improved interpretability. we introduce dr-csc, a frustratingly easy plugand-play detection-and-reasoning module that consists of three subtasks with progressive difficulty. by conducting thorough experiments and detailed analyses, we have empirically demonstrated the effectiveness of our decomposition approach, as well as the valuable information provided by each subtask. furthermore, we believe that dr-csc holds untapped potential for further exploration.