ARM : An Alignment-and-Replacement Module for C hinese Spelling Check Based on LLM s (2024)

Abstract: Chinese Spelling Check (CSC) aims to identify and correct spelling errors in Chinese texts, where enhanced semantic understanding of a sentence can significantly improve correction accuracy. Recently, Large Language Models (LLMs) have demonstrated exceptional mastery of world knowledge and semantic understanding, rendering them more robust against spelling errors. However, the application of LLMs in CSC is a double-edged sword, as they tend to unnecessarily alter sentence length and modify rare but correctly used phrases. In this paper, by leveraging the capabilities of LLMs while mitigating their limitations, we propose a novel plug-and-play Alignment-and-Replacement Module ARM that enhances the performance of existing CSC models and without the need for retraining or fine-tuning. Experiment results and analysis on three benchmark datasets demonstrate the effectiveness and competitiveness of the proposed module.

## introduction chinese spelling check (csc) is a fundamental natural language processing (nlp) task behind many downstream applications, including web search @xcite . it aims to detect and correct spelling errors in chinese texts, with a specific focus on alignment errors @xcite . alignment errors do not alter the length of the text, as corrections are made exclusively through the substitution of characters without the operation of addition or deletion. typically, these errors originate from automatic speech recognition (asr) or optical character recognition (ocr) systems, often involving the incorrect use of characters that are phonologically or visually similar @xcite . according to the characteristics csc errors, previous studies have primarily utilized 他睡得太熟了，睡到忘了时间起床。 he slept so deeply that he forgot the time to get up. ## related work chinese spelling check, an important task in natural language processing, emerged in the 1990s and has increasingly attracted scholarly attention over the past decade. initially, scholars manually devised rules tailored to types of errors to facilitate correction @xcite @xcite . subsequently, researchers adopted statistical methods, utilizing large-scale corpora to both detect and correct textual inaccuracies @xcite @xcite . recently, the advent of deep learning has dramatically influenced csc, particularly with the widespread adoption of plms such as bert @xcite and roberta @xcite . innovations extend to the integration of phonetic and visual character information in models. for instance, realise @xcite employs resnet @xcite to extract the visual information of characters, acquires wordlevel and sentence-level phonetic information by gru @xcite and transformer blocks. similarly, models like plome @xcite and dcn @xcite have also been developed to harness these information. further advancements in csc include the formulation of novel pretraining objectives and mask strategies. scope @xcite uses two parallel decoders with an adaptive weighting scheme and proposes fine granularity pinyin prediction task which predict the initial, final, and tone of pinyin. lead @xcite , craspell @xcite , and mft @xcite also design different training objectives or mask strategies. additionally, some studies have sought to restructure model architectures to optimize correction processes. for example, softmask-bert @xcite and mdcspell @xcite explore the synergy between detection and correction networks. dr-csc @xcite breaks down csc task into three subcomponents: detection, reasoning, and searching, which allows for more efficient leveraging of external chinese linguistic knowledge. following the advent of llms, some researchers @xcite begin to explore the capabilities of llms in csc. their research pointed out that llms have many advan-tages in csc tasks, such as better handling of complex csc samples and better tolerance for errors. in addition, when evaluation criteria are adjusted, the performance of llms is found to be comparable to that of traditional models. nevertheless, significant challenges still persist, such as the inability to constrain output length and the tendency to introduce unnecessary modifications. therefore, our research aims to unveil llms potential and pioneer the integration of llms into csc task. ## methodology in this section, we commence with the formulation of the csc task ( §3.1). we then elaborate on our proposed arm, depicted in figure 2 . comprehensive details on the "alignment operation" are provided in the appendix d. the introduction of our alignment method ers is presented in §3.2, followed by a description of the replacement strategy scp, which is shown in §3.3. ## conclusion we introduces arm, a novel module designed to ameliorate critical deficiencies in llms when ap-plied to csc task. arm encompass two principal approaches: the alignment method ers and replacement strategies scp. ers processes sentences where outputs do not match inputs in length, and enhances the alignment of llms output with the original sentence in terms of length and similarity. concurrently, scp rigorously assesses the appropriateness of candidates provided by llms, determining whether they should supplant the outputs from existing csc models. by incorporating these approaches with current csc models, arm have demonstrated superior performance, achieving state-of-the-art results across three sighan datasets, thereby demonstrating the module's effectiveness and competitiveness.