A Simple yet Effective Training-free Prompt-free Approach to C hinese Spelling Correction Based on Large Language Models (2024)

Abstract: This work proposes a simple training-free prompt-free approach to leverage large language models (LLMs) for the Chinese spelling correction (CSC) task, which is totally different from all previous CSC approaches. The key idea is to use an LLM as a pure language model in a conventional manner. The LLM goes through the input sentence from the beginning, and at each inference step, produces a distribution over its vocabulary for deciding the next token, given a partial sentence. To ensure that the output sentence remains faithful to the input sentence, we design a minimal distortion model that utilizes pronunciation or shape similarities between the original and replaced characters. Furthermore, we propose two useful reward strategies to address practical challenges specific to the CSC task. Experiments on five public datasets demonstrate that our approach significantly improves LLM performance, enabling them to compete with state-of-the-art domain-general CSC models.

## introduction given a chinese character, there may exist many others with the same or similar pronunciations, or with similar shapes. this similarity can lead to incorrect character selection when using certain keyboard input methods. it is worth noting that nowadays most chinese users rely on pinyinbased input methods. besides, optical character recognition (ocr) and automatic speech recognition (asr) systems may also introduce errors during image/speech-to-text conversion. such incorrect characters in texts degrade communication efficiency, and sometimes even lead to misunderstanding. as illustrated in figure 1 , the task of chinese spelling correction (csc) aims to correct each incorrect character in a sentence @xcite , zhenghua li is the corresponding author. partially generated sentence: ## our approach given an input sentence x " x 1 , x 2 , ¨¨¨, x n , where x i denotes a character, a csc model outputs a sentence of the same length, denoted as y " y 1 , y 2 , ¨¨¨, y n . the key to the csc task is how to model the score of the input and output sentence pair, i.e., scorepx, yq. under a perspective of probabilistic modeling, ppx, yq can be decomposed into two parts: the first part corresponds to a distortion model, which captures the relationships between x and y. in other words, it interprets how spelling errors transform y to x. another important function of the distortion model is to make sure that y represents the same "meaning" as x, i.e., faithfulness. the second part corresponds to a large language model, which makes sure that y is fluent and correct from the language use perspective. please note that our use of llms is prompt-free. we do not provide csc-related instructions and examples as the prompt. more importantly, we do not give the input sentence to llms. we use llms as pure traditional language models for evaluating next-token probabilities. ## main results we present the main results in after applying our approach, all three llm families outperforms their prompt-based counterparts on all five datasets by a large margin. ## conclusion in this work, we propose a simple, training-free, and prompt-free approach to leverage llms for the csc task. two components, a large language model and a minimal distortion model, co-operate to correct spelling errors. we alleviate the local optima problem and over-correction issue, with two simple strategies, length reward and faithfulness reward, respectively. our comprehensive experiments have shown that our approach significantly improves llm performance. through our approach, llms demonstrate remarkable domain generalization capabilities, surpassing sota domain-general csc models, that are trained on extensive synthetic csc data, on most datasets.