DISC: Plug-and-Play Decoding Intervention with Similarity of Characters for Chinese Spelling Check
Authors: Ziheng Qiao, Houquan Zhou, Yumeng Liu, Zhenghua Li, Min Zhang, Bo Zhang, Chen Li, Ji Zhang, Fei Huang
Accepted at: acl25

DISC: Plug-and-Play Decoding Intervention with Similarity of Characters for Chinese Spelling Check

Ziheng Qiao1, Houquan Zhou1, Yumeng Liu1, Zhenghua Li1,∗ Min Zhang1
Bo Zhang2, Chen Li2, Ji Zhang2, Fei Huang2
1School of Computer Science and Technology, Soochow University, China
2DAMO Academy, Alibaba Group, China
{zhqiao,hqzhou,ymliu14}@stu.suda.edu.cn,
{zhi13,minzhang}@suda.edu.cn,
{klayzhang.zb,puji.lc,zj122146,f.huang}@alibaba-inc.com

Abstract
One key characteristic of the Chinese spelling check (CSC) task is that incorrect characters are usually similar to the correct ones in either phonetics or glyph. To accommodate this, previous works usually leverage confusion sets, which suffer from two problems, i.e., difficulty in determining which character pairs to include and lack of probabilities to distinguish items in the set. In this paper, we propose a light-weight plug-and-play DISC (i.e., decoding intervention with similarity of characters) module for CSC models. DISC measures phonetic and glyph similarities between characters and incorporates this similarity information only during the inference phase. This method can be easily integrated into various existing CSC models, such as ReaLiSe, SCOPE, and ReLM, without additional training costs. Experiments on three CSC benchmarks demonstrate that our proposed method significantly improves model performance, approaching and even surpassing the current state-of-the-art models.

1 Introduction
Given an input sentence, Chinese spelling check (CSC) aims to detect incorrect characters and modify each into a correct character (Yu and Li, 2014; Xu et al., 2021). Table 1 gives two examples. Spelling errors degrade reading efficiency, and sometimes even lead to misunderstanding. The authority or attitude of the writer may be doubted if their document contains simple spelling errors. Moreover, spelling errors substantially hurt the performance of subsequent NLP models.

As is well known, spelling errors in Chinese texts have three major sources, i.e., 1) from keyboard typing with some input methods, 2) from image or document scanning with some optical character recognition (OCR) software, and 3) from speech-to-text translation with some automatic speech recognition (ASR) software. Nowadays, most Chinese users employ Pinyin-based input methods. Considering the three sources, we can see that the incorrect character in most cases is similar to the underlying correct one in phonetics or glyph, sometimes in both. This is a key characteristic of the CSC task.

Previous works employ confusion sets to leverage such similarities among characters (Yeh et al., 2013; Huang et al., 2014; Xie et al., 2015; Cheng et al., 2020; Huang et al., 2023). Formally, a confusion set is denoted as \( \mathcal{C} = \{ (c_i^1, c_i^2) \}_{i=1}^M \), where each pair \( (c_i^1, c_i^2) \) represents a pair of characters and means that \( c_i^1 \) may be mistakenly replaced by \( c_i^2 \) in real texts.

As a representative work, Wang et al. (2018) construct a confusion set via two channels. First, they add noise into glyph images and apply OCR. Second, they apply ASR to parallel speech/text data. Their confusion set covers about 5K characters and consists of 19K character pairs that are likely to be confused with each other in written texts.

The most direct and popular use of confusion sets is to constrain the search space during the inference phase. The model can only consider character pairs in \( \mathcal{C} \). More specifically, if \( (c_1, c_2) \notin \mathcal{C} \), the model can never change \( c_2 \) into \( c_1 \). The justi-

<table>
  <tr>
    <th>Input</th>
    <th>记得戴眼晴(jīng)。</th>
    <th>Remember to wear eyes.</th>
  </tr>
  <tr>
    <th>Reference</th>
    <th>记得戴眼镜(jìng)。</th>
    <th>Remember to wear glasses.</th>
  </tr>
  <tr>
    <th>Input</th>
    <th>从商场的人(rén)口进去。</th>
    <th>Enter through the mall’s population.</th>
  </tr>
  <tr>
    <th>Reference</th>
    <th>从商场的入(rù)口进去。</th>
    <th>Enter through the mall’s entrance.</th>
  </tr>
</table>

Table 1: Two CSC examples. “晴”(jing, eyes) and “镜”(jing, glasses) are a pair of characters that are similar in phonetics, and “人”(human) and “入”(enter) are similar in glyph.
fication for such constrained decoding is that the resulting sentence may deviate from the meaning of the input sentence (i.e., unfaithfulness), if the model replaces a character with a totally unrelated new one.

Despite their popularity and usefulness, confusion sets have two problems. First, it is difficult to set criteria to decide the inclusion or exclusion of certain character pairs. This renders the construction of confusion sets highly empirical, sometimes requiring manual intervention. Second, there is no probability to distinguish which character pairs are more likely to be confused than others in \( \mathcal{C} \).

As a replacement for confusion sets, we propose a lightweight plug-and-play DISC (decoding intervention with similarity of characters) module. DISC derives probability-based similarities among characters in both phonetics and glyph, and uses them to intervene in the decoding process. Similar to the confusion set, our DISC aims to enhance the model’s precision. However, for datasets that lack or have no in-domain training data, DISC may result in under-corrections due to the model’s conservative predictions, leading to unstable recall. To address this, we propose a copy-punishment solution to balance precision and recall.

It is worth noting that DISC is featured in compatibility. On the one hand, DISC is compatible with the ways to derive probabilities for representing character similarity. On the other hand, DISC is compatible with almost all the current mainstream CSC models, such as SoftMasked-BERT (Zhang et al., 2020), ReaLiSe (Xu et al., 2021), SCOPE (Li et al., 2022), and ReLM (Liu et al., 2024).

Experiments and analyses on popular benchmark datasets, i.e., SIGHANs, ECSpell, and LEMON, demonstrate that our DISC module can significantly enhance the error correction performance of CSC models. This improvement does not require additional training costs and only slightly affects the decoding efficiency of the model. We release our code at https://github.com/zhqiao-nlp/DISC.

2 The Basic CSC Model

Given an input sentence consisting of \( n \) characters, denoted as \( x = x_1 x_2 \cdots x_n \), the goal of a CSC model is to output a corresponding correct sentence, denoted as \( y = y_1 y_2 \cdots y_n \), in which all erroneous characters in \( x \) are replaced with the correct ones.

Presently, mainstream approaches treat CSC as a character-wise classification problem (Zhang et al., 2020; Liu et al., 2021; Xu et al., 2021), i.e., determining whether a current character should be kept the same or be replaced with a new character.

Encoding. Given \( x \), the encoder of the CSC model generates representations for each character:

\[
h_1 \cdots h_n = \textbf{Encoder}(x).
\]

To leverage the power of pre-trained language models, a BERT-like encoder is usually employed.

Classification. For each character position, for instance \( h_i \), the CSC model employs MLP and softmax layers to obtain a probability distribution over the whole character vocabulary \( \mathcal{V} \):

\[
p(y \mid x, i) = \text{softmax}(\text{MLP}(h_i))[y].
\]

During the evaluation phase, the model selects the character with the highest probability, i.e., \( y^* = \arg\max_{y \in \mathcal{V}} p(y \mid x, i) \).

Training. The typical training procedure consists of 2–3 steps for the CSC task. First, automatically synthesize large-scale CSC training data by replacing some characters with others randomly, sometimes constrained by a given confusion set. Second, train the CSC model on the synthesized training data. Third, fine-tune the model on a small-scale in-domain training data, if the data is available.

3 Our Approach

In this paper, we propose a simple plug-and-play module to intervene in the classification (or prediction) process of any off-the-shelf CSC model. The basic idea is to adjust the probability distribution according to the similarity between a candidate character \( y \) and the original character \( x_i \):

\[
\text{Score}(x, i, y) = p(y \mid x, i) + \alpha \times \text{Sim}(x_i, y),
\]

where \( \text{Sim}(\cdot) \) gives the similarity between two characters, and \( \alpha \) is a hyperparameter and we set \( \alpha = 1.1 \) for all datasets and basic models according to a few preliminary experiments. We use \( \text{Score}(\cdot) \) to denote the replacement likelihood since the value is no longer a probability.

Our experiments show that by encouraging the model to prefer similar characters, our approach achieves a consistent and substantial performance boost on all CSC benchmark datasets.
Figure 1: Overview of DISC. It intervenes in the CSC decoder with the similarity between the potential error character and its candidate characters. The DISC module intervenes in the probability distribution results of the CSC model based on specific similarity, favoring the selection of more similar confusing characters.

We measure character similarity from two perspectives, i.e., phonetic and glyph:

\[
\text{Sim}(c_1, c_2) = \beta \times \text{Sim}^{\text{P}}(c_1, c_2) \\
+ (1 - \beta) \times \text{Sim}^{\text{G}}(c_1, c_2),
\]

where \( \beta \) is an interpolation hyperparameter, our experiments in Section 6 demonstrate that the model achieves good and stable performance when it is set to 0.7.

3.1 Phonetic Similarity

Given two characters, we employ the pypinyin library to obtain the Pinyin sequences,\footnote{https://pypi.org/project/pypinyin} e.g., “忠” (zhong) and “仲” (zhong),\footnote{We do not use the tone information, e.g., “忠” (zhōng) and “仲” (zhòng), which is not helpful for model performance according to our preliminary experiments. We suspect the reason is that Pinyin-based input methods do not require users to input the tones. Therefore, tones are not directly related to spelling errors.} and then compute the phonetic similarity based on the edit distance over their Pinyin sequences:

\[
\text{Sim}^{\text{P}}(c_1, c_2) = 1 - \frac{\text{LD}(\text{py}(c_1), \text{py}(c_2))}{\text{len}(\text{py}(c_1) + \text{py}(c_2))},
\]

where \( \text{LD}(\cdot) \) gives the Levenshtein distance,\footnote{Levenshtein distance is a type of edit distance. We set the weights of the three types of operations, i.e., deletion, insertion and substitutions, as 1/1/2 respectively.} and \( \text{len}(\cdot) \) gives the total length of the two sequences.

Handling polyphonic characters. Given two characters, we enumerate all possible Pinyin sequences of each character, and adopt the combination that leads to the highest similarity.

We have also tried more sophisticated strategies. For instance, we follow Yang et al. (2023b) and give higher weights to certain phoneme (consonant or vowel) pairs, since they are more likely to cause spelling errors. However, our preliminary experiments show that our simple strategy in Eq. (5) works quite robustly.

3.2 Glyph Similarity

According to Liu et al. (2010), 83% of Chinese spelling errors are related to pronunciation, while 48% are with glyphs, indicating that a considerable proportion is related to both. Therefore, it is necessary to consider the glyph information when computing character similarity.

Pinyin sequences can largely encode the phonetics of Chinese characters. In contrast, it is much more complex to represent character glyphs. In this work, we compute and fuse glyph similarity from four aspects:

\[
\text{Sim}^{\text{G}}(c_1, c_2) = \frac{\sum_{i=1}^{4} \text{Sim}^{\text{G}}_i(c_1, c_2)}{4}.
\]

Four-corner code. The four-corner method is widely used in Chinese lexicography for indexing characters. Given a character, it gives four digits ranging from 0 to 9, corresponding to the shapes
at the four corners of the character’s glyph, respectively. For instance, the four-corner code is 5033 for “忠”, and 2520 for “仲”.

Then, we use the digit-wise matching rate between two codes as the similarity:

\[
\mathrm{Sim}_1^G(c_1, c_2) = \frac{\sum_{i=1}^{4} \mathbb{1}(\mathrm{FC}(c_1)[i] = \mathrm{FC}(c_2)[i])}{4},
\]

where \( \mathrm{FC}(\cdot) \) gives the four-digit code, and \( \mathbb{1} \) is the indicator function.

Structure-aware four-corner code. One important feature of Chinese characters is that a complex character can usually be decomposed into simpler parts, and each part corresponds to a simpler character or a radical. Most radicals are semantically equivalent to some character, e.g., “亻” to “人”.

Such structural decomposition directly reveals how characters are visually similar to each other. Motivated by this observation, we design a structure-aware four-corner code for each character. For example,
“忠”: C5000C3300 (“中”: 5000; “心”: 3300)
“仲”: B8000B5000 (“人”: 8000; “中”: 5000)
where “C” leading a four-coner code means up-down structure, and “B” means left-right structure.

Then we compute the similarity based on the Levenshtein distance as follows:

\[
\mathrm{Sim}_2^G(c_1, c_2) = 1 - \frac{\mathrm{LD}(\mathrm{SFC}(c_1), \mathrm{SFC}(c_2))}{\mathrm{len}(\mathrm{SFC}(c_1) + \mathrm{SFC}(c_2))},
\]

where \( \mathrm{SFC}(\cdot) \) gives the structure-aware code of a character.

Stroke sequences. Four-corner codes focus on the shapes of the four corners. Some very similar characters may obtain quite different codes, e.g., “木” (4090) vs. “本” (5023). To address this issue, we utilize stroke sequence information, which encodes how a character is handwritten stroke by stroke. For example,
“木”: 一 | ノ、 (4 strokes)
“本”: 一 | ノ、一 (5 strokes)

Then we compute two similarity metrics from two complementary viewpoints. The first metric is based on Levenshtein distance:

\[
\mathrm{Sim}_3^G(c_1, c_2) = 1 - \frac{\mathrm{LD}(\mathrm{SS}(c_1), \mathrm{SS}(c_2))}{\mathrm{len}(\mathrm{SS}(c_1) + \mathrm{SS}(c_2))},
\]

where \( \mathrm{SS}(\cdot) \) gives the stroke sequence of a character.

The second metric considers the longest common subsequence, i.e., LCS(·):

\[
\mathrm{Sim}_4^G(c_1, c_2) = \frac{\mathrm{LCS}(\mathrm{SS}(c_1), \mathrm{SS}(c_2))}{\max(\mathrm{len}(\mathrm{SS}(c_1)), \mathrm{len}(\mathrm{SS}(c_2)))}.
\]

According to Eq. (4), and supposing \( \beta = 0.7 \), we get the similarity between “忠” and “仲” being:

\[
0.7 \times 1 + 0.3 \times \frac{0 + 0.56 + 0.57 + 0.5}{4} = 0.82.
\]

4 Experimental Setup

4.1 Datasets

Following the conventions of previous work, we employ the test sets of the SIGHAN 13/14/15 datasets (Wu et al., 2013; Yu et al., 2014; Tseng et al., 2015) as our evaluation benchmarks.

However, many previous studies have pointed out that the SIGHAN datasets may not represent real-world CSC tasks, as they are derived from Chinese learner texts. To address this limitation, we also conduct experiments on the ECSpell (Lv et al., 2023) and LEMON (Wu et al., 2023) datasets, which are derived from Chinese native-speaker (CNS) texts and encompass a wide range of domains. It is worth noting that LEMON does not have a dedicated training set, making it an excellent test set for evaluating a model’s generalization ability. Due to space constraints, we selected results from four domains for display and provided the average performance across all seven domains.

The details of these datasets are in Appendix B.

4.2 Baseline Models

We select three representative BERT-style models as our baselines: ReaLiSe, SCOPE, and ReLM.

The ReaLiSe model (Xu et al., 2021) employs multi-modal technology to capture semantic, phonetic, and glyph information. The SCOPE model (Li et al., 2022) is one of the SOTA models for CSC, which enhances model correction performance by introducing a character pronunciation prediction task. The ReLM model (Liu et al., 2024) treats CSC as a non-autoregressive paraphrasing task, standing out as a new SOTA model.

Additionally, we include some of the latest work (Cheng et al., 2020; Huang et al., 2023) for performance comparison.

In the era of LLMs, researchers have begun using LLMs to explore the CSC field. We present the results of representative LLMs on certain
<table>
  <tr>
    <th rowspan="2">Models</th>
    <th colspan="4">SIGHAN15</th>
    <th colspan="4">SIGHAN14</th>
    <th colspan="4">SIGHAN13</th>
  </tr>
  <tr>
    <th>C-P<sup>†</sup></th>
    <th>C-R<sup>†</sup></th>
    <th>C-F<sup>†</sup></th>
    <th>FPR<sup>†</sup></th>
    <th>C-P<sup>†</sup></th>
    <th>C-R<sup>†</sup></th>
    <th>C-F<sup>†</sup></th>
    <th>FPR<sup>†</sup></th>
    <th>C-P<sup>†</sup></th>
    <th>C-R<sup>†</sup></th>
    <th>C-F<sup>†</sup></th>
    <th>FPR<sup>†</sup></th>
  </tr>
  <tr>
    <td colspan="14">Previous SOTAs</td>
  </tr>
  <tr>
    <td>SpellGCN</td>
    <td>72.1</td>
    <td>77.7</td>
    <td>75.9</td>
    <td>–</td>
    <td>63.1</td>
    <td>67.2</td>
    <td>65.3</td>
    <td>–</td>
    <td>78.3</td>
    <td>72.7</td>
    <td>75.4</td>
    <td>–</td>
  </tr>
  <tr>
    <td>ReaLiSe</td>
    <td>75.9</td>
    <td>79.9</td>
    <td>77.8</td>
    <td>12.0</td>
    <td>66.3</td>
    <td>70.0</td>
    <td>68.1</td>
    <td>14.9</td>
    <td>87.2</td>
    <td>81.2</td>
    <td>84.1</td>
    <td>10.3</td>
  </tr>
  <tr>
    <td>SCOPE<sup>†</sup></td>
    <td>78.7</td>
    <td>83.5</td>
    <td>81.0</td>
    <td>11.3</td>
    <td>67.1</td>
    <td>71.2</td>
    <td>69.5</td>
    <td>14.8</td>
    <td>86.5</td>
    <td>82.1</td>
    <td>84.2</td>
    <td>17.2</td>
  </tr>
  <tr>
    <td>SCOPE + DR-CSC</td>
    <td><b>80.3</b></td>
    <td>82.3</td>
    <td>81.3</td>
    <td>–</td>
    <td><b>69.3</b></td>
    <td>72.3</td>
    <td>70.7</td>
    <td>–</td>
    <td>87.7</td>
    <td>83.0</td>
    <td>85.3</td>
    <td>–</td>
  </tr>
  <tr>
    <td>ReLM<sup>†</sup></td>
    <td>76.8</td>
    <td><b>83.9</b></td>
    <td>80.2</td>
    <td>12.7</td>
    <td>63.7</td>
    <td>72.3</td>
    <td>67.7</td>
    <td>17.5</td>
    <td>85.0</td>
    <td>82.3</td>
    <td>83.7</td>
    <td>10.8</td>
  </tr>
  <tr>
    <td colspan="14">LLMs Results</td>
  </tr>
  <tr>
    <td>GPT3.5</td>
    <td>32.7</td>
    <td>38.4</td>
    <td>35.3</td>
    <td>33.8</td>
    <td>39.7</td>
    <td>22.1</td>
    <td>28.4</td>
    <td>14.6</td>
    <td>57.1</td>
    <td>27.1</td>
    <td>36.7</td>
    <td>13.8</td>
  </tr>
  <tr>
    <td>GPT4</td>
    <td>36.5</td>
    <td>49.2</td>
    <td>41.9</td>
    <td>40.8</td>
    <td>32.8</td>
    <td>45.0</td>
    <td>38.0</td>
    <td>43.5</td>
    <td>47.3</td>
    <td>45.7</td>
    <td>46.5</td>
    <td>44.8</td>
  </tr>
  <tr>
    <td colspan="14">Ours</td>
  </tr>
  <tr>
    <td>ReaLiSe + DISC</td>
    <td>77.0</td>
    <td>79.9</td>
    <td>78.4<sup>†</sup></td>
    <td>11.3<sup>†</sup></td>
    <td>68.2</td>
    <td>70.2</td>
    <td>69.2<sup>†</sup></td>
    <td>13.7<sup>†</sup></td>
    <td>87.6</td>
    <td>81.1</td>
    <td>84.2<sup>†</sup></td>
    <td>10.3</td>
  </tr>
  <tr>
    <td>SCOPE + DISC</td>
    <td>80.2</td>
    <td>83.4</td>
    <td><b>81.8<sup>†</sup></b></td>
    <td>10.0<sup>†</sup></td>
    <td><b>69.3</b></td>
    <td>72.5</td>
    <td>70.9<sup>†</sup></td>
    <td>13.7<sup>†</sup></td>
    <td>88.0</td>
    <td>83.0</td>
    <td>85.4<sup>†</sup></td>
    <td>17.2</td>
  </tr>
  <tr>
    <td>ReLM + DISC</td>
    <td>79.8</td>
    <td>83.1</td>
    <td>81.4<sup>†</sup></td>
    <td>9.5<sup>†</sup></td>
    <td>68.6</td>
    <td>73.7</td>
    <td><b>71.0<sup>†</sup></b></td>
    <td>14.3<sup>†</sup></td>
    <td><b>88.4</b></td>
    <td><b>83.3</b></td>
    <td><b>85.8<sup>†</sup></b></td>
    <td><b>7.6<sup>†</sup></b></td>
  </tr>
</table>

Table 2: Sentence-level performance on the SIGHAN13, SIGHAN14 and SIGHAN15 test sets. Precision (P), recall (R) and F_1 for correction are reported (%). Results marked with “†” are obtained by rerunning the official code released by Li et al. (2022) and Liu et al. (2024). Other baseline results are directly taken from their literature. Apart from SpellGCN, all models apply post-processing on SIGHAN13, which removes all detected and corrected “地” and “得” from the model output before evaluation. “+ DISC” means adding DISC module in the decoder. α and β are assigned the values 1.1 and 0.7, respectively.

benchmarks for comparison, including the top-performing GPT series in terms of overall capability: GPT3.5 and GPT4, as well as some results on open-source LLMs in the Chinese NLP community from previous work, such as finetuned Baichuan2 (Yang et al., 2023a). Specifically, we demonstrate the results of Baichuan2 on the EC-Spell and LEMON test sets using supervised fine-tuning (SFT) (Liu et al., 2024) and prompt-free training-free approach (Zhou et al., 2024), representing the current SOTA performance.

4.3 Evaluation Metrics

The CSC task comprises two subtasks: error detection and error correction. Following the previous work (Zhang et al., 2020), we report the precision (P), recall (R), and F_1 scores at the sentence level for both subtasks. Additionally, we also evaluate the models with the False Positive Rate (FPR) metric (Liu et al., 2024), which quantifies the CSC model’s frequency of over-correction, i.e., incorrectly identifying correct sentences as erroneous.

4.4 Hyperparameters

Hyperparameters α and β denote the weights assigned to overall similarity and phonetic similarity, respectively. As detailed in Section 6 on grid search results, we set α = 1.1 in Eq. 3 and β = 0.7 in Eq. 4 for all experiments.

5 Main Results

Results on SIGHANs. Table 2 illustrates the main results across SIGHAN benchmarks, demonstrating that the addition of the DISC module in the decoding process leads to notable improvements across all the compared models, reaching state-of-the-art performance. Specifically, ReaLiSe + DISC has increases of 0.1/1.1/0.6, SCOPE + DISC achieves lifts of 1.2/1.4/0.8, and ReLM + DISC sees enhancements of 2.1/3.3/1.2 in correction-level F_1 (C-F) score on the SIGHAN13/14/15 test sets, respectively.

It is worth noting that ReaLiSe and SCOPE have incorporated phonetic or glyph information during training. However, our DISC module can still improve the performance of these models.

In addition to the consistent improvement in the F_1 metric, results demonstrate that the integration of the DISC module into CSC models leads to a significant reduction in FPR across almost all datasets. This implies that DISC can avoid some unnecessary corrections.

Results on Native Datasets. As ReLM has shown outstanding performance on the SIGHAN benchmarks, we continue to utilize it for experiments on the multi-domain datasets of ECSpell and LEMON to demonstrate the DISC module’s domain adaptability.

Table 3 depicts that the incorporation of the
<table>
  <tr>
    <th rowspan="2">Domain</th>
    <th rowspan="2">Model</th>
    <th colspan="3">Correction</th>
    <th rowspan="2">FPR</th>
  </tr>
  <tr>
    <th>P</th>
    <th>R</th>
    <th>F1</th>
  </tr>
  <tr>
    <td rowspan="5">LAW</td>
    <td>ECSpell</td>
    <td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>GPT3.5</td>
    <td>48.5</td>
    <td>43.1</td>
    <td>45.6</td>
    <td>9.4</td>
  </tr>
  <tr>
    <td>GPT4</td>
    <td>62.0</td>
    <td>62.0</td>
    <td>62.0</td>
    <td>7.3</td>
  </tr>
  <tr>
    <td>Baichuan2-7B*</td>
    <td>85.1</td>
    <td>87.1</td>
    <td>86.0</td>
    <td>–</td>
  </tr>
  <tr>
    <td>ReLM</td>
    <td>93.7</td>
    <td><b>98.8</b></td>
    <td>96.2</td>
    <td>6.5</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>96.5</b></td>
    <td><b>98.0</b></td>
    <td><b>97.3</b></td>
    <td><b>2.9</b></td>
  </tr>
  <tr>
    <td rowspan="5">MED</td>
    <td>GPT3.5</td>
    <td>36.5</td>
    <td>42.0</td>
    <td>39.1</td>
    <td>20.1</td>
  </tr>
  <tr>
    <td>GPT4</td>
    <td>45.1</td>
    <td>57.5</td>
    <td>50.6</td>
    <td>24.8</td>
  </tr>
  <tr>
    <td>Baichuan2-7B*</td>
    <td>72.6</td>
    <td>73.9</td>
    <td>73.2</td>
    <td>–</td>
  </tr>
  <tr>
    <td>ReLM</td>
    <td>85.1</td>
    <td>95.8</td>
    <td>90.2</td>
    <td>9.8</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>91.6</b></td>
    <td><b>96.3</b></td>
    <td><b>93.9</b></td>
    <td><b>4.6</b></td>
  </tr>
  <tr>
    <td rowspan="5">ODW</td>
    <td>GPT3.5</td>
    <td>57.3</td>
    <td>52.3</td>
    <td>54.7</td>
    <td>6.3</td>
  </tr>
  <tr>
    <td>GPT4</td>
    <td>71.7</td>
    <td>67.6</td>
    <td>69.5</td>
    <td><b>1.7</b></td>
  </tr>
  <tr>
    <td>Baichuan2-7B*</td>
    <td>86.1</td>
    <td>79.3</td>
    <td>82.6</td>
    <td>–</td>
  </tr>
  <tr>
    <td>ReLM</td>
    <td>89.4</td>
    <td><b>91.5</b></td>
    <td>90.4</td>
    <td>5.8</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>91.1</b></td>
    <td><b>91.1</b></td>
    <td><b>91.1</b></td>
    <td>3.3</td>
  </tr>
  <tr>
    <td rowspan="5">LEMON</td>
    <td>Baichuan2-7B*</td>
    <td>38.2</td>
    <td><b>35.6</b></td>
    <td>36.9</td>
    <td>19.8</td>
  </tr>
  <tr>
    <td>GAM</td>
    <td>35.8</td>
    <td>33.6</td>
    <td>34.6</td>
    <td>20.6</td>
  </tr>
  <tr>
    <td>ReLM</td>
    <td>56.1</td>
    <td>31.5</td>
    <td><b>40.4</b></td>
    <td><b>8.5</b></td>
  </tr>
  <tr>
    <td>Baichuan2-7B*</td>
    <td>64.3</td>
    <td>46.8</td>
    <td>54.2</td>
    <td>6.9</td>
  </tr>
  <tr>
    <td>CAR</td>
    <td>59.2</td>
    <td><b>48.9</b></td>
    <td>53.6</td>
    <td>12.0</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>72.3</b></td>
    <td>45.9</td>
    <td><b>56.2</b></td>
    <td><b>4.6</b></td>
  </tr>
  <tr>
    <td rowspan="5">ENC</td>
    <td>Baichuan2-7B*</td>
    <td>59.8</td>
    <td><b>45.1</b></td>
    <td><b>51.4</b></td>
    <td>10.2</td>
  </tr>
  <tr>
    <td>ReLM</td>
    <td>55.8</td>
    <td>41.6</td>
    <td>47.7</td>
    <td>12.7</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>72.2</b></td>
    <td>39.3</td>
    <td>50.9</td>
    <td><b>5.1</b></td>
  </tr>
  <tr>
    <td>Baichuan2-7B*</td>
    <td>77.5</td>
    <td><b>49.7</b></td>
    <td><b>60.6</b></td>
    <td>3.4</td>
  </tr>
  <tr>
    <td>MEC</td>
    <td>67.3</td>
    <td>44.9</td>
    <td>53.9</td>
    <td>5.8</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>82.2</b></td>
    <td>44.5</td>
    <td><b>57.7</b></td>
    <td><b>2.2</b></td>
  </tr>
  <tr>
    <td>Baichuan2-7B*</td>
    <td>62.1</td>
    <td><b>46.8</b></td>
    <td>53.2</td>
    <td>9.9</td>
  </tr>
  <tr>
    <td>Avg. (all)</td>
    <td>ReLM</td>
    <td>58.1</td>
    <td>45.1</td>
    <td>50.6</td>
    <td>11.7</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>73.7</b></td>
    <td>42.5</td>
    <td><b>53.7</b></td>
    <td><b>4.5</b></td>
  </tr>
</table>

Table 3: Sentence-level performance of LLMs, ReLM, and ReLM + DISC on the test sets of ECSpell and LEMON. Results marked with “*” are from Liu et al. (2024), and “†” are from Zhou et al. (2024).

DISC module into ReLM leads to substantial improvements of 1.1/3.7/0.7 C-F score compared to unenhanced ReLM in the LAW, MED and ODW domains, respectively.

Table 3 also presents the performance of DISC on LEMON. After integrating the DISC module, the results of ReLM + DISC achieve notable improvements across all domains, and the average C-F has an increase of 3.1. This demonstrates that our DISC module yields stable and significant improvements in cross-domain CSC testing.

5.1 Case Study

We present two illustrative examples of DISC-augmented error correction in Figure 2. These examples explain why our DISC module can significantly improve model precision.

Input: 肌肉酸痛是运动过度(dù)导致的。
Muscle soreness is caused by read and exercise.
Reference: 读 → 度 (dù → dù, excessive)
ReLM: 读 → 少 (dù → shǎo, insufficient)
ReLM+DISC: 读 → 度 (dù → dù, excessive)

(a) Select the more similar word

Input: 浓荫蔽空(kōng)，郁郁苍苍。
Thick foliage shades the sky, lush and verdant.
Reference: NONE
ReLM: 空 → 日 (kōng → rì, sun)
ReLM+DISC: NONE

(b) Mitigate over-correction

Figure 2: Cases from the SIGHANs and LEMON.

Figure 2(a) exemplifies how the DISC module retrieves a more plausible alteration resembling the original character. In this example, the ReLM model corrects the erroneous word “读”(dù) to “少”(shǎo). This correction is grammatically correct, but deviates from the original meaning of the sentence. From the perspective of phonetics, a more suitable correction should be “度”(dù), which shares the same pronunciation as the erroneous word. The DISC makes this correction by leveraging semantic and phonetic information.

In Figure 2(b), the DISC alleviates over-correction. The CSC model mistakenly alters “空”(kōng) to “日”(rì), yet the similarity intervention rectifies this error. Specifically, since the most similar to a character is the character itself, when a CSC model incorrectly tends to correct over preserve on a correct sentence, the DISC module can increase the score of the character itself compared to other correction options based on similarity, which sometimes avoids unnecessary corrections.

6 Discussion

We select the SIGHAN15 along with two domains from the LEMON database, ENC and MEC, to conduct further analysis.

Robustness of similarity hyperparameters. As illustrated in Figure 3, the model’s precision steadily improves as \( \alpha \) increases. This is because increased similarity intervention reduces over-correction (Figure 2(a)), boosting precision. However, at the same time, DISC may revert predictions to the original character, as characters are most similar to themselves. This under-correction phenomenon caused by DISC sometimes leads to instability in recall.
Figure 3: The average scores in ENC, MEC and SIGHAN15 with different values of \( \alpha \) and \( \beta \). The solid lines represent the results of ReLM + DISC, and the dashed lines represent the results of the original ReLM.

For \( \beta \), it shows the effect of the proportion of phonetic similarity in the total similarity on correction performance. The F_1 score curve shows a clear trend of rising first and then decreasing, which indicates that phonetic and glyph similarities are complementary, with phonetic similarity being relatively more important than glyph similarity.

The balance between precision and recall. The primary purpose of using a confusion set is to narrow down the retrieval space, thereby improving precision. While a confusion set can enhance recall by enabling the model to make more reasonable edits, it may also reduce recall by discouraging the model from making edits, because the most similar character to the source character is itself.

We observe that this decrease in recall primarily occurs in the LEMON test set. The key distinction between these datasets is that LEMON contains less seen edit pairs in training data. As shown in Table 4, we calculate the proportion of edit pairs from each test set that appear in the training set.\footnote{For LEMON, we conduct statistics using the pairs in the confusion set which is used to generate 34 million monolingual sentences.} In ECSpell, the proportion of seen edit pairs is very low, yet the model performs well. This is due to data leakage, as we explain in Appendix E. Models are prone to copy the source character when the target edit pair is not available in the training data.

For this type of test set, we can use a simple copy punishment combined with DISC, which reduces the probability of copying the original character during inference, to mitigate the decrease in recall. Detailed experimental results can be found in Appendix C.

<table>
  <tr>
    <th>Domain</th>
    <th>Edit Pairs</th>
    <th>Seen Pairs</th>
    <th>Prop.</th>
  </tr>
  <tr>
    <td colspan="4">SIGHANs</td>
  </tr>
  <tr>
    <td>SIGHAN15</td>
    <td>703</td>
    <td>698</td>
    <td>99.29%</td>
  </tr>
  <tr>
    <td>SIGHAN14</td>
    <td>771</td>
    <td>765</td>
    <td>99.22%</td>
  </tr>
  <tr>
    <td>SIGHAN13</td>
    <td>1,224</td>
    <td>1,206</td>
    <td>98.53%</td>
  </tr>
  <tr>
    <td colspan="4">ECSpell</td>
  </tr>
  <tr>
    <td>LAW</td>
    <td>390</td>
    <td>211</td>
    <td>54.10%</td>
  </tr>
  <tr>
    <td>MED</td>
    <td>356</td>
    <td>169</td>
    <td>47.47%</td>
  </tr>
  <tr>
    <td>ODW</td>
    <td>404</td>
    <td>168</td>
    <td>41.58%</td>
  </tr>
  <tr>
    <td colspan="4">LEMON</td>
  </tr>
  <tr>
    <td>GAM</td>
    <td>164</td>
    <td>100</td>
    <td>60.98%</td>
  </tr>
  <tr>
    <td>CAR</td>
    <td>1,911</td>
    <td>1,254</td>
    <td>65.62%</td>
  </tr>
  <tr>
    <td>NOV</td>
    <td>3,415</td>
    <td>2,045</td>
    <td>59.88%</td>
  </tr>
  <tr>
    <td>ENC</td>
    <td>1,787</td>
    <td>1,040</td>
    <td>58.20%</td>
  </tr>
  <tr>
    <td>NEW</td>
    <td>3,260</td>
    <td>2,293</td>
    <td>70.34%</td>
  </tr>
  <tr>
    <td>COT</td>
    <td>486</td>
    <td>309</td>
    <td>63.58%</td>
  </tr>
  <tr>
    <td>MEC</td>
    <td>1,032</td>
    <td>627</td>
    <td>60.76%</td>
  </tr>
</table>

Table 4: Proportion of seen edit pairs in the test sets of SIGHANs, ECSpell, and LEMON.

Effectiveness of DISC module. We conducted experiments using the vanilla BERT without fine-tuning, initialized with bert-base-chinese, as shown in Table 5. Obviously, the general language model, without any fine-tuning or error correction strategies, cannot be applied to error correction tasks. However, after adding our DISC module, the performance of vanilla BERT improves significantly, giving the model basic error correction capabilities.

To further demonstrate the superiority of our method, we degrade the DISC module to a simple confusion set constraint decoding strategy. We investigate two confusion sets: one derived from our similarity computation strategy\footnote{We treat a character pair as confused if their similarity score exceeds 0.5.} and another pre-existing one provided by Wang et al. (2018). The results are shown in the second part of Table 6. From the results, we can see that both confusion sets fail to consistently improve performance, indicating the strategy’s sensitivity to confusion set quality. The confusion set from Wang et al. (2018) improves SIGHAN15 by covering over 99% of its erroneous pairs but degrades performance on other test sets, highlighting the domain-specific limitations of such confusion sets.

Effectiveness of components of the DISC module. We conduct an ablation study on the components of the DISC module. The results are shown in the third part of Table 6. Removing either phonetic or glyph knowledge from the DISC module
<table>
  <tr>
    <th rowspan="2">Domain</th>
    <th colspan="3">Correction</th>
    <th rowspan="2">FPR</th>
  </tr>
  <tr>
    <th>P</th>
    <th>R</th>
    <th>F1</th>
  </tr>
  <tr>
    <td>SIGHAN15</td>
    <td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>vanilla BERT</td>
    <td>2.3</td>
    <td>4.4</td>
    <td>3.1</td>
    <td>91.1</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>71.5</b></td>
    <td><b>25.5</b></td>
    <td><b>37.6</b></td>
    <td><b>2.1</b></td>
  </tr>
  <tr>
    <td>LEMON-ENC</td>
    <td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>vanilla BERT</td>
    <td>3.6</td>
    <td>5.8</td>
    <td>4.4</td>
    <td>73.0</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>79.9</b></td>
    <td><b>18.3</b></td>
    <td><b>29.8</b></td>
    <td><b>1.0</b></td>
  </tr>
  <tr>
    <td>LEMON-MEC</td>
    <td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>vanilla BERT</td>
    <td>2.7</td>
    <td>4.8</td>
    <td>3.5</td>
    <td>77.2</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>88.4</b></td>
    <td><b>18.5</b></td>
    <td><b>30.5</b></td>
    <td><b>0.3</b></td>
  </tr>
</table>

Table 5: Sentence-level performance of vanilla BERT.

<table>
  <tr>
    <th>Model</th>
    <th>ENC</th>
    <th>MEC</th>
    <th>SIG15</th>
    <th>Avg</th>
  </tr>
  <tr>
    <td>ReLM</td>
    <td>47.7</td>
    <td>53.9</td>
    <td>80.2</td>
    <td>60.6</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td>50.9</td>
    <td><b>57.7</b></td>
    <td><b>81.4</b></td>
    <td><b>63.3</b></td>
  </tr>
  <tr>
    <td>+ Confusion set</td>
    <td>47.1</td>
    <td>56.0</td>
    <td>78.0</td>
    <td>60.4</td>
  </tr>
  <tr>
    <td>+ Confusion set<sup>‡</sup></td>
    <td>41.5</td>
    <td>48.7</td>
    <td>80.7</td>
    <td>57.0</td>
  </tr>
  <tr>
    <td>+ DISC (phonetic)</td>
    <td>49.1</td>
    <td>56.1</td>
    <td>80.1</td>
    <td>61.8</td>
  </tr>
  <tr>
    <td>+ DISC (glyph)</td>
    <td>49.4</td>
    <td>53.3</td>
    <td>80.3</td>
    <td>61.0</td>
  </tr>
  <tr>
    <td>+ DISC (phonetic &)</td>
    <td></td><td></td><td></td><td></td>
  </tr>
  <tr>
    <td>\( \sim \mathrm{Sim}_1^G \)</td>
    <td>50.5</td>
    <td>56.8</td>
    <td><b>81.4</b></td>
    <td>62.9</td>
  </tr>
  <tr>
    <td>\( \sim \mathrm{Sim}_2^G \)</td>
    <td>50.5</td>
    <td>57.4</td>
    <td><b>81.4</b></td>
    <td>63.1</td>
  </tr>
  <tr>
    <td>\( \sim \mathrm{Sim}_3^G \)</td>
    <td>51.3</td>
    <td>57.5</td>
    <td>81.2</td>
    <td><b>63.3</b></td>
  </tr>
  <tr>
    <td>\( \sim \mathrm{Sim}_4^G \)</td>
    <td><b>51.6</b></td>
    <td>56.9</td>
    <td>80.8</td>
    <td>63.1</td>
  </tr>
</table>

Table 6: Ablation results in two kinds of confusion sets and different components of DISC. “‡” represents the confusion set from Wang et al. (2018). “Sim<sup>G</sup>” means using similarities of phonetics and the i-th part of glyph.

leads to performance declines across benchmarks. Notably, the absence of phonetic similarity has a lesser effect on SIGHAN15 but a stronger impact on LEMON. The results also show that the four components involved in calculating glyph similarity are independently effective. However, excluding any three typically causes a slight drop in performance, with exceptions like ENC. This phenomenon underscores the necessity of using multi-dimensional similarity measurements for a more comprehensive modeling of glyph similarity. Combining these often results in consistent improvements. Moreover, the fusion of phonetic and glyph similarities achieves the optimal error correction performance, affirming the necessity of integrating these two similarities.

Analysis on Different Error Types. Based on the similarity calculation strategy proposed in this paper, we separately filter the phonetic and glyph error types. Taking the phonetic error type as an example, the specific approach is as follows: for all edit pairs with a phonetic similarity < 0.5, we modify the source character to the golden character, resulting in a test set containing only phonetic edit pairs. The results are shown in Table 7.

According to our classification rule, the average proportions of phonetic and glyph on SIGHAN15 are 90.0% and 40.7%. From the results, it can be seen that DISC significantly improves performance on all error types. We find that, compared to phonetic errors, the original model exhibited a higher Recall metric and lower Precision metric on glyph errors, which makes the improvement brought by the DISC module more pronounced.

Impact on Decoding Efficiency. We examine the influence of the DISC module on decoding speed, with the results shown in Table 8. Phonetic and glyph similarities can be pre-calculated and DISC only need to index them during decoding. Thus, the time taken to decode each sentence increased merely by 14.3%, 3.5%, and 1.0% for ReaLiSe, SCOPE, and ReLM, respectively. The minor slowdown in decoding speed incurred by the DISC module is deemed acceptable considering the substantial enhancement it brings to the model’s performance. Notably, SCOPE exhibits significantly slower decoding speeds compared to the other two models, which we speculate may be attributed to its iterative decoding approach.

7 Related Work

Model architecture shift. Most early works on CSC employed a three-step pipeline, i.e., 1) detecting potential erroneous characters, 2) constructing new sentences by replacing erroneous characters with new ones based on a confusion set; and 3) evaluating the probability of the constructed sentences based on an n-gram language model and choose the one with the highest probability (Yeh et al., 2013; Yu and Li, 2014; Huang et al., 2014; Xie et al., 2015).

In the current deep-learning era, especially with the prevalence of PLMs, recent models directly perform character-level replacement via classification, as introduced in Section 2. There also exist some works that employ a two-step pipeline architecture, which first detects potentially erroneous characters and then replaces them at the detected positions (Zhang et al., 2020; Huang et al., 2023).

Utilizing confusion sets. These works fall into three categories. (1) At only the inference phase.
<table>
  <tr>
    <th rowspan="2">Domain</th>
    <th colspan="4">Correction</th>
  </tr>
  <tr>
    <th>P</th>
    <th>R</th>
    <th>F<sub>1</sub></th>
    <th>FPR</th>
  </tr>
  <tr>
    <td colspan="5">Phonetic errors</td>
  </tr>
  <tr>
    <td>ReLM</td>
    <td>65.2</td>
    <td>77.6</td>
    <td>70.9</td>
    <td>19.3</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>69.8</b></td>
    <td><b>78.0</b></td>
    <td><b>73.7</b></td>
    <td><b>14.3</b></td>
  </tr>
  <tr>
    <td colspan="5">Glyph errors</td>
  </tr>
  <tr>
    <td>ReLM</td>
    <td>44.7</td>
    <td>81.5</td>
    <td>57.7</td>
    <td>19.6</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>51.8</b></td>
    <td><b>83.4</b></td>
    <td><b>63.9</b></td>
    <td><b>15.1</b></td>
  </tr>
</table>

Table 7: Sentence-level performance of different error types on SIGHAN15.

<table>
  <tr>
    <th>Model</th>
    <th>Speed (ms/sent)</th>
    <th>Slowdown</th>
  </tr>
  <tr>
    <td>ReaLiSe</td>
    <td>24.5</td>
    <td>–</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td>27.5</td>
    <td>1.143×</td>
  </tr>
  <tr>
    <td>SCOPE</td>
    <td>138.6</td>
    <td>–</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td>143.4</td>
    <td>1.035×</td>
  </tr>
  <tr>
    <td>ReLM</td>
    <td>12.7</td>
    <td>–</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td>12.8</td>
    <td>1.010×</td>
  </tr>
</table>

Table 8: The decoding time per sentence with a batch size of 1 on SIGHAN15. The results are the average time of three runs.

Wang et al. (2019) and Bao et al. (2020) use the confusion set as constraints upon the search space, i.e., allowing the model to only consider characters in the confusion set.

(2) For data synthesis. Liu et al. (2021) use a confusion set \( \mathcal{C} \) to synthesize data for training CSC models. For a given correct sentence, they randomly select a character (e.g., \( c_i \)), and replace it with an incorrect character (e.g., \( c' \)). The replacement is constrained such that only pairs contained in the confusion set are considered, i.e., \( (c_i, c') \in \mathcal{C} \).

(3) At both training and inference phases. Cheng et al. (2020) construct two character graphs, one based on phonetic relatedness, and the other based on glyph relatedness, and employ GCN to obtain new character representations as extra inputs. Huang et al. (2023) use two confusion sets, one encoding phonetic relatedness, and the other encoding glyph relatedness. Given a potential spelling error, they use a classification module to judge which confusion set the error belongs to, with an extra training loss. During the test phase, the model can only consider characters from the corresponding confusion set according to the classification result.

Utilizing phonetic and glyph information. Besides the use of confusion sets, there exist some works that directly utilize phonetic and glyph information to enhance CSC models. Liu et al. (2021) and Li et al. (2022) add an extra task of predicting the phonetic of each input character. Xu et al. (2021) use GRU to encode Pinyin, and use CNN to encode glyphs (font pictures) for each input character, as extra character representations.

Decoding intervention. Gou and Chen (2021) extract features such as probability and rank of the original character and the top 1 candidate character, and use SVM to judge modification retention. Yin et al. (2024) retrieve similar segments from the training set, and intervene in the decoding process based on the segment (n-gram) similarity between the retrieved segments and the input. Lv et al. (2023) employ a word dictionary in the target domain to assist the decoding process.

8 Conclusions

We propose a plug-and-play decoding intervention strategy that enhances CSC models by utilizing phonetic and glyph similarities through a tailored algorithm. Unlike methods that alter model training, our training-free strategy only modifies the decoding process, making it adaptable to almost all mainstream CSC models. Experiments on multiple CSC benchmarks demonstrate that our method significantly improves baselines, and even surpasses the current SOTA models. Furthermore, experimental analyses demonstrate that our DISC module helps the model better identify similar candidate characters, effectively reducing over-correction. Our research has transcended the limitations of traditional confusion set decoding intervention, proving that specific measures and combinations of phonetic and glyph similarities are necessary.

Limitations

We believe that our work can be further improved from two aspects. First, our experiments focus on the CSC datasets, while our approach can apply to other languages such as Japanese and Korean. Second, as a general-use technique, our proposed approach for determining character similarity may not be optimal for CSC in specific domains or scenarios. In that case, we may need to consider more factors besides phonetic and glyph information to compute character similarity.

Acknowledgements

We are deeply grateful to the anonymous reviewers for their thoughtful comments and dedicated efforts, which have greatly contributed to enhancing the quality and clarity of our work.
This work was supported by National Natural Science Foundation of China (Grant No. 62036004 and 62176173), Alibaba Group through Alibaba Innovative Research Program, and Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions.

References

Zuyi Bao, Chen Li, and Rui Wang. 2020. Chunk-based Chinese Spelling Check with Global Optimization. In Findings of EMNLP, pages 2031–2040, Online.

Xingyi Cheng, Weidi Xu, Kunlong Chen, Shaohua Jiang, Feng Wang, Taifeng Wang, Wei Chu, and Yuan Qi. 2020. SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check. In Proceedings of ACL, pages 871–881, Online.

Wei Gou and Zheng Chen. 2021. Think twice: A post-processing approach for the chinese spelling error correction. Applied Sciences, 11(13).

Haojing Huang, Jingheng Ye, Qingyu Zhou, Yinghui Li, Yangning Li, Feng Zhou, and Hai-Tao Zheng. 2023. A Frustratingly Easy Plug-and-Play Detection-and-Reasoning Module for Chinese Spelling Check. In Findings of EMNLP, pages 11514–11525, Singapore.

Qiang Huang, Peijie Huang, Xinrui Zhang, Weijian Xie, Kaiduo Hong, Bingzhou Chen, and Lei Huang. 2014. Chinese Spelling Check System Based on Tri-gram Model. In Proceedings of CIPS-SIGHAN, pages 173–178, Wuhan, China.

Jiahao Li, Quan Wang, Zhendong Mao, Junbo Guo, Yanyan Yang, and Yongdong Zhang. 2022. Improving Chinese Spelling Check by Character Pronunciation Prediction: The Effects of Adaptivity and Granularity. In Proceedings of EMNLP, pages 4275–4286, Abu Dhabi, United Arab Emirates.

Chao-Lin Liu, Min-Hua Lai, Yi-Hsuan Chuang, and Chia-Ying Lee. 2010. Visually and Phonologically Similar Characters in Incorrect Simplified Chinese Words. In Proceedings of COLING, pages 739–747, Beijing, China.

Linfeng Liu, Hongqiu Wu, and Hai Zhao. 2024. Chinese Spelling Correction as Rephrasing Language Model. In Proceedings of AAAI, pages 18662–18670, Vancouver, Canada.

Shulin Liu, Tao Yang, Tianchi Yue, Feng Zhang, and Di Wang. 2021. PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction. In Proceedings of ACL-IJCNLP, pages 2991–3000, Online.

Qi Lv, Ziqiang Cao, Lei Geng, Chunhui Ai, Xu Yan, and Guohong Fu. 2023. General and Domain-adaptive Chinese Spelling Check with Error-consistent Pre-training. TALLIP, pages 1–18.

Yuen-Hsien Tseng, Lung-Hao Lee, Li-Ping Chang, and Hsin-Hsi Chen. 2015. Introduction to SIGHAN 2015 Bake-off for Chinese Spelling Check. In Proceedings of SIGHAN, pages 32–37, Beijing, China.

Dingmin Wang, Yan Song, Jing Li, Jialong Han, and Haisong Zhang. 2018. A Hybrid Approach to Automatic Corpus Generation for Chinese Spelling Check. In Proceedings of EMNLP, pages 2517–2527, Brussels, Belgium.

Dingmin Wang, Yi Tay, and Li Zhong. 2019. Confusionset-guided Pointer Networks for Chinese Spelling Check. In Proceedings of ACL, pages 5780–5785, Florence, Italy.

Hongqiu Wu, Shaohua Zhang, Yuchen Zhang, and Hai Zhao. 2023. Rethinking Masked Language Modeling for Chinese Spelling Correction. In Proceedings of ACL, pages 10743–10756, Toronto, Canada.

Shih-Hung Wu, Chao-Lin Liu, and Lung-Hao Lee. 2013. Chinese Spelling Check Evaluation at SIGHAN Bake-off 2013. In Proceedings of SIGHAN, pages 35–42, Nagoya, Japan.

Weijian Xie, Peijie Huang, Xinrui Zhang, Kaiduo Hong, Qiang Huang, Bingzhou Chen, and Lei Huang. 2015. Chinese Spelling Check System Based on N-gram Model. In Proceedings of SIGHAN, pages 128–136, Beijing, China.

Heng-Da Xu, Zhongli Li, Qingyu Zhou, Chao Li, Zishen Wang, Yunbo Cao, Heyan Huang, and Xian-Ling Mao. 2021. Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking. In Findings of ACL-IJCNLP, pages 716–728, Online.

Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, Fan Yang, Fei Deng, Feng Wang, Feng Liu, Guangwei Ai, Guosheng Dong, Haizhou Zhao, Hang Xu, Haoze Sun, Hongda Zhang, Hui Liu, Ji-amming Ji, Jian Xie, JunTao Dai, Kun Fang, Lei Su, Liang Song, Lifeng Liu, Liyun Ru, Luyao Ma, Mang Wang, Mickel Liu, MingAn Lin, Nuolan Nie, Peidong Guo, Ruiyang Sun, Tao Zhang, Tianpeng Li, Tianyu Li, Wei Cheng, Weipeng Chen, Xiangrong Zeng, Xiaochuan Wang, Xiaoxi Chen, Xin Men, Xin Yu, Xuehai Pan, Yanjun Shen, Yiding Wang, Yiyu Li, Youxin Jiang, Yuchen Gao, Yupeng Zhang, Zenan Zhou, and Zhiying Wu. 2023a. Baichuan 2: Open large-scale language models.

Liner Yang, Xin Liu, Tianxin Liao, Zhenghao Liu, Mengyan Wang, Xuezhi Fang, and Erhong Yang. 2023b. Is Chinese Spelling Check ready? Understanding the correction behavior in real-world scenarios. AI Open, pages 183–192.

Jui-Feng Yeh, Sheng-Feng Li, Mei-Rong Wu, Wen-Yi Chen, and Mao-Chuan Su. 2013. Chinese Word Spelling Correction Based on N-gram Ranked Inverted Index List. In Proceedings of SIGHAN, pages 43–48, Nagoya, Japan.
Xunjian Yin, Xinyu Hu, Jin Jiang, and Xiaojun Wan. 2024. Error-Robust Retrieval for Chinese Spelling Check. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 6257–6267, Torino, Italia. ELRA and ICCL.

Junjie Yu and Zhenghua Li. 2014. Chinese Spelling Error Detection and Correction Based on Language Model, Pronunciation, and Shape. In Proceedings of CIPS-SIGHAN, pages 220–223, Wuhan, China.

Liang-Chih Yu, Lung-Hao Lee, Yuen-Hsien Tseng, and Hsin-Hsi Chen. 2014. Overview of SIGHAN 2014 Bake-off for Chinese Spelling Check. In Proceedings of CIPS-SIGHAN, pages 126–132, Wuhan, China.

Shaohua Zhang, Haoran Huang, Jicong Liu, and Hang Li. 2020. Spelling Error Correction with Soft-Masked BERT. In Proceedings of ACL, pages 882–890, Online.

Houquan Zhou, Zhenghua Li, Bo Zhang, Chen Li, Shaopeng Lai, Ji Zhang, Fei Huang, and Min Zhang. 2024. A simple yet effective training-free prompt-free approach to Chinese spelling correction based on large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17446–17467, Miami, Florida, USA. Association for Computational Linguistics.

A Implementation Details

We use the official implementation of ReaLiSe and directly utilize the checkpoint provided by its GitHub repository,6 which initializes the semantic encoder with the weights of chinese-roberta-wwm-ext.7 ReLM uses the official BERT weights bert-base-chinese,8 and only offered the checkpoint after pre-training in 34 million monolingual sentences that are synthesized by confusion set. We fine-tune it on SIGHANs and ECSpell with a batch size of 128 and a learning rate of 3e-5, and the MFT strategy (Wu et al., 2023) is used during training. SCOPE utilizes the pre-trained weights from the ChineseBERT-base,9 and we leverage their official implementation for fine-tuning.10 We did not attempt DR-CSC + DISC because they have not fully open-sourced their work. Due to our decoding intervention strategy being deterministic, without any random factors, the experiments are conducted only once. All experiments are conducted on one Tesla V100S-PCIE-32GB GPU.

B Details of Datasets

SIGHANs. Following the setup of previous work, we employ SIGHAN 13/14/15 datasets (Wu et al., 2013; Yu et al., 2014; Tseng et al., 2015) as our training sets, in conjunction with Wang271K (Wang et al., 2018), which consists of 271K synthetically generated instances. We employ the test sets of SIGHAN13/14/15 for evaluation.

ECSpell. ECSpell (Lv et al., 2023) encompasses data from three domains: law, medical treatment, and official document writing. Unlike SIGHANs from Chinese learner texts, the sentences in ECSpell are derived from CNS texts.

LEMON. LEMON (Wu et al., 2023) also originates from CNS texts, containing over 22K instances spanning 7 domains. Given its lack of a dedicated training set, LEMON serves as a benchmark for evaluating the domain adaptation capability of CSC models.

We conduct detailed statistics on the above datasets, and the results are presented in Table 9.

<table>
  <tr>
    <th>Training Set</th>
    <th>#Sent</th>
    <th>Avg. Length</th>
    <th>#Errors</th>
  </tr>
  <tr>
    <td>SIGHAN15</td>
    <td>2,339</td>
    <td>31.3</td>
    <td>2,549</td>
  </tr>
  <tr>
    <td>SIGHAN14</td>
    <td>3,437</td>
    <td>49.6</td>
    <td>3,799</td>
  </tr>
  <tr>
    <td>SIGHAN13</td>
    <td>700</td>
    <td>41.8</td>
    <td>343</td>
  </tr>
  <tr>
    <td>Wang271K</td>
    <td>271,329</td>
    <td>42.6</td>
    <td>381,962</td>
  </tr>
  <tr>
    <td>ECSpell_LAW</td>
    <td>1,960</td>
    <td>30.7</td>
    <td>1,681</td>
  </tr>
  <tr>
    <td>ECSpell_MED</td>
    <td>3,000</td>
    <td>50.2</td>
    <td>2,260</td>
  </tr>
  <tr>
    <td>ECSpell_ODW</td>
    <td>1,720</td>
    <td>41.2</td>
    <td>1,578</td>
  </tr>
  <tr>
    <th colspan="4">Test Set</th>
  </tr>
  <tr>
    <td>SIGHAN15</td>
    <td>1,100</td>
    <td>30.6</td>
    <td>703</td>
  </tr>
  <tr>
    <td>SIGHAN14</td>
    <td>1,062</td>
    <td>50.0</td>
    <td>771</td>
  </tr>
  <tr>
    <td>SIGHAN13</td>
    <td>1,000</td>
    <td>74.3</td>
    <td>1,224</td>
  </tr>
  <tr>
    <td>ECSpell_LAW</td>
    <td>500</td>
    <td>29.7</td>
    <td>390</td>
  </tr>
  <tr>
    <td>ECSpell_MED</td>
    <td>500</td>
    <td>49.6</td>
    <td>356</td>
  </tr>
  <tr>
    <td>ECSpell_ODW</td>
    <td>500</td>
    <td>40.5</td>
    <td>404</td>
  </tr>
  <tr>
    <td>LEMON</td>
    <td>22,252</td>
    <td>35.4</td>
    <td>12,055</td>
  </tr>
</table>

Table 9: Statistics of the datasets.

C Copy Punishment

For datasets like LEMON that lack in-domain training data, we discover a simple recall-boosting solution: reducing the probability of selecting the
<table>
  <tr>
    <th rowspan="2">Domain</th>
    <th rowspan="2">Model</th>
    <th colspan="3">Correction</th>
    <th rowspan="2">FPR</th>
  </tr>
  <tr>
    <th>P</th>
    <th>R</th>
    <th>F<sub>1</sub></th>
  </tr>
  <tr>
    <td colspan="6">LEMON</td>
  </tr>
  <tr>
    <td rowspan="3">GAM</td>
    <td>ReLM</td>
    <td>35.8</td>
    <td>33.6</td>
    <td>34.6</td>
    <td>20.6</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>56.1</b></td>
    <td>31.5</td>
    <td>40.4</td>
    <td><b>8.5</b></td>
  </tr>
  <tr>
    <td>+ DISC*</td>
    <td>52.4</td>
    <td><b>37.0</b></td>
    <td><b>43.4</b></td>
    <td>11.3</td>
  </tr>
  <tr>
    <td rowspan="3">CAR</td>
    <td>ReLM</td>
    <td>59.2</td>
    <td><b>48.9</b></td>
    <td>53.6</td>
    <td>12.0</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>72.3</b></td>
    <td>45.9</td>
    <td><b>56.2</b></td>
    <td><b>4.6</b></td>
  </tr>
  <tr>
    <td>+ DISC*</td>
    <td>68.0</td>
    <td>47.5</td>
    <td>55.9</td>
    <td>6.2</td>
  </tr>
  <tr>
    <td rowspan="3">NOV</td>
    <td>ReLM</td>
    <td>46.3</td>
    <td><b>32.2</b></td>
    <td>38.0</td>
    <td>17.6</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>65.2</b></td>
    <td>29.6</td>
    <td><b>40.8</b></td>
    <td><b>7.1</b></td>
  </tr>
  <tr>
    <td>+ DISC*</td>
    <td>57.8</td>
    <td>31.2</td>
    <td>40.6</td>
    <td>10.2</td>
  </tr>
  <tr>
    <td rowspan="3">ENC</td>
    <td>ReLM</td>
    <td>55.8</td>
    <td>41.6</td>
    <td>47.7</td>
    <td>12.7</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>72.2</b></td>
    <td>39.3</td>
    <td>50.9</td>
    <td><b>5.1</b></td>
  </tr>
  <tr>
    <td>+ DISC*</td>
    <td>66.9</td>
    <td><b>41.7</b></td>
    <td><b>51.4</b></td>
    <td>7.1</td>
  </tr>
  <tr>
    <td rowspan="3">NEW</td>
    <td>ReLM</td>
    <td>68.5</td>
    <td><b>51.5</b></td>
    <td>58.8</td>
    <td>8.4</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>80.4</b></td>
    <td>48.1</td>
    <td>60.2</td>
    <td><b>3.2</b></td>
  </tr>
  <tr>
    <td>+ DISC*</td>
    <td>76.5</td>
    <td>49.7</td>
    <td><b>60.3</b></td>
    <td>4.3</td>
  </tr>
  <tr>
    <td rowspan="3">COT</td>
    <td>ReLM</td>
    <td>73.5</td>
    <td><b>62.8</b></td>
    <td>67.7</td>
    <td>4.9</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>87.4</b></td>
    <td>58.3</td>
    <td><b>69.9</b></td>
    <td><b>1.1</b></td>
  </tr>
  <tr>
    <td>+ DISC*</td>
    <td>80.8</td>
    <td>61.0</td>
    <td>69.5</td>
    <td>1.8</td>
  </tr>
  <tr>
    <td rowspan="3">MEC</td>
    <td>ReLM</td>
    <td>67.3</td>
    <td>44.9</td>
    <td>53.9</td>
    <td>5.8</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>82.2</b></td>
    <td>44.5</td>
    <td><b>57.7</b></td>
    <td><b>2.2</b></td>
  </tr>
  <tr>
    <td>+ DISC*</td>
    <td>76.3</td>
    <td><b>45.0</b></td>
    <td>56.6</td>
    <td>3.2</td>
  </tr>
</table>

Table 10: Sentence-level performance of LLMs, ReLM, and ReLM + DISC on the test sets of ECSpell and LEMON. Results marked with “*” indicate the use of copy-punishment solution.

original character during inference. Specifically, after incorporating the DISC module, we additionally lower the prediction probability of the original character by 0.1 to reduce the model’s tendency to select the original character during inference, thereby improving the model’s recall rate. The experimental results can be found in Table 10.

D Prompt Example

In this work, we use the prompt-based method to activate the CSC ability of the GPT3.5 and GPT4. The prompt is shown in Figure 4.

System and User Prompts for LLMs

System Prompt:
你是一个优秀的中文纠错模型，中文纠错模型模型即更正用户输入句子中的错误。

User Prompt:
请需要识别并纠正用户输入的句子中可能的错别字并输出正确的句子。纠正时必须保证改动前后句子等长。在纠正错别字的同时尽可能减少对原句子的改动(不添加额外标点符号，不添加额外的字，不删除多余空格)，只输出没有错别字的句子。不要添加任何其他解释或说明。如果句子没有错别字，就直接输出和输入相同的句子。

Figure 4: Prompt template used in GPT3.5 and GPT4.

<table>
  <tr>
    <th rowspan="2">Domain</th>
    <th rowspan="2">Model</th>
    <th colspan="3">Correction</th>
    <th rowspan="2">FPR</th>
  </tr>
  <tr>
    <th>P</th>
    <th>R</th>
    <th>F<sub>1</sub></th>
  </tr>
  <tr>
    <td colspan="6">ECSpell</td>
  </tr>
  <tr>
    <td rowspan="3">LAW</td>
    <td>GPT3.5</td>
    <td>48.5</td>
    <td>43.1</td>
    <td>45.6</td>
    <td>9.4</td>
  </tr>
  <tr>
    <td>GPT4</td>
    <td>62.0</td>
    <td>62.0</td>
    <td>62.0</td>
    <td>7.3</td>
  </tr>
  <tr>
    <td>ReLM</td>
    <td>66.1</td>
    <td><b>71.0</b></td>
    <td>70.0</td>
    <td>8.6</td>
  </tr>
  <tr>
    <td rowspan="3">MED</td>
    <td>+ DISC</td>
    <td><b>73.7</b></td>
    <td>70.2</td>
    <td><b>71.9</b></td>
    <td><b>5.7</b></td>
  </tr>
  <tr>
    <td>GPT3.5</td>
    <td>36.5</td>
    <td>42.0</td>
    <td>39.1</td>
    <td>20.1</td>
  </tr>
  <tr>
    <td>GPT4</td>
    <td>45.1</td>
    <td>57.5</td>
    <td>50.6</td>
    <td>24.8</td>
  </tr>
  <tr>
    <td rowspan="3">ODW</td>
    <td>ReLM</td>
    <td>67.4</td>
    <td>70.2</td>
    <td>68.8</td>
    <td>7.0</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>78.2</b></td>
    <td><b>71.6</b></td>
    <td><b>74.8</b></td>
    <td><b>4.6</b></td>
  </tr>
  <tr>
    <td>GPT3.5</td>
    <td>57.3</td>
    <td>52.3</td>
    <td>54.7</td>
    <td>6.3</td>
  </tr>
  <tr>
    <td rowspan="3"></td>
    <td>GPT4</td>
    <td>71.7</td>
    <td>67.6</td>
    <td>69.5</td>
    <td><b>1.7</b></td>
  </tr>
  <tr>
    <td>ReLM</td>
    <td>78.2</td>
    <td>76.4</td>
    <td>77.3</td>
    <td>4.1</td>
  </tr>
  <tr>
    <td>+ DISC</td>
    <td><b>81.8</b></td>
    <td><b>76.7</b></td>
    <td><b>79.2</b></td>
    <td>2.5</td>
  </tr>
</table>

Table 11: Sentence-level performance of LLMs, ReLM, and ReLM + DISC on the test sets of cleaned ECSpell.

E Experiments on Cleaned ECSpell

We discovered a serious data leakage issue in ECSpell. Specifically, the same correct sentence may appear multiple times in both the training and test sets, with only the location and type of errors varying. These sentences respectively account for 52.7%, 19.3%, and 28.2% of the ECSpell-LAW/MED/ODW training sets. We cleaned these duplicate sentences and reorganized the experiments, as shown in Table 11.

The experimental results show that, compared to using the uncleaned training sets, ReLM’s performance on ECSpell significantly decreased, but it still outperforms GPT4. DISC also achieves stable performance improvement, with an impressive 6.0 F1 value increase on ECSpell-MED.

F Detailed Results

In addition to the correction-level performance, we also present the detection-level experimental results of the CSC models, as shown in Table 12. SCOPE + DR-CSC performs well at the detection level, primarily because they incorporate an additional detection network.

Since SIGHANs contain a lot of noise, we also conduct experiments on their revised versions (referred to as SIGHANs (rev.)) released by Yang et al. (2023b), which have undergone manual verification and error correction to ensure higher data quality. As shown in Table 13 and Table 14, our DISC module also achieves consistent performance improvements on SIGHANs (rev.).
<table>
  <tr>
    <th rowspan="2">Models</th>
    <th colspan="3">SIGHAN15</th>
    <th colspan="3">SIGHAN14</th>
    <th colspan="3">SIGHAN13</th>
  </tr>
  <tr>
    <th>D-P†</th><th>D-R†</th><th>D-F†</th>
    <th>D-P†</th><th>D-R†</th><th>D-F†</th>
    <th>D-P†</th><th>D-R†</th><th>D-F†</th>
  </tr>
  <tr>
    <td colspan="9">Previous SOTAs</td>
  </tr>
  <tr>
    <td>SpellGCN</td><td>74.8</td><td>80.7</td><td>77.7</td><td>65.1</td><td>69.5</td><td>67.2</td><td>80.1</td><td>74.4</td><td>77.2</td>
  </tr>
  <tr>
    <td>ReaLiSe</td><td>77.3</td><td>81.3</td><td>79.3</td><td>67.8</td><td>71.5</td><td>69.6</td><td>88.6</td><td>82.5</td><td>85.4</td>
  </tr>
  <tr>
    <td>SCOPE†</td><td>80.5</td><td>85.4</td><td>82.9</td><td>68.8</td><td>73.7</td><td>71.1</td><td>87.5</td><td>83.0</td><td>85.2</td>
  </tr>
  <tr>
    <td>SCOPE + DR-CSC</td><td><b>82.9</b></td><td>84.8</td><td><b>83.8</b></td><td><b>70.2</b></td><td>73.3</td><td>71.7</td><td>88.5</td><td>83.7</td><td>86.0</td>
  </tr>
  <tr>
    <td>ReLM†</td><td>78.3</td><td><b>85.6</b></td><td>81.8</td><td>65.7</td><td>74.5</td><td>69.8</td><td>86.4</td><td>83.7</td><td>85.0</td>
  </tr>
  <tr>
    <td colspan="9">LLMs Results</td>
  </tr>
  <tr>
    <td>GPT3.5</td><td>39.4</td><td>46.4</td><td>42.6</td><td>41.4</td><td>23.1</td><td>29.6</td><td>61.6</td><td>29.2</td><td>39.7</td>
  </tr>
  <tr>
    <td>GPT4</td><td>42.7</td><td>57.5</td><td>49.0</td><td>38.1</td><td>52.3</td><td>44.1</td><td>53.4</td><td>51.6</td><td>52.5</td>
  </tr>
  <tr>
    <td colspan="9">Ours</td>
  </tr>
  <tr>
    <td>ReaLiSe + DISC</td><td>78.3</td><td>81.2</td><td>79.7†</td><td>69.2</td><td>71.2</td><td>70.1†</td><td>88.9</td><td>82.2</td><td>85.4</td>
  </tr>
  <tr>
    <td>SCOPE + DISC</td><td>81.7</td><td>84.8</td><td>83.2†</td><td><b>70.2</b></td><td>73.5</td><td>71.8†</td><td>88.8</td><td>83.7</td><td>86.2†</td>
  </tr>
  <tr>
    <td>ReLM + DISC</td><td>80.8</td><td>84.3</td><td>82.5†</td><td>69.7</td><td><b>74.9</b></td><td><b>72.2</b>†</td><td><b>89.7</b></td><td><b>84.5</b></td><td><b>87.0</b>†</td>
  </tr>
</table>

Table 12: Sentence-level performance on the SIGHAN13, SIGHAN14 and SIGHAN15 test sets. Precision (P), recall (R) and F_1 for detection are reported (%). Results marked with “†” are obtained by reruning the official code released by Li et al. (2022) and Liu et al. (2024). Other baseline results are directly taken from their literature. Apart from SpellGCN, all models apply post-processing on SIGHAN13, which removes all detected and corrected “地” and “得” from the model output before evaluation. “+ DISC” means adding DISC module in the decoder. α and β are assigned the values 1.1 and 0.7, respectively.

<table>
  <tr>
    <th rowspan="2">Models</th>
    <th colspan="4">SIGHAN15 (rev.)</th>
    <th colspan="4">SIGHAN14 (rev.)</th>
    <th colspan="4">SIGHAN13 (rev.)</th>
  </tr>
  <tr>
    <th>C-P†</th><th>C-R†</th><th>C-F†</th><th>FPR†</th>
    <th>C-P†</th><th>C-R†</th><th>C-F†</th><th>FPR†</th>
    <th>C-P†</th><th>C-R†</th><th>C-F†</th><th>FPR†</th>
  </tr>
  <tr>
    <td colspan="12">Previous SOTAs</td>
  </tr>
  <tr>
    <td>BERT*</td><td>73.2</td><td>67.5</td><td>70.2</td><td>–</td><td>62.6</td><td>57.5</td><td>59.9</td><td>–</td><td>71.1</td><td>67.4</td><td>69.2</td><td>–</td>
  </tr>
  <tr>
    <td>ReaLiSe*</td><td>74.4</td><td>69.6</td><td>71.9</td><td>–</td><td>63.6</td><td>59.0</td><td>61.2</td><td>–</td><td>71.9</td><td>68.0</td><td>69.9</td><td>–</td>
  </tr>
  <tr>
    <td><a href="https://arxiv.org/abs/2303.16490">Yang et al. (2023b)</a></td><td>77.0</td><td>67.6</td><td>72.0</td><td>–</td><td>66.0</td><td>57.1</td><td>61.3</td><td>–</td><td>73.2</td><td>67.1</td><td>70.0</td><td>–</td>
  </tr>
  <tr>
    <td>ReLM</td><td>76.4</td><td><b>73.5</b></td><td>74.9</td><td>8.5</td><td>65.5</td><td>62.9</td><td>64.2</td><td>11.3</td><td>74.0</td><td>70.9</td><td>72.4</td><td>10.7</td>
  </tr>
  <tr>
    <td colspan="12">Ours</td>
  </tr>
  <tr>
    <td>ReLM + DISC</td><td><b>79.0</b></td><td>73.0</td><td><b>75.9</b></td><td><b>6.4</b></td><td><b>69.3</b></td><td><b>63.4</b></td><td><b>66.2</b></td><td><b>8.9</b></td><td><b>75.4</b></td><td><b>71.3</b></td><td><b>73.3</b></td><td><b>9.4</b></td>
  </tr>
</table>

Table 13: Sentence-level performance on the revised SIGHAN13-15 test sets. Precision (P), recall (R) and F_1 for correction are reported (%). “*” means that the results of BERT and ReaLiSe in the table are directly copied from Yang et al. (2023b).

<table>
  <tr>
    <th rowspan="2">Models</th>
    <th colspan="3">SIGHAN15 (rev.)</th>
    <th colspan="3">SIGHAN14 (rev.)</th>
    <th colspan="3">SIGHAN13 (rev.)</th>
  </tr>
  <tr>
    <th>D-P†</th><th>D-R†</th><th>D-F†</th>
    <th>D-P†</th><th>D-R†</th><th>D-F†</th>
    <th>D-P†</th><th>D-R†</th><th>D-F†</th>
  </tr>
  <tr>
    <td colspan="9">Previous SOTAs</td>
  </tr>
  <tr>
    <td>BERT*</td><td>75.4</td><td>70.0</td><td>72.4</td><td>64.6</td><td>59.3</td><td>61.8</td><td>72.6</td><td>68.8</td><td>70.6</td>
  </tr>
  <tr>
    <td>ReaLiSe*</td><td>75.8</td><td>70.9</td><td>73.2</td><td>65.6</td><td>60.8</td><td>63.1</td><td>74.9</td><td>70.7</td><td>72.7</td>
  </tr>
  <tr>
    <td><a href="https://arxiv.org/abs/2303.16490">Yang et al. (2023b)</a></td><td>77.7</td><td>68.3</td><td>72.7</td><td>67.2</td><td>58.1</td><td>62.3</td><td>74.4</td><td>68.3</td><td>71.2</td>
  </tr>
  <tr>
    <td>ReLM</td><td>78.8</td><td><b>75.7</b></td><td>77.2</td><td>68.4</td><td><b>65.7</b></td><td>67.0</td><td>76.0</td><td><b>72.8</b></td><td><b>74.4</b></td>
  </tr>
  <tr>
    <td colspan="9">Ours</td>
  </tr>
  <tr>
    <td>ReLM + DISC</td><td><b>80.6</b></td><td>74.4</td><td><b>77.4</b></td><td><b>71.2</b></td><td>65.2</td><td><b>68.1</b></td><td><b>76.4</b></td><td>72.3</td><td>74.3</td>
  </tr>
</table>

Table 14: Sentence-level performance on the revised SIGHAN13-15 test sets. Precision (P), recall (R) and F_1 for detection are reported (%). “*” means that the results of BERT and ReaLiSe in the table are directly copied from Yang et al. (2023b).

Reviewer paper summaries: This paper presents a pluggable module that adjusts the model's probability distribution over the vocabulary by measuring phonetic and glyph similarities between characters, incorporating this similarity information in the decoding stage. This model-agnostic approach can be integrated into most existing CSC models. Experimental results across five datasets have demonstrated its superiority over previous approaches.

This paper presents DISC, a novel plug-and-play decoding intervention module for Chinese Spelling Check that utilizes phonetic and glyph similarities between characters to improve error correction accuracy. DISC operates by integrating character similarity information into the inference phase of CSC models, enhancing their performance without requiring additional training. Experiments on multiple CSC benchmarks, including SIGHAN, ECSpell, and LEMON datasets, demonstrate that DISC significantly boosts the performance of CSC models.
Summary of strengths: 1. This paper proposes leveraging character similarity information to guide the decoding process during the inference phase. A key contribution of this work is the adaptability of the proposed module, which can be easily integrated into most existing CSC models.
2. The effectiveness of the proposed strategy has been validated by extensive experimental results. In the revised version, authors have supplement some necessary explanations to the results.

1. Training-Free: DISC does not require additional training to enhance existing CSC models. Instead, it optimizes error correction by integrating phonetic and glyph similarity information during the inference phase, making it highly efficient and practical for real-world applications.
2. Flexibility and Plug-and-Play: DISC offers flexibility as a plug-and-play solution, easily integrating with existing CSC models like ReaLiSe, SCOPE, and ReLM, without needing modifications or extra training.
Summary of weaknesses: 1. Although the author claims in their response that this work is the first to introduce character similarity knowledge into the decoding process without requiring modifications to the model architecture or re-training, the concept of incorporating character similarity has already been explored in constraint-based decoding. Therefore, I still believe the novelty of this work is quite limited.

2. While large language models (LLMs) can effectively address CSC tasks, I question the practical value of this work in real-world scenarios.

1. The DISC method relies on pre-computed character similarity scores during the inference phase, meaning that once the similarity between two characters is determined, these scores remain fixed throughout the decoding process. This fixed scores may limit the flexibility of the model in certain complex scenarios. For instance, when handling unseen samples or new domains where character relationships might shift, DISC cannot dynamically adjust these scores.
2. Although the DISC method improves spelling correction accuracy by incorporating character similarity, its conservative prediction strategy may lead to under-correction issues, especially in domain-specific data with insufficient domain-specific training data. Since the model heavily relies on similarity scores, it may retain incorrect characters as the original ones, which negatively impacts recall and leads to instability in recall performance. This conservatism limits the stability of the model's performance on domain-specific data, particularly when handling complex or unseen new data.
Comments, suggestions and typos: 1. Does the proposed constraint-based decoding approach lead to an increase in inference time?

2. In Table 3, why does GPT-4 perform significantly worse than ReLM on ECSpell? Given that ReLM has already achieved very high performance (93.7, 98.8, 96.2), could you elaborate on the specific scenarios where DISC can provide additional benefits?

I would be interested in understanding what the result scores would be if the pre-trained model used in DISC were replaced with a language model such as BERT,T5 or LLM, instead of a correction model specifically trained for error correction. Would the performance still hold, or would the model’s error correction ability change significantly?
Overall assessment scores: 3.5

3.5
Metareview score: 4
Metareview: This paper presents a plug-and-play decoding intervention method that employs phonetic and glyph similarities to improve Chinese spelling correction (CSC). The proposed method can be integrated into existing CSC models without additional training. Experiments on multiple benchmarks demonstrate the superiority of the proposed method.
Summary of reasons to publish: - the method is novel and interesting
- comprehensive experiments
- strong empirical results