A Frustratingly Easy Plug-and-Play Detection-and-Reasoning Module for Chinese Spelling Check
Authors: Haojing Huang
Jingheng Ye
Qingyu Zhou
Yinghui Li
Yangning Li
Feng Zhou
Hai-Tao Zheng
Venue: EMNLP 2023 Findings (2023)
Keywords: natural language processing
chinese spelling check

Abstract: In recent years, Chinese Spelling Check (CSC) has been greatly improved by designing task-specific pre-training methods or introducing auxiliary tasks, which mostly solve this task in an end-to-end fashion. In this paper, we propose to decompose the CSC workflow into detection, reasoning, and searching subtasks so that the rich external knowledge about the Chinese language can be leveraged more directly and efficiently. Specifically, we design a plug-and-play detection-and-reasoning module that is compatible with existing SOTA non-autoregressive CSC models to further boost their performance. We find that the detection-and-reasoning module trained for one model can also benefit other models. We also study the primary interpretability provided by the task decomposition. Extensive experiments and detailed analyses demonstrate the effectiveness and competitiveness of the proposed module.

A Frustratingly Easy Plug-and-Play Detection-and-Reasonng Module for Chinese Spelling Check
Haojing Huang': Jingheng Ye',, Qingyu Zhou"! Yinghui Li
Yangning Li Feng Zhou?, Hai-Tar Zhengl 13F
Tinghuu Shenzhen International Graduate School, Tsinghua University 2PPO Research Institute, PPeng Cheng Laboratory (hhj23, yejh22, liyinghu20Jemails tsinghu.. edu. cn tyzhgm@gmail.com com, zheng. hai taoêsz. tsinghua. edu. cn
Abstract
Phnnsloiicl
In recent years, Chinese Spelling Check (CSC) has been greatly improved by designing task- specific pre-training methods introducing auxiliary tasks, which mostly solve this task an end-to-end fashion. In this paper, we propose decompose the CSC workflow into detection, reasoning, and searching subtasks so that the rich external knowledge about Chinese language be leveraged more di- rectly and efficiently. Specifically, we design a plug-and-playy ule that is compatible with existing SOTA non- autoregressive CSC models to further boost their performance. We find that the detection- and-reasoning module trained for one model can also benefit other models. We also study the primary interpretability provided by the task decomposition. Extensive experiments and detailed analyses demonstrate the effectiveness and competitiveness of the proposed module.
AL dk
% siguificacce
The achocvemnnt different octlinary thiiiing
mod-
di
$
Everyuse look and servers thising differently
Figure I: Examples of the CSC task. The first example a phonological error, and the second a morphologi- cal error. We mark the wrong character in red, the right character in blue, and also the error type of the character in green.
1 Introduction
Recently, with the rapid development of Pre- trained Language Models (PLMs) (Dong al., 2023; Li al., 2023c; Ma al., 2023), methods based on PLMs have gradually become the main- stream CSC research (Li and Shi, 2021; Ji et al., 2021; Wang et al., 2021; Li et al., 2022f; Cheng et al., 2023a), most of which are non-autoregressive models making predictions for each input Chinese character. Previous works be roughly divided into two categories: (1) Design tailored pre-training objectives to guide the model to learn the deep se- mantic information (Liu et al., 2021; Zhang et al., 2021 b; Li, 2022: Li al., 2023a); (2) Perform dif- ferent fine-tuning processes like adding auxiliary tasks to fine-tune the models (Zhu et al., 2022; Liu et al., 2022b).
Spelling Check aims to detect and correct spelling errors contained in the text (Wu et al., 20)3a). It benefits various applications, such as search en- gine (Gao et al., 2010; Martins and Silva, 2004), and educational scenarios (Afli al., 2016; Dong and Zhang, 2016; Huang et al., 2016, 2018: Zhang et al., 2022b). Particularly, in this paper, we con- sider spelling check for the Chinese language, i.e., Chinese Spelling Check (CSC), because of the chal- lenge brought by the peculiarities of Chinese to the task (Li et al., 2021b). Suffering from many confus- ing characters, Chinese spelling errors mainly caused by phonologically and visually similar char- acters (Liu al., 2010). As shown in Figure (shi"" is the visual error character of "dai"" because they have almost the same strokes.
Despite the remarkable success achieved by previous works, existing models seem to have some shortcomings incorporating external ex- pert knowledge, such as the confusion set, which contains a set of similar character pairs (Liu et al.,
indicates equal contribution.
Corresponding authors: Hai-Ta Tao Zheng, Qingyu Zhou. 'The source codes are available at https: http::///itttt thub. com/ THUKElab/DR-

---

2010; Wang al., 2019). Cheng al. (2020) makes an attempt that replaces the classifier's weight according to the confusion set using graph neural networks. et al. (2022e) further proposes a neural model that learns knowledge from dictio- nary definitions. However, these approaches im- plicitly learn external knowledge, thus lacking the desired level of interpretability and efficiency, We argue that the CSC task can be decomposed into three subtasks, including Detection, Reasoning, and Searching, aiming to incorporate the external knowledge more naturally and provide more inter- pretability. These subtasks answer the following questions respectively, "Which character is mis- spelled?", "Why did this error occur?" and "How to correct it?". By addressing these questions, the model gains access enror position, misspelling type, and external knowledge from three subtasks, respectively. As a result, deeper understanding of the underlying processes involved in correcting misspelled characters can be achieved.
performance of CSC models.
In summary. the contributions of this work are in three folds:
We design the DR-CSC module, which guides the model to correct Chinese spelling errors by incorporating the confusion set informa- tion. And is also compatible with non- autoregressive CSC models.
We enhance the interpretability of CSC mod- els by explicitly incorporating external knowl- edge through the decomposition of the CSC task into three subtasks.
We conduct extensive experiments on widely used public datasets and achieve state-of art performance. Detailed analyses show the effectiveness of the proposed method.
Related Work
CSC is fundamental language processing task, and it is also important subtask of Chinese Text Correction (Ma et al., 2022; Li et al., 2023b; Ye et al., 2023, 2022: Zhang al., 2023), Around the pre-training and fine-tuning of PLMS (Liu et al., 2022a; Li et al.. 2022d,c; Cheng et al., 2023b; Zhang et al., 2022a; Cheng et al., 2023c), re- searchers have done many efforts to improve the performance of CSC models:
To this end, we propose a frustratingly easy auu--nd--lay Detection-and -ee---nnnd-R--oiii module for Chinese Spelling Check (DR-CSC) to incor- porate external expert knowledge naturally and ef- fectively into existing non-autoreeressive (Cheng et al., 2023d) CSC models. The proposed method is designed to be compatible with wide range of non- autoregressive CSC models, such as Soft-Masked BERT (Zhang et al., 2020), MacBERT (Cui al., 2020), and SCOPE (Li et al., 2022b). We decom- pose the CSC task into three subtasks from easy to hard: (1) Detection: We model the process of detecting misspellings from a sentence through the sequence labeling task, that is, predicting for each character in the sentence whether it is correct or wrong. (2) Reasoning: For the wrong characters predicted in the detection step, sentations of CSC models to further enable the model to consider the attribution of spelling er- rors by classifying them to know whether they are errors caused by phonetics or vision. (3) Search- ing: After determining the spelling error type, we search within its corresponding phonological or vi- sual confusion set to determine the correct charac- ter the correction result. The proposed DR-CSC module performs multi-task learning on these three subtasks. By doing so, naturally incorporates the confusion set information, and effectively nar- rows the search space of candidate characters. As a result, this module greatly helps to improve the
CSC-targeted Pre-training Tasks Researchers design different pre-training strategies to obtain PLMs that are more suitable for CSC. PLOME (Liu et al., 2021) proposes to pre-train models with their designed particular character masking strategy guided by the confusion set and apply task-specific pre-training to enhance the CSC models. Spell- BERT (Ji et al., 2021). DCN (Wang et al., 2021), and MLM MLM- phonetics (Zhang al., 2021a) leverage the phonetic information to improve the adaptation of the pre-training process to the CSC task. For more recent research, WSpeller (Li al., 2022a) trains a word- masked language model to utilize the segment-level information which provides appro- priate word boundaries.
the repre-
CSC-targeted Fine-Tuning Processes Many studies focus the fine-tuning stage of PLMs to obtain various additional knowledge to improve model performance. REALISE (Xu et al., 2021) and PHMOSpell (Huang et al., 2021) focus on the positive impact of multimodal knowledge on

---

1 A K 59 R.
& &
The adtorrmennt detwer foon codanay sigriliianoe
Searching
Plug-and-Play Process
Search Matri
oodnayy diqullicance
Reasoning
Search Matrix
Detection
Encoder
E
The aetievement diflerettion
Encoder
The octievement ditienet ordtaay thikkig
Overview of the proposed DR-CSC. The Detection task detect the character. The Reasoning task is to determine the error type of the typo. The Searching task is to correct the sentence in the search matrix. The right part shows how to plug and play DR-CSC module from models to enhance another model.
Figure 2:
the CSC models. ECOPO (Li et al., 2022f) and EGCM (Sun et al., 2022) propose error-driven methods guide models to avoid over-correction and make predictions that are most suitable for the CSC scenario. Besides, LEAD (Li al., 2022e) is a heterogeneous fine-tuning framework, which for the first time introduces definition knowledge to enhance the CSC model. SCOPE (Li et al., 2022b) proposes the auxiliary task of Chinese pronuncia- tion prediction to achieve better CSC performance. MDCSpell (Zhu et al., 2022) proposes to fuse the detection network output into the correction net- work to reduce the impact of typos.
3.1 Detection
Given the input sentence X (21,422 27 of length T, we utilize the PLMs or CSC models encoder E to get the representations:
= E(X) = {h11, h2 .... hT),
(1)
where h4 E Riidden hidden is the hidden state size of E. The detection task is to detect the wrong characters from the sentence, that is, to predict whether each character in the sentence is correct or wrong. Therefore, compute the binary clas- sification probability of the i-th character in the sentence X as:
Although PLMs have achieved great success in the CSC task, previous studies still lack degree of interpretability. We argue that the CSC task be decomposed into three subtasks introduce ex- ternal knowledge naturally and provide more pos- sibility interpretability. The proposed module compatible with existing non-autoregressivee CSC models to get enhancement.
pe softmax((Wn)) + bp), p. RR2. (2)
where WD 2R2xhidden and bp E R²2 are learnable parameters. Note that pf is probability vector with 2-dimension, based which obtain the detection predicted result:
arg max(p;))
(3)
where yf belongs to (0, 1}, v 0 means that the character is correct, and y/ means that the character is wrong.
3 Methodology
As shown in Figure 2, DR-CSC consists of en- coder (i.e.. E) and three progressive subtasks. DR- CSC is module that can be combined with various cxisting non-autoregressive CSC models or proper PLMs.
For the training of the detection task, we use the cross-entropy as its learning objective:
T
La
Σ
9 log p.d
(4)

---

where of is the training label of the i-th charac- ter in the detection task. is worth mentioning that during the training process, of can be obtained by comparing the original character with its corre- sponding golden character.
are morphological errors. This is because large number of characters exhibit errors in both phono- logical and morphological aspects simultaneously, so is reasonable for us to prioritize this kind of wrong character as "phonological error". During the construction of the confusion set, we ensured that the corresponding characters were included. This approach helps alleviate errors caused by the misidentification of non-erroneous characters rors. Furthermore, the sets of phonetically similar characters and visually similar characters are not mutually exclusive; they exhibit a certain degree of intersection. This characteristic further mitigates the impact of error accumulation.
3.2 Reasoning
After the detection task, we know which characters in the sentence are misspellings. then the reasoning task is to predict whether the wrong character is caused by phoneties vision. It is not difficult know that this is also actually binary classifi- cation process for characters. Therefore, similar to the detection task, we calculate the reasoning prediction probability the i-th character and get the reasoning predicted result as follows:
3.3 Searching
The searching task is to find the correct answers to the spelling errors. Specifically, for each character in X, predict the probability that it should be character in the vocabulary of the PLMs:
pi. softmax SantmaxxWiilhht (Wih; bR), TR?
(5)
v = arg max(p.),
(6)
where Wi E 2xhidden and by E IR? are learnable parameters. When ofp = 1, it means this wrong character is a phonological error. And yf = 0 represents the morphological error. We also utilize the cross-entropy loss to train the reasoning task:
p = Doftmax(Wssh bs), of c I vocob
(8)
p = (pi. pr) P R Txeocab
where Ws E R procabxhidden and by E Rvoeab are trainable parameters, wocab is size of PLMs' vo- cabulary.
T Σ o log
=
(7)
The main innovation of our designed searching task is that utilize the phonologica//visual con- fusion set to enhance the prediction of error cor- rection results. Thanks to the error position and error type information obtained in the detection and reasoning tasks, we can construct a more refined search matrix based on the phonological/viin vivsslliclal/vuuullllalaa con- fusion sets to reduce the search space:
Note that we use 1/5 Equation (3) to ensure that only the wrong characters detected by the detec- tion task participate in the calculation of C,. The reasoning label 9f" of a character is obtained by considering its golden character. If the golden char- acter is in the phonological confusion set of the original character, its label is "phonological error". If the case is visual confusion set, the label is "mor- phological error". Specifically,, we construct the phonological/visual confusion set of a character by comparing the Pinyin/strokes 2 between charac- ters. The phonological confusion set of a character contains all characters similar to its pinyin, and the visual confusion set is all characters similar to its strokes. We also utilize the confusion set pro- vided in SIGHAN133. It contains Similar Pronun- ciation confusion set and Similar Shape confusion set. Previous research (Liu al., 2010) indicates that 83% of errors are phonological errors and 48% enchar toolkit
pe[us]. V = 1 & v =
C
vel [es y = yi =0= O, i Frocab otherwise
(10)
C C2,,
(11)
G E geocab
C I xuveab
where pi and vic are vectors whose dimensions are vocab and whose elements are 0 or 1. And pč[z.] means that the elements the corresponding posi- tion of 2, phonetically similar characters in the vector are set to 1, and the others are The ele- ments of oe[2;] are also set based on the visually similar characters of x;-
We utilize (https://github... http:////giuuu.ooo//eeeeee com/theajack//nehar to get Pinyin/strokes information of Chinese characters.
"http://i.itc.nmmma mitut..ttucdduu ht//tr//iihhn7cscc..ho

---

We augment the correction probabilities based on C:
4
Experiments
4.1 Experimental Setup
P= P C (P.,P,,...,T).
(12)
4.1.1
Datasets
where O means the multiplication of elements in the corresponding positions of the two matrices. Through this operation, for a character that has been determined to be a phonological error, the probability that it is predicted to be character that is not similar to its phonetic will be set to O. The same true for the visually similar case. Therefore, we enhance the probability representation of more suitable candidate characters through the search matrix in the searching task and also narrow the search space of candidate characters. We also use the cross-entropy loss in the searching task:
To ensure fairness, we use the same training data as our baselines, i.e., SIGHANI3 (Wu et al., 2013b), SIGHANI4 (Yu et al., 2014), SIGHAN15 (Tseng et al., 2015) and Wang27IK K (Wang et al., 2018 8). The test data is also widely used in previous CSC task (Wang et al., 2019; Cheng et al., 2020; Li al., 2022a), ie, SIGHANI3//4/15 test datasets.
4.1.2 Baselines Methods
To evaluate the performance of DR-CSC, lect some advanced and strong CSC models our baselines: SoftMakked-BERT (Zhang et al., 2020) is consist of Detection Network and Correction Network. MacBERT (Cui et al., 2020) is an en- hanced BERT with novel MLM as the correction pre-training task. Two-Ways (Li et al., 2021a) utilizes the weak spots of the model to generate pseudo-training data. MLM-pponetiiss (Zhang et al., 2021a) is enhanced ERNIE (Sun et al., 2020) which contains additional phonetic infor- mation. REALISE (Xu al., 2021) is a multi- modal CSC model which leverages semantic, pho- netic, and graphic knowledge. LEAD (Li et al., 2022e) learns heterogeneous knowledge from the dictionary, especially the knowledge of definition. SCOPE (Li et al., 2022b) improve the CSC per- formance with the help of an auxiliary Chinese pronunciation prediction task. is the previous state-of-the-arr method for the SIGHAN datasets.
T
C. - Σ
9i log Ps-
(13)
Note that the label 9i is equivalent to the overall golden label, i.e., the correct character correspond- ing to the wrong character.
Finally, benefiting from multi-task learning, we tackle the three progressive tasks at once in our DR-CSC framework:
== fa B- for y- frs;
(14)
where a, B, and y are hyper-parameter
3.4 Plug-and-Play Process
As depicted in Figure the addg-and integra- tion of the trained encoder E and detection-and- reasoning module (D-R Module) with a new CSC model or PLM E' can be achieved without retrain- ing of the D-R Module. This is accomplished by feeding the text to both encoders, E and E':
4.1.3 Evaluation Metrics
Because CSC aims detect and correct spelling errors, the CSC studies all report the detection and correction performance. For the correction level, the model must not only detect but also correct all wrong characters. More specifically, we report the sentence-level metrics, including Precision, Recall, and F1 score, which are commonly used in previous works (Xu al., 2021; Li et al., 2022b). Note that sentence-level is more challenging than character- level because some sentences may contain multiple wrong characters.
H = E(X), H' = EX)..
(15)
H can then be utilized as input for the D-R Module, yielding the results of detection and reasoning sub- tasks for constructing search matrix C according Equations (1))),, H' can be fed into the output layer to obtain the correction probabili- ties P' of the new CSC model or PLM:
C D-R M-dule (H),
(16)
4.1.4 Implementation Details
In the experiments, we use Pytorch imple- ment the proposed DR-CSC. The implementations of SoftMasked BERT + DR-CSC, MacBERT
P' = Output Layor(H').
(17)
Subsequently, we can augment the correction prob- abilities P', by referring to Equation (12).
DR-CSC and SCOPE + DR-CSC are following

---

Dataset
Model
Delection
Correction
MLM-phonetics (Zhang al., 2021a)
82.0
78.3
80.1
79.5
77.0
78.2
REALISE (Xu al., 2021)
88.6
82.5
85.4
87.2
81.2
84.1
Two-Ways al., 202
84.9
84.4
LEAD (Li 2022e)
88.3
83.4
85.8
87.2
82.4
84.7
SIGHANI3
SoftMasked-BERt (Zhang al., 2020)
81.1
75.7
78.3
75.1
70.1
72.5
SoftMasked-BFFi DR-CSC
80.2
77.3"
78.77
77.9"
75. 17
76.5"
MacBERT (Cui et al., 2020)*
84.6
79.9
82,2
81.0
76.5
78.7
MacBERT DR DR-CSC
87.81
81.67
84.67
86.77
80.5
83,57
SCOPE (Li et al., 2022h)
87.4
83.4
85.4
86.3
82.4
84.3
SCOPE DR-CSC
88.51
83.77
86.07
87.71
83.0
85.3
REALISE (Xu et al., 2021)
67.8
71.5
69.6
66.3
70.0
68.1
Two-Ways (Li et al., 2021a)
70.4
68.6
MLM-phooetics<Zhang et al., 2021a)
66.2
73.8
69.8
64.2
73.8
68.7
LEAD (Li ct al., 2022c)
70.7
71.0
70.8
69.3
69.6
69.5
SIGHANI4
SoftMasked-BBRR (Zhang et al., 2020)²
61.4
67.5
64.3
59.8
65.8
62.6
SoftMasked-BBi BERT DR-CSC
61.4
67.9
64.5
60.3"
66.77
63.4
MacBERT (Cui al., 2020)*
63.9
65.6
64.7
61.0
62.7
61.9
MacBERT DR-CSC
65.67
68.87
67.2
64.8"
68.1
66.4
SCOPE (Li al., 2022h)
70.1
73.1
71.6
68.6
71.5
70.1
SCOPE DR-CSC
70.2
73.37
71.71
69.31
72.31
70.7
MLM-ppooetics/Zhangg et al., 2021a)
77.5
83.1
80.2
74.9
80.2
77.5
REALISE (Xu 2021)
77.3
81.3
79.3
75.9
79.9
77.8
Two-Ways (Li al., 2021a)
80.0
78.2
LEAD 2022c)
79.2
82.8
80.9
77.6
81.2
79.3
SoftMaskedd (Zhang al., 2020)
73.7
73.2
73.5
66.7
66.2
66.4
SIGHANI5
SoftMasked-BERT DR-CSC
74.0
78.87
76.1"
71.6"
76.21
73.9
MacBERT (Cui al., 2020)
71.9
77.9
74.8
68.0
73.6
70.7
MacBERT DR-CSC
75.81
78.37
77.0"
73.6"
76.1"
74.8
SCOPE (Li al., 2022h)
81.1
84.3
82.7
79.2
82.3
80.7
SCOPE DR-CSC
82.9
84.81
83.8
80.31
82.3
81.3
d d/r gt
89.8
85.7
87.7
90.2
95.2
87.0
97.9
89.4
Table I: The performance of DR-CSC and all baselines. + DR-CSC means that we combine DR-CSC with model X. "-with d gt" means with detection ground truth during the inference stage and "with d/r gt" means with detection and reasoning ground truth during the inference stage. t means improvement compared to the baseline model. Results marked with "4" are obtained by running released codes from corresponding papers.
these three github repositories In our exper- iments, initialize the weights of SoftMasked- BERT DR-CSC using the weights of Chinese BERT-wwm (Cui al., 2021) and initialize the weights of MacBERT + DR-CSC using the weights of MacBERT (Cui al., 2020). And the initial weights of SCOPE + DR-CSC are from the Further Pretrained Model proposed by SCOPE (Li et al., 2022b). We set the maximum length of the sentence 192 which contain all train- ing samples' length. We train the model with
Adam W optimizer and set the learning rate 10 5 We set the a, B, y all to 1. The opti- mal models SIGHAN13/14/1 training with batch sizes 96/96/64 for 20/30/30 epochs, respectively. The best-performing mod- els on SIGHAN13/14/15 were trained using batch sizes of 96/96/64, respectively, for 20/30/30 epochs each. All experiments are conducted on one GeForce RTX 3090.
obtained by
4.2 Experimental Results
From Table 1, we can observe that through the optimization the module DR-CSC, SoftMasked- BERT, MacBERT, and SCOPE all obtain further

---

Detection
Correction
Model
F
SoftMasked-BEKI
73.7
73.2
73.5
66.7
66.2
66.4
SoftMasked-BEK1 DR-CSC
74.0
78.7
76.3
71.6
76.2
73.9
-with d/r results from MacBERT DR-CSC
74.4
78.8
76.5
72.1
76.4
74.2
witt d/r results from SCOPE + DR-CSC
73.3
78.5
75.8
71.6
76.6
74.0
MacBERT
71.9
77.9
74.8
68.0
73.6
70.7
MacBERT DR-CSC
75.8
78.3
77.0
73.6
76.1
74.8
with d/r results from SoftMasked-BEKT DR-CSC
75.2
79.4
77.2
75.2
76.8
74.7
with d/r results from SCOPE + DR-CSC
74.7
79.0
76.8
72.8
77.0
74.8
SCOPE
81.1
84.3
82.7
79.2
82.3
80.7
SCOPE DR-CSC
82.9
84.8
83.8
80.3
82.3
81.3
with d/r results from SoftMasked-BERT DR-CSC
82.2
83.9
83.1
80.1
81.7
80.9
with d/r results from MacBERT + DR-CSC
82.2
83.7
83.0
80.2
81.7
81.0
Table 2: The plug-and and-play performance analysis on the SIGHAN15 test The "d/r results from X" means the detection and reasoning tasks' results come from the X model.
improvements all test sets and most evalua- tion metries, which verifies the effectiveness of the proposed method. Specifically, at correction- level, SCOPE + DR-CSC exceeds SCOPE by 1.0% FI on SIGHAN13, 0.6% on SIGHAN14, and 0.6% on SIGHAN15. SoftMasked-BENl + DR- CSC exceeds SoftMasked-BERt by 4.0% Fl on SIGHANI3, 0.8% Fl on SIGHAN14, and 7.5% on SIGHAN15. MacBERT DR-CSC exceeds MacBERT by 4.8% F1 SIGHANI3, 4.5% F1 on SIGHAN14, and 4.1% on SIGHAN15.
ticular, when feeding the detection and reasoning tasks' ground truth simultancously, the correction F-1 of our model reaches 89.4, which shows that the decomposing method indeed intuitive and natural. Besides, this phenomenon shows the great potential of the DR-CSC module, because the meth- ods we design for detection and reasoning tasks are relatively simple (i.e., naive binary classification).
Subtask of SCOPE + DR-CSC
Precision
Recall
F1
Detection Reasoning
82.0
85.5
83.7
Particularly, SCOPE + DR-CSC achieves new state-of-the-ar performance on the three SIGHAN datasets, which reflects the competitiveness of the proposed module.
80.5
82.8
81.6
Searching
80.3
82.3
81.3
Table 3: The performance of each level of subtasks the SIGHAN15 test set.
4.3
Analysis and Discussion
4.3.2 Analysis of Each Subtask
4.3.1 Effect of Decomposing the CSC Models
The three subtasks in the CSC models address spe- cific questions: "Which?". "Why?". and "How?". These subtasks are designed to gradually increase in difficulty, progressing from easy hard. In order to validate the progressive difficulty of our subtasks, we also investigate the individual contributions of each subtask. From Table 3, we reveal a decreas- ing trend in the performance of the three subtasks: detection, reasoning, and searching. This finding aligns with our initial design intuition and motiva- tion, which suggests that these subtasks follow progressive difficulty pattern, with each subsequent task becoming more challenging than the previous one.
The motivation of our work to decompose the CSC task into three subtasks to introduce exter- nal knowledge and enhance the correction abil- ity of CSC models. Specifically, we divide the CSC task into Detection, Reasoning, and Search- ing three subtasks, and we believe that the infor- mation obtained by detection and reasoning tasks is helpful for learning searching task. To verify our motivation, we feed the detection ground truth into SCOPE + DR-CSC to observe performance changes.
From the results of "SCOPE + and "SCOPE + DR-CSC with d/r g!" in Table know that the wrong character position information in the detection task and the misspelling type infor- mation the reasoning task are both very helpful to improve the final correction performance. In par-
with gt"
4.3.3 Plug-and-Play Performance Analysis
We conduct an experiment to assess the compatibil- ity of the proposed DR-CSC module with existing

---

non-autoregressive CSC models for potential en- hancements. This experiment involves utilizing the results of detection and reasoning tasks from vari- ous base models. We construct new search matrix using these results and feed it into the searching task. This upproach not only reduces computation costs but also directly enhances the original model. The objective is to determine if the DR-CSC mod- ule effectively integrate with existing models and yield improvements in performance. From Ta- ble 2, we observe that each setting performs better than the bascline models on the SIGHANI5 test set. For the Soft-Masked BERT model, utilizing detec- tion and reasoning outputs from MacBERT leads to improved performance compared to training the de- tection and reasoning tasks on Soft-Masked BERT. For the MacBERT model and SCOPE model, uti- lizing detection and reasoning output trained on themselves yields the best performance. Overall, the best performance is achieved by utilizing the detection and reasoning task results from SCOPE, possibly due to its different encoders. SCOPE selects ChiTeseBERT (Sun et al., 2021) as their encoder, which incorporates both the glyph and pinyin information of Chinese characters into lan- guage model pretraining. With the assistance of glyph and pinyin information, SCOPE achieves better performance. This approach significantly re- duces the time required for retraining a DR module. It allows for the direct utilization of a trained DR module, resulting in substantial resource savings.
Case 1:
Input Wrong Sentence:
3R TT of
I can't accept and agaiatc it at first eaher.
SCOPE:
RIF Ho shsbo TT (laotis
can't accept at first
SCOPE + DR-CSC(w/a
MArHAA ta Boruyy T (1
I cannt finst either.
Searching):
SCOPE DR-CSC:
MIF 19 sbout TT (haolh8
can't stand first either.
Input Wrong Sentence:
SRNE Hi (zhhoo T
didn't expect to dunce and dance light the midnight.
SCOPE:
SEMBRRR dl REAE RI 4
didn't expect dance and dance light the midnight.
SCOPE + DR-CSC(w/o Searching):
RRSEBIE th th To H++
expect dance dance again midnight.
SCOPE
allee the #
expect dance and through midnight.
Table 5: Cases from the SIGHANI5 test set show the searching subtask can narrow the search space to en- hance the correction ability of the CSC model. We mark the wrong/correct characters.
derstanding. According to the data presented Table 4, phonological errors account for 39% the predicted error characters, while morphological errors make up the remaining 61 %. Among the morphological error characters, only four are not found in the morphological confusion set, prompt- ing assign search matrix value of one these characters. Furthermore, out of total of 769 predicted error characters, only 11 (1.4%) fail find the correct character within their search matrix. The majority of error characters identified by our module were found to have their correct counter- parts within their respective confusion sets. This finding provides a compelling explanation for the performance gains observed in our study.
Item
Number
Predicted phonological emorr
300
Predicted morphological error
469
Not phonological confusion
Not morphological confusion set
Correct character is filtered out phonological
Correct character is filtered out
morphological
Table 4:
We count the outputs of detection and reasoning
4.3.5 Case Study
tasks on SIGHANI5 test set.
From Table 5, we know that SCOPE cannot detect "Ikr" while SCOPE + DR-CSC (w/o Searching) succeeds (even if its correct answer is wrong). This phenomenon indicates the effectiveness of the de- tection task in DR-CSC, which guides the model to focus finding the wrong characters at the begin- ning. In addition, the character (i.e., "Il[(shōu)") that SCOPE DR-CSC (w/o Searching) cannot correct accurately solved by SCOPE + DR-CSC. This is benefited from our searching task, which uses the information provided by detection and
4.3.4 Interpretability Analysis
In the searching task of the proposed module, we construct a search matrix that effectively narrows the search space of candidate characters. This op- timization greatly enhances the performance of CSC models. Additionally, the CSC models con- sist of three subtasks that address the questions "Which?Why?How!". This approach enhances in- terpretability and offers greater potential for un-

---

reasoning tasks to construct a fine- fine-grained search matrix so that the model search in the phonolog- ical confusion set of ""(((sh"""", while avoiding predicting (zud)" whose phonetic is not similar to "(ho("" In Case 2, we know that SCOPE cannot detect "I""". while SCOPE + DR-CSC (w/o Searching) succeeds (even if its correct answer is wrong). However, SCOPE + DR-CSC can correct it accurately, which benefited from the search ma- trix provided in the searching task.
Acknowledging these limitations, further ad- vancements can be made to enhance the detection and reasoning subtasks, expand the scope of error types considered, and optimize the computational demands associated with the search matrix con- struction in the scarching subtask.
A Acknowledgements
This research is supported by National Nat- ural Science Foundation of China (Grant No.62276154), Research Center for Computer Networ(SSenzhen of Education, the Natural Science Foundation of Guangdong Province (Grant No. 2023A151501291 Basic Research Fund of Shenzhen City (Grant No. JCY120210324120011000
5 Conclusion
In this paper, we present enhanced approach for the CSC model by decomposing into three subtasks. The main objective is integrate exter- nal knowledge and offer improved interpretability. We introduce DR- DR-CSC, a frustratingly easy plug- and-play
and JSGG202108022 (10G2002100001554400007,,,, the Major Key Project of PCL for Experiments and Applications (PCL2021 A06), and Overseas Cooperation Research Fund of Tsinghua Shenzhen International Graduate School (HW2021008).
module that con- sists of three subtasks with progressive difficulty. By conducting thorough experiments and detailed analyses, we have empirically demonstrated the effectiveness of our decomposition approach, as well as the valuable information provided by each subtask. Furthermore, we believe that DR-CSC holds untapped potential for further exploration.
References
Haithem Afli, Zhengwei Qiu, Andy Way, and Páraic Sheridan. 2016. Using SMT for OCR error correc- tion of historical texts. In Proceedings of the Tenth International Conference on Language Resources and Evaluation LREC 2016, Portorvū Slovenia, May 23-28, 2016, European Language Resources Associ- ation (ELRA).
6 Limitations
Our proposed module has three potential limita- tions that should be acknowledged. Firstly, the detection and reasoning subtasks within our mod- ule are relatively straightforward and offer room for improvement. Future research could focus on en- hancing the complexity and sophistication of these subtasks to further enhance their performance. Secondly, our module only considers two types of misspellings: phonological errors and morpho- logical errors. While these two types cover a signif- icant portion of common misspellings, other types of errors may exist that are not addressed in our current framework. Exploring and incorporating additional error types could contribute more comprehensive and robust module.
Xingyi Cheng, Weidi Xu, Kunlong Chen, Shaohua Jiang, Feng Wang. Taifeng Wang, Wei Chu, and Yuan Qi. 2020. SpelIGCN: Incorporating phonologi- cal and visual similarities into language models for Chinese spelling check. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 871 $71-881, Online. Association for Computational Linguistics.
Xuxin Cheng, Bowen Cao, Qichen Ye, Zhihong Zhu, Hongxiang Li, and Yuexian Zou. 2023a. MI-Imel: Mutual learning and large-margin contrastive learn- ing for improving ast robustness in spoken language understanding. Findings of the Association for Computational Linguistics: ACL 2023, pages 6492- 6505.
Thirdly, the construction of the search matrix in the searching subtask introduces an additional computational cost during both the training and inference stages. This demands careful considera- tion in terms of resource allocation and efficiency. Future work should focus on optimizing the search matrix construction process to mitigate the compu- tational overhead while maintaining performance.
Xuxin Cheng, Qianqian Dong, Fengpeng Yue, Tom Ko, Mingxuan Wang, and Yuexian Zou. 2023h. M 3 st: Mix at three levels for speech translation. In ICASSP 2023-2023 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE.
Xuxin Cheng, Zhihong Zhu, Hongxiang Li, Yaowei Li, and Yuexian Zou. 2023c. Ssymr: Saliency-based self-

---

training for video- music retrieval. In ICASSP 2023- 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5, IEEE.
Processing (Volume 1: Long Papers), pages 5958- 5967, Online. Association for Computational Lin- guistics.
Tuo Ji, Hang Yan, and Xipeng Qiu. 2021, SpellBERT: A lightweight pretrained model for Chinese spelling check. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3544-3551, Online and Punta Cana, Domini- can Republic. Association for Computational Lin- guisties.
Xuxin Cheng, Zhihong Zhu, Ziyu Yao, Hongxiang Li, Yaowei Li, and Yuexian Zou. 2023d. GhostT5: Gen-
erate More Features with Cheap Operations to Im- prove Textless Spoken Question Answering. In Proc. INTERSPEECH 2023, pages 1134-1138.
Yiming Cui, Wanxiang Che, Ting Liu,, Bing Qin, Shijin Wang, and Guoping Hu. 2020. Revisiting pre-trained models for Chinese natural language processing. In Findings of the Association for Computational Lin-
Chong Li, Cenyuan Zhang, Xiaoqing Zheng, and Xu- anjing Huang. 2021a. Exploration and exploitation: Two ways to improve Chinese spelling correction models. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics (Volume 2: Short Papers), pages 441 446, Online. Association for Computational Linguistics.
guistics: EMNLP 2020, pages 657-668, Online. As- sociation for Computational Linguistics.
Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, and Ziqing Yang. 2021. Pre-training with whole word masking for chinese bert. TEFF/ACM Transac- tions on Audio, Speech, and Language Processing, 29:3504 3514.
Fangfang Li. Youran Shan, Junwen Duan, Xingliang Mao, and Minlie Huang. 2022a. WSpeller: Robust word segmentation for enhancing Chinese spelling check. In Findings of the Association for Computa- tional Linguistics: EMNLP 2022, pages 1179-1188, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Chenhe Dong, Yinghui Li. Haifan Gong, Miaoxin Chen, Junxin Li, Ying Shen, and Min Yang. 2023. A survey of natural language generation. ACM Comput. Surv., 55(8): 173:1-73338.
Jiahao Li, Quan Wang. Zhendong Mao, Junbo Guo, Yanyan Yang. and Yongdong Zhang. 2022b. Improv- ing Chinese spelling check by character pronuncia- tion prediction: The effects of adaptivity and gran- ularity. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. pages 4275-4286, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Fei Dong and Yue Zhang. 2016. Automatic features for essay scoring empirical study. In Proceedings of the 2016 Conference on Empirical Methods in Nat- ural Language Processing, pages 1072-1077, Austin, Texas. Association for Computational Linguistics.
Jianfeng Gao, Xiaolong Li, Daniel Micol, Chris Quirk, and Xu Sun. 2010. A large scale ranker- system for search query spelling correction. In Proceedings of the 23rd International Conference on Computa- tional Linguistics (Coling 2010), pages 358-366, Bei- jing, China. Coling 2010 Organizing Committee.
Jing Li. Dafei Yin, Haozhao Wang, and Yonggang Wang. 2021b. Despell: A detector-correcto framework for chinese spelling error correction. In SIGIR '21: The 44th International ACM SIGIR Conference Research and Development in Information Retrieval, Virtual Event, Canada, July 11 11-15, 2021, pages 1870- 1874. ACM.
Danqing Huang, Shuming Shi, Chin- Yew Lin, Jian Yin, and Wei-Ying Ma. 2016. How well do computers solve math word problems? large-scale dataset con- struction and evaluation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pages 887-896.
Piji Li. 2022. uchecker: Masked pretrained language models as unsupervised chinese spelling checkers. In Proceedings of the 29th International Confer- ence on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12-17, 2022, pages 2812-2822. International Committee on Com- putational Linguistics.
Danqing Huang, Jin-Ge Yao, Chin- Yew Lin, Qingyu Zhou, and Jian Yin. 2018. Using intermediate repre- sentations to solve math word problems. In Proceed- ings of the 56th Annual Meeting of the Association for Compustational Linguistics (Volume 1: Long Papers), pages 419-428, Melbourne, Australia. Association for Computational Linguistics.
Piji Li and Shuming Shi. 2021. Tail-to-tail non- autoregressive sequence prediction for chinese gram- matical error correction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, ( (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 4973-4984. Association for Computational Linguistics.
Li Huang, Junjie Li, Weiwei Jiang, Zhiyu Zhang, Minchuan Chen, Shaojun Wang, and Jing Xiao. 2021. PHMOSpell: Phonological and morpholog- ical knowledge guided Chinese spelling check. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguisties and the 1ith International Joint Conference on Natural Language
Yangning Li. Jiaoyan Chen, Yinghui Li, Tianyu Yu. Xi Chen, and Hai-Tao Zheng. 2023a. Tmbracing ambiguity: Improving similarity-oriented tasks with contextual synonym knowledge. Neurocomputing. 555: 1 26583.

---

Yinghui Li. Haojing Huang, Shirong Ma. Yong Jiang. Yangning Li, Feng Zhou, Hai-Tao Zheng, and Qingyu Zhou. 2023b. On the (injeffecttivnesss of large lan- guage models for chinese text correction. CoRR, abs/2307.09007.
Shulin Liu, Tao Yang, Tianchi Yue, Feng Zhang, and Di Wang. 2021. PLOME: Pre-training with mis- spelled knowledge for Chinese spelling correction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th 11th International Joint Conference on Natural Lan- guage Processing (Volume 1: Long Papers), pages 2991-3000, Online. Association for Computational Linguistics.
Yinghui Li, Shulin Huang, Xinwei Zhang, Qingyu Zhou, Yangning Li, Ruiyang Liu, Yunbo Cao, Hai-Tao Zheng, and Ying Shen. 2022c. Automatic context pattern generation for entity set expansion. CoRR, abs/2207. 00808777.
Shirong Ma, Yinghui Li, Haojing Huang, Shulin Huang, Yangning Li, Hai-Tao Zheng, and Ying Shen. 2023, Progressive multi- learning framework for chi- nese text error correction. CoRR, abs/2306. 174477.
Yinghui Li, Yong Jiang, Shen Huang, Xingyu Lu, Yangning Li. Pengjun Xie, Fei Huang, and Hai-Tao Zheng. 2023c. Bidirectional end-to-end learning of retriever-reader paradigm for entity linking. CoRR, 12s/2306.12245..
Shirong Ma, Yinghui Li, Rongyi Sun, Qingyu Zhou, Shulin Huang. Ding Zhang, Li Yangning, Ruiyang Liu, Zhongli Li, Yunbo Cao. Haitao Zheng. and Ying Shen. 2022. Linguistic rules-based corpus gener- ation for native Chinese grammatical eπror corree- tion. In Findings of the Association for Computa- tional Linguisticx: EMNLP 2022, pages 576-589, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Yinghui Li, Yangning Li. Yuxin He, Tianyu Yu, Ying Shen, and Hai-Tao Zheng. 2022d. Contrastive learn- ing with hard negative entities for entity set expan- sion. In SIGIR 22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 15, 2022, pages 1077-1086 ACM.
Bruno Martins and Mário J. Silva. 2004. Spelling cor- rection for search engine queries. In Advances in Nat- ural Language Processing, pages 372-383, Berlin, Heidelberg. Springer Berlin Heidelberg.
Yinghui Li. Shirong Ma, Qingyu Zhou, Zhongli Li, Li Yangning, Shulin Huang, Ruiyang Liu, Chao Li, Yunbo Cao, and Haitao Zheng. 2022c. Learning from the dictionary: Heterogeneous knowledge guided fine-tuning for Chinese spell checking. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 238-249, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
Rui Sun, Xiuyu Wu, and Yunfang Wu. 2022. An erro- guided correction model for Chinese spelling ertor correction. In Findings of the Association for Com- putational Linguistics: EMNLP 2022. pages 3800- 3810, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Yu Sun, Shuohuan Wang, Yukun Li, Shikun Feng. Hao Tian, Hua Wu, and Haifeng Wang. 2020. Ernie 2.0: A continual pre-training framework for language un- derstanding, Proceedings of the AAA/ Conference Artificial Intelligence, 34(05):8968-89) 8975.
Yinghui Li, Qingyu Zhou, Yangning Li. Zhongli Li, Ruiyang Liu, Rongyi Sun, Zizhen Wang, Chao Li, Yunbo Cao, and Tai-Tao Zheng. 2022f. The past mis- take is the future wisdom: Error-driven contrastive probability optimization for Chinese spell checking. In Findings of the Association for Computatonal Linguistics: ACI. 2022, pages 3202-3213, Dublin, Ireland. Association for Computational Linguistics.
Zijun Sun, Xiaoya Li, Xiaofei Sun, Yuxian Meng. Xiang Ao, Qing He, Fei Wu, and Jiwei Li. 2021. Chinese- BERT: Chinese pretraining enhanced by glyph and Pinyin information. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguisties and the 11th International Joint Confer- ence on Natural Language Processing (Volume 1: Long Papers), pages for Computational Linguistics.
Chao-Lin Liu, Min- Lai, Yi- Yi-Hsuan Chuang, and Chia- Ying Lee. 2010. Visually and phonologically similar characters in incorrect simplified Chinese words. In Coling 2010: Posters. pages 739-747, Beijing, China. Coling 2010 Organizing Committee.
Online. Association
Yuen- Tseng, Lung- Hao Lee, Li-Ping Chang, and Hsin-Hsi Chen. 2015. Introduction to SIGHAN 2015 bake-off for Chinese spelling check. In Proceed- ings of the Eighth SIGHAN Workshop on Chinese Language Processing, pages 32-37, Beijing, China. Association for Computational Linguisties.
Ruiyang Liu, Yinghui Li, Linmi Tao, Dun Liang, and Hai-Tao Zheng. 2022a. Are we ready for new paradigm shift? survey on visual deep MLP. Pat- terns, 100500
Shulin Liu, Shengkang Song, Tianchi Yue, Tao Yang. Huihui Cai, TingHao Yu, and Shengli Sun. 2022b. CRASpell: A contextual typo robust approach to improve Chinese spelling correction. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3008-3018 Dublin, Ireland. Association for Computational Linguistics.
Baoxin Wang. Wanxiang Che, Dayong Wu, Shijin Wang, Guoping Hu, and Ting Liu. 2021. Dynamic connected networks for Chinese spelling check. In Findings of the Association for Computational Lin- guistics: ACL-IJCNLP 2021, pages 2437-2446, On- line. Association for Computational Linguistics.

---

Dingmin Wang, Yan Song, Jing Li. Jialong Han, and Haisong Zhang. 2018. A hybrid approach to auto- matic corpus generation for Chinese spelling check. In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 2517-2527, Brussels, Belgium. Association for Com- putational Linguistics.
Ruiqing Zhang, Chao Pang, Chuanqiang Zhang, Shuo- huan Wang. Zhongjun He, Yu Sun. Hua Wu, and Haifeng Wang. 202 Correcting Chinese spelling errors with phonetic pre-training. In Findings of the Association for Compulational Linguistics: ACL- IJCNPP 2021, pages 2250-2261. Online. Association for Computational Linguistics.
Dingmin Wang. Yi Tay, and Li Zhong. 2019. Confusionset-guidee pointer networks for Chinese spelling check. In Proceedings of the 57th Annual Meeting of the Association for Computational Lin- guistics, pages 5780-5785, Florence, Italy, Associa- tion for Computational Linguistics.
Shaohua Zhang, Haoran Huang, Jicong Liu, and Hang Li. 2020. Spelling error correction with soft-masked BERT. In Proceedings tfe 58th Annual Meeting of the Association for Computational Linguistics, pages 882-890, Online. Association for Computational Lin- guistics.
Xin Zhang, Guangwei Xu, Yueheng Sun, Meishan Zhang. Xiaobin Wang, and Min Zhang. 2022a. Iden- tifying Chinese opinion expressions with extremely- noisy crowdsourcing annotations. In Proceedings of the 60th Anmual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 2801-2813 Dublin, Ireland. Association for Computational Linguistics.
Jian-cheng Wu, Wsun-wen Chiu, and Jason S. Chang. 2013a. Integrating dictionary and web n n-grams for Chinese spell checking. In International Journal of Computational Linguistics & Chinese Language Processing, Volume 18, Number December 2013- Special Issue on Selected Papers from ROCLING XXV.
Shih-Hung Wu, Chao-Lin Liu, and Lung-Hao Lee. 2013b. Chinese spelling check evaluation at SIGHAN bake-off 2013. In Proceedings of the Sev- enth SIGHAN Workshop Chinese Language Pro- cessing, pages 35-42. Nagoya, Japan. Asian Federa- tion of Natural Language Processing.
Xin Zhang, Guangwei Xu, Yueheng Sun, Meishan Zhang, and Pengjun Xie. 2021b. Crowdsourcing learning domain adaptation: A case study on named entity recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence on Natural Language Processing (Volume 1: Long Papers), pages 5558-5570, Online. Association for Computational Linguistics.
Heng-Da Xu, Zhongli Li, Qingyu Zhou, Chao Li, Zizhen Wang, Yunbo Cao, Heyan Huang, and Xian- Ling Mao. 2021. Read, listen, and see: Leveraging multimodal information helps Chinese spell checking. In Findings of the Association for Computational Lin- guisties: ACL-IJCNLP 2021, pages 216-728, Online. Association for Computational Linguistics.
Yusen Zhang, Zhongli Li, Qingyu Zhou, Ziyi Liu, Chao Li, Mina Ma, Yunbo Cao, and Hongzhi Liu. 2022b. Aim: Taking answers in mind to correct chinese cloze tests in educational applications. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3042-3053.
Jingheng Ye, Yinghui Li, Shirong Ma, Rui Xie, Wei Wu, and Hai-Tao Zheng, 2022. Focus is what you need for chinese grammatical error correction. CoRR, abs/2210. 1622.
Chenxi Zhu, Ziqiang Ying, Boyu Zhang, and Feng Mao. 2022. MDCSpell: A multi-task detector-corrector framework for Chinese spelling correction. Find- ings of the Association for Computational Linguis- tics: ACL 2022, pages 1244 1253, Dublin, Ireland. Association for Computational Linguistics.
Jingheng Ye, Yinghui Li, Qingyu Zhou, Yangning Li. Shirong Ma. Hai-Tao Zheng, and Ying Shen. 2023. CLEME: debiasing multi-reference eval- uation for grammatical error correction. CoRR, 10s/2305.00819.
Liang-Chib Yu, Lung-Hao Hao Lcc, Yuen- Tseng, and Hsin-Hsi Chen. 2014. Overview of SIGHAN 2014 bake-off for Chinese spelling check. In Proceedings of The Third CIPS-SIGHAN Joint Conference Chi- nese Language Processing, pages 126-132, Wuhan, China. Association for Computational Linguistics.
Ding Zhang, Yinghui Li, Qingyu Zhou, Shirong Ma. Yangning Li, Yunbo Cao, and Hai-Tao Zheng. 2023. Contextual similarity more valuable than charac- ter similarity: An empirical study for chinese spell checking. In ICASSP 2023 2023 IEEE Interna- tional Conference on Acousties, Speech and Signal Processing (ICASSP), pages 1-5.


Reviews:
[{"review_id": "k4fzUFxf5O", "reviewer": "Reviewer_iyvk", "summary": "", "questions": "Question A: Although previous work has been using SIGHAN13/14/15 datasets, I am concerned that they may be somewhat outdated. Have you considered experimenting with some newer datasets (such [1] or [2] )?\n\nQuetion B: I don't understand a little about \"-with d gt\" and \"with d/r gt\" in Table 1. Do they mean to using a method similar to teach-forcing during training or to input ground truth during testing?\n\nQuestion C: In my opinion the form of pipline seems to be more intuitive to decompose CSC task. Can you provide some reasons or result analysis for using multi-task learning instead of pipline?\n\n\nReferences\n\n[1]https://github.com/nghuyong/cscd-ime\n\n[2]https://github.com/Aopolin-Lv/ECSpell", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments. We attach great importance to your questions and hope to address your concerns.\n\n---\n\n\n**Q1**: The improvement is low. While the module does consistently improve CSC results compared to all prior approaches, it is not clear that the magnitude of the improvement is worth the cost, especially with double PLM parameters.  \n**A1**: Our module can be integrated with any non-autoregressive CSC model. By incorporating our module and retraining, improvements over the baseline model's performance can be achieved. The costs of the model include various aspects, such as the size of the model's parameters and the computational resources required for training. Furthermore, the pre-trained DR module can be directly combined with a new CSC model, leading to an enhancement in the new model's performance. This scenario would entail the utilization of double PLM parameters. **However, it's important to note that there is no need to retrain a new DR module in this case, resulting in substantial cost savings. This situation presents a trade-off.**  \n\n\n---\n\n\n**Q2**: Although previous work has been using SIGHAN13/14/15 datasets, I am concerned that they may be somewhat outdated. Have you considered experimenting with some newer datasets (such as [1] or [2] )?  \n**A2**: Absolutely, thank you for bringing this up. We are aware of the CSCD-IME dataset, which is an open-source collection specifically errors caused by the pinyin input method. \nFurthermore, ECSpell offers a diverse set of domains for CSC, incorporating data from legal, medical, and official document contexts.  \nWe believe that the datasets mentioned above are all too domain-relevant and contrary to our goal of\n**However, we think that the datasets mentioned above are all too domain-relevant and deviate from our goal of achieving a more universally applicable approach to CSC.** In this context, the SIGHAN datasets excel due to their broader, more general nature.  \nIt's noteworthy that as of the latest ACL 2023 [1-4] and EMNLP 2022 [5-8] conferences, **these datasets have not been widely adopted.** However, we greatly appreciate your suggestion, and we plan to conduct more insightful research on these datasets in the future.  \n \n\n---\n\n\n**Q3**: I don't understand a little about \"-with d gt\" and \"with d/r gt\" in Table 1. Do they mean to use a method similar to teach-forcing during training or to input ground truth during testing?  \n**A3**: They actually refer to the latter one – **input ground truth during testing**. We understand that the phrasing in the caption of Table 1 may need improvement to better reflect this.  \n\n\n---\n\n\n**Q4**: In my opinion, the form of pipeline seems to be more intuitive to decompose CSC task. Can you provide some reasons or result analysis for using multi-task learning instead of pipeline?  \n**A4**: While the pipeline approach indeed has its merits, it's important to consider the training cost associated with sequentially training these modules. **Each step in the pipeline necessitates independent training, potentially leading to a lengthier and more resource-intensive training process.** In contrast, we made the decision to adopt a multi-task learning framework, allowing the model to simultaneously learn from multiple tasks. In our experiments, we found that the multi-task learning approach not only streamlined the training process but also demonstrated improved performance on various evaluation metrics. This suggests that the model's ability to learn from multiple tasks simultaneously contributed to its enhanced performance compared to a traditional pipeline approach.  \n  \n\n---\n\n\nReferences  \n[1] UMRSpell: Unifying the Detection and Correction Parts of Pre-trained Models towards Chinese Missing, Redundant, and Spelling Correction. ACL 2023.  \n[2] Disentangled Phonetic Representation for Chinese Spelling Correction. ACL 2023.  \n[3] PTCSpell: Pre-trained Corrector Based on Character Shape and Pinyin for Chinese Spelling Correction. ACL 2023.  \n[4] Investigating Glyph-Phonetic Information for Chinese Spell Checking: What Works and What’s Next?. ACL 2023.  \n[5] An Error-Guided Correction Model for Chinese Spelling Error Correction. EMNLP 2022.  \n[6] Learning from the Dictionary: Heterogeneous Knowledge Guided Fine-tuning for Chinese Spell Checking. EMNLP 2022.  \n[7] WSpeller: Robust Word Segmentation for Enhancing Chinese Spelling Check. EMNLP 2022.  \n[8] Improving Chinese Spelling Check by Character Pronunciation Prediction: The Effects of Adaptivity and Granularity. EMNLP 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The method is simple but effective. And the proposed model shows good potential in incorporating new expert knowledge.\n\n2. The experimental analysis is sufficient and convincing.\n\n3. Clear and direct structure of paper writing.", "reasons_to_reject": "1. The improvement is low.  While the module does consistently improve CSC results compared to all prior approaches, it is not clear that the magnitude of the improvement is worth the cost, especially with double PLM parameters.\n\n2. Experiments use fewer datasets. See \"Questions For The Authors\" for details.", "questions_to_authors": "Question A: Although previous work has been using SIGHAN13/14/15 datasets, I am concerned that they may be somewhat outdated. Have you considered experimenting with some newer datasets (such [1] or [2] )?\n\nQuetion B: I don't understand a little about \"-with d gt\" and \"with d/r gt\" in Table 1. Do they mean to using a method similar to teach-forcing during training or to input ground truth during testing?\n\nQuestion C: In my opinion the form of pipline seems to be more intuitive to decompose CSC task. Can you provide some reasons or result analysis for using multi-task learning instead of pipline?\n\n\nReferences\n\n[1]https://github.com/nghuyong/cscd-ime\n\n[2]https://github.com/Aopolin-Lv/ECSpell", "paper_topic_and_main_contributions": "This paper proposes a simple but effective plug-and-play module named DR-CSC (Detection-and-Reasoning module for Chinese Spelling Check). They decompose CSC task into three sub-tasks (Detection, Reasoning and Search) in a human-intuitive way. The overall structure of the module seems to be a PLM encoder with three prediction heads, which model the three sub-tasks of retrieval, reasoning and search respectively, and adopt the paradigm of multi-task learning to update parameters. This paper conducts extensive experimental analysis on SIGHAN13/14/15 datasets. The evaluation results on show that DR-CSC's SOTA performance and good plug-and-play ability.\n", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 538 6 Limitations -> Limitations"}, "extra_text": {}}, {"review_id": "I9d4SuQ4s2", "reviewer": "Reviewer_cyLs", "summary": "", "questions": "1. The method proposed in this paper uses prior knowledge from the confusion set. Would there be a more pronounced improvement in a low-resource scenario?\n\n2. I would be interested to hear the authors' plan to apply the method from this paper to a decoder-only language model (LLM).", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your positive comments and insightful reviews. Please find below our point-by-point responses:\n\n\n---\n\n\n**Q1**: The information from the confusion set could potentially be directly incorporated into an end-to-end model for learning. This pipeline approach may lead to error accumulation, thereby reducing the final performance.  \n**A1**: Thank you very much for your valuable feedback. The improvements brought about by DR-CSC directly come from the utilization of a search matrix constructed using the confusion set in the Searching subtask. If the search matrix does not contain the correct corresponding Chinese character, the model will not be able to correct that error. The reasons for such error accumulation could be twofold: Firstly, the Detection subtask might incorrectly identify characters that are not actually errors as typos; secondly, there might be misclassifications in the Reasoning subtask. These factors contribute to error accumulation, ultimately leading to performance degradation.  \nWe have taken steps to mitigate the effects of both factors. Firstly, **during the construction of the confusion set, we ensured that the corresponding characters were included.** This approach helps alleviate errors caused by the misidentification of non-erroneous characters as errors. Furthermore, **the sets of phonetically similar characters and visually similar characters are not mutually exclusive**; they exhibit a certain degree of intersection. This characteristic further mitigates the impact of error accumulation.  \nFinally, based on our experimental statistics as presented in \"4.3.4 Interpretability Analysis\", **the proportion of correct characters not present in their corresponding confusion sets is remarkably low only accounting for 1.4%**. This indicates that the performance loss resulting from **error accumulation fails to overshadow the gains our method introduces.** It can be viewed as a trade-off that underscores the effectiveness of our approach.  \n\n\n---\n\n\n**Q2**: The improvements observed from the experiment were not significant, and there's a lack of multiple experiments with averaged results and variance analysis.  \n**A2**:  **We need to clarify that the experimental results presented in the table are the averages of five individual experiments.** However, we acknowledge the need for a more comprehensive approach, involving multiple experiments and a thorough variance analysis to ensure the robustness of our findings. Your feedback highlights an essential aspect of experimental validation, and we will certainly incorporate this suggestion into our future research.\n\n\n---\n   \n\n**Q3**: The scope of errors covered is limited to only two types, phonological errors and morphological errors. Problems outside of these errors may not be addressed.  \n**A3**: In the field of CSC, **the primary errors are primarily attributed to phonological and morphological errors, aligning with the inherent nature of Chinese characters.** The reason for not incorporating a third error type can be explained by the following factors:  \n- **Low Incidence**: Other error types comprise a minimal proportion of errors, as indicated by our analysis of the error types in the SIGHAN15 dataset, the \"other error types\" only account for 1.9%.  Given this low incidence, allocating a new error type for these cases would likely decrease efficiency. Additionally, setting up a new error type would necessitate constructing new confusion sets, which might not be proportional in terms of cost and benefit.  \n- **Coverage by Existing Sets**: The current phonetical and visual confusion sets to some extent encompass some characters that could be classified as \"other errors.\"  \n\nLastly, according to our experimental statistics, the proportion of correct characters not present in their corresponding confusion sets is extremely low. This implies that the absence of certain error types in our confusion sets does not necessarily imply an inability of our model to address those errors.  \n\n\n---\n\n\n**Q4**: The analysis of the three subtasks is too simplistic. Further analysis is needed to understand the reasons for improvements and potential areas for enhancements.  \n**A4**: **Regarding the analysis of the three subtasks, we conducted three distinct experiments**. First, we input ground truth labels for the Detection and Reasoning subtasks into the model for error correction during the inference. The results are summarized in the table below:  \n| Model | D-P | D-R | D-F | C-P | C-R | C-F |\n|------------|-------------------|-------------------|--------------|-------------------|-------------------|--------------|\n| SCOPE + DR-CSC   | 82.9    | 84.8   | 83.8   |80.3| 82.3 |81.3|\n| -with d gt     | 89.8 | 94.1 |91.9 |85.7 |89.8 |87.7|\n|-with d/r gt    | 90.2 | 95.2 |92.6 |87.0 |91.9 |89.4|\n\nThis experimental setup aimed to demonstrate the effectiveness of decomposing the CSC task into three subtasks, highlighting the potential of DR-CSC. Moreover, detailed performance metrics for all three subtasks are provided in Table 3 of the paper.  \nIn addition, we conducted a quality assessment of the search matrix in the search task. The evaluation revealed that only 1.4% of the characters were unable to find the correct character within their corresponding confusion sets, indicating minimal error accumulation. Furthermore, the Case Study reaffirms the effectiveness of the proposed modules.   \nTo address your concerns about the simplicity of the analysis, we acknowledge the need for a more comprehensive investigation into the factors contributing to the observed improvements and areas where enhancements can be made. We plan to delve deeper into these aspects by considering various influencing factors and conducting additional experiments to provide a more insightful analysis.  \n\n\n---\n\n\n**Q5**: The method proposed in this paper uses prior knowledge from the confusion set. Would there be a more pronounced improvement in a low-resource scenario?  \n**A5**: We sincerely appreciate the insights you have provided, as they are instrumental in improving and elevating the quality of our research. In the field of CSC, several mainstream approaches are currently being explored:  \n- **Introduction of Multimodality**: Some methods integrate multimodal information [1].  \n- **Variations of Pretraining Techniques**: Another direction involves adapting variations of pretraining methods [2,3].  \n- **Efficient Utilization of External Knowledge**: External knowledge, such as confusion sets and dictionary information, plays a substantial role in enhancing CSC [4,5].  \n\nOur approach specifically focuses on efficiently incorporating external knowledge. Leveraging this external knowledge has consistently been a means to enhance the effectiveness of CSC methods.  \nHence, in a low-resource scenario, where access to extensive data might be limited, the utilization of external knowledge, as demonstrated in our approach, could indeed yield more significant improvements. The knowledge encoded within **the confusion sets becomes even more valuable in situations where the model has less data to learn from.**  \n\n\n---\n\n\n**Q6**: I would be interested to hear the authors' plan to apply the method from this paper to a decoder-only language model (LLM).  \n**A6**: Thank you for highlighting this point. Indeed, the potential application of our method to a decoder-only language model (LLM) presents an intriguing avenue for future research. **Injecting the knowledge derived from our approach into an LLM is a promising direction that aligns well with our ongoing efforts.** Exploring how to effectively incorporate our technique into a decoder-based framework and leverage its benefits within that context will be a focus of our forthcoming work. Your inquiry underscores the exciting prospects for extending the utility of our approach to different language modeling paradigms.  \n\n\n---\n\n\n**References**   \n[1] Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking. ACL-IJCNLP 2021.  \n[2] The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking. ACL 2022.  \n[3] Improving Chinese Spelling Check by Character Pronunciation Prediction: The Effects of Adaptivity and Granularity. EMNLP 2022.  \n[4] Learning from the Dictionary: Heterogeneous Knowledge Guided Fine-tuning for Chinese Spell Checking. EMNLP 2022.  \n[5] SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check. ACL 2020.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The design of the three subtasks improves interpretability.\n\n2. The method section of the paper is well-articulated and clear.", "reasons_to_reject": "1. The information from the confusion set could potentially be directly incorporated into an end-to-end model for learning. This pipeline approach may lead to error accumulation, thereby reducing the final performance.\n\n2. The improvements observed from the experiment were not significant, and there's a lack of multiple experiments with averaged results and variance analysis.\n\n3. The scope of errors covered is limited to only two types, phonological errors and morphological errors. Problems outside of these errors may not be addressed.\n\n4. The analysis of the three subtasks is too simplistic. Further analysis is needed to understand the reasons for improvements and potential areas for enhancements.", "questions_to_authors": "1. The method proposed in this paper uses prior knowledge from the confusion set. Would there be a more pronounced improvement in a low-resource scenario?\n\n2. I would be interested to hear the authors' plan to apply the method from this paper to a decoder-only language model (LLM).", "paper_topic_and_main_contributions": "The paper designs the DR-CSC module, a mechanism that guides the model to correct Chinese spelling errors by integrating a confusion set. The effectiveness of this method has been confirmed by experimental results.\n\n", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "iEvjRjF92H", "reviewer": "Reviewer_frro", "summary": "", "questions": "see reasons to reject", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your appreciation of our work's contributions. We are honored to have the opportunity to discuss with you.\n\n\n---\n\n\n**Q1**: The purpose of Figure 1 is not clear, as it fails to effectively communicate the necessary information at a preliminary glance. The roles of the various icons and colors used in the figure are not explained, leading to confusion about the nature of the CSC task.   \n**A1**: Thank you for your feedback, we will make the necessary revisions to the figure and provide explicit explanations for each element.\n\n\n---\n\n\n**Q2**: A similar issue arises with Figure 2. It remains unclear what each color signifies and it is not explicitly mentioned which parts were proposed by the user and which parts are tunable parameters.  \n**A2**: We appreciate your observation and comment regarding Figure 2. Your feedback highlights an important aspect that needs clarification. To address this issue, we will thoroughly revise the captions associated with the model diagram.\n\n\n---\n\n\n**Q3**: There seems to be an incorrect portrayal of the \"multi-task\" aspect. While the authors assert that their approach benefits from multi-task learning in line 295, the paper seems to solely focus on the CSC task.  \n**A3**: Thank you for raising this important point.  To address this issue, we want to clarify that we actually decomposed the CSC task into three distinct subtasks. These subtasks were then jointly learned using a multi-task learning approach. Regrettably, we acknowledge that our presentation didn't emphasize this aspect adequately, leading to the misunderstanding that our focus was solely on the CSC task. We will revise our manuscript to provide a more explicit and detailed explanation.\n\n\n---\n\n\n**Q4**: The authors purport their method to be \"plug-and-play\" and to incorporate external language more naturally. However, their experiments are based on previously proposed CSC models such as SoftMasked-BERT, MacBERT, and SCOPE, which, I believe, are already equipped to handle CSC tasks. I question whether the proposed plug-and-play module can function effectively with pure PLMs, thus making the plug-and-play feature more substantial.  \n**A4**: We appreciate your thoughtful inquiry into the \"plug-and-play\" aspect of our approach. The rationale behind selecting three distinct baseline models—SoftMasked-BERT, MacBERT, and SCOPE—was to demonstrate the adaptability and effectiveness of our proposed DR-CSC module in a plug-and-play fashion.\nSoftMasked-BERT is a classic CSC model, **and MacBERT is a pure PLM model with a structure like BERT**. SCOPE is the current state-of-the-art CSC model. By integrating our DR-CSC module with these diverse baselines, we aimed to establish the module's seamless integration and effectiveness across various existing approaches.\nMacBERT is a PLM proposed by Cui et al. In the CSC field, MacBERT is treated as a pure PLM baseline similar to BERT but with better performance [1].  \n\n\n---\n\n\n**Reference**  \n[1] Contextual similarity is more valuable than character similarity: An empirical study for chinese spell checking. ICASSP 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) The paper introduces a novel approach of decomposing the CSC task into three subtasks, which allows for the incorporation of external knowledge and enhances interpretability.\n\n2) The proposed DR-CSC module is compatible with existing non-autoregressive CSC models, making it a practical and easy-to-use tool for enhancing the performance of these models.\n\n3) The paper provides detailed analyses and experimental results to support the effectiveness of the proposed module.", "reasons_to_reject": "1) The purpose of Figure 1 is not clear, as it fails to effectively communicate the necessary information upon a preliminary glance. The roles of the various icons and colors used in the figure are not explained, leading to confusion about the nature of the CSC task.\n\n2) A similar issue arises with Figure 2. It remains unclear what each color signifies and it is not explicitly mentioned which parts were proposed by the user and which parts are tunable parameters.\n\n3) There seems to be an incorrect portrayal of the \"multi-task\" aspect. While the authors assert that their approach benefits from multi-task learning in line 295, the paper seems to solely focus on the CSC task.\n\n4) The authors purport their method to be \"plug-and-play\" and to incorporate external language more naturally. However, their experiments are based on previously proposed CSC models such as SoftMasked-BERT, MacBERT, and SCOPE, which, I believe, are already equipped to handle CSC tasks. I question whether the proposed plug-and-play module can function effectively with pure PLMs, thus making the plug-and-play feature more substantial.\n", "questions_to_authors": "see reasons to reject", "paper_topic_and_main_contributions": "This paper proposes a plug-and-play detection-and-reasoning module for Chinese spelling check. The authors decompose the CSC workflow into three subtasks: detection, reasoning, and searching, to leverage external knowledge and enhance the performance of existing non-autoregressive CSC models. They conduct extensive experiments and demonstrate the effectiveness and competitiveness of the proposed modules.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "see reasons to reject"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers overall agreed that the proposed method is simple and effective and that the experiments are sound, with some concerns regarding the amount of improvement over the baselines. The authors have tried to address most reviewers' questions, by providing a fair amount of discussion and justification.
Decision: Accept-Findings