PTCSpell: Pre-trained Corrector Based on Character Shape and Pinyin for Chinese Spelling Correction
Year: 2023

Abstract:
Chinese spelling correction (CSC) is a challenging task with the goal of correcting each wrong character in Chinese texts. Incorrect characters in a Chinese text are mainly due to the similar shape and similar pronunciation of Chinese characters. Recently, the paradigm of pre-training and fine-tuning has achieved remarkable success in natural language processing. However, the pre-training objectives in existing methods are not tailored for the CSC task since they neglect the visual and phonetic properties of characters, resulting in suboptimal spelling correction. In this work, we propose to pre-train a new corrector named PTCSpell for the CSC task under the detector-corrector architecture. The corrector we propose has the following two improvements. First, we design two novel pre-training objectives to capture pronunciation and shape information in Chinese characters. Second, we propose a new strategy to tackle the issue that the detector’s prediction results mislead the corrector by balancing the loss of wrong characters and correct characters. Experiments on three benchmarks (i.e., SIGHAN 2013, 2014, and 2015) show that our model achieves an average of 5.8% F1 improvements at the correction level over state-of-theart methods, verifying its effectiveness.

heading: 1 Introduction
text: Chinese spelling correction (CSC) is a task which detects incorrect characters in Chinese text and corrects them. CSC is often used as post-processing to ensure the quality of a search engine query text (Gao et al., 2010; Duan et al., 2018) and academic papers (Pollock and Zamora, 1984). CSC plays an important role in correcting recognition errors due to similar pronunciation and character shape (Park et al., 2021; Nguyen et al., 2021), which is a common issue with automatic speech recognition
*Xiao Wei and Jianbao Huang contributed equally. †Hang Yu is the corresponding author.
(ASR) and optical character recognition (OCR) systems.
Compared with other languages, Chinese has distinct characteristics, such as its unique pronunciation system (usually represented as pinyin) and writing norms, which often lead to the two problems, namely the same pronunciation may correspond to multiple characters and different characters have similar shapes. According to Liu et al. (2010), around 83% and 48% of errors in Chinese texts can be attributed to phonological and visual similarity respectively. The first sentence in Table 1 is an example of a character with a similar pronunciation, where "室" is misspelled as "师", and the second sentence is an example of a character with a similar shape, where "千" is misspelled as "于".
Chinese spelling correction is challenging because different people have different writing habits, resulting in a variety of mistakes for each character. As such, it is difficult for previous rule-based methods to address these issues effectively. Most of the recent works based on large pre-trained language models (Zhang et al., 2020; Zheng et al., 2021) perform well in the CSC task. For example, Guo et al. (2021); Liu et al. (2021) used artificially constructed confusion sets to pre-train language models for the CSC task. However, pre-training objectives tailored for CSC have not yet been explored. The visual and phonetic properties of characters are not fully considered in the pre-training process. Moreover, most works are based on the architecture of the detector-corrector (Zhang et al., 2020; Li et al., 2021; Zhu et al., 2022). However,
6330
an inherent problem occurs if the detector predicts a correct character as being wrong, in which case the corrector may change the correct character to another one which is also a reasonable response. For example, in Table 2, the correct character "影" is replaced by "音", and the correct character "安" is replaced by "宁". It can be seen that the modified sentence changes the semantics of the original text.
To solve these problems, we propose a pretrained corrector based on the visual and pronunciation features of characters for the CSC task. By doing so, the corrector can capture the phonetic similarity and visual similarity between characters, and such a pre-training strategy is more tailored to the CSC task. To address the problem of the detector’s prediction results misleading the corrector, our basic idea is to strengthen the ability of the corrector to recognize correct characters to ease the errors caused by detector.
In this work, we propose a two-stage pre-trained corrector. In the first stage of pre-training, the visual and phonological features of characters are taken into account, and the pre-training strategies matching CSC are designed, respectively from similar vision to character and from similar pinyin to character. The pre-trained models are denoted as similar character shape BERT (SCSBERT) and similar pinyin BERT (SPBERT). In the second stage of pre-training, three different pre-trained models, SCSBERT, BERT, and SPBERT are fully fused. In addition, we propose a novel loss function to fine-tune the corrector. We calculate the loss of a small number of correct characters and the loss of all the wrong characters in the original text, so that it not only corrects the wrong characters, it also prevents the correct characters from being corrected by mistake.
To verify the effectiveness of our method, we conduct experiments with our model on the SIGHAN 2013, 2014 and 2015 test set using the official SIGHAN testing tool, which achieve an average improvement in F1 of 5.2% and 5.8% at the detection-level and correction-level compared
to the latest works, MDCSpell (Zhu et al., 2022) and REALISE (Xu et al., 2021).
Our main contributions are summarized as follows: (i) We propose a pre-trained corrector which is tailored for the CSC task; (ii) We design two pre-training objectives based on vision and pronunciation to enable the corrector to capture the shape and pinyin similarity between characters; (iii) we propose a new strategy to solve the problem where the prediction results of the detector mislead the corrector by balancing the loss of incorrect characters and correct characters.

heading: 2 Related Work
text: CSC is the task of detecting and correcting wrong characters in Chinese sentences. The previous works were mainly based on the n-gram language model, rule-based method and confusion set for character error detection and correction (Yeh et al., 2013; Chang et al., 2015; Chu and Lin, 2015). After this, the CSC task is usually transformed into a sequence tagging task. Some machine learning and deep learning methods such as CRF and Bi-LSTM have been used to classify each character in a text (Chang et al., 2015; Wang et al., 2018). These methods detect characters and select a character with the highest probability of being correct.
Large pre-trained language models (PTMs) based on Transformer (Vaswani et al., 2017), such as BERT (Devlin et al., 2019), XLNET (Yang et al., 2019) and SpanBERT (Joshi et al., 2020) have been proposed and are being increasingly used for CSC tasks. In Soft-masked BERT (Zhang et al., 2020), BERT is used as a corrector to correct wrong positions predicted by the gate recurrent unit (GRU). Hong et al. (2019) set different masking schemes to fine-tune BERT, and selected the optimal candidate results as the error correction results. In addition to using BERT as a corrector, ELECTRA’s discriminator (Clark et al., 2020) is also used to detect errors (Zheng et al., 2021). Although BERT can be used in CSC tasks and achieves good results, the similarity between the target character and original character is not easy to learn. Therefore, a confusion set (LEEa et al., 2019) is used to pre-train the BERT-like model to solve the above problem. Guo et al. (2021) used artificially constructed confusion sets to pre-train BERT, which makes BERT more capable of correcting phonologically and visually similar characters. Liu et al. (2021) set different proportions to select phonologically and visually
similar characters in the process of using a confusion set for training.
Chinese spelling errors are mainly caused by characters which are similar in shape and pronunciation. Recent works focus on how to make full use of visual and phonetic features to improve the correction of these two types of errors. To correct errors due to characters having a similar shape, the Chinese characters are encoded into images or split into strokes. To correct errors due to characters having a similar pronunciation, the original text is converted into pinyin or speech features (Wang et al., 2018; Xu et al., 2021; Liu et al., 2021).
Although prior research has achieved good results on the CSC task, the following two problems remain unresolved. First, the visual and phonetic features of similar Chinese characters cannot be directly fused into BERT for pre-training. Second, in relation to the detector-corrector architecture, the corrector cannot effectively alleviate the error caused by detector. To solve these problems, we propose a pre-trained language model based on visual features and pinyin features and a special loss function for the corrector.

heading: 3 Methodology
text: In this section, we briefly introduce the preliminaries of the our method, then we detail the proposed PTCSpell.

heading: 3.1 Preliminaries
text: Task Definition The CSC task is defined as follows. Given a piece of Chinese text X = (x1, x2, x3, . . . , xn) which may contain some errors, the target is to transform it to a corrected text Y = (y1, y2, y3, . . . , yn). This task is generally formed as a mapping of the original sequence X to the corrected sequence Y , i.e., f(X) = Y . The particularity lies in that there are usually only a small number of wrong characters and the wrong character xi ∈ X has some similarity to its correct character yi ∈ Y .
Architecture The mainstream paradigm for the CSC task is based on the detector-corrector architecture (Zhang et al., 2020; Li et al., 2021). The detector identifies whether each character is correct or wrong, and the corrector generates corrections for the detected errors. Our method is designed based on this architecture. We briefly introduce the basic detector and corrector networks used in
our method, and our motivation for designing our PTCSpell method.
Detector The detection network used in our model is based on the ELECTRA discriminator (Clark et al., 2020), as shown in Figure 1. Following Zheng et al. (2021); Li et al. (2021), error detection is defined as a character-level binary classification task. We employ the pre-trained Chinese ELECTRA* to initialize the parameters of the discriminator. The input sequence X is represented as character-level tokens, and then ELECTRA’s discriminator encodes them to Hd. The classification layer can be expressed as follows:
H1 = GELU(W1H d + b1) (1)
H2 = LayerNorm(H1) (2)
H3 = W2H2 + b2 (3)
where GELU is an activation function proposed by Hendrycks and Gimpel (2016), LayerNorm is the layer normalization proposed by Ba et al. (2016), W1 ∈ Rd×d and W2 ∈ R1×d (d is the size of the hidden states from ELECTRA’s discriminator) are trainable parameters, b1 and b2 are bias vectors. H3 = {h1,h2, . . . ,hn} is the classification layer’s output representation of each character. The probability that each character may be wrong can be defined as:
P d(gi = 1|X) = sigmoid(hi) (4)
where sigmoid is the activation function, P d(gi = 1|X) is a conditional probability indicating the probability that xi is an error character.
Corrector The correction network is usually defined as a multi-class classification task at the token level, which is used to correct the characters at the location where the detection network output is "1", as shown in Figure 1. The correction network in some recent works is based on BERT with masked
*We use the released chinese-electra-180g-basediscriminator on Hugging Face https://huggingface.co/ hfl/chinese-electra-180g-base-discriminator.
language model (MLM) task (Hong et al., 2019; Zhang et al., 2020). That is, each wrong character in the original text X detected by the detector is replaced with the [MASK] token, then the model predicts the most likely correct character to replace the [MASK] token.
However, it is important to observe the wrong characters in the original text for the corrector. For example, if people can read wrong characters in context, then it will be more conducive to correct them from a similar shape or similar pronunciation. As such, the disadvantage of the previous works is that the original text is lost when correcting, which may lead to deviation from the semantics of the original text.
Motivations The contribution of this work is mainly in relation to the correction network, which includes the following two aspects as follows. Firstly, we pre-train the correction network using similar character shape and similar pinyin. Secondly, we fine-tune the correction network under a special strategy which balances the loss of correct characters and wrong characters.
To solve the problem in the correction network caused by masked text, Zhu et al. (2022) use original text as input instead of masked text, which is also used in our architecture. In addition, we pretrain the correction network from similar character shape and similar pinyin. These pre-training objectives reduce the gap between pre-training and fine-tuning, and make the error correction process more targeted to two common types of spelling errors.
Another inherent problem caused by the architecture of the detector-corrector is that the detector may predict a correct character as the wrong character, which misleads the corrector. To alleviate the detector prediction error, we not only calculate the loss of wrong characters, but also correct characters during the fine-tuning of the corrector.

heading: 3.2 PTCSpell
text: We propose a pre-trained corrector based on character shape and pinyin for Chinese spelling correction, named PTCSpell. The architecture of the correction network is shown in Figure 2. It consists of three modules, i.e., Similar Character Shape BERT (SCSBERT), BERT and Similar Pinyin BERT (SPBERT), where SCSBERT and SPBERT are new designs. As suggested by the name of SCSBERT and SPBERT, the architecture of these two modules
is exactly the same as BERT, both of which comprise 12 transformer blocks and 12 self-attention heads. The main difference between them is the pre-training objectives, i.e., SCSBERT and SPBERT are pre-trained by our designed pre-training objectives from scratch, while BERT is simply initialized by Chinese BERT with whole word masking† (Cui et al., 2021). More specifically, our proposed pre-training strategies are that SCSBERT is pre-trained by the visual features of characters and SPBERT is pre-trained by the phonetic features of characters (more details are given in Section 3.3).
The input of SCSBERT and BERT is the original text, while SPBERT is pinyin, which is converted from the original text using the pypinyin tool‡. Formally, given input X , SCSBERT, BERT and SPBERT encode it as Hc, Hb and Hp, respectively.
Then we consider how to fuse them to generate a unified representation for X . Since the character and pinyin at each position correspond to each other, it is felicitous to use the concatenation operator to fuse different features. In addition, the operator is conducive to preserving all information about the character and pinyin, thus it is expressed as follows:
Hfused = Concat(Hc,Hb,Hp) (5)
where Hc, Hb, Hp are the last hidden state of SCSBERT, BERT and SPBERT, respectively, Hc ∈ Rb×m×d, Hb ∈ Rb×m×d, Hp ∈ Rb×m×d, Hfused ∈ Rb×m×3d. b is the batch size, m is the maximum length of the text in the batch. d is the size of hidden states of BERT.
After this, we feed Hfused to the classification layer. The formula expression of the classification layer is the same as (1), (2) and (3), except for the dimension of W , i.e., W1 ∈ R3d×3d, W2 ∈ Rv×3d, where v is the size of the vocabulary. The final output of the classification layer is h
′ j , where
j is the position of the character to be corrected.
P c(yj |X) = softmax(h ′ j) (6)
where softmax is the activate function, and P c(yj |X) indicates the probability that xj is corrected to yj .
†We use the released chinese-bert-wwm-ext on Hugging Face https://huggingface.co/hfl/ chinese-bert-wwm-ext
‡We use the released pypinyin to convert Chinese text to pinyin https://github.com/mozillazg/python-pinyin.

heading: 3.3 Pre-training
text: There are two stages of pre-training in the training correction network: the first stage is pre-training SCSBERT and SPBERT from scratch, the second stage is pre-training the whole correction network. The purpose of the first stage of pre-training is to ensure SCSBERT and SPBERT have the ability to correct characters with similar shape and pronunciation, respectively. The second stage of pre-training can fully fuse the features extracted by SCSBERT, BERT and SPBERT.
SCSBERT To obtain the shape similarity of characters visually, we employ the SIFT (Lowe, 2004) image matching algorithm to calculate the visual similarity between two characters. Then, the characters with higher visual similarity are selected to replace the correct characters in the corpus to construct samples which have a similar character shape. As shown in Figure 2 (SCSBERT Pre-training), we use samples which have a similar character shape and the original text of the corpus as the pre-training dataset for SCSBERT.
SPBERT SPBERT Pre-training is illustrated in Figure 2. Firstly, we convert the prepared corpus text to pinyin, and then randomly replace some pinyin tokens with similar pronunciation ones. The sequence with the replaced pinyin is the input text X , and the original corpus data is the corrected
text. The pre-training process of SPBERT converts a similar pinyin to a correct character.
Both the first stage of pre-training and the second stage of pre-training use the following two strategies: (i) randomly replace 10% of tokens of the text with similar characters or similar pinyin (the purple characters as shown in Figure 2); (ii) randomly select 4% of tokens of the text each time, and leave them unchanged (the green characters as shown in Figure 2).

heading: 3.4 Fine-tuning
text: Detection and correction tasks are defined as character classification. The difference is that the detection task is a binary classification task, while the correction task is a multi-classification task, where the number of classes is the size of the vocabulary. The loss function of the detection network is defined as:
Ld = − n∑
i=1
logP d(gi|X) (7)
The correction network loss includes two parts, i.e., the loss of correct characters (the blue characters in Figure 2) and the loss of wrong characters (the red characters in Figure 2) in the original text. Given X = {x1, x2, . . . , xn}, Y = {y1, y2, . . . , yn}, we denote a target set T1 consisting of correct characters in the original text. More specifically, we select characters from Y that are
equal to the characters in X .
T1 = {yi|xi = yi, 1 ≤ i ≤ n, i ∈ N∗} = {t1, t2, . . . , tm1} (8)
where m1 is the size of T1. Characters of α proportion selected from T1 are denoted as T2. The set size is m2 = αm1, where α is a pre-defined the hyper-parameters.
T2 = {ti|ti ∈ T1, 1 ≤ i ≤ m2, i ∈ N∗} = { t′1, t ′ 2, . . . , t ′ m2 } (9)
To obtain the corrected characters corresponding to the wrong characters in the original text, we denote F as a set with size m3:
F = {yi|xi ̸= yi, 1 ≤ i ≤ n, i ∈ N∗} = {f1, f2, . . . , fm3}
(10)
Therefore, the loss of the correction network can be expressed as:
Lc=− m2∑
i=1
logP c(t′i|X)− m3∑
i=1
logP c(fi|X) (11)
We train the detection network and correction network by minimizing Ld and Lc.

heading: 4 Experiments

heading: 4.1 Datasets and Metrics
text: To make a fair comparison with previous works, we use SIGHAN training data (Wu et al., 2013; Yu et al., 2014; Tseng et al., 2015) and the generated pseudo data (Wang et al., 2018) as the datasets for pre-training and fine-tuning. We use the official SIGHAN2015 testing tool to evaluate the performance of our method in the three SIGHAN test sets. Following previous works, we use sentence-level precision, recall, and F1 as the metrics.
Our data preprocessing methods include converting traditional characters to simplified characters in the whole dataset using OpenCC tool§ and removing a few mislabeled data in the SIGHAN2014 and SIGHAN2015 training sets. The SIGHAN dataset is repeated 5 times and joined with the Wang271K dataset as the final training set. The results are shown in Table 3.

heading: 4.2 Implementation Details
text: We fine-tune the detection network with 10 epochs, in which the learning rate is 1.5e-5 and the batch
§https://github.com/BYVoid/OpenCC
size is 8. The learning process of the correction network consists of two stages of pre-training and one stage of fine-tuning. We train the correction network with 10 epochs in the three stages, in which the learning rate is 2e-5 and the batch size is 8. The optimizer for the detection network and the correction network is AdamW (Loshchilov and Hutter, 2018). We use a warm-up (He et al., 2016) strategy to adjust the learning rate. Specifically, the learning rate increases linearly in the first quarter of iterations, but decreases linearly in the next three quarters of iterations. Following previous works (Xu et al., 2021), "的", "地" and "得" in SIGHAN2013 are not considered during evaluating because they are mixed in the dataset.

heading: 4.3 Baselines
text: We compare our model with the following methods:
• SpellGCN (Cheng et al., 2020): Graph convolutional networks are used to incorporate phonological and visual similarity knowledge into BERT.
• GAD (Guo et al., 2021): This method captures rich global context information to reduce the impact of local error context information.
• DCN (Wang et al., 2021): The candidate Chinese characters are generated by pinyin and then an attention-based network is used to model the dependencies between two adjacent characters.
• REALISE (Xu et al., 2021): This method selectively mixes the semantic, phonetic and graphic information of Chinese characters.
• MDCSpell (Zhu et al., 2022): This method designs a multi-task framework, where BERT is used as a corrector to capture the visual and phonological features of the characters and integrated the hidden states of the detector to reduce the impact of error.

heading: 4.4 Results
text: As shown in Table 4, we observe that our PTCSpell achieves significant performance gain over the other baselines. It can be seen that the performance of our PTCSpell greatly exceeds that of ELECTRA+BERT. Specifically, at the correction level, our PTCSpell outperforms it in terms of F1 by 2.2%, 11.9% and 7.1% on the three SIGHAN test sets, respectively.
Then, we compare our PTCSpell with the most competitive works, such as MDCSpell and REALISE. At the correction level, our PTCSpell outperforms both of these in terms of F1 by 4.2%, 7.6% and 5.5%, respectively, and in terms of precision by 12.2%, 14.8% and 11% on the three SIGHAN test sets, respectively.
The main reasons are two-fold. First, PTCSpell learns the similarity between characters, thus the corrector selects characters which are similar to the original text in the correction process. Second,
the novel loss function proposed on the corrector alleviates the problem of the detector predicting correct characters as errors. Although PTCSpell has made great improvements in precision and F1, recall is not improved compared with the baselines. A possible reason is that the detector fails to detect all errors, leading to a bottleneck in the corrector.

heading: 4.5 Ablation Study
text: We explore the influence of parameter α in the loss function on model performance and the contribution of the SCSBERT and SPBERT modules to the PTCSpell model. We evaluate our model at the sentence-level on the 2013, 2014 and 2015 SIGHAN test sets. The average F1 performance of the three test sets is shown in Table 5 and Table 6 (the detailed results are provided in Appendix A.1).
Table 5 shows the influence of different parameter α in the loss function on the performance of the PTCSpell model. We test the performance of our model by varying α in range of [0, 0.1] and [0.92, 1] with a step of 0.02. We find that a smaller α is better than a larger α. When α is 0, the model loses the ability to retain the original text semantics, so the model does not perform well. When α is 1, this means that all characters participate in the calculation of loss. It is observed that when α
is 0.04, the model achieves the best performance. Interestingly, according to our statistics, we find that wrong characters accounted for 3.3% of all characters in the training set. We infer that when α is set at 0.04, the number of correct characters and wrong characters is balanced, which is helpful to train the model.
Table 6 analyses the second stage of pre-training, SCSBERT and SPBERT for our PTCSpell. To verify the effectiveness of the pre-training in the second stage, we conduct experiments PTCSpell and PTCSpell(-PT). We find that the second stage of pre-training greatly improves the performance of the model. To verify the effectiveness of SCSBERT and SPBERT, we conduct experiments PTCSpell (-PT), SCSBERT + BERT (-PT), BERT + SPBERT (-PT) and BERT (-PT), where SCSBERT and SPBERT are pre-trained by character shape and pinyin, respectively. We find that SCSBERT and SPBERT both improve the performance of the model. Through these experiments, The effectiveness of SCSBERT, SPBERT and the second stage of pre-training are verified.

heading: 4.6 Case Study
text: Table 7 provides details on two cases of Chinese spelling correction. Given an input sentence, we compare the error correction effect of the baseline (ELECTRA + BERT) and PTCSpell. The detector (ELECTRA) detects character errors, where
"0" means the current character is correct and "1" means the current character is wrong. If the output of the detector in the current position is "1", the character is corrected by the corrector. In the first case, "乍" is correctly predicted to be the wrong character, but the baseline corrects it to "什" and forms a phrase with the following "么", which fails to capture the character shape of "乍". Our PTCSpell corrects "乍" to "怎", because it captures the character shape successfully. For the other two wrong characters "蝉声", it is difficult for baseline to correct consecutive wrong characters, causing "蝉" to be corrected to "蚊" due to its similar shape. In contrast, our PTCSpell corrects them to "生产" based on similar pronunciations and is not affected by consecutive errors. The first case shows that PTCSpell achieves good performance in both similar shape and pronunciation errors. In the second case, "影" is a correct character but the detector predicts it to be a wrong character. We can see that the baseline does not handle this well, so it is replaced with "音", while PTCSpell leaves this character unchanged. The second case shows that PTCSpell is not misled by the detector because it learns whether the current character is really wrong and tries its best to maintain the semantics of the original sentence during error correction.

heading: 5 Conclusion
text: We propose a pre-trained corrector, which is a part of the detector-corrector architecture, for modeling similar shape and pronunciation errors in the CSC task. Our main contribution is to enhance the ability of the corrector. The architecture of PTCSpell is based on BERT, but is pre-trained from scratch based on similar character shapes and pinyin re-
spectively. In addition, we propose a special loss function that enhances the ability of the corrector to retain the correct characters in the original text. Finally, the experimental results show that our model is effective. In the future, we will design a better detector to further improve the recall score for the whole detector-corrector model.
Limitations
Our model achieves outstanding performance in relation to Chinese spelling correction. However, it has several potential limitations: (i) Errors of missing and redundant characters cannot be corrected by our model. The PTCSpell model only focuses on spelling errors, and requires that the input text has no grammatical or semantic errors. (ii) The error-correcting language is targeted at Chinese. The pre-trained model based on similar pinyin cannot adapt to other languages, while pretrained model based on similar character shape can adapt to other languages well, because the pinyin input method is unique to Chinese, but character error due to a similar shape is a common problem in many languages. Nevertheless, we put forward the idea of matching the pre-trained model with error correction tasks, which is suitable for all languages.

heading: Acknowledgments
text: This rescarch was supported by the Shanghai Science and Technology Young Talents Sailing Program (22YF1413600). We thank Maoxin Shen for helpful discussions, and the anonymous reviewers for their insightful comments.

heading: A Appendix
text: A.1 Ablation We show detailed results on the effect of different alpha on model performance. After two stages of pre-training, we set different alpha for the model training, and record the performance of the model on three SIGHAN test sets. The detailed results are shown in Table 8.
At the same time, we explore the contribution of SCSBERT and SPBERT to the whole model. In the following experiment, SCSBERT and SPBERT are pre-trained respectively, but the second stage pre-training is not carried out. Then they are finetuned, where the alpha in the loss function is set to 0.04. The detailed results are shown in Table 9.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Limitations
3 A2. Did you discuss any potential risks of your work? 4.5 Ablation Study; Limitations
3 A3. Do the abstract and introduction summarize the paper’s main claims? Abstract
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Left blank.
3 B1. Did you cite the creators of artifacts you used? Footnote 2 and 3 on the second page.
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Footnote 2 and 3 on the second page.
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? 4 Experiments
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? We use datasets published online.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Footnote 2 and 3 on the second page.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 4.1 Datasets and Metrics
C 3 Did you run computational experiments? Left blank.
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? 4.2 Implementation Details
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? 4.2 Implementation Details
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 4.5 Ablation Study
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 4.2 Implementation Details
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: In the future, we will design a better detector to further improve the recall score for the whole detector-corrector model. Limitations
Our model achieves outstanding performance in relation to Chinese spelling correction. However, it has several potential limitations: (i) Errors of missing and redundant characters cannot be corrected by our model. The PTCSpell model only focuses on spelling errors, and requires that the input text has no grammatical or semantic errors. (ii) The error-correcting language is targeted at Chinese. The pre-trained model based on similar pinyin cannot adapt to other languages, while pretrained model based on similar character shape can adapt to other languages well, because the pinyin input method is unique to Chinese, but character error due to a similar shape is a common problem in many languages. Nevertheless, we put forward the idea of matching the pre-trained model with error correction tasks, which is suitable for all languages